* wire-api: add ProtocolMixedTag (with undefined stubs)

* wip

* wire-api

* fix bug in schema

* galley int test: fix some types

* libs/api-client fix type

* brig-integration: fix some types

* Add endpoint type

* wip

* add updateMixedProtocol

* wire-api add json instances to ProtocolTag

* add test

* Rename test group

* fix bug

* Add TODO

* Add failing test

* Add creator client to conversation

* Revert "Add TODO"

This reverts commit c3ed0c8.

* refactor tests and add failing test step

* Allow proteus messages on mixed protocol

* Add changelog entry

* Add nginz route

* hi ci

* hi ci

* finish leftover TODOs in protocolValidAction

* Fix bug in schema of ProtocolUpdate

* update docs: start-serices-only -> run-services

* Allow no-op state transitions

* Add variable-sized integer serialisation

* Implement new MLS structures

* Fix KeyPackage parser

* Fix MLS signature verification

Signatures in MLS are computed on a special `SignContent` structure, so
we need to replicate that for verification.

* Update paths now contain leaf nodes

* Remove proposals now have indices instead of refs

* Adapt integration tests to remove proposal changes

* Compute new node index for add proposals

* New commit bundle API

Also replace PublicGroupState with GroupInfo

* Add instances for roundtrip tests of MLS types

* fix adding users to MLS conversations

* change content-type of commit bundle in integration tests
* fix keypackage ref serialisation
* add context to commit bundle parsing

* fix integration test: send other user's commit

* keep track of index map while processing proposals
* add creator client to ProposalAction in epoch 0

* readGroupState for the new group.json format

* Generate welcome recipients when processing bundle

Also remove old unsupported welcome endpoints. All welcome messages now
need to be sent through commit bundles.

* Send recipients as part of a welcome RPC

* Use commit bundles in failure tests

* Implement new proposal ref computation

* fix integration test admin removes user from a conversation

* switch mls-test-cli call to external-proposal

* Implement validation of leaf nodes in galley

- extract core validation function to wire-api
- generalise validation of leaf node source
- implement validation of key packages and leaf nodes in galley
- remove all internal brig endpoints related to validation
- validate leaf node in external commits
- validate leaf node signature

* Apply proposals in the correct order

* Remove redundant GroupContext structure

* Re-implement processing of external commits

* add references from data types to MLS spec

* Remove key package mapping code

* fix more integration tests

* track client scheduled for removal in Cassandra

[ ] conversations
[x] subconversations

* minor typos

* split executing proposals for int and ext commits

* execute remove proposals before add proposals

This makes sure that all leaf indices are freed in the database before
they are occupied again.

* rename Word32 and ref to LeafIndex and idx

* Remove MissingSenderClient error

* Remove some prefixes from MLS structures

* Remove prefixes from RawMLS fields

* Reorganise TODOs

* Check epoch again after taking commit lock

* Remove MLSPackageRefNotFound error

* Simplify testRemoveUserParent

* Simplify testRemoveCreatorParent

* Pass correct list of clients to planClientRemoval

* Fix assertion in external add proposal test

* Propagate actual message, not just commit

* Fix signature calculation when generating messages

* Pass removal key to mls-test-cli on group creation

* Take pending clients into account in removal logic

* Fix assertion in remove proposal test

* apply linter suggestions

* fix unit test: MLS remove proposal

* Upgrade mls-test-cli in the nix environment

* Update cassandra-schema.cql

* disable testing the keypackage lifetime

* remove checks for keypackage assignments

* validate bare proposals and inline proposal

* rephrase and filter the left TODOs

* Verify that capabilities include basic credentials

* Add nonce to PreSharedKeyID structure

* Split Galley.API.MLS.Message

* Inline executeIntCommitProposalAction

* Use more specific type for external commit actions

* Re-organise TODOs

* Simplify processProposal arguments

* Remove LWT in planMLSClientRemoval

* Restore unsupported proposal test

* Restore disabled MLS unit tests

* Add CHANGELOG entries

* Document IndexMap and ClientMap

* fixup! Restore unsupported proposal test

* Linter fix

* fixup! Upgrade mls-test-cli in the nix environment

* Fix: make git-add-cassandra-schema-impl lists to many keyspaces

* postMLSMessageToLocalConv: return no events

* Remove unused paExternalInit

* Renew certificates for e2e integration tests (#3243)

* Renew certificates for e2e integration tests

* Document how to renew e2e integration test certs

Co-authored-by: Igor Ranieri <igor@elland.me>

* fix broken tests

* ExternalCommitAction: remove superfluous ClientIdentity

---------

Co-authored-by: Stefan Matting <stefan@wire.com>
Co-authored-by: Stefan Berthold <stefan.berthold@wire.com>
Co-authored-by: Akshay Mankar <akshay@wire.com>
Co-authored-by: Igor Ranieri <igor@elland.me>

* Move some MLS tests to new integration suite

* Add CHANGELOG entry

* Add endpoint to delete key packages

* Add integration test for key package deletion

* migrate test that adds user via mls

* mls-test-cli: make show use json

* testlib: assertOne, asByteString

* Add test: user leaves -> remove proposal

* wip test: adding partial client set to mixed

* migrate test testAddUserPartial

* make test fail

* Use new test parametrization

* asInt -> asIntegral

* migrate testRemoveClientsIncomplete

* Add HasCallStack to getJSON

* Add test for removing partial clients

* Refactor: use fields

* integration: rename functions and improve errors

* Add test: remote backend doesnt know about about mixed protocol convs

* Deny application msgs for mixed (with test)

* fix mixed remote test

* Add testFirstCommitAllowsPartialAdds

* Only allow protocol updates for team conversations

* Call on-new-remote-conversation on protocol update

* Test remote user adding

* remove migrated test

---------

Co-authored-by: Paolo Capriotti <paolo@capriotti.io>

* Fix golden tests for MLSMessageSendigStatus
* Port what used to be MLS/Message.hs
* Throw instead of relying on failed_to_add

In MLS a commit is rejected anyway so there is no point in passing
through FailedToProcess. Instead, a federator error is thrown if there
are unreachable backends when submitting an add commit.

* Test adding to an MLS conversation
* Fix: use the mls_migration_lock_status DB column

- This is a bug to be fixed in the main MLS branch too

- A c/p mistake that is fixed now

…bconversation ID (#3309)

Removed federation endpoints
- on-new-remote-conversation,
- on-new-remote-subconversation, and
- on-delete-mls-conversation.

Removed effects
- Galley.Effects.SubConversationSupply.

It should be possible for the `Capabilities` structure to contain
unknown version, proposal, credential and extension tags.

* [FS-1915]  Include conversation id into welcome message event

* Hi ci

* Initial implementation of mixed -> mls transition

* Migration criteria check stub

* Simple mixed->mls test

* Check migration criteria on finalisation

* More specific migration criteria error

* Test extraneous client removal

* One2One GET endpoint stub

* Make one2one conversation ID protocol-dependent

* Create MLS 1-1 conversation object

* Add MLS one2one integration tests

* Test MLS 1-1 for teammates

* More general MLS 1-1 conversation GET handler

- Fetch 1-1 conversation from database if present
- Get conversation from the other backend when the conversation is
  supposed to be remote

- 16 bit version
- 16 bit conversation type

* Remove user & client thresholds from MLS migration

* remove columns from DB

* review comments

* Add test for happy path

* Rename `meta` field of ConvOrSubConv to `mlsMeta`

* Make creator field optional in conv metadata

* Create MLS 1-1 conversation on commit

* Fix connection check for MLS 1-1 conversations

* Merge BaseProtocolTag and ProtocolCreateTag

* Return empty MLS 1-1 conversation on GET

* Check that subconv group ID matches message

* Check welcome message on first 1-1 commit

---------

Co-authored-by: Paolo Capriotti <paolo@capriotti.io>

Merge dev into MLS

Merge develop

…anager or own)

* Do not send welcome message to sender

merge dev into mls

* Remove member limit for MLS conversations

* Restore HardTruncationLimit to 2000

* Fix client dir path in performance script

* WPB-3611: Notifying remote domains about defederation.

During defederation, remaining federation members will now be notified
about the defederation by a new notification type
"federation.connectionRemoved" that carries the domains that were
involved in the defederation. This allows federation members to clean up
their conversations where users from both domains are present.
Additionally, this is sent out to all of their local clients so they can
also clean up. Normal notifications are suppressed for these operations,
similar to "federation.delete" handling.

Changes:
- Added a new federation event type.
- Added a new federation endpoint in galley. This route cleans up local
  conversations and notifies local clients of the defederation using the
  new event type.
- Updated background-worker to send notifications to federation members.
- Added tests for the new notifications, and that local conversations
  are cleaned up as expected.

---------

Co-authored-by: Leif Battermann <leif.battermann@wire.com>
Co-authored-by: Leif Battermann <leif.battermann@gmail.com>

…o-develop

  Master->Develop after release

…3507)

Co-authored-by: Sven Tennie <sven.tennie@wire.com>

Merge develop into MLS

…otifications (#3515)