diff --git a/proposals/2199-canonical-dms.md b/proposals/2199-canonical-dms.md new file mode 100644 index 00000000000..77bdffc8d7d --- /dev/null +++ b/proposals/2199-canonical-dms.md @@ -0,0 +1,722 @@ +# Canonical DMs (server-side middle ground edition) + +In the messaging space it is common for apps to expose exactly 1 chat for a conversation +between two users, and in some cases a specific set of users. Matrix currently does not +adequately support this as it allows for multiple rooms to represent a DM between two users. +Although supporting multiple chats with individuals is potentially useful for some use cases, +clients are increasingly wanting to be able to support exactly one DM for a user. + +Any messaging app is likely to want to differentiate DMs from all the noise, and Riot in +particular is looking at using the DM between users for a more personal experience within +the app (such as key verification between users). This proposal focuses on immutable DMs +between users to assist in identifying the "One True DM" between users and prevent the +room's scope from increasing. + +For comparison to Matrix, the following chat apps/platforms all support and encourage a +single DM between users. They all redirect the user to an existing chat with the target +user, creating a new DM if needed. + +* Slack +* Discord +* IRC +* Telegram +* Gitter +* Facebook Messenger +* Twitter +* Hangouts +* Instagram +* Rocket.Chat +* Steam +* WhatsApp +* SMS +* and probably many others... + +Platforms which allow/encourage users to have multiple DMs with a specific person are: + +* Matrix (currently) +* Email +* XMPP (technically speaking) + +This proposal covers how servers enforce exactly one DM (ideally immutable) between a set +of users. Clients have some responsibilities to render this appropriately, however the server +is expected to do most of the heavy lifting here. Immutable DMs are direct chats between +users where the participants can't change the room's aesthetics. For example, users can't +change the name or topic of the room, and they'll always be DMs. + +This proposal supports "direct chats" or "DMs" in the sense of private conversations with +a set of users. This means that a DM between 3 users is possible, however the common case +is likely to be 1:1 (2 person) chats. + +Immutability of DMs is also proposed in this MSC, however as an optional feature of DMs and +not a requirement. + + +## Proposal + +In short, the entire direct chat module in Matrix is deprecated by this proposal. The direct +chat module relies on clients behaving themselves and tracking all the information for themselves +while somehow coming to the same conclusions as any other client the user happens to run. In +practice this becomes very difficult to do and maintain, particularly given the direct chats +are stored in an account data blob. If two clients were to update the account data at the same +time (which happens *a lot*), the clients would end up overwriting each other. + +Direct chats consist of "important" and "unimportant" users to allow for bots and personal +assistants in a chat. Unimportant users don't affect the DM-ness of the room and are essentially +observers on the room (with speaking rights). How classification is done is defined later in +this proposal. + +Direct chats are now expected to be identified and labelled by the server through the `summary` +response field on `/sync`. The `m.heroes` MUST contain the important users in a DM, regardless +of the presence of a name, canonical alias, etc on the room. The heroes must never be truncated +for a DM, and must not contain the syncing user's ID. A new field, `m.kind`, is mandatory for +direct chats which can only be `m.dm`. Future values may be added in a future proposal. Like +the other fields in the summary, `m.kind` may be omitted if there was no change in the field. + +For the purposes of demoting a DM from its DM status, `m.kind` can also be the literal `null` +to identify it as "unknown or irrelevant kind". + +Implementation note: clients can identify whether a room is a 1:1 or multi-way chat by looking +at the `m.kind` and `m.heroes`: if the kind is `m.dm` and there's only 1 user in `m.heroes` +then the room is a 1:1 chat. If the kind is `m.dm` and there's 2+ users in the `m.heroes` then +it is a multi-way/private group chat. + +Implementations should also be wary of semantics regarding what causes a room to jump between +a multi-way DM and a 1:1 DM. Users can be invited (and joined) to either kind of room without +causing the room to jump to a different kind: for example, a 1:1 chat can have several unimportant +users in it (bots, etc) which do not make it a multi-way DM. Similarly, if an important user +were to leave a multi-way DM the room does not become a 1:1 DM. This is all described in further +detail later in this proposal. + +With irrelevant fields not included, an example sync response would be: +```json +{ + "rooms": { + "join": { + "!a:example.org": { + "summary": { + "m.heroes": ["@alice:example.org"], + "m.joined_member_count": 1, + "m.invited_member_count": 1, + "m.kind": "m.dm" + } + }, + "!b:example.org": { + "summary": { + "m.heroes": ["@bob:example.org"], + "m.joined_member_count": 2, + "m.invited_member_count": 0, + "m.kind": "m.dm" + } + }, + "!c:example.org": { + "summary": { + "m.heroes": ["@alice:example.org", "@bob:example.org"], + "m.joined_member_count": 3, + "m.invited_member_count": 0, + "m.kind": "m.dm" + } + }, + "!d:example.org": { + "summary": { + "m.joined_member_count": 5, + "m.invited_member_count": 2, + } + } + } + } +} +``` + +Clients rendering the above should end up with something like the following: + +![alice-bob-over-banquet](./images/2199-dms-client.png) + + +Servers are expected to be able to identify direct chats for users and flag them appropriately +in the room summary. How existing DMs are migrated is covered later in this proposal. The +server must use these rules to determine if a room is a direct chat: +* The join rules are `invite`. +* The rejoin rules are `invite` or `join` (as per [MSC2213](https://github.com/matrix-org/matrix-doc/pull/2213)). +* The user's most recent membership event has `"m.dm": true` in the `content`. +* The room has an explicit power level event (running a DM without a PL event is not supported). +* The room has at least 2 possible important users (who may or may not be joined yet). +* The user themselves is important in the room. +* The room is not tombstoned. +* The room is not soft-tombstoned by the server (described later in this proposal). +* No important users have been banned. Kicking or leaving the room does not affect the DM-ness + of the room. + +Assuming no other conflicts arise (duplicate chats, etc) the room is considered a DM between +the important users in the room. Important users are identified simply by having a power level +greater than or equal to the `state_default` power level requirement. + +The server MUST maintain the `m.dm` flag when updating or transitioning any user's membership +event. The server SHOULD refuse to let the user manipulate the flag directly through `PUT /state` +and similar APIs (such as inviting users to a room). Servers MUST consider a value of `false` +the same as the field not being present. + +Servers are not expected to identify a room as a direct chat until the server resides in the +room. Servers MUST populate the room summary for invites if the server is a resident of the +room. Clients SHOULD use the room summary on invites and joined rooms to identify if a room +is a direct chat or not. Where the summary is not available and an `m.dm` field is on the +invite event, clients should be skeptical but otherwise trusting of the room's DM status. +For example, a skeptical client might list the room with a warning stating that the room +is flagged as a DM without being appropriately decorated, but not prevent the user from +accepting the invite. + +Servers MUST attempt to identify (or re-affirm) rooms as a DM whenever the relevant state +events in the rules above change or are updated. + +Soft tombstoned rooms are rooms which are considered tombstoned without an actual tombstone +state event being present. This is typically used to flag rooms where tombstone events cannot +be sent as tombstoned. This proposal goes into more detail on this a bit later. + +#### A note about leaving DMs + +Unless an important user is banned, the DM is still considered alive and should be able to +be rejoined even if some important people were to leave. However, there's a good chance that +when all important users leave that the room becomes unjoinable and therefore dead. To help +combat this, servers should approach rooms where important users have left with caution by +assuming the room cannot be brought back to life. + +If [MSC1777](https://github.com/matrix-org/matrix-doc/pull/1777) or similar were to land, +the server should abuse the capability to determine how alive the room is. By refreshing +its perspective, it can avoid scenarios where it has to make assumptions about the state +of the room it is no longer in. The server should never autojoin the user to the room, +regardless of MSC1777 support or not. + +The following cases are reinforcement reading for the conditions mentioned so far. They +describe how the server is expected to behave given a common scenario - not all scenarios +are covered here. + +**Case 1: The DM resides on a single homeserver**: when all important users leave the DM, +the DM is to be considered soft-tombstoned for those users. This will cause a new DM to be +created. The presence of unimportant users can be enough for a homeserver to consider the +room not dead and therefore rejoinable. + +**Case 2: The DM resides on 2+ homeservers**: this gets a bit tricky. When the last important +user leaves, that homeserver would not have visibility on the room anymore but does not have +enough information for it to be tombstoned (soft or otherwise). Another server can still rejoin +the room due to unimportant users being left behind, or keep the room alive without the other +homeserver continuing participation. The homeservers involved will still converge on the room +when other conversations start. + +**Case 3: A duplicate DM comes in for a room the server has left**: when the server does not +have visibility on the members of a room anymore it cannot tombstone newly created rooms and +point to the room it can't see. If the server happens to be a resident of the room, it can +absolutely tombstone the new room in favour of the old room, even if the server's membership +is just unimportant users and the important users having left. When the server does encounter +an invite for a DM which duplicates a room it has left, it must assume that its perspective +of the older room is antiquated and that something happened to cause a new invite to come +in. After joining, if a server happened to tombstone the room to point to the older room +then the user could then try and join the room and refresh the server's perspective. + +*Note*: Originally case 3 had a step for the server to autojoin the user into the existing +room to refresh its state, however this left a glaring hole for abuse to filter its way down +to a given user. + + +#### Creating DMs + +The room creation preset `trusted_private_chat` is deprecated and to be removed in a future +specification version. Clients should stop using that preset and instead use the new endpoint +for creating DMs, as defined here. + +**`POST /_matrix/client/r0/create_dm`**: + +This is mostly a clone of `/createRoom` with the unnecessary parts left behind. This does not +replace `/createRoom`. This requires authentication and can be rate-limited. + +Request parameters (JSON body): +* `room_version` (`string`) - The version to create the room with. Same as `room_version` from + `/createRoom`. +* `invite` (`[string]`) - The list of user IDs to invite to the room. Same as `invite` from + `/createRoom` with the added behaviour described below. +* `invite_3pid` (`[3pid invite]`) - The list of 3rd party users to invite to the room. Same as + `invite_3pid` from `/createRoom` with the added behaviour described below. +* `creation_content` (`object`) - Additional content for the creation event. This is also the + same as `creation_content` from `/createRoom`. + +Note: `name`, `topic`, `visibility`, `room_alias_name`, `initial_state`, `preset`, `is_direct`, +and `power_level_content_override` are all intentionally excluded from the clone. There is no +preset support, aesthetic characteristics of the room (name, topic, etc) are not supported for +immutable DMs under this MSC, and the remaining fields do not have strong use cases for DMs - +clients can always send state event updates after creating the room, for instance. + +Response (JSON): +The server creates the room similar to how `/createRoom` operates: + +1. An `m.room.create` event is created, as is `m.room.power_levels` using the defaults. +2. Join rules of `invite` are set with a rejoin rule of `join`, as per + [MSC2213](https://github.com/matrix-org/matrix-doc/pull/2213)). This is to allow DMs to be + re-used (described later) without sacrificing security of the DM by letting random people + who were invited at some point into the room. +3. History visibility of `invited` is set. +4. Guest access of `can_join` is set so that guests can be included in DMs where needed. +5. Encryption is enabled by default using the most preferred megolm algorithm in the spec. + Currently this would be `m.megolm.v1.aes-sha2`. +6. Power levels with the following non-default structure are set: + + ```json + { + "events_default": 0, + "state_default": 50, + "users_default": 0, + "ban": 50, + "invite": 50, + "kick": 50, + "redact": 50, + "notifications": { + "room": 50 + }, + "events": { + "m.room.name": 100, + "m.room.avatar": 100, + "m.room.topic": 100, + "m.room.history_visibility": 100, + "m.room.power_levels": 100, + "m.room.join_rules": 100, + "m.room.encryption": 100, + "m.room.canonical_alias": 100 + }, + "users": { + + }, + "third_party_users": { + + } + } + ``` + + This power level event is not applied until after the invites have been sent as otherwise + it would be impossible to give third party users power. Servers should apply a default + power level event to the room and then apply the power level event described here after + the invites have been sent. Servers can optimize this process and use this power level + event at step 0 of the room creation process if there are no third party invites to send. + Users invited to the room get power level 50, including the creator. + * Third party users invited to the room get power level 50, as described by MSC2212 (see + later on in this proposal for how this works). Like the invited users, all third party + users invited as a result of the `/create_dm` call are considered important. +7. Users (3rd party and otherwise) are invited. Important users (those invited and the creator) + MUST have `"m.dm": true` in their membership event content. Third party important users get + the same `m.dm` flag on their `m.room.third_party_invite` event contents. + +Servers MUST return an existing room ID if the server already knows about a DM between the +important users. The important users which have left the DM MUST be explicitly re-invited. +If the user trying to create the room is not in the DM themselves, the server MUST try and +re-join the user to the room. If the re-join fails, the server should create a new room +for the DM and consider the existing one as soft-tombstoned. + +The rationale for preventing either party from manipulating the room is to ensure that there is +equal representation of the room from both parties (automatically named, automatic avatar, etc). +Users generally expect that a DM will be predictably dressed for searching and scanning, meaning +that the other parties cannot change the aesthetics of the room. For predictable history and +privacy, the room's history & join rules are locked out from the users. The users can upgrade the +room at any time to escape power level struggles, although this may not maintain the DM status. + +*Note*: The `private_chat` preset is untouched as it does not affect DMs. `trusted_private_chat` +was intended for DMs despite its name and is deprecated as a result. + +*Note*: Servers are still expected to be able to identify DMs from `/createRoom`, and clients +are able to craft DMs using `/createRoom`. When this happens, servers MUST return an existing +room ID for a DM if one exists and it looks like the user is trying to create a DM through the +`/createRoom` endpoint. + +#### Glare and duplicated rooms + +There is a potential for two users to start a DM with each other at the same time, preventing +either server from short-circuiting the `/createRoom` call. Servers MUST NOT prevent users +from accepting or rejecting invites to known conflicting DM rooms. Servers SHOULD NOT retract +the invite and SHOULD NOT automatically join the user to the room. The rationale is that the +user is likely to have already received a notification about a room invite and would be +confused if the room went missing or was automatically accepted. Clients are welcome to adjust +their UX as desired to do something else instead (like automatically reject/accept the invite +or hide it when it is an apparent duplicate). + +When the server joins a room it has identified as conflicting/duplicating an existing DM it +MUST apply the conflict resolution algorithm: +1. Order all the conflicting room's creation events by their `origin_server_ts`. +2. Pick the room with the **oldest** creation event as the canonical DM. +3. Send a tombstone state event to the non-canonical DM rooms pointing at the canonical room. + * If the tombstone cannot be sent by the server due to auth rules, it must consider the + room as "soft-tombstoned". A soft-tombstoned room is just a flag for preventing a room + from being counted as a DM and has no further behaviour or implication in this proposal. + +The rationale for picking the oldest creation event is it is likely the room with the most +context for the user. It is possible that a malicious user intentionally creates a new room +with an ancient creation event however there's very little gain in doing so. The algorithm +is chosen such that any two servers come to the same conclusion on which room to use. + + +#### Upgrading DM rooms + +There may come a time where DM rooms need to be upgraded. When DM rooms are upgraded through the +`/upgrade` endpoint, servers MUST preserve the `content` of the previous room's creation event and +otherwise apply the `immutable_dm` preset over the new room. If conflicts arise between +the previous room's state and the state applied by the preset, the preset takes preference over +the previous room's state. + +Servers MUST create the new room by using the the room creation rules listed earlier in this +proposal. This means inviting the other parties of the previous room to the new room automatically +and ensuring the room state is as described above. Auth errors with transferring state (such as +the room name) must be considered non-fatal to the upgrade process: just skip that state event. + +Note: normally room upgrades modify the power levels in the old room in an attempt to mute all +participants. However, no one will have power to modify the power levels in the old room. +Therefore, servers MUST NOT fail to upgrade a room due to being unable to update the power levels +in the old room. This is considered acceptable by this proposal because upgraded DM rooms will +lose their DM status, making them at worst just another room for the user. + +Servers should do their best to follow tombstones to new DM rooms, however they should not assume +that those new rooms are valid DM rooms nor should they automatically join/invite the important +users to the room. + + +#### Detecting server support + +Servers which implement this MSC prior to it appearing in a spec release MUST advertise as such +through the unstable feature flag `m.immutable_dms` on `/versions`. Clients which detect server +support should not only check for the feature flag but also the presence of a supported spec +version on the server (as the flag may disappear once the feature lands in a release). Currently +this proposal is expected to land in r0.6.0 of the Client-Server API. + +Some servers have thousands and even millions of users which will need their DMs migrated. In +order to allow those servers to advertise the feature (both as an unstable flag and as a released +spec version) the client SHOULD assume that the server has not migrated its user's DMs until the +server sends the `m.direct_merged` account data event. More information about migration can be +found in the next section. + + +#### Migration + +Users already have DMs which need to be correctly mapped by the server. Servers MUST use the +following process for migration for each user: +1. Grab a list of room IDs from their `m.direct` account data (if present). Ignore the user IDs + that are associated with the room IDs. +2. Identify each room as either a DM or some other room, flagging them as appropriate, using the + following steps: + + * If the room does not have a `rejoin_rule`, consider the rejoin rule as `join` for the + purposes of identification. + * Identify the room using the rules specified earlier in this proposal. + * If the room did not have a `rejoin_rule`, attempt to set the rejoin rule to `join`. If that + fails, do not consider the room a DM. Note that this will not cause the rule to magically + work on prior room versions: it is just set to ensure that the DM flagging conditions + are met as they don't care for the behaviour, just the value being present. + + Identification of DMs may involve conflict resolution, which should only happen after the steps + above have been executed. +3. Set `m.direct_merged` account data with `content` consisting of a single array, `rooms`, listing + all the room IDs the server iterated over, regardless of result. + +The server SHOULD support automatically migrating any additions to `m.direct` even after migrating +the user. This is to allow for the user having an older client which does not yet support proper +DMs. Removals from `m.direct` are recommended to be left unhandled to ensure consistency with the +updated DMs. + +Clients SHOULD NOT assume that the server will migrate direct chats if the user's `m.direct` +account data does not list any rooms. This is more important for new accounts created after the +DMs feature has been introduced in Matrix: if the server had to set `m.direct_merged` for every +user in the future, the server would be collecting largely useless data. Instead, the server is +given the option to skip migrations for users that have no data to migrate (such as new users). + + +#### Sugar APIs (for appservices/thin bots) + +Appservice users and some bots are unlikely to see the room summary information. In an attempt to +combat this, some APIs are provided for users and thin clients to call and poke into the server's +view of their DM list. + +The expected use cases for not using `/sync` are: +* When appservices want to contact a user in a DM fashion (IRC bridge sending NickServ info). +* When thin bots aren't syncing and want to alert users (monitoring bots triggered by external event). +* Clients which don't trust their local cache of DMs. + +Appservices in particular are expected to heavily use the DM-finding API below and should cache +the result. When the appservice sees an event which the server might have reidentified the room +for it should invalidate its cache. Servers should make both of these endpoints highly cached. + +**`GET /_matrix/client/r0/user/:userId/dms`** + +The only parameter, `:userId`, is the user ID requesting their DMs. The auth provided must be valid +for this user. Example response: +```json +{ + "dms": { + "!a:example.org": { + "important": ["@alice:example.org"] + }, + "!b:example.org": { + "important": ["@bob:example.org"] + }, + "!c:example.org": { + "important": ["@alice:example.org", "@bob:example.org"] + } + } +} +``` + +The `important` array does not include the user themselves. + +**`GET /_matrix/client/r0/user/:userId/dm?involves=@alice:example.org&involves=@bob:example.org`** + +The `:userId` is the user ID requesting their DMs - the auth provided must be valid for this user. +This additionally takes `?involves` (specified multiple times) to search for a DM involving the given +users. It should be considered a bad request if the user asks for a DM involving themselves. + +Example response: +```json +{ + "room_id": "!c:example.org" +} +``` + +If no DM exists involving those users, an empty object should be returned with 200 OK. + + +#### Third party invites + +Third party invites (email invites) are hard to handle as the participants in the room are +unlikely to be able to modify the power levels in the room because the immutable DM preset +forbids this by design. To remedy this, this proposal requires that third party users get +their own power level, as described in [MSC2212](https://github.com/matrix-org/matrix-doc/pull/2212). + +In addition to MSC2212, this proposal requires that a `m.dm` field be added to the +`m.room.third_party_invite` event. When the invite is claimed, the `m.dm` field must +be copied to the generated membership event. + +#### Complete list of deprecations + +Earlier in this proposal it was mentioned that the existing direct chats module is replaced +by this proposal, however it didn't exactly explain the scope. The existing module is +optionally supported by servers (which requires very little effort on their part), however +this proposal recommends that servers drop support completely per the deprecations below. + +Deprecated things: +* `is_direct` on `/createRoom` now does nothing (ignored field, like any other unrecognized + field). +* `is_direct` on membership events now means nothing (replaced by `m.dm`). +* `m.direct` account data is deprecated (replaced with the behaviour described in this + proposal). + +After sufficient time has passed, the deprecated components should be removed from the +specification. + + +#### Test cases for the server/proposal to consider + +**Disclaimer**: This section might add clarity but might also be confusing. Sorry. + +**Simple case** + +The happy path of DMs. + +1. Alice starts a DM with Bob. Alice and Bob end up with a single room. +2. Bob then starts a DM with Alice and gets redirected to the existing room by their client + or server. +3. Alice tries to start a new DM with Bob. Like Bob in the last step, Alice is redirected + to the existing room. +4. Bob then starts a DM with Alice and Charlie. A new room is created. Similar redirection + semantics take place when the three parties attempt to duplicate the room. + +**Simple glare** + +The slightly less happy path of DMs: when servers collide. + +1. Alice and Bob reside on different homeservers. +2. Alice starts a DM with Bob, who does not accept nor reject it. +3. Bob starts a DM with Alice, who does not accept not reject it. +4. Alice accepts Bob's invite. Alice's server realizes this is a duplicate room and sends + a tombstone to the room, pointing at the DM Alice created earlier. + * During this, Bob's server would have originally flagged the room as a DM for Bob and + Alice, but the tombstone signals to Bob's homeserver that something is up and assumes + that Bob has no DM with Alice (yet). +5. Bob gets a tombstone in Alice's DM and realizes there's a pending invite. Bob accepts. +6. Bob's homeserver sees that the room is a DM and flags it as the DM for Bob and Alice. + +Alice and Bob's homeserver have come to the same conclusion. In the worst case, both servers +could accept the invites at the same time and conflict with each other on the tombstone. The +worst case for this is a split DAG on the tombstone with two tombstone state events, resolved +through state resolution. Both servers would have still come to the same conclusion on which +room to use so either of the tombstone events sent could be considered a no-op. + +**De-duplication during migration / tombstone tree** + +1. Alice has 3 existing DMs with Bob, who resides in both rooms. +2. The server updates (Alice and Bob happen to be on the same server in this example, however + the scenario works cross-server too). +3. The server realizes that Alice and Bob share multiple DMs and pick the oldest DM. The + server tombstones the other 2 rooms. +4. Alice and Bob now have exactly one DM with each other. + +The room map looks a bit complicated now, but there's a tree of rooms where two point to +a single room. Clients aren't exactly expected to represent this nicely but can try if they +want to. There will be no `predecessor` for them to work off of. + +If the canonical DM room was then upgraded or otherwise tombstoned itself, the tree starts +to become more linear. + +**Tombstone tree with glare** + +Following the example of a "tombstone tree" then creating a conflicting room with glare +the servers should resolve to use the canonical room identified by the tombstone tree. The +conflicting room should be tombstoned to point at the canonical room. + +**Change in power levels, join rules, or membership** + +If the power levels or join rules change then the server needs to re-evaluate the room. If +the room ends up being no longer a DM, the server must flag it as such to the relevant clients +and *not* send a tombstone into the room. The parties are considered to not have a DM. + +If the membership changes such that someone ends up leaving the room (on their own or through +a kick/ban) then the DM is no longer considered a DM, similar to the paragraph above. + +If the room starts to look like a DM again (members rejoin, join rules become acceptable again, +etc) then the conflict resolution algorithm might take place depending on the user's DMs. + +**Malicious takeover of a DM** + +1. Alice and Bob are on different homeservers, but don't have to be for this scenario. +2. Alice starts a DM with Bob. Bob accepts. +3. Bob's homeserver sends a tombstone into the DM to point it at Matrix HQ. +4. Both Alice and Bob no longer have a viable DM: both homeservers do not consider Matrix HQ + as the replacement room, and do not treat the tombstoned room as a DM. + +Ultimtely this is a wasted effort: both homeservers would start a new DM after the room was +directed at HQ. + +**The cases covered by leaving DMs above** + +This proposal covers a few cases that servers are supposed to consider when a user (or many +users) leaves a room. Please reference that section for more detail. + + +## Tradeoffs / issues + +This is really complicated for servers to implement. The alternative is that the client +does all the work (see below in 'Alternative solutions') however this is deemed unreasonable +by the author and early reviewers of the proposal. + +This allows for bots and personal assistants to be residents of a room. There is an argument +for restricting DMs to just important users (excluding any chance of unimportant users), +namely to ensure that security is not sacrificed through leaky bots/assistants. The author +believes there is a stronger argument for assistant-style/audit bots in a DM, such as a reminder +bot or Giphy bot. The rationale being that although bots pose a security risk to the conversation, +the rooms are supposed to have a history visibility setting which prevents the history from +being exposed to the new parties. Important parties additionally should have the power to remove +the new user account from the room if they don't agree. + +This proposal enables encryption by default for DMs. Encryption in the specification is +moderately incomplete as of writing, making the move to enable encryption by default somewhat +questionable. For instance, a large number of bots and bridges in the ecosystem do not currently +support encryption and generally fall silent in encrypted rooms. Bots in today's ecosystem +can use [Pantalaimon](https://github.com/matrix-org/pantalaimon) or add dedicated support +if they so choose, however bridges are still unable to decrypt messages even with Pantalaimon. +Bridges being able to support encryption (they'll need EDUs and a device ID) is a problem +for a different MSC, and likely a different spec version. In the meantime, this proposal +specifically does not prohibit DMs which are unencrypted and bridges can start DMs as such, +or refuse to participate in DMs which are encrypted. Generally speaking, bridges in the +current ecosystem start the DMs with users instead of the users contacting the bridge, +however there are some prominent examples of this being reversed (IRC, most puppet bridges, +etc). Clients can combat this by identifying bridge namespaces (also a future spec problem) +and offering users the option to not encrypt the room, or by just having that checkbox +always present. Ideally, this proposal and cross-signing, better encryption for bridges, +etc all land concurrently in the next specification version. + +This proposal is vague about which encryption algorithm to support. This is an intentional +choice by the author to support the possibility of a given encryption algorithm being +deemed unsuitable by Matrix in the future. For example, if a given algorithm was broken or +found to have security flaws it would be difficult to update the specification if the preset +had baked in a specific algorithm. Instead, the proposal asks the specification to make +a recommendation and for servers to think critically of the available algorithms (of which +only one is actually available at the moment) to make the best decision for their users. + + +## Alternative solutions + +The sugar APIs are a bit awkward for the value they provide. Bridges are more commonly in +need of direct chats and thin bots arguably can still sync their way through life. Bridges, +and possibly clients, could receive changes to their DMs over a purpose-built room managed +by the server. Similar to server notices, the server could maintain state events in a readonly +immutable room for the user/client to sit in and receive updates. The author feels as though +this is more complicated than it should be, requiring servers to maintain hundreds, thousands, +or even millions of dedicated rooms for users which can easily overwhelm an appservice. For +instance, the Freenode IRC bridge has millions of users which would equate to millions of +rooms which all need to be synced down the appservice pipe which is already fairly congested. + +Servers could be less involved and all this work could be done by the client. Although +easier for servers, clients are more likely to run into conflicts between themselves. The +most common type of conflicts for clients to have is updating account data at the same time, +which happens a lot more often than it should. When there's conflicts in updating account +data, there's a strong possibility that data is lost due to overwrites. Sure, the client +could be told to use room tagging instead however a similar problem occurs when one client +wants to add a tag and another wants to remove it. An earlier draft of this proposal did +cover what a mostly client-side implementation of immutable DMs could look like: it can +be found [here](https://gist.github.com/turt2live/6a4e21437508bce76be89d6cbaf65723). + +Servers could be even more involved and require a whole new room version to change the +event auth rules for DMs. The author strongly believes that this is far too complicated +and way overkill for what Matrix needs/wants to achieve, despite having written an earlier +draft of this proposal calling for exactly that. Many of the problems with this approach +are described in that draft, found [here](https://gist.github.com/turt2live/ed0247531d07c666b19dd95f7471eff4). + +There is also an argument that the existing solution is fine and clients should just +make it work for their purposes. The author believes this is an invalid argument given +the introduction of this proposal and the problems highlighted regarding client-driven +DMs. + + +## Security considerations + +Some clients are investigating the usage of immutable DMs for security and privacy related +functionality of their design. For example, client designers are looking into handling key +verification and cross-signing within the immutable DM (if such a concept existed) rather than +in a floating dialog. To accomplish this in a safe and secure way it may be desirable to have +high levels of enforcement regarding immutable DMs. This may increase the user's comfort in +knowing that none of the users in the room can manipulate it. Clients which need this level of +confidence may wish to ignore insecure DMs and attempt to start new ones by upgrading the +existing DM through a predefined `preset` (ideally acquiring permission first). + +There is a theoretical scenario where a homeserver or user could maliciously prevent a user +from opening a new DM with them. This is considered a feature given modules like ignoring users +exists, however a homeserver/user could continously set up a scenario where an existing DM +becomes unjoinable while sending tombstones for all new DM rooms which point to the unjoinable +room. This has a largely social impact on the room that technology cannot resolve (if people +are going to be mean, they're going to be mean). Attempts to alter the DM such that the user +cannot join without being notified are possible (changing the rejoin rules) however in those +cases both homeservers should be considering the room as no longer a DM, unless of course +the homeserver was being the malicious actor. Because of how tombstones currently work in +Matrix, users would have to perform an action to try and join the new DM and eventually the +user may get frustrated with the other user and ignore them, breaking the cycle of new DMs +being created. + + +## Pitfalls with the current system + +The existing DM system in Matrix support multiple DMs with a user, however that system is +not necsarily more complex than this proposal (in fact, it's probably simpler than what +is proposed here). The existing system has the disadvantage of being non-deterministic where +two users in the same room can have a different view of that room in their clients: Alice +can see it as a DM whereas Bob sees it as just another room. Having non-deterministic DMs +can lead to conversational confusion ("hey Bob, let's go to our DM" and Bob asking where that +is) or uncertainty about where to send sensitive information. + +Using the example of clients wanting to use DMs as a channel for exchanging key information, +knowing that all parties in the conversation perceive it as a DM is important. If the +verification were to land in a random room, the other party may miss it or be concerned it is +not a real verification request. + +Further, the existing system is managed completely client-side and every client is expected +to clone the same logic. By pushing the DM calculations to the server, all the client has to +do is render them. Most messenger Matrix clients want or have a concept of DMs, and instead of +duplicating the same code over and over the server could just inform the client of its DMs. + + +## Conclusion + +Immutable DMs are a hard problem to solve and involve a bunch of tiny changes to multiple parts +of the specification. Of all the iterations of this proposal, this proposal is targeted at being +a safe balance of client complexity and user safety, sacrificing the server's complexity in favour +of predictable results. Most clients are interested in implementing DMs in some capacity, and this +proposal helps make it less painful and more reliable for them. Users should feel safer to use +DMs with this proposal given the enabled encryption and permissions models imposed on the room. diff --git a/proposals/images/2199-dms-client.png b/proposals/images/2199-dms-client.png new file mode 100644 index 00000000000..49eda49844a Binary files /dev/null and b/proposals/images/2199-dms-client.png differ