From b215c2413182e97b94014d8964d22ea33fc60dad Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 24 Jul 2019 14:18:10 -0600 Subject: [PATCH 01/39] Over-the-top MSC for immutable DMs --- proposals/0000-immutable-dms.md | 287 ++++++++++++++++++++++++++++++++ 1 file changed, 287 insertions(+) create mode 100644 proposals/0000-immutable-dms.md diff --git a/proposals/0000-immutable-dms.md b/proposals/0000-immutable-dms.md new file mode 100644 index 00000000000..2e1697e7c17 --- /dev/null +++ b/proposals/0000-immutable-dms.md @@ -0,0 +1,287 @@ +# Immutable DMs + +In the messaging space it is common for apps to expose exactly 1 chat for a conversation +between two users, and in some cases a specific set of users. Matrix currently does not +adequately support this as it allows for multiple rooms to represent a DM between two users. +Although supporting multiple chats with individuals is potentially useful for some use cases, +clients are increasingly wanting to be able to support exactly one DM for a user. + +The Fractal team has previously published a blog post for the Banquets versus Barbecues use +cases (available [here](https://blogs.gnome.org/tbernard/2018/05/16/banquets-and-barbecues/)). +In short, Banquets are public rooms and Barbecues are DMs and small private chats (family, +friends, colleagues, etc). This proposal aims to address the lacking Barbecue support in +Matrix. + +Any messaging app is likely to have a similar desire as Fractal to differentiate DMs from +all the noise, and Riot in particular is looking at using the DM between users for a more +personal experience within the app (such as key verification between users). This proposal +focuses on immutable DMs between users to assist in identifying the "One True DM" between +users and prevent the room's scope from increasing. + +For comparison to Matrix, the following chat apps/platforms all support and encourage a +single DM between users. They all redirect the user to an existing chat with the target +user, creating a new DM if needed. + +* Slack +* Discord +* IRC +* Telegram +* Gitter +* Facebook Messenger +* Twitter +* Hangouts +* Instagram +* Rocket.Chat +* Steam +* WhatsApp +* SMS +* and probably many others... + +Platforms which allow/encourage users to have multiple DMs with a specific person are: +* Matrix (currently) +* Email +* XMPP (technically speaking) + +This proposal covers how clients and servers can work together to encourage and have exactly +one DM between users. These are known as immutable DMs as users are unable to alter the rooms +to make them become regular rooms (they'll always be DMs). + +**Disclaimer**: this proposal explores the most extreme direction immutable rooms could take, +up to and including altering event auth rules to achieve immutability and consistency. Not +every aspect of this proposal is required for it to move forward. + +## Proposal + +The existing direct chat module in Matrix allow for multiple users in a direct chat and multiple +rooms to act as a direct chat between users. Rooms are identified by the `m.direct` account data +event and through invites using a similar property. More information about direct chats as they +currently stand in Matrix can be read here: https://matrix.org/docs/spec/client_server/r0.5.0#id185 + +The `m.direct` account data event is to be deprecated and replaced with `m.direct_chats` (defined +below). The reason for replacing rather than extending the existing event type is account data +cannot be versioned, and `m.direct` was not created with extensibility in mind: one would have +to encode information in the form of user IDs to be compatible with existing clients. Therefore, +a new event type is introduced. Clients which support immutable DMs should ignore `m.direct` +entirely. + +`m.direct_chats` looks as follows: +```json +{ + "type": "m.direct_chats", + "content": { + "direct": { + "@alice:example.org": "!room_a:example.org", + "@bob:example.org": "!room_b:example.org" + }, + "group": { + "!room_c:example.org": ["@alice:example.org", "@bob:example.org"] + } + } +} +``` + +The event maps target users to rooms. Rooms under the `group` property are mapped by room ID +as a hopefully key to find the room or cache it in an easier structure for the client. The +account data is not meant to be a source of truth, just a faster lookup than clients scanning +room state. Clients are responsible for maintaining this account data. (**TODO: If we make it +part of the protocol like below, should the server manage this?**). + +Clients rendering the account data event above would result in something like so: + +![alice-bob-over-banquet](https://i.imgur.com/h4BHZSw.png) + +In order to make the rooms immutable, a new preset, `immutable_direct_chat`, is introduced. The +preset has the following effects on the room state: + +* Join rules of `invite`. +* History visibility of `shared`. +* Guest access of `can_join`. +* Power levels with the following non-default structure: + ```json + { + "events_default": 0, + "state_default": 50, + "users_default": 0, + "ban": 100, + "invite": 100, + "kick": 100, + "redact": 50, + "notifications": { + "room": 50 + }, + "events": { + "m.room.name": 100, + "m.room.avatar": 100, + "m.room.topic": 100, + "m.room.history_visibility": 100, + "m.room.power_levels": 100, + "m.room.join_rules": 100, + "m.room.encryption": 100, + "m.room.canonical_alias": 100 + }, + "users": { + + } + } + ``` +* Users invited to the room get power level 50. +* The room creation event gets a field of `m.direct` containing an array of users that were + invited to the room. The generated invited users array overrides any values in the + `creation_content`. +* Encryption is enabled by default using the `m.megolm.v1.aes-sha2` algorithm. + +The power level event content described above is applied *after* the users are invited. If it +wasn't, the users could not be invited. When using the immutable DM preset, servers MUST ignore +the `power_level_content_override` field of the `/createRoom` request. + +The `m.direct` field of a creation event is to be preserved by the redaction algorithm (provided +[MSC2176](https://github.com/matrix-org/matrix-doc/pull/2176) doesn't ban redactions of the +create event, in which case this requirement is satisfied). + +Clients SHOULD use the `m.direct` field on the creation event to determine if the room is a DM. +The power levels created by the preset reinforce this restriction. A room must not be considered +a DM if the join rules are anything other than `invite`. + +Servers SHOULD assist clients by reusing room IDs for known DMs on repeated calls to `/createRoom` +using the same invited users and immutable DM preset. + +The rationale for preventing either party from manipulating the room is to ensure that there is +equal representation of the room from both parties (automatically named, automatic avatar, etc). +Users generally expect that a DM will be predictably dressed for searching and scanning, meaning +that the other parties cannot change the aesthetics of the room. For predicatble history and +privacy, the room's history & join rules are locked out from the users. The users can upgrade the +room at any time to escape power level struggles, although this may not maintain the DM status. + +Bots and assistant users are not handled in this proposal - see the tradeoffs section for more +information on this. Invites are disabled as a result. + +Tombstoned (previously upgraded) immutable DMs MUST be removed from the `m.direct_chats` account +data event and cannot be considered an immutable DM. Tombstoned immutable DMs are dead rooms on +the user's account, preserved for history retention. This applies later in this proposal when +the server is required to make decisions on immutable DM rooms. + +#### Upgrading DM rooms + +There may come a time where DM rooms need to be upgraded. When DM rooms are upgraded through the +`/upgrade` endpoint, servers MUST preserve the `content` of the previous room's creation event and +otherwise apply the `immutable_direct_chat` preset over the new room. If conflicts arise between +the previous room's state and the state applied by the preset, the preset takes preference over +the previous room's state. + +Servers MUST create the new room by using the the room creation rules listed earlier in this proposal +(`/createRoom`, but applying power level changes after invites). This means inviting the other +parties of the previous room to the new room automatically and ensuring the room state is as +described above. + +#### Migrating existing DMs + +Existing DMs cannot be added to the `m.direct_chats` account data event because they are lacking +the `m.direct` field on the creation event. In order to combat this, a new parameter is to be +added to the `/upgrade` endpoint: `preset`. This behaves similar to the above section on +upgrading rooms: the state implied by the `preset` overrides the previous room's state to more +easily return a room to its defaults. + +When `preset` is not provided, no default is assumed unless the room was an immutable DM. When +the `preset` is explicitly given as `immutable_direct_chat`, the rules about upgrading a room +above take affect. + +An upgrade request body can now look like: +```json +{ + "new_version": "5", + "preset": "immutable_direct_chat" +} +``` + +An immutable DM room can be upgraded to a mutable room (ie: a `preset` of `public_chat` will +create a new public room, losing the DM status on the room and the users). + +After upgrading, the client SHOULD update the `m.direct_chats` account data event. Similar +behaviour should apply when the other parties join the upgraded room. + +The users of the previous room are the seed users for the new room, as described above in +upgrading a DM room. + +#### Auth rule changes + +In order to enforce that only the invited users are ever in the immutable DM, servers MUST +reject membership events for members not listed in the `m.direct` array on the creation +event. + +#### Handling conflicting rooms + +The federation invite API allows for a user to receive an invite to an unverified DM room +with potentially conflicting participants with an existing DM room. Altering the invite API +to include signed membership and creation events is one possibility for ensuring that the +room is unique, however that becomes difficult and expansive to maintain (the target server +would likely end up trying to rebuild the room's state by requesting the gaps in the DAG). +Instead, it is proposed that clients when joining a room check for the `m.direct` field on +the creation event. + +Clients MUST always check the flag when they join a new room, not just when they are accepting +an invite. If the room conflicts with an immutable DM the client already knows about, a tombstone +state event must be sent to the older room with a reference to the newly joined room. If the +tombstone event fails to send, the client and server MUST consider the room tombstoned anyways. + + +## Tradeoffs / issues + +This proposal has a number of downsides which may or may not be acceptable to readers. + +#### Bots aren't supported + +This proposal doesn't support bots or personal assistants. They both end up being treated as +first-class users, resulting in a group DM. This proposal doesn't cover trying to identify +unimportant users, however in future when bots can be identified this module can be adapted. + +One possible option which requires minimal changes (unlike [MSC1206](https://github.com/matrix-org/matrix-doc/issues/1206)) +is to have the Integration Manager suggest/expose the bots it supports and for clients to +include the list of relevant users in an `unimportant_invites` array on `/createRoom`. The +invites would go out normally to the bots, however they would not have power or identification +as part of the immutability of the DM. How a user adds a bot after the DM is created is +undefined, hence the limitation being listed here rather than being described as a solution +in the proposal. + +Invites after the DM is created could potentially be handled by automatically treating them +as unimportant (doesn't affect the room name, avatar, etc). The unimportant users would not +have anythign higher than a default power level in the room. + +#### Clients do all the heavy lifting + +Instead of having clients track tombstones, room members, creation events, and account data they +could instead rely on the server to track all this for them. This could take the shape of the +client having read-only access to `m.direct_chats` account data and the server keeping it updated +for the user. + +#### Migration plan sucks real bad + +Upgrading your DMs sounds like a horrible way to slow down the server and feels tedious. Clients +would additionally have to put in a non-trivial amount of work to migrate users, handling common +cases of users with hundreds (and thousands) of direct chats. + + +## Alternative solutions + +This proposal is intentionally written at the far side of the spectrum where enforcement is an +essential part of how the DM works. Just as easily this could be flagged as a state event in +the room (`m.room.direct`?) which when combined with an invite-only room results in a cosmetic +immutable DM. + +This proposal could also take a stance somewhere in the middle in the spectrum and mix cosmetic +requirements with enforcement, however not to the point of altering the auth rules for rooms. + + +## Security considerations + +Some clients are investigating the usage of immutable DMs for security and privacy related +functionality of their design. For example, client designers are looking into handling key verification +and cross-signing within the immutable DM (if such a concept existed) rather than in a floating +dialog. To accomplish this in a safe and secure way it may be desirable to have high levels of +enforcement regarding immutable DMs. This may increase the user's comfort in knowing that none +of the users in the room can manipulate it. + + +## Conclusion + +Immutable DMs are a hard problem to solve and involve a bunch of tiny changes to multiple parts +of the specification for full enforcement. From 912cee23b4668b1f1dc4b8e9e3409e3e77db9045 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 24 Jul 2019 14:58:35 -0600 Subject: [PATCH 02/39] Softer MSC --- proposals/0000-immutable-dms.md | 185 ++++++++++++++++---------------- 1 file changed, 93 insertions(+), 92 deletions(-) diff --git a/proposals/0000-immutable-dms.md b/proposals/0000-immutable-dms.md index 2e1697e7c17..f868ed3d716 100644 --- a/proposals/0000-immutable-dms.md +++ b/proposals/0000-immutable-dms.md @@ -46,10 +46,6 @@ This proposal covers how clients and servers can work together to encourage and one DM between users. These are known as immutable DMs as users are unable to alter the rooms to make them become regular rooms (they'll always be DMs). -**Disclaimer**: this proposal explores the most extreme direction immutable rooms could take, -up to and including altering event auth rules to achieve immutability and consistency. Not -every aspect of this proposal is required for it to move forward. - ## Proposal The existing direct chat module in Matrix allow for multiple users in a direct chat and multiple @@ -69,22 +65,21 @@ entirely. { "type": "m.direct_chats", "content": { - "direct": { - "@alice:example.org": "!room_a:example.org", - "@bob:example.org": "!room_b:example.org" - }, - "group": { - "!room_c:example.org": ["@alice:example.org", "@bob:example.org"] - } + "rooms": [ + "!room_a:example.org", + "!room_b:example.org", + "!room_c:example.org" + ] } } ``` -The event maps target users to rooms. Rooms under the `group` property are mapped by room ID -as a hopefully key to find the room or cache it in an easier structure for the client. The -account data is not meant to be a source of truth, just a faster lookup than clients scanning -room state. Clients are responsible for maintaining this account data. (**TODO: If we make it -part of the protocol like below, should the server manage this?**). +The event lists all the rooms which are considered DMs by other clients. Ideally, all the rooms +in the list will have been created using the immutable DMs preset defined below. Mapping out +which users are participating in DMs is difficult to represent through JSON, and the identification +of important users is simple, therefore the event is not believed to need more metadata about +the rooms. The event schema additionally supports future expansion in that the array is nested +in a dedicated property, giving room for future changes to the event. Clients rendering the account data event above would result in something like so: @@ -103,7 +98,7 @@ preset has the following effects on the room state: "state_default": 50, "users_default": 0, "ban": 100, - "invite": 100, + "invite": 50, "kick": 100, "redact": 50, "notifications": { @@ -125,22 +120,22 @@ preset has the following effects on the room state: } ``` * Users invited to the room get power level 50. -* The room creation event gets a field of `m.direct` containing an array of users that were - invited to the room. The generated invited users array overrides any values in the - `creation_content`. * Encryption is enabled by default using the `m.megolm.v1.aes-sha2` algorithm. -The power level event content described above is applied *after* the users are invited. If it -wasn't, the users could not be invited. When using the immutable DM preset, servers MUST ignore -the `power_level_content_override` field of the `/createRoom` request. +Clients which support the direct messaging module of the Client-Server API MUST use the new +`m.direct_chats` account data to determine which DMs the user is in. Clients MUST use the power +levels of users in those DMs to determine who they are with: users with power greater than or +equal to `state_default` are considered "important". When a user attempts to start a new direct +chat with an important user, the existing room should be re-used instead. Unimportant users are +all other users in the room, such as bots and personal assistants. Unimportant users do not affect +who the DM is considered to be with. -The `m.direct` field of a creation event is to be preserved by the redaction algorithm (provided -[MSC2176](https://github.com/matrix-org/matrix-doc/pull/2176) doesn't ban redactions of the -create event, in which case this requirement is satisfied). +For example, if Alice and Bob (both power level 50) are in a DM and invite Charlie, Charlie is +not to be considered important for the DM. This means that when Alice or Bob starts a chat with +Charlie, the existing room with all 3 of them will not be re-used. However, if Alice were to start +a chat with Bob, the existing room containing Bob and Charlie would be re-used. -Clients SHOULD use the `m.direct` field on the creation event to determine if the room is a DM. -The power levels created by the preset reinforce this restriction. A room must not be considered -a DM if the join rules are anything other than `invite`. +A room must not be considered a DM if the join rules are anything other than `invite`. Servers SHOULD assist clients by reusing room IDs for known DMs on repeated calls to `/createRoom` using the same invited users and immutable DM preset. @@ -152,13 +147,14 @@ that the other parties cannot change the aesthetics of the room. For predicatble privacy, the room's history & join rules are locked out from the users. The users can upgrade the room at any time to escape power level struggles, although this may not maintain the DM status. -Bots and assistant users are not handled in this proposal - see the tradeoffs section for more -information on this. Invites are disabled as a result. +Unimportant users MUST NOT be used to name the room (cannot appear in heroes or in automatic naming +for rooms). Tombstoned (previously upgraded) immutable DMs MUST be removed from the `m.direct_chats` account data event and cannot be considered an immutable DM. Tombstoned immutable DMs are dead rooms on -the user's account, preserved for history retention. This applies later in this proposal when -the server is required to make decisions on immutable DM rooms. +the user's account, preserved for history retention. + +DMs may involve multiple important users, and are entirely valid in the eyes of this proposal. #### Upgrading DM rooms @@ -166,24 +162,35 @@ There may come a time where DM rooms need to be upgraded. When DM rooms are upgr `/upgrade` endpoint, servers MUST preserve the `content` of the previous room's creation event and otherwise apply the `immutable_direct_chat` preset over the new room. If conflicts arise between the previous room's state and the state applied by the preset, the preset takes preference over -the previous room's state. +the previous room's state. Servers can identify whether a room is a DM or not by peeking at the +user's account data: if the room is listed under the upgrading user's `m.direct_chats`, it is +considered a DM. The room does not have to be listed on all user's direct chats, just the user +who is performing the upgrade. -Servers MUST create the new room by using the the room creation rules listed earlier in this proposal -(`/createRoom`, but applying power level changes after invites). This means inviting the other -parties of the previous room to the new room automatically and ensuring the room state is as -described above. +Servers MUST create the new room by using the the room creation rules listed earlier in this +proposal. This means inviting the other parties of the previous room to the new room automatically +and ensuring the room state is as described above. #### Migrating existing DMs -Existing DMs cannot be added to the `m.direct_chats` account data event because they are lacking -the `m.direct` field on the creation event. In order to combat this, a new parameter is to be -added to the `/upgrade` endpoint: `preset`. This behaves similar to the above section on -upgrading rooms: the state implied by the `preset` overrides the previous room's state to more -easily return a room to its defaults. - -When `preset` is not provided, no default is assumed unless the room was an immutable DM. When -the `preset` is explicitly given as `immutable_direct_chat`, the rules about upgrading a room -above take affect. +Existing DMs can be automatically added to the `m.direct_chats` account data event provided they +meet the power level requirements. Most DMs at this point in Matrix will have been created using +the `trusted_private_chat` preset, giving both participants admin (power level 100) rights in the +room, therefore making them eligible for inclusion in the new direct chats account data. Clients +are encouraged to walk the user through the upgrade/migration process instead of automatically +migrating data. This is to prevent potential collisions between clients while the process happens. + +Users are encouraged to upgrade their rooms to be immutable: this can be done in two ways. The +first way is altering the power levels and other state events to match the preset defined above. +The second is using the `/upgrade` endpoint with a new field: `preset`. This proposal encourages +clients to use the first method when migrating existing DMs for users, however the second option +is proposed as an addition to the protocol. + +The `preset` field on `/upgrade` acts similar to `/createRoom` and the behaviour described above +regarding upgrading a room: the state implied by the `preset` takes precedence over existing state +in the old room. This applies regardless of the preset being used. When not provided, no default +preset is assumed (ie: all applicable state is transferred without any taking precedence over +another). An upgrade request body can now look like: ```json @@ -196,18 +203,13 @@ An upgrade request body can now look like: An immutable DM room can be upgraded to a mutable room (ie: a `preset` of `public_chat` will create a new public room, losing the DM status on the room and the users). -After upgrading, the client SHOULD update the `m.direct_chats` account data event. Similar -behaviour should apply when the other parties join the upgraded room. +After upgrading, the client MUST update the `m.direct_chats` account data event. Similar +behaviour should apply when the other parties join the upgraded room (for the client which +joined/accepted the invite). The users of the previous room are the seed users for the new room, as described above in upgrading a DM room. -#### Auth rule changes - -In order to enforce that only the invited users are ever in the immutable DM, servers MUST -reject membership events for members not listed in the `m.direct` array on the creation -event. - #### Handling conflicting rooms The federation invite API allows for a user to receive an invite to an unverified DM room @@ -215,36 +217,32 @@ with potentially conflicting participants with an existing DM room. Altering the to include signed membership and creation events is one possibility for ensuring that the room is unique, however that becomes difficult and expansive to maintain (the target server would likely end up trying to rebuild the room's state by requesting the gaps in the DAG). -Instead, it is proposed that clients when joining a room check for the `m.direct` field on -the creation event. +Instead, it is proposed that clients check the power levels of the room to determine if the +room should be considered a direct chat. This renders the `is_direct` flag on invites only +useful for aesthetics for rendering invitations - the flag serves no other purpose if this +proposal were to be accepted. Clients MUST always check the flag when they join a new room, not just when they are accepting an invite. If the room conflicts with an immutable DM the client already knows about, a tombstone state event must be sent to the older room with a reference to the newly joined room. If the -tombstone event fails to send, the client and server MUST consider the room tombstoned anyways. +tombstone event fails to send, the client MUST consider the room tombstoned anyways. +#### Server assistance -## Tradeoffs / issues +As mentioned previously in this proposal, servers SHOULD already be re-using rooms for which +the user has a direct chat with another set of users. Similar restrictions SHOULD be in place +to ensure that clients do not accidentally add duplicate direct chats to the account data event. -This proposal has a number of downsides which may or may not be acceptable to readers. - -#### Bots aren't supported +#### Abandoning/leaving rooms -This proposal doesn't support bots or personal assistants. They both end up being treated as -first-class users, resulting in a group DM. This proposal doesn't cover trying to identify -unimportant users, however in future when bots can be identified this module can be adapted. +When an important user leaves or gets kicked/banned from the room the room MUST NOT be counted +as a DM anymore by any clients. For example, a 3 person DM where 1 person leaves would cause +all 3 users to lose the DM status on the room. The room is not downgraded to a 2 person DM due +to potential conflicts with existing DMs. -One possible option which requires minimal changes (unlike [MSC1206](https://github.com/matrix-org/matrix-doc/issues/1206)) -is to have the Integration Manager suggest/expose the bots it supports and for clients to -include the list of relevant users in an `unimportant_invites` array on `/createRoom`. The -invites would go out normally to the bots, however they would not have power or identification -as part of the immutability of the DM. How a user adds a bot after the DM is created is -undefined, hence the limitation being listed here rather than being described as a solution -in the proposal. +## Tradeoffs / issues -Invites after the DM is created could potentially be handled by automatically treating them -as unimportant (doesn't affect the room name, avatar, etc). The unimportant users would not -have anythign higher than a default power level in the room. +This proposal has a number of downsides which may or may not be acceptable to readers. #### Clients do all the heavy lifting @@ -253,35 +251,38 @@ could instead rely on the server to track all this for them. This could take the client having read-only access to `m.direct_chats` account data and the server keeping it updated for the user. -#### Migration plan sucks real bad +#### This is reliant on trust -Upgrading your DMs sounds like a horrible way to slow down the server and feels tedious. Clients -would additionally have to put in a non-trivial amount of work to migrate users, handling common -cases of users with hundreds (and thousands) of direct chats. +A user could remove a room from their direct chats without leaving the room, which the other +users in the room would be unaware of. This could potentially lead to confusion when that user +tries to initiate a new direct chat with the same users, however it is expected that "power users" +(people who are experienced with the protocol) will be the only ones doing this and likely for +good cause. ## Alternative solutions -This proposal is intentionally written at the far side of the spectrum where enforcement is an -essential part of how the DM works. Just as easily this could be flagged as a state event in -the room (`m.room.direct`?) which when combined with an invite-only room results in a cosmetic -immutable DM. - -This proposal could also take a stance somewhere in the middle in the spectrum and mix cosmetic -requirements with enforcement, however not to the point of altering the auth rules for rooms. +DMs could be enforced through atuh rule changes and server enforcement, however this feels +far too complicated and over the top for the simplicity of just using a preset to fix rooms. +An example of how this could look is described [here](https://gist.github.com/turt2live/ed0247531d07c666b19dd95f7471eff4). ## Security considerations Some clients are investigating the usage of immutable DMs for security and privacy related -functionality of their design. For example, client designers are looking into handling key verification -and cross-signing within the immutable DM (if such a concept existed) rather than in a floating -dialog. To accomplish this in a safe and secure way it may be desirable to have high levels of -enforcement regarding immutable DMs. This may increase the user's comfort in knowing that none -of the users in the room can manipulate it. +functionality of their design. For example, client designers are looking into handling key +verification and cross-signing within the immutable DM (if such a concept existed) rather than +in a floating dialog. To accomplish this in a safe and secure way it may be desirable to have +high levels of enforcement regarding immutable DMs. This may increase the user's comfort in +knowing that none of the users in the room can manipulate it. Clients which need this level of +confidence may wish to ignore insecure DMs and attempt to start new ones by upgrading the existing +DM through a predefiend `preset` (ideally acquiring permission first). ## Conclusion Immutable DMs are a hard problem to solve and involve a bunch of tiny changes to multiple parts -of the specification for full enforcement. +of the specification. Of the options described, this proposal focuses on making immutable DMs +accessible to modern clients without needing complicated migration processes whereas prior drafts +focused on enforcing immutability as much as possible. This proposal is targeted towards being +a balance between complete enforcement and reasonable expectations of users. From a9255a5cbba92504f77b173b19acb5a05086ef4d Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 28 Jul 2019 13:06:48 -0600 Subject: [PATCH 03/39] Minor touchups --- proposals/0000-immutable-dms.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/proposals/0000-immutable-dms.md b/proposals/0000-immutable-dms.md index f868ed3d716..bad4729f6ad 100644 --- a/proposals/0000-immutable-dms.md +++ b/proposals/0000-immutable-dms.md @@ -1,4 +1,4 @@ -# Immutable DMs +# Immutable DMs (middle ground) In the messaging space it is common for apps to expose exactly 1 chat for a conversation between two users, and in some cases a specific set of users. Matrix currently does not @@ -89,7 +89,7 @@ In order to make the rooms immutable, a new preset, `immutable_direct_chat`, is preset has the following effects on the room state: * Join rules of `invite`. -* History visibility of `shared`. +* History visibility of `invited`. * Guest access of `can_join`. * Power levels with the following non-default structure: ```json @@ -97,9 +97,9 @@ preset has the following effects on the room state: "events_default": 0, "state_default": 50, "users_default": 0, - "ban": 100, + "ban": 50, "invite": 50, - "kick": 100, + "kick": 50, "redact": 50, "notifications": { "room": 50 @@ -119,7 +119,7 @@ preset has the following effects on the room state: } } ``` -* Users invited to the room get power level 50. +* Users invited to the room get power level 50, including the creator. * Encryption is enabled by default using the `m.megolm.v1.aes-sha2` algorithm. Clients which support the direct messaging module of the Client-Server API MUST use the new @@ -171,6 +171,12 @@ Servers MUST create the new room by using the the room creation rules listed ear proposal. This means inviting the other parties of the previous room to the new room automatically and ensuring the room state is as described above. +Note: normally room upgrades modify the power levels in the old room in an attempt to mute all +participants. However, no one will have power to modify the power levels in the old room. +Therefore, servers MUST NOT fail to upgrade a room due to being unable to update the power levels +in the old room. This is considered acceptable by this proposal because upgraded DM rooms will +lose their DM status, making them at worst just another room for the user. + #### Migrating existing DMs Existing DMs can be automatically added to the `m.direct_chats` account data event provided they @@ -262,7 +268,7 @@ good cause. ## Alternative solutions -DMs could be enforced through atuh rule changes and server enforcement, however this feels +DMs could be enforced through auth rule changes and server enforcement, however this feels far too complicated and over the top for the simplicity of just using a preset to fix rooms. An example of how this could look is described [here](https://gist.github.com/turt2live/ed0247531d07c666b19dd95f7471eff4). From 05b9115564155823751acf1b33573e2120476a4a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 29 Jul 2019 18:15:23 -0600 Subject: [PATCH 04/39] What if we made it the server's problem --- proposals/0000-immutable-dms.md | 549 ++++++++++++++++++++++++-------- 1 file changed, 414 insertions(+), 135 deletions(-) diff --git a/proposals/0000-immutable-dms.md b/proposals/0000-immutable-dms.md index bad4729f6ad..97b27dca68f 100644 --- a/proposals/0000-immutable-dms.md +++ b/proposals/0000-immutable-dms.md @@ -1,4 +1,4 @@ -# Immutable DMs (middle ground) +# Immutable DMs (server-side middle ground edition) In the messaging space it is common for apps to expose exactly 1 chat for a conversation between two users, and in some cases a specific set of users. Matrix currently does not @@ -35,6 +35,8 @@ user, creating a new DM if needed. * Steam * WhatsApp * SMS +* Skype +* MSN Messenger * and probably many others... Platforms which allow/encourage users to have multiple DMs with a specific person are: @@ -42,51 +44,128 @@ Platforms which allow/encourage users to have multiple DMs with a specific perso * Email * XMPP (technically speaking) -This proposal covers how clients and servers can work together to encourage and have exactly -one DM between users. These are known as immutable DMs as users are unable to alter the rooms -to make them become regular rooms (they'll always be DMs). +This proposal covers how servers enforce exactly one DM (ideally immutable) between a set +of users. Clients have some responsibilities to render this appropriately, however the server +is expected to do most of the heavy lifting here. Immutable DMs are direct chats between +users where the participants can't change the room's aesthetics. For example, users can't +change the name or topic of the room, and they'll always be DMs. -## Proposal +This proposal supports "direct chats" or "DMs" in the sense of private conversations with +a set of users. This means that a DM between 3 users is possible, however the common case +is likely to be 1:1 (2 person) chats. -The existing direct chat module in Matrix allow for multiple users in a direct chat and multiple -rooms to act as a direct chat between users. Rooms are identified by the `m.direct` account data -event and through invites using a similar property. More information about direct chats as they -currently stand in Matrix can be read here: https://matrix.org/docs/spec/client_server/r0.5.0#id185 -The `m.direct` account data event is to be deprecated and replaced with `m.direct_chats` (defined -below). The reason for replacing rather than extending the existing event type is account data -cannot be versioned, and `m.direct` was not created with extensibility in mind: one would have -to encode information in the form of user IDs to be compatible with existing clients. Therefore, -a new event type is introduced. Clients which support immutable DMs should ignore `m.direct` -entirely. +## Proposal -`m.direct_chats` looks as follows: +In short, the entire direct chat module in Matrix is deprecated by this proposal. The direct +chat module relies on clients behaving themselves and tracking all the information for themselves +while somehow coming to the same conclusions as any other client the user happens to run. In +practice this becomes very difficult to do and maintain, particularly given the direct chats +are stored in an account data blob. If two clients were to update the account data at the same +time (which happens *a lot*), the clients would end up overwriting each other. + +Direct chats consist of "important" and "unimportant" users to allow for bots and personal +assistants in a chat. Unimportant users don't affect the DM-ness of the room and are essentially +observers on the room (with speaking rights). How classification is done is defined later in +this proposal. + +Direct chats are now expected to be identified and labelled by the server through the `summary` +response field on `/sync`. The `m.heroes` MUST contain the important users in a DM, regardless +of the presence of a name, canonical alias, etc on the room. The heroes must never be truncated +for a DM, and must not contain the syncing user's ID. A new field, `m.kind`, is mandatory for +direct chats which can only be `m.dm`. Future values may be added in a future proposal. Like +the other fields in the summary, `m.kind` may be omitted if there was no change in the field. + +For the purposes of demoting a DM from its DM status, `m.kind` can also be the literal `null` +to identify it as "unknown or irrelevant kind". + +Implementation note: clients can identify whether a room is a 1:1 or multi-way chat by looking +at the `m.kind` and `m.heroes`: if the kind is `m.dm` and there's only 1 user in `m.heroes` +then the room is a 1:1 chat. If the kind is `m.dm` and there's 2+ users in the `m.heroes` then +it is a multi-way/private group chat. + +With irrelevant fields not included, an example sync response would be: ```json { - "type": "m.direct_chats", - "content": { - "rooms": [ - "!room_a:example.org", - "!room_b:example.org", - "!room_c:example.org" - ] + "rooms": { + "join": { + "!a:example.org": { + "summary": { + "m.heroes": ["@alice:example.org"], + "m.joined_member_count": 1, + "m.invited_member_count": 1, + "m.kind": "m.dm" + } + }, + "!b:example.org": { + "summary": { + "m.heroes": ["@bob:example.org"], + "m.joined_member_count": 2, + "m.invited_member_count": 0, + "m.kind": "m.dm" + } + }, + "!c:example.org": { + "summary": { + "m.heroes": ["@alice:example.org", "@bob:example.org"], + "m.joined_member_count": 2, + "m.invited_member_count": 0, + "m.kind": "m.dm" + } + }, + "!d:example.org": { + "summary": { + "m.joined_member_count": 5, + "m.invited_member_count": 2, + } + } + } } } ``` -The event lists all the rooms which are considered DMs by other clients. Ideally, all the rooms -in the list will have been created using the immutable DMs preset defined below. Mapping out -which users are participating in DMs is difficult to represent through JSON, and the identification -of important users is simple, therefore the event is not believed to need more metadata about -the rooms. The event schema additionally supports future expansion in that the array is nested -in a dedicated property, giving room for future changes to the event. - -Clients rendering the account data event above would result in something like so: +Clients rendering the above should end up with something like the following: ![alice-bob-over-banquet](https://i.imgur.com/h4BHZSw.png) -In order to make the rooms immutable, a new preset, `immutable_direct_chat`, is introduced. The -preset has the following effects on the room state: + +Servers are expected to be able to identify direct chats for users and flag them appropriately +in the room summary. How existing DMs are migrated is covered later in this proposal. The +server must use these rules to determine if a room is a direct chat: +* The join rules are `invite`. +* The user's most recent membership event has `"m.direct": true` in the `content`. +* The room has an explicit power level event (running a DM without a PL event is not supported). +* The room has at least 2 important users in it. +* The user themselves is important in the room. +* The room is not tombstoned. +* The room is not soft-tombstoned by the server. +* No important users have left the room (through kick, ban, or other means). + +Assuming no other conflicts arise (duplicate chats, etc) the room is considered a DM between +the important users in the room. Important users are identified simply by having a power level +greater than or equal to the `state_default` power level requirement. + +The server MUST maintain the `m.direct` flag when updating or transitioning any user's membership +event. The server SHOULD refuse to let the user manipulate the flag directly through `PUT /state` +and similar APIs (such as inviting users to a room). Servers MUST consider a value of `false` +the same as the field not being present. + +Servers are not expected to identify a room as a direct chat until the server resides in the +room. Servers MUST populate the room summary for invites if the server is a resident of the +room. Clients SHOULD use the room summary on invites and joined rooms to identify if a room +is a direct chat or not. Where the summary is not available and an `m.direct` field is on the +invite event, clients should be skeptical but otherwise trusting of the room's DM status. + +Servers SHOULD attempt to identify (or re-affirm) rooms as a DM whenever the relevant state +events in the rules above change or are updated. + + +#### Creating DMs + +The room creation presets `private_chat` and `trusted_private_chat` are deprecated and to be +removed in a future specification version. Clients should stop using those presets and instead +use `immutable_direct_chat` as defined here. The new preset has the following effects on the +room state: * Join rules of `invite`. * History visibility of `invited`. @@ -119,26 +198,21 @@ preset has the following effects on the room state: } } ``` -* Users invited to the room get power level 50, including the creator. -* Encryption is enabled by default using the `m.megolm.v1.aes-sha2` algorithm. - -Clients which support the direct messaging module of the Client-Server API MUST use the new -`m.direct_chats` account data to determine which DMs the user is in. Clients MUST use the power -levels of users in those DMs to determine who they are with: users with power greater than or -equal to `state_default` are considered "important". When a user attempts to start a new direct -chat with an important user, the existing room should be re-used instead. Unimportant users are -all other users in the room, such as bots and personal assistants. Unimportant users do not affect -who the DM is considered to be with. - -For example, if Alice and Bob (both power level 50) are in a DM and invite Charlie, Charlie is -not to be considered important for the DM. This means that when Alice or Bob starts a chat with -Charlie, the existing room with all 3 of them will not be re-used. However, if Alice were to start -a chat with Bob, the existing room containing Bob and Charlie would be re-used. - -A room must not be considered a DM if the join rules are anything other than `invite`. - -Servers SHOULD assist clients by reusing room IDs for known DMs on repeated calls to `/createRoom` -using the same invited users and immutable DM preset. + The power level event is considered the `power_level_content_override` implicitly and + therefore applied at step 0 of the room creation process. +* Users invited to the room get power level 50, including the creator. Important users + (those invited and the creator) MUST have `"m.direct": true` in their membership event + content. +* Encryption is enabled by default using the most preferred megolm algorithm in the spec. + Currently this would be `m.megolm.v1.aes-sha2`. + +The preset prevents the use of `visibility`, `room_alias_name`, `name`, `topic`, `initial_state`, +and `power_level_content_override` during creation. Servers MUST reject the request with +`400 M_BAD_STATE` when the request contains conflicting properties. Future extensions to +`/createRoom` are expected to be included in the forbidden list where appropriate. + +Servers MUST return an existing room ID if the server already knows about a DM between the +important users. The rationale for preventing either party from manipulating the room is to ensure that there is equal representation of the room from both parties (automatically named, automatic avatar, etc). @@ -147,14 +221,32 @@ that the other parties cannot change the aesthetics of the room. For predicatble privacy, the room's history & join rules are locked out from the users. The users can upgrade the room at any time to escape power level struggles, although this may not maintain the DM status. -Unimportant users MUST NOT be used to name the room (cannot appear in heroes or in automatic naming -for rooms). -Tombstoned (previously upgraded) immutable DMs MUST be removed from the `m.direct_chats` account -data event and cannot be considered an immutable DM. Tombstoned immutable DMs are dead rooms on -the user's account, preserved for history retention. +#### Glare and duplicated rooms + +There is a potential for two users to start a DM with each other at the same time, preventing +either server from short-circuiting the `/createRoom` call. Servers MUST NOT prevent users +from accepting or rejecting invites to known conflicting DM rooms. Servers SHOULD NOT retract +the invite and SHOULD NOT automatically join the user to the room. The rationale is that the +user is likely to have already received a notification about a room invite and would be +confused if the room went missing or was automatically accepted. Clients are welcome to adjust +their UX as desired to do something else instead (like automatically reject/accept the invite +or hide it when it is an apparent duplicate). + +When the server joins a room it has identified as conflicting/duplicating an existing DM it +MUST apply the conflict resolution algorithm: +1. Order all the conflicting room's creation events by their `origin_server_ts`. +2. Pick the room with the **oldest** creation event as the canonical DM. +3. Send a tombstone state event to the non-canonical DM rooms pointing at the canonical room. + * If the tombstone cannot be sent by the server due to auth rules, it must consider the + room as "soft-tombstoned". A soft-tombstoned room is just a flag for preventing a room + from being counted as a DM and has no further behaviour or implication in this proposal. + +The rational for picking the oldest creation event is it is likely the room with the most +context for the user. It is possible that a malicious user intentionally creates a new room +with an ancient creation event however there's very little gain in doing so. The algorithm +is chosen such that any two servers come to the same conclusion on which room to use. -DMs may involve multiple important users, and are entirely valid in the eyes of this proposal. #### Upgrading DM rooms @@ -162,14 +254,12 @@ There may come a time where DM rooms need to be upgraded. When DM rooms are upgr `/upgrade` endpoint, servers MUST preserve the `content` of the previous room's creation event and otherwise apply the `immutable_direct_chat` preset over the new room. If conflicts arise between the previous room's state and the state applied by the preset, the preset takes preference over -the previous room's state. Servers can identify whether a room is a DM or not by peeking at the -user's account data: if the room is listed under the upgrading user's `m.direct_chats`, it is -considered a DM. The room does not have to be listed on all user's direct chats, just the user -who is performing the upgrade. +the previous room's state. Servers MUST create the new room by using the the room creation rules listed earlier in this proposal. This means inviting the other parties of the previous room to the new room automatically -and ensuring the room state is as described above. +and ensuring the room state is as described above. Auth errors with transferring state (such as +the room name) must be considered non-fatal to the upgrade process: just skip that state event. Note: normally room upgrades modify the power levels in the old room in an attempt to mute all participants. However, no one will have power to modify the power levels in the old room. @@ -177,101 +267,289 @@ Therefore, servers MUST NOT fail to upgrade a room due to being unable to update in the old room. This is considered acceptable by this proposal because upgraded DM rooms will lose their DM status, making them at worst just another room for the user. -#### Migrating existing DMs +Servers should do their best to follow tombstones to new DM rooms, however they should not assume +that those new rooms are valid DM rooms nor should they automatically join/invite the important +users to the room. + + +#### Detecting server support + +Servers which implement this MSC prior to it appearing in a spec release MUST advertise as such +through the unstable feature flag `m.immutable_dms` on `/versions`. Clients which detect server +support should not only check for the feature flag but also the presence of a supported spec version +on the server (as the flag may disappear once the feature lands in a release). Currently this +proposal is expected to land in r0.6.0 of the Client-Server API. + +Some servers have thousands and even millions of users which will need their DMs migrated. In +order to allow those servers to advertise the feature (both as an unstable flag and as a released +spec version) the client SHOULD assume that the server has not migrated its user's DMs until the +server sends the `m.direct_merged` account data event. + + +#### Migration + +Users already have DMs which need to be correctly mapped by the server. Servers MUST use the +following process for migration for each user: +1. Grab a list of room IDs from their `m.direct` account data (if present). Ignore the user IDs + that are associated with the room IDs. +2. Identify each room as either a DM or some other room and flag appropriately. This may include + conflict resolution. +3. Set `m.direct_merged` account data with `content` consisting of a single array, `rooms`, listing + all the room IDs the server iterated over, regardless of result. -Existing DMs can be automatically added to the `m.direct_chats` account data event provided they -meet the power level requirements. Most DMs at this point in Matrix will have been created using -the `trusted_private_chat` preset, giving both participants admin (power level 100) rights in the -room, therefore making them eligible for inclusion in the new direct chats account data. Clients -are encouraged to walk the user through the upgrade/migration process instead of automatically -migrating data. This is to prevent potential collisions between clients while the process happens. +The server SHOULD support automatically migrating any additions to `m.direct` even after migrating +the user. This is to allow for the user having an older client which does not yet support proper +DMs. Removals from `m.direct` are recommended to be left unhandled to ensure consistency with the +updated DMs. -Users are encouraged to upgrade their rooms to be immutable: this can be done in two ways. The -first way is altering the power levels and other state events to match the preset defined above. -The second is using the `/upgrade` endpoint with a new field: `preset`. This proposal encourages -clients to use the first method when migrating existing DMs for users, however the second option -is proposed as an addition to the protocol. -The `preset` field on `/upgrade` acts similar to `/createRoom` and the behaviour described above -regarding upgrading a room: the state implied by the `preset` takes precedence over existing state -in the old room. This applies regardless of the preset being used. When not provided, no default -preset is assumed (ie: all applicable state is transferred without any taking precedence over -another). +#### Sugar APIs (for appservices/thin bots) -An upgrade request body can now look like: +Appservice users and some bots are unlikely to see the room summary information. In an attempt to +combat this, some APIs are provided for users and thin clients to call and poke into the server's +view of their DM list. + +The expected use cases for not using `/sync` are: +* When appservices want to contact a user in a DM fashion (IRC bridge sending NickServ info). +* When thin bots aren't syncing and want to alert users (monitoring bots triggered by external event). +* Clients which don't trust their local cache of DMs. + +Appservices in particular are expected to heavily use the DM-finding API below and should cache +the result. When the appservice sees an event which the server might have reidentified the room +for it should invalidate its cache. Servers should make both of these endpoints highly cached. + +**`GET /_matrix/client/r0/user/:userId/dms`** + +Takes no parameters besides the obvious `:userId`. Example response: ```json { - "new_version": "5", - "preset": "immutable_direct_chat" + "direct_chats": { + "!a:example.org": { + "important": ["@alice:example.org"] + }, + "!b:example.org": { + "important": ["@bob:example.org"] + }, + "!c:example.org": { + "important": ["@alice:example.org", "@bob:example.org"] + } + } } ``` -An immutable DM room can be upgraded to a mutable room (ie: a `preset` of `public_chat` will -create a new public room, losing the DM status on the room and the users). +The `important` array does not include the user themselves. -After upgrading, the client MUST update the `m.direct_chats` account data event. Similar -behaviour should apply when the other parties join the upgraded room (for the client which -joined/accepted the invite). +**`GET /_matrix/client/r0/user/:userId/dm?involves=@alice:example.org&involves=@bob:example.org`** -The users of the previous room are the seed users for the new room, as described above in -upgrading a DM room. +Takes a `:userId` in the route and `?involves` (specified multiple times) to search for a DM +involving the given users. It should be considered a bad request if the user asks for a DM +involving themselves. -#### Handling conflicting rooms +Example response: +```json +{ + "room_id": "!c:example.org" +} +``` -The federation invite API allows for a user to receive an invite to an unverified DM room -with potentially conflicting participants with an existing DM room. Altering the invite API -to include signed membership and creation events is one possibility for ensuring that the -room is unique, however that becomes difficult and expansive to maintain (the target server -would likely end up trying to rebuild the room's state by requesting the gaps in the DAG). -Instead, it is proposed that clients check the power levels of the room to determine if the -room should be considered a direct chat. This renders the `is_direct` flag on invites only -useful for aesthetics for rendering invitations - the flag serves no other purpose if this -proposal were to be accepted. +If no DM exists involving those users, an empty object should be returned with 200 OK. -Clients MUST always check the flag when they join a new room, not just when they are accepting -an invite. If the room conflicts with an immutable DM the client already knows about, a tombstone -state event must be sent to the older room with a reference to the newly joined room. If the -tombstone event fails to send, the client MUST consider the room tombstoned anyways. -#### Server assistance +#### Third party invites -As mentioned previously in this proposal, servers SHOULD already be re-using rooms for which -the user has a direct chat with another set of users. Similar restrictions SHOULD be in place -to ensure that clients do not accidentally add duplicate direct chats to the account data event. +Third party invites (email invites) are hard to handle as the participants in the room are +unlikely to be unable to modify the power levels in the room because the immutable DM preset +forbids this by design. There's several solutions to this problem which are up for debate: -#### Abandoning/leaving rooms +*Note*: All solutions here assume that the third party invite gets flagged with an `m.direct` +flag that is transferred to the generated membership event. -When an important user leaves or gets kicked/banned from the room the room MUST NOT be counted -as a DM anymore by any clients. For example, a 3 person DM where 1 person leaves would cause -all 3 users to lose the DM status on the room. The room is not downgraded to a 2 person DM due -to potential conflicts with existing DMs. +1. The auth rules get altered such that when a user claims a third party invite they can + empower themselves one time only. They'd only be able to set themselves (no others) to + the `state_default` power level. The disadvantage here is that one could not have 3rd + party invites for unimportant users (assitants, strange bots, etc) without a mix of some + of the other solutions here. -## Tradeoffs / issues +2. The important users get tracked in an immutable state event (generated during room creation + when using the preset). This sacrifices reusability of the power levels for additional + tracking, and is not enforceable in pre-existing DMs (although neither are power level + enforcements so does it matter?). The state event would list the user IDs and third party + IDs (which can then be traced to membership events) which are "important". -This proposal has a number of downsides which may or may not be acceptable to readers. +3. Alter power levels to support a third party user listing, similar to the existing `users` + definition. Power in this map means nothing until the identifier is claimed, in which case + it behaves just like the user was in `users` with the power level. This also requires a + change to the auth rules to make it work correctly. -#### Clients do all the heavy lifting +4. Declare bankrupcy and don't support DMs with non-Matrix users. This option includes the + option of leaving it for another MSC to figure out, putting it out of scope here. -Instead of having clients track tombstones, room members, creation events, and account data they -could instead rely on the server to track all this for them. This could take the shape of the -client having read-only access to `m.direct_chats` account data and the server keeping it updated -for the user. +There are potentially other solutions as well, and the author welcomes them. As of writing, +the author is leaning towards option 4 due to the infrequent use of third party invites today. -#### This is reliant on trust -A user could remove a room from their direct chats without leaving the room, which the other -users in the room would be unaware of. This could potentially lead to confusion when that user -tries to initiate a new direct chat with the same users, however it is expected that "power users" -(people who are experienced with the protocol) will be the only ones doing this and likely for -good cause. +#### Complete list of deprecations +Earlier in this proposal it was mentioned that the existing direct chats module is replaced +by this proposal, however it didn't exactly explain the scope. The existing module is +optionally supported by servers (which requires very little effort on their part), however +this proposal recommends that servers drop support completely per the deprecations below. -## Alternative solutions +Deprecated things: +* `is_direct` on `/createRoom` now does nothing (ignored field, like any other unrecognized + field). +* `is_direct` on membership events now means nothing (replaced by `m.direct`). +* `m.direct` account data is deprecated (replaced with the bhevaiour described in this + proposal). + +After sufficient time has passed, the deprecated components should be removed from the +specification. + + +#### Test cases for the server/proposal to consider + +**Disclaimer**: This section might add clarity but might also be confusing. Sorry. + +**Simple case** + +The happy path of DMs. + +1. Alice starts a DM with Bob. Alice and Bob end up with a single room. +2. Bob then starts a DM with Alice and gets redirected to the existing room by their client + or server. +3. Alice tries to start a new DM with Bob. Like Bob in the last step, Alice is redirected + to the existing room. +4. Bob then starts a DM with Alice and Charlie. A new room is created. Similar redirection + semantics take place when the three parties attempt to duplicate the room. + +**Simple glare** + +The slightly less happy path of DMs: when servers collide. + +1. Alice and Bob reside on different homeservers. +2. Alice starts a DM with Bob, who does not accept nor reject it. +3. Bob starts a DM with Alice, who does not accept not reject it. +4. Alice accepts Bob's invite. Alice's server realizes this is a duplicate room and sends + a tombstone to the room, pointing at the DM Alice created earlier. + * During this, Bob's server would have originally flagged the room as a DM for Bob and + Alice, but the tombstone signals to Bob's homeserver that something is up and assumes + that Bob has no DM with Alice (yet). +5. Bob gets a tombstone in Alice's DM and realizes there's a pending invite. Bob accepts. +6. Bob's homeserver sees that the room is a DM and flags it as the DM for Bob and Alice. + +Alice and Bob's homeserver have come to the same conclusion. In the worst case, both servers +could accept the invites at the same time and conflict with each other on the tombstone. The +worst case for this is a split DAG on the tombstone with two tombstone state events, resolved +through state resolution. Both servers would have still come to the same conclusion on which +room to use so either of the tombstone events sent could be considered a no-op. + +**De-duplication during migration / tombstone tree** + +1. Alice has 3 existing DMs with Bob, who resides in both rooms. +2. The server updates (Alice and Bob happen to be on the same server in this example, however + the scenario works cross-server too). +3. The server realizes that Alice and Bob share multiple DMs and pick the oldest DM. The + server tombstones the other 2 rooms. +4. Alice and Bob now have exactly one DM with each other. -DMs could be enforced through auth rule changes and server enforcement, however this feels -far too complicated and over the top for the simplicity of just using a preset to fix rooms. -An example of how this could look is described [here](https://gist.github.com/turt2live/ed0247531d07c666b19dd95f7471eff4). +The room map looks a bit complicated now, but there's a tree of rooms where two point to +a single room. Clients aren't exactly expected to represent this nicely but can try if they +want to. There will be no `predecessor` for them to work off of. + +If the canonical DM room was then upgraded or otherwise tombstoned itself, the tree starts +to become more linear. + +**Tombstone tree with glare** + +Following the example of a "tombstone tree" then creating a conflicting room with glare +the servers should resolve to use the canonical room identified by the tombstone tree. The +conflicting room should be tombstoned to point at the canonical room. + +**Change in power levels, join rules, or membership** + +If the power levels or join rules change then the server needs to re-evaluate the room. If +the room ends up being no longer a DM, the server must flag it as such to the relevant clients +and *not* send a tombstone into the room. The parties are considered to not have a DM. + +If the membership changes such that someone ends up leaving the room (on their own or through +a kick/ban) then the DM is no longer considered a DM, similar to the paragraph above. + +If the room starts to look like a DM again (members rejoin, join rules become acceptable again, +etc) then the conflict resolution algorithm might take place depending on the user's DMs. + +**Malicious takeover of a DM** + +1. Alice and Bob are on different homeservers, but don't have to be for this scenario. +2. Alice starts a DM with Bob. Bob accepts. +3. Bob's homeserver sends a tombstone into the DM to point it at Matrix HQ. +4. Both Alice and Bob no longer have a viable DM: both homeservers do not consider Matrix HQ + as the replacement room, and do not treat the tombstoned room as a DM. + + +## Tradeoffs / issues + +This is really complicated for servers to implement. The alternative is that the client +does all the work (see below in 'Alternative solutions') however this is deemed unreasonable +by the author and early reviewers of the proposal. + +This allows for bots and personal assistants to be residents of a room. There is an argument +for restricting DMs to just important users (excluding any chance of unimportant users), +namely to ensure that security is not sacrificed through leaky bots/assitants. The author +believes there is a stronger argument for assistant-style bots in a DM, such as a reminder +bot or Giphy bot. The rationale being that although bots pose a security risk to the conversation, +the rooms are supposed to be encrypted therefore preventing past history from being exposed +to the bot. Both parties additionally should have the power to remove the bot if they don't +agree. + +This proposal enables encryption by default for DMs. Encryption in the specification is +moderately incomplete as of writing, making the move to enable encryption by default somewhat +questionable. The author is predicting much of the future here and is expecting that key +backup and cross-signing land in a released version of the spec at the same time or shortly +after this proposal ends up in a released specification. Additionally, most clients which +support encryption are targetting a stable release with better UX for around the same time +this feature would be released in a production build ("out of labs" as Riot calls it). + +This proposal is vague about which encryption algorithm to support. This is an intentional +choice by the author to support the possibility of a given encryption algorithm being +deemed unsuitable by Matrix in the future. For example, if a given algorithm was broken or +found to have security flaws it would be difficult to update the specification if the preset +had baked in a specific algorithm. Instead, the proposal asks the specification to make +a recommendation and for servers to think critically of the available algorithms (of which +only one is actually available at the moment) to make the best decision for their users. + + +## Alternative solutions +The sugar APIs are a bit awkward for the value they provide. Bridges are more commonly in +need of direct chats and thin bots arguably can still sync their way through life. Bridges, +and possibly clients, could receive changes to their DMs over a purpose-built room managed +by the server. Similar to server notices, the server could maintain state events in a readonly +immutable room for the user/client to sit in and receive updates. The author feels as though +this is more complicated than it should be, requiring servers to maintain hundreds, thousands, +or even millions of dedicated rooms for users which can easily overwhelm an appservice. For +instance, the Freenode IRC bridge has millions of users which would equate to millions of +rooms which all need to be synced down the appservice pipe which is already fairly congested. + +Servers could be less involved and all this work could be done by the client. Although +easier for servers, clients are more likely to run into conflicts between themselves. The +most common type of conflicts for clients to have is updating account data at the same time, +which happens a lot more often than it should. When there's conflicts in updating account +data, there's a strong possibility that data is lost due to overwrites. Sure, the client +could be told to use room tagging instead however a similar problem occurs when one client +wants to add a tag and another wants to remove it. An earlier draft of this proposal did +cover what a mostly client-side implementation of immutable DMs could look like: it can +be found [here](https://gist.github.com/turt2live/6a4e21437508bce76be89d6cbaf65723). + +Servers could be even more involved and require a whole new room version to change the +event auth rules for DMs. The author strongly believes that this is far too complicated +and way overkill for what Matrix needs/wants to achieve, despite having written an earlier +draft of this proposal calling for exactly that. Many of the problems with this approach +are described in that draft, found [here](https://gist.github.com/turt2live/ed0247531d07c666b19dd95f7471eff4). + +There is also an argument that the existing solution is fine and clients should just +make it work for their purposes. The author believes this is an invalid argument given +the introduction of this proposal and the problems highlighted regarding client-driven +DMs. ## Security considerations @@ -288,7 +566,8 @@ DM through a predefiend `preset` (ideally acquiring permission first). ## Conclusion Immutable DMs are a hard problem to solve and involve a bunch of tiny changes to multiple parts -of the specification. Of the options described, this proposal focuses on making immutable DMs -accessible to modern clients without needing complicated migration processes whereas prior drafts -focused on enforcing immutability as much as possible. This proposal is targeted towards being -a balance between complete enforcement and reasonable expectations of users. +of the specification. Of all the iterations of this proposal, this proposal is targeted at being +a safe balance of client complexity and user safety, sacrificing the server's complexity in favour +of predictable results. Most clients are interested in implementing DMs in some capacity, and this +proposal helps make it less painful and more reliable for them. Users should feel safer to use +DMs with this proposal given the enabled encryption and permissions models imposed on the room. From fd6e09b536955b708da8283b5dbeeddd340b7622 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 29 Jul 2019 18:17:57 -0600 Subject: [PATCH 05/39] Assign number --- proposals/{0000-immutable-dms.md => 2199-immutable-dms.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename proposals/{0000-immutable-dms.md => 2199-immutable-dms.md} (100%) diff --git a/proposals/0000-immutable-dms.md b/proposals/2199-immutable-dms.md similarity index 100% rename from proposals/0000-immutable-dms.md rename to proposals/2199-immutable-dms.md From 5fefd4604ecf8fe3ac8be27df262a1b73de3ddb6 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 29 Jul 2019 18:37:00 -0600 Subject: [PATCH 06/39] Clarify what the DM behaviour is early in the proposal It's described later, but no harm in saying it early when the reader is missing context. --- proposals/2199-immutable-dms.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index 97b27dca68f..f5f59aa81ee 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -84,6 +84,14 @@ at the `m.kind` and `m.heroes`: if the kind is `m.dm` and there's only 1 user in then the room is a 1:1 chat. If the kind is `m.dm` and there's 2+ users in the `m.heroes` then it is a multi-way/private group chat. +Implementations should also be wary of semantics regarding what causes a room to jump between +a multi-way DM and a 1:1 DM. Users can be invited (and joined) to either kind of room without +causing the room to jump to a different kind: for example, a 1:1 chat can have several unimportant +users in it (bots, etc) which do not make it a multi-way DM. Similarly, if an important user +were to leave a multi-way DM the room loses its DM status for all users and does not become +a 1:1 DM. Unimportant users can leave and join either kind of DM at any time without consequence +on the DM status or kind. This is all described in further detail later in this proposal. + With irrelevant fields not included, an example sync response would be: ```json { From fec96ca58f2a4e6a7d69246aad8b2bbd1dd42005 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 29 Jul 2019 18:37:18 -0600 Subject: [PATCH 07/39] Maybe these platforms don't support single DMs? --- proposals/2199-immutable-dms.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index f5f59aa81ee..4474da3f4ac 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -35,8 +35,8 @@ user, creating a new DM if needed. * Steam * WhatsApp * SMS -* Skype -* MSN Messenger +* Skype [for Business] (maybe?) +* MSN Messenger (maybe?) * and probably many others... Platforms which allow/encourage users to have multiple DMs with a specific person are: From d3e3cc3d8a194fd6b70a8f179e873125bb0aaacd Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 29 Jul 2019 18:37:29 -0600 Subject: [PATCH 08/39] Spelling --- proposals/2199-immutable-dms.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index 4474da3f4ac..9bc56f9c6ef 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -250,7 +250,7 @@ MUST apply the conflict resolution algorithm: room as "soft-tombstoned". A soft-tombstoned room is just a flag for preventing a room from being counted as a DM and has no further behaviour or implication in this proposal. -The rational for picking the oldest creation event is it is likely the room with the most +The rationale for picking the oldest creation event is it is likely the room with the most context for the user. It is possible that a malicious user intentionally creates a new room with an ancient creation event however there's very little gain in doing so. The algorithm is chosen such that any two servers come to the same conclusion on which room to use. @@ -407,7 +407,7 @@ Deprecated things: * `is_direct` on `/createRoom` now does nothing (ignored field, like any other unrecognized field). * `is_direct` on membership events now means nothing (replaced by `m.direct`). -* `m.direct` account data is deprecated (replaced with the bhevaiour described in this +* `m.direct` account data is deprecated (replaced with the behaviour described in this proposal). After sufficient time has passed, the deprecated components should be removed from the From 39f534dec10f39b02ac5f5ea1cfb56d3e9aa2ec6 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 29 Jul 2019 18:37:35 -0600 Subject: [PATCH 09/39] Backlink to soft-tombstones --- proposals/2199-immutable-dms.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index 9bc56f9c6ef..c6d505752d1 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -146,7 +146,7 @@ server must use these rules to determine if a room is a direct chat: * The room has at least 2 important users in it. * The user themselves is important in the room. * The room is not tombstoned. -* The room is not soft-tombstoned by the server. +* The room is not soft-tombstoned by the server (described later in this proposal). * No important users have left the room (through kick, ban, or other means). Assuming no other conflicts arise (duplicate chats, etc) the room is considered a DM between From ab7a963d717fd82576c7ae9cdd03ae5705877d39 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 29 Jul 2019 18:39:55 -0600 Subject: [PATCH 10/39] Clarify who the user ID is in sugar APIs --- proposals/2199-immutable-dms.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index c6d505752d1..55c22d67b2f 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -328,7 +328,8 @@ for it should invalidate its cache. Servers should make both of these endpoints **`GET /_matrix/client/r0/user/:userId/dms`** -Takes no parameters besides the obvious `:userId`. Example response: +The only parameter, `:userId`, is the user ID requesting their DMs. The auth provided must be valid +for this user. Example response: ```json { "direct_chats": { @@ -349,9 +350,9 @@ The `important` array does not include the user themselves. **`GET /_matrix/client/r0/user/:userId/dm?involves=@alice:example.org&involves=@bob:example.org`** -Takes a `:userId` in the route and `?involves` (specified multiple times) to search for a DM -involving the given users. It should be considered a bad request if the user asks for a DM -involving themselves. +The `:userId` is the user ID requesting their DMs - the auth provided must be valid for this user. +This additionall takes `?involves` (specified multiple times) to search for a DM involving the given +users. It should be considered a bad request if the user asks for a DM involving themselves. Example response: ```json From de8b3631bd9ade75c8480fec414bca7cc1b635dd Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 29 Jul 2019 18:47:37 -0600 Subject: [PATCH 11/39] More spelling --- proposals/2199-immutable-dms.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index 55c22d67b2f..8fb98341f44 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -225,7 +225,7 @@ important users. The rationale for preventing either party from manipulating the room is to ensure that there is equal representation of the room from both parties (automatically named, automatic avatar, etc). Users generally expect that a DM will be predictably dressed for searching and scanning, meaning -that the other parties cannot change the aesthetics of the room. For predicatble history and +that the other parties cannot change the aesthetics of the room. For predictable history and privacy, the room's history & join rules are locked out from the users. The users can upgrade the room at any time to escape power level struggles, although this may not maintain the DM status. @@ -376,7 +376,7 @@ flag that is transferred to the generated membership event. 1. The auth rules get altered such that when a user claims a third party invite they can empower themselves one time only. They'd only be able to set themselves (no others) to the `state_default` power level. The disadvantage here is that one could not have 3rd - party invites for unimportant users (assitants, strange bots, etc) without a mix of some + party invites for unimportant users (assistants, strange bots, etc) without a mix of some of the other solutions here. 2. The important users get tracked in an immutable state event (generated during room creation @@ -390,7 +390,7 @@ flag that is transferred to the generated membership event. it behaves just like the user was in `users` with the power level. This also requires a change to the auth rules to make it work correctly. -4. Declare bankrupcy and don't support DMs with non-Matrix users. This option includes the +4. Declare bankruptcy and don't support DMs with non-Matrix users. This option includes the option of leaving it for another MSC to figure out, putting it out of scope here. There are potentially other solutions as well, and the author welcomes them. As of writing, @@ -503,7 +503,7 @@ by the author and early reviewers of the proposal. This allows for bots and personal assistants to be residents of a room. There is an argument for restricting DMs to just important users (excluding any chance of unimportant users), -namely to ensure that security is not sacrificed through leaky bots/assitants. The author +namely to ensure that security is not sacrificed through leaky bots/assistants. The author believes there is a stronger argument for assistant-style bots in a DM, such as a reminder bot or Giphy bot. The rationale being that although bots pose a security risk to the conversation, the rooms are supposed to be encrypted therefore preventing past history from being exposed @@ -515,7 +515,7 @@ moderately incomplete as of writing, making the move to enable encryption by def questionable. The author is predicting much of the future here and is expecting that key backup and cross-signing land in a released version of the spec at the same time or shortly after this proposal ends up in a released specification. Additionally, most clients which -support encryption are targetting a stable release with better UX for around the same time +support encryption are targeting a stable release with better UX for around the same time this feature would be released in a production build ("out of labs" as Riot calls it). This proposal is vague about which encryption algorithm to support. This is an intentional @@ -569,7 +569,7 @@ in a floating dialog. To accomplish this in a safe and secure way it may be desi high levels of enforcement regarding immutable DMs. This may increase the user's comfort in knowing that none of the users in the room can manipulate it. Clients which need this level of confidence may wish to ignore insecure DMs and attempt to start new ones by upgrading the existing -DM through a predefiend `preset` (ideally acquiring permission first). +DM through a predefined `preset` (ideally acquiring permission first). ## Conclusion From 5ca0946499cf4e6389b23acca7adaf78546ed005 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 29 Jul 2019 18:47:45 -0600 Subject: [PATCH 12/39] Mention audit bots --- proposals/2199-immutable-dms.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index 8fb98341f44..0660bcef42f 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -504,7 +504,7 @@ by the author and early reviewers of the proposal. This allows for bots and personal assistants to be residents of a room. There is an argument for restricting DMs to just important users (excluding any chance of unimportant users), namely to ensure that security is not sacrificed through leaky bots/assistants. The author -believes there is a stronger argument for assistant-style bots in a DM, such as a reminder +believes there is a stronger argument for assistant-style/audit bots in a DM, such as a reminder bot or Giphy bot. The rationale being that although bots pose a security risk to the conversation, the rooms are supposed to be encrypted therefore preventing past history from being exposed to the bot. Both parties additionally should have the power to remove the bot if they don't From 82c8984ad18b98220945868caacfe006f76cb46b Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 31 Jul 2019 09:07:41 -0600 Subject: [PATCH 13/39] Fix words Co-Authored-By: J. Ryan Stinnett --- proposals/2199-immutable-dms.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index 0660bcef42f..7475d74db15 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -351,7 +351,7 @@ The `important` array does not include the user themselves. **`GET /_matrix/client/r0/user/:userId/dm?involves=@alice:example.org&involves=@bob:example.org`** The `:userId` is the user ID requesting their DMs - the auth provided must be valid for this user. -This additionall takes `?involves` (specified multiple times) to search for a DM involving the given +This additionally takes `?involves` (specified multiple times) to search for a DM involving the given users. It should be considered a bad request if the user asks for a DM involving themselves. Example response: @@ -367,7 +367,7 @@ If no DM exists involving those users, an empty object should be returned with 2 #### Third party invites Third party invites (email invites) are hard to handle as the participants in the room are -unlikely to be unable to modify the power levels in the room because the immutable DM preset +unlikely to be able to modify the power levels in the room because the immutable DM preset forbids this by design. There's several solutions to this problem which are up for debate: *Note*: All solutions here assume that the third party invite gets flagged with an `m.direct` From 9a7bf8bc58a83dad4c6bfb4d2ea0de843e495b88 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 1 Aug 2019 16:59:50 -0600 Subject: [PATCH 14/39] Describe third party invites --- proposals/2199-immutable-dms.md | 45 ++++++++++++--------------------- 1 file changed, 16 insertions(+), 29 deletions(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index 7475d74db15..2a3e011a5a1 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -203,14 +203,23 @@ room state: }, "users": { + }, + "third_party_users": { + } } ``` - The power level event is considered the `power_level_content_override` implicitly and - therefore applied at step 0 of the room creation process. + This power level event is not applied until after the invites have been sent as otherwise + it would be impossible to give third party users power. Servers should apply a default + power level event to the room and then apply the power level event described here after + the invites have been sent. Servers can optimize this process and use this power level + event at step 0 of the room creation process if there are no third party invites to send. * Users invited to the room get power level 50, including the creator. Important users (those invited and the creator) MUST have `"m.direct": true` in their membership event content. +* Third party users invited to the room get power level 50, as described by MSC2212 (see + later on in this proposal for how this works). These users are considered important, + and get `"m.direct": true` on the `m.room.third_party_invite` event contents. * Encryption is enabled by default using the most preferred megolm algorithm in the spec. Currently this would be `m.megolm.v1.aes-sha2`. @@ -368,34 +377,12 @@ If no DM exists involving those users, an empty object should be returned with 2 Third party invites (email invites) are hard to handle as the participants in the room are unlikely to be able to modify the power levels in the room because the immutable DM preset -forbids this by design. There's several solutions to this problem which are up for debate: - -*Note*: All solutions here assume that the third party invite gets flagged with an `m.direct` -flag that is transferred to the generated membership event. - -1. The auth rules get altered such that when a user claims a third party invite they can - empower themselves one time only. They'd only be able to set themselves (no others) to - the `state_default` power level. The disadvantage here is that one could not have 3rd - party invites for unimportant users (assistants, strange bots, etc) without a mix of some - of the other solutions here. - -2. The important users get tracked in an immutable state event (generated during room creation - when using the preset). This sacrifices reusability of the power levels for additional - tracking, and is not enforceable in pre-existing DMs (although neither are power level - enforcements so does it matter?). The state event would list the user IDs and third party - IDs (which can then be traced to membership events) which are "important". - -3. Alter power levels to support a third party user listing, similar to the existing `users` - definition. Power in this map means nothing until the identifier is claimed, in which case - it behaves just like the user was in `users` with the power level. This also requires a - change to the auth rules to make it work correctly. - -4. Declare bankruptcy and don't support DMs with non-Matrix users. This option includes the - option of leaving it for another MSC to figure out, putting it out of scope here. - -There are potentially other solutions as well, and the author welcomes them. As of writing, -the author is leaning towards option 4 due to the infrequent use of third party invites today. +forbids this by design. To remedy this, this proposal requires that third party users get +their own power level, as described in [MSC2212](https://github.com/matrix-org/matrix-doc/pull/2212). +In addition to MSC2212, this proposal requires that a `m.direct` field be added to the +`m.room.third_party_invite` event. When the invite is claimed, the `m.direct` field must +be copied to the generated membership event. #### Complete list of deprecations From 7ad3b05a1629bfeb142e475c88407ad43e9991be Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 6 Aug 2019 12:29:10 -0600 Subject: [PATCH 15/39] Counting is hard --- proposals/2199-immutable-dms.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index 2a3e011a5a1..bc58aa371f1 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -116,7 +116,7 @@ With irrelevant fields not included, an example sync response would be: "!c:example.org": { "summary": { "m.heroes": ["@alice:example.org", "@bob:example.org"], - "m.joined_member_count": 2, + "m.joined_member_count": 3, "m.invited_member_count": 0, "m.kind": "m.dm" } From 69ea33d551dd61bff69b555b88341817ed7be22b Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 6 Aug 2019 16:56:59 -0600 Subject: [PATCH 16/39] Encourage re-use of DMs --- proposals/2199-immutable-dms.md | 82 +++++++++++++++++++++++++++++---- 1 file changed, 74 insertions(+), 8 deletions(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index bc58aa371f1..5db07e4555f 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -141,13 +141,15 @@ Servers are expected to be able to identify direct chats for users and flag them in the room summary. How existing DMs are migrated is covered later in this proposal. The server must use these rules to determine if a room is a direct chat: * The join rules are `invite`. +* The rejoin rules are `invite` or `join` (as per [MSC2213](https://github.com/matrix-org/matrix-doc/pull/2213)). * The user's most recent membership event has `"m.direct": true` in the `content`. * The room has an explicit power level event (running a DM without a PL event is not supported). -* The room has at least 2 important users in it. +* The room has at least 2 possible important users (who may or may not be joined yet). * The user themselves is important in the room. * The room is not tombstoned. * The room is not soft-tombstoned by the server (described later in this proposal). -* No important users have left the room (through kick, ban, or other means). +* No important users have been banned. Kicking or leaving the room does not affect the DM-ness + of the room. Assuming no other conflicts arise (duplicate chats, etc) the room is considered a DM between the important users in the room. Important users are identified simply by having a power level @@ -167,6 +169,37 @@ invite event, clients should be skeptical but otherwise trusting of the room's D Servers SHOULD attempt to identify (or re-affirm) rooms as a DM whenever the relevant state events in the rules above change or are updated. +#### A note about leaving DMs + +Unless an important user is banned, the DM is still considered alive and should be able to +be rejoined even if some important people were to leave. However, there's a good chance that +when all important users leave that the room becomes unjoinable and therefore dead. To help +combat this, servers should approach rooms where important users have left with caution. + +**Case 1: The DM resides on a single homeserver**: when all important users leave the DM, +the DM should be considered soft-tombstoned for those users. This will cause a new DM to be +created. The presences of unimportant users can be enough for a homeserver to consider the +room not dead and therefore rejoinable. + +**Case 2: The DM resides on 2+ homeservers**: this gets a bit tricky. When the last important +user leaves, that homeserver might be inclined to consider it soft-tombstoned. However, this +is unlikely in practice: another server could rejoin thanks to unimportant users and invite +the last homeserver back. Both homeservers would converge on the room being the active DM for +those two users. + +**Case 3: A duplicate DM comes in for a room the server has left**: the server would normally +be inclined to automatically tombstone the new DM in reference of the existing DM (which the +server has left). It must not do this unless it is a resident of the room and can verify +that the room is alive as otherwise its perspective of the room may be antiquated. Instead, +the server must assume that the new DM means that the existing one is dead for one reason +or another. If another server were to tombstone the new room and point to the existing room, +the user could then try and join the existing room and the server's perspective of the state +would be refreshed. + +*Note*: Originally case 3 had a step for the server to autojoin the user into the existing +room to refresh its state, however this left a glaring hole for abuse to filter its way down +to a given user. + #### Creating DMs @@ -176,6 +209,9 @@ use `immutable_direct_chat` as defined here. The new preset has the following ef room state: * Join rules of `invite`. +* Rejoin rules of `join` (as per [MSC2213](https://github.com/matrix-org/matrix-doc/pull/2213)). + This is to allow DMs to be re-used (described later) without sacrificing security of the DM + by letting random people who were invited at some point into the room. * History visibility of `invited`. * Guest access of `can_join`. * Power levels with the following non-default structure: @@ -229,7 +265,10 @@ and `power_level_content_override` during creation. Servers MUST reject the requ `/createRoom` are expected to be included in the forbidden list where appropriate. Servers MUST return an existing room ID if the server already knows about a DM between the -important users. +important users. The important users which have left the DM MUST be explicitly re-invited. +If the user trying to create the room is not in the DM themselves, the server MUST try and +re-join the user to the room. If the re-join fails, the server should create a new room +for the DM and consider the existing one as soft-tombstoned. The rationale for preventing either party from manipulating the room is to ensure that there is equal representation of the room from both parties (automatically named, automatic avatar, etc). @@ -309,8 +348,17 @@ Users already have DMs which need to be correctly mapped by the server. Servers following process for migration for each user: 1. Grab a list of room IDs from their `m.direct` account data (if present). Ignore the user IDs that are associated with the room IDs. -2. Identify each room as either a DM or some other room and flag appropriately. This may include - conflict resolution. +2. Identify each room as either a DM or some other room, flagging them as appropriate, using the + following steps: + + * If the room does not have a `rejoin_rule`, consider the rejoin rule as `join` for the purposes + of identification. + * Identify the room using the rules specified earlier in this proposal. + * If the room did not have a `rejoin_rule`, attempt to set the rejoin rule to `join`. If that + fails, do not consider the room a DM. + + Identification of DMs may involve conflict resolution, which should only happen after the steps + above have been executed. 3. Set `m.direct_merged` account data with `content` consisting of a single array, `rooms`, listing all the room IDs the server iterated over, regardless of result. @@ -482,6 +530,12 @@ etc) then the conflict resolution algorithm might take place depending on the us as the replacement room, and do not treat the tombstoned room as a DM. +**The cases covered by leaving DMs above** + +This proposal covers a few cases that servers are supposed to consider when a user (or many +users) leaves a room. Please reference that section for more detail. + + ## Tradeoffs / issues This is really complicated for servers to implement. The alternative is that the client @@ -555,9 +609,21 @@ verification and cross-signing within the immutable DM (if such a concept existe in a floating dialog. To accomplish this in a safe and secure way it may be desirable to have high levels of enforcement regarding immutable DMs. This may increase the user's comfort in knowing that none of the users in the room can manipulate it. Clients which need this level of -confidence may wish to ignore insecure DMs and attempt to start new ones by upgrading the existing -DM through a predefined `preset` (ideally acquiring permission first). - +confidence may wish to ignore insecure DMs and attempt to start new ones by upgrading the +existing DM through a predefined `preset` (ideally acquiring permission first). + +There is a theoretical scenario where a homeserver or user could maliciously prevent a user +from opening a new DM with them. This is considered a feature given modules like ignoring users +exists, however a homeserver/user could continously set up a scenario where an existing DM +becomes unjoinable while sending tombstones for all new DM rooms which point to the unjoinable +room. This has a largely social impact on the room that technology cannot resolve (if people +are going to be mean, they're going to be mean). Attempts to alter the DM such that the user +cannot join without being notified are possible (changing the rejoin rules) however in those +cases both homeservers should be considering the room as no longer a DM, unless of course +the homeserver was being the malicious actor. Because of how tombstones currently work in +Matrix, users would have to perform an action to try and join the new DM and eventually the +user may get frustrated with the other user and ignore them, breaking the cycle of new DMs +being created. ## Conclusion From 369961cd40d9ebdf3a716f95ddc57e55e705e521 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 6 Aug 2019 16:57:15 -0600 Subject: [PATCH 17/39] Make some minor clarifications --- proposals/2199-immutable-dms.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index 5db07e4555f..713304dc012 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -166,7 +166,7 @@ room. Clients SHOULD use the room summary on invites and joined rooms to identif is a direct chat or not. Where the summary is not available and an `m.direct` field is on the invite event, clients should be skeptical but otherwise trusting of the room's DM status. -Servers SHOULD attempt to identify (or re-affirm) rooms as a DM whenever the relevant state +Servers MUST attempt to identify (or re-affirm) rooms as a DM whenever the relevant state events in the rules above change or are updated. #### A note about leaving DMs @@ -213,7 +213,7 @@ room state: This is to allow DMs to be re-used (described later) without sacrificing security of the DM by letting random people who were invited at some point into the room. * History visibility of `invited`. -* Guest access of `can_join`. +* Guest access of `can_join` so that guests can be included in DMs where needed. * Power levels with the following non-default structure: ```json { @@ -339,7 +339,8 @@ proposal is expected to land in r0.6.0 of the Client-Server API. Some servers have thousands and even millions of users which will need their DMs migrated. In order to allow those servers to advertise the feature (both as an unstable flag and as a released spec version) the client SHOULD assume that the server has not migrated its user's DMs until the -server sends the `m.direct_merged` account data event. +server sends the `m.direct_merged` account data event. More information about migration can be +found in the next section. #### Migration @@ -529,6 +530,8 @@ etc) then the conflict resolution algorithm might take place depending on the us 4. Both Alice and Bob no longer have a viable DM: both homeservers do not consider Matrix HQ as the replacement room, and do not treat the tombstoned room as a DM. +Ultimtely this is a wasted effort: both homeservers would start a new DM after the room was +directed at HQ. **The cases covered by leaving DMs above** From 21e0e858360a242c6c79d30a2c8b749931e91b63 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 6 Aug 2019 16:57:46 -0600 Subject: [PATCH 18/39] Add prose to allow servers to not migrate DMs for new users --- proposals/2199-immutable-dms.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index 713304dc012..ffd62e9717f 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -368,6 +368,12 @@ the user. This is to allow for the user having an older client which does not ye DMs. Removals from `m.direct` are recommended to be left unhandled to ensure consistency with the updated DMs. +Clients SHOULD NOT assume that the server will migrate direct chats if the user's `m.direct` +account data does not list any rooms. This is more important for new accounts created after the +DMs feature has been introduced in Matrix: if the server had to set `m.direct_merged` for every +user in the future, the server would be collecting largely useless data. Instead, the server is +given the option to skip migrations for users that have no data to migrate (such as new users). + #### Sugar APIs (for appservices/thin bots) From 340036758a0209c46fa460fedf0ef78d92ec1071 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 6 Aug 2019 17:05:29 -0600 Subject: [PATCH 19/39] Missed a thing on leaving users --- proposals/2199-immutable-dms.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index ffd62e9717f..a28606f0955 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -88,9 +88,8 @@ Implementations should also be wary of semantics regarding what causes a room to a multi-way DM and a 1:1 DM. Users can be invited (and joined) to either kind of room without causing the room to jump to a different kind: for example, a 1:1 chat can have several unimportant users in it (bots, etc) which do not make it a multi-way DM. Similarly, if an important user -were to leave a multi-way DM the room loses its DM status for all users and does not become -a 1:1 DM. Unimportant users can leave and join either kind of DM at any time without consequence -on the DM status or kind. This is all described in further detail later in this proposal. +were to leave a multi-way DM the room does not become a 1:1 DM. This is all described in further +detail later in this proposal. With irrelevant fields not included, an example sync response would be: ```json From 75f98a26fbaf803e7cd6ef01cc0b941b0f1f3dad Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 16 Aug 2019 19:13:44 -0600 Subject: [PATCH 20/39] Remove uncertain platforms from list --- proposals/2199-immutable-dms.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index a28606f0955..f7cfc62b93b 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -35,8 +35,6 @@ user, creating a new DM if needed. * Steam * WhatsApp * SMS -* Skype [for Business] (maybe?) -* MSN Messenger (maybe?) * and probably many others... Platforms which allow/encourage users to have multiple DMs with a specific person are: From fbf12ea45fe55c55161932b7e06f0a34b5727815 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 16 Aug 2019 19:15:03 -0600 Subject: [PATCH 21/39] Skeptical clients are skeptical --- proposals/2199-immutable-dms.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index f7cfc62b93b..edbdc4d3468 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -162,6 +162,9 @@ room. Servers MUST populate the room summary for invites if the server is a resi room. Clients SHOULD use the room summary on invites and joined rooms to identify if a room is a direct chat or not. Where the summary is not available and an `m.direct` field is on the invite event, clients should be skeptical but otherwise trusting of the room's DM status. +For example, a skeptical client might list the room with a warning stating that the room +is flagged as a DM without being appropriately decorated, but not prevent the user from +accepting the invite. Servers MUST attempt to identify (or re-affirm) rooms as a DM whenever the relevant state events in the rules above change or are updated. From 5e4e55b5e92b15876f95d6e266687cd67262ef1c Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 16 Aug 2019 19:26:36 -0600 Subject: [PATCH 22/39] Clarify soft tombstone --- proposals/2199-immutable-dms.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index edbdc4d3468..315d46cf792 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -169,6 +169,10 @@ accepting the invite. Servers MUST attempt to identify (or re-affirm) rooms as a DM whenever the relevant state events in the rules above change or are updated. +Soft tombstoned rooms are rooms which are considered tombstoned without an actual tombstone +state event being present. This is typically used to flag rooms where tombstone events cannot +be sent as tombstoned. This proposal goes into more detail on this a bit later. + #### A note about leaving DMs Unless an important user is banned, the DM is still considered alive and should be able to From f13a9fcba0c5f48627a7c0844c872c053947a10e Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 16 Aug 2019 19:26:44 -0600 Subject: [PATCH 23/39] Better wording for the server behaviour cases --- proposals/2199-immutable-dms.md | 39 +++++++++++++++++++-------------- 1 file changed, 23 insertions(+), 16 deletions(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index 315d46cf792..2433f5bde88 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -178,27 +178,34 @@ be sent as tombstoned. This proposal goes into more detail on this a bit later. Unless an important user is banned, the DM is still considered alive and should be able to be rejoined even if some important people were to leave. However, there's a good chance that when all important users leave that the room becomes unjoinable and therefore dead. To help -combat this, servers should approach rooms where important users have left with caution. +combat this, servers should approach rooms where important users have left with caution by +assuming the room cannot be brought back to life. + +The following cases are reinforcement reading for the conditions mentioned so far. They +describe how the server is expected to behave given a common scenario - not all scenarios +are covered here. **Case 1: The DM resides on a single homeserver**: when all important users leave the DM, -the DM should be considered soft-tombstoned for those users. This will cause a new DM to be -created. The presences of unimportant users can be enough for a homeserver to consider the +the DM is to be considered soft-tombstoned for those users. This will cause a new DM to be +created. The presence of unimportant users can be enough for a homeserver to consider the room not dead and therefore rejoinable. **Case 2: The DM resides on 2+ homeservers**: this gets a bit tricky. When the last important -user leaves, that homeserver might be inclined to consider it soft-tombstoned. However, this -is unlikely in practice: another server could rejoin thanks to unimportant users and invite -the last homeserver back. Both homeservers would converge on the room being the active DM for -those two users. - -**Case 3: A duplicate DM comes in for a room the server has left**: the server would normally -be inclined to automatically tombstone the new DM in reference of the existing DM (which the -server has left). It must not do this unless it is a resident of the room and can verify -that the room is alive as otherwise its perspective of the room may be antiquated. Instead, -the server must assume that the new DM means that the existing one is dead for one reason -or another. If another server were to tombstone the new room and point to the existing room, -the user could then try and join the existing room and the server's perspective of the state -would be refreshed. +user leaves, that homeserver would not have visibility on the room anymore but does not have +enough information for it to be tombstoned (soft or otherwise). Another server can still rejoin +the room due to unimportant users being left behind, or keep the room alive without the other +homeserver continuing participation. The homeservers involved will still converge on the room +when other conversations start. + +**Case 3: A duplicate DM comes in for a room the server has left**: when the server does not +have visibility on the members of a room anymore it cannot tombstone newly created rooms and +point to the room it can't see. If the server happens to be a resident of the room, it can +absolutely tombstone the new room in favour of the old room, even if the server's membership +is just unimportant users and the important users having left. When the server does encounter +an invite for a DM which duplicates a room it has left, it must assume that its perspective +of the older room is antiquated and that something happened to cause a new invite to come +in. After joining, if a server happened to tombstone the room to point to the older room +then the user could then try and join the room and refresh the server's perspective. *Note*: Originally case 3 had a step for the server to autojoin the user into the existing room to refresh its state, however this left a glaring hole for abuse to filter its way down From 1fce23abdb76ad221ddb905a1c2c272a0d3c772d Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 16 Aug 2019 19:30:06 -0600 Subject: [PATCH 24/39] Mention MSC1777 for updating left rooms --- proposals/2199-immutable-dms.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index 2433f5bde88..ef5a5d2876c 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -181,6 +181,12 @@ when all important users leave that the room becomes unjoinable and therefore de combat this, servers should approach rooms where important users have left with caution by assuming the room cannot be brought back to life. +If [MSC1777](https://github.com/matrix-org/matrix-doc/pull/1777) or similar were to land, +the server should abuse the capability to determine how alive the room is. By refreshing +its perspective, it can avoid scenarios where it has to make assumptions about the state +of the room it is no longer in. The server should never autojoin the user to the room, +regardless of MSC1777 support or not. + The following cases are reinforcement reading for the conditions mentioned so far. They describe how the server is expected to behave given a common scenario - not all scenarios are covered here. From 8a2b31695859c75874072d93e4b46ecc16edfa0e Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 16 Aug 2019 19:32:08 -0600 Subject: [PATCH 25/39] Support private chats --- proposals/2199-immutable-dms.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index ef5a5d2876c..2aabd9d75ce 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -220,9 +220,9 @@ to a given user. #### Creating DMs -The room creation presets `private_chat` and `trusted_private_chat` are deprecated and to be -removed in a future specification version. Clients should stop using those presets and instead -use `immutable_direct_chat` as defined here. The new preset has the following effects on the +The room creation preset `trusted_private_chat` is deprecated and to be removed in a future +specification version. Clients should stop using those presets and instead use the new preset +`immutable_direct_chat`, as defined here. The new preset has the following effects on the room state: * Join rules of `invite`. @@ -294,6 +294,9 @@ that the other parties cannot change the aesthetics of the room. For predictable privacy, the room's history & join rules are locked out from the users. The users can upgrade the room at any time to escape power level struggles, although this may not maintain the DM status. +*Note*: The `private_chat` preset is untouched as it does not affect DMs. `trusted_private_chat` +was intended for DMs despite its name and is deprecated as a result. + #### Glare and duplicated rooms From 21eb08d49886c80fd130cf935cd297d80484ef03 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 16 Aug 2019 19:36:27 -0600 Subject: [PATCH 26/39] Shuffle descriptions of power levels for members in preset --- proposals/2199-immutable-dms.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index 2aabd9d75ce..1c65ead4616 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -267,12 +267,13 @@ room state: power level event to the room and then apply the power level event described here after the invites have been sent. Servers can optimize this process and use this power level event at step 0 of the room creation process if there are no third party invites to send. -* Users invited to the room get power level 50, including the creator. Important users - (those invited and the creator) MUST have `"m.direct": true` in their membership event - content. + Users invited to the room get power level 50, including the creator. * Third party users invited to the room get power level 50, as described by MSC2212 (see - later on in this proposal for how this works). These users are considered important, - and get `"m.direct": true` on the `m.room.third_party_invite` event contents. + later on in this proposal for how this works). Like the invited users, all third party + users invited as a result of the `/createRoom` call are considered important. +* Important users (those invited and the creator) MUST have `"m.direct": true` in their + membership event content. Third party important users get the same `m.direct` flag on + their `m.room.third_party_invite` event contents. * Encryption is enabled by default using the most preferred megolm algorithm in the spec. Currently this would be `m.megolm.v1.aes-sha2`. From 90e35de52549e7973c09d1b939642c58c33f7cc3 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 16 Aug 2019 19:37:40 -0600 Subject: [PATCH 27/39] Spacing --- proposals/2199-immutable-dms.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index 1c65ead4616..3215aeb2e76 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -353,9 +353,9 @@ users to the room. Servers which implement this MSC prior to it appearing in a spec release MUST advertise as such through the unstable feature flag `m.immutable_dms` on `/versions`. Clients which detect server -support should not only check for the feature flag but also the presence of a supported spec version -on the server (as the flag may disappear once the feature lands in a release). Currently this -proposal is expected to land in r0.6.0 of the Client-Server API. +support should not only check for the feature flag but also the presence of a supported spec +version on the server (as the flag may disappear once the feature lands in a release). Currently +this proposal is expected to land in r0.6.0 of the Client-Server API. Some servers have thousands and even millions of users which will need their DMs migrated. In order to allow those servers to advertise the feature (both as an unstable flag and as a released From 498705f22f5d161a2ee4b0dfa92b20dbbcd94291 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 16 Aug 2019 19:39:15 -0600 Subject: [PATCH 28/39] Clarify that we're not altering previous room version rules --- proposals/2199-immutable-dms.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index 3215aeb2e76..ae8b57ffacd 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -373,11 +373,13 @@ following process for migration for each user: 2. Identify each room as either a DM or some other room, flagging them as appropriate, using the following steps: - * If the room does not have a `rejoin_rule`, consider the rejoin rule as `join` for the purposes - of identification. + * If the room does not have a `rejoin_rule`, consider the rejoin rule as `join` for the + purposes of identification. * Identify the room using the rules specified earlier in this proposal. * If the room did not have a `rejoin_rule`, attempt to set the rejoin rule to `join`. If that - fails, do not consider the room a DM. + fails, do not consider the room a DM. Note that this will not cause the rule to magically + work on prior room versions: it is just set to ensure that the DM flagging conditions + are met as they don't care for the behaviour, just the value being present. Identification of DMs may involve conflict resolution, which should only happen after the steps above have been executed. From 901f9c3e488c56e6ab28432512d458943982da29 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 16 Aug 2019 19:41:14 -0600 Subject: [PATCH 29/39] Clarify that history is supposed to be protected --- proposals/2199-immutable-dms.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index ae8b57ffacd..b4ec860ab61 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -579,9 +579,9 @@ for restricting DMs to just important users (excluding any chance of unimportant namely to ensure that security is not sacrificed through leaky bots/assistants. The author believes there is a stronger argument for assistant-style/audit bots in a DM, such as a reminder bot or Giphy bot. The rationale being that although bots pose a security risk to the conversation, -the rooms are supposed to be encrypted therefore preventing past history from being exposed -to the bot. Both parties additionally should have the power to remove the bot if they don't -agree. +the rooms are supposed to have a history visibility setting which prevents the history from +being exposed to the new parties. Important parties additionally should have the power to remove +the new user account from the room if they don't agree. This proposal enables encryption by default for DMs. Encryption in the specification is moderately incomplete as of writing, making the move to enable encryption by default somewhat From 3545a35a978ef5179f64cffdb4e5c60d40b09f61 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 16 Aug 2019 19:48:46 -0600 Subject: [PATCH 30/39] More words about why enabling encryption is safe enough --- proposals/2199-immutable-dms.md | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index b4ec860ab61..29a3591a864 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -585,11 +585,20 @@ the new user account from the room if they don't agree. This proposal enables encryption by default for DMs. Encryption in the specification is moderately incomplete as of writing, making the move to enable encryption by default somewhat -questionable. The author is predicting much of the future here and is expecting that key -backup and cross-signing land in a released version of the spec at the same time or shortly -after this proposal ends up in a released specification. Additionally, most clients which -support encryption are targeting a stable release with better UX for around the same time -this feature would be released in a production build ("out of labs" as Riot calls it). +questionable. For instance, a large number of bots and bridges in the ecosystem do not currently +support encryption and generally fall silent in encrypted rooms. Bots in today's ecosystem +can use [Pantalaimon](https://github.com/matrix-org/pantalaimon) or add dedicated support +if they so choose, however bridges are still unable to decrypt messages even with Pantalaimon. +Bridges being able to support encryption (they'll need EDUs and a device ID) is a problem +for a different MSC, and likely a different spec version. In the meantime, this proposal +specifically does not prohibit DMs which are unencrypted and bridges can start DMs as such, +or refuse to participate in DMs which are encrypted. Generally speaking, bridges in the +current ecosystem start the DMs with users instead of the users contacting the bridge, +however there are some prominent examples of this being reversed (IRC, most puppet bridges, +etc). Clients can combat this by identifying bridge namespaces (also a future spec problem) +and offering users the option to not encrypt the room, or by just having that checkbox +always present. Ideally, this proposal and cross-signing, better encryption for bridges, +etc all land concurrently in the next specification version. This proposal is vague about which encryption algorithm to support. This is an intentional choice by the author to support the possibility of a given encryption algorithm being From dd292bd042350e7479a4e8501d496089d569c45f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 27 Aug 2019 17:42:50 -0600 Subject: [PATCH 31/39] Remove confusing banquets section --- proposals/2199-immutable-dms.md | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index 29a3591a864..88dc3906b48 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -6,17 +6,11 @@ adequately support this as it allows for multiple rooms to represent a DM betwee Although supporting multiple chats with individuals is potentially useful for some use cases, clients are increasingly wanting to be able to support exactly one DM for a user. -The Fractal team has previously published a blog post for the Banquets versus Barbecues use -cases (available [here](https://blogs.gnome.org/tbernard/2018/05/16/banquets-and-barbecues/)). -In short, Banquets are public rooms and Barbecues are DMs and small private chats (family, -friends, colleagues, etc). This proposal aims to address the lacking Barbecue support in -Matrix. - -Any messaging app is likely to have a similar desire as Fractal to differentiate DMs from -all the noise, and Riot in particular is looking at using the DM between users for a more -personal experience within the app (such as key verification between users). This proposal -focuses on immutable DMs between users to assist in identifying the "One True DM" between -users and prevent the room's scope from increasing. +Any messaging app is likely to want to differentiate DMs from all the noise, and Riot in +particular is looking at using the DM between users for a more personal experience within +the app (such as key verification between users). This proposal focuses on immutable DMs +between users to assist in identifying the "One True DM" between users and prevent the +room's scope from increasing. For comparison to Matrix, the following chat apps/platforms all support and encourage a single DM between users. They all redirect the user to an existing chat with the target @@ -38,6 +32,7 @@ user, creating a new DM if needed. * and probably many others... Platforms which allow/encourage users to have multiple DMs with a specific person are: + * Matrix (currently) * Email * XMPP (technically speaking) From 0cbd1dce42b3311db1ef178589f721d90c4f1999 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 27 Aug 2019 17:43:01 -0600 Subject: [PATCH 32/39] Inline the example image --- proposals/2199-immutable-dms.md | 2 +- proposals/images/2199-dms-client.png | Bin 0 -> 7040 bytes 2 files changed, 1 insertion(+), 1 deletion(-) create mode 100644 proposals/images/2199-dms-client.png diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index 88dc3906b48..e2440718fdb 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -126,7 +126,7 @@ With irrelevant fields not included, an example sync response would be: Clients rendering the above should end up with something like the following: -![alice-bob-over-banquet](https://i.imgur.com/h4BHZSw.png) +![alice-bob-over-banquet](./images/2199-dms-client.png) Servers are expected to be able to identify direct chats for users and flag them appropriately diff --git a/proposals/images/2199-dms-client.png b/proposals/images/2199-dms-client.png new file mode 100644 index 0000000000000000000000000000000000000000..49eda49844a68746588b8b885cb31785f2e1c9ec GIT binary patch literal 7040 zcmbVRcTiJbn~ox&R1s8401*VFcL=>nQ=0S+g3=KP0qIB!NK=p=2$3o!lpsw&igXCQ zH#HCly-R1K-#4?fJG-;L*?;c6=bZPPxp&TUp67ku8?B?IN=`yg0ssKW)zy@r0|3{k z@y|;)uj8*G+(xJP+cofWRYd?24aDIaH|!NO6##(B1k!UGLI8k)OI=w(|BdBlmX(g# zbYPb-ia+bf5<1mjhsZTCtp$Q~gbXQ*&m; zon;OR{Lc2@8Xh0NaNh!*?w@s{22ueb=5IlOZU7kY{u&nWfS~`HJW(1!2$=vefXF8f>k*B-@B z3{ENB4Y#O5unm|b981SHybU*|=#AgXw7QjTtT-Od=(FsahLYB?z2!woS4Idfv%R}qbk6=nEephG;bCk%$uXU&Gxs%kyg(XHr3Wl`fG#V*Bz4~qJ~&waS9E(J_>ujN|l=t%E);wODtQ;Bo6Ll zW?!!JrdoE%m5hN@b0?;bOC+}x8S68$c6sy&Vg8b@0q$xc@X;$@qd&4#lgQyv&1+@> zSm~iNVjS@Y9x#RuN=hZAlv>{MXt6VO-0r#6d$;jM`~ zGhs6stYQ=z^tpH0qDN*MXM!|2`(952L;1d20YU0l-2|=NZS17SPh31Ob|%>u^BM7F zpgJ>dZnW2&*gHgy1S`!QeUm8};Rr!!TDq0fE!7A`XQ7OweX_S%AX6)49=Tda;71{U zpGf({WkhbTU50V?c&S%%`BQjuP0*qWtc?9Dy8yRE&<4)L5pX1wGOs0B?=5lT4|tXx zy3~-YK*BTv4LF%hg7WKJ09f8czMVEvc|F+Sts$6kmiGUP;D14!W(2laE`7OlIlxSw zedea!E|mrSi{?B{#g~A08Et7wFV4^Ir_z{h{VYwNpH;*0f}`D#s~yuz7+@@&K4 zS#P1L@~Y2)Dy4vy$+$dF_L(>#Ia$E-eJ}fUNBG|F!HzP?v8W-Jl-P)KY0@ZTiWUZ? z5!i2{!CUmJb4=-4ZeC=lh>j(v_04LM$3;|#X`Z(r{JS{DA3c8Sb-BN5=&3~aQ$)mt z&4}5GipxQVRS|`IrrhpUn`+Oz(-4-+LsdQut1tR0G?uHc6`NhU|5)_*IHbcjF(KL^ zGx0{N$^5-mO-4>&BK9W5>cPUhriFUZqv(cSB57@UzX6Cu@bKoz*0wC8pEPf^%d4sm zW*qz{%H6rrG?l+4k&eFK#>7K$z7+cm{H2h~E5ZJT3B3ztZ9?O-ITDZs;-)s8(0x*) zdMOjFCjb(Kw*x1;b1I(BOI?2I&g-u{J&*<^IXmJ_Z?(65g7MoKG5r>K;<7l4JMxV` zsw5-@t4MO0yIw^0E@}5sX+u)QeqR*T&jfoupCysERLj`U$_{xzkh^X=D3+;5jXBH> zv+(=!t#MZ!6UGG56Y>pY39>~SG6mIQ!385vrZYzGYdl@(6t2zpUV5``+erfG*6XGV z-5!uuchhDDI-DS)8S+MtV`ceHzYc26nWR5&QfDs|PYIG4=cq0hQhISrP?>zUnZ9Re zg)AZq>Fx&yi#YLI%du9=uodN=pJ{mMVXLxZR<{e!GrU^!Y3}|NwWcWqs*?2gc%|cD z6_G!ANZbh@-Rpg07aS>T4;K1S;DKiI;SojSNI@l1V`EXyq!*jRLA;b=fD;zS_zo#V zJ0?af>Dso}eBEKjpkATILc*6$BYU{(x3n4Ky~XvQ)ANX-=14J+;m`HgyS`4ZD-|!Y zbVlb)d318oWd5`dY_j# zTYy)H{W*QPMtnM(6$aq!jm75F7~=aPTWr$5l+H?IE}<=bEo!>+=>@ky`bc!YtI zeQAKj^CG1cZXvV5gwyE~$KKQomaVCJP)*sA{fHkN1#6x@+GJRmm{u6S3!bygZj?OB zACdt0rvwbJ6s`2^)oi|&+rQe{En0gA+Cs{b$$eBulDC*40NC3i*l9>x2-i7Q_k6S* znVr7%bS0RhVYYxc{MEH76ip|PaIXt|h4l|xsn(_)x?c7iaby5%{Z1g2?NagE6~IYC z;z$8l{zuXnshF|vzvM#jQpu*DMesd}{Nr6bTabfhgUC$=J$lgn zMHY~V$5s+*pci;lQa^#U`+y?HhRD48J7=Un+R|lVO5KgDsU(@U`=LF9I17E5Y)df1 zX-j%iW%5`-ezvt9q4u^-cUKBR77(F40&IOt(+?D^VV8YBIIY6|2B<2T3^#(uM8cBo zA4)STNUwtRxt9kEKbvb_dtt^L()Rof*f8}qZQ_cTNU7^1D{Tg0Z7P!$7h87DbU6uA z`kK1QWztxOWu6|I@>fPy+}Cmh@b8<28t{aT`j)LggpM90gG6d!uIF2Q0&e(IU92cv zpSu1CKoedb9GQc{WO$*)Dk)96D+hfkC^@}>7;M_5dW7xAQ@SxM4Uw3u6{ zP2NDKy;KlcijR7@#v^FgEwn>BPWc$=-Z=3eG@_HCzES*W-sly8+boZI`b>tprgh%!18Cqg!iyIwP7&V9G4H`(C$MsBK+2BL^k&S9ppI4P|_LvGdjl^0=g0B z`xgfhf~s%N70_Zj>@*`lKrit>%CaIv>>uu)Pli1;*hI-W2kit7(khTvYeZD57m>;9 z>d}JLVy{jIU@u}A^{-uqB;wI-&TMgUc3OTzn+M^9*JQ?!7i?DGw#t|Pw=%WqeYJXb zJ?G|o@s)ONx`G=_;mjOK3GmBjJd=p|2%>A857`C~6siKP%9EYKN0{s7a@!iScJhA3 zuBU5`P`S<&4%RcS%$Y^4#2RDQW!w=6{L_5Qf^JPP#g2XQ)OzrNBRaj6Bb10}OLXw+ z=khuA^0wSNGss^Rq9Z+FY-U01#IatUO4$lMH=tV+`_E~E?Wj;9$yU+iqlgV{(i<#;Oa8vI(?tE};eW^;Q&G)vYmTac!?f;@*7iwPllZ?PDOhf~y z^J~`k)gDRsbFh^~>R#>!^>7VW>dx#a(Vbl0W}?D9%-Gh@%UR|^t#LSAaT%5v<5a}F zRx4Z>$mz116K$NMBGbrqHS`^M*t(#CKL*#bu*v>X!L+!hLS)so!9I6XJ;e&M7J0MJ z7HZzJMziLa^+H%WuFJG^VwrR&##mhS3NABhzWJGC{%OT>e(RC@S$E%e-ju!viPm5H zJK7Ak&=Rx7!i<|m30mwlU1)L88qx6<4@)SDPqMQY-W3+q^O`W>BuV;F=( z7u#gcUOFAZXZ^|t6G|5B(9>e~=e4mr8@k+>4;OT&K#aU1!^%MfY@9JH0H%~1b9v(=8Ef5KA5xKiYr@q0i3EeO%H-hTL3bW`EHv(JbE7O&peLH~#@ z1(|stpM9q6e^=QIo4wvi{{AwMdC>3AW3efbVvH+JBwmmq38;}Z*m+mBJxszim zXZSkJ4CT*pP$g@0Z%eoF5zoqm&=|2lgE}}W7T(5Fw?6+_^B|M4I)3g9XJEgyB(dtBWx9j4k<}29r>ploZ zlgXa=zA-!tzs5Vnb=S$NgKDpu?hoxL|1;W)eEm#j4)3K&PXBl3`QNRnyh%kx^+MqO zLV7%lRRuJM&@^8WVS3a8)H!|6_D%pd#E$Sx!MA=nT*NZ!D@;9N&}fIfam~kLWNx1# zA`53CDW5(ZxX;~6{j?~f0)uQM2d5Aho6ysdZ#Op-r5JkiO+fT}boFo^4TG>)UYwL0 zpR$(drr!R~05k)ULQ4{)jQrdjn@$s71r+*Q|M7_r)|Cc;zu8yA}5<+aOW3}ns41^|6rikkv<@f1=o~T7_%khQh*?0P6H{G*jK?hOW zcRi96{Zd3GV4K?!q(#^3}<$U*JUkowM5!NKm`px^=`A?#$6+9Ih#A(gW0^}%Q=+-UN(3@Q zn~^nNXeRVdmjj{trvtI_8Y}AFJu@+A&f6~jcvxPH(Cx;DE>$gFwPN5z6@Irz7+*^t zXK7c+T;H5TV7i|`5=fGIB{QwW^ty#k0}NN^N~5IUp~A_A_j`D6ZA&32{B7(`JW0ev zTEFj!OP(!rS+;%2N&jumTZ=1^4-YLGmp_82R~Ud>H7N*nF&05;Ip6Q=?h$9nP@+cz z3t24h?b5AQPiOhpWrE6c*^9kI3+R^CbYeD3s}}<`GLz%S4gESmA?e7A-ZPMM!a);P z)7^1bDlduU*6i$LB5Quz>CL<(XQ>4t!E4tj?6)egi+1c*Npio~+0 zS6@Q!IfzbH#!>yFgyT=$M;R<;**N2S0hPPlO;Eq_DK?De0hJoD!fo;-kbXj*uFsPP zy|QY=QohfyF58WUzvUm?mR2LlWu<2a4=p~pt@xfsMOEY(wg^IXGn^$h-T`m0KMxeb zW$O+xvt^6xu5SwWZ~r@sDGQgUF{5Q1=8OfgMY2vHrd|*mgvdp`Aj7XlD6lfpc3k^V zG?XJnN3^(fD^i|D9Ut302^&QT{>^8`vZ$h7+{DNFrYkWlQ^d>*IXrdV8^np1a+yHYOCP3q8}y$C50{HSB}jxXCwT#8<0 zEcx74%1{qS9{bt3gW4>xO9uTT5K?xteKjgF57fmpcSq#MO|1J1>XM$gcpD{k5+`L> zwAiZYF7e7(axIe=;TTzQ-}LwP{h%(7XVA2^01Dip>!x1nZ?`{RBC?`di?{FYrTuhF zR@H(~dfiMEd77S+?iSVoen%kbmDa(F{Z1`U+f|FQ?*P$HV)hJku*-#Z9$ZG3N6sUO zBlG#hC_>qzeY+f{l`bR9dTHz;BRLr{gKz!hU7!^P;#em-J^j3u+X%S6DYaO%AkEfb zo?J%INYtU|pNmq@Ixt)P_Tv|h5(ec{#(4`4`@i2t>WqHN*VlW~m zE-|Y%{>H5DbI=^TQJZvbTw6%&MDh4B+ZrYOTRBX8Ip9cirx)2NAK;28GyuhI7oy8N z*$@fG%^NzCzwEVl35u=+Ao`^gjFBfmCX$Nqr;klPZ3tv{4LbAdrS4@sk1FqApv2?j zPdy^f8XHw9x~ovGqp4bE%U~DZwL5IEJZ`c68Zz=23--ih-4ES;Da%|bJoF>*YJ-Qj zlKV?lrDN>3f^{D!^Mz9hv$>7e9(AU<=gCvq&Apa#%c0q}Scj|qYSKV(-FSNOH!<&? z!;+FDm*JX0gs0!!j{^!blZ0W+1X&d|A`|7wal29YB-BXl{3z#?X9@*oS`X~bo<}d4 ze8c1}aFnVN0K$^sl_n{_yj>R=j{BXvM|{wLirusrGg9muBQJRa{#psv!UaCE$H5 zNk37OSYAl!eM{6}4C@kRp%h8oBpv#E~08&6K9k4h8*AQx^>|m zeQ9*fgxpxR%?;-1F z38{`0<9_1c<3vo&A$&5D&d{%+o# zqk{hqrEi8GhpY&KY7G(Cr04wRKQ%NB3b@C;I@ZTa2W$?KO?>3!x+3}pvIVqs&k64#L&iwikM#AChQgV`TD6j;u<>%75ouhjB_z`mgE2h-wU>E;Auq z3rAo)_W9rDeRhBR;+k*$vM{l>YA4R^%Cu5vdNy_QtrfNXiF|;+*s(5FmZN@R)I|nu zvpJgIsY(iNf_^5(EN6z5p_Nj{sn(A6-0imon9YaPCVc~bM1Lw_S$*4&0S?;8M&vM4 zezi0k%Y=xXA7}LUyn+eO9&mH%V+Be(L0+;6A94IY-cDu&PRhaz$FK zMb*bkZGU**r|9v&AxW+dc3HTkc}1q+>Fiv2vK3vO5zDi9OLS0Zwl z+r?nPoo2xo=&%zoRYvwYHc6~Wx1F%^-nTsTY(T)_sYy(|DE(d#Z7eTm{w&HUUfd8^ zRE|^E+}-_Q6lqh?CnBB?Oz;LQgxM2kP(0SNJ2j_^QBLiHMElTwO)WtgIO$N9&bQO9 zOsF8!ZanQcRU7SEo6A@~4*8URF_+;jJKds`3C#odsTyKg5lS*+`Ptmw`i82`ve0vn zb?((Cc5W4@gJ!=O24~6f_KG0h1?#^Q4{ci;6$kCylM7_^(ma9I6Hh5o$+%_ED|AN; zQ<38{_^#uE4GvkyuG-`1!8>C1djU@^t{O*n_otoOEtnO>@FI(^wCUz83R!Qzxp!cz zkM%FCrH>Sk!bKJ$t$h3LelYqYoP9u&02I0I9)9;(t-zc`1o*O9T=7pS*QigVe+3u% z@06Y}Ad_4-79W-?faO%U1p2$L<3-33F7PL{ZsTFkp^B2B$BR6^Fm_!x5g*rXM-jw$ z|LyIEC01P>{8g;C73~gVHoC8e%G0vW@;taPu7rJj<6&*T9v}o?asFrjo4T_GQm-h{ YhXHyvwM?4$zi0sUCtAu#MeET20Oq-s<^TWy literal 0 HcmV?d00001 From d504efa01a4b4170393cbba9cf258646ff392d59 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 27 Aug 2019 17:43:26 -0600 Subject: [PATCH 33/39] Use "dm" as the keyword --- proposals/2199-immutable-dms.md | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index e2440718fdb..54d7be75876 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -134,7 +134,7 @@ in the room summary. How existing DMs are migrated is covered later in this prop server must use these rules to determine if a room is a direct chat: * The join rules are `invite`. * The rejoin rules are `invite` or `join` (as per [MSC2213](https://github.com/matrix-org/matrix-doc/pull/2213)). -* The user's most recent membership event has `"m.direct": true` in the `content`. +* The user's most recent membership event has `"m.dm": true` in the `content`. * The room has an explicit power level event (running a DM without a PL event is not supported). * The room has at least 2 possible important users (who may or may not be joined yet). * The user themselves is important in the room. @@ -147,7 +147,7 @@ Assuming no other conflicts arise (duplicate chats, etc) the room is considered the important users in the room. Important users are identified simply by having a power level greater than or equal to the `state_default` power level requirement. -The server MUST maintain the `m.direct` flag when updating or transitioning any user's membership +The server MUST maintain the `m.dm` flag when updating or transitioning any user's membership event. The server SHOULD refuse to let the user manipulate the flag directly through `PUT /state` and similar APIs (such as inviting users to a room). Servers MUST consider a value of `false` the same as the field not being present. @@ -155,7 +155,7 @@ the same as the field not being present. Servers are not expected to identify a room as a direct chat until the server resides in the room. Servers MUST populate the room summary for invites if the server is a resident of the room. Clients SHOULD use the room summary on invites and joined rooms to identify if a room -is a direct chat or not. Where the summary is not available and an `m.direct` field is on the +is a direct chat or not. Where the summary is not available and an `m.dm` field is on the invite event, clients should be skeptical but otherwise trusting of the room's DM status. For example, a skeptical client might list the room with a warning stating that the room is flagged as a DM without being appropriately decorated, but not prevent the user from @@ -217,7 +217,7 @@ to a given user. The room creation preset `trusted_private_chat` is deprecated and to be removed in a future specification version. Clients should stop using those presets and instead use the new preset -`immutable_direct_chat`, as defined here. The new preset has the following effects on the +`immutable_dm`, as defined here. The new preset has the following effects on the room state: * Join rules of `invite`. @@ -266,8 +266,8 @@ room state: * Third party users invited to the room get power level 50, as described by MSC2212 (see later on in this proposal for how this works). Like the invited users, all third party users invited as a result of the `/createRoom` call are considered important. -* Important users (those invited and the creator) MUST have `"m.direct": true` in their - membership event content. Third party important users get the same `m.direct` flag on +* Important users (those invited and the creator) MUST have `"m.dm": true` in their + membership event content. Third party important users get the same `m.dm` flag on their `m.room.third_party_invite` event contents. * Encryption is enabled by default using the most preferred megolm algorithm in the spec. Currently this would be `m.megolm.v1.aes-sha2`. @@ -324,7 +324,7 @@ is chosen such that any two servers come to the same conclusion on which room to There may come a time where DM rooms need to be upgraded. When DM rooms are upgraded through the `/upgrade` endpoint, servers MUST preserve the `content` of the previous room's creation event and -otherwise apply the `immutable_direct_chat` preset over the new room. If conflicts arise between +otherwise apply the `immutable_dm` preset over the new room. If conflicts arise between the previous room's state and the state applied by the preset, the preset takes preference over the previous room's state. @@ -414,7 +414,7 @@ The only parameter, `:userId`, is the user ID requesting their DMs. The auth pro for this user. Example response: ```json { - "direct_chats": { + "dms": { "!a:example.org": { "important": ["@alice:example.org"] }, @@ -453,8 +453,8 @@ unlikely to be able to modify the power levels in the room because the immutable forbids this by design. To remedy this, this proposal requires that third party users get their own power level, as described in [MSC2212](https://github.com/matrix-org/matrix-doc/pull/2212). -In addition to MSC2212, this proposal requires that a `m.direct` field be added to the -`m.room.third_party_invite` event. When the invite is claimed, the `m.direct` field must +In addition to MSC2212, this proposal requires that a `m.dm` field be added to the +`m.room.third_party_invite` event. When the invite is claimed, the `m.dm` field must be copied to the generated membership event. #### Complete list of deprecations @@ -467,7 +467,7 @@ this proposal recommends that servers drop support completely per the deprecatio Deprecated things: * `is_direct` on `/createRoom` now does nothing (ignored field, like any other unrecognized field). -* `is_direct` on membership events now means nothing (replaced by `m.direct`). +* `is_direct` on membership events now means nothing (replaced by `m.dm`). * `m.direct` account data is deprecated (replaced with the behaviour described in this proposal). @@ -637,6 +637,7 @@ make it work for their purposes. The author believes this is an invalid argument the introduction of this proposal and the problems highlighted regarding client-driven DMs. + ## Security considerations Some clients are investigating the usage of immutable DMs for security and privacy related From 79ba50969cca19cbdaa8ac7d28445a33ca512507 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 27 Aug 2019 17:43:35 -0600 Subject: [PATCH 34/39] Explain pitfalls of the current system --- proposals/2199-immutable-dms.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-immutable-dms.md index 54d7be75876..5382b89605b 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-immutable-dms.md @@ -662,6 +662,28 @@ Matrix, users would have to perform an action to try and join the new DM and eve user may get frustrated with the other user and ignore them, breaking the cycle of new DMs being created. + +## Pitfalls with the current system + +The existing DM system in Matrix support multiple DMs with a user, however that system is +not necsarily more complex than this proposal (in fact, it's probably simpler than what +is proposed here). The existing system has the disadvantage of being non-deterministic where +two users in the same room cna have a different view of that room in their clients: Alice +can see it as a DM whereas Bob sees it as just another room. Having non-deterministic DMs +can lead to conversational confusion ("hey Bob, let's go to our DM" and Bob asking where that +is) or uncertainty about where to send sensitive information. + +Using the example of clients wanting to use DMs as a channel for exchanging key information, +knowing that all parties in the conversation perceive it as a DM is important. If the +verification were to land in a random room, the other party may miss it or be concerned it is +not a real verification request. + +Further, the existing system is managed completely client-side and every client is expected +to clone the same logic. By pushing the DM calculations to the server, all the client has to +do is render them. Most messenger Matrix clients want or have a concept of DMs, and instead of +duplicating the same code over and over the server could just inform the client of its DMs. + + ## Conclusion Immutable DMs are a hard problem to solve and involve a bunch of tiny changes to multiple parts From 55f1c0778cb565a30b3442b3972cd408389ab8f7 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 28 Aug 2019 11:17:14 -0600 Subject: [PATCH 35/39] We're calling them canonical DMs --- proposals/{2199-immutable-dms.md => 2199-canonical-dms.md} | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) rename proposals/{2199-immutable-dms.md => 2199-canonical-dms.md} (99%) diff --git a/proposals/2199-immutable-dms.md b/proposals/2199-canonical-dms.md similarity index 99% rename from proposals/2199-immutable-dms.md rename to proposals/2199-canonical-dms.md index 5382b89605b..ee11d7eb8ea 100644 --- a/proposals/2199-immutable-dms.md +++ b/proposals/2199-canonical-dms.md @@ -1,4 +1,4 @@ -# Immutable DMs (server-side middle ground edition) +# Canonical DMs (server-side middle ground edition) In the messaging space it is common for apps to expose exactly 1 chat for a conversation between two users, and in some cases a specific set of users. Matrix currently does not @@ -47,6 +47,9 @@ This proposal supports "direct chats" or "DMs" in the sense of private conversat a set of users. This means that a DM between 3 users is possible, however the common case is likely to be 1:1 (2 person) chats. +Immutability of DMs is also proposed in this MSC, however as an optional feature of DMs and +not a requirement. + ## Proposal From 17108f5b6ee87120c2f982be9f6f63c108ef9d87 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 28 Aug 2019 12:07:37 -0600 Subject: [PATCH 36/39] Support a dedicate /createDm endpoint for DMs This is a bit simpler to implement, mostly. --- proposals/2199-canonical-dms.md | 145 +++++++++++++++++++------------- 1 file changed, 85 insertions(+), 60 deletions(-) diff --git a/proposals/2199-canonical-dms.md b/proposals/2199-canonical-dms.md index ee11d7eb8ea..34c6d2862a7 100644 --- a/proposals/2199-canonical-dms.md +++ b/proposals/2199-canonical-dms.md @@ -219,66 +219,87 @@ to a given user. #### Creating DMs The room creation preset `trusted_private_chat` is deprecated and to be removed in a future -specification version. Clients should stop using those presets and instead use the new preset -`immutable_dm`, as defined here. The new preset has the following effects on the -room state: - -* Join rules of `invite`. -* Rejoin rules of `join` (as per [MSC2213](https://github.com/matrix-org/matrix-doc/pull/2213)). - This is to allow DMs to be re-used (described later) without sacrificing security of the DM - by letting random people who were invited at some point into the room. -* History visibility of `invited`. -* Guest access of `can_join` so that guests can be included in DMs where needed. -* Power levels with the following non-default structure: - ```json - { - "events_default": 0, - "state_default": 50, - "users_default": 0, - "ban": 50, - "invite": 50, - "kick": 50, - "redact": 50, - "notifications": { - "room": 50 - }, - "events": { - "m.room.name": 100, - "m.room.avatar": 100, - "m.room.topic": 100, - "m.room.history_visibility": 100, - "m.room.power_levels": 100, - "m.room.join_rules": 100, - "m.room.encryption": 100, - "m.room.canonical_alias": 100 - }, - "users": { - - }, - "third_party_users": { - - } - } - ``` - This power level event is not applied until after the invites have been sent as otherwise - it would be impossible to give third party users power. Servers should apply a default - power level event to the room and then apply the power level event described here after - the invites have been sent. Servers can optimize this process and use this power level - event at step 0 of the room creation process if there are no third party invites to send. - Users invited to the room get power level 50, including the creator. -* Third party users invited to the room get power level 50, as described by MSC2212 (see - later on in this proposal for how this works). Like the invited users, all third party - users invited as a result of the `/createRoom` call are considered important. -* Important users (those invited and the creator) MUST have `"m.dm": true` in their - membership event content. Third party important users get the same `m.dm` flag on - their `m.room.third_party_invite` event contents. -* Encryption is enabled by default using the most preferred megolm algorithm in the spec. - Currently this would be `m.megolm.v1.aes-sha2`. - -The preset prevents the use of `visibility`, `room_alias_name`, `name`, `topic`, `initial_state`, -and `power_level_content_override` during creation. Servers MUST reject the request with -`400 M_BAD_STATE` when the request contains conflicting properties. Future extensions to -`/createRoom` are expected to be included in the forbidden list where appropriate. +specification version. Clients should stop using that preset and instead use the new endpoint +for creating DMs, as defined here. + +**`POST /_matrix/client/r0/createDm`**: + +This is mostly a clone of `/createRoom` with the unnecessary parts left behind. This does not +replace `/createRoom`. This requires authentication and can be rate-limited. + +Request parameters (JSON body): +* `room_version` (`string`) - The version to create the room with. Same as `room_version` from + `/createRoom`. +* `invite` (`[string]`) - The list of user IDs to invite to the room. Same as `invite` from + `/createRoom` with the added behaviour described below. +* `invite_3pid` (`[3pid invite]`) - The list of 3rd party users to invite to the room. Same as + `invite_3pid` from `/createRoom` with the added behaviour described below. +* `creation_content` (`object`) - Additional content for the creation event. This is also the + same as `creation_content` from `/createRoom`. + +Note: `name`, `topic`, `visibility`, `room_alias_name`, `initial_state`, `preset`, `is_direct`, +and `power_level_content_override` are all intentionally excluded from the clone. There is no +preset support, aesthetic characteristics of the room (name, topic, etc) are not supported for +immutable DMs under this MSC, and the remaining fields do not have strong use cases for DMs - +clients can always send state event updates after creating the room, for instance. + +Response (JSON): +The server creates the room similar to how `/createRoom` operates: + +1. An `m.room.create` event is created as is `m.room.power_levels` using the defaults. +2. Join rules of `invite` are set with a rejoin rule of `join`, as per + [MSC2213](https://github.com/matrix-org/matrix-doc/pull/2213)). This is to allow DMs to be + re-used (described later) without sacrificing security of the DM by letting random people + who were invited at some point into the room. +3. History visibility of `invited` is set. +4. Guest access of `can_join` is set so that guests can be included in DMs where needed. +5. Encryption is enabled by default using the most preferred megolm algorithm in the spec. + Currently this would be `m.megolm.v1.aes-sha2`. +6. Power levels with the following non-default structure are set: + + ```json + { + "events_default": 0, + "state_default": 50, + "users_default": 0, + "ban": 50, + "invite": 50, + "kick": 50, + "redact": 50, + "notifications": { + "room": 50 + }, + "events": { + "m.room.name": 100, + "m.room.avatar": 100, + "m.room.topic": 100, + "m.room.history_visibility": 100, + "m.room.power_levels": 100, + "m.room.join_rules": 100, + "m.room.encryption": 100, + "m.room.canonical_alias": 100 + }, + "users": { + + }, + "third_party_users": { + + } + } + ``` + + This power level event is not applied until after the invites have been sent as otherwise + it would be impossible to give third party users power. Servers should apply a default + power level event to the room and then apply the power level event described here after + the invites have been sent. Servers can optimize this process and use this power level + event at step 0 of the room creation process if there are no third party invites to send. + Users invited to the room get power level 50, including the creator. + * Third party users invited to the room get power level 50, as described by MSC2212 (see + later on in this proposal for how this works). Like the invited users, all third party + users invited as a result of the `/createDm` call are considered important. +7. Users (3rd party and otherwise) are invited. Important users (those invited and the creator) + MUST have `"m.dm": true` in their membership event content. Third party important users get + the same `m.dm` flag on their `m.room.third_party_invite` event contents. Servers MUST return an existing room ID if the server already knows about a DM between the important users. The important users which have left the DM MUST be explicitly re-invited. @@ -296,6 +317,10 @@ room at any time to escape power level struggles, although this may not maintain *Note*: The `private_chat` preset is untouched as it does not affect DMs. `trusted_private_chat` was intended for DMs despite its name and is deprecated as a result. +*Note*: Servers are still expected to be able to identify DMs from `/createRoom`, and clients +are able to craft DMs using `/createRoom`. When this happens, servers MUST return an existing +room ID for a DM if one exists and it looks like the user is trying to create a DM through the +`/createRoom` endpoint. #### Glare and duplicated rooms From 553caf101d724247b72a45232f5ca1bfc36c5982 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 28 Aug 2019 12:54:32 -0600 Subject: [PATCH 37/39] createDm -> create_dm we don't prefer camelcasing in our API where possible --- proposals/2199-canonical-dms.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/2199-canonical-dms.md b/proposals/2199-canonical-dms.md index 34c6d2862a7..de43411a0a3 100644 --- a/proposals/2199-canonical-dms.md +++ b/proposals/2199-canonical-dms.md @@ -222,7 +222,7 @@ The room creation preset `trusted_private_chat` is deprecated and to be removed specification version. Clients should stop using that preset and instead use the new endpoint for creating DMs, as defined here. -**`POST /_matrix/client/r0/createDm`**: +**`POST /_matrix/client/r0/create_dm`**: This is mostly a clone of `/createRoom` with the unnecessary parts left behind. This does not replace `/createRoom`. This requires authentication and can be rate-limited. @@ -296,7 +296,7 @@ The server creates the room similar to how `/createRoom` operates: Users invited to the room get power level 50, including the creator. * Third party users invited to the room get power level 50, as described by MSC2212 (see later on in this proposal for how this works). Like the invited users, all third party - users invited as a result of the `/createDm` call are considered important. + users invited as a result of the `/create_dm` call are considered important. 7. Users (3rd party and otherwise) are invited. Important users (those invited and the creator) MUST have `"m.dm": true` in their membership event content. Third party important users get the same `m.dm` flag on their `m.room.third_party_invite` event contents. From 54dbbbdc64003dc7b35afe17c87a9858dc006968 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 11 Sep 2019 15:54:07 -0600 Subject: [PATCH 38/39] Update proposals/2199-canonical-dms.md Co-Authored-By: Hubert Chathi --- proposals/2199-canonical-dms.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2199-canonical-dms.md b/proposals/2199-canonical-dms.md index de43411a0a3..f91c5a57899 100644 --- a/proposals/2199-canonical-dms.md +++ b/proposals/2199-canonical-dms.md @@ -696,7 +696,7 @@ being created. The existing DM system in Matrix support multiple DMs with a user, however that system is not necsarily more complex than this proposal (in fact, it's probably simpler than what is proposed here). The existing system has the disadvantage of being non-deterministic where -two users in the same room cna have a different view of that room in their clients: Alice +two users in the same room can have a different view of that room in their clients: Alice can see it as a DM whereas Bob sees it as just another room. Having non-deterministic DMs can lead to conversational confusion ("hey Bob, let's go to our DM" and Bob asking where that is) or uncertainty about where to send sensitive information. From 8ab1af3c603e95c09153a495f42a8fcacd2980bd Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 8 Oct 2019 13:11:53 +0100 Subject: [PATCH 39/39] Update proposals/2199-canonical-dms.md Co-Authored-By: David Baker --- proposals/2199-canonical-dms.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2199-canonical-dms.md b/proposals/2199-canonical-dms.md index f91c5a57899..77bdffc8d7d 100644 --- a/proposals/2199-canonical-dms.md +++ b/proposals/2199-canonical-dms.md @@ -246,7 +246,7 @@ clients can always send state event updates after creating the room, for instanc Response (JSON): The server creates the room similar to how `/createRoom` operates: -1. An `m.room.create` event is created as is `m.room.power_levels` using the defaults. +1. An `m.room.create` event is created, as is `m.room.power_levels` using the defaults. 2. Join rules of `invite` are set with a rejoin rule of `join`, as per [MSC2213](https://github.com/matrix-org/matrix-doc/pull/2213)). This is to allow DMs to be re-used (described later) without sacrificing security of the DM by letting random people