From 52d8d772d900314db0f0fad7ca9d5ed282a2658b Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 29 Aug 2020 21:02:12 +0300 Subject: [PATCH 01/21] MSC2753: Peeking via sync (take 2) --- proposals/2753-peeking-via-sync-v2.md | 69 +++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) create mode 100644 proposals/2753-peeking-via-sync-v2.md diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md new file mode 100644 index 00000000000..42f012484b2 --- /dev/null +++ b/proposals/2753-peeking-via-sync-v2.md @@ -0,0 +1,69 @@ +# MSC2753: Peeking via Sync (Take 2) + +## Problem + +Peeking into rooms without joining them currently relies on the deprecated v1 /initialSync and /events APIs. + +This poses the following issues: + + * Servers and clients must implement two separate sets of event-syncing logic, doubling complexity. + * Peeking a room involves setting a stream of long-lived /events requests + going. Having multiple streams is racey, competes for resources with the + /sync stream, and doesn't scale given each room requires a new /events + stream. + * v1 APIs are deprecated and not implemented on new servers. + +This MSC likely obsoletes [MSC1763](https://github.com/matrix-org/matrix-doc/pulls/1763). + +## Proposal + +We add an CS API called `/peek/{roomIdOrAlias}`, very similar to `/join/{roomIdOrAlias}`. + +Calling `/peek`: + * Resolves the given room alias to a room ID, if needed. + * Adds the room (if permissions allow) to a new section of the /sync response called `peeking` - but only for the device which called `/peek`. + * The `peeking` section is identical to `joined`, but shows the live activity of rooms for which that device is peeking. + +The API returns 404 on an unrecognised room ID or alias, or 403 if the room does not allow peeking. + +Similar to `/join`, `/peek` lets you specify `server_name` querystring parameters to specify which server(s) to try to peek into the room via (when coupled with [MSC2444](https://github.com/matrix-org/matrix-doc/pulls/2444)). + +If a user subsequently `/join`s the room they're peeking, we atomically move the room to the `joined` block of the `/sync` response, allowing the client to build on the state and history it has already received without re-sending it down /sync. + +To stop peeking, the user calls `/unpeek` on the room, similar to `/leave` or `/forget`. This returns 200 on success, 404 on unrecognised ID, or 400 if the room was not being peeked in the first place. + +## Potential issues + +It could be seen as controversial to add another new block to the `/sync` response. We could use the existing `joined` block, but: + + a) it's a misnomer (given the user hasn't joined the rooms) + b) `joined` refers to rooms which the *user* is in, rather than that they are peeking into using a given device + c) we risk breaking clients who aren't aware of the new style of peeking. + d) there's already a precedent for per-device blocks in the sync response (for to-device messages) + +It could be seen as controversial to make peeking a per-device rather than per-user feature. When thinking through use cases for peeking, however: + + 1. Peeking a chatroom before joining it is the common case, and is definitely per-device - you would not expect peeked rooms to randomly pop up on other devices, or consume their bandwidth. + 2. [MSC1769](https://github.com/matrix-org/matrix-doc/pulls/1769) (Profiles as rooms) is also per device: if a given client wants to look at the Member Info for a given user in a room, it shouldn't pollute the others with that data. + 3. [MSC1772](https://github.com/matrix-org/matrix-doc/pulls/1772) (Groups as rooms) uses room joins to indicate your own membership, and peeks to query the group membership of other users. Similarly to profiles, it's not clear that this should be per-user rather than per-device (and worse case, it's a matter of effectively opting in rather than trying to filter out peeks you don't care about). + +## Alternatives + +[MSC1763](https://github.com/matrix-org/matrix-doc/pulls/1763) defined an alternative approach, where you could use filters to add peeked rooms into a given `/sync` response as needed. This however had some issues: + + * You can't specify what servers to peek remote rooms via. + * You can't identify rooms via alias, only ID + * It feels hacky to bodge peeked rooms into the `joined` block of a given `/sync` response + * Fiddling around with custom filters feels clunky relative to just calling a `/peek` endpoint similar to `/join`. + +While experimenting with implementing MSC1763, I came up with this as an alternative that empirically feels much simpler and tidier. + +## Security considerations + +Servers should ratelimit calls to `/peek` to stop someone DoSing the server. + +Servers may stop maintaining a `/peek` if its device has not `/sync`ed recently, and thus reclaim resources. At the next `/sync` the server would need to restore the `peek` and provide a gappy update. This is most relevant in the context of peeking into a remote room via [MSC2444](https://github.com/matrix-org/matrix-doc/pulls/2444) however. + +## Unstable prefix + +TBD \ No newline at end of file From 2f883435bed93e7bc95acad47ea1b18bde2d971c Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 29 Aug 2020 21:06:43 +0300 Subject: [PATCH 02/21] fix MSC refs --- proposals/2753-peeking-via-sync-v2.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index 42f012484b2..1d43f0c7567 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -13,7 +13,7 @@ This poses the following issues: stream. * v1 APIs are deprecated and not implemented on new servers. -This MSC likely obsoletes [MSC1763](https://github.com/matrix-org/matrix-doc/pulls/1763). +This MSC likely obsoletes [MSC1776](https://github.com/matrix-org/matrix-doc/pulls/1776). ## Proposal @@ -49,14 +49,14 @@ It could be seen as controversial to make peeking a per-device rather than per-u ## Alternatives -[MSC1763](https://github.com/matrix-org/matrix-doc/pulls/1763) defined an alternative approach, where you could use filters to add peeked rooms into a given `/sync` response as needed. This however had some issues: +[MSC1776](https://github.com/matrix-org/matrix-doc/pulls/1776) defined an alternative approach, where you could use filters to add peeked rooms into a given `/sync` response as needed. This however had some issues: * You can't specify what servers to peek remote rooms via. * You can't identify rooms via alias, only ID * It feels hacky to bodge peeked rooms into the `joined` block of a given `/sync` response * Fiddling around with custom filters feels clunky relative to just calling a `/peek` endpoint similar to `/join`. -While experimenting with implementing MSC1763, I came up with this as an alternative that empirically feels much simpler and tidier. +While experimenting with implementing MSC1776, I came up with this as an alternative that empirically feels much simpler and tidier. ## Security considerations From 4aba8c847790d58702c23ee796d0c61318f85cff Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 29 Aug 2020 21:07:08 +0300 Subject: [PATCH 03/21] fix MSC refs again --- proposals/2753-peeking-via-sync-v2.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index 1d43f0c7567..b339141c30b 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -13,7 +13,7 @@ This poses the following issues: stream. * v1 APIs are deprecated and not implemented on new servers. -This MSC likely obsoletes [MSC1776](https://github.com/matrix-org/matrix-doc/pulls/1776). +This MSC likely obsoletes [MSC1776](https://github.com/matrix-org/matrix-doc/pull/1776). ## Proposal @@ -26,7 +26,7 @@ Calling `/peek`: The API returns 404 on an unrecognised room ID or alias, or 403 if the room does not allow peeking. -Similar to `/join`, `/peek` lets you specify `server_name` querystring parameters to specify which server(s) to try to peek into the room via (when coupled with [MSC2444](https://github.com/matrix-org/matrix-doc/pulls/2444)). +Similar to `/join`, `/peek` lets you specify `server_name` querystring parameters to specify which server(s) to try to peek into the room via (when coupled with [MSC2444](https://github.com/matrix-org/matrix-doc/pull/2444)). If a user subsequently `/join`s the room they're peeking, we atomically move the room to the `joined` block of the `/sync` response, allowing the client to build on the state and history it has already received without re-sending it down /sync. @@ -44,12 +44,12 @@ It could be seen as controversial to add another new block to the `/sync` respon It could be seen as controversial to make peeking a per-device rather than per-user feature. When thinking through use cases for peeking, however: 1. Peeking a chatroom before joining it is the common case, and is definitely per-device - you would not expect peeked rooms to randomly pop up on other devices, or consume their bandwidth. - 2. [MSC1769](https://github.com/matrix-org/matrix-doc/pulls/1769) (Profiles as rooms) is also per device: if a given client wants to look at the Member Info for a given user in a room, it shouldn't pollute the others with that data. - 3. [MSC1772](https://github.com/matrix-org/matrix-doc/pulls/1772) (Groups as rooms) uses room joins to indicate your own membership, and peeks to query the group membership of other users. Similarly to profiles, it's not clear that this should be per-user rather than per-device (and worse case, it's a matter of effectively opting in rather than trying to filter out peeks you don't care about). + 2. [MSC1769](https://github.com/matrix-org/matrix-doc/pull/1769) (Profiles as rooms) is also per device: if a given client wants to look at the Member Info for a given user in a room, it shouldn't pollute the others with that data. + 3. [MSC1772](https://github.com/matrix-org/matrix-doc/pull/1772) (Groups as rooms) uses room joins to indicate your own membership, and peeks to query the group membership of other users. Similarly to profiles, it's not clear that this should be per-user rather than per-device (and worse case, it's a matter of effectively opting in rather than trying to filter out peeks you don't care about). ## Alternatives -[MSC1776](https://github.com/matrix-org/matrix-doc/pulls/1776) defined an alternative approach, where you could use filters to add peeked rooms into a given `/sync` response as needed. This however had some issues: +[MSC1776](https://github.com/matrix-org/matrix-doc/pull/1776) defined an alternative approach, where you could use filters to add peeked rooms into a given `/sync` response as needed. This however had some issues: * You can't specify what servers to peek remote rooms via. * You can't identify rooms via alias, only ID @@ -62,7 +62,7 @@ While experimenting with implementing MSC1776, I came up with this as an alterna Servers should ratelimit calls to `/peek` to stop someone DoSing the server. -Servers may stop maintaining a `/peek` if its device has not `/sync`ed recently, and thus reclaim resources. At the next `/sync` the server would need to restore the `peek` and provide a gappy update. This is most relevant in the context of peeking into a remote room via [MSC2444](https://github.com/matrix-org/matrix-doc/pulls/2444) however. +Servers may stop maintaining a `/peek` if its device has not `/sync`ed recently, and thus reclaim resources. At the next `/sync` the server would need to restore the `peek` and provide a gappy update. This is most relevant in the context of peeking into a remote room via [MSC2444](https://github.com/matrix-org/matrix-doc/pull/2444) however. ## Unstable prefix From 752d5694fe812e319861cc1160b011d6ec5ddf53 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 29 Aug 2020 23:35:29 +0300 Subject: [PATCH 04/21] fix md --- proposals/2753-peeking-via-sync-v2.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index b339141c30b..48c7094c182 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -36,10 +36,10 @@ To stop peeking, the user calls `/unpeek` on the room, similar to `/leave` or `/ It could be seen as controversial to add another new block to the `/sync` response. We could use the existing `joined` block, but: - a) it's a misnomer (given the user hasn't joined the rooms) - b) `joined` refers to rooms which the *user* is in, rather than that they are peeking into using a given device - c) we risk breaking clients who aren't aware of the new style of peeking. - d) there's already a precedent for per-device blocks in the sync response (for to-device messages) + * it's a misnomer (given the user hasn't joined the rooms) + * `joined` refers to rooms which the *user* is in, rather than that they are peeking into using a given device + * we risk breaking clients who aren't aware of the new style of peeking. + * there's already a precedent for per-device blocks in the sync response (for to-device messages) It could be seen as controversial to make peeking a per-device rather than per-user feature. When thinking through use cases for peeking, however: @@ -66,4 +66,4 @@ Servers may stop maintaining a `/peek` if its device has not `/sync`ed recently, ## Unstable prefix -TBD \ No newline at end of file +TBD From 74cc1c33cad28bf2717a9d881a759b2d2dc7a38a Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 31 Aug 2020 13:49:16 +0300 Subject: [PATCH 05/21] s/peeking/peek/ and s/joined/join/ --- proposals/2753-peeking-via-sync-v2.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index 48c7094c182..3320f6cfda7 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -21,23 +21,23 @@ We add an CS API called `/peek/{roomIdOrAlias}`, very similar to `/join/{roomIdO Calling `/peek`: * Resolves the given room alias to a room ID, if needed. - * Adds the room (if permissions allow) to a new section of the /sync response called `peeking` - but only for the device which called `/peek`. - * The `peeking` section is identical to `joined`, but shows the live activity of rooms for which that device is peeking. + * Adds the room (if permissions allow) to a new section of the /sync response called `peek` - but only for the device which called `/peek`. + * The `peek` section is identical to `join`, but shows the live activity of rooms for which that device is peeking. The API returns 404 on an unrecognised room ID or alias, or 403 if the room does not allow peeking. Similar to `/join`, `/peek` lets you specify `server_name` querystring parameters to specify which server(s) to try to peek into the room via (when coupled with [MSC2444](https://github.com/matrix-org/matrix-doc/pull/2444)). -If a user subsequently `/join`s the room they're peeking, we atomically move the room to the `joined` block of the `/sync` response, allowing the client to build on the state and history it has already received without re-sending it down /sync. +If a user subsequently `/join`s the room they're peeking, we atomically move the room to the `join` block of the `/sync` response, allowing the client to build on the state and history it has already received without re-sending it down /sync. To stop peeking, the user calls `/unpeek` on the room, similar to `/leave` or `/forget`. This returns 200 on success, 404 on unrecognised ID, or 400 if the room was not being peeked in the first place. ## Potential issues -It could be seen as controversial to add another new block to the `/sync` response. We could use the existing `joined` block, but: +It could be seen as controversial to add another new block to the `/sync` response. We could use the existing `join` block, but: * it's a misnomer (given the user hasn't joined the rooms) - * `joined` refers to rooms which the *user* is in, rather than that they are peeking into using a given device + * `join` refers to rooms which the *user* is in, rather than that they are peeking into using a given *device* * we risk breaking clients who aren't aware of the new style of peeking. * there's already a precedent for per-device blocks in the sync response (for to-device messages) @@ -53,7 +53,7 @@ It could be seen as controversial to make peeking a per-device rather than per-u * You can't specify what servers to peek remote rooms via. * You can't identify rooms via alias, only ID - * It feels hacky to bodge peeked rooms into the `joined` block of a given `/sync` response + * It feels hacky to bodge peeked rooms into the `join` block of a given `/sync` response * Fiddling around with custom filters feels clunky relative to just calling a `/peek` endpoint similar to `/join`. While experimenting with implementing MSC1776, I came up with this as an alternative that empirically feels much simpler and tidier. From c4c4b35807265de17d3d7b39968689358124fbe2 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 31 Aug 2020 17:50:47 +0300 Subject: [PATCH 06/21] spell out that joins should automatically cancel peeks, and that clients should tidy up stale peeks --- proposals/2753-peeking-via-sync-v2.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index 3320f6cfda7..06d094b63b7 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -32,6 +32,10 @@ If a user subsequently `/join`s the room they're peeking, we atomically move the To stop peeking, the user calls `/unpeek` on the room, similar to `/leave` or `/forget`. This returns 200 on success, 404 on unrecognised ID, or 400 if the room was not being peeked in the first place. +The act of joining a peeked room automatically cancels any ongoing /peeks by that user. + +Clients should check for any irrelevant peeked rooms on launch (left over from previous instances of the app) and explicitly `/unpeek` them to conserve resources. + ## Potential issues It could be seen as controversial to add another new block to the `/sync` response. We could use the existing `join` block, but: From 4fba9278228f77690312e269ec079daf410f01a1 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Fri, 4 Sep 2020 13:19:24 +0100 Subject: [PATCH 07/21] incorporate review --- proposals/2753-peeking-via-sync-v2.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index 06d094b63b7..3e109dea43f 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -24,7 +24,11 @@ Calling `/peek`: * Adds the room (if permissions allow) to a new section of the /sync response called `peek` - but only for the device which called `/peek`. * The `peek` section is identical to `join`, but shows the live activity of rooms for which that device is peeking. -The API returns 404 on an unrecognised room ID or alias, or 403 if the room does not allow peeking. +The API returns 404 on an unrecognised room ID or alias, or 403 if the room does not allow peeking. Rooms allow peeking if they have history_visibility of "world_readable". N.B. join_rules do not affect peekability - it's possible to have an invite-only room which joe public can still peek into, if history_visibility has been set to "world_readable". + +If the history_visibility of a room changes to not be world_readable, any peeks on the room are cancelled. + +In order to clear up stale peeks if a client restarts without cleaning up nicely, the act of calling /sync without a `since` token must cancel any peeks created by that device. Similar to `/join`, `/peek` lets you specify `server_name` querystring parameters to specify which server(s) to try to peek into the room via (when coupled with [MSC2444](https://github.com/matrix-org/matrix-doc/pull/2444)). From a0709352593cddcc7e30799512151d00e10e0be1 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 8 Sep 2020 10:49:54 +0100 Subject: [PATCH 08/21] spell out that we should flash unpeeks in leave to let the client know they've vanished --- proposals/2753-peeking-via-sync-v2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index 3e109dea43f..5ed207d5f19 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -34,7 +34,7 @@ Similar to `/join`, `/peek` lets you specify `server_name` querystring parameter If a user subsequently `/join`s the room they're peeking, we atomically move the room to the `join` block of the `/sync` response, allowing the client to build on the state and history it has already received without re-sending it down /sync. -To stop peeking, the user calls `/unpeek` on the room, similar to `/leave` or `/forget`. This returns 200 on success, 404 on unrecognised ID, or 400 if the room was not being peeked in the first place. +To stop peeking, the user calls `/unpeek` on the room, similar to `/leave` or `/forget`. This returns 200 on success, 404 on unrecognised ID, or 400 if the room was not being peeked in the first place. Having stopped peeking, the empty room block appears in the `leave` block of the next sync response to tell the client that the user is no longer peeking. The act of joining a peeked room automatically cancels any ongoing /peeks by that user. From acec3005a28b689185b79b226bb69acf59d4ed1f Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 20 Oct 2020 18:06:42 +0100 Subject: [PATCH 09/21] format and edit minor edits --- proposals/2753-peeking-via-sync-v2.md | 108 ++++++++++++++++++-------- 1 file changed, 76 insertions(+), 32 deletions(-) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index 5ed207d5f19..2c187747049 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -21,56 +21,100 @@ We add an CS API called `/peek/{roomIdOrAlias}`, very similar to `/join/{roomIdO Calling `/peek`: * Resolves the given room alias to a room ID, if needed. - * Adds the room (if permissions allow) to a new section of the /sync response called `peek` - but only for the device which called `/peek`. - * The `peek` section is identical to `join`, but shows the live activity of rooms for which that device is peeking. - -The API returns 404 on an unrecognised room ID or alias, or 403 if the room does not allow peeking. Rooms allow peeking if they have history_visibility of "world_readable". N.B. join_rules do not affect peekability - it's possible to have an invite-only room which joe public can still peek into, if history_visibility has been set to "world_readable". - -If the history_visibility of a room changes to not be world_readable, any peeks on the room are cancelled. - -In order to clear up stale peeks if a client restarts without cleaning up nicely, the act of calling /sync without a `since` token must cancel any peeks created by that device. - -Similar to `/join`, `/peek` lets you specify `server_name` querystring parameters to specify which server(s) to try to peek into the room via (when coupled with [MSC2444](https://github.com/matrix-org/matrix-doc/pull/2444)). - -If a user subsequently `/join`s the room they're peeking, we atomically move the room to the `join` block of the `/sync` response, allowing the client to build on the state and history it has already received without re-sending it down /sync. - -To stop peeking, the user calls `/unpeek` on the room, similar to `/leave` or `/forget`. This returns 200 on success, 404 on unrecognised ID, or 400 if the room was not being peeked in the first place. Having stopped peeking, the empty room block appears in the `leave` block of the next sync response to tell the client that the user is no longer peeking. - -The act of joining a peeked room automatically cancels any ongoing /peeks by that user. - -Clients should check for any irrelevant peeked rooms on launch (left over from previous instances of the app) and explicitly `/unpeek` them to conserve resources. + * Adds the room (if permissions allow) to a new section of the /sync response + called `peek` - but only for the device which called `/peek`. + * The `peek` section is identical to `join`, but shows the live activity of + rooms for which that device is peeking. + +The API returns 404 on an unrecognised room ID or alias, or 403 if the room +does not allow peeking. Rooms allow peeking if they have `history_visibility` +of `world_readable`. N.B. `join_rules` do not affect peekability - it's +possible to have an invite-only room which joe public can still peek into, if +`history_visibility` has been set to `world_readable`. + +If the `history_visibility` of a room changes to not be `world_readable`, any +peeks on the room are cancelled. + +In order to clear up stale peeks if a client restarts without cleaning up +nicely, the act of calling `/sync` without a `since` token must cancel any peeks +created by that device. + +Similar to `/join`, `/peek` lets you specify `server_name` querystring +parameters to specify which server(s) to try to peek into the room via (when +coupled with [MSC2444](https://github.com/matrix-org/matrix-doc/pull/2444)). + +If a user subsequently `/join`s the room they're peeking, we atomically move +the room to the `join` block of the `/sync` response, allowing the client to +build on the state and history it has already received without re-sending it +down `/sync`. + +To stop peeking, the user calls `/unpeek` on the room, similar to `/leave` or +`/forget`. This returns 200 on success, 404 on unrecognised ID, or 400 if the +room was not being peeked in the first place. Having stopped peeking, the +empty room block appears in the `leave` block of the next sync response to tell +the client that the user is no longer peeking. + +The act of joining a peeked room automatically cancels any ongoing `/peeks` by +that user. + +Clients should check for any irrelevant peeked rooms on launch (left over from +previous instances of the app) and explicitly `/unpeek` them to conserve +resources. ## Potential issues -It could be seen as controversial to add another new block to the `/sync` response. We could use the existing `join` block, but: +It could be seen as controversial to add another new block to the `/sync` +response. We could use the existing `join` block, but: * it's a misnomer (given the user hasn't joined the rooms) - * `join` refers to rooms which the *user* is in, rather than that they are peeking into using a given *device* + * `join` refers to rooms which the *user* is in, rather than that they are + peeking into using a given *device* * we risk breaking clients who aren't aware of the new style of peeking. - * there's already a precedent for per-device blocks in the sync response (for to-device messages) - -It could be seen as controversial to make peeking a per-device rather than per-user feature. When thinking through use cases for peeking, however: - - 1. Peeking a chatroom before joining it is the common case, and is definitely per-device - you would not expect peeked rooms to randomly pop up on other devices, or consume their bandwidth. - 2. [MSC1769](https://github.com/matrix-org/matrix-doc/pull/1769) (Profiles as rooms) is also per device: if a given client wants to look at the Member Info for a given user in a room, it shouldn't pollute the others with that data. - 3. [MSC1772](https://github.com/matrix-org/matrix-doc/pull/1772) (Groups as rooms) uses room joins to indicate your own membership, and peeks to query the group membership of other users. Similarly to profiles, it's not clear that this should be per-user rather than per-device (and worse case, it's a matter of effectively opting in rather than trying to filter out peeks you don't care about). + * there's already a precedent for per-device blocks in the sync response (for + to-device messages) + +It could be seen as controversial to make peeking a per-device rather than +per-user feature. When thinking through use cases for peeking, however: + + 1. Peeking a chatroom before joining it is the common case, and is definitely + per-device - you would not expect peeked rooms to randomly pop up on other + devices, or consume their bandwidth. + 2. [MSC1769](https://github.com/matrix-org/matrix-doc/pull/1769) (Profiles as + rooms) is also per device: if a given client wants to look at the Member + Info for a given user in a room, it shouldn't pollute the others with that + data. + 3. [MSC1772](https://github.com/matrix-org/matrix-doc/pull/1772) (Groups as + rooms) uses room joins to indicate your own membership, and peeks to query + the group membership of other users. Similarly to profiles, it's not clear + that this should be per-user rather than per-device (and worse case, it's a + matter of effectively opting in rather than trying to filter out peeks you + don't care about). ## Alternatives -[MSC1776](https://github.com/matrix-org/matrix-doc/pull/1776) defined an alternative approach, where you could use filters to add peeked rooms into a given `/sync` response as needed. This however had some issues: +[MSC1776](https://github.com/matrix-org/matrix-doc/pull/1776) defined an +alternative approach, where you could use filters to add peeked rooms into a +given `/sync` response as needed. This however had some issues: * You can't specify what servers to peek remote rooms via. * You can't identify rooms via alias, only ID - * It feels hacky to bodge peeked rooms into the `join` block of a given `/sync` response - * Fiddling around with custom filters feels clunky relative to just calling a `/peek` endpoint similar to `/join`. + * It feels hacky to bodge peeked rooms into the `join` block of a given + `/sync` response + * Fiddling around with custom filters feels clunky relative to just calling a + `/peek` endpoint similar to `/join`. -While experimenting with implementing MSC1776, I came up with this as an alternative that empirically feels much simpler and tidier. +While experimenting with implementing MSC1776, I came up with this as an +alternative that empirically feels much simpler and tidier. ## Security considerations Servers should ratelimit calls to `/peek` to stop someone DoSing the server. -Servers may stop maintaining a `/peek` if its device has not `/sync`ed recently, and thus reclaim resources. At the next `/sync` the server would need to restore the `peek` and provide a gappy update. This is most relevant in the context of peeking into a remote room via [MSC2444](https://github.com/matrix-org/matrix-doc/pull/2444) however. +Servers may stop maintaining a `/peek` if its device has not `/sync`ed +recently, and thus reclaim resources. At the next `/sync` the server would +need to restore the `peek` and provide a gappy update. This is most relevant +in the context of peeking into a remote room via +[MSC2444](https://github.com/matrix-org/matrix-doc/pull/2444) however. ## Unstable prefix From a2551407bc44599b3abb68aff61c15f4bd081939 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 20 Oct 2020 18:08:57 +0100 Subject: [PATCH 10/21] add information about responses --- proposals/2753-peeking-via-sync-v2.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index 2c187747049..2f63b6ec27e 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -54,6 +54,10 @@ room was not being peeked in the first place. Having stopped peeking, the empty room block appears in the `leave` block of the next sync response to tell the client that the user is no longer peeking. +The new `/peek` and `/unpeek` endpoints require authentication and can be +ratelimited. Their responses are analogous to their `/join` and `/leave` +counterparts (eg: `room_id` in the response and empty object when stopping). + The act of joining a peeked room automatically cancels any ongoing `/peeks` by that user. @@ -61,6 +65,7 @@ Clients should check for any irrelevant peeked rooms on launch (left over from previous instances of the app) and explicitly `/unpeek` them to conserve resources. + ## Potential issues It could be seen as controversial to add another new block to the `/sync` From 558f66fcd1c87b78e0457e471d74aaa17ea66f4d Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 20 Oct 2020 18:19:34 +0100 Subject: [PATCH 11/21] remove duplicated info --- proposals/2753-peeking-via-sync-v2.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index 2f63b6ec27e..7dd2b681cb2 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -58,14 +58,10 @@ The new `/peek` and `/unpeek` endpoints require authentication and can be ratelimited. Their responses are analogous to their `/join` and `/leave` counterparts (eg: `room_id` in the response and empty object when stopping). -The act of joining a peeked room automatically cancels any ongoing `/peeks` by -that user. - Clients should check for any irrelevant peeked rooms on launch (left over from previous instances of the app) and explicitly `/unpeek` them to conserve resources. - ## Potential issues It could be seen as controversial to add another new block to the `/sync` From 4dea820c28c695c309daa5f0f292c89de2ba27ef Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 20 Oct 2020 18:19:40 +0100 Subject: [PATCH 12/21] add unstable prefix --- proposals/2753-peeking-via-sync-v2.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index 7dd2b681cb2..248854f1797 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -119,4 +119,11 @@ in the context of peeking into a remote room via ## Unstable prefix -TBD + +The following mapping will be used for identifiers in this MSC during +development: + +Proposed final identifier | Purpose | Development identifier +------------------------------- | ------- | ---- +`/_matrix/client/r0/peek` | API endpoint | `/_matrix/client/unstable/org.matrix.msc2753/peek` +`/_matrix/client/r0/unpeek` | API endpoint | `/_matrix/client/unstable/org.matrix.msc2753/unpeek` From 8b08c0998ef58c23d26e46ddfd69079543883332 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 20 Oct 2020 18:23:47 +0100 Subject: [PATCH 13/21] more updates --- proposals/2753-peeking-via-sync-v2.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index 248854f1797..c785adc00d0 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -48,11 +48,11 @@ the room to the `join` block of the `/sync` response, allowing the client to build on the state and history it has already received without re-sending it down `/sync`. -To stop peeking, the user calls `/unpeek` on the room, similar to `/leave` or -`/forget`. This returns 200 on success, 404 on unrecognised ID, or 400 if the -room was not being peeked in the first place. Having stopped peeking, the -empty room block appears in the `leave` block of the next sync response to tell -the client that the user is no longer peeking. +To stop peeking, the user calls `/unpeek` on the room, similar to `/leave`. +This returns 200 on success, 404 on unrecognised ID, or 400 if the room was not +being peeked in the first place. Having stopped peeking, the empty room block +appears in the `leave` block of the next sync response to tell the client that +the user is no longer peeking. The new `/peek` and `/unpeek` endpoints require authentication and can be ratelimited. Their responses are analogous to their `/join` and `/leave` @@ -126,4 +126,4 @@ development: Proposed final identifier | Purpose | Development identifier ------------------------------- | ------- | ---- `/_matrix/client/r0/peek` | API endpoint | `/_matrix/client/unstable/org.matrix.msc2753/peek` -`/_matrix/client/r0/unpeek` | API endpoint | `/_matrix/client/unstable/org.matrix.msc2753/unpeek` +`/_matrix/client/r0/rooms/{roomId}/unpeek` | API endpoint | `/_matrix/client/unstable/org.matrix.msc2753/rooms/{roomId}/unpeek` From b2cb3f2f374280fc8baf5d3620c14064950708a9 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Wed, 21 Oct 2020 02:24:18 +0100 Subject: [PATCH 14/21] clarify interaction with e2ee --- proposals/2753-peeking-via-sync-v2.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index c785adc00d0..f6bb9b9f966 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -62,6 +62,16 @@ Clients should check for any irrelevant peeked rooms on launch (left over from previous instances of the app) and explicitly `/unpeek` them to conserve resources. +## Encrypted rooms + +(this para taken from MSC #2444): + +It is considered a feature that you cannot peek into encrypted rooms, given +the act of peeking would leak the identity of the peeker to the joined users +in the room (as they'd need to encrypt for the peeker). This also feels +acceptable given there is little point in encrypting something intended to be +world-readable. + ## Potential issues It could be seen as controversial to add another new block to the `/sync` From a80364d5b75cc6c44d0a393525c52014b641033c Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Mon, 16 Nov 2020 11:46:21 +0000 Subject: [PATCH 15/21] minor updates --- proposals/2753-peeking-via-sync-v2.md | 27 ++++++++++++++++++++------- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index f6bb9b9f966..4eb233f57ba 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -2,18 +2,31 @@ ## Problem -Peeking into rooms without joining them currently relies on the deprecated v1 /initialSync and /events APIs. +[Room previews](https://matrix.org/docs/spec/client_server/r0.6.1#id116), more +commonly known as peeking, has a number of usecases, such as: + + * Look at a room before joining it, to preview it. + * Look at a user's profile room (see + [MSC1769](https://github.com/matrix-org/matrix-doc/issues/1769)). + * Browse the metadata or membership of a space (see + [MSC1772](https://github.com/matrix-org/matrix-doc/issues/1772)). + * Monitor [moderation policy lists](https://matrix.org/docs/spec/client_server/r0.6.1#moderation-policy-lists). + +Currently, peeking relies on the deprecated v1 `/initialSync` and `/events` +APIs. This poses the following issues: - * Servers and clients must implement two separate sets of event-syncing logic, doubling complexity. + * Servers and clients must implement two separate sets of event-syncing logic, + doubling complexity. * Peeking a room involves setting a stream of long-lived /events requests going. Having multiple streams is racey, competes for resources with the /sync stream, and doesn't scale given each room requires a new /events stream. * v1 APIs are deprecated and not implemented on new servers. -This MSC likely obsoletes [MSC1776](https://github.com/matrix-org/matrix-doc/pull/1776). +This proposal suggests a new API in which events in peeked rooms would be +returned over `/sync`. ## Proposal @@ -21,7 +34,7 @@ We add an CS API called `/peek/{roomIdOrAlias}`, very similar to `/join/{roomIdO Calling `/peek`: * Resolves the given room alias to a room ID, if needed. - * Adds the room (if permissions allow) to a new section of the /sync response + * Adds the room (if permissions allow) to a new section of the `/sync` response called `peek` - but only for the device which called `/peek`. * The `peek` section is identical to `join`, but shows the live activity of rooms for which that device is peeking. @@ -50,9 +63,9 @@ down `/sync`. To stop peeking, the user calls `/unpeek` on the room, similar to `/leave`. This returns 200 on success, 404 on unrecognised ID, or 400 if the room was not -being peeked in the first place. Having stopped peeking, the empty room block -appears in the `leave` block of the next sync response to tell the client that -the user is no longer peeking. +being peeked in the first place. Having stopped peeking, the unpeeked room +will appear in the `leave` block of the next sync response to tell the client +that the user is no longer peeking. The new `/peek` and `/unpeek` endpoints require authentication and can be ratelimited. Their responses are analogous to their `/join` and `/leave` From 90eba26cc230b0722bd08d28d24295c55b22ed81 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Mon, 16 Nov 2020 12:16:32 +0000 Subject: [PATCH 16/21] initialsync does not clear peeks --- proposals/2753-peeking-via-sync-v2.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index 4eb233f57ba..0e3e47fb0de 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -48,10 +48,6 @@ possible to have an invite-only room which joe public can still peek into, if If the `history_visibility` of a room changes to not be `world_readable`, any peeks on the room are cancelled. -In order to clear up stale peeks if a client restarts without cleaning up -nicely, the act of calling `/sync` without a `since` token must cancel any peeks -created by that device. - Similar to `/join`, `/peek` lets you specify `server_name` querystring parameters to specify which server(s) to try to peek into the room via (when coupled with [MSC2444](https://github.com/matrix-org/matrix-doc/pull/2444)). @@ -132,7 +128,8 @@ alternative that empirically feels much simpler and tidier. ## Security considerations -Servers should ratelimit calls to `/peek` to stop someone DoSing the server. +Servers should ratelimit calls to the new endpoints to stop someone DoSing the +server. Servers may stop maintaining a `/peek` if its device has not `/sync`ed recently, and thus reclaim resources. At the next `/sync` the server would From 3b629ba524ac62b87077282b891a740654aea965 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Mon, 16 Nov 2020 15:18:18 +0000 Subject: [PATCH 17/21] lots of updates --- proposals/2753-peeking-via-sync-v2.md | 228 +++++++++++++++++++------- 1 file changed, 171 insertions(+), 57 deletions(-) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index 0e3e47fb0de..932aca6bf80 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -3,7 +3,7 @@ ## Problem [Room previews](https://matrix.org/docs/spec/client_server/r0.6.1#id116), more -commonly known as peeking, has a number of usecases, such as: +commonly known as "peeking", has a number of usecases, such as: * Look at a room before joining it, to preview it. * Look at a user's profile room (see @@ -19,57 +19,129 @@ This poses the following issues: * Servers and clients must implement two separate sets of event-syncing logic, doubling complexity. - * Peeking a room involves setting a stream of long-lived /events requests + * Peeking a room involves setting a stream of long-lived `/events` requests going. Having multiple streams is racey, competes for resources with the - /sync stream, and doesn't scale given each room requires a new /events + `/sync` stream, and doesn't scale given each room requires a new `/events` stream. - * v1 APIs are deprecated and not implemented on new servers. + * `/initialSync` and `/events` are deprecated and not implemented on new + servers. This proposal suggests a new API in which events in peeked rooms would be returned over `/sync`. ## Proposal -We add an CS API called `/peek/{roomIdOrAlias}`, very similar to `/join/{roomIdOrAlias}`. +Peeking into a room remains per-device: if the user has multiple devices, each +of which wants to peek into a given room, then each device must make a separate +request. -Calling `/peek`: - * Resolves the given room alias to a room ID, if needed. - * Adds the room (if permissions allow) to a new section of the `/sync` response - called `peek` - but only for the device which called `/peek`. - * The `peek` section is identical to `join`, but shows the live activity of - rooms for which that device is peeking. +To help avoid situations where clients create peek requests and forget about +them, each peek request is given a lifetime by the server. The client must +*renew* the peek request before this lifetime expires. The server is free to +pick any lifetime. -The API returns 404 on an unrecognised room ID or alias, or 403 if the room -does not allow peeking. Rooms allow peeking if they have `history_visibility` -of `world_readable`. N.B. `join_rules` do not affect peekability - it's -possible to have an invite-only room which joe public can still peek into, if -`history_visibility` has been set to `world_readable`. +### Starting a peek + +We add an CS API called `/peek_room`, which starts peeking into a given +room. This is similar to +[`/join/{roomIdOrAlias}`](https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-join-roomidoralias) +but has a slightly different API shape. + +For example: + +``` +POST /_matrix/client/r0/peek_room HTTP/1.1 + +{ + "room": "", + "servers": [ + "server1", "server2" + ] +} +``` + +A successful response has the following format: + +``` +{ + "room_id": "", + "peek_expiry_ts": 1605534210000 +} +``` + +The `room` parameter is required and must be a valid room id or alias. The +`servers` parameter is optional and, if present, gives a list of servers to try +to peek through. + +Both `room_id` and `peek_expiry_ts` are required in the +response. `peek_expiry_ts` gives a timestamp (milliseconds since the unix +epoch) when the server will *expire* the peek if the client does not renew it. + +The server ratelimit requests to `/peek_room` and returns a 429 error with +`M_LIMIT_EXCEEDED` if the limit is exceeded. + +Otherwise, the server first resolves the given room alias to a room ID, if +needed. + +If there is already an active peek for the room in question, it is renewed and +a successful response is returned with the updated `peek_expiry_ts`. + +If the user is already *joined* to the room in question, the server returns a +400 error with `M_BAD_STATE`. + +If the room in question does not exist, the server returns a 404 error with +`M_NOT_FOUND`. + +If the room does not allow peeking (ie, it does not have `history_visibility` +of `world_readable` [1](#f1)), the server returns a 403 +error with `M_FORBIDDEN`. + +Otherwise, the server starts a peek for the calling device into the given room, +and returns a 200 response as above. -If the `history_visibility` of a room changes to not be `world_readable`, any -peeks on the room are cancelled. +When a peek first starts, the current state of the room is returned in the +`peek` section of the next `/sync` response. -Similar to `/join`, `/peek` lets you specify `server_name` querystring -parameters to specify which server(s) to try to peek into the room via (when -coupled with [MSC2444](https://github.com/matrix-org/matrix-doc/pull/2444)). +### `/sync` response -If a user subsequently `/join`s the room they're peeking, we atomically move -the room to the `join` block of the `/sync` response, allowing the client to -build on the state and history it has already received without re-sending it -down `/sync`. +We add a new `peek` section to the `rooms` field of the `/sync` +response. `peek` has the same shape as `join`. -To stop peeking, the user calls `/unpeek` on the room, similar to `/leave`. -This returns 200 on success, 404 on unrecognised ID, or 400 if the room was not -being peeked in the first place. Having stopped peeking, the unpeeked room -will appear in the `leave` block of the next sync response to tell the client -that the user is no longer peeking. +While a peek into a given room is active, any new events in the room cause that +room to be included in the `peek` section (but only for the device with the +active peek). When a peek first starts, the entire state of the room is +included in the same way as when a room is first joined. -The new `/peek` and `/unpeek` endpoints require authentication and can be -ratelimited. Their responses are analogous to their `/join` and `/leave` -counterparts (eg: `room_id` in the response and empty object when stopping). +If the client requests lazy-loading via `lazy_load_members`, then redundant +membership events are excluded in the same way as they are for joined rooms. -Clients should check for any irrelevant peeked rooms on launch (left over from -previous instances of the app) and explicitly `/unpeek` them to conserve -resources. +If a user subsequently `/join`s a room they are peeking, the room will +thenceforth appear in the `join` section instead of `peek`. For devices which +were already peeking into the room, the server should *not* include the entire +room state for the room in the `/sync` response, allowing the client to build +on the state and history it has already received without re-sending it down +`/sync`. + +### Stopping a peek + +To stop peeking, the client calls `rooms//unpeek`: + +``` +POST /_matrix/client/r0/rooms/{room_id}/unpeek HTTP/1.1 + +{} +``` + +The body must be a JSON dictionary, but no parameters are specified. + +A successful response has an empty body. + +If the room is unknown or was not previously being peeked the server returns a +400 error with `M_BAD_STATE`. + +Having stopped peeking, the unpeeked room will appear in the `leave` block of +the next sync response to tell the client that the user is no longer +peeking. XXX: why do we do this? Will it be empty? ## Encrypted rooms @@ -83,6 +155,33 @@ world-readable. ## Potential issues + * Expiring peeks might be hard for clients to manage? + +## Alternatives + +### Keep /peek_room closer to /join + +Given that peeking has parallels to joining, it might be preferable to keep the +API closer to `/join`. On the other hand, they probabably aren't actually +similar enough to make it worth propagating the oddities of `/join` (in +particular, the use of query-parameters +([matrix-doc#2864](https://github.com/matrix-org/matrix-doc/issues/2864)). + +### Filter-based API + +[MSC1776](https://github.com/matrix-org/matrix-doc/pull/1776) defined an +alternative approach, where you could use filters to add peeked rooms into a +given `/sync` response as needed. This however had some issues: + + * You can't specify what servers to peek remote rooms via. + * You can't identify rooms via alias, only ID + * It feels hacky to bodge peeked rooms into the `join` block of a given + `/sync` response + * Fiddling around with custom filters feels clunky relative to just calling a + `/peek` endpoint similar to `/join`. + +### Use the `join` block + It could be seen as controversial to add another new block to the `/sync` response. We could use the existing `join` block, but: @@ -93,6 +192,8 @@ response. We could use the existing `join` block, but: * there's already a precedent for per-device blocks in the sync response (for to-device messages) +### Per-account peeks + It could be seen as controversial to make peeking a per-device rather than per-user feature. When thinking through use cases for peeking, however: @@ -110,40 +211,53 @@ per-user feature. When thinking through use cases for peeking, however: matter of effectively opting in rather than trying to filter out peeks you don't care about). -## Alternatives +### Alternatives to expiring peeks -[MSC1776](https://github.com/matrix-org/matrix-doc/pull/1776) defined an -alternative approach, where you could use filters to add peeked rooms into a -given `/sync` response as needed. This however had some issues: +Having servers expire peek requests could be fiddly, so we considered a number +of alternatives: - * You can't specify what servers to peek remote rooms via. - * You can't identify rooms via alias, only ID - * It feels hacky to bodge peeked rooms into the `join` block of a given - `/sync` response - * Fiddling around with custom filters feels clunky relative to just calling a - `/peek` endpoint similar to `/join`. + * Allow peeks to stack up without limit and trust that clients will not forget + about them: after all, it is in clients' best interest not to leak + resources, to reduce the amount of data to be handled, and it is not obvious + that leaking peeks is easier than leaking joins. + + Ultimately this does not align with our experience of administering + `matrix.org`: it seems that where a resource *can* be leaked, it ultimately + will be, and it is better to design the API to prevent it. -While experimenting with implementing MSC1776, I came up with this as an -alternative that empirically feels much simpler and tidier. + * Limit the number of peeks that can be active at once, to force clients to be + fastidious in their peek cleanups. However, it is hard to see what a good + limit would be. Furthermore: peeks could be lost through no fault of the + client (for example: when a `/peek_room` request succeeds but the client + does not receive the response), and these leaked peaks could stack up until + peeking becomes inoperative. + + * Automatically clear active peeks when a `/sync` request is made without a + `since` parameter. However, this feels like magic at a distance, and also + means that if you initial-sync separately (e.g. you stole an access token + from the DB to manually debug something) then existing clients will be + broken. + + * Have the client resubmit the list of active peeks every time it wants to add + or remove one. This could amount to a sigificant quantity of data. ## Security considerations -Servers should ratelimit calls to the new endpoints to stop someone DoSing the +Servers should ratelimit calls to `/peek_room` to stop someone DoSing the server. -Servers may stop maintaining a `/peek` if its device has not `/sync`ed -recently, and thus reclaim resources. At the next `/sync` the server would -need to restore the `peek` and provide a gappy update. This is most relevant -in the context of peeking into a remote room via -[MSC2444](https://github.com/matrix-org/matrix-doc/pull/2444) however. - ## Unstable prefix - The following mapping will be used for identifiers in this MSC during development: Proposed final identifier | Purpose | Development identifier ------------------------------- | ------- | ---- -`/_matrix/client/r0/peek` | API endpoint | `/_matrix/client/unstable/org.matrix.msc2753/peek` +`/_matrix/client/r0/peek_room` | API endpoint | `/_matrix/client/unstable/org.matrix.msc2753/peek_room` `/_matrix/client/r0/rooms/{roomId}/unpeek` | API endpoint | `/_matrix/client/unstable/org.matrix.msc2753/rooms/{roomId}/unpeek` + +## Footnotes + +[1]: `join_rules` do not affect peekability - it's +possible to have an invite-only room which joe public can still peek into, if +`history_visibility` has been set to `world_readable`.[↩](#a1) From 405c05d7d7dd629c91d4b06d1ed4c2e1f6996351 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Mon, 16 Nov 2020 16:17:44 +0000 Subject: [PATCH 18/21] ore updates --- proposals/2753-peeking-via-sync-v2.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index 932aca6bf80..1acd269a1df 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -73,6 +73,12 @@ The `room` parameter is required and must be a valid room id or alias. The `servers` parameter is optional and, if present, gives a list of servers to try to peek through. +XXX: should we limit this API to room IDs, and require clients to do a `GET +/_matrix/client/r0/directory/room/{roomAlias}` request if they have a room +alias? (In which case, `/_matrix/client/r0/room/{room_id}/peek` might be a +better name for it.) On the one hand: cleaner, simpler API. On the other: more +requests needed for each operation. + Both `room_id` and `peek_expiry_ts` are required in the response. `peek_expiry_ts` gives a timestamp (milliseconds since the unix epoch) when the server will *expire* the peek if the client does not renew it. @@ -153,6 +159,18 @@ in the room (as they'd need to encrypt for the peeker). This also feels acceptable given there is little point in encrypting something intended to be world-readable. +## Future extensions + + * "snapshot" API, for a one-time peek operation which returns the current + state of the room without adding the room to future `/sync` responses. Might + be useful for certain usecases (eg, looking at a user's public profile)? + + * "bulk peek" API, for peeking into many rooms at once. Might be useful for + flair (which requires peeking into lots of users' profile rooms), though + realistically that usecase will need server-side support. + + * "cross-device" peeks could be useful for microblogging etc? + ## Potential issues * Expiring peeks might be hard for clients to manage? From 12740e5ac3a03eebf0db970c453067026d4805c6 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Mon, 16 Nov 2020 18:20:19 +0000 Subject: [PATCH 19/21] unpeeked rooms go in the leave section of /sync --- proposals/2753-peeking-via-sync-v2.md | 56 +++++++++++++++++++-------- 1 file changed, 39 insertions(+), 17 deletions(-) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index 1acd269a1df..9ad53521cdf 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -108,6 +108,23 @@ and returns a 200 response as above. When a peek first starts, the current state of the room is returned in the `peek` section of the next `/sync` response. +### Stopping a peek + +To stop peeking, the client calls `rooms//unpeek`: + +``` +POST /_matrix/client/r0/rooms/{room_id}/unpeek HTTP/1.1 + +{} +``` + +The body must be a JSON dictionary, but no parameters are specified. + +A successful response has an empty body. + +If the room is unknown or was not previously being peeked the server returns a +400 error with `M_BAD_STATE`. + ### `/sync` response We add a new `peek` section to the `rooms` field of the `/sync` @@ -128,27 +145,32 @@ room state for the room in the `/sync` response, allowing the client to build on the state and history it has already received without re-sending it down `/sync`. -### Stopping a peek - -To stop peeking, the client calls `rooms//unpeek`: +When a room stops being peeked (either because the client called `/unpeek` or +because the server timed out the peek), the room will be included in the +`leave` section of the `/sync` response, including any events that occured +between the previous `/sync` and the the peek ending. If there are no such +events, the room's entry in the `leave` section will be empty. -``` -POST /_matrix/client/r0/rooms/{room_id}/unpeek HTTP/1.1 +For example: -{} +```js +{ + "rooms": { + "join": { /* ... */ }, + "leave": { + "!unpeeked:example.org": { + "timeline": { + "events": [ + { "type": "m.room.message", "content": {"body": "just one more thing"}} + ] + } + }, + "!alsounpeeked:example.com": {} + } + } +} ``` -The body must be a JSON dictionary, but no parameters are specified. - -A successful response has an empty body. - -If the room is unknown or was not previously being peeked the server returns a -400 error with `M_BAD_STATE`. - -Having stopped peeking, the unpeeked room will appear in the `leave` block of -the next sync response to tell the client that the user is no longer -peeking. XXX: why do we do this? Will it be empty? - ## Encrypted rooms (this para taken from MSC #2444): From c75784991d9accd1cdac0db7db20675d00450808 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 17 Nov 2020 11:31:27 +0000 Subject: [PATCH 20/21] Drop `_room` suffix Matthew wants us to drop the `_room` suffix. I worry that this is somewhat confusing (it's not obvious to the casual observer what it is that we are peeking), and risks a lot of confusion in the future when we find something else to peek, but he seems to feel strongly. --- proposals/2753-peeking-via-sync-v2.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index 9ad53521cdf..5a34d1f0e0f 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -42,7 +42,7 @@ pick any lifetime. ### Starting a peek -We add an CS API called `/peek_room`, which starts peeking into a given +We add an CS API called `/peek`, which starts peeking into a given room. This is similar to [`/join/{roomIdOrAlias}`](https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-join-roomidoralias) but has a slightly different API shape. @@ -50,7 +50,7 @@ but has a slightly different API shape. For example: ``` -POST /_matrix/client/r0/peek_room HTTP/1.1 +POST /_matrix/client/r0/peek HTTP/1.1 { "room": "", @@ -83,7 +83,7 @@ Both `room_id` and `peek_expiry_ts` are required in the response. `peek_expiry_ts` gives a timestamp (milliseconds since the unix epoch) when the server will *expire* the peek if the client does not renew it. -The server ratelimit requests to `/peek_room` and returns a 429 error with +The server ratelimit requests to `/peek` and returns a 429 error with `M_LIMIT_EXCEEDED` if the limit is exceeded. Otherwise, the server first resolves the given room alias to a room ID, if @@ -199,7 +199,7 @@ world-readable. ## Alternatives -### Keep /peek_room closer to /join +### Keep /peek closer to /join Given that peeking has parallels to joining, it might be preferable to keep the API closer to `/join`. On the other hand, they probabably aren't actually @@ -268,7 +268,7 @@ of alternatives: * Limit the number of peeks that can be active at once, to force clients to be fastidious in their peek cleanups. However, it is hard to see what a good limit would be. Furthermore: peeks could be lost through no fault of the - client (for example: when a `/peek_room` request succeeds but the client + client (for example: when a `/peek` request succeeds but the client does not receive the response), and these leaked peaks could stack up until peeking becomes inoperative. @@ -283,7 +283,7 @@ of alternatives: ## Security considerations -Servers should ratelimit calls to `/peek_room` to stop someone DoSing the +Servers should ratelimit calls to `/peek` to stop someone DoSing the server. ## Unstable prefix @@ -293,7 +293,7 @@ development: Proposed final identifier | Purpose | Development identifier ------------------------------- | ------- | ---- -`/_matrix/client/r0/peek_room` | API endpoint | `/_matrix/client/unstable/org.matrix.msc2753/peek_room` +`/_matrix/client/r0/peek` | API endpoint | `/_matrix/client/unstable/org.matrix.msc2753/peek` `/_matrix/client/r0/rooms/{roomId}/unpeek` | API endpoint | `/_matrix/client/unstable/org.matrix.msc2753/rooms/{roomId}/unpeek` ## Footnotes From 50537aa3c33f05bed6d86d101b964913a5b4f227 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Thu, 26 Nov 2020 17:05:08 +0000 Subject: [PATCH 21/21] for /peek, put room id in the uri --- proposals/2753-peeking-via-sync-v2.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/proposals/2753-peeking-via-sync-v2.md b/proposals/2753-peeking-via-sync-v2.md index 5a34d1f0e0f..763cf88a44d 100644 --- a/proposals/2753-peeking-via-sync-v2.md +++ b/proposals/2753-peeking-via-sync-v2.md @@ -50,10 +50,9 @@ but has a slightly different API shape. For example: ``` -POST /_matrix/client/r0/peek HTTP/1.1 +POST /_matrix/client/r0/peek/{roomIdOrAlias} HTTP/1.1 { - "room": "", "servers": [ "server1", "server2" ] @@ -69,9 +68,8 @@ A successful response has the following format: } ``` -The `room` parameter is required and must be a valid room id or alias. The -`servers` parameter is optional and, if present, gives a list of servers to try -to peek through. +The `servers` parameter is optional and, if present, gives a list of servers to +try to peek through. XXX: should we limit this API to room IDs, and require clients to do a `GET /_matrix/client/r0/directory/room/{roomAlias}` request if they have a room