Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Remove functionality associated with unused historical stats tables #9721

Merged
merged 6 commits into from
Jul 8, 2021
Merged

Remove functionality associated with unused historical stats tables #9721

merged 6 commits into from
Jul 8, 2021

Conversation

xmunoz
Copy link
Contributor

@xmunoz xmunoz commented Mar 31, 2021 •
edited by richvdh
Loading

  • Purge all associated functionality
  • Update the docs
  • Get unit tests passing

Pull Request Checklist

  • Pull request is based on the develop branch
  • Pull request includes a changelog file. The entry should:
    • Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
    • Use markdown where necessary, mostly for code blocks.
    • End with either a period (.) or an exclamation mark (!).
    • Start with a capital letter.
  • Pull request includes a sign off
  • Code style is correct (run the linters)

Fixes #9602

@xmunoz xmunoz force-pushed the remove_historical_stats branch 3 times, most recently from fbbe0e6 to 16be4ae Compare March 31, 2021 21:50
@richvdh richvdh requested a review from a team April 1, 2021 08:58
@richvdh richvdh changed the title Fixes #9602: Remove unused historical stats tables Remove unused historical stats tables Apr 1, 2021
richvdh
richvdh reviewed Apr 6, 2021
View reviewed changes
Copy link
Member

@richvdh richvdh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @xmunoz, thank you very much for doing all this work!

I haven't yet read through it in detail, though at a skim it looks like it makes all the right sort of changes.

However, we need to be a bit careful here: if we simply drop the *_historical tables from the database, that means that it will be difficult to roll Synapse back to an earlier version. Generally we try to avoid that sort of situation, because it means that if there is a bug in the next version of Synapse, our users are a bit stuck.

So, one solution here is to just remove the code that updates the tables, but leave the tables in place for now.

(I've been thinking a bit recently about how to manage this situation: https://hackmd.io/@richvdh/BJYXQMQHO has some more details)

@xmunoz
Copy link
Contributor Author

xmunoz commented Apr 6, 2021 •
edited
Loading

Ok, that makes sense. Is there a suggested PR procedure for actioning this kind of change? It sounds like you'd like me to split this PR into 3 PRs, each with a number indicating the order that they should be merged. Is there a way to tag PRs with a version number, so that you will know when it needs to get merged? Are there any examples of changes that have been merged in this way? If not, that's fine too. I'm happy to be the guinea pig for this.

@richvdh
Copy link
Member

richvdh commented Apr 6, 2021

since there isn't any code that reads from the _historical tables currently (as far as I know?), I think we only need two PRs here: one to remove the code that writes to the tables, and later, another that finally removes the tables.

Generally what we do is just write the first PR, and then file a maintenance issue to remind us to do the next step. (The Z-Future-Maintenance issue label is meant for that sort of thing, though we haven't yet used it for much.)

@xmunoz xmunoz changed the title Remove unused historical stats tables Remove functionality associated with unused historical stats tables Apr 6, 2021
@xmunoz
Copy link
Contributor Author

xmunoz commented Apr 6, 2021

Build is failing now, but I'm not sure why. I simply added a commit to remove the sql schema change and update the changelog.

@richvdh
Copy link
Member

richvdh commented Apr 6, 2021

oh, the build is failing because flake8-bugbear has had a new release. Can you merge develop into your branch, so that you pick up #9734 ?

@xmunoz xmunoz requested a review from richvdh April 6, 2021 17:36
richvdh
richvdh reviewed Apr 8, 2021
View reviewed changes
Copy link
Member

@richvdh richvdh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is amazing, thank you so much!

The only thing is that I'd prefer to hold off merging it until we've done the groundwork for schema versions outlined in https://hackmd.io/@richvdh/BJYXQMQHO. If you fancy taking that on (in a separate PR) that would be amazing - otherwise hopefully we'll get to it within the next week or so.

changelog.d/9721.removal Outdated Show resolved Hide resolved
docs/room_and_user_statistics.md Outdated Show resolved Hide resolved
@xmunoz
Copy link
Contributor Author

xmunoz commented Apr 9, 2021

The only thing is that I'd prefer to hold off merging it until we've done the groundwork for schema versions outlined in https://hackmd.io/@richvdh/BJYXQMQHO. If you fancy taking that on (in a separate PR) that would be amazing - otherwise hopefully we'll get to it within the next week or so.

I'm not sure what this groundwork is or what it entails, so I'll leave it up to you.

@anoadragon453
Copy link
Member

I believe this is blocked on #9933.

@richvdh
Copy link
Member

richvdh commented Jun 11, 2021

#9933 is now landed, so this is unblocked. We should bump SCHEMA_VERSION, per the example in https://matrix-org.github.io/synapse/develop/development/database_schema.html#synapse-schema-versions.

@richvdh richvdh self-assigned this Jun 11, 2021
@richvdh
Copy link
Member

richvdh commented Jun 11, 2021

That said: I'd like us to have at least one release of Synapse which checks schema_compat_version rather than schema_version before we bump schema_version, otherwise we'll break downgrade.

In other words: Synapse 1.37 will introduce support for schema_compat_version. We can land this PR in Synapse 1.38 along with a bump of schema_version, and it will still be possible to downgrade from 1.38 to 1.37.

@erikjohnston erikjohnston added the z-blocked (Deprecated Label) label Jun 16, 2021
@richvdh richvdh mentioned this pull request Jun 29, 2021
Closed
@richvdh richvdh removed the z-blocked (Deprecated Label) label Jun 29, 2021
xmunoz and others added 5 commits July 8, 2021 16:07
@xmunoz @reivilibre
Fixes matrix-org#9602: Remove unused historical stats tables
832ffae 
- Purge all associated functionality
- Update the docs
- Get unit tests passing
- Newsfile

Signed-off-by: Cristina Muñoz <hi@xmunoz.com>
@xmunoz @reivilibre
Don't remove the historical stats table quite yet.
540ed48 
Do this in a follow-up PR for a subsequent release.

Signed-off-by: Cristina Muñoz <hi@xmunoz.com>
@xmunoz @reivilibre
Wrap line in docs at 80 characters
ab71ba5
@xmunoz @richvdh @reivilibre
Update changelog.d/9721.removal
570c341 
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
@reivilibre
Increment the SCHEMA_VERSION.
f988e1c
@reivilibre reivilibre requested a review from a team July 8, 2021 15:22
richvdh
richvdh approved these changes Jul 8, 2021
View reviewed changes
Copy link
Member

@richvdh richvdh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@richvdh richvdh enabled auto-merge (squash) July 8, 2021 15:29
@richvdh richvdh mentioned this pull request Jul 8, 2021
Closed
@richvdh richvdh merged commit f6767ab into matrix-org:develop Jul 8, 2021
anoadragon453 added a commit that referenced this pull request Jul 20, 2021
@anoadragon453
Merge tag 'v1.39.0rc1' into develop
b181dc4 
Synapse 1.39.0rc1 (2021-07-20)
==============================

The Third-Party Event Rules module interface has been deprecated in favour of the generic module interface introduced in Synapse v1.37.0. Support for the old interface is planned to be removed in September 2021. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information.

Features
--------

- Add the ability to override the account validity feature with a module. ([\#9884](#9884))
- The spaces summary API now returns any joinable rooms, not only rooms which are world-readable. ([\#10298](#10298), [\#10305](#10305))
- Add a new version of the R30 phone-home metric, which removes a false impression of retention given by the old R30 metric. ([\#10332](#10332), [\#10427](#10427))
- Allow providing credentials to `http_proxy`. ([\#10360](#10360))

Bugfixes
--------

- Fix error while dropping locks on shutdown. Introduced in v1.38.0. ([\#10433](#10433))
- Add base starting insertion event when no chunk ID is specified in the historical batch send API. ([\#10250](#10250))
- Fix historical batch send endpoint (MSC2716) rejecting batches with messages from multiple senders. ([\#10276](#10276))
- Fix purging rooms that other homeservers are still sending events for. Contributed by @ilmari. ([\#10317](#10317))
- Fix errors during backfill caused by previously purged redaction events. Contributed by Andreas Rammhold (@andir). ([\#10343](#10343))
- Fix the user directory becoming broken (and noisy errors being logged) when knocking and room statistics are in use. ([\#10344](#10344))
- Fix newly added `synapse_federation_server_oldest_inbound_pdu_in_staging` prometheus metric to measure age rather than timestamp. ([\#10355](#10355))
- Fix PostgreSQL sometimes using table scans for queries against `state_groups_state` table, taking a long time and a large amount of IO. ([\#10359](#10359))
- Fix `make_room_admin` failing for users that have left a private room. ([\#10367](#10367))
- Fix a number of logged errors caused by remote servers being down. ([\#10400](#10400), [\#10414](#10414))
- Responses from `/make_{join,leave,knock}` no longer include signatures, which will turn out to be invalid after events are returned to `/send_{join,leave,knock}`. ([\#10404](#10404))

Improved Documentation
----------------------

- Updated installation dependencies for newer macOS versions and ARM Macs. Contributed by Luke Walsh. ([\#9971](#9971))
- Simplify structure of room admin API. ([\#10313](#10313))
- Refresh the logcontext dev documentation. ([\#10353](#10353)), ([\#10337](#10337))
- Add delegation example for caddy in the reverse proxy documentation. Contributed by @moritzdietz. ([\#10368](#10368))
- Fix and clarify some links in `docs` and `contrib`. ([\#10370](#10370)), ([\#10322](#10322)), ([\#10399](#10399))
- Make deprecation notice of the spam checker doc more obvious. ([\#10395](#10395))
- Add instructions on installing Debian packages for release candidates. ([\#10396](#10396))

Deprecations and Removals
-------------------------

- Remove functionality associated with the unused `room_stats_historical` and `user_stats_historical` tables. Contributed by @xmunoz. ([\#9721](#9721))
- The third-party event rules module interface is deprecated in favour of the generic module interface introduced in Synapse v1.37.0. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information. ([\#10386](#10386))

Internal Changes
----------------

- Convert `room_depth.min_depth` column to a `BIGINT`. ([\#10289](#10289))
- Add tests to characterise the current behaviour of R30 phone-home metrics. ([\#10315](#10315))
- Rebuild event context and auth when processing specific results from `ThirdPartyEventRules` modules. ([\#10316](#10316))
- Minor change to the code that populates `user_daily_visits`. ([\#10324](#10324))
- Re-enable Sytests that were disabled for the 1.37.1 release. ([\#10345](#10345), [\#10357](#10357))
- Run `pyupgrade` on the codebase. ([\#10347](#10347), [\#10348](#10348))
- Switch `application_services_txns.txn_id` database column to `BIGINT`. ([\#10349](#10349))
- Convert internal type variable syntax to reflect wider ecosystem use. ([\#10350](#10350), [\#10380](#10380), [\#10381](#10381), [\#10382](#10382), [\#10418](#10418))
- Make the Github Actions workflow configuration more efficient. ([\#10383](#10383))
- Add type hints to `get_{domain,localpart}_from_id`. ([\#10385](#10385))
- When building Debian packages for prerelease versions, set the Section accordingly. ([\#10391](#10391))
- Add type hints and comments to event auth code. ([\#10393](#10393))
- Stagger sending of presence update to remote servers, reducing CPU spikes caused by starting many connections to remote servers at once. ([\#10398](#10398))
- Remove unused `events_by_room` code (tech debt). ([\#10421](#10421))
- Add a github actions job which records success of other jobs. ([\#10430](#10430))
erikjohnston added a commit that referenced this pull request Jul 29, 2021
@erikjohnston
Merge tag 'v1.39.0'
a6e2c16 
Synapse 1.39.0 (2021-07-29)
===========================

No significant changes.

Synapse 1.39.0rc3 (2021-07-28)
==============================

Bugfixes
--------

- Fix a bug introduced in Synapse 1.38 which caused an exception at startup when SAML authentication was enabled. ([\#10477](#10477))
- Fix a long-standing bug where Synapse would not inform clients that a device had exhausted its one-time-key pool, potentially causing problems decrypting events. ([\#10485](#10485))
- Fix reporting old R30 stats as R30v2 stats. Introduced in v1.39.0rc1. ([\#10486](#10486))

Internal Changes
----------------

- Fix an error which prevented the Github Actions workflow to build the docker images from running. ([\#10461](#10461))
- Fix release script to correctly version debian changelog when doing RCs. ([\#10465](#10465))

Synapse 1.39.0rc2 (2021-07-22)
==============================

This release also includes the changes in v1.38.1.

Internal Changes
----------------

- Move docker image build to Github Actions. ([\#10416](#10416))

Synapse 1.39.0rc1 (2021-07-20)
==============================

The Third-Party Event Rules module interface has been deprecated in favour of the generic module interface introduced in Synapse v1.37.0. Support for the old interface is planned to be removed in September 2021. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information.

Features
--------

- Add the ability to override the account validity feature with a module. ([\#9884](#9884))
- The spaces summary API now returns any joinable rooms, not only rooms which are world-readable. ([\#10298](#10298), [\#10305](#10305))
- Add a new version of the R30 phone-home metric, which removes a false impression of retention given by the old R30 metric. ([\#10332](#10332), [\#10427](#10427))
- Allow providing credentials to `http_proxy`. ([\#10360](#10360))

Bugfixes
--------

- Fix error while dropping locks on shutdown. Introduced in v1.38.0. ([\#10433](#10433))
- Add base starting insertion event when no chunk ID is specified in the historical batch send API. ([\#10250](#10250))
- Fix historical batch send endpoint (MSC2716) rejecting batches with messages from multiple senders. ([\#10276](#10276))
- Fix purging rooms that other homeservers are still sending events for. Contributed by @ilmari. ([\#10317](#10317))
- Fix errors during backfill caused by previously purged redaction events. Contributed by Andreas Rammhold (@andir). ([\#10343](#10343))
- Fix the user directory becoming broken (and noisy errors being logged) when knocking and room statistics are in use. ([\#10344](#10344))
- Fix newly added `synapse_federation_server_oldest_inbound_pdu_in_staging` prometheus metric to measure age rather than timestamp. ([\#10355](#10355))
- Fix PostgreSQL sometimes using table scans for queries against `state_groups_state` table, taking a long time and a large amount of IO. ([\#10359](#10359))
- Fix `make_room_admin` failing for users that have left a private room. ([\#10367](#10367))
- Fix a number of logged errors caused by remote servers being down. ([\#10400](#10400), [\#10414](#10414))
- Responses from `/make_{join,leave,knock}` no longer include signatures, which will turn out to be invalid after events are returned to `/send_{join,leave,knock}`. ([\#10404](#10404))

Improved Documentation
----------------------

- Updated installation dependencies for newer macOS versions and ARM Macs. Contributed by Luke Walsh. ([\#9971](#9971))
- Simplify structure of room admin API. ([\#10313](#10313))
- Refresh the logcontext dev documentation. ([\#10353](#10353)), ([\#10337](#10337))
- Add delegation example for caddy in the reverse proxy documentation. Contributed by @moritzdietz. ([\#10368](#10368))
- Fix and clarify some links in `docs` and `contrib`. ([\#10370](#10370)), ([\#10322](#10322)), ([\#10399](#10399))
- Make deprecation notice of the spam checker doc more obvious. ([\#10395](#10395))
- Add instructions on installing Debian packages for release candidates. ([\#10396](#10396))

Deprecations and Removals
-------------------------

- Remove functionality associated with the unused `room_stats_historical` and `user_stats_historical` tables. Contributed by @xmunoz. ([\#9721](#9721))
- The third-party event rules module interface is deprecated in favour of the generic module interface introduced in Synapse v1.37.0. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information. ([\#10386](#10386))

Internal Changes
----------------

- Convert `room_depth.min_depth` column to a `BIGINT`. ([\#10289](#10289))
- Add tests to characterise the current behaviour of R30 phone-home metrics. ([\#10315](#10315))
- Rebuild event context and auth when processing specific results from `ThirdPartyEventRules` modules. ([\#10316](#10316))
- Minor change to the code that populates `user_daily_visits`. ([\#10324](#10324))
- Re-enable Sytests that were disabled for the 1.37.1 release. ([\#10345](#10345), [\#10357](#10357))
- Run `pyupgrade` on the codebase. ([\#10347](#10347), [\#10348](#10348))
- Switch `application_services_txns.txn_id` database column to `BIGINT`. ([\#10349](#10349))
- Convert internal type variable syntax to reflect wider ecosystem use. ([\#10350](#10350), [\#10380](#10380), [\#10381](#10381), [\#10382](#10382), [\#10418](#10418))
- Make the Github Actions workflow configuration more efficient. ([\#10383](#10383))
- Add type hints to `get_{domain,localpart}_from_id`. ([\#10385](#10385))
- When building Debian packages for prerelease versions, set the Section accordingly. ([\#10391](#10391))
- Add type hints and comments to event auth code. ([\#10393](#10393))
- Stagger sending of presence update to remote servers, reducing CPU spikes caused by starting many connections to remote servers at once. ([\#10398](#10398))
- Remove unused `events_by_room` code (tech debt). ([\#10421](#10421))
- Add a github actions job which records success of other jobs. ([\#10430](#10430))
aaronraimist added a commit to aaronraimist/synapse that referenced this pull request Jul 29, 2021
@aaronraimist
Merge tag 'v1.39.0' into 2021-05
bd221ce 
Synapse 1.39.0 (2021-07-29)
===========================

No significant changes.

Synapse 1.39.0rc3 (2021-07-28)
==============================

Bugfixes
--------

- Fix a bug introduced in Synapse 1.38 which caused an exception at startup when SAML authentication was enabled. ([\matrix-org#10477](matrix-org#10477))
- Fix a long-standing bug where Synapse would not inform clients that a device had exhausted its one-time-key pool, potentially causing problems decrypting events. ([\matrix-org#10485](matrix-org#10485))
- Fix reporting old R30 stats as R30v2 stats. Introduced in v1.39.0rc1. ([\matrix-org#10486](matrix-org#10486))

Internal Changes
----------------

- Fix an error which prevented the Github Actions workflow to build the docker images from running. ([\matrix-org#10461](matrix-org#10461))
- Fix release script to correctly version debian changelog when doing RCs. ([\matrix-org#10465](matrix-org#10465))

Synapse 1.39.0rc2 (2021-07-22)
==============================

This release also includes the changes in v1.38.1.

Internal Changes
----------------

- Move docker image build to Github Actions. ([\matrix-org#10416](matrix-org#10416))

Synapse 1.39.0rc1 (2021-07-20)
==============================

The Third-Party Event Rules module interface has been deprecated in favour of the generic module interface introduced in Synapse v1.37.0. Support for the old interface is planned to be removed in September 2021. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information.

Features
--------

- Add the ability to override the account validity feature with a module. ([\matrix-org#9884](matrix-org#9884))
- The spaces summary API now returns any joinable rooms, not only rooms which are world-readable. ([\matrix-org#10298](matrix-org#10298), [\matrix-org#10305](matrix-org#10305))
- Add a new version of the R30 phone-home metric, which removes a false impression of retention given by the old R30 metric. ([\matrix-org#10332](matrix-org#10332), [\matrix-org#10427](matrix-org#10427))
- Allow providing credentials to `http_proxy`. ([\matrix-org#10360](matrix-org#10360))

Bugfixes
--------

- Fix error while dropping locks on shutdown. Introduced in v1.38.0. ([\matrix-org#10433](matrix-org#10433))
- Add base starting insertion event when no chunk ID is specified in the historical batch send API. ([\matrix-org#10250](matrix-org#10250))
- Fix historical batch send endpoint (MSC2716) rejecting batches with messages from multiple senders. ([\matrix-org#10276](matrix-org#10276))
- Fix purging rooms that other homeservers are still sending events for. Contributed by @ilmari. ([\matrix-org#10317](matrix-org#10317))
- Fix errors during backfill caused by previously purged redaction events. Contributed by Andreas Rammhold (@andir). ([\matrix-org#10343](matrix-org#10343))
- Fix the user directory becoming broken (and noisy errors being logged) when knocking and room statistics are in use. ([\matrix-org#10344](matrix-org#10344))
- Fix newly added `synapse_federation_server_oldest_inbound_pdu_in_staging` prometheus metric to measure age rather than timestamp. ([\matrix-org#10355](matrix-org#10355))
- Fix PostgreSQL sometimes using table scans for queries against `state_groups_state` table, taking a long time and a large amount of IO. ([\matrix-org#10359](matrix-org#10359))
- Fix `make_room_admin` failing for users that have left a private room. ([\matrix-org#10367](matrix-org#10367))
- Fix a number of logged errors caused by remote servers being down. ([\matrix-org#10400](matrix-org#10400), [\matrix-org#10414](matrix-org#10414))
- Responses from `/make_{join,leave,knock}` no longer include signatures, which will turn out to be invalid after events are returned to `/send_{join,leave,knock}`. ([\matrix-org#10404](matrix-org#10404))

Improved Documentation
----------------------

- Updated installation dependencies for newer macOS versions and ARM Macs. Contributed by Luke Walsh. ([\matrix-org#9971](matrix-org#9971))
- Simplify structure of room admin API. ([\matrix-org#10313](matrix-org#10313))
- Refresh the logcontext dev documentation. ([\matrix-org#10353](matrix-org#10353)), ([\matrix-org#10337](matrix-org#10337))
- Add delegation example for caddy in the reverse proxy documentation. Contributed by @moritzdietz. ([\matrix-org#10368](matrix-org#10368))
- Fix and clarify some links in `docs` and `contrib`. ([\matrix-org#10370](matrix-org#10370)), ([\matrix-org#10322](matrix-org#10322)), ([\matrix-org#10399](matrix-org#10399))
- Make deprecation notice of the spam checker doc more obvious. ([\matrix-org#10395](matrix-org#10395))
- Add instructions on installing Debian packages for release candidates. ([\matrix-org#10396](matrix-org#10396))

Deprecations and Removals
-------------------------

- Remove functionality associated with the unused `room_stats_historical` and `user_stats_historical` tables. Contributed by @xmunoz. ([\matrix-org#9721](matrix-org#9721))
- The third-party event rules module interface is deprecated in favour of the generic module interface introduced in Synapse v1.37.0. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information. ([\matrix-org#10386](matrix-org#10386))

Internal Changes
----------------

- Convert `room_depth.min_depth` column to a `BIGINT`. ([\matrix-org#10289](matrix-org#10289))
- Add tests to characterise the current behaviour of R30 phone-home metrics. ([\matrix-org#10315](matrix-org#10315))
- Rebuild event context and auth when processing specific results from `ThirdPartyEventRules` modules. ([\matrix-org#10316](matrix-org#10316))
- Minor change to the code that populates `user_daily_visits`. ([\matrix-org#10324](matrix-org#10324))
- Re-enable Sytests that were disabled for the 1.37.1 release. ([\matrix-org#10345](matrix-org#10345), [\matrix-org#10357](matrix-org#10357))
- Run `pyupgrade` on the codebase. ([\matrix-org#10347](matrix-org#10347), [\matrix-org#10348](matrix-org#10348))
- Switch `application_services_txns.txn_id` database column to `BIGINT`. ([\matrix-org#10349](matrix-org#10349))
- Convert internal type variable syntax to reflect wider ecosystem use. ([\matrix-org#10350](matrix-org#10350), [\matrix-org#10380](matrix-org#10380), [\matrix-org#10381](matrix-org#10381), [\matrix-org#10382](matrix-org#10382), [\matrix-org#10418](matrix-org#10418))
- Make the Github Actions workflow configuration more efficient. ([\matrix-org#10383](matrix-org#10383))
- Add type hints to `get_{domain,localpart}_from_id`. ([\matrix-org#10385](matrix-org#10385))
- When building Debian packages for prerelease versions, set the Section accordingly. ([\matrix-org#10391](matrix-org#10391))
- Add type hints and comments to event auth code. ([\matrix-org#10393](matrix-org#10393))
- Stagger sending of presence update to remote servers, reducing CPU spikes caused by starting many connections to remote servers at once. ([\matrix-org#10398](matrix-org#10398))
- Remove unused `events_by_room` code (tech debt). ([\matrix-org#10421](matrix-org#10421))
- Add a github actions job which records success of other jobs. ([\matrix-org#10430](matrix-org#10430))
babolivier added a commit to matrix-org/synapse-dinsic that referenced this pull request Sep 1, 2021
@babolivier
Merge tag 'v1.39.0' into babolivier/dinsic_1.41.0
262007f 
Synapse 1.39.0 (2021-07-29)
===========================

No significant changes.

Synapse 1.39.0rc3 (2021-07-28)
==============================

Bugfixes
--------

- Fix a bug introduced in Synapse 1.38 which caused an exception at startup when SAML authentication was enabled. ([\#10477](matrix-org/synapse#10477))
- Fix a long-standing bug where Synapse would not inform clients that a device had exhausted its one-time-key pool, potentially causing problems decrypting events. ([\#10485](matrix-org/synapse#10485))
- Fix reporting old R30 stats as R30v2 stats. Introduced in v1.39.0rc1. ([\#10486](matrix-org/synapse#10486))

Internal Changes
----------------

- Fix an error which prevented the Github Actions workflow to build the docker images from running. ([\#10461](matrix-org/synapse#10461))
- Fix release script to correctly version debian changelog when doing RCs. ([\#10465](matrix-org/synapse#10465))

Synapse 1.39.0rc2 (2021-07-22)
==============================

This release also includes the changes in v1.38.1.

Internal Changes
----------------

- Move docker image build to Github Actions. ([\#10416](matrix-org/synapse#10416))

Synapse 1.39.0rc1 (2021-07-20)
==============================

The Third-Party Event Rules module interface has been deprecated in favour of the generic module interface introduced in Synapse v1.37.0. Support for the old interface is planned to be removed in September 2021. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information.

Features
--------

- Add the ability to override the account validity feature with a module. ([\#9884](matrix-org/synapse#9884))
- The spaces summary API now returns any joinable rooms, not only rooms which are world-readable. ([\#10298](matrix-org/synapse#10298), [\#10305](matrix-org/synapse#10305))
- Add a new version of the R30 phone-home metric, which removes a false impression of retention given by the old R30 metric. ([\#10332](matrix-org/synapse#10332), [\#10427](matrix-org/synapse#10427))
- Allow providing credentials to `http_proxy`. ([\#10360](matrix-org/synapse#10360))

Bugfixes
--------

- Fix error while dropping locks on shutdown. Introduced in v1.38.0. ([\#10433](matrix-org/synapse#10433))
- Add base starting insertion event when no chunk ID is specified in the historical batch send API. ([\#10250](matrix-org/synapse#10250))
- Fix historical batch send endpoint (MSC2716) rejecting batches with messages from multiple senders. ([\#10276](matrix-org/synapse#10276))
- Fix purging rooms that other homeservers are still sending events for. Contributed by @ilmari. ([\#10317](matrix-org/synapse#10317))
- Fix errors during backfill caused by previously purged redaction events. Contributed by Andreas Rammhold (@andir). ([\#10343](matrix-org/synapse#10343))
- Fix the user directory becoming broken (and noisy errors being logged) when knocking and room statistics are in use. ([\#10344](matrix-org/synapse#10344))
- Fix newly added `synapse_federation_server_oldest_inbound_pdu_in_staging` prometheus metric to measure age rather than timestamp. ([\#10355](matrix-org/synapse#10355))
- Fix PostgreSQL sometimes using table scans for queries against `state_groups_state` table, taking a long time and a large amount of IO. ([\#10359](matrix-org/synapse#10359))
- Fix `make_room_admin` failing for users that have left a private room. ([\#10367](matrix-org/synapse#10367))
- Fix a number of logged errors caused by remote servers being down. ([\#10400](matrix-org/synapse#10400), [\#10414](matrix-org/synapse#10414))
- Responses from `/make_{join,leave,knock}` no longer include signatures, which will turn out to be invalid after events are returned to `/send_{join,leave,knock}`. ([\#10404](matrix-org/synapse#10404))

Improved Documentation
----------------------

- Updated installation dependencies for newer macOS versions and ARM Macs. Contributed by Luke Walsh. ([\#9971](matrix-org/synapse#9971))
- Simplify structure of room admin API. ([\#10313](matrix-org/synapse#10313))
- Refresh the logcontext dev documentation. ([\#10353](matrix-org/synapse#10353)), ([\#10337](matrix-org/synapse#10337))
- Add delegation example for caddy in the reverse proxy documentation. Contributed by @moritzdietz. ([\#10368](matrix-org/synapse#10368))
- Fix and clarify some links in `docs` and `contrib`. ([\#10370](matrix-org/synapse#10370)), ([\#10322](matrix-org/synapse#10322)), ([\#10399](matrix-org/synapse#10399))
- Make deprecation notice of the spam checker doc more obvious. ([\#10395](matrix-org/synapse#10395))
- Add instructions on installing Debian packages for release candidates. ([\#10396](matrix-org/synapse#10396))

Deprecations and Removals
-------------------------

- Remove functionality associated with the unused `room_stats_historical` and `user_stats_historical` tables. Contributed by @xmunoz. ([\#9721](matrix-org/synapse#9721))
- The third-party event rules module interface is deprecated in favour of the generic module interface introduced in Synapse v1.37.0. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information. ([\#10386](matrix-org/synapse#10386))

Internal Changes
----------------

- Convert `room_depth.min_depth` column to a `BIGINT`. ([\#10289](matrix-org/synapse#10289))
- Add tests to characterise the current behaviour of R30 phone-home metrics. ([\#10315](matrix-org/synapse#10315))
- Rebuild event context and auth when processing specific results from `ThirdPartyEventRules` modules. ([\#10316](matrix-org/synapse#10316))
- Minor change to the code that populates `user_daily_visits`. ([\#10324](matrix-org/synapse#10324))
- Re-enable Sytests that were disabled for the 1.37.1 release. ([\#10345](matrix-org/synapse#10345), [\#10357](matrix-org/synapse#10357))
- Run `pyupgrade` on the codebase. ([\#10347](matrix-org/synapse#10347), [\#10348](matrix-org/synapse#10348))
- Switch `application_services_txns.txn_id` database column to `BIGINT`. ([\#10349](matrix-org/synapse#10349))
- Convert internal type variable syntax to reflect wider ecosystem use. ([\#10350](matrix-org/synapse#10350), [\#10380](matrix-org/synapse#10380), [\#10381](matrix-org/synapse#10381), [\#10382](matrix-org/synapse#10382), [\#10418](matrix-org/synapse#10418))
- Make the Github Actions workflow configuration more efficient. ([\#10383](matrix-org/synapse#10383))
- Add type hints to `get_{domain,localpart}_from_id`. ([\#10385](matrix-org/synapse#10385))
- When building Debian packages for prerelease versions, set the Section accordingly. ([\#10391](matrix-org/synapse#10391))
- Add type hints and comments to event auth code. ([\#10393](matrix-org/synapse#10393))
- Stagger sending of presence update to remote servers, reducing CPU spikes caused by starting many connections to remote servers at once. ([\#10398](matrix-org/synapse#10398))
- Remove unused `events_by_room` code (tech debt). ([\#10421](matrix-org/synapse#10421))
- Add a github actions job which records success of other jobs. ([\#10430](matrix-org/synapse#10430))
Fizzadar pushed a commit to Fizzadar/synapse that referenced this pull request Oct 26, 2021
@bradtgmurray
Merge tag 'v1.39.0'
c9a3235 
Synapse 1.39.0 (2021-07-29)
===========================

No significant changes.

Synapse 1.39.0rc3 (2021-07-28)
==============================

Bugfixes
--------

- Fix a bug introduced in Synapse 1.38 which caused an exception at startup when SAML authentication was enabled. ([\matrix-org#10477](matrix-org#10477))
- Fix a long-standing bug where Synapse would not inform clients that a device had exhausted its one-time-key pool, potentially causing problems decrypting events. ([\matrix-org#10485](matrix-org#10485))
- Fix reporting old R30 stats as R30v2 stats. Introduced in v1.39.0rc1. ([\matrix-org#10486](matrix-org#10486))

Internal Changes
----------------

- Fix an error which prevented the Github Actions workflow to build the docker images from running. ([\matrix-org#10461](matrix-org#10461))
- Fix release script to correctly version debian changelog when doing RCs. ([\matrix-org#10465](matrix-org#10465))

Synapse 1.39.0rc2 (2021-07-22)
==============================

This release also includes the changes in v1.38.1.

Internal Changes
----------------

- Move docker image build to Github Actions. ([\matrix-org#10416](matrix-org#10416))

Synapse 1.39.0rc1 (2021-07-20)
==============================

The Third-Party Event Rules module interface has been deprecated in favour of the generic module interface introduced in Synapse v1.37.0. Support for the old interface is planned to be removed in September 2021. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information.

Features
--------

- Add the ability to override the account validity feature with a module. ([\matrix-org#9884](matrix-org#9884))
- The spaces summary API now returns any joinable rooms, not only rooms which are world-readable. ([\matrix-org#10298](matrix-org#10298), [\matrix-org#10305](matrix-org#10305))
- Add a new version of the R30 phone-home metric, which removes a false impression of retention given by the old R30 metric. ([\matrix-org#10332](matrix-org#10332), [\matrix-org#10427](matrix-org#10427))
- Allow providing credentials to `http_proxy`. ([\matrix-org#10360](matrix-org#10360))

Bugfixes
--------

- Fix error while dropping locks on shutdown. Introduced in v1.38.0. ([\matrix-org#10433](matrix-org#10433))
- Add base starting insertion event when no chunk ID is specified in the historical batch send API. ([\matrix-org#10250](matrix-org#10250))
- Fix historical batch send endpoint (MSC2716) rejecting batches with messages from multiple senders. ([\matrix-org#10276](matrix-org#10276))
- Fix purging rooms that other homeservers are still sending events for. Contributed by @ilmari. ([\matrix-org#10317](matrix-org#10317))
- Fix errors during backfill caused by previously purged redaction events. Contributed by Andreas Rammhold (@andir). ([\matrix-org#10343](matrix-org#10343))
- Fix the user directory becoming broken (and noisy errors being logged) when knocking and room statistics are in use. ([\matrix-org#10344](matrix-org#10344))
- Fix newly added `synapse_federation_server_oldest_inbound_pdu_in_staging` prometheus metric to measure age rather than timestamp. ([\matrix-org#10355](matrix-org#10355))
- Fix PostgreSQL sometimes using table scans for queries against `state_groups_state` table, taking a long time and a large amount of IO. ([\matrix-org#10359](matrix-org#10359))
- Fix `make_room_admin` failing for users that have left a private room. ([\matrix-org#10367](matrix-org#10367))
- Fix a number of logged errors caused by remote servers being down. ([\matrix-org#10400](matrix-org#10400), [\matrix-org#10414](matrix-org#10414))
- Responses from `/make_{join,leave,knock}` no longer include signatures, which will turn out to be invalid after events are returned to `/send_{join,leave,knock}`. ([\matrix-org#10404](matrix-org#10404))

Improved Documentation
----------------------

- Updated installation dependencies for newer macOS versions and ARM Macs. Contributed by Luke Walsh. ([\matrix-org#9971](matrix-org#9971))
- Simplify structure of room admin API. ([\matrix-org#10313](matrix-org#10313))
- Refresh the logcontext dev documentation. ([\matrix-org#10353](matrix-org#10353)), ([\matrix-org#10337](matrix-org#10337))
- Add delegation example for caddy in the reverse proxy documentation. Contributed by @moritzdietz. ([\matrix-org#10368](matrix-org#10368))
- Fix and clarify some links in `docs` and `contrib`. ([\matrix-org#10370](matrix-org#10370)), ([\matrix-org#10322](matrix-org#10322)), ([\matrix-org#10399](matrix-org#10399))
- Make deprecation notice of the spam checker doc more obvious. ([\matrix-org#10395](matrix-org#10395))
- Add instructions on installing Debian packages for release candidates. ([\matrix-org#10396](matrix-org#10396))

Deprecations and Removals
-------------------------

- Remove functionality associated with the unused `room_stats_historical` and `user_stats_historical` tables. Contributed by @xmunoz. ([\matrix-org#9721](matrix-org#9721))
- The third-party event rules module interface is deprecated in favour of the generic module interface introduced in Synapse v1.37.0. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information. ([\matrix-org#10386](matrix-org#10386))

Internal Changes
----------------

- Convert `room_depth.min_depth` column to a `BIGINT`. ([\matrix-org#10289](matrix-org#10289))
- Add tests to characterise the current behaviour of R30 phone-home metrics. ([\matrix-org#10315](matrix-org#10315))
- Rebuild event context and auth when processing specific results from `ThirdPartyEventRules` modules. ([\matrix-org#10316](matrix-org#10316))
- Minor change to the code that populates `user_daily_visits`. ([\matrix-org#10324](matrix-org#10324))
- Re-enable Sytests that were disabled for the 1.37.1 release. ([\matrix-org#10345](matrix-org#10345), [\matrix-org#10357](matrix-org#10357))
- Run `pyupgrade` on the codebase. ([\matrix-org#10347](matrix-org#10347), [\matrix-org#10348](matrix-org#10348))
- Switch `application_services_txns.txn_id` database column to `BIGINT`. ([\matrix-org#10349](matrix-org#10349))
- Convert internal type variable syntax to reflect wider ecosystem use. ([\matrix-org#10350](matrix-org#10350), [\matrix-org#10380](matrix-org#10380), [\matrix-org#10381](matrix-org#10381), [\matrix-org#10382](matrix-org#10382), [\matrix-org#10418](matrix-org#10418))
- Make the Github Actions workflow configuration more efficient. ([\matrix-org#10383](matrix-org#10383))
- Add type hints to `get_{domain,localpart}_from_id`. ([\matrix-org#10385](matrix-org#10385))
- When building Debian packages for prerelease versions, set the Section accordingly. ([\matrix-org#10391](matrix-org#10391))
- Add type hints and comments to event auth code. ([\matrix-org#10393](matrix-org#10393))
- Stagger sending of presence update to remote servers, reducing CPU spikes caused by starting many connections to remote servers at once. ([\matrix-org#10398](matrix-org#10398))
- Remove unused `events_by_room` code (tech debt). ([\matrix-org#10421](matrix-org#10421))
- Add a github actions job which records success of other jobs. ([\matrix-org#10430](matrix-org#10430))
@buffless-matt buffless-matt mentioned this pull request Mar 18, 2022
Open
@dklimpel dklimpel mentioned this pull request Apr 22, 2022
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@richvdh richvdh richvdh approved these changes

Assignees

@richvdh richvdh

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

room_stats_historical and user_stats_historical appear unused
5 participants
@xmunoz @richvdh @anoadragon453 @erikjohnston @reivilibre