Release 2024-02-12 - (expected chart version 4.40.0)

[zebot]

[2024-02-12] (Chart Release 4.40.0)

Release notes

• The settings setDisabledAPIVersions (brig) and disabledAPIVersions (in cannon, cargohold, galley, gundeck, proxy, and spar) are now required.
  The default defined in charts/<service>/values.yaml is set to [ development ] and disables all development API versions.
  For more information see https://docs.wire.com/developer/reference/config-options.html#disabling-api-versions (WPB-4657 Disabling development versions #3772)

• The mls team feature now has a lock status. If the current installation has default settings defined in wire-server's values.yaml, the lockStatus has to be added, e.g.:

  mls:
    defaults:
      status: enabled
      config:
        protocolToggleUsers: []
        defaultProtocol: mls
        allowedCipherSuites: [1]
        defaultCipherSuite: 1
        supportedProtocols: [proteus, mls] # must contain defaultProtocol
      lockStatus: locked
  ``` (#3681)
  

• • Replace fake-sqs with ElasticMQ

  ElasticMQ is an actively maintained project, fake-sqs hasn't seen a commit since
  2018. This is not expected to have any noticeable effect on deployments that
  don't have any extra configurations for the SQS queues. If the fake-aws-sqs
  chart had configured custom queue names, they have couple of extra limitations:

  • The queue names must only contain alphanumeric characters and hyphens.
  • The FIFO queue names must end in .fifo. (Use ElasticMQ instead of fake_sqs #3750)

• Upgrade team-settings version to 4.15.1-v0.31.19-0-ee1dbce (Update team-settings version in Helm chart [skip ci] #2180)

• Upgrade webapp to 2023-12-11-production.0-v0.31.17-0-1e91445

  Beside using up-to-date versions in Helm charts is generally beneficial,
  this version also provides multi-ingress support. (webapp: Upgrade to 2023-12-11-production.0-v0.31.17-0-1e91445 #3803)

API changes

• Un-verified users can no longer upload assets (WPB-1906 - Unverified users can no longer create assets #3604)

• • Remove IP request header from add-client endpoint.
  • Remove longitude and latitude response fields from add-client, list-clients, and get-client endpoints.

  These are not considered breaking changes, since clients are not using this information. (remove geoip (dead) code #3792)

• Create new API version V6 and finalize V5 (WPB-6012 create new API version v6 #3815)

• Block changes of userDisplayName, userHandle in mlsE2EI-enabled teams on the backend without SCIM; report "managed_by" == "scim" in GET /self, but only there (Disallow changing user display name, handle in mlsE2EId-enabled teams #3827)

• The federation API can now be versioned. Multiple versions of an RPC can be defined on the same path. After version negotiation, the federation client now sets the X-Wire-API-Version header, and federator propagates it to the destination service. (Federation API versioning #3762)

• Improved formatting of federation errors. No extra copy of the response body, and nested errors are now part of the JSON structure, not quoted inside the message. (Federation error wrapping #3742)

• New endpoint for replacing MLS key packages in bulk: PUT /mls/key-packages/self/:client. It replaces all existing key packages that match the given ciphersuites with the new key packages provided in the body. ([WPB-4981] replace unclaimed keypackages atomically #3654)

Features

• The lifetime of conversation guest links is now configurable (WPB-1436 make guest link maximum lifetime configurable #3796)

• Events for a member update, join and leave are not sent to everyone in the team any longer. Only team admins get them. ([WPB-2565] Do not send member updates to all #3703)

• Allowlist for who on cloud can connect to on-prem:

  • Internal API to configure allowlist
  • Restrict federated user search according to team federation policy
    (WPB-5385 Extend internal federation config API with team ID #3697, WBP-5388 restrict contact search results according to team federation policy #3732, [fix] WPB-5715 data access layer of federation_remotes #3758)

• The mls team feature now has a lock status (WPB-5143 locked status for mls config #3681)

• add a uniform timeout to the integration test-suite set by the environment variable TEST_TIMEOUT_SECONDS with a default of 10 seconds if the variable isn't set ([WPB-5241] add the timeout to the global and local environment #3692)

• Apply team-level federation policies when establishing and updating user connections ([WPB-5389] Guard user connection requests by team-level federation settings #3774)

• Introduce a feature flag that controls whether the limited event fanout should be used when a team member is deleted ([WPB-5883] Feature flag for a limited event fanout #3797)

• Send a conversation.member-leave event to team admins for each conversation the deleted team member used to be part of ([WPB-5936] Send conversation.member-leave events to team admins #3790)

• Allow the configuration of TLS-secured connections to Cassandra. TLS is used
  when a certificate is provided. This is either done with
  --tls-ca-certificate-file for cli commands or the configuration attribute
  cassandra.tlsCa for services. In Helm charts, the certificate is provided as
  literal PEM string; either as attribute cassandra.tlsCa (analog to service
  configuration) or by a reference to a secret (cassandra.tlsCaSecretRef.) (TLS connections to Cassandra #3587)

• [SFT chart] Add option to enable serviceMonitor to scrape prometheus metrics (add optional serviceMonitor field for SFTD chart #3770)

Bug fixes and other updates

• galley's DB migrations fixed (Fix galley DB migrations #3680)

• The X509 client identity parser supports a new format: wireapp://{userid}!{deviceid}@{host} (fix: X509 Client Identity parser #3808)

• Updated demo-smtp Helm chart from deprecated docker image namshi/smtp to ixdotai/smtp (fix: WPB-5064 Moved namshi to ix-ai smtp image  #3791)

• External partners search restriction enforced by backend (WBP-5133 External partners search restriction enforced by backend #3708)

• File upload size is now limited to 100 MiB (WPB-5417 limit file upload to 100MiB #3752)

• Fix a bug where non-team conversation members that are remote would not get a conversation.member-leave event ([WPB-5603] Deleting a team member does not result in a conversation event #3745, [WPB-5603] Fix the team member deleted event reason #3764)

• Enforce external partner permissions on the backend (WPB-5695 Enforce group conversation permission for external partner role #3788)

• Various improvements around LH policy conflict detection:

  • Fix LH policy conflict detection logic when posting messages
  • Better policy conflict error messages (distinguish between old clients and missing consent)
  • Add first LH scaffolding and tests to /integration
  • Annotate some API functions in /integration with links to openapi3 docs (Various improvements around LH policy conflict detection #3773)

• Do not match on the Accept header for service provider endpoints with no response body ([WPB-5810] Fix the service provider endpoints that return no body #3766)

• Guests should not be added to conversations that are under legalhold (WPB-5845 guests should not be able to join conversations under legalhold #3853)

• Intra-service calls from brig to galley's public API are now aware of disabled API versions (WPB-6351 Use max available version for internal API calls #3863)

• fix Helm pretty-printer for disabledAPIVersions (WPB-6428 fix Helm pretty-printer for disabledAPIVersions #3877)

• Adjust the requested memory and upper bound limit of nginz pods in the related Helm chart. (We experienced OOM errors with the old settings.) (increase nginz memory limit #3821)

• don't use shell when communicating with mls-test-cli, move flaking brig tests over to new integration testsuite ([WPB 5356] fix brig flaking #3701)

• set notificationTimeOut to 28 days, make it legible ([fix] set notificationTimeOut to 28 days, make it legible #3714)

• Update coturn image with bugfix to its prestop-hook from Fixup pre-stop-hook that is currently not working. coturn#10 to allow coturn pods to terminate once their traffic has drained. (Update coturn default image #3872)

• Extra remove proposals were being sent when a user was removed from a conversation (Fix extra remove proposal bug #3672)

• Remove client check for subconversations (Remove client check for subconversations #3677)

• Ensure that SCIM can find users even after the team admin has changed the SAML issuer for the user. ( Spar: Ensure mkValidExternalId returns a valid URef  #3747)

• addClient used the internal brig API in the integration testsuite when it should use the public one ([fix] use the correct API in the integration tests #3869)

• Ensure that HTTP 1.1 connections are grafully closed

  To fix this warp had to be patched to fix the bug upstream: warp: Send Connection: close when closing the connection yesodweb/wai#958 (Use fork of warp which closes connections gracefully #3775)

Documentation

• Fix missing code sections on docs.wire.com, notably on "configuring TLS" page. (fixing grepinclude references for docs.wire.com and adjusting nix build context; updating TLS documentation #3839)

• Swagger generation no longer adds tags containing information about federation calls.

  Added the federation calling graph to the Federation API Conventions page. (WPB-4853: Swagger cleanup #3674)

• Backend-to-backend OpenApi Docs added (WPB-5098 Backend-to-backend OpenApi Docs #3666)

• Documentation for creating a new API version updated (WPB-6177 document steps for creating new API version #3817)

• Update documentation of MLS group ID (Update group ID documentation #3705)

• Turn long summaries in openapi documentation into descriptions (Turn long summaries into descriptions #3706)

• update the build instructions for wire-server ([WPB-6210] update documentation on how to build wire-server #3854)

Internal changes

• stern/backoffice PUT /teams/{teamId}/features/conferenceCalling fixed ([fix] stern/backoffice conference calling TTL #3723)

• Removed client ID conversion round trip ([chore] Remove client ID conversion roundtrip #3727)

• Migrate to Servant the Galley conversation internal endpoints ([WPB-1226] Servantify internal Galley conversation endpoints #3718)

• The development API version is now disabled by default (WPB-4657 Disabling development versions #3772)

• Attempt to fix flaky integration test provider.service.delete (WPB-4848 Flaky test #3689)

• The fedcalls tool no longer walks the Swagger/OpenAPI structure when generating call graphs. These graphs are now generated directly from the Servant API types. (WPB-4853: Swagger cleanup #3674, Swagger docs: new line after fed call tag #3691)

• Increased ingress payload size from 256k to 512k (WPB-4887 increased ingress payload size from 256k to 512k #3756)

• Request tracing across federated requests (WPB-4888: Implement request tracing across federation #3765)

• upgrade nixpkgs to upgrade haskell-language-server ([WPB-5042] upgrade nixpkgs to upgrade haskell-language-server #3650)

• upgrade the GHC version to GHC 9.4 ([WPB-5175] upgrade to ghc 9.4 #3679)

• Removed APNS_VOIP code. (APNS_VOIP is a native push notification channel which we aren't using anymore.) (WPB-5204 Removing unused code #3695)

• Improve error logs (WPB-5312 Improve federator error logs #3782)

• Migrating tests for Cargohold to the new integration test suite. (WPB-5382 - Migrating tests from Cargohold into the new integration test suite. #3741)

• Fix calendar integration setting in backoffice / stern (Fix calendar integration setting in backoffice / stern #3761)

• Reply-Nonce is added to Access-Control-Expose-Headers (WBP-5577 make replay nonce header accessible for frontend #3729)

• Add custom feature flag; only supported for some on-prem installations; locked & disabled by default (New team feature EnforceFileDownloadLocation #3779)

• Improved how tests are automatically extracted from the integration test suite.

  The test extractor parser has been improved to handle block comments, and to more strictly check for Haddock documentation for each test. (WPB-5667: Updating integration tests to better handle comments and haddock. #3749)

• Additional logging on user/team suspension (WPB-6001 suspend user logging #3795)

• cleanup the haskell-pins

  • remove many pins
  • remove many overrides
  • restructure the files such that it's easier to see what is going on ([WPB-6073] cleanup haskell pins #3814)

• Version of rusty-jwt-tools bumped to v0.8.0 (WPB-6101 make feature enforceFileDownloadLocation unlockable for QA #3805)

• Feature enforceFileDownloadLocation lockstatus can be set with basic auth on staging (WPB-6099 Bump the version of rusty-jwt-tools in wire server #3802)

• Version of rusty-jwt-tools bumped to v0.8.5 (WPB-6181 Update rusty-jwt-tools #3820)

• Translate integration tests: manually add / delete LH device (Clean up LH tests #3830)

• adds a new executable, hs-run, to quickly run haskell scripts ([feat] add support for ghc-flakr's hs-run executable #3716)

• Represent client IDs as Word64 internally (Use Word64 to represent a ClientId #3713)

• Allow to install the coturn chart multiple times in multiple namespaces on the same cluster. (Coturn: avoid name clashes when installing multiple times #3698)

• For some rust packages (cryptobox and libzauth-c), we now use crate2nix as a build tool, rather than the more coarse and FOD-based nixpkgs rustPlatform.buildRustPackage approach. (#PR_NOT_FOUND)

• Delete shell.nix. It has been broken for quite some time. The supported way to get a development nix environment is to use direnv. (Delete shell.nix #3726)

• Deploy a backend with federation API V0 while setting up services for local testing (docker-ephemeral: Run federation-v0 services for backwards compat testing #3719)

• Improve integration test coverage (Improve spar test coverage #3757)

• Increase timeout for waiting for SQS notifications in galley's integration tests (Increase SQS timeout in galley integration #3699)

• Simplify process spawning of dynamic backends in integration tests (Simplify process spawning in integration tests #3759)

• More robust consuming of MLS messages: the behaviour of sendAndConsumeMessage and sendAndConsumeCommitBundle is changed to actually wait for those messages on the client's websocket (Consume MLS messages from websocket #3671)

• Update group state after application message (Update group state after application messages #3678)

• bump the nixpkgs version to allow updating curl ([feat] nixpkgs bump #3781)

• Simplify the definition of the servant notification API (Refactor notification API descriptions #3685)

• Start refactoring code into subsystems, first subsystem being the NotificationSubsystem. (Introduce NotificationSubsystem to push notifications #3786)

• Remove apply-refact from CI image

  This gets rid of GHC in the image, making the image smaller. (Reduce the size for CI image by getting rid of 2 GHCs #3712)

• Refactor getOptions (Refactor getOptions #3707)

• Restored Brig memory quota to 512mb down from 1gb. (prev bump CI: Increase memory limit for brig to 1Gi #3751) (Revert brig memory setup back to 512mb #3806)

• Add tool to analyse test results in junit/ant xml format (Add tool to aggregate and push test statistics from junit/ant XML reports #3652)

• updated annotation for enabling Topology Aware Routing to service.kubernetes.io/topology-mode for k8s 1.27+ (update annotation key for k8s 1.27+ #3878)

• replace runAsNonRoot with runAsUser and runAsGroup 1000 (Update helm charts to support k8s v1.28 #3826)

• Update SFTD default to 4.0.10 and its nginx to 1.25.3. (Update SFTD and its nginx images used by default in the helm charts #3768)

• add a Makefile target to make it possible to upload a bom of all services to s3 on every CI run ([feat] bombon derivations #3744)

• Upload bill-of-material (BOM) files directly to the Dependency Tracker via REST.
  This eases the life of the security team and prevents cluttering our release
  artifact page. (Upload bombon bom files directly to deptrack (WPB-6142) #3810)

• Passively migrate user passwords from scrypt to argon2id.

  By passively we mean that whenever a user re-enters their passwords, if it was hashed using scrypt, it is then rehashed using argon2id and stored as such.
  If that user has a legacy short password (under 8 characters in length), it does not migrate to argon2id. (Add Argon2id support on top of Scrypt for password hashing #3720)

Federation changes

• Define a few tests for adding members to an MLS conversation when unreachable backends are involved ([WPB-5103] Add users to MLS conversation when some backends are unreachable #3673)

• Make sure that remote users can be added to both a Proteus and an MLS conversation when other users are unreachable ([WPB-5208] Allow adding users to conversations when other backends are unreachable #3688)

[CLAassistant]

All committers have signed the CLA.

[battermann]

webapp version bump missing