Client-side search for E2E

[ara4n]

Server-side search for E2E isn't happening anytime soon. We should consider our options for client-side search, though. Rather than paginating in history, decrypting it, and searching clientside, perhaps we should finally finish the localstorage history store in js-sdk (which would also then speed up reloads...)

[arxcode]

Maybe it would make sense in the meantime to display at least a helpful message to users trying to search an encrypted room? All you get right now is "No results", which is confusing to people who do not know the technical background. Something like "Search is not available for encrypted rooms."?

[kannes]

Having a message would definitely help... Also please add a warning to the message box when one turns on a room's encryption. Discovering this months later sucks.

[hukaze]

The warning at enablment would be huge.. When we built our first servers some time ago we literally rebuilt them multiple times thinking that the search had somehow broken along the way.. Only now, ages later, do we discover that this is a natural behavior.. Also, don't mean to ask the unanswerable but.. Any idea when this will be a functional ability both at the client and the server?

[t3chguy]

I am currently tasked with a generic search daemon, which you'll be able to run client-side and it'll offer e2e search.

[adrianog]

Is there a roadmap? Next milestone / alpha quality build?

[ara4n]

so the daemon exists over at https://github.com/matrix-org/matrix-search. however, it's experimental still; we need to do the work to get it hooked into riot nicely (e.g. by bundling it with riot/desktop).

An alternate approach here could be for clients to slowly download full e2e history in the background and decrypt and index it themselves (potentially uploading encrypted indexes to the server to speed up the operation).

[MountainX]

https://github.com/matrix-org/matrix-search is showing no updates for 9 months. Has the roadmap changed? Last fall it seemed client side search of E2E rooms would be part of the Riot 1.0 release.

[jryans]

The feature of E2E search is still quite high on the Riot roadmap. We weren't able to have it ready for Riot 1.0, but it's clear that some form of search in E2E rooms is required before we can make E2E rooms the default mode. The Riot team is actively planning our post-1.0 roadmap, so we should have more timing details to share soon.

[MountainX]

The feature of E2E search is still quite high on the Riot roadmap. ... The Riot team is actively planning our post-1.0 roadmap, so we should have more timing details to share soon.

Hi. We're looking for an update on E2E search. It's been 6-7 months since last comment here. Hope to know more details. Thank you. Riot is the best.

[SharkyRawr]

I just ran in to this, using matrix/riot with my own homeserver as an alternative to Telegram and other IMs. Search yielded "No results" whatever the search term was, in a conversation that's been E2E encrypted since the beginning. Took me a while to connect the dots. 🤣

Searching your messages, encrypted or not, is really a fundamental feature that I'm missinga lot -- but I also understand that it's difficult to implement without breaking something because of the current architecture.

In the mean time a warning or explanation "cannot search encrypted rooms" in Riot (or synapse?) would be really appreciated while the feature is still being worked on (being planned? whats the status?).

[flooguu]

This is insane... the text is right there on my screen, yet i can not find a message even from yesterday? Is there absolutely no way to do a client-side search on a best effort basis... this is a huge drawback for 3 years now. And there is not even a sensible error message, just 'no results'.

[jryans]

The latest news is that search in encrypted rooms will be included as part of the cross-signing / E2E by default for DMs release coming up in a few weeks. It will only be available in Riot Desktop initially, as it requires native components that can't easily be wired up in the web version.

If you are feeling adventurous, you can try out the feature right now by building Riot Desktop with it enabled.

[MurzNN]

Will seshat do search in all room messages, or only on last messages, that available on local cache? And how it will work with global search in all rooms (server-size search in regular rooms, and local search on e2e rooms)?

[jryans]

Will seshat do search in all room messages, or only on last messages, that available on local cache? And how it will work with global search in all rooms (server-size search in regular rooms, and local search on e2e rooms)?

It crawls all room messages in the background. The initial implementation will use the existing server search for regular rooms and Seshat's local index for encrypted rooms, but we may eventually change to a local index for all rooms.

[michaelsmoody]

Good evening. I'm curious if you have a rough release date plan that will include seshat? I'm looking forward to it quite a bit.

[jryans]

I'm curious if you have a rough release date plan that will include seshat?

Nightly builds of Riot Desktop with Seshat are coming together and should be ready this week (still a few updating issues to resolve).

We're on track for a stable release with Seshat in a few weeks.

[michaelsmoody]

Good afternoon. Just checking in to see if I need to enable a lab on Riot Desktop in order to enable this, or if I should just sit tight? Also, any need for French, Spanish, or English translation for this feature? We're fluent and can help with those languages if you need anything.

[t3chguy]

It will be available in riot-desktop 1.6, its not available in stable as it requires a different build process for the native module seshat. All translation efforts can be done via https://translate.riot.im

[geckolinux]

Really looking forward to this, thanks for listening to user requests.

[turt2live]

Going to close this as we've landed it on develop and have nightly builds for it. Please report issues and feature requests for capabilities as new issues.

[tredondo]

FWIW, Pidgin with the OTR plugin allowed searching in encrypted rooms 5+ years ago.

It also optionally saved clear-text logs in a location of the user's choice (e.g. a Veracrypt drive or similar), so they could search across rooms at any time later.

I suspect riot is doing more than reinventing the wheel. What the advantages are over tried & true solutions like Pidgin + OTR?

[michaelsmoody]

While Double Ratchet (OLM) was originally based on OTR, they're really quite different at this point from what I understand. Not to mention that Riot primarily runs on the web, and even the Riot Desktop app is an Electron app. Having also used Pigdin and OTR back in the day, I did find the experience left some to be desired. But at the same time, use whichever solution works for you? I personally use Riot/Matrix, Signal, and several other messengers with E2EE.

EDIT: One MAJOR difference being the asynchronous messaging. Both parties don't need to be online at the same time.

[michaelsmoody]

@turt2live Good afternoon. Is there a place where the nightlies or release candidates are released as builds for the varying platforms? (Yes, I could build, but I figured I'd ask before I do so). Essentially, is there a dmg/exe/etc for 1.6.0-rc4 on a build server somewhere? If so, I haven't yet been able to find it.

[lub]

@michaelsmoody see https://packages.riot.im/nightly/install/ and https://packages.riot.im/debian/pool/main/r/riot-nightly/ for nightlies

[michaelsmoody]

Thank you. Just curious, did I miss those things in the README.md somewhere? Or on the website?

[aaronraimist]

No you didn’t. It hasn’t be documented on the website yet #13173