Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vector search and embeddings generation #37

Merged
merged 12 commits into from
Jul 12, 2024
Merged

Vector search and embeddings generation #37

merged 12 commits into from
Jul 12, 2024

Conversation

justyns
Copy link
Owner

@justyns justyns commented Jul 11, 2024

For #34

Needs more testing, but so far:

  • Added support for generating embeddings using openai, ollama, and gemini
  • Stores generated embeddings in silverbullet's native datastore
  • Listens for page:index events and indexes pages automatically when they change
  • Generates an embedding per paragraph
  • Added a new 🤖 virtual page to test searching with

Todo:

  • make this whole thing optional
  • add docs and make sure to recommend using local first, like ollama
  • more testing to make sure nothing blows up

@justyns justyns changed the title WIP: Vector search and embeddings generation Vector search and embeddings generation Jul 12, 2024
@justyns justyns marked this pull request as ready for review July 12, 2024 03:47
@justyns justyns merged commit a675760 into main Jul 12, 2024
3 checks passed
@justyns justyns deleted the vector-embeddings branch July 12, 2024 03:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant