There is currently no way to erase documents from the SimpleStore with LlamaIndex #1892
automaton82
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
If you setup a flow like this and upsert:
The documents go into the vector storage which is great.
Now if you create another chatflow with a different file and upsert it, both files are now in the vector store and there's no way to erase the original ones. This problem doesn't present itself with other vector stores because they're already isolated. In-memory is isolated per session, and persistence ones can have the name of the storage.
The other problem here is each upsert adds which means you can get into infinite growth with no way to clean up.
SimpleStore should have an option when upserting to erase all existing content.
Also -- as a note, unlike LangChain, when doing a prediction it won't automatically upsert the documents into the storage. It must be manually upsert first, which is not ideal. See #1883 for more information.
Beta Was this translation helpful? Give feedback.
All reactions