Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LlamaIndex Integration #31

Open
steventkrawczyk opened this issue Jul 25, 2023 · 4 comments
Open

LlamaIndex Integration #31

steventkrawczyk opened this issue Jul 25, 2023 · 4 comments
Assignees

Comments

@steventkrawczyk
Copy link
Contributor

🚀 The feature

We need an experiment or example for LlamaIndex. This will be similar to the work we need to do for LangChain and MindsDB.

Motivation, pitch

LlamaIndex is a popular framework for connecting LLMs to data. It would be good to support LlamaIndex testing using our existing vectorDB and LLM evaluations.

Alternatives

N/A

Additional context

N/A

@rachittshah
Copy link

hi @steventkrawczyk I'd love to work on this.

My guess is we'd need abstractions over LlamaIndex over core prompttools classes, but we'll need to update it with each release of LlamaIndex.

@steventkrawczyk
Copy link
Contributor Author

Hey @rachittshah we'd love your help! I think you're right that we'll need to build prompttools abstractions on top of the llamaindex abstractions. I'm hoping there are a few abstractions which are stable enough for us to build once, but if we need to update with each release we can explore that too.

Feel free to open a draft PR with what you're thinking now, and we can work together to iterate. If you want to chat in depth, I can add you to the dev channel of our discord, and we can set up a call to brainstorm if you'd find it useful.

Discord invite: https://discord.gg/72a9xh5Z7P

@rachittshah
Copy link

Hi @steventkrawczyk ! Joining discord, I think I'd open a draft PR with a proposed implementation of changing(since this would be a breaking change).

Joining discord, and keen to work on this!

@baggiponte
Copy link

+1 on this feature 😊 A client I work uses llama-index and there aren't notable libraries for LLM eval that are compatible. Yesterday deeplearning.ai released a course on RAG and LLM Eval using llama-index and truelens but I feel that promptools design would feel more natural and provide a simpler integration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants