This is a Rest-Backend for a Conversational Agent, that allows to embedd Documentes, search for them using Semantic Search, to QA based on Documents and do document processing with Large Language Models.
- Changed to LiteLLM to allow for every model provider. Default is Cohere for Embedding and Google AI Studio Gemini for Generation.
I have decided to stop creating different services for different provider and switchting to LiteLLM which allows to use basically every provider you want.
Some providers i would recommend are:
- Cohere Awesome models and great free tier.
- Ollama If you want to keep your data your data.
- Google AI Studio The Google Integration that is not really suited for enterprise but perfect for everybody else.
To run the complete system with docker use this command:
git clone https://github.com/mfmezger/conversational-agent-langchain.git
cd conversational-agent-langchainCreate a .env file from the .env-template and set the necessary API Keys. Absolutely necessary are:
- GEMINI_API_KEY
- COHERE_API_KEY
Then start the system with
docker compose up -dThen go to http://127.0.0.1:8001/docs or http://127.0.0.1:8001/redoc to see the API documentation.
Frontend: localhost:8501 Qdrant Dashboard: localhost:6333/dashboard
This project is a conversational rag agent that uses Google Gemini Large Language Models to generate responses to user queries. The agent also includes a vector database and a REST API built with FastAPI.
Features
- Uses Large Language Models to generate responses to user queries.
- Includes a vector database to store and retrieve information.
- Provides a REST API built with FastAPI for easy integration with other applications.
- Has a basic gui.
Semantic search is an advanced search technique that aims to understand the meaning and context of a user's query, rather than matching keywords. It involves natural language processing (NLP) and machine learning algorithms to analyze and interpret user intent, synonyms, relationships between words, and the structure of content. By considering these factors, semantic search improves the accuracy and relevance of search results, providing a more intuitive and personalized user experience.
For Hybrid Search the BM25 FastEmbedd from Qdrant is used.
On Linux or Mac you need to adjust your /etc/hosts file to include the following line:
127.0.0.1 qdrantFirst install Python Dependencies:
You need to install uv if you want to use it for syncing the requirements.lock file. UV Installation.
uv syncIn src/agent/scripts use the load dummy data script to load some example data in the rag.
Start the complete system with:
docker compose up -dTo run the Qdrant Database local just run:
docker compose up qdrantTo run the Backend use this command in the root directory:
poetry run uvicorn agent.api:app --reloadTo run the tests you can use this command:
poetry run coverage run -m pytest -o log_cli=true -vvv testsTo run the Frontend use this command in the root directory:
poetry run streamlit run gui.py --theme.base="dark"mypy rag --explicit-package-bases
Qdrant Dashboard is available at http://127.0.0.1:6333/dashboard. There you need to enter the api key.
To use the Qdrant API you need to set the correct parameters in the .env file. QDRANT_API_KEY is the API key for the Qdrant API. And you need to change it in the qdrant.yaml file in the config folder.
If you want to ingest large amount of data i would recommend you use the scripts located in agent/ingestion.
To Test the API i would recommend Bruno. The API Requests are store in ConvAgentBruno folder.