This project leverages Mistral AI models to create an interactive emotional AI companion, aimed at supporting the elderly and those with memory challenges. The AI Companion uses the Pixtral-12B model to process chat-based queries and image-based descriptions. Additionally, we have integrated a Telegram Bot for easy and personalized interactions.
We have also fine-tuned Mistral Large using the Empathetic Dialogues dataset to create a more empathetic conversational agent (API fine-tuning for Pixtral-12B is not yet available).
- Text-Based Chat: Interact with the AI using natural language. The bot is designed to provide empathetic and context-aware responses to support users.
- Image Analysis: Get AI-powered descriptions of images. The bot can analyze photos, such as old family pictures, to provide descriptive and emotional insight.
- Memory Integration: The bot maintains an ongoing conversation history, allowing for more personalized interactions.
- Telegram Bot: The AI Mind Companion is also available as a Telegram bot for easy access and direct conversations on any device. The bot can handle both text and images seamlessly.
- Dynamic Name & Background Setting: Users can dynamically set their name and provide additional background information to make responses more personalized.
- Voice Interaction: Voice-to-text functionality is available for text-based interactions through the terminal, providing enhanced accessibility.
git clone <your-repo-url>
cd <your-repo-directory>
Create a Python virtual environment and activate it.
python3 -m venv pixtral-env
source pixtral-env/bin/activate # For Mac/Linux
Install the required Python packages using:
pip install -r requirements.txt
You’ll need an API key from Mistral AI. Once you have it, set it as an environment variable:
export MISTRAL_API_KEY="your_api_key" # For Mac/Linux
To use the Telegram bot functionality, you'll need a bot token from Telegram. Set it as an environment variable:
export TELEGRAM_TOKEN="your_telegram_bot_token" # For Mac/Linux
To start the application, simply run:
python main.py
- Interactive Chat: Run the application, and you will be prompted to choose between a text-based interaction or using the Telegram bot.
- Telegram Bot: You can use the Telegram bot for more convenient access. Just search for the bot using the handle provided (e.g.,
@AI_Mind_Companion_Bot
) and start chatting or sending images for analysis. - Image Analysis: Upload an image, and the AI will generate a detailed description and emotional context.
- Voice Interaction: Voice input is supported for text-based prompts in the terminal, providing a hands-free experience for users.
Pull requests are welcome. For significant changes, please open an issue first to discuss what you would like to change.
- Enhanced Emotional Analysis: Further improve the AI's emotional understanding to provide more nuanced responses.
- Telegram Voice Integration: Connect the voice interaction to the Telegram bot for enhanced accessibility.
- Extended Multi-Language Support: Add support for additional languages to reach a broader audience.
For any questions or suggestions, please feel free to reach out via the repository's issue tracker or by contacting the bot's developer directly. You can see more here: https://devpost.com/software/companionai#updates