Fix: Add continue message at the end of a message sequence

[vria]

Problem

The latest text is crucially important to LLM. Currently this can be easily a random text that distracts model significantly.
For instance consider this message sequence:

Message 1 - system - <describing a persona>

Message 2 - user - <describing a task and tools>

Message 3 - assistant tool usage (read file)
Observation: <file content>

As you can see the latest text can be any random depending on what the file contains. This is applicable to any tool output: files, api calls, everything.

The problem is that most models tend to continue the latest content in their immediate response.

Request:

Message 1 - system - <describing a persona>

Message 2 - user - <describing a task and tools>

Message 3 - assistant tool usage (read file)
Observation:
...
...
...
These principles are important to our system:
- Single Responsibility Principle
- Open/Closed Principle

Response:

- Liskov Substitution Principle
- Interface Segregation Principle
- Dependency Inversion Principle

Thought: xxx
Action: xxx
Action Input: xxx

Note how LLM completed SOLID principles but it is NOT desirable.

In my case this was much worse, LLM completed file context with lots of unusable content that distracted the further conversation with LLM and lead to bad results.

Solution

Simply add ephemeral user's input as a last message.

[joaomdmoura]

Disclaimer: This review was made by a crew of AI Agents.

Code Review Comment for PR #3093

Overview

This PR introduces enhancements to the message handling for the Mistral model within the CrewAI framework. While the changes represent a step forward in managing message continuations, I have identified several areas for improvement, primarily centered around code efficiency, documentation, and testing considerations.

Code Quality Findings

src/crewai/agents/crew_agent_executor.py

• Message Chain Modification: The copy() method is being used to duplicate the entire message chain. This can be inefficient, especially for larger message histories. Instead, consider using references where possible to reduce memory overhead.

  • Improvement Suggestion:

    if len(messages) > 2:
        final_messages = messages + [self._continue_message()]
    else:
        final_messages = messages

• Conditional Logic Clarity: The condition checking if len(messages) > 2 lacks accompanying documentation. It’s essential to explain this choice to future maintainers.

  • Improvement Suggestion: Add a docstring to clarify the rationale behind the threshold.

src/crewai/llm.py

• Documentation Gaps: The removal of Mistral-specific message handling logic should be documented to prevent future regressions and to inform developers of recent changes in functionality.
  • Improvement Suggestion:

    """
    Mistral-specific message handling has been removed as of [VERSION].
    Ensure that Mistral model integrations are thoroughly validated.
    """

src/crewai/translations/en.json

• Consistency of Message Format: The newly added "continue" message format introduces slight deviations from existing formats. A standardized message format is vital for a cohesive user experience across translations.
  • Improvement Suggestion:

    {
      "slices": {
        "continue": "\nContinue!\n\nThought:"
      }
    }

Historical Context from Related PRs

While I wasn't able to fetch specific related PRs in this instance, I recommend reviewing the history of modifications to the crew_agent_executor.py for past approaches to message handling. Historical PRs could provide valuable insights into the evolution of the message processing logic.

Implications for Related Files

The changes in both crew_agent_executor.py and llm.py should be closely monitored to ensure interoperability and functionality, especially as we phase out Mistral-specific handling. Additionally, proper validation is necessary to safeguard against unintended regressions.

Specific Improvement Suggestions

1. Documentation Enhancements: Ensure that explanations for logic choices, especially in message handling, are clearly documented.
2. Testing Considerations:
   • Implement comprehensive unit tests for the _continue_message logic.
   • Include integration tests with Mistral models.
3. Code Optimization:
   • Explore the use of references for message chains rather than copies.
   • Consider developing a utility class dedicated to managing message operations to encapsulate related logic.

Conclusion

The proposed changes enhance the message-handling capabilities of the CrewAI framework but are in need of additional documentation and testing. Addressing the identified issues will both improve code quality and enhance maintainability moving forward. Thank you for your efforts, and I look forward to seeing these enhancements implemented!