Skip to content

Commit

Permalink
Merge pull request #1236 from Codium-ai/tr/tldr
Browse files Browse the repository at this point in the history
TLDR
  • Loading branch information
mrT23 committed Sep 16, 2024
2 parents 16763d8 + da398ce commit b447080
Show file tree
Hide file tree
Showing 3 changed files with 21 additions and 9 deletions.
5 changes: 5 additions & 0 deletions docs/docs/core-abilities/dynamic_context.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
## TL;DR

PR-Agent uses an **asymmetric and dynamic context strategy** to improve AI analysis of code changes in pull requests.
It provides more context before changes than after, and dynamically adjusts the context based on code structure (e.g., enclosing functions or classes).
This approach balances providing sufficient context for accurate analysis, while avoiding needle-in-the-haystack information overload that could degrade AI performance or exceed token limits.

## Introduction

Expand Down
8 changes: 4 additions & 4 deletions docs/docs/core-abilities/metadata.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ PR-Agent initially retrieves for each PR the following data:
By default, the first command that PR-Agent executes is [`describe`](https://pr-agent-docs.codium.ai/tools/describe/), which generates three types of outputs:

- PR Type (e.g. bug fix, feature, refactor, etc)
- PR Description - a bullet points summary of the PR
- Changes walkthrough - going file-by-file, PR-Agent generate a one-line summary and longer bullet points summary of the changes in the file
- PR Description - a bullet point summary of the PR
- Changes walkthrough - for each modified file, provide a one-line summary followed by a detailed bullet point list of the changes.

These AI-generated outputs are now considered as part of the PR metadata, and can be used in subsequent commands like `review` and `improve`.
This effectively enables multi-stage chain-of-thought analysis, without doing any additional API calls which will cost time and money.
Expand All @@ -25,7 +25,7 @@ For example, when generating code suggestions for different files, PR-Agent can

```
## File: 'src/file1.py'
### AI-generated changes summary:
### AI-generated file summary:
- edited function `func1` that does X
- Removed function `func2` that was not used
- ....
Expand All @@ -49,7 +49,7 @@ __old hunk__
...
```

(3) The entire PR files that where retrieved are also used to expand and enhance the PR context (see [Dynamic Context](https://pr-agent-docs.codium.ai/core-abilities/dynamic-context/)).
(3) The entire PR files that were retrieved are also used to expand and enhance the PR context (see [Dynamic Context](https://pr-agent-docs.codium.ai/core-abilities/dynamic-context/)).


(4) All the metadata described above represents several level of cumulative analysis - ranging from hunk level, to file level, to PR level, to organization level.
Expand Down
17 changes: 12 additions & 5 deletions docs/docs/core-abilities/self_reflection.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
## TL;DR

PR-Agent implements a **self-reflection** process where the AI model reflects, scores, and re-ranks its own suggestions, eliminating irrelevant or incorrect ones.
This approach improves the quality and relevance of suggestions, saving users time and enhancing their experience.
Configuration options allow users to set a score threshold for further filtering out suggestions.

## Introduction - Efficient Review with Hierarchical Presentation

### Introduction - fast review with hierarchical structure

Given that not all generated code suggestions will be relevant, it is crucial to enable users to review them in a fast and efficient way, allowing quick identification and filtering of non-applicable ones.

Expand All @@ -9,9 +15,10 @@ To achieve this goal, PR-Agent offers a dedicated hierarchical structure when pr
- Each suggestion is first described by a one-line summary, which can be expanded to a full description by clicking on a collapsible.
- Upon expanding a suggestion, the user receives a more comprehensive description, and a code snippet demonstrating the recommendation.

This hierarchical structure is designed to facilitate rapid review of each suggestion, with users spending an average of ~5-10 seconds per item.
!!! note "Fast Review"
This hierarchical structure is designed to facilitate rapid review of each suggestion, with users spending an average of ~5-10 seconds per item.

### Self-reflection and re-ranking
## Self-reflection and Re-ranking

The AI model is initially tasked with generating suggestions, and outputting them in order of importance.
However, in practice we observe that models often struggle to simultaneously generate high-quality code suggestions and rank them well in a single pass.
Expand All @@ -30,13 +37,13 @@ Note that presenting all generated suggestions simultaneously provides the model
To conclude, the self-reflection process enables PR-Agent to prioritize suggestions based on their importance, eliminate inaccurate or irrelevant proposals, and optionally exclude suggestions that fall below a specified threshold of significance.
This results in a more refined and valuable set of suggestions for the user, saving time and improving the overall experience.

### Example results
## Example Results

![self_reflection](https://codium.ai/images/pr_agent/self_reflection1.png){width=768}
![self_reflection](https://codium.ai/images/pr_agent/self_reflection2.png){width=768}


### Appendix - relevant configuration options
## Appendix - Relevant Configuration Options
```
[pr_code_suggestions]
self_reflect_on_suggestions = true # Enable self-reflection on code suggestions
Expand Down

0 comments on commit b447080

Please sign in to comment.