Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: remove context_relevancy from library #1111

Merged
merged 2 commits into from
Jul 19, 2024

Conversation

jjmachan
Copy link
Member

No description provided.

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Jul 18, 2024
@shahules786 shahules786 merged commit 387c452 into explodinggradients:main Jul 19, 2024
15 checks passed
@xiaokening
Copy link

why do you remove context_relevancy from library? can you give me a explanation? thanks!

@jjmachan
Copy link
Member Author

that turned out to be not a useful metric, so it was under deprecation for a while. use this instead https://docs.ragas.io/en/stable/concepts/metrics/context_precision.html

@zhiwei2017
Copy link

Hey,
just want to comment on the best practises of semantic versioning, usually people remove functions in at least minor version, not patch version, even though there were deprecation warning for the removed functions. In patch versions, only the bugs should be fixed. Please check the documentation of semantic versioning

@jjmachan
Copy link
Member Author

@zhiwei2017 that is a good point, thanks for bringing it up. Will keep this in mind moving forward! 🙂

sorry for the inconvenience

@mongodben
Copy link

mongodben commented Aug 5, 2024

since this has been removed, it'd be great if you could have some other reference-free evaluation metric that looks at how relevant retrieved context is to answer an input.
currently, the only metrics that assess context require ground truths, which might not always be available or obvious.

Could you also expand on why context relevancy "turned out to be not a useful metric" meriting its removal?

@jjmachan
Copy link
Member Author

jjmachan commented Aug 5, 2024

@shahules786 could you help @mongodben out?

@jjmachan jjmachan deleted the chore/remove_cr branch August 5, 2024 18:36
@shahules786
Copy link
Member

Hi @mongodben thanks for the suggestions.
I just added #1174 to the documentation. We had it for a while but it wasn't displayed in doc making it almost invisible to users.
Try it out and let us know :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants