Skip to content

Issues: stanford-crfm/helm

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

131 Open 939 Closed
131 Open 939 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Help me understand the HELM Classic Leaderboard's missing results user question
#2994 opened Sep 16, 2024 by PaulJoeMaliakel
1
Add metric tooltip to Predictions
#2987 opened Sep 11, 2024 by farzaank
Add amazon bedrock 3p models to the model configuration. user question
#2967 opened Sep 3, 2024 by subhaviv
2
Run.tsx fetches all run specs but we only need one
#2954 opened Aug 27, 2024 by farzaank
Incorrect scoring due to answer format mismatch in MMLU evaluation user question
#2939 opened Aug 16, 2024 by DerryChan
1
Add wrapper with consistent padding for MiniLeaderboards
#2933 opened Aug 15, 2024 by farzaank
@farzaank
Breaking change in nltk 3.8.2
#2926 opened Aug 14, 2024 by yifanmai
RAFT evaluation user question
#2909 opened Aug 8, 2024 by divyashan
2
Add Windows support
#2907 opened Aug 8, 2024 by yifanmai
Add sorting/filtering functionality to predictions pages frontend Frontend issues
#2887 opened Aug 1, 2024 by farzaank
Add metadata field to HELM predictions enhancement New feature or request frontend Frontend issues
#2886 opened Aug 1, 2024 by farzaank
2
VHELM images do not scale when window is resized
#2885 opened Aug 1, 2024 by yifanmai
HEIMHumanEvalScenario requires permissions to download data from codalab HEIM (Text2Image) user question
#2865 opened Jul 30, 2024 by slymane
1
pip needs to be temporarily downgraded to 24.1.2
#2855 opened Jul 30, 2024 by yifanmai
Update tutorial for helm-server
#2805 opened Jul 17, 2024 by farzaank
Benchmark Runner Error user question
#2803 opened Jul 16, 2024 by carina-ding
2
How can I use HELM to evaluate local model? user question
#2800 opened Jul 11, 2024 by Zoey384
1
Link to Image2Struct is broken user question
#2785 opened Jul 2, 2024 by zhimin-z
1
Sort by correctness functionality on Predictions page in front end
#2783 opened Jun 29, 2024 by farzaank
@farzaank
Chart runs out of colors when there are too many organizations frontend Frontend issues
#2764 opened Jun 21, 2024 by yifanmai
@farzaank
Annotator issues
#2697 opened May 31, 2024 by yifanmai
4 tasks
@farzaank
Support for Hugging Face Local Models
#2690 opened May 30, 2024 by NoushNabi
1
warning when on outdated site frontend Frontend issues good first issue Good for newcomers
#2681 opened May 24, 2024 by farzaank
@farzaank
Unable to download HELM leaderboard results (v1.3.0) user question
#2680 opened May 24, 2024 by jasonwright38
8
Change gsutil rsync to gcloud storage rsync in documentation documentation Improvements or additions to documentation
#2676 opened May 23, 2024 by yifanmai
ProTip! Follow long discussions with comments:>50.