Skip to content

Pull requests: huggingface/lighteval

New pull request New
100 Open 665 Closed
100 Open 665 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Fix callable type hint in parallelism helper
#1239 opened May 20, 2026 by GoparapukethaN Loading…
fix: guard choices[0] and message=None before content access in llm_as_judge
#1238 opened May 17, 2026 by qizwiz Loading…
docs: fix custom model examples
#1237 opened May 15, 2026 by MukundaKatta Loading…
1
fix: prevent IndexError in Doc.get_golds() for out-of-bounds gold_index
#1236 opened May 13, 2026 by AmSach Loading…
1
fix typo
#1235 opened May 8, 2026 by fpetrakov Loading…
fix: transpose references before passing to sacrebleu in CorpusLevelTranslationMetric
#1234 opened May 8, 2026 by jaydenC88 Loading…
2 tasks done
1
Popotest patch 1
#1231 opened May 5, 2026 by popotest Loading…
8
test: style-bot trigger
#1221 opened May 4, 2026 by paulinebm Contributor Loading…
14
Add Bayes@N metric
#1219 opened Apr 29, 2026 by mohsenhariri Loading…
Log per-sample details as trackio.Trace in push_to_wandb
#1217 opened Apr 27, 2026 by abidlabs Member Loading…
1
Add LICA-Bench: graphic design VLM evaluation (39 tasks, 7 domains)
#1212 opened Apr 15, 2026 by purvanshi Loading…
3 of 4 tasks
1
POLLUX LLM-Judge metric
#1210 opened Apr 10, 2026 by ulyanaisaeva Loading…
1
catch task has no docs instead of throw
#1207 opened Apr 8, 2026 by BuiHoangTu Loading…
add multilingual flag to vllm
#1206 opened Apr 8, 2026 by BuiHoangTu Loading…
fix(vllm): Enhance VLLMModel context size handling for batch inputs
#1205 opened Apr 6, 2026 by paulovsantanas Loading…
1
examples: add RAIL Score responsible AI custom task template
#1203 opened Apr 2, 2026 by SumitVermakgp Loading…
Add --load-tasks-multilingual and fix --custom-tasks for inspect backend
#1199 opened Mar 25, 2026 by dzautner Loading…
4 tasks done
[Bugfix] Check all responses when n>1 instead of only the first one
#1197 opened Mar 23, 2026 by eldarkurtic Contributor Loading…
[Litellm Enhancement] Enable extra sampling args for litellm backend
#1195 opened Mar 20, 2026 by eldarkurtic Contributor Loading…
[Bugfix] presence_penalty is silently dropped from sampling args in litellm backend
#1193 opened Mar 18, 2026 by eldarkurtic Contributor Loading…
[Bugfix] litellm backend should iterate over docs in a split not entire dataset
#1192 opened Mar 18, 2026 by eldarkurtic Contributor Loading…
Remove deprecated prompt_token_ids wrapping in vLLM backend
#1191 opened Mar 18, 2026 by sihyeonn Loading…
Fix litellm connection pool limiting concurrent_requests
#1190 opened Mar 18, 2026 by sihyeonn Loading…
feat(utils): show count of evaluated samples in Markdown summary table
#1188 opened Mar 13, 2026 by anzzyspeaksgit Loading…
1
Fix typos in math_comparison.py and sample_comparison.py
#1186 opened Mar 12, 2026 by joshuaswanson Loading…
ProTip! Adding no:label will show everything without a label.