DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation Paper • 2503.01622 • Published Mar 3, 2025
Growing Pains: Extensible and Efficient LLM Benchmarking Via Fixed Parameter Calibration Paper • 2604.12843 • Published Apr 15 • 1
A Matter of TASTE: Improving Coverage and Difficulty of Agent Benchmarks Paper • 2605.28556 • Published 7 days ago • 55
Holmes: Benchmark the Linguistic Competence of Language Models Paper • 2404.18923 • Published Apr 29, 2024
JuStRank: Benchmarking LLM Judges for System Ranking Paper • 2412.09569 • Published Dec 12, 2024 • 20
Benchmark Agreement Testing Done Right: A Guide for LLM Benchmark Evaluation Paper • 2407.13696 • Published Jul 18, 2024 • 5
Unitxt: Flexible, Shareable and Reusable Data Preparation and Evaluation for Generative AI Paper • 2401.14019 • Published Jan 25, 2024 • 23