GMLRVigil/BenchCheck-FramesPixels
Preview • Updated • 40
GMLR
university
AI & ML interests
None defined yet.
Recent Activity
View all activity
Enxin
updated a
dataset about 21 hours ago
Enxin
published a
dataset about 22 hours ago
Enxin
updated 2
datasets about 23 hours ago
Enxin
published 2
datasets about 23 hours ago
Enxin
published a
dataset 9 days ago
Enxin
updated a
dataset 10 days ago
YaoTang23
authored a
paper 8 months ago
SuhaoYu1020
authored a
paper 12 months ago
Enxin
authored 3
papers about 1 year ago
Devil in the Number: Towards Robust Multi-modality Data Filter
Paper • 2309.13770 • Published
AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark
Paper • 2410.03051 • Published • 6
Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmark
Paper • 2504.14693 • Published
YaoTang23
authored 2
papers about 1 year ago
Post
1616
🎉 Introducing Video-MMLU, a new benchmark for evaluating large multimodal models on classroom-style lectures in math, physics, and chemistry!
🧑🏫📚Video-MMLU requires strong reasoning capabilities and world knowledge compared to the previous benchmarks for video LMMs.
Each video comes with two tasks:
📝 Take Notes — detailed captioning of multi-discipline lectures
🧠 Do Quiz — open-ended QA to test reasoning over visuals & proofs
We evaluated 90+ models, including vision-blind baselines, open-source models and proprietary ones.
📉 We find that existing models generally perform poorly, with accuracy ranging from only 10% to 50%.
📉We also explore how the number of visual tokens and the base LLMs influence performance, offering insights into the interplay between multimodal perception and reasoning in lecture comprehension.
For more details, please check below：
📄 Paper: https://arxiv.org/abs/2504.14693
💻 Code: https://github.com/Espere-1119-Song/Video-MMLU
🧠 Data: Enxin/Video-MMLU
🌐 Website: https://enxinsong.com/Video-MMLU-web/
🧑🏫📚Video-MMLU requires strong reasoning capabilities and world knowledge compared to the previous benchmarks for video LMMs.
Each video comes with two tasks:
📝 Take Notes — detailed captioning of multi-discipline lectures
🧠 Do Quiz — open-ended QA to test reasoning over visuals & proofs
We evaluated 90+ models, including vision-blind baselines, open-source models and proprietary ones.
📉 We find that existing models generally perform poorly, with accuracy ranging from only 10% to 50%.
📉We also explore how the number of visual tokens and the base LLMs influence performance, offering insights into the interplay between multimodal perception and reasoning in lecture comprehension.
For more details, please check below：
📄 Paper: https://arxiv.org/abs/2504.14693
💻 Code: https://github.com/Espere-1119-Song/Video-MMLU
🧠 Data: Enxin/Video-MMLU
🌐 Website: https://enxinsong.com/Video-MMLU-web/
Enxin
authored a
paper almost 2 years ago
Enxin
authored a
paper about 3 years ago