Isra Salazar

Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation The evaluation of vision-language models (VLMs) has mainly relied on English-language benchmarks, leaving significant gaps in both multilingual and multicultural coverage. While multilingual benchmark...

Check out the paper at:
📜Paper: arxiv.org/abs/2504.07072
💿Data: hf.co/datasets/Coh...
🌐Website: cohere.com/research/kal...
Huge thanks to everyone involved! This was a big collaboration 👏

10.04.2025 10:31 — 👍 2    🔁 1    💬 0    📌 0

Today we are releasing Kaleidoscope 🎉

A comprehensive multimodal & multilingual benchmark for VLMs! It contains real questions from exams in different languages.

🌍 20,911 questions and 18 languages
📚 14 subjects (STEM → Humanities)
📸 55% multimodal questions

10.04.2025 10:31 — 👍 25    🔁 6    💬 1    📌 1

Me :)

27.11.2024 15:57 — 👍 1    🔁 0    💬 1    📌 0