Multilingual Representation Workshop @ EMNLP 2025

Announcing the two Best Paper Awards for this year’s workshop. Congratulations to all the authors for your great work!

Finally, we recognize the Hawaiian submission to the shared task. Thank you for your contributions!

The second is 7PERFECTION, a dataset for seven Nigerian languages!

The first of our best contribution awards goes to AraPIQA!

We would like to recognize three honorable mention submissions. Great work!

This afternoon, @catherinearnett.bsky.social presented the result of the shared task and presented the best contribution awards. Congratulations to all contributors!

@aliceatkaist.bsky.social joins us to give the final keynote of the day about code switching in multilingual language models!

Join us in hall C3 at posters 137-168 for our in-person poster session. Join us online on Zoom via Underline for our virtual poster session!

Now Pontus Stenetorp shares an oral history of UK-LLM!

Research poster of the paper "Sub-1B Language Models for Low-resource Languages: Training Strategies and Insights for Basque."

Tomorrow, I'll be presenting (virtually) our research from @orainlp.bsky.social on pre-training SLMs for low-resource languages as a poster during the @mrl-workshop.bsky.social.

Come check it out!

📝 aclanthology.org/2025.mrl-mai...

@kellymarchisio.bsky.social from Cohere presents “Building Multilingual LLMs in Industry”, sharing insights on training multilinguality at scale!

We have kicked off proceedings with some brief opening remarks from @catherinearnett.bsky.social

We are kicking off this year’s workshop in Suzhou at #EMNLP2025! Come join us in room A106-107 or online!

Global PIQA: Evaluating Physical Commonsense Reasoning Across 100+ Languages and Cultures To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we present Global PIQA, a ...

Preprint: arxiv.org/abs/2510.24081
Dataset: huggingface.co/datasets/mrl...

Global PIQA Contributor Interest Form Thanks for your interest in contributing to Global PIQA! Please fill out the form and we will contact you with details about how to get involved!

It’s not too late to get involved! Until early 2026, we will be accepting submissions for languages not already represented in Global PIQA. If you’re interested, please fill out this form and we will contact you with details!
docs.google.com/forms/d/e/1F...

There are seven languages where even the best proprietary LLM scores less than 80% (chance: 50%). Sub-Saharan African languages lag behind Western European languages by ~15%. Thus Global PIQA highlights languages which are very poorly served by large, proprietary models.

The top proprietary models achieve ~90% accuracy, which falls short of human accuracy (~95%). The best open models perform significantly worse, with the best open model performance from Gemma 3 (27B) at 82.4%.

This dataset is created and owned by the contributors, all of whom were offered authorship. We believe this is more fair to annotators and is likely to result in a higher-quality dataset, as it is constructed by the NLP researchers who will use it.

Global PIQA includes subsets for 116 unique language varieties. These cover five continents, 14 language families, and 23 writing systems. Over 50% of examples reference local foods, customs, traditions, or other culturally-specific elements.

Introducing Global PIQA, a new multilingual benchmark for 100+ languages. This benchmark is the outcome of this year’s MRL shared task, in collaboration with 300+ researchers from 65 countries. This dataset evaluates physical commonsense reasoning in culturally relevant contexts.

We are in need of some emergency reviewers for MRL. If you are available, please fill out this form!

sigtyp.github.io/ws2025-mrl.h...

This year MRL is also accepting papers that have been submitted to ARR and have received reviews and a metareview! Submit your papers by September 23rd! See the workshop website for details on how to submit ⬇️

Correct, you can submit ARR papers that already have their reviews by Sep 23. More instructions soon!

TBD on whether the workshop will be hybrid.

We extended the deadline by one day, so you have until the end of today (Aug 24) AoE to submit! Good luck!

MRL 2025 Shared Task Info Meeting MRL 2025 Shared Task at EMNLP: Multilingual Physical Commonsense Reasoning Datasets Info meeting, 2025/08/14 Contact: mrl2025-workshop@googlegroups.com Last updated: 2025/08/14

Check out more information, including answers to FAQs: docs.google.com/presentation...

MRL Shared Task volunteers Call for participation to create physical reasoning datasets for various languages! While there has been much progress in developing benchmarks for diverse languages, we still have very few multiling...

If you plan to participate, fill in this google form so we can better plan the shared task: forms.gle/zxhpCfL6wvBz...

We have over 200 volunteers now for 90+ languages! We are hoping to expand the diversity of our language coverage and are still looking for participants who speak these languages. Check out how to get involved below, and please help us spread the word!

The deadline for MRL at #EMNLP2025 is next week!

⏰ Submission Deadline: August 23rd (AoE)

🔗 CfP: sigtyp.github.io/ws2025-mrl.h...

See the shared task page for more information: sigtyp.github.io/st2025-mrl.h...