McGill NLP

A blizzard is raging through Montreal when your friend says “Looks like Florida out there!” Humans easily interpret irony, while LLMs struggle with it. We propose a 𝘳𝘩𝘦𝘵𝘰𝘳𝘪𝘤𝘢𝘭-𝘴𝘵𝘳𝘢𝘵𝘦𝘨𝘺-𝘢𝘸𝘢𝘳𝘦 probabilistic framework as a solution.
Paper: arxiv.org/abs/2506.09301 to appear @ #ACL2025 (Main)

"Build the web for agents, not agents for the web"

This position paper argues that rather than forcing web agents to adapt to UIs designed for humans, we should develop a new interface optimized for web agents, which we call Agentic Web Interface (AWI).

arxiv.org/abs/2506.10953

Excited to share the results of my recent internship!

We ask 🤔
What subtle shortcuts are VideoLLMs taking on spatio-temporal questions?

And how can we instead curate shortcut-robust examples at a large-scale?

We release: MVPBench

Details 👇🔬

Exciting work on hallucinations from @ziling-cheng.bsky.social

Incredibly proud of my students @adadtur.bsky.social and Gaurav Kamath for winning a SAC award at #NAACL2025 for their work on assessing how LLMs model constituent shifts.

Congratulations to Mila members @adadtur.bsky.social , Gaurav Kamath and @sivareddyg.bsky.social for their SAC award at NAACL! Check out Ada's talk in Session I: Oral/Poster 6. Paper: arxiv.org/abs/2502.05670

Presenting ✨ 𝐂𝐇𝐀𝐒𝐄: 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐧𝐠 𝐜𝐡𝐚𝐥𝐥𝐞𝐧𝐠𝐢𝐧𝐠 𝐬𝐲𝐧𝐭𝐡𝐞𝐭𝐢𝐜 𝐝𝐚𝐭𝐚 𝐟𝐨𝐫 𝐞𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧 ✨

Work w/ fantastic advisors Dima Bahdanau and @sivareddyg.bsky.social

Thread 🧵:

Overview figure for paper, showing creation of constituent movement data, in addition to three step experimentation: "Model Shifting Preference", "Motivating Factors of Model Preference", "Human-Model Preference Correlation"

Super excited to finally announce our NAACL 2025 main conference paper “Language Models Largely Exhibit Human-like Constituent Ordering Preferences”!

We examine constituent ordering preferences between humans and LLMs; we present two main findings… 🧵

At McGill we have an NLP lab that works on a lot of things, from human-AI collaboration, to evaluation, to low resource NLP (me).

@emnlpmeeting.bsky.social just happened in Miami, and my colleagues just presented six papers there:

Thank you for trying again! I haven't a solution to the search issue and might contact support soon. Will let you know once we're indexed!

The Causal Influence of Grammatical Gender on Distributional Semantics How much meaning influences gender assignment across languages is an active area of research in linguistics and cognitive science. We can view current approaches as aiming to determine where gender as...

The Causal Influence of Grammatical Gender on Distributional Semantics
Featuring @karstanczak.bsky.social

arxiv.org/abs/2311.18567

Benchmarking Vision Language Models for Cultural Understanding Foundation models and vision-language pre-training have notably advanced Vision Language Models (VLMs), enabling multimodal processing of visual and linguistic data. However, their performance has bee...

Benchmarking Vision Language Models for Cultural Understanding
Featuring @karstanczak.bsky.social

arxiv.org/abs/2407.10920

Does This Summary Answer My Question? Modeling Query-Focused Summary Readers with Rational Speech Acts Query-focused summarization (QFS) is the task of generating a summary in response to a user-written query. Despite its user-oriented nature, there has been limited work in QFS in explicitly considerin...

Does This Summary Answer My Question? Modeling Query-Focused Summary Readers with Rational Speech Acts
By @cesare-spinoso.bsky.social

arxiv.org/abs/2411.06524

It turns out we had even more papers at EMNLP!

Let's complete the list with three more🧵

From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models Mehar Bhatia, Sahithya Ravi, Aditya Chinchure, EunJeong Hwang, Vered Shwartz. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models
By @meharbhatia.bsky.social

aclanthology.org/2024.emnlp-m...

Social Bias Probing: Fairness Benchmarking for Language Models Marta Marchiori Manerba, Karolina Stanczak, Riccardo Guidotti, Isabelle Augenstein. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

Social Bias Probing: Fairness Benchmarking for Language Models
By @karstanczak.bsky.social

aclanthology.org/2024.emnlp-m...

From Insights to Actions: The Impact of Interpretability and Analysis Research on NLP Marius Mosbach, Vagrant Gautam, Tomás Vergara Browne, Dietrich Klakow, Mor Geva. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

From Insights to Actions: The Impact of Interpretability and Analysis Research on NLP
By @mariusmosbach.bsky.social

aclanthology.org/2024.emnlp-m...

Our lab members recently presented 3 papers at @emnlpmeeting.bsky.social in Miami ☀️ 📜

From interpretability to bias/fairness and cultural understanding -> 🧵

Hello 👋 could you add us? Great initiative!