Isabel Papadimitriou @ ICML 🌄🌲🐋

Thinking fast, slow, and everywhere in between in humans and language models Author(s): Prystawski, Ben; Goodman, Noah | Abstract: How do humans adapt how they reason to varying circumstances? Prior research has argued that reasoning comes in two types: a fast, intuitive type ...

How do people trade off between speed and accuracy in reasoning tasks without easy heuristics? Come to my talk, "Thinking fast, slow, and everywhere in between in humans and language models," in the Reasoning session this afternoon #CogSci2025 to find out!
paper: escholarship.org/uc/item/5td9...

Idiosyncratic but not opaque: Linguistic conventions formed in reference games are interpretable by naïve humans and vision–language models Author(s): Boyce, Veronica; Prystawski, Ben; Tan, Alvin Wei Ming; Frank, Michael C. | Abstract: When are in-group linguistic conventions opaque to non-group members (teen slang like "rizz") or general...

When people form conventions in reference games, how easy are they for outsiders to interpret? (for values of "outsider" that include naïve humans and vision-language models) Check out @vboyce.bsky.social's poster today at #CogSci2025 to find out.
paper: escholarship.org/uc/item/16c4...

@antararb.bsky.social is applying for PhDs this fall! She’s super impressive and awesome to work with, and conceived of this project independently and carried it out very successfully! Keep an eye out 🙂

Investigating the interaction of linguistic and mathematical reasoning in language models using multilingual number puzzles Across languages, numeral systems vary widely in how they construct and combine numbers. While humans consistently learn to navigate this diversity, large language models (LLMs) struggle with linguist...

More in the preprint! arxiv.org/abs/2506.13886 This project was led by Antara, with @dmelis.bsky.social and Kate Davidson

So is it really this implicit operators thing that’s tripping them up? We try many other ablations, looking at the effect of giving extra context in the prompt, using numbers vs words, left-to-right ordering, and subtractive systems, and none of them seem to affect the models that much.

Our experiments are based on Linguistics Olympiad problems that deal with number systems, like the one here. We created additional hand-standardized versions of each puzzle in order to be able to do all of the operator ablations.

This shows the types of reasoning and variable binding jumps that are hard for LMs. It’s hard to go one level up, and bind a variable to have the meaning of an operator, or to understand that an operator is implicit.

If we alter the problems to make the operators explicit, the models can solve these problems pretty easily. But it’s still harder to bind a random symbol or word to mean an operator like +. It’s much easier when we use the familiar symbols for the operators, like + and x.

Our main finding: LMs find it hard when *operators* are implicit. We don’t say “5 times 100 plus 20 plus 3”, we say “five hundred and twenty-three”. The Linguistics Olympiad puzzles are pretty simple systems of equations that an LM should solve – but the operators aren’t explicit.

Why can’t LMs solve puzzles about the number systems of languages, when they can solve really complex math problems? Our new paper, led by @antararb.bsky.social looks at why this intersection of language and math is difficult, and what this means for LM reasoning! arxiv.org/abs/2506.13886

ACL paper alert! What structure is lost when using linearizing interp methods like Shapley? We show the nonlinear interactions between features reflect structures described by the sciences of syntax, semantics, and phonology.

Committee selfie!

Congrats to Veronica Boyce on her dissertation defense! That’s three amazing talks by three great students in 8 days!

(the unfortunate truth is that I am really enjoying this mac and its battery life oops)

This work Mac (my first ever) is great because every time something seriously breaks, instead of becoming distressed and despondent like I usually do, it's just like "ooooooh yeahhh, yet another win for team Linux 😎😎😎🎉🐧"

😼SMOL DATA ALERT! 😼Anouncing SMOL, a professionally-translated dataset for 115 very low-resource languages! Paper: arxiv.org/pdf/2502.12301
Huggingface: huggingface.co/datasets/goo...

New paper in Psychological Review!

In "Causation, Meaning, and Communication" Ari Beller (cicl.stanford.edu/member/ari_b...) develops a computational model of how people use & understand expressions like "caused", "enabled", and "affected".

📃 osf.io/preprints/ps...
📎 github.com/cicl-stanfor...
🧵

Where are all of the phoneticians of the Boston area and why isn't there a storied subfield of fieldwork studying the Cambridge shopkeeper who seems to have a mix between a West Country (rhotic English!) and a Boston (non-rhotic American!) accent.

Apparently the shop's been open for decades, smh

Quanta write-up of our Mission: Impossible Language Models work, led by @juliekallini.bsky.social. As the photos suggest, Richard, @isabelpapad.bsky.social, and I do all our work sitting together around a single laptop and pointing at the screen.

My most controversial take is that you should never use commit -m, just let it open the damn vim file, let yourself think for a second, and then write something descriptive

I will be at NeurIPS starting tomorrow! Would love to chat about interpretability, linguistics, language structure, meaning in LLMs. Reach out!

Aaaand if you love Vancouver, apply to do a PhD at UBC! Fun research in a lovely place! linguistics.ubc.ca/graduate/adm...

Aerial picture of the UBC campus, with an arrow pointing to a building and text asking "Your PhD lab?"

Do you want to understand how language models work, and how they can change language science? I'm recruiting PhD students at UBC Linguistics! The research will be fun, and Vancouver is lovely. So much cool NLP happening at UBC across both Ling and CS! linguistics.ubc.ca/graduate/adm...