Adina Williams

This is super cool! I had been contemplating starting a project like this a few years ago to actualize a vague interest I have in the math of knots... but this benchmark is probably much better than I could've managed. Can't wait to take a closer look 🪢!

We've create a simple and easy framework for documenting AI evaluation methods! Check it out and submit some for your own datasets with a PR!

You can learn more here: github.com/facebookrese...

And here: arxiv.org/abs/2512.04062

My spouse is doing a week of science communication (in German) this week.

Topics may include cognitive science, linguistics, AI, political economy, science and technology studies, maybe even some philosophy too.

Check it out👨‍🔬!

Yes! Social accountability seems to have been a major part of the process...if you get a ping telling you you're late from an (S)AC you know, you're much more likely to feel embarassed, drop everything, and do what you agreed to do...the fields growing pangs are a challenge for sure!

I did once (arxiv.org/pdf/2311.18567) if you count this as compling paper (and not NLP). I know my coauthor Ryan has continued thinking about Judea Pearl style causality more afterwards too probably.

Like gen1 may just be comitative. Which also had me doubting whether he1 really corresponds to 'and' either... but i suspect Haspelmath has tests or at least refs to some in that link.

I stared trying to construct an argument that maybe mandarin differentiates NP (he2) and DP (gen1) conjunction. Then I tried to think about basic coordination of mandarin VPs (may only be periphrastic)... so perhaps this is a next reasonable place to point you to instead:
wals.info/chapter/64

Top: A syntax tree for the sentence "the doctor by the lawyer saw the artist". Bottom: A continuous vector.

🤖🧠I'll be considering applications for PhD students & postdocs to start at Yale in Fall 2026!

If you are interested in the intersection of linguistics, cognitive science, & AI, I encourage you to apply!

PhD link: rtmccoy.com/prospective_...
Postdoc link: rtmccoy.com/prospective_...

Screenshot of a figure with two panels, labeled (a) and (b). The caption reads: "Figure 1: (a) Illustration of messages (left) and strings (right) in toy domain. Blue = grammatical strings. Red = ungrammatical strings. (b) Surprisal (negative log probability) assigned to toy strings by GPT-2."

New work to appear @ TACL!

Language models (LMs) are remarkably good at generating novel well-formed sentences, leading to claims that they have mastered grammar.

Yet they often assign higher probability to ungrammatical strings than to grammatical strings.

How can both things be true? 🧵👇

Nice! Might this shed some light on unnatural language processing effects? Thinking of older work I was doing with @koustuvsinha.com. We found examples where word reordering causing ungrammaticality could improve classifier results, and much discussion followed. Could be fun to think these together!

Please respond to this survey if you have changed or have thought about changing your name in academic publishing! For any reason, whether it be transition, recognizability, marriage, privacy, immigration, cultural reasons, etc.

Please RT for reach :)

UT Austin Linguistics is hiring in computational linguistics!

Asst or Assoc.

We have a thriving group sites.utexas.edu/compling/ and a long proud history in the space. (For instance, fun fact, Jeff Elman was a UT Austin Linguistics Ph.D.)

faculty.utexas.edu/career/170793

🤘

Deflating “Hype” Won’t Save Us The problem with AI isn’t hype. The problem is who and what it’s useful for.

We wrote a thing about AI, fascism, and why framing this as "hype" is too apolitical

www.liberalcurrents.com/deflating-hy...

I agree this thread's headline claim seems premature. Let me add our recent ACL Findings paper, with Dexter Ju and @hagenblix.bsky.social, which found syntactic simplification in at least some LMs, in a novel domain regeneration setting: aclanthology.org/2025.finding...

Our team is hiring a postdoc in (mechanistic) interpretability! The ideal candidate will have research experience in interpretability for text and/or image generation models and be excited about open science!

Please consider applying or sharing with colleagues: metacareers.com/jobs/2223953961352324

Ingeborg and I wrote a thing about "hype", and why we think that framing AI through that lens is increasingly inadequate - check it out!

My spouse's new book has a chapter getting into the weeds around AI and high paid jobs...AI powered deskilling, drawing historical connections back as far as the steam engine, economic reasoning pushing technical innovation etc. Check it 👇 I think you might like it.

bsky.app/profile/hage...

Have you heard about this year's shared task? 📢

Mechanistic Interpretability (MI) is quickly advancing, but comparing methods remains a challenge. This year at #BlackboxNLP, we're introducing a shared task to rigorously evaluate MI methods in language models 🧵

Excited to share the IntPhys 2.0 benchmark for intuitive physics understanding of video! It's super challenging for models but not humans, making it a great testbed for pushing modeling improvements.

Check it out: ai.meta.com/research/pub...

Come by to our panel at APS to share your thoughts, and ask us all the hard stuff!

Check it out! Guy et al. explores the impact of format on function vectors, and invites further conversation about what it would mean to have universal goal representations in LLMs.

(I've hung around interp communities for a while but this is my 1st mech-interp project. feedback much appreciated!)

COLM 2025 workshops are announced now, check it out: colmweb.org/Workshops.html

Awesome, can't wait to read it; congrats Maya!

Book Launch: Auto-Correct: The Fantasies and Failures of AI, Ethics, and the Driverless Car · Luma A conversation with Maya Indira Ganesh on how driverless cars are reshaping governance, responsibility, and values What can a driverless car tell us about…

Pls come to my book launch on May 22, 5pm, at Newnham College Cambridge cohosted w @mctd.bsky.social and @cfi-cambridge.bsky.social and with stellar discussants @gsvoss.bsky.social and Jennifer Schooling Register here: lu.ma/9vm405sk

This is such a fun example of LM weirdness (which also shows how they match form over fact!)

More linguistically: it looks like ending a query with "meaning" triggers the bot to accommodate the presupposition that the input contains an idiom! (Hard to run normal preposition tests here tho)

Foundations of AI — Initiative for the Theoretical Sciences Friday, 11 April 9:30 AM - 4:00 PM Rooms 9206/9207, Graduate Center CUNY Join us at the Initiative for Theoretical Sciences for an interdisciplinary exploration of the foundations of artificial i...

Happy to be at CUNY today to workshop about the theoretical foundations of AI! Fun to have several perspectives---math, philosophy, linguistics and more---together in one place!

w/@giannig.bsky.social @seiller.bsky.social J. Terilla et al.

itsatcuny.org/calendar/202...

Fantastic news, congrats to you and to BU 🎉

Content I'm so here for 🤩

AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons The rapid advancement and deployment of AI systems have created an urgent need for standard safety-evaluation frameworks. This paper introduces AILuminate v1.0, the first comprehensive industry-standa...

Happy to share that the paper describing the AILuminate v1.0 benchmark is now out! arxiv.org/abs/2503.05731
The benchmark is designed with @mlcommons.org to assess LLM risk and reliability across 12 hazard categories. AILuminate is available for testing models and helping ensure safer deployment!