Martin Tutek

Can you solve this algebra puzzle? 🧩

cb=c, ac=b, ab=?

A small transformer can learn to solve problems like this!

And since the letters don't have inherent meaning, this lets us study how context alone imparts meaning. Here's what we found:🧵⬇️

⏳ Deadline approaching! We’re hiring 2 fully funded postdocs in #NLP.

Join the MilaNLP team and contribute to our upcoming research projects (SALMON & TOLD)

🔗 Details + how to apply: milanlproc.github.io/open_positio...

⏰ Deadline: Jan 31, 2026

I'd like a link as well!

Nathan Stringham, Fateme Hashemi Chaleshtori, Xinyuan Yan, Zhichao Xu, Bei Wang, Ana Marasovi\'c
Teaching People LLM's Errors and Getting it Right
https://arxiv.org/abs/2512.21422

🚀 We’re opening 2 fully funded postdoc positions in #NLP!

Join the MilaNLP team and contribute to our upcoming research projects.

🔗 More details: milanlproc.github.io/open_positio...

⏰ Deadline: Jan 31, 2026

Llama enjoying a mug of hot cocoa in an office with Tuesday, March 31 circled on a calendar behind them

COLM 2026 is just around the corner! Mark your calendars for:

💡 Abstract deadline: Thursday, March 26, 2026
📄 Full paper submission deadline: Tuesday, March 31, 2026

Call for papers (website coming soon):
docs.google.com/document/d/1...

The Doge of Venice visits a Murano glassworks in the 17th century. I will talk about why glassmaking in this era has some similarities to AI research today.

At the #Neurips2025 mechanistic interpretability workshop I gave a brief talk about Venetian glassmaking, since I think we face a similar moment in AI research today.

Here is a blog post summarizing the talk:

davidbau.com/archives/202...

I’m recruiting a postdoc to work on algorithms for cancer genome reconstruction. We have access to a rich set of tumour samples sequenced across multiple technologies. If interested, feel free to DM. Please share.

🧑‍🔬I’m recruiting PhD students in Natural Language Processing @unileipzig.bsky.social Computer Science, together with @scadsai.bsky.social!

Topics include, but aren’t limited to:

🔎Linguistic Interpretability
🌍Multilingual Evaluation
📖Computational Typology

Please share!

#NLProc #NLP

I will be @euripsconf.bsky.social this week to present our paper as non-archival at the PAIG workshop (Beyong Regulation:
Private Governance & Oversight Mechanisms for AI). Very much looking forward to the discussions!

If you are at #EurIPS and want to chat about LLM's training data. Reach out!

📢 Postdoc position 📢

I’m recruiting a postdoc for my lab at NYU! Topics include LM reasoning, creativity, limitations of scaling, AI for science, & more! Apply by Feb 1.

(Different from NYU Faculty Fellows, which are also great but less connected to my lab.)

Link in 🧵

wait this is not the routine?

Comic. Panels up to the 10-year point are grayed out. New panels since the Ten Years comic, which chronicles the first ten years of PERSON 1's journey with cancer: (1) [two people in bed] PERSON 1 (woman): One more chapter? PERSON 2 (man): Don’t we both have to get up early? PERSON 1: Nnnnnggggh PERSON 2: Sure, good point. (2) [many people wearing masks, walking while looking at graphs on their phones] (3) [birds landing on people] PERSON 2 in beanie and scarf: Hah! They like *my* seeds best. PERSON 1 in scarf holding phone with a bird sitting on it: Wait, how do I take a picture of this one? (4) [two people rowing boats with tree landscape] (5) [Person 1 carries overflowing stack of things to Person 2 in bed] PERSON 1: I brought you honey lemon tea, more pillows, a cinnamon roll, Tylenol, another blanket, a– PERSON 2: It was just Appendicitis, I’m really– PERSON 1: *It is my turn to take care of you and I am going to do it right!* (6) [Two people in car] (7) [still in car) PERSON 1: Oh my god. PERSON 2: Oh my god. (8) [car driving] PERSON 1: Pull over! PERSON 2: I am! (9) [both people get out of car] (10) [Large colored panel of aurora borealis over water with both people looking on] (11) [Person 1 sits against tree while Person 2 lies on the ground] PERSON 1: Fifteen years. No sign of the cancer. (12) I *am* having some weird symptoms. Joint pain. Fatigue. I think I’m losing my close-up vision. PERSON 2: Yeah. Me too. (13) PERSON 2: I think we’re getting old. (14) PERSON 1: I guess that’s okay. PERSON 2: It’s all I wanted.

Fifteen Years

xkcd.com/3172/

There's a reviewer at ICLR who apparently always writes *exactly* 40 weaknesses and comments no matter what paper he's reviewing.

Exhibit A: openreview.net/forum?id=8qk...
Exhibit B: openreview.net/forum?id=GlX...
Exhibit C: openreview.net/forum?id=kDh...

within the next 3-4 days, so sadly that doesn't work

*Urgently* looking for emergency reviewers for the ARR October Interpretability track 🙏🙏

ReSkies much appreciated

Full house at BlackboxNLP at #EMNLP2025!! Getting ready for my 1.45PM keynote 😎 Join us in A102 to learn about "Memorization: myth or mystery?"

𝙒𝙚'𝙧𝙚 𝙝𝙞𝙧𝙞𝙣𝙜 𝙣𝙚𝙬 𝙛𝙖𝙘𝙪𝙡𝙩𝙮 𝙢𝙚𝙢𝙗𝙚𝙧𝙨!

KSoC: utah.peopleadmin.com/postings/190... (AI broadly)

Education + AI:
- utah.peopleadmin.com/postings/189...
- utah.peopleadmin.com/postings/190...

Computer Vision:
- utah.peopleadmin.com/postings/183...

Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps Martin Tutek, Fateme Hashemi Chaleshtori, Ana Marasovic, Yonatan Belinkov. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Outstanding paper (5/7):

"Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps"
by Martin Tutek, Fateme Hashemi Chaleshtori, Ana Marasovic, and Yonatan Belinkov
aclanthology.org/2025.emnlp-m...

6/n

hvala Josipa 🎉🥳

Thank you Gabriele :)

Very honored to be one out of seven outstanding papers at this years' EMNLP :)

Huge thanks to my amazing collaborators @fatemehc.bsky.social @anamarasovic.bsky.social @boknilev.bsky.social , this would not have been possible without them!

Unsupervised Word-level Quality Estimation for Machine Translation Through the Lens of Annotators (Dis)agreement Gabriele Sarti, Vilém Zouhar, Malvina Nissim, Arianna Bisazza. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Presenting today our work "Unsupervised Word-level Quality Estimation Through the Lens of Annotator (Dis)agreement" at the Machine Translation morning session (Room A301, 11:45 China time). See you there! 🤗

Paper: aclanthology.org/2025.emnlp-m...
Slides/video/poster: underline.io/events/502/s...

Here’s a custom feed for #EMNLP2025. Click the pin to save it to your home screen!

Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps When prompted to think step-by-step, language models (LMs) produce a chain of thought (CoT), a sequence of reasoning steps that the model supposedly used to produce its prediction. Despite much work o...

Flying out to @emnlpmeeting soon🇨🇳
I'll present our parametric CoT faithfulness work (arxiv.org/abs/2502.14829) on Wednesday at the second Interpretability session, 16:30-18:00 local time A104-105

If you're in Suzhou, reach out to talk all things reasoning :)

literally any book by sally rooney

(jk I know you don't like her)

⏰ One week left to apply for the two PhD Fellowships in Trustworthy NLP and Explainable NLU! The two positions have a starting date in spring 2026. Check the original post for more details👇

The only benefit of them being humanoid is training data I guess?

Companies have a bunch of videos of e.g. factory workers doing repetetive tasks, so you have more signal on intermediate steps of some actions to train the robots behavior

📣Tomorrow at #COLM2025:

1️⃣ Purbid's 𝐩𝐨𝐬𝐭𝐞𝐫 at 𝐒𝐨𝐋𝐚𝐑 (𝟏𝟏:𝟏𝟓𝐚𝐦-𝟏:𝟎𝟎𝐩𝐦) on catching redundant preference pairs & how pruning them hurts accuracy; www.anamarasovic.com/publications...

2️⃣ My 𝐭𝐚𝐥𝐤 at 𝐗𝐋𝐋𝐌-𝐑𝐞𝐚𝐬𝐨𝐧-𝐏𝐥𝐚𝐧 (𝟏𝟐𝐩𝐦) on measuring CoT faithfulness by looking at internals, not just behaviorally

1/3

If you're at COLM, check out various works by Ana and her group!