Lukas Muttenthaler

Aligning machine and human visual representations across abstraction levels - Nature Aligning foundation models with human judgments enables them to more accurately approximate human behaviour and uncertainty across various levels of visual abstraction, while additionally improving th...

What aspects of human knowledge do vision models like CLIP fail to capture, and how can we improve them? We suggest models miss key global organization; aligning them makes them more robust. Check out LukasMuttenthaler's work, finally out (in Nature!?) www.nature.com/articles/s41... + our blog! 1/3

Teaching AI to See the World More Like Humans Do Aligning AI vision models with human knowledge, improves their robustness and ability to generalize.

We summarize the ideas in a new DeepMind blog post: deepmind.google/blog/teachin... 2/3

When AI “thinks” like us New research forms a bridge between human and machine representation

When #AI “thinks” like us - New research out now in @nature.com from @lukasmut.bsky.social & @lampinen.bsky.social & @tuberlin.bsky.social & @mpib-berlin.bsky.social forms a bridge between #human and #machine representation. tinyurl.com/nhd4ycsh

🥳 I am incredibly humbled and grateful to share that our work, "Aligning machine and human visual representations across abstraction levels," has been published today in @nature.com ⬇️

Pleased to share that our survey "Getting aligned on representational alignment" — on representational alignment across cognitive (neuro)science and machine learning — is now published in TMLR! openreview.net/forum?id=Hiq...
Kudos to @sucholutsky.bsky.social @lukasmut.bsky.social for leading this!

Though it’s traumatic for me to bring this up, I was targeted by a group allied with Turning Point USA in 2018, with the help of Tucker Carlson on Fox. My home address circulated online and I got death threats. 1/

I’m recruiting committee members for the Technical Program Committee at #CCN2026.

Please apply if you want to help make submission, review & selection of contributed work (Extended Abstracts & Proceedings) more useful for everyone! 🌐

Helps to have: programming/communications/editorial experience.

w/ @eringrant.me @dotadotadota.bsky.social @sucholutsky.bsky.social @siddsuresh97.bsky.social

Join us for the Re-Align^{3} community event & hackathon at #CCN2025 in room A2.11! There will be lots of 🌶️ discussions on 🧠🤖

Interesting reaction from ChatGPT to the HHS mRNA memo. It finds it so implausible that it thinks it's fake. From the perspective of a ~2024(?) trained model, 2025 policies are so absurd as to be unbelievable...

chatgpt.com/share/689364...

🎉 Presenting at #ICML2025 tomorrow!
Come and explore how representational similarities behave across datasets :)

📅 Thu Jul 17, 11 AM-1:30 PM PDT
📍 East Exhibition Hall A-B #E-2510

Huge thanks to @lorenzlinhardt.bsky.social, Marco Morik, Jonas Dippel, Simon Kornblith, and @lukasmut.bsky.social!

If two models are more similar to each other than a third on ImageNet, will this hold for medical/satellite images?

Our #icml2025 paper analyses how vision model similarities generalize across datasets, the factors that influence them, and their link to downstream task behavior. 🧵1/7

Dimensions underlying the representational alignment of deep neural networks with humans - Nature Machine Intelligence An interpretability framework that compares how humans and deep neural networks process images has been presented. Their findings reveal that, unlike humans, deep neural networks focus more on visual ...

What makes humans similar or different to AI? In a paper out in @natmachintell.nature.com led by @florianmahner.bsky.social & @lukasmut.bsky.social, w/ Umut Güclü, we took a deep look at the factors underlying their representational alignment, with surprising results.

www.nature.com/articles/s42...

Objective drives the consistency of representational similarity across datasets The Platonic Representation Hypothesis claims that recent foundation models are converging to a shared representation space as a function of their downstream task performance, irrespective of the obje...

I am deeply grateful to @lorenzlinhardt.bsky.social, Marco Morik, Jonas Dippel, Simon Kornblith, and @lukasmut.bsky.social for their great work and support in this project! We also thank our collaborators, @bifold.berlin and HFA 7/7
📄Paper: arxiv.org/abs/2411.05561
💻Code: github.com/lciernik/sim...

🚨Have a look at our most recent work on the driving factors of representational similarities! This work will be presented at ICML 2025! See thread below 🧵

Please everyone gather and go outside, protest, march, do something! We cannot let science and academia be destroyed by the worst administration the US has ever had.

How do language models generalize from information they learn in-context vs. via finetuning? In arxiv.org/abs/2505.00661 we show that in-context learning can generalize more flexibly, illustrating key differences in the inductive biases of these modes of learning — and ways to improve finetuning. 1/

That’s a wrap for Re²-Align 🤖 🧠👤 at #ICLR2025 🦁🥳— thanks for being part of it! We’re probably just getting started… see you all at Re³-Align🌷😉

✨ Come join us at the Second edition of the Re-Align workshop
@iclr_conf
! 🚀🧠 The workshop explores the fascinating question of how artificial and biological systems align in their representations of the world. #ReAlign #ICLR2025

Harvard has set an example for other higher-ed institutions - rejecting an unlawful and ham-handed attempt to stifle academic freedom, while taking steps to make sure students can benefit from an environment of intellectual inquiry, rigorous debate and mutual respect. Let’s hope others follow suit.

The trump administration is the worst that could have happened.

This is terrible

video-muttenthaler Lukas Muttenthaler (Google DeepMind, TU Berlin)

Here’s an extended version of my PhD defense talk about our work on *representational alignment between humans 🧠 and machines 🤖 for computer vision* that I gave two weeks ago at the Max Planck Institute for Human Cognitive and Brain Sciences

www.cbs.mpg.de/cbs-coconut/...

🚨 We're hiring! If you're excited about 🤖 ML/LLMs, 🧠 cognitive science, or 💭 computational psychiatry, come join us in Munich. Two fully funded PhDs @www.helmholtz-munich.de: tailored mentorship, international vibe, lots of room to grow.
📅 May 16th
🔗 hcai-munich.com/PhDHCAI.pdf

Thank you, Rob! 🙏🏼

Thank you Desmond! Copenhagen definitely played a big part in preparing me for the PhD.

Thank so much, Andrew! You know you played a big part in all of this and I am grateful that I got to know you during my PhD and can call you both a mentor and a friend.

Oh wow, then congrats Dr. Greta!! How did I miss this? I probably saw it and said congrats but forgot about this lol. For some reason I was under the impression that you and I will defend at about the same time 🤔 Very happy for you!!

Thank you Greta! 🫶🏼 When’s your turn?

Thanks a lot for your kind words, Dota! 🙏🏼🫶🏼