Martin Schrimpf

As part of #CCN2025 our satellite event on Monday will explore how we can model the brain as a physical system, from topography to biophysical detail -- and how such models can potentially lead to impactful applications neuroailab.github.io/modeling-the-physical-brain. Join us! 🧪🧠 🤖

this is all to say that I think it is very cool the idea of "diverse representations driven by a unified objective" is coming to fruition, and I find the consistently high performance and alignment of powerful video models a strong support for it

which enables a fine-grain mapping of cortical space with a new multi-task relevance analysis; the accurate (R~0.5) prediction of second-by-second human brain activity, which makes us more confident in the characterization of action understanding pathways; and a couple more

The mouse work is definitely relevant, will make sure to reference (apologies for the oversight). I do think there are substantial novelties that have only been made possible with more recent powerful video models: the tight relation to behavior and a variety of tasks,

I don't know what the policy is for parallel discussions on BlueSky and X so I'll post twice for now 😄

It was a steep climb in the "early days" (~2012) up the ImageNet gradient towards better visual system models. That tapped out ~2015 after resnet ...

But now w/ video models starting to perform, we can push forward again. Task-driven brain models ftw. amazing...

@mschrimpf.bsky.social

Great work by @davidtyt.bsky.social with @akgokce.bsky.social, Khaled Jedoui, @dyamins.bsky.social (and me).

Check out the full thread for more details bsky.app/profile/davi... and of course the paper biorxiv.org/content/10.1... #NeuroAI #Vision #Neuroscience #AI

Where models really shine is their ability to integrate disparate findings. Our findings not only recapitulate known brain structures, they also characterize action understanding pathways. The models help us make sense of hierarchy, behavioral relevance, and functional processing

Brain-like computations support object and motion recognition that map onto classic visual ventral and dorsal streams. But looking deeper, we found a much more distributed computational landscape -- which may emerge from a single computational goal: modeling the visual world

What makes visual processing in the brain so powerful and flexible? Very excited to share our new work where we started from SOTA models that accurately predict dynamic brain activity during hours of video watching, and investigated core computations underlying visual perception

Have you by any chance tested recent larger models? Would they be more similar? At least for alignment with human classification choices, it seems that scale is really key in closely matching human behavior (plot from arxiv.org/abs/2411.05712)

🚨 New Preprint!!

Thrilled to share with you our latest work: “Mixture of Cognitive Reasoners”, a modular transformer architecture inspired by the brain’s functional networks: language, logic, social reasoning, and world knowledge.

1/ 🧵👇

Announcement: Workshop at #CCN2025
🧠 Modeling the Physical Brain: Spatial Organization & Biophysical Constraints
🗓️ Monday, Aug 11 | 🕦 11:30–18:00 CET | 📍 Room A2.07
🔗 Register: tinyurl.com/CCN-physical...
#NeuroAI @cogcompneuro.bsky.social

Does the model training process also mimic the development of representations in the brain? Maybe I'm missing it in the plot but the finding seems to be that `alignment(LLM_trained, brain_adult) > alignment(LLM_trained, brain_juvenile)`. What about the alignment of juvenile LLMs?

Agree! Let's hope our optimism caches out :D

It depends on how extensive you think "*training* on interventional data" will have to be. Eg, topographic models with a simple "perturbation module" to describe the change in neural activity from interventions were quite powerful in predicting behavioral effects www.biorxiv.org/content/10.1...

Excited to be at #NAACL2025 in Albuquerque! I’ll be presenting our paper “The LLM Language Network” as an Oral tomorrow at 2:00 PM in Ballroom C, hope to see you there!

Looking forward to all the discussions! 🎤 🧠

(Poster session actually this morning SG time, timeline confusion)

I'm at #ICLR, lmk if you'd like to chat.

Come by our TopoLM Oral on Friday (poster tonight!) iclr.cc/virtual/2025..., arxiv.org/html/2410.11516

AI vision is insanely good nowadays—but is it really like human vision or something else entirely? In our new pre-print, we pinpoint a fundamental visual mechanism that's trivial for humans yet causes most models to fail spectacularly. Let's dive in👇🧠
[https://arxiv.org/abs/2504.05253]

🚨 New Preprint!!

LLMs trained on next-word prediction (NWP) show high alignment with brain recordings. But what drives this alignment—linguistic structure or world knowledge? And how does this alignment evolve during training? Our new paper explores these questions. 👇🧵

(NAACL) @bkhmsi.bsky.social showed that neuroscience localizers uncover brain-like functional specializations in LLMs. Across 18 LLMs, he discovered neuron groups that mirror the brain’s language, theory of mind, and multiple demand networks! Work w/ @mschrimpf.bsky.social @gretatuckute.bsky.social

Excited to share that our paper, 'The LLM Language Network,' has been accepted to NAACL 2025! Looking forward to presenting it in Albuquerque—see you there! 🏜️ #NAACL2025

🚀 Applications are OPEN for the CAJAL NeuroAI course!

Learn how AI & deep learning help us model brain activity & behavior. Work with experts, get hands-on training & join a global network!

📅 Apply by March 7
🔗 loom.ly/xg_uRKE

#NeuroAI #DeepLearning #Neuroscience

🚨 New Paper!

Can neuroscience localizers uncover brain-like functional specializations in LLMs? 🧠🤖

Yes! We analyzed 18 LLMs and found units mirroring the brain's language, theory of mind, and multiple demand networks!

w/ @gretatuckute.bsky.social, @abosselut.bsky.social, @mschrimpf.bsky.social
🧵👇

Then I would hereby formally apply to be considered a BlueSky verified vision scientist

Do you accept applications to the list? 😄

🙋‍♂️

Faculty Position in Neuromodulation The Schools of Engineering (STI), Life Sciences (SV) and Computer and Communication Sciences (IC) at EPFL invite applications for a tenure-track assistant faculty position to hold the Medtronic chair ...

One month left to apply to this faculty position in neuromodulation: www.epfl.ch/about/workin... The EPFL NeuroX institute is a truly unique environment combining neuro-technology, -science, and -computation with the ultimate goal of applications in medicine. Come be our colleague! #neuro #tech 🧠

A few people asked why my slides looked jumbled up at the NIH BRAIN workshop -- I promise they were fine when I sent them in last week 😄 Copy of them here for reference: mschrimpf.altervista.org/wp-content/u... #NeuroAI 🧠🤖🧪