Emalie McMahon

Congratulations, Ko! So well deserved

Humans are sensitive to many different types of social interactions, but it has been difficult to isolate these differences in the human brain

In this new preprint @yuanfangzhao.bsky.social w/ @emaliemcmahon.bsky.social applied data-driven methods to two fMRI video datasets to address this

1/n

Alternative title: "LLMs don't have ears (or eyes)"

What do humans and machines miss out on when processing language as purely written text, without all the embodied audiovisual richness that scaffolds language in daily human contexts?

Very proud of this elegant work from @tommybotch.bsky.social

On Friday, my son stayed home from school because he had a sore throat and wasn’t feeling well. He went with my husband to his office for the morning while I handled our daughter. When he came back at lunch, he was not interested in eating, saying his throat hurt too much. 1/n

I'm incredibly honored to be a recipient of this year's Glushko Prize. I've looked up to so many past winners over the years, so it's surreal to be getting one myself. Thanks to Bob Glushko and the prize committee, as well as to my advisor Brian Scholl and Sam McDougle for their letters of support.

In closing, I want to thank my amazing advisors for supervising this work. I could not have imagined a better place to earn my PhD than the Department of Cognitive Science at @jhuartssciences.bsky.social

Taken together, our results do support a spatiotemporal hierarchy for computing social interactions in the lateral visual stream.

In our joint regression procedure we find that communicative interactions predict STS activity within 200 ms of stimulus onset. This reveals a rapid representation of high-level social interaction features in the brain.

In the joint regression, we find that social primitives are broadly represented in mid- and high-level regions of the STS with a similar latency. Combined with the finding that LOC and aSTS are predicted with similar latency, this suggests there may be skip connections between EVC and the STS.

Joint prediction of three ROIs of the brain (EVC, LOC, and aSTS) by EEG and stimulus features (AlexNet, agent distance, and communication)

In our final analysis, we use our novel regression procedure to jointly predict fMRI responses using EEG and stimulus features. We find that low-level visual features predict EVC activity with a short temporal latency and that communication is represented in the STS with the longest latency.

A) The time course of EEG prediction of three ROIs in the brain: EVC, LOC, and aSTS. B) The whole brain prediction of fMRI activity from the EEG signal over time.

The EEG signal predicts early visual cortex (EVC) with a very short latency. However, mid-level and high-level regions in the lateral visual pathway are represented with a similar latency, suggesting that the lateral visual stream may not be organized in a strict feedforward hierarchy.

The decoding time course of two stimulus features (agent distance and communication) from the EEG activity.

We find that visual social primitives, such as how far apart two people are in a video, are decodable from EEG earlier than social interactions features like whether two people are communicating.

Shown is an example image that participants viewed either in EEG, fMRI, and a behavioral annotation task. There is also a schematic of a regression procedure for jointly predicting fMRI responses from stimulus features and EEG activity.

I am excited to share our recent preprint and the last paper of my PhD! Here, @imelizabeth.bsky.social, @lisik.bsky.social, Mick Bonner, and I investigate the spatiotemporal hierarchy of social interactions in the lateral visual stream using EEG-fMRI.

osf.io/preprints/ps...

#CogSci #EEG

I still can't believe this! I'm deeply grateful to Bob Glushko and to @cogscisociety.bsky.social for this award. Huge thanks to my advisors @sabinehunnius.bsky.social and @neuroecologylab.bsky.social, and to many more collaborators, all the work in my thesis would have been impossible without them!

It is truly an honor to be recognized as a Glushko prize winner among all these incredible scientists. Thank you to the prize committee and to my PhD advisors @lisik.bsky.social and Mick Bonner.

The cerebellar components of the human language network The cerebellum's capacity for neural computation is arguably unmatched. Yet despite evidence of cerebellar contributions to cognition, including language, its precise role remains debated. Here, we sy...

New paper! 🧠 **The cerebellar components of the human language network**

with: @hsmall.bsky.social @moshepoliak.bsky.social @gretatuckute.bsky.social @benlipkin.bsky.social @awolna.bsky.social @aniladmello.bsky.social and @evfedorenko.bsky.social

www.biorxiv.org/content/10.1...

1/n 🧵

Seniors Are More Conservative Because the Poor Die Off Sometimes older voters aren’t just naturally more conservative. It’s just that wealthier and whiter people tend to live longer and healthier lives.

Hey, just a quiet reminder: it’s not true that people get more conservative as they age. *Populations* get more conservative as they age because poor and marginalized people die sooner.

nymag.com/intelligence...

New preprint “Monkey See, Model Knew: LLMs accurately predict visual responses in humans AND NHPs”
Led by Colin Conwell with @emaliemcmahon.bsky.social Akshay Jagadeesh, Kasper Vinken @amrahs-inolas.bsky.social @jacob-prince.bsky.social George Alvarez @taliakonkle.bsky.social & Marge Livingstone 1/n

We wrote a paper on how we think language interacts with Theory of Mind, why the connection is tricky to find experimentally, and what this means for pragmatics: www.sciencedirect.com/science/arti...

I promised to write about my thoughts on the status of the field of neuroAI, some of the big challenges we are facing, and the approaches we are taking to address them. This is super selective on the topic of finding a good model but in my view it affects the field as a whole. Here we go. 🧵

Monkey See, Model Knew: LLMs accurately Predict Human AND Macaque Visual Brain Activity
Colin Conwell, @emaliemcmahon.bsky.social, Akshay Vivek Jagadeesh, Kasper Vinken, @amrahs-inolas.bsky.social, @jacob-prince.bsky.social, George Alvarez, @taliakonkle.bsky.social and Marge Livingstone

Woo! So excited to read it!

This new article perfectly concludes my time & lessons in Jim’s lab.@JamesJDiCarlo and I propose +review SMART models of object recognition
✅ Sensory computable
✅ Mechanistic
✅ Anatomically Referenced
✅ Testable
Coming in Annual Reviews 2024
Preprint: bit.ly/3tk7u8D

A dog puppet and a lion puppet lifting the lid of a transparent box which contains a toy rattle

First post here - sharing that our paper on infants' representation of third-party helping interactions is now published (OA) at the Annual Review of Developmental Psychology! www.annualreviews.org/doi/full/10....

I am super excited to share our preprint "Early neural development of social perception: evidence from voxel-wise encoding in young children and adults" with Angira Shirahatti and @lisik.bsky.social !! osf.io/preprints/ps...

Our paper "Hierarchical organization of social action features in the lateral visual stream" led by @emaliemcmahon.bsky.social with Mick Bonner is now out in @currentbiology.bsky.social

www.sciencedirect.com/science/arti...

Seeing social interactions Seeing the interactions between other people is a critical part of our everyday visual experience, but recognizing the social interactions of others i…

My link above is broken, so adding here:

www.sciencedirect.com/science/arti...

We also provide a computational framework for how this is accomplished in the mind and brain. This framework proposes exciting directions for future research.

We review behavioral, computational, and neuroimaging evidence that converge to suggest that social interaction perception is visual and distinct from higher level social processes like theory of mind.

In our new paper out today at Trends in Cognitive Science, @lisik.bsky.social and I argue that social interaction perception is a visual process–computed by the visual system. (1/3)

tinyurl.com/nhh2dhx

#PsychSciSky #CogSci #CogPsyc #compneuro