Congratulations, Ko! So well deserved
Humans are sensitive to many different types of social interactions, but it has been difficult to isolate these differences in the human brain
In this new preprint @yuanfangzhao.bsky.social w/ @emaliemcmahon.bsky.social applied data-driven methods to two fMRI video datasets to address this
1/n
Alternative title: "LLMs don't have ears (or eyes)"
What do humans and machines miss out on when processing language as purely written text, without all the embodied audiovisual richness that scaffolds language in daily human contexts?
Very proud of this elegant work from @tommybotch.bsky.social
On Friday, my son stayed home from school because he had a sore throat and wasn’t feeling well. He went with my husband to his office for the morning while I handled our daughter. When he came back at lunch, he was not interested in eating, saying his throat hurt too much. 1/n
I'm incredibly honored to be a recipient of this year's Glushko Prize. I've looked up to so many past winners over the years, so it's surreal to be getting one myself. Thanks to Bob Glushko and the prize committee, as well as to my advisor Brian Scholl and Sam McDougle for their letters of support.
In closing, I want to thank my amazing advisors for supervising this work. I could not have imagined a better place to earn my PhD than the Department of Cognitive Science at @jhuartssciences.bsky.social
Taken together, our results do support a spatiotemporal hierarchy for computing social interactions in the lateral visual stream.
In our joint regression procedure we find that communicative interactions predict STS activity within 200 ms of stimulus onset. This reveals a rapid representation of high-level social interaction features in the brain.
In the joint regression, we find that social primitives are broadly represented in mid- and high-level regions of the STS with a similar latency. Combined with the finding that LOC and aSTS are predicted with similar latency, this suggests there may be skip connections between EVC and the STS.
In our final analysis, we use our novel regression procedure to jointly predict fMRI responses using EEG and stimulus features. We find that low-level visual features predict EVC activity with a short temporal latency and that communication is represented in the STS with the longest latency.
The EEG signal predicts early visual cortex (EVC) with a very short latency. However, mid-level and high-level regions in the lateral visual pathway are represented with a similar latency, suggesting that the lateral visual stream may not be organized in a strict feedforward hierarchy.
We find that visual social primitives, such as how far apart two people are in a video, are decodable from EEG earlier than social interactions features like whether two people are communicating.
I am excited to share our recent preprint and the last paper of my PhD! Here, @imelizabeth.bsky.social, @lisik.bsky.social, Mick Bonner, and I investigate the spatiotemporal hierarchy of social interactions in the lateral visual stream using EEG-fMRI.
osf.io/preprints/ps...
#CogSci #EEG
I still can't believe this! I'm deeply grateful to Bob Glushko and to @cogscisociety.bsky.social for this award. Huge thanks to my advisors @sabinehunnius.bsky.social and @neuroecologylab.bsky.social, and to many more collaborators, all the work in my thesis would have been impossible without them!
It is truly an honor to be recognized as a Glushko prize winner among all these incredible scientists. Thank you to the prize committee and to my PhD advisors @lisik.bsky.social and Mick Bonner.
New paper! 🧠 **The cerebellar components of the human language network**
with: @hsmall.bsky.social @moshepoliak.bsky.social @gretatuckute.bsky.social @benlipkin.bsky.social @awolna.bsky.social @aniladmello.bsky.social and @evfedorenko.bsky.social
www.biorxiv.org/content/10.1...
1/n 🧵
Hey, just a quiet reminder: it’s not true that people get more conservative as they age. *Populations* get more conservative as they age because poor and marginalized people die sooner.
nymag.com/intelligence...
New preprint “Monkey See, Model Knew: LLMs accurately predict visual responses in humans AND NHPs”
Led by Colin Conwell with @emaliemcmahon.bsky.social Akshay Jagadeesh, Kasper Vinken @amrahs-inolas.bsky.social @jacob-prince.bsky.social George Alvarez @taliakonkle.bsky.social & Marge Livingstone 1/n
We wrote a paper on how we think language interacts with Theory of Mind, why the connection is tricky to find experimentally, and what this means for pragmatics: www.sciencedirect.com/science/arti...
I promised to write about my thoughts on the status of the field of neuroAI, some of the big challenges we are facing, and the approaches we are taking to address them. This is super selective on the topic of finding a good model but in my view it affects the field as a whole. Here we go. 🧵
Monkey See, Model Knew: LLMs accurately Predict Human AND Macaque Visual Brain Activity
Colin Conwell, @emaliemcmahon.bsky.social, Akshay Vivek Jagadeesh, Kasper Vinken, @amrahs-inolas.bsky.social, @jacob-prince.bsky.social, George Alvarez, @taliakonkle.bsky.social and Marge Livingstone
Woo! So excited to read it!
This new article perfectly concludes my time & lessons in Jim’s lab.@JamesJDiCarlo and I propose +review SMART models of object recognition
✅ Sensory computable
✅ Mechanistic
✅ Anatomically Referenced
✅ Testable
Coming in Annual Reviews 2024
Preprint: bit.ly/3tk7u8D
First post here - sharing that our paper on infants' representation of third-party helping interactions is now published (OA) at the Annual Review of Developmental Psychology! www.annualreviews.org/doi/full/10....
I am super excited to share our preprint "Early neural development of social perception: evidence from voxel-wise encoding in young children and adults" with Angira Shirahatti and @lisik.bsky.social !! osf.io/preprints/ps...
Our paper "Hierarchical organization of social action features in the lateral visual stream" led by @emaliemcmahon.bsky.social with Mick Bonner is now out in @currentbiology.bsky.social
www.sciencedirect.com/science/arti...
We also provide a computational framework for how this is accomplished in the mind and brain. This framework proposes exciting directions for future research.
We review behavioral, computational, and neuroimaging evidence that converge to suggest that social interaction perception is visual and distinct from higher level social processes like theory of mind.
In our new paper out today at Trends in Cognitive Science, @lisik.bsky.social and I argue that social interaction perception is a visual process–computed by the visual system. (1/3)
tinyurl.com/nhh2dhx
#PsychSciSky #CogSci #CogPsyc #compneuro
