Sathvik

If any friends are at Cog Sci, I’ll be in SF tomorrow! Let me know if you’d like to meet!

DEFINITELY not talking about predictive coding the next time i go back to the bay area

The sycophantic tone of ChatGPT always sounded familiar, and then I recognized where I'd heard it before: author response letters to reviewer comments.

"You're exactly right, that's a great point!"

"Thank you so much for this insight!"

Also how it always agrees even when it contradicts itself.

When it comes to text prediction, where does one LM outperform another? If you've ever worked on LM evals, you know this question is a lot more complex than it seems. In our new #acl2025 paper, we developed a method to find fine-grained differences between LMs:

🧵1/9

Table 1 Typology of traps, how they can be avoided, and what goes wrong if not avoided. Note that all traps in a sense constitute category errors (Ryle & Tanney, 2009) and the success-to-truth inference (Guest & Martin, 2023) is an important driver in most, if not all, of the traps.

NEW paper! 💭🖥️

“Combining Psychology with Artificial Intelligence: What could possibly go wrong?”

— Brief review paper by @olivia.science & myself, highlighting traps to avoid when combining Psych with AI, and why this is so important. Check out our proposed way forward! 🌟💡

osf.io/preprints/ps...

Capturing Online SRC/ORC Effort with Memory Measures from a Minimalist Parser Aniello De Santo. Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics. 2025.

A bit late but since I really like this paper, a bit of self-advertising! I am presenting at CMCL today work showing that metrics measuring how a Minimalist Grammar parser modulates memory usage can help us model Self-paced reading data for SRC/ORC contrasts: aclanthology.org/2025.cmcl-1.5/

New preprint on controlled generation from LMs!

I'll be presenting at NENLP tomorrow 12:50-2:00pm

Longer thread coming soon :)

thanks a lot Kanishka! happy i made a little contribution & that the probabilities are corrected :)

from minicons import scorer from nltk.tokenize import TweetTokenizer lm = scorer.IncrementalLMScorer("gpt2") # your own tokenizer function that returns a list of words # given some sentence input word_tokenizer = TweetTokenizer().tokenize # word scoring lm.word_score_tokenized( ["I was a matron in France", "I was a mat in France"], bos_token=True, # needed for GPT-2/Pythia and NOT needed for others tokenize_function=word_tokenizer, bow_correction=True, # Oh and Schuler correction surprisal=True, base_two=True ) ''' First word = -log_2 P(word | <beginning of text>) [[('I', 6.1522440910339355), ('was', 4.033324718475342), ('a', 4.879510402679443), ('matron', 17.611848831176758), ('in', 2.5804288387298584), ('France', 9.036953926086426)], [('I', 6.1522440910339355), ('was', 4.033324718475342), ('a', 4.879510402679443), ('mat', 19.385351181030273), ('in', 6.76780366897583), ('France', 10.574726104736328)]] '''

another day another minicons update (potentially a significant one for psycholinguists?)

"Word" scoring is now a thing! You just have to supply your own splitting function!

pip install -U minicons for merriment

I’ll also be presenting a talk based on this work Friday afternoon at HSP. Very excited to share it with a psycholinguistics-focused audience!

I’ll be at #HSP2025! I’m presenting a poster in session 4 on how semantic factors might affect timing data from a speeded cloze task (w @virmalised.us, Philip Resnik, and @colinphillips.bsky.social)

hsp2025.github.io/abstracts/19...

I’ll be presenting a poster at HSP 2025 in about a week. It’s on memory for pronominal clitic placement in Spanish, come stop by and say hi if you can!

YouTube video by Society for Mathematical Psychology Iris van Rooij keynote at MathPsych/ICCM 2024

🎬🎥🍿 Video of my keynote at MathPsych2024 now available online www.youtube.com/watch?v=WrwN...

#CogSci #CriticalAI #AIhype #AGI #PsychSci #PhilSci 🧪

What do YOU mean by "intelligence", and does ChatGPT fit your definition?
We collected the major criteria used in CogSci and other fields, and designed a survey to find out!

Access link: www.survey-xact.dk/collect
Code: 4S7V-SN4M-S536
Time: 5-10 mins

starter pack for the Computational Linguistics and Information Processing group at the University of Maryland - get all your NLP and data science here!

go.bsky.app/V9qWjEi

@kanishka.bsky.social and I have made a starter pack for researchers working broadly on linguistic interpretability and LLMs!

go.bsky.app/F9qzAUn

Please message me or comment on this post if you've noticed someone who we forgot or would like to be added yourself!

"Hey everyone! 👋 I’ve created a starter pack of South Asian artists, authors, academics, activists, and orgs. I’ll keep it updated—DM me or reply if you or someone you know should be added! ✨" go.bsky.app/GGd6dxU

I'll be there as well - excited to chat!

Generalizations across filler-gap dependencies in neural language models Katherine Howitt, Sathvik Nair, Allison Dods, Robert Melvin Hopkins. Proceedings of the 28th Conference on Computational Natural Language Learning. 2024.

RNN LMs can learn many syntactic relations but fail to capture a shared generalization across constructions. Augmenting the training data with more examples helps, but not how we'd expect!

(with Katherine Howitt, @allidods.bsky.social , and Robert Hopkins)
aclanthology.org/2024.conll-1...

A Psycholinguistic Evaluation of Language Models’ Sensitivity to Argument Roles Eun-Kyoung Rosa Lee, Sathvik Nair, Naomi Feldman. Findings of the Association for Computational Linguistics: EMNLP 2024. 2024.

Language models can identify who did what to whom, but they may not be using human-like mechanisms, based on materials from different psycholinguistic studies finding systematic patterns in human processing.

(with Rosa Lee & Naomi Feldman)
aclanthology.org/2024.finding...

I’ll be presenting two posters on (psycho)linguistically motivated perspectives on LM generalization at #EMNLP2024!

1. Sensitivity to Argument Roles - Session 2 & #BlackBoxNLP
2. Learning & Filler-Gap Dependencies - #CoNLL

Excited to chat with other folks interested in compling x cogsci!

papers⬇️

5-gram of the day: "language models from computational linguistics"

I’m so sorry but cool to see another psycholinguist who came from the CS route! I interned on a database team and hated it. The other reviews were good. It’s just a very random process

GRFP reviews sigh

Today I learned that I may not have a successful psycholinguistics career because I got a B in databases.

Panicked after seeing AGI on my tax form

“There is no ethical way to use the major AI image generators. All of them are trained on stolen images, and all of them are built for the purpose of deskilling, disempowering and replacing real human artists.”

This sums it up perfectly. It’s not a conversation.

yelled about lexicalism in my NLU seminar do i get a prize

LLMs are so weird because one side is people with five PhDs who have been studying neuron activations for the past three decades and on the other side is someone called leetm5n with an anime avatar just casually releasing increasingly better performing fine tunes of mistral

Large-scale single-neuron speech sound encoding across the depth of human cortex - Nature High-density single-neuron recordings show diverse tuning for acoustic and phonetic features across layers in human auditory speech cortex.

We’re excited about our first paper looking at speech encoding in single neurons across the depth of human cortex. Out today in @nature! www.nature.com/articles/s41... [1/6]