Sean Trott

A confounding thing for the linguistics of LMs: the best way to assess their grammatical ability is string probability. Yet string probability and grammaticality are famously not the same!

Really excited to have this out, where we give a formal account, w/ experiments, of how to make sense of that!

(By @raphaelmilliere.com and @cameronbuckner.bsky.social )

Raphaël Millière & Cameron Buckner, Interventionist Methods for Interpreting Deep Neural Networks - PhilPapers Recent breakthroughs in artificial intelligence have primarily resulted from training deep neural networks (DNNs) with vast numbers of adjustable parameters on enormous datasets. Due to their complex ...

Thank you! I'd also recommend this philosophy-oriented overview of "interventionist" methods for studying neural networks: philpapers.org/rec/MILIMF-2

Congratulations!

Hard to process the news about Harvard and international students. Other universities should stand in solidarity with our colleagues who are being persecuted.

Why I’m Resigning from the NSF and Library of Congress I cannot participate in systems that require dishonesty as the price of belonging.

Please read my essay in TIME, which @science.org
did not do carefully before publishing this assertion. time.com/7285045/resi...

LLM-ology: the challenges ahead Three hard questions for a new paradigm.

In case you're interested I make a similar argument in this blog post, that construct validity (and external validity) are both key problems facing the scientific study of LLMs: seantrott.substack.com/p/llm-ology-...

This is great—there's definitely a measurement problem in much (maybe all) of benchmarking.

Very cool work!

Opinion | How Does A.I. Think? Here’s One Theory. It may employ a kind of reasoning humans use all the time — but you’ve probably never heard of it.

My awesome colleague @seantrott.bsky.social quoted on understanding the nature of our new robot overlords in @nytimes.com

www.nytimes.com/2024/12/16/o...

Thank you!