Stella Biderman

🚨 AI keeps scaling, but social impact evaluations aren’t–and the data proves it 🚨

Our new paper, 📎“Who Evaluates AI’s Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluations,” analyzes hundreds of evaluation reports and reveals major blind spots ‼️🧵 (1/7)

Our #NeurIPS2025 paper shows that even comparable monolingual tokenizers have different compression rates across languages. But by getting rid of whitespace tokenization and using a custom vocab size for each language, we can reduce token premiums. Preprint out now!

The Pile: An 800GB Dataset of Diverse Text for Language Modeling Recent work has demonstrated that increased training dataset diversity improves general cross-domain knowledge and downstream generalization capability for large-scale language models. With this in mi...

Small caveat: I misunderstood arXiv's ToS when I wrote this paper. While a large portion of arXiv has an open license, the majority (last time I checked) does not. That shouldn't have a check under "author."

PG-19 lacks one because of how radically technology has changed.

arxiv.org/abs/2101.00027

In the original Pile paper we talked about various conceptions of consent (though I don't stand by everything I wrote about this topic 5 years ago). None of this data has EIC, though I think that the ones marked "author" in the table are ones where authorial objection would be unreasonable.

Adding to what @mmitchell.bsky.social said, EIC cannot be use-agnostic by definition. It must be explicit to the use in question. If you put a notice that says "everyone can use this for every purpose" that's *not* EIC.

That was the best I was able to find, I swear I've read others though.

It doesn't directly address your original question though... maybe I should write a blog post about it.

GitHub - EleutherAI/cookbook: Deep learning for dummies. All the practical details and useful utilities that go into working with real models. Deep learning for dummies. All the practical details and useful utilities that go into working with real models. - EleutherAI/cookbook

We have a repo documenting resources to learn about this and all other HPC aspects of LLM training: github.com/EleutherAI/c...

You can't train a LLM like that without multiple revolutionary breakthroughs. It's a common talking point from people who are grifters or clueless, but the technology simply doesn't work like that.

I feel like there are several blog posts or papers that put forth a research agenda of "making AI research a scientific field" or "advancing the science of AI" or something like that. I'm trouble finding them, does this ring a bell to anyone / does anyone have links to notable examples?

Obviously she assaulted him /s

No, it’s not The Incentives—it’s you There’s a narrative I find kind of troubling, but that unfortunately seems to be growing more common in science. The core idea is that the mere existence of perverse incentives is a valid and sufficient reason to knowingly behave in an antisocial way, just as long as one first acknowledges the existence of those perverse incentives. The way this dynamic usually unfolds is that someone points out some fairly serious problem with the way many scientists behave—say, our collective propensity to p-hack as if it’s going out of style, or the fact that we insist on submitting our manuscripts to publishers that are actively trying to undermine our interests—and then someone else will say, “I know, right—but what are you going to do, those are the incentives.”

I want to print it out giant and put it everywhere

This is always dire, but especially so when the US is being run by an authoritarian who delights in using state power to go after people he dislikes. The second OpenAI starts asking for your ID, the government will be asking OpenAI for your chats.

We Need to Think Beyond Police in Mental Health Crises In March of 2020, Joe Prude called 911 for assistance. His brother, Daniel Prude, was behaving erratically and had just bolted out the back door of Joe…

OpenAI says adult chats deserve confidentiality, then single out teens for surveillance and says that they'll call the cops on people with mental health crises.

This will kill people and not help them get the care they need. It happens all the time

www.vera.org/news/we-need...

Building towards age prediction Learn how OpenAI is building age prediction and parental controls in ChatGPT to create safer, age-appropriate experiences for teens while supporting families with new tools.

The new “teen safety” program from OpenAI repeats the same lies that companies and governments have been saying since the internet began. This won't achieve better online safety for kids, but it will suppress individual liberty and promote censorship.

openai.com/index/buildi...

the *cato* institute says less than 10% of politically motivated terrorism is caused by leftists. the *cato* institute.

more than two-thirds is from the far-right.

There are some papers demonstrating that this improves performance, especially in translation contexts IIRC.

It's also a pretty notable comment about my friend group that when I wrote this comment I was considering "partner" to be the opposite-gender counterpart of "girlfriend"

How can an imitative model like an LLM outperform the experts it is trained on? Our new COLM paper outlines three types of transcendence and shows that each one relies on a different aspect of data diversity. arxiv.org/abs/2508.17669

How did you learn to present code? Are there resources that you recommend using to help teach people?

Good luck! Maybe you'll succeed where people have failed for decades.

Same

(Since I don't know most of the people in this thread, the joke is that I run one of the servers Naomi mentioned. Except it's not a joke.)

I have so few straight friends that "partner" to me is mostly coded as "bi but in a relationship with someone of the opposite gender"

You're right that this is an active area of research but I'm unaware of any meaningful successes coming out of it.

Can you name an example of an idea that is well-grounded in biology that has proven successful for neural networks? I don't mean "oh DL was inspired by how non-neuroscientists thinks the brain works," I mean an actual case of making a model work better by making it more brain-like

Digging into unpopular positions with no evidence while the rest of the world passes them by is basically my expectation for sufficiently senior folk.

Welcome! You are invited to join a webinar: Exploring the 2025 Latino Data Hub Updates. After registering, you will receive a confirmation email about joining the webinar. At a time when federal data systems are being defunded, decommissioned, or delayed, public access to reliable, community-level information has never been more critical. Join the UCLA Latino Policy…

On Thursday we will be joining the #UCLA Latino Policy & Politics Institute for their demo of the newly updated Latino Data Hub (LDH), a public bilingual data platform built to democratize access to critical data about #Latino communities across the country. Register here:

Here are a couple of slides that I presented yesterday at #aitechgov about open-weight model risk management.

AI safety tip: if you don’t want it giving bioweapon instructions, maybe don’t put them in the training data, say researchers New research shows that scrubbing risky material from AI training data can build safeguards that are harder to bypass — and one author calls out tech giants for keeping such work under wraps.

Thanks to @stellaathena.bsky.social for chatting with me about Deep Ignorance: the new paper/project from Eleuther AI and the UK AISI. Bottom line: Worried AI could teach people to build bioweapons? Don’t teach it how

fortune.com/2025/08/14/w...

“Your driver's license contains a ton of somewhat immutable information about you” like your name, address, DOB, and face, EFF’s Lisa Femia told the @thetennesean.bsky.social. It's not like a credit card number that can be replaced if it's leaked.