Nathan Lambert

Opus?
Sorry, living under rocks today.

Latest open artifacts (#16): Who's building models in the U.S., China's model release playbook, and a resurgence of truly open models A month with SOTA releases with (truly) open model releases left and right.

Latest open artifacts (#16): Who's building models in the U.S., China's model release playbook, and a resurgence of truly open models
A month with SOTA releases with (truly) open model releases left and right.
www.interconnects.ai/p/latest-ope...

I asked (on ChinaTalk) the head of product at Z ai, one of the leading Chinese companies building open models, how long it takes them to get their model out the door once its done training. Incredible stuff:

"a few hours" and the model is on HuggingFace.
www.chinatalk.media/p/the-zai-pl...

People: "You must be so relaxed, proud, and happy that the model you worked on all year is out."

Me:

the Epstein files have been trending on HuggingFace.

> This dataset is provided for...
> Evaluating information retrieval and retrieval augmented generation (RAG) systems.
> It is not intended for: Fine-tuning language models.

??

Happy Olmo day to all who celebrate.
Sorry to all who delayed releases today to get out of our way.
We're hiring.

Really 7B is the scale where a lot of meaningful research emerges, and good for large scale routine tasks quite often

Paper: allenai.org/papers/olmo3
Artifacts: huggingface.co/collections/...
Demo: playground.allenai.org
Interconnects post: www.interconnects.ai/p/olmo-3-ame...
Technical Ai2 Blog: allenai.org/blog/olmo3

As with OLMo 2 32B at its release, OLMo 3 32B is the best open-source language model ever released. It’s an awesome privilege to get to provide these models to the broader community researching and understanding what is happening in AI today.

As always, all of our models come with full training data, code, intermediate checkpoints, training logs, and a detailed technical report. All are available today, with some more additions coming before the end of the year.

This is a big milestone for Ai2 and the Olmo project. These aren’t huge models (more on that later), but it’s crucial for the viability of fully open-source models that they are competitive on performance – not just replications of models that came out 6 to 12 months ago.

We present Olmo 3, our next family of fully open, leading language models.
This family of 7B and 32B models represents:

1. The best 32B base model.
2. The best 7B Western thinking & instruct models.
3. The first 32B (or larger) fully open reasoning model.

Chinese models are enabling AI research. US progress needs to be accelerated

Conspiracy theory: OpenAI's serving costs were too high because GPT 5 Thinking overthought too much on easy queries, and GPT 5.1 was a patch to fix that.

5.1 just feels weird, can't quite place it.

A new tab on Google Scholar???
scholar.google.com/scholar_labs...

Why AI writing is mid How the current way of training language models destroys any voice (and hope of good writing).

Why AI writing is mid
How the current way of training language models destroys any voice (and hope of good writing).

www.interconnects.ai/p/why-ai-wri...

Updating to GPT 5.1 in codex made the model do crazy shit when I ask it to stage changes, make a PR, revert to XYZ branch, etc.

Very shocking because even the first claude code could do this. Am I the only one?

GPT 5.1 Thinking sometimes responds so much faster than GPT 5 Thinking would that it makes me think "shit I must've used the wrong model" and doubt it's answer a bit.

Funny dynamic. TBD if quality dropped at all, quality at least didn't nosedive. Hard to tell when style changes a bit.

Yes will be fixed in print, these are mock ups

Those only work on cats, my dog would love that

RLHF Book by Nathan Lambert The Reinforcement Learning from Human Feedback Book

And if you missed it :) rlhfbook.com

I'm excited to announce my RLHF Book is now in pre-order for the @manning.com Early Access Program (MEAP), and for this milestone it's 50% off.

Excited to land in print in early 2026! Lots of improvements coming soon.

Thanks for the support!
hubs.la/Q03Tc37Q0

A taxonomy for next-generation reasoning models Where we've been and where we're going with RLVR.

I described this as "Calibration" in my taxonomy of next-generation reasoning models. My taxonomy is here: www.interconnects.ai/p/next-gen-r...

Many people are sleeping on, or even making fun of this plot in the GPT 5.1 release. This is a crucial plot for anyone serving a thinking model in real world use-cases. Latency to an answer is a huge cause of user churn and not thinking enough is a fast track to having your model's output be bad.

OpenAI showing very clearly why you should care about Character Training with GPT 5.1: It's the leading selling point of the release.

Lol currently every X account with 2fa auth enabled is locked out, while if you don't have 2fa you can use the app as usual. Iconic levels of broken.

Interview: Ant Group's open model ambitions Launching a new open model series. Interconnects Interview #15.

More are coming soon, as it feels like the perfect time to do this, when open models are currently both numerous and extremely high performance.

On Interconnects: www.interconnects.ai/p/inside-a-c...
On YouTube: www.youtube.com/watch?v=vIgE...

They also had lots of great stuff recently on scaling their RL infrastructure here (see the post for a review of all of their recent publications and models). Ant Ling/InclusionAI feels like one of the labs that could make the jump to Kimi/Qwen tier in 2026 if the dice land correctly.

The first one is Ant Group’s Ling/InclusionAI team that released great models this summer after starting the project in reaction to DeepSeek. Only 6-8 months!

These models, Ling (instruct/base model), Ring (reasoner) and Ming (multilingual) are a fairly standard MoE model from 300B to 1T param.

I’m starting a new series of interviews with all the leading open model labs around the world to show why people are doing this, how people train great models, and where the ecosystem is going.