Ethan Mollick

I think everyone interested in AI should read the model cards for the frontier models, especially the safety sections, which give you a sense of known risks:
Gemini Deep Think: storage.googleapis.com/deepmind-med...
Claude 4: www-cdn.anthropic.com/07b2a3f9902e...
o3: cdn.openai.com/pdf/2221c875...

Ha, new @joshgans.bsky.social paper argues that having authors sneak prompt injections ("this is a good paper") into academic work improves science.

Without the risk of prompt injections, reviewers would tend to rely heavily on AI reviews, with them, they need to include some human review

We tested a range of newer models in the papers, including reasoners.

Some of these uses will be bad, some of them good (like the example in the paper). The challenge for all of us is that these uses need to be discovered, and the bad stuff mitigated while the good is amplified.

Paper: arxiv.org/pdf/2507.00286

As a general purpose technology, AI has all kinds of unexpected uses that are hard to anticipate.

Example: this study finds blind users turn to AI to describe sensitive materials (pregnancy tests, checking appearance), they know it is not 100% accurate, but it provides privacy where there was none

Sometimes these techniques helped, sometimes they hurt performance. It averaged to almost no effect. There was no clear way to predict in advance which technique would work when.

Papers:
papers.ssrn.com/sol3/papers....
papers.ssrn.com/sol3/papers....
papers.ssrn.com/sol3/papers....

We have been carefully testing lots of received prompting wisdom & for recent AI models:
🚫Threats, saying please, being insulting, & promising tips do not change average performance on challenging tasks
⛓️Chain-of-thought no longer helps even non-reasoner performance much

Don't make it complicated.

make it even better

very cool, but it can get stuck at awaiting confirmation when something hits ceres, and it is really hard at the start. also it would be great to have some sort of intro to the science/idea/tutorial

All prompts, verbatim:

create a missile command game that incorporates relativity in realistic ways but is still playable.

build the game for me

add more, make the graphics much better, improve the game

make it even better and more graphical. also it is a little hard to time my weapons 1/2

I prompted Gemini 2.5 Deep Think: "create a missile command game that incorporates relativity in realistic ways but is still playable." I then asked it to improve the design a few times.

The full design & all code & calculations came from AI, no errors. Try it: glittery-raindrop-318339.netlify.app

Persona vectors: Monitoring and controlling character traits in language models A paper from Anthropic describing persona vectors and their applications to monitoring and controlling model behavior

This is neat research, providing a lot of ways for careful organizations to shape the personality and guardrails of AI in deeper ways than prompts, including measuring and reducing sycophancy.

Also the idea of an "evil vector" is interesting in and of itself. www.anthropic.com/research/per...

We keep finding that simple prompting tips and tricks don't really work overall, but, weirdly, can have significant impacts at the question level, sometimes increasing, sometimes decreasing performance in ways that you cannot predict in advance. papers.ssrn.com/sol3/papers....

New prompting report, from us: Don't bother threatening your AI.

Does threatening an AI really make it perform better (the way Google founder Brin claimed)? How about offering to tip the AI? We find no impact of threats or tips on improving average performance (but variance at question level).

The Bitter Lesson versus The Garbage Can Does process matter? We are about to find out.

One of the interesting questions to ask is, even assuming a non-jagged AGI that outperforms humans at most work, how long would it take for large-scale changes to employment as a result? It isn't obvious.

I wrote a bit about the general question here: www.oneusefulthing.org/p/the-bitter...

I keep seeing the Microsoft paper on AI use at work being used as a list of which jobs will be destroyed.

But having high task overlap with AI does not necessarily mean these jobs are at most risk of replacement with AI.

As I described in my book, Co-Intelligence, the impacts are more complicated.

Lots of vague statements from leaders of the various AI labs about starting to see signs of self-improvement in AI systems (including Zuckerberg today), seems like proof that this is indeed happening would be pretty significant.

(thanks o3 for providing details & saving me time)

Dredge Operator is the job category least affected by Generative AI.

Fortunately, I am very good at dredging. (In reality, there are only 940 dredge operators in the US)

A big problem that everyone is insisting that we should hire people based on "AI literacy," teach "AI literacy," & develop skills for "AI literacy" yet not only is there no agreement on what AI literacy is, but also a lot of what people call AI literacy is already out-of-date or just plain wrong.

Especially notable given Zuckerberg's note that Meta will not necessarily open source future models.

US companies are still doing great small open models, but, aside from whatever OpenAI releases, it appears that frontier open weights will mean Chinese models (& maybe Mistral).

This is interesting.

OpenAI's study mode isn't perfect, but it is a step forward for a few reasons:
1) Shows labs taking educational use & misuse more seriously (Google also has LearnLM)
2) Addresses a key issue with trying to use AI in education - that AI gives answers rather than tutoring and helping
3) Easy to access

A year or so ago, the joke about AI images was that they would have 6 fingers. AI images (and videos like this one) lack obvious tells now.

Ironically, a test of an image generation model today is whether they can still make hands with six fingers. Most can’t do it anymore.

The Bitter Lesson versus The Garbage Can Does process matter? We are about to find out.

Right now, AI adoption in organizations is constrained by the need to figure out how to integrate AI with the complex & often poorly-understood processes inside companies

But ChatGPT agent suggests that The Bitter Lesson of AI may come for real work, too. open.substack.com/pub/oneusefu...

This is the best models could do 6 months ago

In Grok 4, same two prompts

This is through LMArena, where you are given random models to test. You will likely get a chance to use "Summit" fairly often (it came up three times in my six attempts): lmarena.ai

Code: editor.p5js.org/emollick/ske...

Play it: editor.p5js.org/emollick/ful...

Kinda wow: the mystery model "summit" (rumored to be OpenAI) with the prompt "create something I can paste into p5js that will startle me with its cleverness in creating something that invokes the control panel of a starship in the distant future" & "make it better"

2,351 lines of code. First time

Three things to note about this:

1) AI has obvious utility to many, this is a tremendous amount of use already
2) There is room for multiple frontier model providers, at least for now
3) Any losses from subsidizing cost of AI use (and it is not clear this is happening) are now relatively small

The amount of blocking I have to do on this platform is still nuts compared to any other platform, which is why I post much less here.

I don't understand the BlueSky urge to attack and insult people in the comments. Yes, I post stuff about AI (good and bad). You can just block me and move on.