MJ Yang

Odd, or signal that ChatGPT-5 is lower quality, but cheaper to run, because it decides when a high-token test-time compute run is triggered..

This also raises huge issues for replicability of research based on older ChatGPT models..

Urg

Confirmed, geez

There are two Bs!

ChatGPT-5 is here.

Will data centers crash the economy? This time let's think about a financial crisis before it happens.

You should read @noahpinion.blogsky.venki.dev on the data centre investments and whether they will lead to a financial crisis. I think he downplays the risk. It is potentially very high. www.noahpinion.blog/p/will-data-...

Disruption of the traditional MBA is here, whether we want it or not..

Consider: Google has a 90% market share in web and mobile searches. And has Gemini LLM results on top by default on any search. Are 1 trillion processed tokens really a sign of usefulness? (Same question about MSFT forcing LLMs on their developers..)

We see "revealed preference", just not for LLMs.

I have been thinking about this a lot, lately, after finding the troubling productivity impacts of AI on older firms ( conference.nber.org/conf_papers/... ), combined with prior work showing the impacts of digitization on older workers (www.nber.org/papers/w28094).
#EconSky

Importantly, the negative productivity effects are concentrated at old manufacturing establishments and firms. We can narrow down the mechanism as well, by showing that roughly half of the productivity loss is driven by removal of (human-supporting) structured management practices.

We use 2 separate datasets from the US Census Bureau and 3 separate identification strategies (matching, first-difference, IV) to establish credible causal effects. We find as of 2021 strongly negative productivity effects in the short-run and some evidence for potentially positive long-run effects.

SI 2025 Digital Economics and Artificial Intelligence

Naked self-promotion: My fantastic co-author @kmcelheran.bsky.social will present our empirical paper on "Industrial AI" (self-driving forklifts, self-optimizing production lines, autonomous quality control) and its effect on productivity in US manufacturing.

www.nber.org/conferences/...

Its not me, its the LLMs: in domains including coding, math, mapping, logic and now simple physics, they overfit and are unable to generalize. Still waiting for ChatGPT-5 btw..

I agree that teachers (=experts) using LLMs has clear benefits. I think the evidence of benefits of LLMs for students is less consistent and more context dependent. LLMs are not a hoax, but also clearly over-hyped.

Potemkin Understanding in Large Language Models Large language models (LLMs) are regularly evaluated using benchmark datasets. But what justifies making inferences about an LLM's capabilities based on its answers to a curated set of questions? This...

Part of the avalanche of recent papers showing incoherence of LLMs: @sendhil.bsky.social and coauthors show that LLM answers contradict their reasoning in the majority of cases, among 7 current LLM/LRM models. Another datapoint on "LLM reasoning is just pretense".

arxiv.org/abs/2506.21521

Amazon.com : empire of ai

Finished "Empire of AI" by @karenhao.bsky.social: important & compelling read for anybody worried about the broader social consequences of LLMs, even if we never reach AGI. I was initially skeptical about the "empire/colonialism" framing but now appreciate it.

www.amazon.com/s?k=empire+o...

More evidence on the educational costs of LLMs: users underperform peers using traditional web-search.

papers.ssrn.com/sol3/papers....

Yup, that OpEd reminds me of the recent MIT-brain study on LLM use..

I guess it depends on how long "short term" is? Even if data center investments have broken even, the opportunity cost really should be measured by a long-run average of alternative investments at their social benefits..

Very impressive study showing that LLM-use reduces brain activity for an essay writing task, over a 4-month time horizon. Letting college students use LLMs can significantly reduce educational benefits, not increase them.

More and more, I'm thinking the answer is "no". However, that doesn't prevent employers to think that it can and for example not sure junior software developers anymore. More importantly, AI is sometimes cheaper than hiring entry-level skilled employees..

At the same time, he says "AI is NOT a fad" and emphasizes that this (LLM-supported code generation) will "never go away". I agree, more generally for higher-ed than just software engineering.

I like this guy's idea of "AI Hype Derangement Syndrome", and might refer to it. Ask yourself: If MSFT, META etc really could create 30% of their code through LLMs, why isn't there more high-quality LLM-created code in open source software? Or why is "AI-code" banned on many open-source platforms?

YouTube video by Spec Again: Reinventing Modern Software Careers Generative AI and Code Security — recent developments as of June 2025

LLM-generated code is highly likely to generate cybersecurity risks. Yet, a majority of IT professionals say they believe that LLM-generated code is more secure than human code. Maybe because MSFT and others claim "up to 30%" of their code is LLM-based? Wait a sec..

youtu.be/NxOehfOfB6c?...

P.S.: Is it telling that joke "rebuttals" of the Apple paper (as co-written by Alex Lawsen and Antropic's Claude) have gone viral as a real rebuttal? Or does this just show that social media users believe what they want to believe?

The Leaderboard Illusion Measuring progress is fundamental to the advancement of any scientific field. As benchmarks play an increasingly central role, they also grow more susceptible to distortion. Chatbot Arena has emerged ...

Sad to see, but true: LLM companies are systematically gaming leaderboards, to make LLMs seem smarter than they are. This is why the Apple paper from 2 weeks ago had so much impact: it is part of a string of work that shows the misleading claims of (leading) LLM companies.

arxiv.org/abs/2504.20879

Having this reported by Sabine Hossenfelder (former professor of physics and now full-time YouTuber + QC skeptic), makes this news even more potentially credible. Although you should always take announcements by people who try to sell you something with a lot of skepticism!

YouTube video by Sabine Hossenfelder Surprise Progress in Quantum Computing

Interesting new developments on the Quantum Computing (QC) front: IBM reports that it developed new error-correction to cut the required physical qbits for one logical qbit by 90%. Commercially useful QCs might be with us by the end of the decade!

youtu.be/ZNQIF_q3NVA?...

A knockout blow for LLMs? LLM “reasoning” is so cooked they turned my name into a verb

Yup, "LLM reasoning" is just pretense. But OpenAI, Google and Anthropic still charge extra for these "frontier" models. My question: What will LLM companies come up next to try and convince us that AI scaling does not run into diminishing returns?