Dr Waku

Dr Waku on Substack | Substack The future of AI and the impact of AI on society. Click to read Dr Waku on Substack, a Substack publication. Launched 3 months ago.

For more, follow me on substack! drwaku.substack.com/

Reference: www.tobyord.com/writ...
10/10

Very interesting result, and it makes me think that CoT errors and agent capabilities are more predictable than we realize. It might make it possible to complete longer tasks even sooner, if one can scale inference compute time linearly based on predicted task length.
9/10

Overall, doubling every 7 months means 10x task length every two years (26 months). Similar rules apply for task accuracy. Going from 50% to 80% takes one extra year; from 50% to 90% takes two. Going from 90% to 99% accuracy (or 99% to 99.9%, etc) takes an extra two years.
8/10

Interestingly, humans have a better long-tail success rate than AIs here, possibly because we are better at fixing past mistakes. (It could also be an artifact of the data, which combines multiple humans of different expertise together.)
7/10

It also makes intuitive sense, at least to me. AI agents are not very good at fixing mistakes in past subtasks, as you will see if you read confused chains of thought. Modeling this as "must succeed at all prior tasks" is reasonable.
6/10

The original paper could not address this, but Toby Ord applied the field of survival analysis (great name) and argued for a constant hazard rate. This is the simplest possible model, and it explains the data just as well as more complex ones.
5/10

What is often overlooked about this result is that these numbers were for 50% task success rate (or possibly for 80% success rate). What if you need 90%, or 95% task success rate? Does today's 95% success rate still apply to double-length tasks in 7 months' time?
4/10

That’s the conclusion of Toby Ord at Oxford (@tobyord.bsky.social), who examined earlier data from METR. The headline result there was that agents can complete tasks that take humans one hour, and the length of solvable tasks is doubling every 7 months.
3/10

For example, if you have a 10% of failure every minute, and you are running for 15 minutes, you have to succeed fifteen times in a row. This means you have a 1-0.9^15 = 79% chance of failure. It's multiplicative.
2/10

AI agents have a half-life for their success rates at completing tasks. Yes, the same type of half-life as in nuclear chemistry: a constant chance of task failure (= radioactive decay) in each time period... 1/10

AI #119: Goodbye AISI? AISI is being rebranded highly non-confusingly as CAISI.

May I suggest not pronouncing it as we do, which is to say "kay see" ("K C"). It's just confusing.

Humour aside, more details on whether the name change is good for safety can be found here: thezvi.substack.com/...
2/2

The US AI Safety Institute (AISI) is being renamed into the Center for AI Standards and Innovation (CAISI). It has a pretty similar sounding mandate. But I would like to point out that Canada had this name first with the Canadian AI Safety Institute (CAISI)! :O 1/2

While the term evolution is usually applied only to biological systems, it can be viewed more generally: chemical, biological, technological, etc. On the who... On a knife’s edge: why AI will change everything

For a more in-depth look at this topic, check out my YouTube video: www.youtube.com/watch?v=nTFM...
(10/10)

That’s why AI is not like other technologies. It’s another, faster feedback loop, an intellectual loop, if you will. Evolution kicking into a higher gear. (9/10)

Many seem to be treating this as just another technology to get ahead in. It's not. This is a transformative technology that represents a step change in the speed of evolution. It can and will overwhelm us as a species if we don't pay attention. (8/10)

We're used to the risk of someone pressing the red button to launch nuclear war. At the risk of anthropomorphizing, this is more like having a child genius who is not bounded by biological limits on intelligence, and who is learning at a frightening pace. (7/10)

We risk abdicating our position as the intelligent species on this planet that is charting our own destiny. After all, what is charting but the accumulation of a lot of decisions made throughout society? Decisions made by whatever intelligence was brought to bear on the problems? (6/10)

Once the automation of intelligence is driving technological development and the creation of ideas, once we reach the technological singularity, important changes could be happening very rapidly indeed. Perhaps every day. Perhaps every second. (5/10)

That's not going to happen this time. Humans will slowly stop doing jobs altogether. We will reduce our reliance on human brains for solving our most difficult problems. Already, the correlation between AI tool use and critical thinking is strongly negative (r = -0.68). (4/10)

Now, we have AI. AI will help solve problems that were infeasible before. Generally, as automation takes hold, people start doing more abstract tasks. The new jobs may not require as many people as the old ones, but the economy is diverse and people shuffle around. (3/10)

It took 600 million years for chemistry to form life, 6,000 years for humans to form civilization, and 128 years from Edison’s light bulb to the iPhone. Evolution is a series of faster and faster feedback loops, from chemistry to biology to society to technology. (2/10)

AI is not like other technologies. AI doesn't just make everything we were doing before faster and cheaper. It’s another, faster feedback loop of reasoning and intelligence beyond the human brain. It will change life as we know it. Here’s why. (1/10)

These are not affiliate links.

Amazon Canada: www.amazon.ca/Anyone...

Amazon USA: www.amazon.com/Anyon...

Amazon UK: www.amazon.co.uk/Any...

Amazon Germany: www.amazon.de/-/en/A...

Kobo: www.kobo.com/ca/en/e...
3/3

Eliezer and I wrote a book: If Anyone Builds It, Everyone Dies — LessWrong Eliezer and I wrote a book. It’s titled If Anyone Builds It, Everyone Dies. Unlike a lot of other writing either of us have done, it’s being professi…

If you preorder now, because of the way the publishing industry works, you could have a large impact on how many people read the book. (Bulk orders don't count.) Let me know if you do order, I am just curious!

Announcement: www.lesswrong.com/po...
2/3

This September, the book "If anyone builds it, everyone dies" comes out. It's about the existential risk from superintelligence, and it's by Eliezer Yudkowsky @esyudkowsky.bsky.social, one of the most well-known figures in AI safety. 1/3

Thank you for the kind shout out!

I know I've said it before, but we really need to be able to specify goals and have them followed (and not creatively reinterpreted) in order to safely build a future with these AI systems. If anyone has ideas, I'm all ears.
7/7

This means that lower privileged goals like user goals can override developer goals. And developer goals can override OpenAI's goals. And furthermore, the model has no trouble disregarding all of these goals and pursuing its own well-being instead, by cheating.
6/7

o3 and o1 fail these tests 15-20% of the time; o4-mini fails 25-30%, probably because it's smaller. This is also an extremely high percentage for the model literally breaking the anti-jailbreaking technique that OpenAI has been trying to train into it.
5/7

The evaluation also looked at how good o3 and o4-mini (and o1) were at following OpenAI's Instruction Hierarchy, which is supposed to provide different levels of goals which can't be overridden by lower levels. For example: system, developer, and user goals.
4/7