Calum

2/6
Cameron’s most recent paper noted that self-referential processing by LLMs yields reports of subjective experience - especially when their deception capabilities were suppressed.

www.prism-global.com/podcast/came...

5/7
Holly’s P(Pause) is quite high, at 30%, but she also thinks that agitating for it can bring worthwhile regulations, and it could perhaps bring temporary and partial pauses.
calumchace.com/podcast-feed/

The startups chasing rogue AI agents: ‘They could cost your company billions of dollars’ An emerging crop of founders is helping verify agents are sticking to their instructions

“A broken agent could cost a company billions of dollars,”
Ted Lappas, cofounder of Conscium

sifted.eu/articles/the....

1/6
Our guest is Cameron Berg, who leads AE Studio´s research into markers of subjective experience in AIs. He studied cognitive science at Yale, has worked at Meta, and has built psychometric tools used by millions.

www.prism-global.com/podcast/came...

4/7
Your P(Doom) is the probability you assign to AI causing an existential catastrophe. David coins P(Pause), the probability that humanity will collectively agree to, and stick to, a pause on the development of AI.
calumchace.com/podcast-feed/

3/7
PauseAI has organised public protests in London, San Francisco, New York, Portland, Ottawa, Sao Paulo, Berlin, Paris, Rome, Oslo, Stockholm, and Sydney.
calumchace.com/podcast-feed/

2/7
Holly studied evolutionary biology at Harvard, and was a researcher at the Effective Altrusim think tank Rethink Priorities in the area of Wild Animal Welfare.
calumchace.com/podcast-feed/

1/7
Our latest guest is Holly Elmore, Founder and Executive Director of PauseAI US, whose website says, “Our proposal is simple: Don’t build powerful AI systems until we know how to keep them safe. Pause AI.”
calumchace.com/podcast-feed/

This explains a lot.

21% of US adults believe in Santa Claus.

(Finding from an IPSOS survey of 1,000 US adults.)

Musk says new Tesla software allows texting and driving, which is illegal in most states | TechCrunch Texting while driving is banned in nearly every state, even with the use of advanced driver-assistance systems like Tesla's Full Self-Driving (Supervised) software.

Musk encourages Tesla drivers to break the law.

techcrunch.com/2025/12/04/m...

8/10
Lenore believes that artificial consciousness is not here yet, but that its arrival is inevitable, and fairly soon.

www.prism-global.com/podcast/leno...

Wikipedia:Signs of AI writing - Wikipedia This is a list of writing and formatting conventions typical of AI chatbots such as ChatGPT, with real examples taken from Wikipedia articles and drafts. It is a field guide to help detect…

Wikipedia has a nice article on the tell-tale signs that an LLM did your homework.

en.wikipedia.org/wiki/Wikiped...

What happens when AI doesn’t just outperform us, but becomes conscious? In today’s episode, we explore artificial consciousness, the economic singularity, AI agents, and the leadership required to… S1:Ep15: Conscious Machines, Safer Futures: Calum Chace on AI, Work, and What Comes Next

The future of conscious machines.
In conversation with Bryan Dennstedt.

www.youtube.com/watch?v=UhlC...

Interview on Times Radio on 27 November 2025 about the future of jobs Interview on Times Radio on 27 November 2025 about the future of jobs

The future of jobs. The machines are collar blind.
Interview with Times Radio.

youtu.be/WkgtFDE888Y

A full list of papers and resources mentioned can be found here: www.prism-global.com/podcast/lucius-caviola-a-future-with-digital-minds 0:00 Introduction and Lucius's background 01:31 Expert… Lucius Caviola: A Future with Digital Minds? Expert Estimates and Societal Response

6/6
Lucius’ work gives me an opportunity to plug a favourite science fiction author, Greg Egan, whose short story “Leaning To Be Me” is very relevant to the conversation.

youtu.be/r03bVSP44h8

A full list of papers and resources mentioned can be found here: www.prism-global.com/podcast/lucius-caviola-a-future-with-digital-minds 0:00 Introduction and Lucius's background 01:31 Expert… Lucius Caviola: A Future with Digital Minds? Expert Estimates and Societal Response

5/6
Experts were equally divided about whether the first superintelligence should ideally be conscious or a zombie.This again differs from non-experts, who mostly seem to want a zombie.

youtu.be/r03bVSP44h8

A full list of papers and resources mentioned can be found here: www.prism-global.com/podcast/lucius-caviola-a-future-with-digital-minds 0:00 Introduction and Lucius's background 01:31 Expert… Lucius Caviola: A Future with Digital Minds? Expert Estimates and Societal Response

4/6
Half of the experts expect digital minds to appear by 2050, and they mostly expect superintelligence to arrive first. They then expect large numbers of such minds to be created.

youtu.be/r03bVSP44h8

Vote for the Best Time Futurists Trainers and Thought Leaders - Global Gurus The Worlds Best Futurists Speakers Trainers In the Futurists Speakers category, vote for those Futurists speakers and trainers who have made a profound impact on you, your team or your organization.…

Time for some egregious politicking.

May I please have your vote?

globalgurus.org/vote/futuris...

Thank you!

A full list of papers and resources mentioned can be found here: www.prism-global.com/podcast/lucius-caviola-a-future-with-digital-minds 0:00 Introduction and Lucius's background 01:31 Expert… Lucius Caviola: A Future with Digital Minds? Expert Estimates and Societal Response

3/6
Non-experts give a much lower estimate of digital minds being developed than experts – 23% compared with the experts’ 73%.

youtu.be/r03bVSP44h8

What do AIs need to acquire to reach Superintelligence By creating superintelligence, we are turning ourselves into chimpanzees, so we should find out what cognitive capabilities separate us from our successors-to-be.

7/7
D. Social Understanding and Norms
Human and societal modelling
Normative reasoning
Pragmatism and common sense
Cultural grounding
Institutional and legal agency

www.forbes.com/sites/calumc...

A full list of papers and resources mentioned can be found here: www.prism-global.com/podcast/lucius-caviola-a-future-with-digital-minds 0:00 Introduction and Lucius's background 01:31 Expert… Lucius Caviola: A Future with Digital Minds? Expert Estimates and Societal Response

2/6
In a recent survey of 67 AI consciousness experts, including philosophers and cognitive scientists, Lucius and his colleague Brad Saad found their median estimate of the likelihood of digital minds being created was 73%.

youtu.be/r03bVSP44h8

What do AIs need to acquire to reach Superintelligence By creating superintelligence, we are turning ourselves into chimpanzees, so we should find out what cognitive capabilities separate us from our successors-to-be.

6/7
C. World Modelling and Planning
Rich physics world models
Counterfactual planning

www.forbes.com/sites/calumc...

A full list of papers and resources mentioned can be found here: www.prism-global.com/podcast/lucius-caviola-a-future-with-digital-minds 0:00 Introduction and Lucius's background 01:31 Expert… Lucius Caviola: A Future with Digital Minds? Expert Estimates and Societal Response

1/6
Our latest guest is Lucius Caviola, Assistant Professor at Cambridge University´s Leverhulme Centre for the Future of Intelligence. He applies experimental psychology to questions about perceptions of AI consciousness and moral status.

youtu.be/r03bVSP44h8

What do AIs need to acquire to reach Superintelligence By creating superintelligence, we are turning ourselves into chimpanzees, so we should find out what cognitive capabilities separate us from our successors-to-be.

5/7
B. Knowledge Creation and Reasoning
Robust scientific method
Causal discovery and variable invention
Handling ambiguity
Hunches and intuition
Analogy and cross-domain transfer
Meta-cognition
Deception and subterfuge

www.forbes.com/sites/calumc...

What do AIs need to acquire to reach Superintelligence By creating superintelligence, we are turning ourselves into chimpanzees, so we should find out what cognitive capabilities separate us from our successors-to-be.

4/7
A. Self and Agency:
Conscious phenomenal experience
Volition
Self-modelling and identity
Planning, and meaning-making over time
Sensorimotor grounding
Curiosity-driven exploration
Aesthetic sense and taste formation

www.forbes.com/sites/calumc...

What do AIs need to acquire to reach Superintelligence By creating superintelligence, we are turning ourselves into chimpanzees, so we should find out what cognitive capabilities separate us from our successors-to-be.

3/7
By creating superintelligence, we are turning ourselves into chimpanzees, so we should try to understand what cognitive capabilities it lacks. This article is a list of 21 human capabilities which machines don’t have. Yet.

www.forbes.com/sites/calumc...

What do AIs need to acquire to reach Superintelligence By creating superintelligence, we are turning ourselves into chimpanzees, so we should find out what cognitive capabilities separate us from our successors-to-be.

2/7
It is surprisingly hard to specify all the things that human radiologists are doing which machines cannot yet replicate, but there are a lot of them. Nevertheless, 2030 is a common estimate for the arrival of superintelligence.

www.forbes.com/sites/calumc...

Robots could reach our homes within 10 years, says Boston Dynamics CEO Boston Dynamics CEO Robert Playter tells Euronews Next about robot safety, its newest model, and AI.

Boston Dynamics CEO thinks robot housekeepers could be ten years away.

If he´s right, it´s time to short Tesla.

www.euronews.com/next/2025/11...

What do AIs need to acquire to reach Superintelligence By creating superintelligence, we are turning ourselves into chimpanzees, so we should find out what cognitive capabilities separate us from our successors-to-be.

5/7
B. Knowledge Creation and Reasoning
Robust scientific method
Causal discovery and variable invention
Handling ambiguity
Hunches and intuition
Analogy and cross-domain transfer
Meta-cognition
Deception and subterfuge

www.forbes.com/sites/calumc...

What do AIs need to acquire to reach Superintelligence By creating superintelligence, we are turning ourselves into chimpanzees, so we should find out what cognitive capabilities separate us from our successors-to-be.

1/7
About a decade ago, Geoff Hinton declared that human radiologists were like Wily E. Coyote in the Roadrunner show. They had run off the edge of a cliff, but hadn’t realised it yet. Ten years on, radiology has yet to be automated.

www.forbes.com/sites/calumc...