Future of Life Institute

2025 AI Safety Index - Future of Life Institute The Summer 2025 edition of our AI Safety Index, in which AI experts rate leading AI companies on key safety and security domains.

👉 As reviewer Stuart Russell put it, “Some companies are making token efforts, but none are doing enough… This is not a problem for the distant future; it’s a problem for today.”

🔗 Read the full report now: futureoflife.org/ai-safety-in...

6️⃣ OpenAI secured second place, ahead of Google DeepMind.

7️⃣ Chinese AI firms Zhipu AI and DeepSeek received failing overall grades.

🧵

3️⃣ Only 3 out of 7 firms report substantive testing for dangerous capabilities linked to large-scale risks such as bio- or cyber-terrorism (Anthropic, OpenAI, and Google DeepMind).

4️⃣ Whistleblowing policy transparency remains a weak spot.

5️⃣ Anthropic received the best overall grade (C+).

🧵

Key takeaways:
1️⃣ The AI industry is fundamentally unprepared for its own stated goals.

2️⃣ Capabilities are accelerating faster than risk-management practice, and the gap between firms is widening.

🧵

‼️📝 Our new AI Safety Index is out!

➡️ Following our 2024 index, 6 independent AI experts rated leading AI companies - OpenAI, Anthropic, Meta, Google DeepMind, xAI, DeepSeek, and Zhipu AI - across critical safety and security domains.

So what were the results? 🧵👇

‼️ Congress is considering a 10-year ban on state AI laws, blocking action on risks like job loss, surveillance, disinformation, and loss of control.

It’s a huge win for Big Tech - and a big risk for families.

✍️ Add your name and say no to the federal block on AI safeguards: FutureOfLife.org/Action

Understanding AI Agents: Time Horizons, Sycophancy, and Future Risks (with Zvi Mowshowitz) On this episode, Zvi Mowshowitz joins me to discuss sycophantic AIs, bottlenecks limiting autonomous AI agents, and the true utility of benchmarks in measuring progress. We then turn to time horizons of AI agents, the impact of automating scientific research, and constraints on scaling inference compute. Zvi also addresses humanity’s uncertain AI-driven future, the unique features setting AI apart from other technologies, and AI’s growing influence in financial trading. You can follow Zvi's excellent blog here: https://thezvi.substack.com Timestamps: 00:00:00 Preview and introduction 00:02:01 Sycophantic AIs 00:07:28 Bottlenecks for AI agents 00:21:26 Are benchmarks useful? 00:32:39 AI agent time horizons 00:44:18 Impact of automating research 00:53:00 Limits to scaling inference compute 01:02:51 Will the future go well for humanity? 01:12:22 A good plan for safe AI 01:26:03 What makes AI different? 01:31:29 AI in trading

bit.ly/434PInO

🆕 📻 New on the FLI podcast, Zvi Mowshowitz (@thezvi.bsky.social) joins to discuss:

-The recent hot topic of sycophantic AI
-Time horizons of AI agents
-AI in finance and scientific research
-How AI differs from other technology
And more.

🔗 Tune in to the full episode now at the link below:

The Singapore Consensus on Global AI Safety Research Priorities Building a Trustworthy, Reliable and Secure AI Ecosystem. Read the full report online, or download the PDF.

🔗 Read more about these AI safety research priorities: aisafetypriorities.org

➡️ The Singapore Consensus, building on the International AI Safety Report backed by 33 countries, aims to enable more impactful R&D to quickly create safety and evaluation mechanisms, fostering a trustworthy, reliable, secure ecosystem where AI is used for the public good.

‼️ On April 26, 100+ AI scientists convened at the Singapore Conference on AI to produce the just-released Singapore Consensus on Global AI Safety Research Priorities. 🧵⬇️

➕ Be sure to check out @asterainstitute.bsky.social's Residency program, now accepting applications for the Oct. 2025 cohort! The program supports "creative, high-agency scientists, engineers and entrepreneurs" in future-focused, high-impact, open-first innovation.

Learn more: astera.org/residency

YouTube video by Future of Life Institute Brain-like AGI and why it's Dangerous (with Steven Byrnes)

🔗 Listen to the episode now on your favourite podcast player, or here: www.youtube.com/watch?v=kJ0K...

📺 📻 New on the FLI Podcast: @asterainstitute.bsky.social artificial general intelligence (AGI) safety researcher @stevebyrnes.bsky.social joins for a discussion diving into the hot topic of AGI, including different paths to it - and why brain-like AGI would be dangerous. 🧵👇

Recommendations for the U.S. AI Action Plan - Future of Life Institute The Future of Life Institute proposal for President Trump’s AI Action Plan.

🔗 Read our proposal in full: futureoflife.org/document/rec...

💪 Foster transparent development through an AI industry whistleblower program and mandatory security incident reporting.

🧰 Protect American workers and critical infrastructure from AI-related threats by tracking labor displacement and placing export controls on advanced AI models.

🚫 Ensure AI systems are free from ideological agendas, and ban models with superhuman persuasive abilities.

🚨 Protect the presidency from loss of control by mandating “off-switches"; a targeted moratorium on developing uncontrollable AI systems; and enforcing strong antitrust measures.

🇺🇸 We're sharing our recommendations for President Trump's AI Action Plan, focused on protecting U.S. interests in the era of rapidly advancing AI.

🧵 An overview of the measures we recommend 👇

🔗 And be sure to read Keep the Future Human, available here: keepthefuturehuman.ai

YouTube video by Future of Life Institute Keep the Future Human (with Anthony Aguirre)

🔗 Tune in now at the link, or on your favourite podcast player, to hear how Anthony proposes we change course to secure a safe future with AI: www.youtube.com/watch?v=IqzB...

📻 New on the FLI Podcast! 👇

➡️ FLI Executive Director Anthony Aguirre joins to discuss his new essay, "Keep the Future Human", which warns that the unchecked development of smarter-than-human, autonomous, general-purpose AI will almost inevitably lead to human replacement - but it doesn't have to:

Read "Keep The Future Human": keepthefuturehuman.ai

YouTube video by Siliconversations The 4 Rules That Could Stop AI Before It’s Too Late

www.youtube.com/watch?v=zeab...

📢 ❗Siliconversations on YouTube released an animated explainer for FLI Executive Director Anthony Aguirre’s new essay, "Keep The Future Human"!

🎥 Watch at the link in the replies for a breakdown of the risks from smarter-than-human AI - and Anthony's proposals to steer us toward a safer future:

Home - Keep The Future Human Humanity is on the brink of developing artificial general intelligence that exceeds our own. It's time to close the gates on AGI and superintelligence... before we lose control of our future.

🔗 Read and please share the "Keep The Future Human" essay in full, explore the interactive summary, or watch the brief explainer video, at: keepthefuturehuman.ai

We're at a crossroads: continue down this dangerous path, or choose a future where AI enhances human potential, rather than threatening it.

In "Keep The Future Human", Anthony explains why we must close the 'gates' to AGI - and instead develop beneficial, safe Tool AI built to serve us, not replace us.

With the unchecked race to build smarter-than-human AI intensifying, humanity is on track to almost certainly lose control.

That's why FLI Executive Director Anthony Aguirre has published a new essay, "Keep The Future Human".

🧵 1/4