The International AGI Project Series | Government Collaboration Research series on international AGI collaboration. Explores the "Intelsat for AGI" model for government cooperation on artificial general intelligence.

The full series has discussion of why this might be desirable, what the AGI project should focus on, and how to make this more likely.

Link: www.forethought.org/research/th...

By making non-US influence circumscribed in this way, and letting the US call the shots day to day, the proposal becomes both more feasible and less likely to get bogged down in bureaucracy.

The core idea is that we can get most of the benefits of an international project by giving non-US countries meaningful influence over only a relatively small number of decisions.

The main result is a proposal I call "Intelsat for AGI” — modelled the international project that developed the first global satellite communications network.

Today I’m publishing a series of research notes on the idea of an international AGI project.

The aim is to assess how desirable an international AGI project is, and what the best version of such a project is (taking feasibility into account).

Against Maxipok: existential risk isn’t everything — EA Forum We argue against the view that reducing existential risk should be the sole priority for improving the long-term future: non-catastrophic lock-in events can matter just as much.

EA Forum post here: forum.effectivealtruism.org/posts/qhdk8...

Against Maxipok Existential risk isn’t everything

Substack here: newsletter.forethought.org/p/against-m...

Beyond Existential Risk Bostrom's Maxipok principle suggests reducing existential risk should be the overwhelming focus for improving humanity’s long-term prospects. But this rests on an implicitly dichotomous view of future value, where most outcomes are either near-worthless or near-best. Against Maxipok, we argue it is possible to substantially influence the long-term future by other channels than reducing existential risk — including how values, institutions, and power distributions become locked in.

Full paper here: www.forethought.org/research/be...

This paper is aimed at a more academic audience, offers some new arguments and counterarguments, and provides a formal framework for thinking about existential vs. trajectory impact.

This paper complements my "Better Futures" series from last year. Better Futures argued that Flourishing (the quality of the future given survival) deserves as much attention as Surviving.

We argue longtermists should focus on the broader category of "grand challenges"—decisions that substantially affect the expected value of Earth-originating life, whether or not they involve existential risk.

This has practical implications: Maxipok can even recommend actions that are actively harmful, if they reduce x-risk while making the surviving future worse.

Lock-in events—around AGI, space governance, digital rights—could shape civilization's trajectory without being "existential" in the traditional sense. The value of the future lies on a spectrum, and we can shift where on that spectrum we land.

Third, that the very best goods are so much better than almost all others that only near-optimal futures matter.

None of these hold up. The future isn't binary. Power could be divided among groups with different values; some might optimize for good, others won't.

We assess three potential justifications for Dichotomy and reject each. First, that there's a wide basin of attraction, such that any sufficiently good civilization would tend towards the best. Second, that value is bounded above, making "good enough" essentially optimal.

If Dichotomy holds, then all that matters is shifting probability from the bad cluster to the good cluster—i.e., reducing existential risk.

The key assumption behind Maxipok is what we call "Dichotomy": that the future will either have little-to-zero value (catastrophe), or some specific extremely high value (everything else), and our actions can only move probability mass between these two poles.

Beyond Existential Risk:

In a new paper, Guive Assadi and I argue against Bostrom's "Maxipok" principle—that altruists should seek to maximize the probability of an "OK outcome," where OK just means avoiding existential catastrophe.

What sort of post-superintelligence society should we aim for? The case for ‘viatopia’: a state of society that is on track for a near-best future, whatever that might look like.

newsletter.forethought.org/p/viatopia

My current guesses for what viatopia looks like: material abundance, technological progress, coordination to avoid conflict, low catastrophic risk—plus preserving society-wide optionality, cultivating reflection, and structuring deliberation so better ideas win out.

A viatopia is a state of society that is *on track* for a near-best future, whatever that might look like. A teenager might not know what they want to do with their life, but know that a good education keeps their options open.

The transition to superintelligence will present many problems all at once, and may need to choose between very different solutions to the same problems. We need a way to prioritise and plan.

So I want to introduce a third framing: viatopia.

The main alternative is “protopianism”: solving the most urgent problems one by one, not guided by any big-picture view of society’s long-run course. I prefer protopianism to utopianism, but it gives up too much.

Almost no one has articulated a positive vision for what comes after superintelligence. What should we be trying to aim for?

Utopias from history look clearly dystopian to us, and we should expect the same for our own attempts. We don’t know enough, or have the authority, to decide the details.

Leisure & Social:
* How often do you socialize with friends?
* Do you take any vacations/holidays per year?
* How many hours of leisure time do you have per day?

* What's your highest level of education completed?
* Are you currently learning any new skills or pursuing education?
Living Situation:
* Do you live in an urban, suburban, or rural area?
* How many people share your living space?
* Do you have reliable electricity, water, and internet?

* Do you have regular contact with extended family?
Health & Lifestyle:
* How would you rate your overall health? (excellent/good/fair/poor)
* How often do you exercise per week?
* Do you have access to healthcare when needed?
* How many hours of sleep do you typically get?
Education & Growth:

* Do you have any retirement savings/pension?
* Can you handle a surprise expense of $1,000 (or equivalent) without borrowing?
Family & Relationships:
* What's your relationship status? (married/partnered/single/divorced)
* Do you have children? If so, how many?

* What type of work do you do? (employed/self-employed/not working)
* How many hours per week do you typically work?
* Do you have job security/stable income?
Financial Situation:
* Do you own or rent your home?
* Are you able to save money regularly?

And if you want to try it for yourself, here’s a prompt (put the answers in after the questions):

I’m a [age] [gender] who lives in [location].

Please consider my answers to these questions, and tell me how I compare to both global and developed country averages:

Work & Career: