Dominik Tornow

Durable Executions, defined

A crash? An interruption.
A sleep? An interruption.

Same suspension. Same resumption. Same solution.

The full definition: 12 words.
The minimal implementation: 50 lines.

journal.resonatehq.io/p/durable-ex...

See the Write Last, Read First rule in action, in Lisbon, today at 6:00pm

Join Matklad and me for an evening of systems engineering

luma.com/7d47f4et

Durable Executions, defined

Every durable execution framework: write normal code, get fault tolerance. Yet nobody defines what normal code or fault tolerance are

The alternative: a rigorous definition, grounded in first principles, and a minimal implementation

Dropping today

In distributed systems, you can never know "what is", you can only know "what was"

Even in consensus, you never know the current value, you only know the value at an offset in the log

Every decision is made on a stale understanding of the world

Resonate's Distributed Async Await doesn't add events, event types, event logs, or complex replay machinery.

Resonate makes promises durable. That's it.

Distributed Async Await, @resonatehqio.bsky.social Durable Executions framework, rests on one mechanic:

skip-when-done

- Checkpoint before the function starts
- Checkpoint after the function returns
- On restart, skip what's done

Everything else follows from there

Dead simple

Join Matklad and me in Lisbon next week for an evening of systems engineering with @tigerbeetle.com and @resonatehqio.bsky.social

I will talk about Write Last, Read First: How to keep systems consistent without transactions

bsky.app/profile/tige...

Dijkstra argued against unconstrained goto, not goto itself:

The unbridled use of goto statements makes it terribly hard to find a meaningful set of coordinates in which to describe the process progress

Unconstrained Event-Driven Architecture is distributed goto

Today's GoTo is Event-Driven Architecture

(Read first. Yell later)

Why your code should crash more

Assertions seem extreme: they don't throw exceptions, they crash the process. Yet continuing after a violation is worse than crashing.

Shut it down. Enable assertions in production

dtornow.substack.com/p/assert-in-...

📝 Blogged: "Building a Durable Execution Engine With SQLite"

Exploring the fundamentals of DE, based on a minimal engine built from scratch, using #SQLite as an execution log, and modern #Java (👋 virtual threads).

👉https://www.morling.dev/blog/building-durable-execution-engine-with-sqlite/

The Distributed Async Await protocol builds on two (abstract) primitives:

Execute until Blocked

Make progress by taking one incremental step

Send

Communicate by sending one message

Enabling any durability model on any transport

When durable and ephemeral timelines collide

What happens when your agent crashes while the LLM is streaming tokens to the user?!

Chapter 4: Lifetime Management in Agentic Systems

Mastering Continuations = Mastering Control Flow

A good starting point to understand continuations and programming with continuations is

Control structures in programming languages: from goto to algebraic effects

by Xavier Leroy

Orchestration vs Choreography

Forget the clumsy analogies

Orchestration

The platform manages the continuation

Choreography

The application manages the continuation

Everything else is noise

The Write Last, Read First Rule

How to guarantee strict serializability across database systems

A special guest post from @dominiktornow.bsky.social

tigerbeetle.com/blog/2025-11...

Partial Failure & Partial Retry

RPCs create recovery boundaries: When an execution crashes, only the executions within its recovery boundaries restart

Executions in different recovery boundaries (across RPCs) continue running.

Partial Failure but Total Retry?!

If we follow the familiar model of chained HTTP calls then a failure in the chain forces a restart from the failure point

This works for short-lived, cheap executions, but collapses under long-lived, expensive ones

From the terminal to the cloud, from prototype to production

Learn how agents scale beyond a single machine and master identity, coordination, and recovery

Build a recursive, distributed Deep Research Agent with @resonatehqio.bsky.social + OpenAI—in under 120 lines

Chapter 3 is live

A Promise doesn’t just represent eventual completion

A Promise also represents eventual invocation

We just tend to forget the latter

I'm experimenting with animations for the 3rd chapter of Systems Engineering for Agentic Applications

This shows the call graph of the recursive, distributed Research Agent unfolding over time, highlighting durable promises

I dig the result but they are a lot of work 🥲

Weekend Treat ... sorry ... Read 🎃

Formal models are magic. Not only as a verification tool but as a thinking tool, lifting the fog of uncertainty and assumption

Their magic lies in rigorous, ruthless, relentless clarity

Two sentences explain I/O better than any CS course

If an expression yields an effect, it percolates through the program to the top, with the context stored as a continuation. An external authority handles the effect and passes the result back to the continuation.

Uncertainty, formalized: Possible Worlds

The fewer worlds an agent considers possible, the less his uncertainty, and the more he knows

In Reasoning about Knowledge

Trampolined Style

If you want to understand Event Loops, you need to read this paper from 1999:

A trampolined program is organized as a single loop in which computations are scheduled and allowed to proceed in discrete steps.

Grokking Event Loops = Grokking Continuations

Join me in 30 min. I'll be in the live chat

Building a recursive, distributed Deep Research Agent, scaling to hundreds of agents across dozens of nodes

Chapter 3 covers:

➡️ Agent Identity
➡️ Coordination
➡️ Recovery

When agents leave the terminal, you need to become a distributed systems engineer

Roll up your newspapers, we’re squashing Heisenbugs with Deterministic Simulation Testing 🗞️

Join my talk tomorrow 11/22, 11:20 AM PDT

I'll be in the live chat for Q&A

Sign up below ⬇️

T-2d

Run your app for minutes, hours, days, weeks, or month

On FaaS like AWS Lambda or GCP Functions

Serverless @resonatehqio.bsky.social