John Allspaw

Folks who would respond that way completely misunderstand the concept. ‘blameless’ doesn’t mean “getting off the hook” or not accountable. Ugh.

Does anyone have first-hand experience with using an AI agent that has “participated” in the response to an actual incident in a genuinely diagnostic — or otherwise contextually-specific way?

Comedy is about taking chances, Av. You took a chance.

There are a couple of phrases I genuinely despise. One of them is "...we're at an inflection point..."

1. That's not how inflection points work.
2. Hindsight's a helluva drug, isn't it?
3. Just say you are at a loss to explain some recent surprises. It's honest and not unnecessarily dramatic.

Misunderstanding the blameless postmortem post Accountability vs. Punishment The biggest misunderstanding is that it seems like you’re conflating "blameless" with "no accountability." Your post claims that blameless cultures mean "people believe...

@staysaasy.bsky.social I’m not on X very much, finally collected notes on the post docs.google.com/document/d/1...

Hi, 👋 that right there, is me.

For leaders with expectations about how more productive, efficient, etc. the software engineers in their org will be with AI:

1. How are you handling the "Left-Over Principle" challenges?

2. Also: customer comms about the incidents involving code produced AI?

(Seems clear #2 is depends on #1)

Another real life demonstration of Ironies of Automation

Incidents happen because people do things that have always worked successfully, up until the incident. Doing something that always worked in the past is completely rational!

Come on, @gergely.pragmaticengineer.com. You and I talked about the hindsight bias trap on your podcast. Don’t also fall for this trap as well:

www.adaptivecapacitylabs.com/2021/08/22/w...

Yes, and when inevitably: “Oh…huh…whoa, how come you’re doing it like that?”

They’ve got a story for you. 😀

As if there was any other route!

[Incident Fest] - The Catch-22 of Executives in Incidents | LinkedIn Join us for a short-but-sweet live session that focuses on the role of executives during incident management. Although the session is prerecorded, our experts will participate in a Q&A in the comment...

I'm now talking with Hamed Silatani, Alex Hibbitt, and Beth Adele Long about the various Catch-22 situations leaders find themselves in when responding to incidents!

www.linkedin.com/events/incid...

[Incident Fest] - What Does Great Incident Response Look Like? YouTube video by Uptime Labs

Watching www.youtube.com/live/Uxe_mw8...

Labeling a root cause is predicting the future, poorly Why do we retrospect on our incidents? Why spend the time doing those write-ups and holding review meetings? We don’t do this work as some sort of intellectual exercise for amusement. Rather,…

New blog post: surfingcomplexity.blog/2025/05/15/l...

When a bad analysis is worse than none at all One of the most famous physics experiments in modern history is the double-split experiment, originally performed by the English physicist Thomas Young back in 1801. You probably learned about this…

New blog post: surfingcomplexity.blog/2025/05/10/w...

As Richard Cook notes, in reflecting on the data collected from the SNAFU Catchers Consortium: "It's hard to know who is monitoring the system, who is looking at things, and who is available. We know for certain in (some organizations) that there is a pretty well defined group of people who are in high-frequency update mode more or less continuously while awake. This is a huge contributor to reducing the costs of coordination and it is invisible to everyone and therefore not on any dashboard." (Cook, personal communication)

1. PEOPLE keep things working.
2. When things break down, PEOPLE work to make the consequences much less than they might have been otherwise.

Both dynamics are, for the most part, invisible to management.
(via @lauramaguire.bsky.social's dissertation)

I recently read the paper "Towards Joint Activity Design Heuristics: Essentials for Human-Machine Teaming" which I loved so much I wanted to make it easier to share. To that end, I've excerpted the Ten Heuristics from the paper here: human-machine.team with anchors for each heuristic.

What a great podcast! Honored to be talking about Resilience Engineering with @colettecello.bsky.social and @spamaps.org!

Two years ago, at the first LFI Conference, we spoke alongside a client of ours (Indeed) about the amazing progress they had made in learning effectively from incidents.

This is what "good" looks like with respect to learning effectively from incidents.

www.adaptivecapacitylabs.com/2025/02/28/w...

YouTube video by thisisfinepodcast Episode 10 - When They go Full ITIL on You w/special guest John Allspaw

@allspaw.bsky.social joined us last week to help contrast ITIL's approach of "Counting and tabulating incidents" with the resilience engineering way of looking directly at the messy reality underneath them.

www.youtube.com/watch?v=cimc...

Preach! youtu.be/cimcogNc02I?... 13:30-16:30

What a great podcast! Honored to be talking about Resilience Engineering with @colettecello.bsky.social and @spamaps.org!

Three Takes on Four Concepts for Resilience Engineering Ed note: The first time I read Dr. David Woods' paper Four Concepts for Resilience Engineering, I felt so many things click in my brain. While the field of Resilience Engineering is not new, those of ...

The Resilience in Software Foundation blog included notes I wrote for the paper "Four Concepts for Resilience Engineering" in their post today: resilienceinsoftware.org/news/1149720

Very few know we've (@adaptivecapacity.bsky.social) been developing tools to help support us. After 7 years & 6 patents (!) we realized others were interested in integrating/commercializing "Churchkey."

We’re looking for partners interested in licensing Churchkey's IP. churchkey.info

What we talk about when we talk about ‘root cause’ Instead of finding the ‘root cause’ to incidents and issues, @allspaw says it’s more accurate to try and break down what created the ‘perfect storm’:

Beyond just the language setting ourselves up for failure, "root cause" have deeper issues. For me it triggers anxiety everytime i hear it. @allspaw.bsky.social has articulated it very well the issues here. Recommended material for SRE-201: github.com/readme/guide...

The beauty of your incident categorization scene is no match for the messiness of the real world.

I am so excited to announce the Resilience in Software Foundation - a project we've been working on for awhile now. 💜

YouTube video by thisisfinepodcast Episode 7 - AI and Resilience with special guest Courtney Nash

Hey folks, new episode up today! Come listen to us chat with @courtneynash.bsky.social about the ironies of automation, AI, HABA-MABA, and the impending job shortage of people who can grok how AI broke it...

youtu.be/i4qMmp6-hXg