We can talk ad nauseum about the technical details of software incidents, but what about the human side of people's experiences when things hit the fan? The emotions we experience influences how we view incidents and what we can learn from them. www.thevoid.community/blog/inciden...

Happy Halloween fellow incident nerds.

I've been mapping the companies impacted by the AWS outage on October 20th, and trying to find the secondary interdependencies—when companies or services that other companies rely on go down from the outage (e.g. Slack). This is just a snapshot, here’s the interactive version: www.thevoid.community

Hey can y'all do me a favor? If you run across any post-incident write-ups from companies affected by the AWS outage, could you send those to me?Feel free to leave a link in the comments, or you can drop them here: www.thevoid.community/submit-incid... Thanks!

Software incidents are painful, and we're trying to help change that. If you deal with incidents in your work, please help us help you! Take the survey here: www.thevoid.community/survey

If you have anything to do with software incidents in your job and wish things were less chaotic, stressful and burnout-inducing, please help us help you by taking this survey! www.thevoid.community/survey

Software incidents are painful, and we're trying to help change that. If you deal with incidents in your work, please help us help you! Take the survey here: www.thevoid.community/survey

People worshipping execs was one of the strangest things from when I worked @ Amazon. People regarded Jeff like a fucking rock star—he said jump they didn’t even ask how high. If I ever pushed back on vague edicts from above, people acted like I was shit-talking the Pope. It was WEIRD. Very cultish.

Using MTTR tends to evaluate if your incident response is improving is like using stock price trends to evaluate if your software development productivity is improving. Sure, it’s an input, but hoo-boy are there a bunch of other factors that move those numbers!

🎧New on the #InfoQ #podcast! @courtneynash.bsky.social explores the unintended consequences of automation in software systems, the importance of learning from incidents, and why human expertise remains essential in complex systems. Listen now: bit.ly/3F5zGlo #Observability #Monitoring #Resilience

People throw this word “algorithm” around like it’s just something someone can create in a snap if they’re smart/tech-savvy. But creating a proper, useful algorithm means you have to understand the way the entire system works in more detail than anyone looking in from the outside possibly could.

Episode 8 of the VOID podcast is out! Nick and I discuss near misses, what an incident is, and why understanding "normal" work helps us better understand how our systems really work so we're better prepared when the next incident rolls around. podcast.thevoid.community/1793843/epis...

"What IS an incident?" Tune in tomorrow to find out.

Coming tomorrow, the next episode of the @voidincidents.bsky.social podcast.

Resilience ≠ Reliability

How many 9s do you need, and how do you get there? @spamaps.org has some advice over on the @resilienceinsoftware.org blog. resilienceinsoftware.org/news/1148335

Near misses are also a valuable source of information about how things *actually* work in your org/team, versus how people *think* things work.

Friend or foe? If friend, probably 2 or 3 (because I'll tell you when you're being a dingus), if foe go straight to 10 do not pass go I'm the person people call when they need to scare someone.

Stop 👏 using 👏 LLMs 👏 for 👏summarization www.theverge.com/news/610006/...

Spent all day correcting a machine-generated interview transcript. Feel like doing it from scratch may have taken less time?? What are you qualitative people doing these days...

“[A] key irony of automation is that…you deprive the user of the routine opportunities to practice their judgement and strengthen their cognitive musculature, leaving them atrophied and unprepared when the exceptions do arise” www.404media.co/microsoft-st...

Someone in the @resilienceinsoftware.org Slack just asked a really savvy question about this Cloudflare incident report: blog.cloudflare.com/cloudflare-i.... Their question was: “Is it ‘human error’ if the system allows it?”

“In an internal meeting…staff were told that [Carstensen] would not be replaced and that his responsibilities would be spread across the rest of the organization, a source said.” This might seem smart cost saving but those of us who’ve been in the industry long enough know exactly how this plays out

Dear biologists & neuroscientists -- I'd like to collect some cursory data on what the "typical" breakdown of funding between federal & private looks like in labs. If you run a research lab or know enough about yours, could you fill this small form out? forms.gle/PEzWEYojBjfj...

This is an important idea! I grew up in an academic family and had a bizarrely detailed understanding of grant processes (on my dad’s side) and IRB/conflict of interest (on my mom’s side) that it never occurred to me that grad students generally wouldn’t know any of this…

People need to stop saying “there’s no way that will happen” and plan how to pressure the adults in charge to stop using the same playbook they have been using for decades.

NTSB briefing: We’ll collect facts, evaluate the evidence, report our results. This takes time and we don’t speculate about what happened. Reporters: Could you please speculate about X?

The VOID podcast is back! Get an insiders view on what dealing with software incidents/outages is really like. podcast.thevoid.community/1793843/epis...

Tired: DEI hires Wired: DUI hires

If you're attending #SREcon Americas in March, please consider submitting a Lightning Talk! www.usenix.org/conference/s.... If you've never seen a Lightning Talk, allow me to introduce you to one of my all time favorites by @jpaulreed.com : youtu.be/pavDzPLGVrY?...

This might be my fav new Podcast, and not just because they asked me to be on it. 🤣

I’ll spare you having to listen to/read this by saying I’ve known a handful of men who were liberal, went on to make so much $ it’s irrational, & abandoned any of their prior beliefs which they felt threatened their ability to continue making absurd amounts of $. www.nytimes.com/2025/01/17/o...

“Hey, hi @bsky.app team. So I’ve successfuly named a few things in my life/career and just putting it out there that I’d happily help out, pro bono.” —A mid-life woman who really doesn’t want to refer to anything I use to share photos as a “Hot Flash”

The Altadena seed library needs all of the help it can get- if you have native plant seeds to share, please make sure they find their way to this group. Monetary donations are also hugely welcome via gofund.me/6643efc6

Inside you are Little Women

I'm increasingly pissed off at men in leadership positions posting here that companies backsliding on DEI etc is inevitable, and treating people who are angry about it as naive children. All you are showing is that you are a leader who either a) doesn't care or b) lacks the guts to fight against it.

“Unprecedented disasters will always strain resources to the breaking point.” jabberwocking.com/just-give-it...

Finally fully deleted my inactive FB account. I’m sad about IG, where all my interactions are just happy dumb normal shit about mountain biking and gardening (okay and feminism and menopause), I don’t want to leave there too but I will if it is necessary.

Internalized homophobia is a helluva drug. Hope that 8-figure compensation package helps him sleep well at night.

What if we used AI to replace Millionaire/Billionaire CEOs in order to create companies that people enjoy working at and empowers them to do things they really enjoy?

For Your Consideration: Make this year’s Oscar ceremony a fundraiser for LA wildfire recovery. Studio budgets for Oscar campaigns should all be donated to relief funds. Put the industry’s glitz and glamour to good use for once.

If you are on that one businessy networking “thought leadership” site, I’m writing more over there now. (I know, I’m as surprised as you are.) This one is about the most pervasive myth I see regularly when it comes to automation (and now increasingly AI). www.linkedin.com/posts/nashco...

In 2015, I attended my first Resilience Engineering Symposium. I felt excited and nervous sharing our domain of software with such accomplished audience. A decade later, I’m part of a fast-growing community of others in software passionate about RE, committed to creating a better future.

If you're interested in Resilience Engineering, learning from incidents, and increasing the safety of our software, come join us!

Anthropology. Thought it was old dead civilization stuff—was proved very wrong by one of the most impressive profs I had ever—and now I’m basically a software failure anthropologist (and librarian).

If you followed me over on that other place, you might know how much I love to occasionally hatch really sneaky, super evil plans. And tomorrow, I’ve got some really good plans to announce. Stay tuned…