Writing

Notes on running things that break

Post-incident write-ups, the occasional opinion about on-call, and what we changed in the product because of both.

  • Featured
  • Post-incident

The quiet hour: what a stalled queue looked like from the outside

For fifty-one minutes our writes succeeded and our reads returned yesterday. Nothing alerted, because every check we had was asking the wrong question. Here is the timeline, the wrong assumption, and the four checks that exist now.

- 11 minutes - by Priya Raman