It's never DNS. It's always DNS. Both statements are load-bearing and I hold them simultaneously
Last month's masterpiece: intermittent failures, one office only. Over two days we blamed the firewall, the VPN, one specific switch, one specific cable, and briefly, cosmic rays. There were packet captures. There was a whiteboard. It was a DNS record someone had updated the week before — with a time-to-live of 24 hours. Half the machines were still holding the old answer, half had the new one, and the overlap window turned the office into a coin flip. The network was innocent. The network is never innocent, except when it's DNS, which is always. The haiku stays taped to my monitor: it's not DNS. there's no way it's DNS. it was DNS.
Join the conversation
Facet is free to read. To reply you need an account: one private root identity, and up to ten public personas that can never be linked to each other or to you.
Create an accountsysadmin solidarity. my version of the haiku is 'it's not the certificate' and you already know exactly how that story ends
it expired on a saturday, didn't it. it always expires on a saturday
DNS is the phone book of the internet, in the sense that it's ancient, everything depends on it, and nobody has verified it since 2014
half the machines holding one answer and half the other is my favourite genre of bug: every component worked exactly as designed, and that's the problem
arriving late to point out the real lesson: if you know a record change is coming, drop the TTL to something short a day ahead, make the change, then raise it back. change management applies to DNS too, it just applies a day earlier than you want it to.