When the system that monitors health becomes a participant in the system it monitors, observation becomes a failure vector.
Health checks consume the same resources they monitor, creating a feedback loop under stress
If disabling monitoring improves system performance under load — if health checks create the connections they're checking — if the monitoring system's failure mode is indistinguishable from the application's failure mode — the observer has become a participant.
Every monitoring system is also a system. Kubernetes liveness probes, load balancer health checks, APM agents, log collectors — each adds overhead that is invisible under normal conditions and catastrophic under stress.
Both consume resources they should not — health checks under stress, initializers at startup. Same mechanic: observation/preparation becomes consumption
A runaway migration under load triggers health check failures, creating a cascading feedback loop
Year
2014–2020
Context
Microservices and Kubernetes made health checks mandatory. Every service needed a /health endpoint. Load balancers, orchestrators, and monitoring systems all hit this endpoint continuously. The pattern seemed simple: if the service can respond, it's healthy. But what happens when the health check itself causes the service to become unhealthy?
Who Built This
Platform engineers and SREs building observability into microservice architectures. They followed the documentation. Kubernetes required health checks. The documentation showed how to build them. It didn't show what could go wrong.
Threat Model at Time
Health checks were considered safe — read-only, lightweight, harmless. Nobody modeled the health check itself as a load source.
Why It Made Sense
A health check that verifies database connectivity, cache availability, and downstream dependencies gives you the most accurate picture of service health. A shallow health check that just returns 200 OK doesn't catch real problems.
This pattern has been found in applications built by talented developers at respected organizations across every decade of software history. Its presence in a codebase is not a reflection of the developer who wrote it — it is a reflection of what that developer was taught, what tools they had, and the path that was easiest given what they were taught. The goal is not to find fault. The goal is to find the pattern — before it finds you.
Katie's Law: The developers were not wrong. The shortcut was not wrong. The context changed and the shortcut didn't.