testing in production
expressionops cultureprogramming
The practice - reckless or disciplined, depending entirely on tooling - of validating changes on the live system.
As a punchline it mocks missing test environments; as a discipline (feature flags, canaries, dark launches, chaos engineering) it acknowledges that production is the only environment that tells the whole truth. The joke and the strategy are separated by observability and a rollback path.
Testing in production began as an accusation and became, carefully qualified, a defensible practice. The accusation is real: shipping untested code and letting users find the defects is negligence. The defensible version is that some properties cannot be established anywhere else.
The reasoning is that a staging environment differs from production in every dimension that matters at scale: real traffic patterns, real data volumes and shapes, real third-party behaviour, real concurrency and the accumulated state of a system that has been running for years. A system can pass every pre-production test and fail on contact with conditions nobody could reproduce, which is why the industry built mechanisms to observe production safely.
Those mechanisms are what separate the practice from the accusation. Feature flags to expose a change to a small population, canary deployments to route a fraction of traffic, shadow traffic to exercise a new path without user impact, and observability good enough to see the difference. With those, testing in production means learning from reality with a bounded blast radius. Without them, the phrase means what it originally meant.
Also known as: test in prod