Schroedinger's backup
expressionops cultureIT support
The state of any backup that has never been restored: simultaneously good and bad until someone opens the box.
Backup jobs succeed for years while quietly producing nothing restorable - wrong scope, silent corruption, expired credentials. The operational form of the proverb: a backup does not exist until a restore has been tested.
The state of any backup is unknown until a restore is attempted. The joke is precise, and the underlying claim is that an untested backup exists in superposition: it is simultaneously a working recovery plan and a directory of useless files, and only the restore collapses it.
The failure modes are numerous and quiet. The job succeeded but excluded a directory added last year. The database was backed up while running without a consistent snapshot, so the file is present and unusable. The encryption key is stored in the system being restored. The tapes are fine and the drive that reads them has been decommissioned. The retention policy silently rotated away the only copy from before the corruption. In every case the monitoring is green, because backup software reports on whether it ran.
The only remedy is restoring on a schedule, to separate hardware, verifying the contents rather than the exit code, and timing how long it takes so the recovery objective is measured rather than assumed. Organizations that do this discover problems on an ordinary Tuesday. The ones that do not discover the same problems during the incident that made them matter.
Also known as: untested backup