“Everything fails, all the time.”Werner Vogels · All Things Distributed · 2008 · Werner Vogels — formulare reluată în prezentări și pe blogul All Things Distributed, din 2008
A backup never tested by restoring is an assumption, not a backup.
In the cloud, compute is separated from storage and elastic: you pay per second or per query, buy no hardware, upgrades belong to the vendor, regions are a click away. The reverse: data egress costs, unbounded queries that produce surprise bills, vendor dependence and data residency, which for regulated data demands a European region and keys you manage. On-prem: predictable cost at steady load, full control, low latency to internal systems, sensitive data that never leaves. The reverse: capacity planning (you pay for the peak all year), upgrades, people, a single site. The decision rests on a few variables, not fashion: how variable the load is, what regulation demands, what the team knows, what licences already exist, how much goes out as egress. Hybrid is legitimate: the identity vault at home, token-keyed facts in the cloud. And Vogels' sentence holds in both places: the disk fails in your data centre too, the vendor has outages too. So backups tested by restoring, not by existing; recovery drills; two zones. "Where" is the small question; "what happens when it fails" is the big one.
The card says the decision rests on variables, not fashion. Let's take one of them and turn it over fully: the workload profile. An accounting firm has huge traffic for three months and almost none the rest of the year. In the cloud, it pays only while working. In its own data center, it would buy powerful machines sitting idle for nine months. Now flip the example: a hospital with constant, round-the-clock load pays the same cloud bill every month, forever. At some point, that sum exceeds the price of owning the hardware. So there is no universal right answer, only a right answer for the shape of your load. The practical question: sketch your traffic curve over a year and put it next to both bills. Then a second question appears, one the profile alone cannot settle: what does the law say about where your data may live?
Passage one left the law question hanging. Let's take it. Some data may not leave the country or the Union: medical records, banking data, public archives. Then the cloud isn't forbidden, but conditional: a European region, keys only you hold, a contract that guarantees where things actually sit. A concrete example: a dental practice wants automatic backups in the cloud, but patient data must stay in the EU, so it picks a European region and verifies that in the contract, not in the brochure. And here comes the trap nobody puts in the spreadsheet: data egress. Getting data in is cheap; getting it out, migrating to another provider, costs money. The provider knows this, and the price is set precisely so you won't leave. Today's decision becomes a narrower door tomorrow. But even with the law satisfied and the price clear, something is still missing: who, concretely, operates it all?
The third variable is the most human one: who operates what. In the cloud, updates, hardware failures and baseline security belong to the provider. Your team only has to use the service well. In your own data center, someone must apply patches, swap disks, be on call at night. A restaurant with three employees has nobody to keep a server alive; a bank with an infrastructure team does. So ask honestly what your team knows, not what you wish it knew. And whoever operates anything, anywhere, hits the same reality: things fail. That is why the next step is not choosing the location, but the test that survives any choice: the restore.
Passage three ended with the restore test. Let's make it concrete. An unverified backup is only a hope, not a plan. The real test is simple: take the backup, rebuild the system in a clean environment, time how long it takes, and note what's missing. An online bookstore discovers this way that the backup exists, but the product catalog takes three days to rebuild, which kills holiday sales. From this, two habits: recovery drills at regular intervals, not once a year out of obligation, and copies in two separate locations, because neither your provider nor your own data center is immune to outages. Here everything connects: once you have tested a restore, you already hold the data needed to choose between cloud, on-prem, or a mix of both. The next step is to look at that mix, the hybrid, without treating it as a fear-driven compromise.
So far the hybrid appeared as a synthesis. Let us treat it as a decision in its own right, with clear rules. The idea: separate data by sensitivity and by rhythm, not by fashion. Identities, keys, and the legal archive stay home, under your control. Variable workloads, analytics, and the public website go to the cloud, where you pay for what you use. An everyday example: an architecture firm keeps signed drawings and contracts on its own server, but renders 3D scenes in the cloud only during deadline weeks. No fearful compromise; each thing lives where it performs best. Yet the hybrid brings a bill many discover late: traffic between the two worlds costs money on every crossing. A careless nightly sync over the whole volume can outprice the machine you avoided buying. The question that opens next: who in your organization reads the bill before it becomes a surprise?
The previous passage ended with a question: who reads the bill? Let's take that further. Cost is not a result you receive; it is a design variable, like security. In the cloud, every technical choice has a price: how often a query runs, how long a machine stays on, how much data you move out. A courier firm regenerates a report hourly across all deliveries; nobody reads it in between. Each run is small, the sum is not. So the question is not only who reads the bill, but who decides beforehand what deserves to run. A budget with alerts, a simple rule like 'big reports run at night, by hand', a person named by name, not by role. Just as the restore test is the drill for failure, cost control is the drill for peak load. The next step is to see what happens when both fail at once.
Why it matters The cloud bill grows with every careless query; the on-prem bill is fixed and includes the peak you hit one day a year. Both are lost if you never tested a restore.