Author
W. Edwards Deming
2 reading cards from 1 book · 1986.
2 cards
Out of the Crisis · 1986
A test at intake costs one line; the same error found in the report costs a week.
Data quality has dimensions that can be measured: completeness (are rows or values missing?), uniqueness (duplicate keys?), validity (values from the allowed set, correct types), consistency (do the lines sum to the order total?), timeliness (did it arrive on time?) and accuracy (does it match reality?). Deming's point applied here: tests are code and sit next to the transformations, not in a quarterly audit report. At intake, contracts (lesson 12). On every model: not null, unique, accepted values, relationships between tables, freshness, expected volume. At the output, anomaly monitors on the figures that matter — rows per day, distributions, sums. What you do when a test fails depends on what is at stake: for critical figures, the load stops (fail-closed) and someone looks; for the rest, suspect rows go to quarantine and the remainder passes. Every dataset has an owner and an SLA, and the cause is fixed upstream, not in the report. The report is where you find out; it is not where you fix.
“Cease dependence on inspection to achieve quality. Eliminate the need for massive inspection by building quality into the product in the first place.”
Out of the Crisis · 1986
A closed ticket is not an eliminated cause.
In the 2022 edition the order changed: 10.1 is continual improvement and 10.2 is nonconformity and corrective action. The requirement has two halves. First you react to the nonconformity: control it, correct it and deal with the consequences. Then you evaluate whether action is needed to eliminate the causes, so that it does not recur or occur elsewhere — you review the nonconformity, determine its causes and check whether similar nonconformities exist or could occur. Finally you review the effectiveness of the corrective action and change the system if necessary. The nature of the nonconformities, the actions taken and their results are retained. A lost laptop is replaced: that is the correction. The reason full-disk encryption was not enforced on every laptop is removed: that is the corrective action. The step skipped most often is the last one, the effectiveness review — without it the ticket is closed but the nonconformity is not.
“Quality comes not from inspection, but from improvement of the production process.”