‹ Shelves ■ Data warehousingA 27-lesson course, each with a book, a verified quotation and a video lesson to watch in place: from Codd and Inmon to the lakehouse and the data mesh, dimensional modelling and SCDs, ETL/ELT and quality, preparing data for AI, pseudonymisation, k-anonymity and differential privacy, security, cloud or on-prem, joins, indexes and memory.
27 lessons · 43 min
The reading track 27 lessons All With video · 27 All authors Ralph Kimball, Margy Ross · 2
01 Lesson 1 · A data warehouse is not a bigger database; it is a database built for a different question. Building the Data Warehouse · W. H. Inmon · 1992 ▶ Video lesson › 02 Lesson 2 · Data independence: the user describes what they want, not where it lives. A Relational Model of Data for Large Shared Data Banks · E. F. Codd · 1970 ▶ Video lesson › 03 Lesson 3 · An engine good at transactions is weak at analysis, and vice versa — not a defect, physics. “One Size Fits All”: An Idea Whose Time Has Come and Gone · Michael Stonebraker, Uğur Çetintemel · 2005 ▶ Video lesson › 04 Lesson 4 · Declare the grain before anything else; the rest of the model follows from it. The Data Warehouse Toolkit · Ralph Kimball, Margy Ross · 2013 ▶ Video lesson › 05 Lesson 5 · Data Vault separates what is stable (keys), what links (relations) and what changes (attributes). Data Vault Series 1 — Data Vault Overview · Dan Linstedt · 2002 ▶ Video lesson › 06 Lesson 6 · Lakehouse: open files on object storage, with transactions and a schema on top. Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics · Michael Armbrust, Ali Ghodsi, Reynold Xin, Matei Zaharia · 2021 ▶ Video lesson › 07 Lesson 7 · The data mesh moves ownership of data to the domain that produces it; the platform and the rules stay shared. Data Mesh Principles and Logical Architecture · Zhamak Dehghani · 2020 ▶ Video lesson › 08 Lesson 8 · Column storage, compression and vectorised execution are why a query over a billion rows takes seconds. C-Store: A Column-oriented DBMS · Michael Stonebraker et al. · 2005 ▶ Video lesson › 09 Lesson 9 · The log is the source of truth; the database is a cache of the latest value in the log. The Log: What every software engineer should know about real-time data's unifying abstraction · Jay Kreps · 2013 ▶ Video lesson › 10 Lesson 10 · Accountants don't use erasers: record facts, derive states, recompute when you were wrong. Immutability Changes Everything · Pat Helland · 2015 ▶ Video lesson › 11 Lesson 11 · SCD type 2: when an attribute changes, you do not overwrite — you add a row and close the old one's period. The Data Warehouse Toolkit · Ralph Kimball, Margy Ross · 2013 ▶ Video lesson › 12 Lesson 12 · Data outlives code: every schema change has to be readable by the old code and the new. Designing Data-Intensive Applications · Martin Kleppmann · 2017 ▶ Video lesson › 13 Lesson 13 · ELT moves the transformation into the warehouse; what is left to design is orchestration and idempotency. Fundamentals of Data Engineering · Joe Reis, Matt Housley · 2022 ▶ Video lesson › 14 Lesson 14 · Quality is built into the pipeline, not inspected in the report. Out of the Crisis · W. Edwards Deming · 1986 ▶ Video lesson › 15 Lesson 15 · Garbage in, garbage out — and provenance is the only way to find where the garbage got in. Passages from the Life of a Philosopher · Charles Babbage · 1864 ▶ Video lesson › 16 Lesson 16 · Preparing data for ML: the same warehouse, but with correct time, no leakage and reproducible features. The Unreasonable Effectiveness of Data · Alon Halevy, Peter Norvig, Fernando Pereira · 2009 ▶ Video lesson › 17 Lesson 17 · Embeddings: meaning becomes a vector, and the warehouse gets a column it cannot compare with "=". A synopsis of linguistic theory 1930–1955 · John Rupert Firth · 1957 ▶ Video lesson › 18 Lesson 18 · Separate identity from facts: an identity vault, a token in the warehouse and no way back without the key. Regulamentul general privind protecția datelor (GDPR) · Parlamentul European și Consiliul Uniunii Europene · 2016 ▶ Video lesson › 19 Lesson 19 · Personal data is a liability: inventory it, collect the minimum, retain with a deadline and delete provably. Data Is a Toxic Asset, So Why Not Throw It Out? · Bruce Schneier · 2016 ▶ Video lesson › 20 Lesson 20 · No name does not mean anonymous: postcode, date of birth and sex identify most people. k-anonymity: a model for protecting privacy · Latanya Sweeney · 2002 ▶ Video lesson › 21 Lesson 21 · Differential privacy: you add calibrated noise to the answer, not the data, and the promise survives any other released set. The Algorithmic Foundations of Differential Privacy · Cynthia Dwork, Aaron Roth · 2014 ▶ Video lesson › 22 Lesson 22 · The warehouse is the biggest target in the company because it gathers everything; the defence is least privilege, applied to rows and columns, with audit. The Protection of Information in Computer Systems · Jerome Saltzer, Michael Schroeder · 1975 ▶ Video lesson › 23 Lesson 23 · Cloud or on-prem is not a question about place, but about who operates what and what it costs idle versus at peak. All Things Distributed · Werner Vogels · 2008 ▶ Video lesson › 24 Lesson 24 · A join executes in three ways, and Knuth's critical 3% in a warehouse is almost always a join. Structured Programming with go to Statements · Donald Knuth · 1974 ▶ Video lesson › 25 Lesson 25 · In a columnar warehouse, the data structure is the layout on disk: partition and order decide how many blocks need not be read. Mesaj pe lista de discuții git · Linus Torvalds · 2006 ▶ Video lesson › 26 Lesson 26 · Every rung of the memory hierarchy is a hundred times slower than the one above; a query is fast exactly as long as it stays on the top rung. Tape is Dead, Disk is Tape, Flash is Disk, RAM Locality is King · Jim Gray · 2006 ▶ Video lesson › 27 Lesson 27 · What to watch for: warehouses do not die of technology, they die of the lack of a model, an owner and a user. The Mythical Man-Month · Frederick P. Brooks Jr. · 1975 ▶ Video lesson ›