“This paper argues that the data warehouse architecture as we know it today will wither in the coming years and be replaced by a new architectural pattern, the Lakehouse, which will (i) be based on open direct-access data formats, such as Apache Parquet, (ii) have first-class support for machine learning and data science, and (iii) offer state-of-the-art performance.”Michael Armbrust, Ali Ghodsi, Reynold Xin, Matei Zaharia · Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics · 2021 · Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics, CIDR 2021 — rezumat
Bronze is the evidence, silver is the cleaned truth, gold is the answer.
The data lake promised cheap storage for anything: Parquet files on object storage, no enforced schema, no transactions. It often delivered a swamp: nobody knows what is clean, two jobs write over the same directory, a read catches half of a write. The classic warehouse has transactions, schema and speed, but in a closed format, as a copy of the data. The lakehouse puts an open table format between them (Delta, Iceberg, Hudi): a metadata log over the Parquet files that brings ACID transactions, "time travel" to an earlier version, schema evolution and, above all, several engines — SQL, Spark, an ML framework — reading the same files without copies. The medallion architecture organises the layers: bronze (data exactly as it arrived, never modified), silver (cleaned, deduplicated, correctly typed), gold (aggregated, modelled for consumption — usually as a star). The golden rule is that bronze is the evidence: anything in silver and gold must be rebuildable from it.
The card says what the metadata journal brings. Let's see what it actually changes in practice. When two jobs wrote to the same directory, the fix was human coordination: who writes, when, with whose approval. The journal moves that understanding into the system. A write either lands completely or not at all, and readers always see a coherent version. Nobody needs to guard the folder anymore. A familiar example: a shared document with version history, where you can return to yesterday's version without saving manual copies. This opens a new question: if every write is recorded, who is allowed to write, and how do you decide that?
The previous passage ended with a question: who is allowed to write? The lakehouse answer is simple: the log becomes the checkpoint. Every write is a recorded operation, so it can be checked before it reaches the table. The system can reject a file with wrong types or a missing column, like a doorman who only lets in ticket holders. In the old data lake, broken data got in anyway; cleaning happened later, painfully. Here, the gate is at the entrance. An everyday example: an online form that refuses an email without the @ sign, instead of letting the mistake into the database and hunting for it afterwards. But a strict gate only guarantees that data looks right, not that it is right. So how do we show that data was cleaned with care?
Why it matters When a transformation turns out to be wrong six months later, the only question that matters is whether you still have the raw data. Bronze answers yes.