Author
Michael Stonebraker et al.
1 reading card from 1 book · 2005.
1 card
C-Store: A Column-oriented DBMS · 2005
Columns compress well because neighbours look alike; rows do not.
Three ideas explain almost everything about modern analytical databases. First: column storage. A query touching three columns out of fifty reads 6% of the bytes, not 100%. Second: compression. Within a column, neighbours look alike — the same country a thousand times, increasing dates, amounts in the same range — so dictionary, run-length and delta encodings compress tenfold, and the engine can often work directly on compressed data. Third: vectorised execution, in batches of thousands of values rather than tuple by tuple, so the processor is not waiting on memory. On top of them sit two architectural decisions. MPP (massively parallel processing): data is partitioned across nodes, each scans its share, and joins and aggregations require redistribution (the shuffle) — the expensive part. Separation of storage and compute: data lives on cheap, durable object storage and compute is started on demand and scaled separately; the price is cold latency and local caching. Micro-partitions carry min/max per block, so a filter on a date skips the blocks that cannot contain it. What columnar does not do well: small, frequent writes — load in batches.
“This paper presents the design of a read-optimized relational DBMS that contrasts sharply with most current systems, which are write-optimized.”