From books

Lesson 16 · Preparing data for ML: the same warehouse, but with correct time, no leakage and reproducible features.

Alon Halevy, Peter Norvig, Fernando Pereira · The Unreasonable Effectiveness of Data · 2009 · The Unreasonable Effectiveness of Data, IEEE Intelligent Systems 24(2), 20092 minutes read
But invariably, simple models and a lot of data trump more elaborate models based on less data.Alon Halevy, Peter Norvig, Fernando Pereira · The Unreasonable Effectiveness of Data · 2009 · The Unreasonable Effectiveness of Data, IEEE Intelligent Systems 24(2), 2009

A feature is computed "as it was at the time of the event", not "as it is now".

The 2009 argument became doctrine: more data beats a cleverer model. The unspoken condition is that the data is correct in three ways a reporting warehouse does not demand. First, point-in-time correctness: a feature is computed "as it was at the time of the event", not "as it is now" — a customer who is premium today may have been on the free tier when they placed the 2023 order; type 2 dimensions are exactly the tool here. Then, leakage: a column that contains the answer, directly or indirectly — the cancellation date when predicting cancellation; the model looks excellent in training and fails in production. Finally, training–serving skew: the features computed in batch for training and those computed live for prediction must come from the same code; the feature store exists to guarantee that. The rest is warehouse discipline: split sets by time, not at random; document the label definition; version the training set so you can reproduce it a year later; keep class balance in mind. Only then does "a lot of data" mean more than "a lot of rows".

Why it mattersA model trained on "today's" features for "back then" events learns the future and forgets it in production. The warehouse must be able to answer "how it was at that moment".

Training setPoint-in-time correctNo leakageSame features online andoffline
Three conditions a report does not demand, but a model does.
Open on YouTube

Back to the feed