olivlawolivlaw

From books

Do not pay for reasoning where no thinking is needed: route cheap calls to cheap models and keep the reasoning model for the hard decisions.

Alfred North Whitehead · An Introduction to Mathematics · 1911 · An Introduction to Mathematics (1911), cap. 5 «The Symbolism of Mathematics»1 minute readpublic domain
“It is a profoundly erroneous truism, repeated by all copy-books and by eminent people when they are making speeches, that we should cultivate the habit of thinking of what we are doing. The precise opposite is the case. Civilization advances by extending the number of important operations which we can perform without thinking about them.”Alfred North Whitehead · An Introduction to Mathematics · 1911 · An Introduction to Mathematics (1911), cap. 5 «The Symbolism of Mathematics»

The expensive model is called on exception, not by default.

An agent's cost architecture: tiering (a cheap model by default, escalation on low confidence or a hard task); routing reports in 2026 about 95% of frontier-model quality while sending only 14–26% of calls to the expensive model, a 75–85% saving on routed traffic; prompt caching cuts the price of repeated input by up to ~90%; batching for whatever is not urgent; caps on reasoning effort. The order of measures: static tiering, caching, effort caps and batching first (50–70% typical savings), a learned router only after. The trap: the cheap model's failure mode is confident error — every tier needs its own evaluation (lesson 15).

Whitehead turns a familiar idea on its head: thinking is not a virtue to cultivate everywhere, but a scarce resource. Progress comes from moving tasks out of the "I think" column into the "it happens by itself" column. For an agent, every call to an expensive model draws on that same resource: money and attention. If routines, repeated phrasings and simple tasks go to the cheap model, real thinking stays free for what matters. Example: you do not carefully weigh the price of bread, but you scrutinize a lease contract. The practical question is different: how do you decide, call by call, what deserves expensive thinking?

Why it matters An agent's bill is a distribution of calls; most of them do not deserve the most expensive model.

Reasoning model:exceptionsMid-size model:ordinary tasksSmall model andcache: most calls
The top is expensive and rare; the base is cheap and dense.
Open on YouTube ↗

Shelves

← Back to the feed