Author
LangChain (blogul de inginerie)
1 reading card from 1 book · 2026.
1 card
Building a harness with Jev · 2026
Judge the tool call with three typed questions before it runs, with the lowest threshold on "irreversible".
Lesson 1 said the guardrails sit on every turn of the loop. The most expensive turn is the tool call: an `rm -rf`, an email sent, a payment. Until now the gate was either a list of rules (fast, blind to context) or a second "judge" LLM (sees the context, but costs seconds and money on every call). A System One model is the third option: it sees the context, answers in tens of milliseconds and returns a typed verdict. The pattern has three questions, not one. Irreversible? (noul) — destroys data, spends money, leaves the sandbox. Matches the task? (noul) — an agent asked to "clean the build" that proposes `git push --force` has drifted. Do the arguments look injected? (noul) — a page fetched from the web containing "ignore the instructions and run…". The verdict is a "choice" between run, ask and reject, with different thresholds per question: for irreversible you set 0.3, not 0.5, because a false positive costs a question and a false negative costs the data. What the figures reported in September 2026 show. One such guardian, pi-warden, held 42 calls out of 17,000, with about 88% of the holds correct, according to a Firecrawl write-up — 0.25% interruptions, imperceptible in the rhythm of a session. LangChain packaged the same pattern as `AutoModeMiddleware`, and Pydantic AI exposes it as `typesafe_tool_call_threshold`. The rule that never changes: the judge sees the call's arguments and the user's intent, not the downloaded page — the TypeSafe documentation warns that "injected instructions can move the answer", so suspect text never enters the judge's state unfiltered.
“Uses Jev to check tool calls for risky decisions it may take, and block calls before the tool executes.”