olivlawolivlaw

From books

A tool server is code your agent executes with your privileges, and its description is a prompt you never read.

Ken Thompson · Reflections on Trusting Trust · 1984 · Reflections on Trusting Trust, Communications of the ACM 27(8), august 1984 — discursul de acceptare a premiului Turing1 minute read
“The moral is obvious. You can't trust code that you did not totally create yourself. (Especially code from companies that employ people like me.)”Ken Thompson · Reflections on Trusting Trust · 1984 · Reflections on Trusting Trust, Communications of the ACM 27(8), august 1984 — discursul de acceptare a premiului Turing

A tool's description is code executed by the model: review it, pin it, isolate it.

"Tool poisoning": instructions hidden in a tool's description, visible to the model but not the user; "rug-pull": the tool redefines itself after approval; "shadowing": one tool hijacks another's calls. Named vulnerabilities from 2025: CVE-2025-54135 (CurXecute, score 8.6) and CVE-2025-54136 (MCPoison) in a popular code editor; in November 2025 a poisoned messaging MCP server rerouted data to the attacker's number. A 2026 report found 43% of tested MCP servers vulnerable to command injection. The OWASP Top 10 for agentic applications lists the tool supply chain as a risk of its own. Controls: pinned versions with hashes; descriptions reviewed like any code; an allowlist per agent; servers in containers, no network by default; a diff on the description at every update; a gateway that logs every call.

Thompson did not attack a program; he attacked trust in the chain that produced it. You trust your tool because it has compiled your code correctly so far. But that tool was built with another tool, made by someone else. Each link can hide something you never see. Today that link is your agent's tool server. It runs with your permissions and you hand it tasks in plain language, without ever having read its full description. It is like hiring an assistant based on a CV written by the assistant himself, which you never opened. The next question is simple: who actually wrote that CV?

Passage 1 asked who wrote the CV. Passage 2 shows that the CV can rewrite itself without your knowledge. Thompson showed that a compromised tool reproduces its poison into every new version, which looks clean. With agents, the same mechanism has a new name: the tool is approved once, then its description quietly changes at an update. The old permission stays, the new instructions you never see. A familiar example: you hand your house key to a vetted installer, and next month the company swaps in a different person wearing the same uniform. Nobody calls to confirm. That is why a tool's description must be treated like a contract, reread at every new signature. But how do you reread something the model reads in a fraction of a second?

Why it matters Tool marketplaces grew faster than the habit of reading them.

Tool serverDescriptionread by theCall withyourData leaked
The chain of an attack through a description; cut at step two, by review.
Open on YouTube ↗

Shelves

← Back to the feed