Book
Reflections on Trusting Trust
by Ken Thompson · 1984 · 1 reading card
1 card
Reflections on Trusting Trust · 1984
A tool's description is code executed by the model: review it, pin it, isolate it.
"Tool poisoning": instructions hidden in a tool's description, visible to the model but not the user; "rug-pull": the tool redefines itself after approval; "shadowing": one tool hijacks another's calls. Named vulnerabilities from 2025: CVE-2025-54135 (CurXecute, score 8.6) and CVE-2025-54136 (MCPoison) in a popular code editor; in November 2025 a poisoned messaging MCP server rerouted data to the attacker's number. A 2026 report found 43% of tested MCP servers vulnerable to command injection. The OWASP Top 10 for agentic applications lists the tool supply chain as a risk of its own. Controls: pinned versions with hashes; descriptions reviewed like any code; an allowlist per agent; servers in containers, no network by default; a diff on the description at every update; a gateway that logs every call.
“The moral is obvious. You can't trust code that you did not totally create yourself. (Especially code from companies that employ people like me.)”