Peter Belcak et al. (NVIDIA Research)

1 card

1. 01

   Inside an agent, most calls are narrow and repetitive; a small specialised model does them cheaper, faster and often just as well.

   Small Language Models are the Future of Agentic AI · 2025

   The planner can be large; the workers can be small, specialised and local.

   The paper's argument: an agent's tasks are narrow, formatted, repetitive — exactly what fits a model under 10 billion parameters. The system becomes heterogeneous: a frontier model plans, small models work. The proposed procedure: log the calls, cluster them by task type, train a small model per cluster and put it in place of the large call. The 2026 pattern — a large planner, small fine-tuned workers — is reported at roughly a tenth of the cost. Open-weight families (Qwen, Gemma, Mistral, gpt-oss and others) run on a single GPU or on-prem. The price: the evaluations and the serving are yours. Where small models fail: open-ended reasoning, long context, rare cases — there you escalate.

   “Here we lay out the position that small language models (SLMs) are sufficiently powerful, inherently more suitable, and necessarily more economical for many invocations in agentic systems, and are therefore the future of agentic AI.”

   Open the card →