Also known as: tool calling, tool invocation, actions
Tool use is the pattern where an LLM emits a structured request to call an external function — a search API, a code runner, a database query — and the runtime executes it and returns the result.
Tool use is the mechanism by which a language model reaches outside its own weights. The model emits a structured request — “call function search_docs with argument query='context window benchmarks'” — the runtime executes that function, and the result is fed back into the model’s context. With this primitive, the LLM stops being a closed text generator and starts being a system that can act.
A tool exposed to the model has three things:
At inference, the model produces either a normal text response or a tool-call request — name plus arguments, in a structured format the runtime parses. The runtime executes the underlying function, captures the result, and either returns it to the user or feeds it back to the model as the next turn’s input.
Almost every agent capability resolves to tool use:
The agent loop is mostly just a while-loop around this primitive: model emits a tool call, runtime executes, model sees the result, repeat.
At the API level, a tool call is a structured field in the assistant’s response, not free-text. OpenAI’s shape is roughly:
{
"role": "assistant",
"content": null,
"tool_calls": [{
"id": "call_abc",
"type": "function",
"function": { "name": "search_docs", "arguments": "{\"query\":\"...\"}" }
}]
}The runtime parses arguments as JSON, dispatches to the named function, and returns a tool role message with the result keyed to tool_call_id: "call_abc". The model’s next turn sees that tool result inline.
Constrained decoding under the hood is what makes the JSON parseable — providers either run a grammar over the function-arguments slot or post-process and reject invalid arguments, retrying the model. Either way, by the time the runtime sees the call, the schema has been enforced.
Tool selection is a narrow classification task: given the query and the catalog of tools, pick the right one. That’s the shape a small specialized model excels at, at a fraction of the LLM’s per-call cost. Same applies to reranking the results that come back from a search tool. The agent stays generalist; the high-frequency narrow calls inside the loop go to specialists.
LLMs can't access fresh data, can't run code reliably, can't hit your private APIs, and can't take side effects. Tool use is how the model reaches outside its weights — for retrieval, computation, communication, or any action whose result isn't already in its context.
Tool selection. With more than ~10 available tools, frontier LLMs start picking the wrong one — sometimes the second-best, sometimes one that's plainly irrelevant. Tool selection is itself a narrow classification task and a strong candidate for a specialized small model in front of the LLM.
RAG is one specific tool-use pattern: 'call a retrieval function, splice results into the prompt.' Tool use is the more general primitive — the LLM can call any function exposed to it, not just search. Modern agents combine retrieval, code execution, web fetching, and domain APIs through the same tool-use surface.
