Function Calling

Function calling is the structured API surface that lets an LLM invoke external functions. You describe each function with a JSON schema — name, parameters, types, descriptions — pass it alongside the user message, and the model can respond either with normal text or with a typed call request: function name plus a JSON object of arguments. The runtime parses the call, executes the underlying code or API, and returns the result on the next turn.

The schema

A function definition looks roughly like this on every major provider:

{
  "name": "search_docs",
  "description": "Search the customer's internal documentation for a query.",
  "parameters": {
    "type": "object",
    "properties": {
      "query": { "type": "string", "description": "Search query" },
      "top_k": { "type": "integer", "default": 10 }
    },
    "required": ["query"]
  }
}

The model is fine-tuned during post-training to honor the schema — to emit calls only with valid argument shapes, only to functions that exist, only when a tool call is actually appropriate. Names, descriptions, and required-field lists are all load-bearing for the model’s decisions.

Provider differences

OpenAI, Anthropic, and Google all expose function-calling APIs, with provider-specific message-formatting differences:

• OpenAI uses a tools parameter and tool_calls in the assistant message
• Anthropic uses tools and emits tool_use content blocks
• Google uses function_declarations and function_call parts

The schema vocabulary is roughly compatible, so cross-provider portability is achievable but rarely free. Most production agent harnesses (LangChain, LlamaIndex, the MCP ecosystem) abstract over the differences.

What “good” function descriptions look like

The model picks tools mostly off the description and parameter docstrings. Good descriptions:

• Say when to use the tool, not just what it does (“Use this when the user asks about company-internal information that wouldn’t be in the model’s training data”)
• Disambiguate from neighbors (“Unlike search_web, this only searches internal docs”)
• Note constraints the model otherwise wouldn’t know (“Returns at most 50 results; for broader queries, use list_documents”)

The classic anti-pattern is one-line descriptions copy-pasted from internal API docs. The model doesn’t know what your acronyms mean.

Where this gets fragile

Argument-shape mismatches are common: the model invents an enum value, drops a required field, or sends a string where a number was expected. Strict-mode schemas (JSON-schema validation enforced at decode time) and retry-with-error-message patterns are the standard mitigations.

Token cost is the other failure axis. Every tool definition lives in the prompt on every turn. With 50 tools and verbose descriptions, you pay 5K+ tokens of overhead per turn before the user has said anything. Production agents prune the catalog to per-turn-relevant tools, often via a specialized tool-selection model .