Function calling turns a text-generating LLM into an agent operating systems: the model returns structured JSON describing a tool call, your code executes it, and the result feeds back into context. Demos are easy; production charges for guardrails. The patterns below separate demos that impress from agents that survive real traffic.

Schema quality decides reliability

A great model fills a poor schema with plausible garbage. Schema discipline comes before model choice: strict types in JSON Schema, enums over free strings, and detailed descriptions per parameter with value examples. A date field described as just "date" receives "tomorrow"; described as ISO 8601 with a valid range, it receives 2026-05-24. A tool catalog that grows too large for one request confuses the model as well; keep it below 20 per call and compose flows instead of stacking options.

The agent loop and step budgets

The classic cycle: plan, act, observe, repeat until the goal is met. Without a step cap, an impossible goal becomes an infinite loop burning tokens against rate limits. Cap iterations between 10 and 25 and return an explicit failure message on breach: "could not complete X after 15 attempts" is product behavior; a stack trace is a bug. Log the termination reason; goal reached, budget exceeded, and repeated error tell different stories.

Tools that survive retries

  • A few composable tools beat many overlapping ones: 8 well-designed tools outperform 40 confusing ones
  • Idempotent operations survive retries; pass an agent-generated request_id for deduplication
  • Structured errors the model can reason about: {"error": "card_declined", "retryable": false} shapes the next action

Autonomy tiers matched to blast radius

Autonomy is a spectrum, and blast radius picks the tier. Read queries run on their own within limits; writes to a production database require human approval; payments execute behind explicit confirmation. Three tiers cover almost everything: suggest-only, execute-with-approval, autonomous-with-limits. Tier changes track risk changes: a fresh payment integration starts suggest-only for two weeks.

Observability for every call

Log every tool call with inputs, outputs, latency, and token spend; OpenTelemetry spans tie multi-step chains into a single trace. Store the assembled prompt and raw result on each trace; incident replay depends on both. Without this, debugging an agent means guessing; with it, "why did the agent pick that tool" becomes a query.

Known failure patterns

  • Infinite retry loops on a failing tool: circuit breakers in the executor, backoff with a hard ceiling
  • Hallucinated tool names: validate at dispatch against a closed list and return a descriptive error to the model
  • Context window overflow from verbose tool results: truncate with summaries and paginate large payloads