Tool use and agents — Seminal AI

The mechanics of tool calling

A model cannot call anything. Tool use is a structured-output convention with a loop around it, and you own the loop.

You send a request carrying tool definitions: a name, a natural-language description, and a JSON Schema for the arguments.
The model either answers normally or emits one or more tool-call blocks — a tool name plus a JSON argument object — and the response ends with a stop reason marking a pending call (tool_use on the Anthropic API).
Your code executes the call. The provider never touches your database; whatever runs, runs on your infrastructure under your credentials.
You append the assistant message verbatim, then append a user message containing a result block for every call, keyed by the call's ID.
You resend the whole conversation and repeat until the stop reason is a normal end of turn.

Two consequences of step 5. The conversation is stateless and resent every turn, so token spend grows roughly quadratically with loop length. And the model's only view of the world is the text you put in tool results — a truncated, misordered, or silently empty result is indistinguishable from reality.

Turn on strict schema validation where the API offers it (Anthropic: strict: true on the tool definition, with additionalProperties: false and an explicit required list). It guarantees arguments that match your schema, not arguments that are correct — a perfectly-typed call naming an account that does not exist still comes back as an error. Parse tool inputs as JSON rather than string-matching the serialized form; escaping conventions differ across models.

Parallel tool calls

A single assistant message can contain several tool-call blocks. Execute them concurrently and return all of the results in one user message. Split them across messages and the transcript shows the model that its parallel call was answered piecemeal; it stops issuing parallel calls for the rest of the conversation, and the latency win disappears with no error anywhere.

Parallelism only helps when calls are independent. Three lookups against three services collapse to one round trip. A read that determines the argument to a write does not.

The agent loop, and whether you need one

The loop above is the same code whether you have written a workflow or an agent. The difference is who chooses the next step.

	Workflow	Agent
Control flow	You write it; the model fills in steps	The model chooses each step
Step count	Fixed and known	Unbounded until a stop condition
Failure mode	Wrong output at a known stage	Plausible wrong path, discovered late
Cost per task	Predictable within a narrow band	Varies by an order of magnitude across runs
Debugging	Reproduce the failing stage	Replay a whole trace
Best for	Known decomposition: classify, then route, then extract	Unknown decomposition: investigate, repair, research
Evaluate by	Per-stage accuracy	Task completion rate and steps per completed task

Before you build the agent, answer four questions honestly:

Complexity. Is the task genuinely hard to specify in advance? "Turn this bug report into a patch" qualifies. "Extract the invoice total from this PDF" does not — that is one call.
Value. Does the outcome justify latency in minutes and a cost orders of magnitude above a single call?
Viability. Is the model actually good at this class of work? Check a real sample before committing architecture to the answer. Compare models on the capability, not the benchmark headline.
Cost of error. Can a wrong action be detected and undone? Agents earn their keep where there is a cheap verifier — a test suite, a type checker, a reviewer, a rollback.

A "no" to any one of them means drop a tier: to a workflow, or to a single call.

Errors compound

Assume each step independently succeeds with probability p. End-to-end success over n steps is p^n.

98% per step, 20 steps: 67%
95% per step, 20 steps: 36%
99% per step, 50 steps: just over 60%

Independence is wrong in both directions. Errors correlate — one bad early read poisons every later step, so long traces cluster into clean runs and total garbage rather than degrading smoothly. But agents also observe their failures: a result saying "no such file" often produces recovery on the next step, which is why returning errors to the model beats raising them. The rule survives either correction: anything requiring dozens of unverified sequential steps to be right is not a job for an agent. Shorten the chain, or add a verifier the loop can run itself.

Tool design

The tool list is part of the prompt. It renders ahead of the system prompt and the messages, is read in full every turn, and is the first thing to stabilize if you want prompt caching to work.

Fewer, better-described tools beat many overlapping ones. Two tools whose descriptions could each plausibly answer the same request produce a coin flip on every call. Merge them, or make the boundary explicit in both ("use this only when you already have an account ID; otherwise use search_accounts").
Model tools on intent, not on your REST surface. One tool per endpoint reproduces your API's accidental structure inside the model's decision space. get_customer_orders(email) is one tool; a lookup plus a list plus a filter is three chances to go wrong.
Constrain arguments. Enums beat free strings. Required fields beat twelve optional ones. Every optional parameter is a decision you have delegated.
Return what the model needs, not the payload you got. Raw API JSON is mostly keys the model will never use, priced at input rates on every remaining turn. Project the fields, cap the row count, and say how many rows were dropped.
When the surface is genuinely large, do not just add more tools. Current APIs support deferred loading, where definitions stay out of context until a search tool retrieves them (at least one tool, including the search tool itself, must stay loaded), and programmatic tool calling, where the model writes sandboxed code that calls your tools in a loop so only the results re-enter context.

Write descriptions for a competent new hire with no access to your codebase: what it does, when to use it, when not to, and what the result looks like. Then read your traces and fix the description that produced the wrong call. Tool descriptions are the highest-leverage prompt text in an agent.

Error handling

Two kinds of failure, routed differently.

	Examples	Handled where	What the model sees
Transport	Timeout, 429, 5xx, connection reset	Retry with backoff inside the executor	Nothing
Semantic	Bad argument, not found, permission denied, precondition unmet	Returned as a result block flagged as an error	The message, and what to do differently

A transport failure costs a full round trip to show the model something it cannot act on. A semantic failure is the signal that lets the loop correct itself, so word it as an instruction ("account_id must be a UUID; call search_accounts first"), not a stack trace. Never drop the result block — a missing result for an emitted call breaks the message sequence and the request is rejected.

Cap repetition: the same tool called with the same arguments three times means the loop is stuck. Sanitize before returning, too; stack traces carry file paths, connection strings, and internal hostnames into a context you may later log, cache, or show a user.

Approval gates

Classify every tool as read, write, or destructive, and enforce that classification in the executor.

A prompt instruction is not an access control. "Always ask before deleting" is a preference the model usually honors and an attacker's tool result can override. If a tool can destroy data, spend money, or send external messages, the gate belongs in the code that dispatches the call — and the credentials it runs under should be scoped so that the worst possible argument is still survivable.

A workable gate shows the human the exact tool name and arguments, never a model-written summary of them. Add idempotency keys so an approved action cannot double-execute on retry, a dry-run variant for anything bulk, and a reversible path for everything destructive. Treat content that entered context from outside — web pages, tickets, emails, files — as untrusted input that may be trying to steer the loop.

Context growth and observability

In a long loop, tool results dominate context. Three defenses, in increasing order of complexity: cap the size of any single result at the source; clear stale tool results out of the history; summarize the earlier conversation into a compact form. Anthropic ships the second and third as context editing and compaction.

Keep the tool list and system prompt byte-stable so the cached prefix survives. A cache read costs roughly a tenth of a fresh input token and a cache write a modest premium above one, so a stable prefix pays for itself within a couple of turns and dominates the bill by turn thirty. Price it in the cost calculator at your real step count; output tokens cost several times input on every current model, so a chatty model costs more than a chatty tool.

Log per step: run ID, step index, tool name, arguments, latency, result size, input and output tokens, stop reason. The unit of debugging is the run, not the request — without a trace ID threading the steps you are reading disconnected requests and guessing. Alert on distribution shifts — steps per run, error-result rate, tools never chosen — because a degraded agent does not throw exceptions, it takes eleven steps where it used to take four.

Measure cost and success per completed task, not per request: a cheaper model that needs three more turns is not cheaper. See the model index for current context and capability figures.

What people get wrong

Splitting parallel tool results across several user messages. Kills parallelism for the rest of the conversation.
Putting the safety rule in the prompt. Gates go in the dispatcher.
One tool per API endpoint. A large, overlapping surface the model picks from badly.
Assuming strict mode means correct arguments. It means schema-valid ones.
No step cap and no repetition cap. The runaway loop is the standard first production incident.
Feeding a non-retryable error back forever. A permission denial does not become permitted on the fourth try.
Reaching for an agent when the decomposition is already known. If you can draw the flowchart, write the flowchart.
Evaluating on single-call accuracy. Per-step accuracy that looks fine at 97% is a 54% end-to-end success rate over 20 steps.