Function Calling in Practice: Designing Good Tools for LLMs

Function calling is the mechanism that turns a language model from a text generator into something that can do things: you describe your functions in the request, and instead of (or alongside) prose, the model replies with a structured request to call one — a name and arguments matching the schema you declared. Your code executes the function, feeds the result back, and the model continues with real data in hand. I covered the concept in the agents post; this one is about the craft.

Because here's what building with it teaches you quickly: most "the model is being dumb" failures are actually tool-design failures. The model can only be as good as the interface you hand it. It can't read your codebase, your database schema, or your team's assumptions — the tool definitions are its entire knowledge of your system. That makes tool design a genuine API-design discipline, with the unusual twist that your API consumer is a language model that takes everything you write literally and fills every gap with a guess.

Write descriptions for a stranger, not a colleague

A description like "Fetches user data" forces the model to guess: which user? what data? when should this be used instead of that other similar tool? Every guess is a failure waiting for production traffic.

Write each description the way you'd brief a smart temp on their first day — what it does, when to use it, and when not to:

{
  "name": "search_orders",
  "description": "Search the customer's own orders by status or date range.
    Use when the user asks about their orders or deliveries.
    Not for product searches — use search_products for that.",
  ...
}

The "not for X — use Y" line matters whenever two tools could plausibly overlap. Tool-choice confusion is one of the most common failure modes in practice, and it's far cheaper to fix in the description than to detect afterward. If you find yourself writing a long disambiguation paragraph, though, take the hint: the two tools probably shouldn't both exist.

Parameter descriptions deserve the same care as tool descriptions. "date_from": "string" invites chaos; "date_from": "Start of range, ISO 8601 date, e.g. 2026-03-01. Defaults to 30 days ago." gets you correct calls. Format examples in descriptions are the cheapest reliability upgrade available.

Design parameters that are hard to get wrong

Every parameter is a chance for the model to hallucinate a value. Shrink that surface:

Use enums wherever values come from a fixed set. A free-text status string invites "shipped?" and "SHIPPED" and "sent"; an enum of ["pending", "shipped", "delivered"] doesn't.
Make parameters optional with sane defaults instead of requiring the model to invent values it doesn't have. A required parameter the model can't know is an instruction to hallucinate.
Never ask for what the system already knows. If the user is authenticated, the tool shouldn't have a user_id parameter at all — inject it server-side from the session. Anything the model supplies, the model can supply wrongly; anything security-relevant must come from your code, not the conversation. This is the single most important rule in this post.
Prefer a few coarse tools over many fine ones. Ten overlapping micro-tools multiply choice errors. One search_orders with good filters beats four variants of it. Models handle a handful of well-differentiated tools much better than a drawer of similar ones.

Results are prompts too

Whatever your tool returns gets fed straight back into the model's context, so the return value is part of your prompt engineering — it's the model's only window into what actually happened. Three practical rules:

Return compact, relevant fields — not a raw database row with thirty columns of internal noise. Every token of junk costs money and dilutes the model's attention on the fields that matter. Design the result shape for the conversation, not for your ORM.

Make errors instructive. A bare 500 or "error" gives the model nothing to act on, so it will either give up or make something up. "Date range too large: maximum is 90 days. Retry with a narrower range." frequently gets you a corrected retry on the very next turn, with no code on your side. You're writing error messages for a reader that will actually follow the instructions in them — an odd luxury.

Say "nothing found" explicitly. An empty string or [] with no explanation is ambiguous — some models will fill the vacuum with something plausible-sounding. "No orders matched status=shipped in the last 30 days" closes the door on that and gives the model something honest to tell the user.

Assume every call can be wrong

Treat the model like an untrusted client, because it is one — a well-meaning but suggestible client whose inputs are partly authored by whatever the end user typed:

Validate arguments against your schema before executing — check types, ranges, and that referenced entities exist — and return a clear message on failure rather than throwing. The model will often self-correct if told what was wrong.
Enforce authorization in the tool, never in the prompt. "Only query the current user's data" as an instruction is a suggestion that prompt injection can override; a server-side filter is a guarantee. Prompts are UX; the tool boundary is security.
Gate destructive actions. Reads can be free; deletes, payments, and messages to other humans deserve friction — either a human-in-the-loop confirmation or a two-phase design (prepare_refund returns the details for review, confirm_refund executes). Design so that the worst plausible mistake is annoying, not catastrophic.
Cap the loop. Set a maximum number of tool calls per request and log every call with its arguments and results. When something goes strange in production — and it will — that log is the only way to reconstruct what the model was "thinking."

Test tools like an API, evaluate them like a prompt

Tools sit in both worlds, so they need both kinds of testing. Unit-test the functions themselves with ordinary tests — validation, authorization, error paths. Then build a small evaluation set of realistic user requests with the tool calls you'd expect, and run it whenever you change a description, add a tool, or upgrade the model. Description changes feel harmless — it's just prose, right? — but they're behaviour changes, and they deserve the same regression safety net as code.

The takeaway

Function calling puts a language model behind an API surface you design, and the quality of that surface decides everything. Describe tools like documentation for a stranger, constrain parameters so mistakes are hard to express, return results that teach the model what happened, and validate as if the caller were hostile — because sometimes, via the user's input, it is. Do this well and the "unreliable" agent you were debugging mostly fixes itself; the model was never the weak link.