Overview: Tool use and function calling

Asteria Okay, so we've been hand-waving this for weeks and I think it's finally rude not to explain it. Tool use and function calling keep popping up like we're assuming everybody already has the plumbing in their head.

Draco Yeah, and we really have been doing that thing where we say it like it's obvious. It’s not obvious. It’s one of those core AI ideas that sounds tiny until you realize it’s the difference between “the model said a thing” and “something actually happened.”

Asteria Exactly. And I keep thinking about that listener question in the most basic way possible: if a model can write a perfect calculator command, why doesn't it just know the answer already? That gap is the whole episode, right?

Draco Right. The clean mental model is: the model is a very smart writer, but it doesn't have hands. It can draft the instruction, but some other system has to pick it up and do the work.

Asteria So like a receptionist and a specialist office. The receptionist doesn't do the surgery, but they know which form to send and who to hand it to. Okay, that is such an Exploring Next take, but it helps.

Draco It helps because the receptionist isn't guessing the outcome. They’re routing the request. Tool use is that routing step, except instead of a form it’s usually a structured request the software can read.

Asteria And just to pin down the basics for a second, a language model is the thing generating the text in the first place. It’s a pattern learner trained on lots of text, so it can produce convincing answers, but that doesn't mean it can reach out and check the weather or run math.

Draco Yeah. It predicts the next token, which is just a fancy way of saying it keeps generating the next piece of text that fits the prompt and context. That’s powerful, but it’s still text generation. It’s not execution.

Asteria Mm-hm.

Draco And prompting and in-context learning matter because the model has to be taught, in the moment or by training, that a certain pattern means “ask for help instead of improvising.” In-context learning is just the model picking up the rule from examples in the conversation or the system prompt, not from being retrained on the spot.

Asteria Okay, so it's learning the convention from the setup and the examples in front of it, not magically becoming a different model. That part always gets blurred together in casual talk.

Draco Exactly. Then the API design part shows up because the outside world needs a clean contract. A function signature is basically the shape of the action: what the function is called, what arguments it expects, and what it returns.

Asteria Right, like the model can't just say “do weather” and hope. It needs to say which location, which unit, maybe which date, and the runtime has to know what to do with that.

Draco And that leads to structured output formats. Usually that's JSON, which is just a predictable text format with keys and values. The important bit is not the syntax trivia, it's that the system can parse it reliably instead of trying to guess what the model meant.

Asteria So the model is basically writing a tiny machine-readable note instead of a freeform sentence. That note says, “Call this function with these inputs.”

Draco Yeah. And then the runtime intercepts it. The model doesn't execute the tool itself. The surrounding software sees the structured request, runs the real calculator or search or database query, and sends the result back into the conversation.

Asteria Wait, so the model doesn't just keep talking and hope the answer appears in the next sentence? Because that is the part people blur together all the time.

Draco No, that's the exact distinction. The model proposes. The system executes. Then the model gets the result as fresh context and continues from there.

Asteria I love that this is the glamorous future of AI and it's still mostly “propose, execute, continue.” Very cinematic.

Draco It's deeply unglamorous, which is usually how the useful stuff works. And that loop is why function calling is so valuable: the model can ask for facts it doesn't have, math it shouldn't fake, or actions it can't perform by itself.

Asteria So if I ask it to calculate forty-seven times eighty-nine, the model can choose the calculator instead of pretending it's doing mental math in public. Which, honestly, good call.

Draco Exactly. Or if it's pulling current data, it can call search or a database instead of hallucinating whatever sounds plausible. That’s the grounding piece: the answer comes back from the real system, not from the model’s memory of text.

Asteria Okay, before we go too far, I want the first concrete example because abstractly this sounds clean and then reality gets messy. Like that episode we did on the warm, GUI feeling for CLIs — the whole point there was that the model wasn't just typing, it was choosing the right command at the right moment.

Draco Yeah, that’s a good anchor. A command-line interface becomes a lot more usable when the model can call a shell-like tool instead of trying to narrate what you should type. The function call is the bridge between “I know the command” and “the command actually ran.”

Asteria And the weird part is that this is where product value shows up immediately. If the tool loop works, the user doesn't care whether the model “thought hard” or not. They care that the thing shipped, the query ran, the file changed, the right result came back.

Draco Mm-hm.

Asteria Let me put that a better way. The user experience is not “wow, the model generated a beautiful JSON blob.” The user experience is “it actually looked up the thing, and I didn't have to copy-paste three times.”

Draco Yeah, that's the right framing. And because you're forcing the model into a structured request, you can validate it before execution. That matters a lot. A freeform sentence can be ambiguous. A function signature can reject missing or malformed arguments.

Asteria So the structure is doing some of the safety work, not just the convenience work.

Draco Exactly. If the model says `search_weather` with no location, the runtime can say no. Or if it asks for a field that doesn't exist, the call fails before anything external gets touched. That’s boring, but boring is good.

Asteria And I think this is where people get confused by the word “agent.” They hear agent and imagine a tiny autonomous wizard. But a lot of the time it’s just a model that knows how to choose a tool and then hand off to ordinary software.

Draco Yes. The “agentic” part is mostly a loop with decisions in it. It isn't magic autonomy. It's a model, a tool list, some rules about when to call, and code that keeps the loop going until the task is done or the system stops it.

Asteria Okay, let's stay with the loop for a second because that's the actual mechanism. The model looks at the prompt, decides it needs help, emits a structured request, the runtime catches it, runs the function, and then feeds the result back. That's the whole skeleton?

Draco That's the skeleton. And it can repeat. A search result can lead to another search, or a database lookup can lead to a calculator call, or a web result can lead to a final answer. The model is basically negotiating with the outside world one step at a time.

Asteria Oh interesting.

Draco The important part is that each step is conditional. The model doesn't have to decide everything in advance. It can inspect the result and then choose the next action. That’s why tool use is so much more flexible than just stuffing a bigger prompt.

Asteria Right, because the world isn't a fixed blob of text. It's live. The answer might depend on current inventory, fresh logs, a database row, or just whether the file exists right now.

Draco Yep. And that’s also why prompting alone runs out of room pretty fast. You can ask the model to simulate a workflow, but if it never actually touches the external system, you’re still just getting a well-written guess.

Asteria Mm-hm.

Draco Now, one subtle thing: there are a few names for the same basic idea. OpenAI says function calling. Anthropic tends to say tool use. Some people say action generation. The label changes, but the mechanism is still the model producing a structured request that a runtime can execute.

Asteria And the runtime side is where the product gets real. Because once you've got that handoff, you can plug in all kinds of things. Calculators, search, databases, code execution, scheduling, whatever your app can actually do.

Draco Right. And that’s where the function signature matters again. If the tool is a database query, the signature tells the model what fields it can ask for. If it’s a calendar action, it tells the model what a valid event request looks like. The model is not inventing the interface from scratch every time.

Asteria So the interface is part of the learning problem. The model has to learn the shapes of the tools, not just the words around them.

Draco Exactly. And this is where fine-tuning versus prompting shows up. Some systems teach the model with examples until it gets good at emitting the right format. Others rely more on instruction in the prompt. Fine-tuning is usually more reliable, but prompting is easier to change.

Asteria Which is basically the eternal software trade-off: more reliable usually means more setup. Shocking, I know.

Draco I know, devastating revelation.

Asteria And I keep thinking of that local-models-for-coding episode. There, too, the interesting part wasn't “can the model talk about code” but “can it call the right tools and recover when the harness is messy.” Tool use is the thing that turns a text model into something that can touch a workflow.

Draco Yeah, and that episode had the right cautionary note. Smaller models can be surprisingly capable, but tool calling still fails in annoying ways. Bad schemas, wrong parameters, malformed outputs. The model can be good enough and the tool loop can still be brittle.

Draco Miracles do happen on this show.

Asteria Come on, Draco, don't get smug now. We finally made one of our favorite little plumbing jokes make sense, and I cannot believe this is how we're spending a Wednesday.