Tool calls that do not break the chat

Three ways tool calling fails

• Broken arguments. The model sends arguments like {. In llama.cpp #22948 this stopped generation, and the user had to restart everything.
• A schema that breaks the grammar. llama-server builds one grammar from all tool schemas. One bad schema can break every tool. llama.cpp #25923 shows an empty-object schema and a very large maxLength making invalid grammar. llama.cpp #30056 shows a request with about 60 tools failing before any token is made. llama.cpp #30321 shows an optional field listed before a required one getting dropped with no warning.
• Errors in long tool loops. Ollama #17778 reports a 500 error ("no user query found in messages") after many tool calls in one chat.

On the test box (build b11624, Qwen3 1.7B), a tool with "properties": {} and a tool with maxLength: 524288 both worked. That does not mean your build, model, and template are safe. Test your own tool list on your own build.

Keep schemas small and flat

• Send only the tools this step needs. Fewer tools means a smaller grammar.
• Use flat objects with string, number, and boolean fields. Avoid deep nesting.
• Do not use huge maxLength values. Check length in your own code instead.
• List required fields first. Give every tool at least one real parameter.

Check, then run, then save

This loop uses the parser from lesson 02 (safe_parse.py). It checks each tool call before it runs. If the reply is bad, it logs it, does not save it, and asks again.

"""A small agent loop that checks tool arguments and keeps bad turns out of history."""
import json
import urllib.request

from safe_parse import ParseProblem, read_reply

URL = "http://127.0.0.1:8080/v1/chat/completions"
SYSTEM = "You are a helpful assistant. Use tools when they help."  # never put a timestamp here

TOOLS = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the weather for one city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string", "description": "City name, like Paris"}},
            "required": ["city"],
        },
    },
}]


def get_weather(city: str) -> str:
    return json.dumps({"city": city, "temp_c": 18, "sky": "cloudy"})


HANDLERS = {"get_weather": get_weather}


def check_args(name: str, raw: str) -> dict:
    if name not in HANDLERS:
        raise ValueError(f"unknown tool {name!r}")
    args = json.loads(raw)  # raises on "{" and other broken JSON
    schema = next(t["function"]["parameters"] for t in TOOLS if t["function"]["name"] == name)
    for key in schema.get("required", []):
        if not isinstance(args.get(key), str) or not args[key].strip():
            raise ValueError(f"missing or empty argument {key!r}")
    extra = set(args) - set(schema["properties"])
    if extra:
        raise ValueError(f"unexpected arguments {sorted(extra)}")
    return args


def chat(messages, think=False, max_tokens=512):
    body = {
        "messages": messages,
        "tools": TOOLS,
        "max_tokens": max_tokens,
        "chat_template_kwargs": {"enable_thinking": think},
    }
    req = urllib.request.Request(URL, json.dumps(body).encode(), {"Content-Type": "application/json"})
    return json.load(urllib.request.urlopen(req))


def run(question: str, max_steps: int = 4, retries: int = 2) -> str:
    history = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": question}]
    for _ in range(max_steps):
        for attempt in range(retries + 1):
            try:
                reply = read_reply(chat(history))
                calls = [(c, check_args(c["function"]["name"], c["function"]["arguments"]))
                         for c in reply["tool_calls"]]
                break
            except (ParseProblem, ValueError, json.JSONDecodeError) as err:
                print(f"bad turn (attempt {attempt + 1}): {err}")  # log it, do not append it
        else:
            return "Sorry, the model did not give a usable answer."
        if not calls:
            return reply["content"]
        history.append({"role": "assistant", "content": "", "tool_calls": reply["tool_calls"]})
        for call, args in calls:
            result = HANDLERS[call["function"]["name"]](**args)
            history.append({"role": "tool", "tool_call_id": call.get("id", ""), "content": result})
    return "Stopped after too many steps."


if __name__ == "__main__":
    print(run("What is the weather in Paris?"))

Start the server and run it:

llama-server -hf Qwen/Qwen3-1.7B-GGUF:Q8_0 --port 8080 -c 8192 -np 1
uv run python tool_loop.py
# The weather in Paris is currently 18°C with clouds.

The important line is the comment # log it, do not append it. A bad turn goes to your logs, not into history. The next request is then the same as before, and the model gets a clean second try.

Cap the thinking

Reasoning models can think for a very long time before a tool call. Set max_tokens on every request. llama-server also has --reasoning-budget N to limit thinking tokens, and --reasoning off to turn it off. Set max_steps in your loop so a model that calls tools forever stops.

To compare engines or quantizations, run the same small set of tool tasks on each and count how many pass. The agent harness course shows how to grow that set from real traces.

Sources

• llama.cpp docs: Function calling
• Ollama docs: Tool calling