Thinking output in the wrong field

Where the answer can land

An OpenAI-style reply has content, tool_calls, and (on local servers) reasoning_content. A normal reply puts the answer in content or a call in tool_calls. These reports show other results:

• llama.cpp #30246: with K2 Horizon at low or medium reasoning effort, the final text and tool calls stay inside reasoning_content. finish_reason is stop, so the client sees a successful empty reply.
• llama.cpp #30256: with Gemma 4, a JSON schema request still allows thinking even with enable_thinking: false. On long prompts the thought never ends and the request stops at length with empty content.
• mlx-lm #1946: mlx-lm 0.32.0 returns finish_reason: "tool_calls" but no tool_calls field.

See it on your own server

curl -s localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "What is the weather in Paris? Use the tool."}],
  "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get weather for a city",
    "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}],
  "max_tokens": 400,
  "chat_template_kwargs": {"enable_thinking": true}
}' | jq '.choices[0] | {finish_reason, message}'

Run it with enable_thinking set to true and then false. On the test box with Qwen3 1.7B, thinking on gave a long reasoning_content and finish_reason: "tool_calls"; thinking off gave the tool call with no reasoning. Write down what your model does for each setting.

A parser that checks every case

"""Read a chat completion safely: answer, tool calls, and reasoning may land in odd places."""
import json
import re

TOOL_MARKUP = re.compile(r"<\|?[a-z_|]*tool_call", re.IGNORECASE)


class ParseProblem(Exception):
    pass


def read_reply(response: dict) -> dict:
    choice = response["choices"][0]
    msg = choice.get("message") or {}
    finish = choice.get("finish_reason")
    content = (msg.get("content") or "").strip()
    reasoning = (msg.get("reasoning_content") or msg.get("reasoning") or "").strip()
    calls = msg.get("tool_calls") or []

    if finish == "tool_calls" and not calls:
        raise ParseProblem("finish_reason is tool_calls but tool_calls is missing")
    if not content and not calls:
        if TOOL_MARKUP.search(reasoning):
            raise ParseProblem("tool call markup is inside reasoning_content")
        if finish == "length":
            raise ParseProblem("ran out of tokens before any answer (thinking never ended?)")
        raise ParseProblem("empty reply: no content and no tool_calls")
    return {"content": content, "tool_calls": calls, "reasoning": reasoning, "finish": finish}


def read_json_content(content: str) -> dict:
    text = content.strip()
    if text.startswith("```"):
        text = text.strip("`").split("\n", 1)[-1]
    try:
        return json.loads(text)
    except json.JSONDecodeError as err:
        raise ParseProblem(f"content is not JSON: {err}") from err

Test the bad cases without a server:

from safe_parse import ParseProblem, read_reply

bad = [
    {"choices": [{"finish_reason": "tool_calls", "message": {"role": "assistant", "content": None}}]},
    {"choices": [{"finish_reason": "stop", "message": {"content": "", "reasoning_content": "ok <|tool_call|>read"}}]},
    {"choices": [{"finish_reason": "length", "message": {"content": "", "reasoning_content": "thinking..."}}]},
]
for reply in bad:
    try:
        read_reply(reply)
    except ParseProblem as err:
        print("caught:", err)

caught: finish_reason is tool_calls but tool_calls is missing
caught: tool call markup is inside reasoning_content
caught: ran out of tokens before any answer (thinking never ended?)

What to do when it happens

• Retry once with thinking off, or with a higher reasoning effort if the model has levels (in #30246 high effort worked).
• Never parse tool calls out of reasoning_content and run them. Treat that as a failed turn.
• Keep a small test script for each thinking setting and run it after every runtime update.