New models and runtime upgrades

Day one: unknown architecture

Each GGUF file has a general.architecture field. The runtime needs code for that architecture. If it does not have it, loading fails:

Error: 500 Internal Server Error: llama-server process has terminated: exit status 1:
error loading model: unknown model architecture: 'k2-horizon'

That error is from Ollama #18698, a request for the K2 Horizon family. Ollama ships its own llama.cpp build, so a model can work in upstream llama.cpp before it works in Ollama. Read the architecture before you download a big model, or before you file a bug:

uv run python gguf_arch.py model.gguf
# architecture: qwen3
llama-server --version   # which build do you have?

Then try the model with an upstream llama-server build. If upstream loads it and your app's runtime does not, wait for the runtime update. Do not ship a workaround.

Upgrades can break working models

• Ollama #18856: qwen3.6:35b-mlx worked on 0.35.0 and crashed on 0.40.0 and 0.40.1-rc0 with panic: cache: offset 5335 exceeds 5334 stored keys.
• Ollama #18898: gemma4:12b failed to load on 0.40.2 with Gemma4Assistant requires ctx_other to be set.
• Ollama #18968: a user reports 25% to 30% lower tokens per second on 0.40.3 than on 0.40.1.
• Ollama #18909: 0.40.2 started upgrading downloaded models in the background on first run. The user asked for a way to turn that off because of disk space. The 0.40.3 release paused the background upgrade.

Pin, then test, then upgrade

• Pin the runtime version (for example a fixed Ollama release, or a fixed llama.cpp build tag like b11624).
• Pin the model by tag and record its digest from ollama list.
• Run a smoke test on the new version before users get it. Keep the old version ready to roll back.

"""Smoke test a local model before you ship a runtime or model update (Ollama)."""
import json
import sys
import time
import urllib.request

HOST = "http://127.0.0.1:11434"
MODEL = sys.argv[1] if len(sys.argv) > 1 else "qwen3:1.7b"
MIN_TOK_S = float(sys.argv[2]) if len(sys.argv) > 2 else 5.0


def get(path):
    return json.load(urllib.request.urlopen(HOST + path))


def post(path, body):
    req = urllib.request.Request(HOST + path, json.dumps(body).encode(), {"Content-Type": "application/json"})
    return json.load(urllib.request.urlopen(req))


def main():
    version = get("/api/version")["version"]
    show = post("/api/show", {"model": MODEL})
    arch = show.get("model_info", {}).get("general.architecture")
    print(f"ollama {version} | model {MODEL} | architecture {arch}")

    start = time.perf_counter()
    r = post("/api/chat", {"model": MODEL, "stream": False, "think": False,
                           "messages": [{"role": "user", "content": "What is 12 plus 30? Reply with the number only."}],
                           "options": {"temperature": 0, "num_predict": 20}})
    wall = time.perf_counter() - start
    answer = r["message"]["content"].strip()
    tok_s = r["eval_count"] / (r["eval_duration"] / 1e9)
    print(f"answer={answer!r} load+reply={wall:.1f}s gen={tok_s:.1f} tok/s")

    problems = []
    if "42" not in answer:
        problems.append("wrong answer")
    if tok_s < MIN_TOK_S:
        problems.append(f"slower than {MIN_TOK_S} tok/s")
    if problems:
        sys.exit("FAIL: " + ", ".join(problems))
    print("PASS")


if __name__ == "__main__":
    main()

$ uv run python smoke_test.py qwen3:1.7b 3
ollama 0.40.4 | model qwen3:1.7b | architecture qwen3
answer='42' load+reply=0.5s gen=40.5 tok/s
PASS

Set the speed floor to about 80% of what you measure today. Then a drop like #18968 fails the test. Add the context check from lesson 01 to the same job.