---
name: Test Local Decision Models (/v1/systemone)
description: >-
  Test a local decision model on llama.cpp or Ollama /v1/systemone before using it in a pipeline: labelled set, option order bias, quant comparison, confidence threshold with human fallback, and latency.
---
# Test Local Decision Models (/v1/systemone)

Use this course before a local decision model routes, labels, or scores anything for real. The scripts run against Kev-0.8B served by llama-server on port 8080. The same request works on Ollama at port 11434 with a `model` field.

## End-to-end agent recipe

1. Use [lesson 01](01-what-a-decision-model-returns.llms.md). Serve Kev-0.8B with llama-server, check `output_modalities`, and call `/v1/systemone` with choice, noul, and score questions.
2. Use [lesson 02](02-labelled-set-and-order-bias.llms.md). Run `eval_decision.py` on a labelled JSONL set with shuffled option orders and record accuracy, flips, and confidence.
3. Use [lesson 03](03-compare-quantizations.llms.md). Run `eval_decision.py` on two quants of the same model, compare accuracy, flips, confidence, and p50/p95 latency, and check llama.cpp #30064 and Ollama #18865 and #18769.
4. Use [lesson 04](04-thresholds-fallback-latency.llms.md). Sweep thresholds with `eval_decision.py --threshold`, route low-confidence answers to a human, measure p50/p95 latency, and run decision models with `-np 1` because of llama.cpp #30482.
5. Keep these links exact: [Test AI behavior before you ship](../on-device-eval-harness/), [The agent harness (and how to improve it)](../agent-harness-improve-loop/), [Reliable Local Agents (llama.cpp + Ollama)](../local-agent-reliability/), [Local LLM Runtime Traps (Ollama + llama.cpp)](../local-runtime-traps/).
6. Use the human HTML pages for the reader. Use the `.llms.md` briefs for agent steps and acceptance checks.

## Constraints

- Plain spoken English for a non-native reader. Literal sentences. No quizzes. No em dashes.
- Brand Edge FDE only. Do not name a course author.
- Use the request and response shape from the llama-server README. Do not invent fields.
- Use a small model (Kev-0.8B). Report numbers with the build and hardware they came from.
- Environment setup comes before the first runnable example.

## Key concepts

- Decision models return probabilities and confidence, not text.
- Probabilities are not calibrated for your data.
- Option order can change the answer on hard cases.
- Quantization can change confidence even when picks stay right.
- Multi-slot servers can fail on shared-prefix questions (llama.cpp #30482).

## Takeaways

- Test on your own labelled set before you connect the model to anything.
- Shuffle options and treat flips as uncertain.
- Choose the quant by confidence and latency.
- Act only above a threshold you picked from data. Send the rest to a person.

## Sources

- llama-server README, POST /v1/systemone: https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md
- Ollama releases 0.35.0 and 0.40.4: https://github.com/ollama/ollama/releases
- llama.cpp #30064 and #30482 at https://github.com/ggml-org/llama.cpp/issues
- Ollama #18865 and #18769 at https://github.com/ollama/ollama/issues
- Kev-0.8B GGUF: https://huggingface.co/ggml-org/Kev-0.8B-GGUF