← All courses ← Course home

Lesson 05 of 05

Read a slow or wrong answer

A useful trace answers two questions: which hop was late, and which hop produced the wrong path.

Agent brief (llms.md)

Open your signpost lane and the Foundation Models tree together. Start from ai.request. Then read children left to right.

Longai.context_pack
Longai.context_pack
Orai.model
Longai.context_pack
Orai.model
Orai.tool / ai.guardrail

Slow

If ai.context_pack is long, cut history, trim Dynamic Instructions, or move a disk read off the request. If ai.model is long and Instruments shows a long yellow bar, the prompt is large. If PCC is in use, the first token includes the network. If ai.tool is long, fix the tool, not the model.

Wrong

If the model called the wrong tool, the Instruments tree and your ai.tool spans will disagree with the path you wanted. Turn that path into a TrajectoryExpectation in Test AI behavior before you ship. If the model was right and the UI was wrong, ai.guardrail or the view is the hop.

If a Dynamic Profile never changed, you will see one instruction set for the whole request. Check the profile body and the app state it reads. Version prompts like product copy keeps the instruction version on the request so you can replay it.

Primary sources: WWDC26 243 and Recording Performance Data.

Short form. Name the hop. Fix that hop. Do not restart the model first.

Key concepts

  • Read the signpost lane and the Foundation Models tree together from ai.request.
  • A long ai.context_pack means cut history or trim instructions.
  • A long ai.model means check prompt size or PCC network.
  • If the model is right and the UI is wrong, suspect ai.guardrail or the view.

Takeaways

  • Name the hop, then fix that hop. Do not restart the model first.
  • Wrong tool paths become eval samples.
  • Keep the instruction version on the request so you can replay.