Three lanes compete:
- Memory: facts the app already stored about this user or project.
- Search hits: notes or files from the retrieval course or Spotlight.
- Tools: only the tools this turn is allowed to call. Tool schemas cost tokens too.
ScoreRelevance now
CostToken count
FillUntil leftover is gone
enum SliceLane: String, Sendable { case memory, search, tool }
struct ContextSlice: Sendable {
let id: String
let lane: SliceLane
let text: String
let score: Float
let tokens: Int
}
func pack(slices: [ContextSlice], leftover: Int, perLaneCap: Int = 4) -> [ContextSlice] {
var used = 0
var taken: [SliceLane: Int] = [:]
var packed: [ContextSlice] = []
for slice in slices.sorted(by: { $0.score == $1.score ? $0.tokens < $1.tokens : $0.score > $1.score }) {
let count = taken[slice.lane, default: 0]
if count >= perLaneCap { continue }
if used + slice.tokens > leftover { continue }
packed.append(slice)
used += slice.tokens
taken[slice.lane] = count + 1
}
return packed
}
Score memory and search with the same embedder you use in the semantic cache course, or with the retrieval ranker. Tools get a hard allow-list from the current app state, not a similarity score. A calendar tool does not belong in a packing pass about a photo.
Cap each lane so one noisy index cannot eat the window. Four memories and four hits is a sane start.
Drop, do not summarize yet. First ship drops low slices. Summaries come later, and they need their own eval.
Next, put the packed slices into a Dynamic Profile.
Key concepts
- Three competing lanes: memory, search hits, and tools.
- Each slice gets a score, a token cost, and a lane.
- Fill leftover budget from the top. Drop the rest.
- Cap each lane. Four memories and four hits is a sane start.
Takeaways
- Score memory and search with your ranker. Tools use a hard allow-list.
- Tool schemas cost tokens too.
- Drop low slices first. Do not summarize until you have eval for summaries.