Cosine similarity is a simple score: 1 means the same direction, 0 means no shared direction. You do not need a vector database. A few hundred rows in memory is enough for a first ship.
New vectorThis question
ScanBest cosine
GateHit or miss
struct CacheRow: Identifiable, Sendable {
let id: UUID
let question: String
let answer: String
let embedding: [Float]
let createdAt: Date
let expiresAt: Date?
}
func cosine(_ a: [Float], _ b: [Float]) -> Float {
precondition(a.count == b.count)
var dot: Float = 0
var na: Float = 0
var nb: Float = 0
for i in a.indices {
dot += a[i] * b[i]
na += a[i] * a[i]
nb += b[i] * b[i]
}
let denom = (na.squareRoot() * nb.squareRoot())
return denom == 0 ? 0 : dot / denom
}
func lookup(question: [Float], rows: [CacheRow], minimum: Float, now: Date) -> CacheRow? {
let live = rows.filter { $0.expiresAt.map { $0 > now } ?? true }
return live.max { cosine($0.embedding, question) < cosine($1.embedding, question) }
.flatMap { cosine($0.embedding, question) >= minimum ? $0 : nil }
}Pick a starting threshold
Start high, around 0.90 to 0.94, then lower only after you read misses. A low threshold looks like a high hit rate and serves the wrong answer. Time-sensitive rows need expiresAt.
Normalize lightly before you embed: trim space, collapse repeats, keep the user's words. Do not stem so hard that "cancel" and "reschedule" look the same.
Product rule. A miss that generates is better than a hit that lies.
Next, count hits so you can defend the threshold.
Key concepts
- Store embedding, question, answer, created time, and optional expiry as a cache row.
- Score with cosine similarity and gate on a minimum threshold.
- Start high, around 0.90 to 0.94. Lower only after you read misses.
- Time-sensitive rows need an expiry.
Takeaways
- A miss that generates beats a hit that lies.
- A few hundred in-memory rows is enough for a first ship.
- A low threshold can look like a high hit rate while serving wrong answers.