How attacks and leaks show up on device · Stop prompt attacks and leaks

Environment setup

Use Xcode 27. New SwiftUI app, iOS 27 or macOS 27. Add Foundation Models. You will wrap session.respond, not replace the model.

xcodebuild -version
# New App, SwiftUI, iOS 27 or macOS 27. Add FoundationModels.

Two failures show up on device. A prompt attack is untrusted text that tries to change the model's job. A leak is private data that reaches the model, a tool, or a cloud model when it should have stayed in your store.

Untrusted textMay steer the model

Private dataMay enter the prompt

DeviceYour last chance to stop both

Untrusted text includes mail, web pages, OCR, chat paste, and MCP tool results. If you drop that text into respond next to your instructions, the model may follow the pasted words instead of your instructions. That is prompt injection. It is not a special Apple bug. It is how you built the prompt.

LanguageModelSession accepts guardrails: and defaults to Guardrails.default. That filter is Apple's safety layer. It is not a complete injection detector. It does not redact names, emails, or account numbers for you. An Edge FDE still writes middleware.

import FoundationModels

// Illustrative. Confirm Guardrails.default and the session initializer in the SDK.
let session = LanguageModelSession(guardrails: .default)

func ask(_ pasted: String) async throws -> String {
    let clean = try rejectInjection(pasted) // your check first
    let reply = try await session.respond(to: clean)
    return redactPII(String(describing: reply))
}

Your checksInjection and PII

SessionGuardrails.default

ModelOn-device or PCC

If you later switch a Dynamic Profile to PrivateCloudComputeLanguageModel, the packet leaves the device into Apple's Private Cloud Compute. Redact first. See WWDC26 session 242 for model switches. Read Run tools in a safe box, Talk to tools with MCP, and Make several agents agree first for tools that must also refuse bad arguments.

Rule. Treat pasted and fetched text as untrusted. Treat account data as leave-the-store-only-if-redacted.

Next, put one middleware type in front of the session and the tools.

Key concepts

Untrusted sources include mail, web, OCR, chat paste, and tool results.
Injection drops untrusted text next to instructions in respond.
Guardrails.default does not document injection detection or PII redaction.
PrivateCloudComputeLanguageModel sends packets off device. Redact first.

Takeaways

Treat pasted and fetched text as untrusted.
Treat account data as leave-the-store only if redacted.
Write middleware for injection and PII. Do not rely on Apple's default alone.