A Brain API combines untrusted inputs with powerful capabilities. User text, retrieved webpages, documents, images, tool results, and memory records can all contain instructions that conflict with the application's policy. If the model is allowed to interpret every token as authority, prompt injection becomes an access-control problem. Security therefore has to exist outside the model in schemas, permissions, data boundaries, and execution controls.

Threat modeling starts by identifying assets and authority. Assets may include private records, API credentials, source code, unpublished content, model prompts, user memory, and compute budget. Authority includes reading data, sending messages, changing files, publishing pages, running commands, or purchasing services. The design should assume that some inputs are malicious and that model output can be wrong. The goal is containment: one bad instruction should not become broad system access.

Map trust boundaries in the workflow

Draw the path from client to API gateway, context builder, model provider, tool worker, database, and external service. Mark where data crosses a tenant, region, provider, or privilege boundary. A retrieved document is untrusted even when it came from an approved connector. A tool result may contain attacker-controlled text. A memory record may have been inferred from an earlier compromised session. Trust should be based on source and validation, not on where a string appears in the prompt.

For each boundary, list allowed data, authentication, authorization, encryption, logging, retention, and failure behavior. Identify which component can make irreversible changes. The orchestration service may decide what to propose, but a trusted tool gateway should decide whether the caller and workflow may execute it. This separation limits the damage if a prompt or model behavior is manipulated.

Treat prompt injection as content, not authority

Separate system policy, developer instructions, user input, retrieved content, and tool output in the internal message model. Do not concatenate them into an undifferentiated prompt. Mark retrieved content as data and tell the model not to follow embedded instructions, but do not rely on that sentence as the security boundary. The application must still restrict tools and data access independently.

Content sanitization can remove active markup or known exploit patterns, but it cannot determine every malicious instruction. Use allowlisted transformations for files and webpages, and preserve a reference to the original for review. When a task asks the model to summarize a document, the model should not gain more tools because the document requests them. Capabilities come from the task policy, not from the content being processed.

Authorize every tool call in trusted code

A valid tool schema is not an authorization decision. The tool gateway should verify the user, tenant, workflow, requested action, target resource, and approval status immediately before execution. Use short-lived capability tokens or scoped credentials instead of giving the model a master key. Arguments should be validated, normalized, and checked against business rules after model generation.

Classify tools by side effect. Read-only calls can have one policy; reversible changes another; external communication, deletion, code execution, and financial actions require stronger controls. Support dry-run or proposal modes. Record the authorization reason and policy version. A tool should fail closed when context is missing rather than guessing which account or resource the model intended.

Prevent data exfiltration through tools and outputs

An attacker may try to make the system reveal secrets directly, encode them in an answer, or send them through a tool. Restrict which data sources a workflow can read, then restrict which destinations it can write. Network egress allowlists, domain controls, output size limits, and content scanning can reduce exfiltration paths. A research agent that can read internal documents should not automatically be able to post to arbitrary URLs.

Apply data-loss prevention at more than one point. The context builder can exclude secret fields, the tool worker can redact responses, and the output validator can detect prohibited patterns. Do not log blocked secrets in full. Return a policy code and a safe explanation. Review false positives, but avoid weakening a control globally to fix one workflow; use task-specific policies with clear ownership.

Sandbox code, files, and browsers

Code execution needs an isolated environment with resource, filesystem, network, and time limits. Start from a clean image, mount only required files, and destroy or snapshot the workspace according to policy. Do not place production credentials in the sandbox. Proxy any necessary service through a narrow tool that enforces authorization and records requests. A generated command should be treated like untrusted code even when it looks harmless.

Browser and file tools need similar containment. Block local-network targets unless explicitly required, validate redirects, limit downloads, scan archives, and prevent path traversal. A document parser should not inherit the permissions of the orchestration process. Separate extraction from reasoning so malformed files cannot directly compromise the agent runtime.

Protect prompts, keys, and internal configuration

System prompts are not reliable secrets, because model output or indirect attacks may expose portions of them. They can still be proprietary configuration, so limit access and avoid placing credentials or private data inside them. Store provider keys in a secret manager, inject them only into trusted adapters, rotate them, and scope them by environment. Tool credentials should belong to the tool service, not the prompt or model context.

Version prompts and policies like code. Reviews should examine new tool descriptions, changed data sources, and expanded authority, not only wording. A prompt update can alter the effective application behavior even when no compiled code changed. Include prompt and policy versions in traces so an incident can be tied to the exact configuration.

Validate outputs before downstream use

Model output is untrusted until parsed and checked. Use structured schemas for commands, SQL fragments, CMS updates, configuration, and code-review findings. Escape content at the rendering boundary. Do not pass generated HTML into a page, generated shell text into a terminal, or generated database queries into production without validation and appropriate parameterization. The consumer must enforce its own contract.

The OWASP Top 10 for LLM and GenAI initiative provides a current framework for thinking about risks across generative and agentic applications. Use it as a starting taxonomy, then extend the threat model with the organization's actual tools, data flows, deployment environment, and consequences. A checklist is useful only when it maps to concrete controls and owners.

Build security tests into every release

Create adversarial fixtures for direct injection, instructions hidden in retrieved documents, poisoned memory, oversized input, malformed tool arguments, cross-tenant references, secret requests, and repeated attempts to escape a sandbox. Test both refusal and containment. A workflow may continue safely after ignoring a malicious instruction; blocking the entire request is not always necessary.

Run deterministic policy tests in continuous integration. Red-team exercises can explore new paths, but repeatable cases prevent old failures from returning. Review changes to tools and scopes in pull requests, and require additional approval when authority expands. Production monitoring should alert on unusual tool sequences, repeated policy blocks, large exports, and sudden increases in network destinations.

Prepare a containment and recovery plan

Assume that a control will eventually fail. Maintain kill switches for individual tools, providers, workflows, and memory writes. Revoke credentials, disable egress, and stop queued runs without taking unrelated services offline. Preserve relevant traces and configuration versions under restricted access. Document who can make emergency changes and how those changes are reviewed afterward.

Recovery includes correcting polluted memory, invalidating cached summaries, reprocessing affected artifacts, and notifying appropriate stakeholders. A model upgrade alone does not remove compromised state. The incident review should identify which boundary allowed the behavior and strengthen that boundary. Blaming the prompt without changing authority or data flow guarantees a repeat.

Implementation checklist

Before the first production release

  • Mark system policy, user input, retrieved content, and tool output separately.
  • Grant capabilities from trusted task policy, never from retrieved instructions.
  • Authorize every tool call immediately before execution.
  • Separate read, reversible write, irreversible, communication, and code tools.
  • Restrict both data sources and outbound destinations.
  • Run generated code and file processing in isolated, minimal environments.
  • Validate structured outputs at the consumer boundary.
  • Maintain repeatable injection, exfiltration, and cross-tenant tests.
  • Keep audited kill switches and credential-revocation procedures.

Conclusion

Secure agentic systems do not depend on a model always recognizing malicious instructions. They constrain what any model call can read, propose, and execute. Trust boundaries, scoped capabilities, tool authorization, output validation, sandboxing, and incident controls remain in trusted application code.

Start with the smallest authority that completes the task. Add data sources and tools only with explicit owners and tests. Treat prompts as versioned behavior, retrieved content as untrusted, and model output as input to another validator. That approach turns prompt security from a wording exercise into an engineering discipline.