Task contracts
Route by task type, output schema, modality, context, latency, and risk.
LLM model routing
Route each Brain API request to the right model and execution path.
A model router translates task requirements into an approved plan across OpenAI, Anthropic, local models, deterministic services, and future providers—without coupling clients to one vendor name.
Architecture overview
Optimize the service objective, not the model leaderboard.
Model quality depends on the actual prompt, language, modality, tool schema, context, and failure tolerance. A capability registry and application-specific evaluations provide better routing evidence than broad benchmark rankings.
Begin with transparent rules. Enforce privacy, region, modality, and risk constraints first; then choose among compatible routes by latency, cost, and measured task success. Preserve the decision record for replay and analysis.
Classify the taskDetermine the requested capability, modality, size, and service class.
Apply constraintsRemove routes that violate privacy, region, feature, or risk requirements.
Rank candidatesUse measured quality, latency, capacity, and total task cost.
Execute through an adapterTranslate the common contract into provider-specific semantics.
Observe the outcomeRecord route, version, tokens, errors, retries, cost, and task result.
Update the registryUse evaluations and incidents to change traffic deliberately.
Design principles
Make the boundary explicit
These principles keep the implementation understandable as models, data sources, and product requirements change.
Capability registry
Version model features, restrictions, evaluation results, region, and cost.
Policy first
Apply data-location, provider, and authority constraints before optimization.
Thin adapters
Normalize transport and provider features while keeping workflow logic shared.
Safe fallback
Define task-specific retries, circuit breakers, and no-route failure behavior.
Measured outcomes
Compare cost and latency per successful task, including retries and review.
Recommended reading
Go deeper with field guides
Detailed articles connect the architecture to implementation, testing, operations, and safety controls.
LLM Model Routing: Choosing the Right AI Model for Every Request
A practical guide to routing Brain API requests by capability, latency, cost, context size, provider availability, and risk without coupling clients to one model vendor.
On-Device Brain APIs for iOS, Android, and Mac Applications
An architecture guide to local AI inference, mobile and desktop model packaging, hybrid routing, privacy, updates, and test matrices for iOS, Android, and Mac apps.
Self-Hosted Brain API: Deployment, Git Review, CLI, and QA Checklist
A production-oriented plan for self-hosting a Brain API with containers, gateways, model runtimes, queues, observability, CLI operations, Git review, and release QA.
Questions
Common implementation decisions
Should clients request a model by name?
Usually no. A task-oriented service class allows the application to change providers or versions without forcing every client to migrate.
Can a router use more than price and latency?
Yes. Capability, structured-output reliability, tool support, context size, privacy, region, quality, capacity, and risk should all be represented.
When should routing use a learned classifier?
After a deterministic baseline and enough labeled outcomes exist. A learned router should still operate inside hard policy constraints and produce a reviewable decision record.
Is cloud fallback always acceptable for on-device tasks?
No. The route must respect the data class, user setting, and connectivity policy. Silent cloud fallback can violate expectations and privacy requirements.
Next step
Turn the concept into one testable workflow.
Start with a bounded task, versioned request and response schemas, explicit permissions, and a replayable evaluation set.