Databricks updated its documentation to highlight Unity Gateway as a centralized control plane for managing model inference across hosted and external providers. By creating custom model services, platform teams can decouple applications from specific model backends, enabling them to centrally enforce identity-aware policies, PII masking, and spending limits. This architecture allows administrators to implement weighted traffic splitting for rollouts and configure ordered fallbacks for high availability. Crucially, teams must treat these gateways as governed data assets, ensuring that both primary and fallback destinations comply with necessary security and regional requirements.
Databricks’ hosted-inference documentation, updated October 30, 2026, directs teams to Unity Gateway for models hosted both by Databricks and by third-party providers. Applications can use that access layer while platform teams manage permissions, spending, and routing centrally. The date marks a documentation update, not a separately documented product launch.
The practical building block is a custom model service: a governed endpoint that can send requests to multiple model destinations. An application can keep calling the same service while administrators change its underlying models, split traffic during a rollout, or add backup destinations.
A shared endpoint consolidates control without making every destination operationally identical. Hosting location, provider terms, regional availability, and routing behavior still matter. This guide analyzes Databricks’ published documentation; it includes no independent performance measurements.
What Unity Gateway Consolidates
Databricks describes Unity Gateway as a control plane for model inference, tools, and skills across AI applications and coding agents. Its documented capabilities include identity-aware policies, personally identifiable information masking, audit logs, usage tracking, budgets, and spend limits. Teams can also bring existing capacity from providers including OpenAI, Anthropic, Amazon Bedrock, and Azure Foundry. These are documented product capabilities, not measured outcomes.
For inference, the documentation describes several access arrangements:
Access Arrangement
Who Hosts the Model?
Practical Consequence
Databricks-served models
Databricks
Model services can use pay-per-token or provisioned-throughput destinations.
Provider-hosted catalog models
The model provider
Databricks supplies access, authentication, and routing; the provider operates the hosting environment.
Connected external providers
The external provider
Teams can bring provider access, including bring-your-own-key connections, under Gateway governance.
The hosted-inference page lists Databricks-hosted model families from companies including Anthropic, OpenAI, Google, and Meta. A model’s developer and its hosting operator are therefore separate facts. Seeing an OpenAI or Anthropic model name alone does not identify where that particular destination runs.
Databricks also makes customers responsible for applicable provider terms. My view is that platform inventories should record the hosting arrangement alongside the model name. Otherwise, a seemingly simple model substitution can conceal a change in the environment receiving application data.
Create a Stable Endpoint for Each Workload
A custom model service translates and routes inference requests to one or more models, supporting real-time and batch inference. It can mix Databricks-served destinations with external models connected through a model provider.
Services are Unity Catalog securable objects, meaning access is governed through catalog privileges. Callers address them using a fully qualified name such as production.ai.support-assistant, including across workspaces or from outside Databricks. Databricks uses this workload-oriented naming pattern in its custom model services documentation.
That separation is useful: an application depends on a support-assistant service, while the platform team owns its model configuration. It reduces the number of application changes required for a model migration. It cannot guarantee that a replacement model produces compatible answers, tool calls, or formatting, so evaluation remains necessary.
Check Region Support and Permissions First
The documented prerequisites include a supported Databricks workspace region and Unity Catalog enabled. Unity Gateway is not supported on AWS GovCloud.
Creating a service requires:
USE CATALOG, USE SCHEMA, and CREATE SERVICE on the target catalog and schema.
EXECUTE on each referenced model.
EXECUTE, USE CATALOG, and USE SCHEMA on referenced model providers.
Appropriate catalog and schema access plus CREATE TABLE where an inference table will be created, if inference logging is enabled.
These are creation prerequisites. Teams must also grant intended callers access to the service.
A separate availability check is essential. Model assets in system.ai are listed globally, but availability depends on the workspace region, cross-geography settings, and the model itself. Databricks instructs users to check the corresponding service in the Unity Gateway UI and confirm query permissions. Merely listing a model does not send customer data to it for inference, according to the availability guidance.
Use the UI or Infrastructure Tooling
The documented UI path is straightforward:
Open AI Gateway in the workspace sidebar and select Create.
Enter the service name and choose its catalog and schema.
Select an eligible Databricks-served model or model provider as the primary destination.
Create the service, then configure features such as inference logging and additional routing.
Catalog Explorer offers another creation path. Programmatic options include REST APIs, Databricks SDKs, the CLI, Terraform, and Declarative Automation Bundles, with DAB support marked Beta in the supplied documentation.
A custom service is optional for basic access. System-provided services in system.ai let teams query models without creating their own endpoint. I would introduce a custom service when a workload needs a distinct owner, access boundary, routing policy, or spending scope.
Separate Permissions, Budgets, and Rate Limits
Unity Gateway brings several controls together, but they solve different problems.
Permissions determine who may call a service. Unity Catalog privileges can separate teams and application identities. Databricks also documents service policies that govern how requests and responses proceed based on their content and the caller’s identity, alongside PII masking and other guardrail capabilities.
Rate limits constrain consumption speed. The model-services documentation gives illustrative configurations of 100 requests per minute for testing and 1,000 requests per minute for production. Those are configuration examples, not published platform ceilings.
Budgets and spend limits govern financial exposure. Databricks documents per-user thresholds and hard caps. Its workload example tags a service with project=support-assistant and scopes a monthly budget to that tag, with an alert at $1,000 of spend. That dollar amount is an example alert threshold, not product pricing.
Usage and inference logs support investigation. The governance documentation describes tracking requests, tokens, and latency, attributing Databricks costs to services and principals, and optionally recording requests and responses in Unity Catalog Delta tables. See the governance overview for these separate control areas.
The operational implication is to design these settings together. A budget alert informs someone; a hard cap stops spending at its configured boundary. A rate limit controls request frequency but cannot by itself predict a monthly bill when token consumption varies.
I would also give inference-log tables deliberate permissions and retention rules. Recording prompts and responses creates another governed data asset, potentially containing the same sensitive material the application handles.
Traffic Splitting and Fallbacks Solve Different Problems
Unity Gateway supports weighted traffic splitting across up to five destinations. Percentages must total 100%. In the UI, teams add destinations under a service’s Routing section and assign each its share.
For example, a platform team could allocate 90% to an established destination and 10% to a candidate replacement. This is a rollout configuration, not evidence that the replacement is cheaper or better.
Databricks documents two routing stages:
Mechanism
When It Applies
What It Does
Traffic splitting
Initial destination selection
Distributes requests according to configured weights, subject to session affinity.
Ordered fallbacks
After a failed attempt
Tries backup destinations sequentially in the configured order.
Failures such as HTTP 429 responses, indicating rate limiting, or 5xx server errors can lead to fallback attempts. The system proceeds through the configured backups until one succeeds or the list is exhausted.
Fallback retries do not repeat the weighted selection process. Databricks also states that traffic splitting cannot be configured on fallback destinations. These mechanics are detailed in the routing and fallback documentation.
Session Affinity Changes How to Read a Rollout
When traffic splitting is configured, Databricks automatically enables session affinity. Requests carrying a session-identifying header stay with the same destination. Requests without such a header follow the weighted split.
That distinction matters when evaluating a 90/10 rollout. A few long sessions can generate many requests, so a short-window request count may differ noticeably from the configured allocation. Teams should inspect their client’s session-header behavior before treating that difference as a routing fault.
Databricks says affinity supports prefix caching and predictable results. The corresponding tradeoff is that a session stays on its assigned destination instead of sampling models on every request.
Approve the Backup Path Too
A fallback can preserve availability while changing the model handling a request. Because one service can mix hosting arrangements, the backup may also involve a different hosting operator.
The practical consequence follows directly from those capabilities: approve every fallback destination for the workload’s data and provider terms before enabling it. Reviewing only the primary destination leaves the failure path outside that review.
Sequential fallback attempts can also add end-to-end latency. Application timeout testing should cover that failure path. Databricks records routing decisions in the routing_information field of system.ai_gateway.usage; inspecting those records helps distinguish initial routing from failover activity.
Automatic Routing Has a More Specific Scope
The Unity Gateway overview describes automatic routing for cost, speed, and quality. Its more detailed governance page specifically presents Smart Routing for coding tasks, selecting a model and harness for each task.
That distinction prevents a misleading interpretation of the product update. A configured 90/10 split is weighted allocation. An ordered fallback reacts to a failed attempt. Neither mechanism, by itself, evaluates response quality and selects the best model for each arbitrary application request.
Databricks’ governance documentation describes Smart Routing separately. Teams should match the documented routing mechanism to their workload instead of assuming that placing models behind a common endpoint enables universal cost optimization.
For an initial deployment, I would start with one workload-specific service, explicit permissions, a spending policy, and observable routing. Add a candidate model and approved fallback after establishing those boundaries. That sequence makes it easier to identify whether an unexpected result came from access configuration, the model change, or a retry to another destination.
Frequently Asked Questions
4 questions
1
Does Unity Gateway Host Every Connected Model?
No. Some models run on Databricks infrastructure, while provider-hosted models and connected external APIs run in provider-operated environments.
2
Is a Custom Model Service Required?
No. System-provided services in system.ai support direct model access. Custom services add workload-specific endpoints, routing, and governance configuration.
3
Does a 10% Traffic Split Mean Exactly 10% of Requests?
Not in every observation window. Routing is weighted, and session-identifying headers pin related requests to one destination.
4
Do Fallbacks Retry Requests With Poor Answers?
The cited documentation describes fallback after request failures, including 429 and 5xx errors. It does not document response-quality judgments as a fallback trigger.
