Databricks’ hosted-inference documentation, updated October 30, 2026, directs teams to Unity Gateway for models hosted both by Databricks and by third-party providers. Applications can use that access layer while platform teams manage permissions, spending, and routing centrally. The date marks a documentation update, not a separately documented product launch.
The practical building block is a custom model service: a governed endpoint that can send requests to multiple model destinations. An application can keep calling the same service while administrators change its underlying models, split traffic during a rollout, or add backup destinations.
A shared endpoint consolidates control without making every destination operationally identical. Hosting location, provider terms, regional availability, and routing behavior still matter. This guide analyzes Databricks’ published documentation; it includes no independent performance measurements.
What Unity Gateway Consolidates
Databricks describes Unity Gateway as a control plane for model inference, tools, and skills across AI applications and coding agents. Its documented capabilities include identity-aware policies, personally identifiable information masking, audit logs, usage tracking, budgets, and spend limits. Teams can also bring existing capacity from providers including OpenAI, Anthropic, Amazon Bedrock, and Azure Foundry. These are documented product capabilities, not measured outcomes.
For inference, the documentation describes several access arrangements:
| Access Arrangement | Who Hosts the Model? | Practical Consequence |
|---|---|---|
| Databricks-served models | Databricks | Model services can use pay-per-token or provisioned-throughput destinations. |
| Provider-hosted catalog models | The model provider | Databricks supplies access, authentication, and routing; the provider operates the hosting environment. |
| Connected external providers | The external provider | Teams can bring provider access, including bring-your-own-key connections, under Gateway governance. |
The hosted-inference page lists Databricks-hosted model families from companies including Anthropic, OpenAI, Google, and Meta. A model’s developer and its hosting operator are therefore separate facts. Seeing an OpenAI or Anthropic model name alone does not identify where that particular destination runs.
Databricks also makes customers responsible for applicable provider terms. My view is that platform inventories should record the hosting arrangement alongside the model name. Otherwise, a seemingly simple model substitution can conceal a change in the environment receiving application data.
Create a Stable Endpoint for Each Workload
A custom model service translates and routes inference requests to one or more models, supporting real-time and batch inference. It can mix Databricks-served destinations with external models connected through a model provider.
Services are Unity Catalog securable objects, meaning access is governed through catalog privileges. Callers address them using a fully qualified name such as production.ai.support-assistant, including across workspaces or from outside Databricks. Databricks uses this workload-oriented naming pattern in its custom model services documentation.
That separation is useful: an application depends on a support-assistant service, while the platform team owns its model configuration. It reduces the number of application changes required for a model migration. It cannot guarantee that a replacement model produces compatible answers, tool calls, or formatting, so evaluation remains necessary.
Check Region Support and Permissions First
The documented prerequisites include a supported Databricks workspace region and Unity Catalog enabled. Unity Gateway is not supported on AWS GovCloud.
Creating a service requires:
USE CATALOG,USE SCHEMA, andCREATE SERVICEon the target catalog and schema.EXECUTEon each referenced model.EXECUTE,USE CATALOG, andUSE SCHEMAon referenced model providers.- Appropriate catalog and schema access plus
CREATE TABLEwhere an inference table will be created, if inference logging is enabled.
These are creation prerequisites. Teams must also grant intended callers access to the service.
A separate availability check is essential. Model assets in system.ai are listed globally, but availability depends on the workspace region, cross-geography settings, and the model itself. Databricks instructs users to check the corresponding service in the Unity Gateway UI and confirm query permissions. Merely listing a model does not send customer data to it for inference, according to the availability guidance.
Use the UI or Infrastructure Tooling
The documented UI path is straightforward:
- Open AI Gateway in the workspace sidebar and select Create.
- Enter the service name and choose its catalog and schema.
- Select an eligible Databricks-served model or model provider as the primary destination.
- Create the service, then configure features such as inference logging and additional routing.
Catalog Explorer offers another creation path. Programmatic options include REST APIs, Databricks SDKs, the CLI, Terraform, and Declarative Automation Bundles, with DAB support marked Beta in the supplied documentation.
A custom service is optional for basic access. System-provided services in system.ai let teams query models without creating their own endpoint. I would introduce a custom service when a workload needs a distinct owner, access boundary, routing policy, or spending scope.
Separate Permissions, Budgets, and Rate Limits
Unity Gateway brings several controls together, but they solve different problems.
Permissions determine who may call a service. Unity Catalog privileges can separate teams and application identities. Databricks also documents service policies that govern how requests and responses proceed based on their content and the caller’s identity, alongside PII masking and other guardrail capabilities.
Rate limits constrain consumption speed. The model-services documentation gives illustrative configurations of 100 requests per minute for testing and 1,000 requests per minute for production. Those are configuration examples, not published platform ceilings.
Budgets and spend limits govern financial exposure. Databricks documents per-user thresholds and hard caps. Its workload example tags a service with project=support-assistant and scopes a monthly budget to that tag, with an alert at $1,000 of spend. That dollar amount is an example alert threshold, not product pricing.
Usage and inference logs support investigation. The governance documentation describes tracking requests, tokens, and latency, attributing Databricks costs to services and principals, and optionally recording requests and responses in Unity Catalog Delta tables. See the governance overview for these separate control areas.
The operational implication is to design these settings together. A budget alert informs someone; a hard cap stops spending at its configured boundary. A rate limit controls request frequency but cannot by itself predict a monthly bill when token consumption varies.
I would also give inference-log tables deliberate permissions and retention rules. Recording prompts and responses creates another governed data asset, potentially containing the same sensitive material the application handles.
Traffic Splitting and Fallbacks Solve Different Problems
Unity Gateway supports weighted traffic splitting across up to five destinations. Percentages must total 100%. In the UI, teams add destinations under a service’s Routing section and assign each its share.
For example, a platform team could allocate 90% to an established destination and 10% to a candidate replacement. This is a rollout configuration, not evidence that the replacement is cheaper or better.
Databricks documents two routing stages:
Frequently Asked Questions
4 questions
1Does Unity Gateway Host Every Connected Model?
No. Some models run on Databricks infrastructure, while provider-hosted models and connected external APIs run in provider-operated environments.
