OpenAI says Habitat, the storage platform behind ChatGPT and its other products, serves more than 1 billion users across 40 geographic regions and manages over 500 petabytes of data. Its October 23, 2026 engineering post describes the decisions behind that scale: returning to shared database containers, co-managing Azure Cosmos DB with Microsoft, and combining regional replication, caching and a Rust proxy rewrite.
The reported performance result is substantial. OpenAI says global median read latency fell from roughly 120 milliseconds to 10 milliseconds, a 12× improvement. These are OpenAI’s production figures, without independent validation in the published account. They describe storage reads, not end-to-end ChatGPT response time. OpenAI’s engineering post
The reusable lesson is the sequence of changes. Habitat’s team first needed reliable isolation, then faster global reads, then an operating model that could handle exceptional scale. Infrastructure teams can apply that sequence well before they reach hundreds of petabytes.
Why Habitat Returned to Multi-Tenancy
Habitat moved through three configurations: one shared Cosmos DB container, dedicated single-tenant backing containers, and finally shared containers by default again.
Each configuration solved one problem while exposing another.
Dedicated containers gave product teams clearer ownership and stronger workload separation. A demanding feature had fewer opportunities to interfere with unrelated traffic. But each team also inherited capacity planning, and separately provisioned workloads could leave substantial capacity unused.
Shared containers moved more responsibility to the central database team, which could provision resources across workloads. That improved the opportunity for efficient utilization, while reintroducing noisy neighbors: one tenant consuming enough resources to degrade service for others.
OpenAI describes this as a “circular journey,” and says it still relies heavily on Cosmos DB’s built-in capabilities. OpenAI
I’d read that reversal as a useful warning against treating either tenancy model as an architectural destination. Sharing becomes attractive when isolation controls are strong enough. Dedicated capacity remains a reasonable response when those controls cannot protect critical workloads.
Logical Keys Do Not Own Physical Capacity
Cosmos DB’s storage hierarchy explains why Habitat’s original shared design struggled.
An account contains containers and defines a replication policy. Within a container, logical partition keys determine how application data is grouped. Cosmos DB places those logical partitions onto backing physical partitions.
Throughput is measured in request units per second, or RU/s. Request units represent database resource consumption; they are not interchangeable with queries per second, because operations can have different costs.
| Layer | Role in Habitat’s account | Practical consequence |
|---|---|---|
| Account | Groups containers and defines replication policy | Replication decisions affect a broader storage boundary |
| Container | Holds partitioned data and throughput configuration | Sharing a container pools workloads |
| Logical partition key | Groups related application data | A hot key can become a throughput bottleneck |
| Physical partition | Hosts multiple logical partitions and enforces capacity limits | Different keys can still compete for the same resources |
OpenAI initially relied on the stated 10,000 RU/s limit per logical partition key to provide backpressure. But multiple logical keys shared physical capacity. The reproduced account explains that this allowed niche feature traffic to disrupt core login and signup flows. OpenAI · Reproduced engineering account
The distinction matters: a logical key’s upper limit is not a reservation. As an illustrative capacity check, two keys each demanding 6,000 RU/s would require 12,000 RU/s in aggregate. If they share a 10,000 RU/s physical resource, both requests cannot be satisfied simultaneously, despite each key remaining below its individual ceiling.
Habitat also exposed partition-key selection directly to clients. Common identifiers such as user_id and workspace_id collided across product use cases, making separate features contend under the same logical-key limits.
For a new storage API, I’d review key namespacing before optimizing partition counts. Product boundaries that look separate in application code can disappear inside a shared database keyspace.
Make Sharing Safe Before Consolidating Tenants
OpenAI’s return to multi-tenancy depended on three capabilities: hot-key throttling, request-priority quality of service, and an improved serverless mode with usage-based partition splitting. The published summary says each physical partition in that serverless setup can scale to 10,000 RU/s and splits based on RU usage. OpenAI · How They Scaled
These mechanisms address different failure modes.
Hot-key throttling contains concentrated demand. A heavily used organization, workspace or record can create disproportionate pressure. Limiting that traffic helps protect workloads sharing its underlying resources.
Request-priority QoS controls who gets served under contention. If background work and an interactive operation compete for constrained capacity, priorities provide a way to preserve the more important request path. The useful implementation question is which operations can tolerate delay and how that priority travels through the storage layer.
Usage-based splitting responds to compute pressure. Storage volume alone is an incomplete signal. A relatively small dataset can be extremely busy, so partition growth needs to account for demand as well as bytes.
These controls complement each other. Additional partitions can provide more aggregate capacity, but a concentrated hot key still needs protection. Priorities can protect critical work, while throttling prevents one workload from consuming an excessive share.
For teams considering consolidation, the resulting design review is concrete:
- Identify hot keys independently of total tenant traffic.
- Assign priorities according to product consequences.
- Measure contention at the physical resource boundary.
- Check how quickly capacity responds to sustained demand.
- Test whether a low-priority workload can disrupt a critical operation.
That checklist follows directly from Habitat’s failure mode: application-level separation did not guarantee resource isolation.
A Managed Database Still Needs Shared Operations
OpenAI says its largest Cosmos DB account stores tens of petabytes and serves more than 20 million queries per second. That account-level figure is separate from Habitat’s platform-wide total of more than 500 petabytes. The billion-user figure also should not be read as a concurrent-user count. OpenAI
At this scale, the database supplier becomes part of the operational design. The published account describes joint ownership with Azure; the accompanying summary traces that arrangement to failover and failback outages in late 2024, followed by shared testing and monitoring. How They Scaled
Failover moves service to an alternate serving path. Failback returns it after recovery. Both can expose capacity assumptions that ordinary operation never exercises.
The reusable practice is to make those transitions joint engineering work. A managed-service agreement can define responsibilities, but a recovery exercise reveals whether the application, database and operators behave coherently.
For a smaller team, the practical equivalent is a shared recovery plan covering:
- The signals that trigger intervention.
- The capacity available on the alternate path.
- Application behavior during retries and recovery.
- The conditions required before returning traffic.
- Who can authorize and execute each step.
My view is that this operational boundary deserves as much design attention as the database schema. A storage service can meet normal demand and still struggle when recovery changes where requests land.
Fix the Global Read Path as a System
OpenAI combined three changes to improve reads: selective regional replication, distributed caching with change-data-capture invalidation, and a Rust rewrite of the proxy path.
The reported reduction from approximately 120 ms to 10 ms is a 12× improvement in median latency, or about a 92% reduction in elapsed time. The evidence presents a combined result; it does not isolate the contribution of each change or provide a tail-latency result. OpenAI
Replicate According to Workload
Habitat replicated small, latency-critical datasets widely, according to the published summary.
Frequently Asked Questions
4 questions
1What Is OpenAI’s Habitat?
Habitat is OpenAI’s online storage platform for ChatGPT, the API, Codex and other services. OpenAI reports that it manages more than 500 petabytes across a platform serving over a billion users.
