Azure Cosmos DB

Azure Cosmos DB is a fully managed, globally distributed NoSQL and vector database service run by Microsoft on its Azure cloud. It stores JSON documents and other data models, replicates them to any number of Azure regions, and lets developers choose among five consistency levels, from strong to eventual.^[1][8] Throughput is billed in an abstract unit called the request unit (RU), and data is spread across partitions by a customer-chosen partition key.^[9][10] The service began inside Microsoft in 2010 as "Project Florence", was previewed as Azure DocumentDB in 2014 and made generally available in 2015, and was relaunched under the Cosmos DB name at Microsoft Build in May 2017.^[2][5][6]

Microsoft now markets Cosmos DB as a "database for the AI era" and has cited its use by OpenAI for ChatGPT since 2023.^[1][32] In 2024 the service added built-in vector indexing based on Microsoft Research's DiskANN algorithms, along with full-text (BM25) and hybrid search.^[19][20] In September and October 2026 OpenAI published a two-part engineering series describing how its internal storage platform, Habitat, runs on Azure Cosmos DB; OpenAI reported that caching, replication and a rewrite of its storage service had cut global median read latency by about 12 times.^[34][35]

History

Cosmos DB started in late 2010 as Project Florence, a Microsoft effort to address problems faced by developers building internet-scale applications inside the company. Dharma Shukla, the Microsoft Technical Fellow who wrote the service's 2017 technical overview, said the project "eventually grew into Azure DocumentDB before expanding and blossoming into its current form".^[2] Shukla also credited the work of Turing Award winner Leslie Lamport as an influence on the system's design.^[2]

At the 2017 launch Microsoft described Cosmos DB as "the first and only globally distributed database service in the industry to offer financially-backed comprehensive SLAs", covering high availability, latency at the 99th percentile, consistency and throughput.^[6] The launch-era design goals included 99.99% availability regardless of the number of regions.^[2]

Architecture and data model

Cosmos DB's engine uses an internal type system that Microsoft calls atom-record-sequence (ARS): atoms are primitive types such as strings, booleans and numbers, records are structs, and sequences are arrays. Each supported API is projected onto this core model, so that a container appears as a collection to the MongoDB API, a table to the Table API and a graph to the Gremlin API.^[2] Microsoft describes the engine as write-optimized, resource-governed and schema-agnostic; it indexes every item it ingests automatically by default, and the index is made durable before a write is acknowledged.^[2] The 2025 vector-search paper describes the engine's index as stored in a Bw-Tree, a latch-free structure that supports fast random reads and writes in a tiered memory and SSD setup.^[21]

An account contains databases, which contain containers, which hold items. A single item can be up to 2 MB.^[39] Within a container, all items with the same partition key value form a logical partition, and a logical partition is also the scope of transactions, which run with snapshot isolation.^[10]

APIs

Cosmos DB exposes several APIs over the same underlying service. Apart from the native API for NoSQL, they are wire-protocol-compatible "compatibility APIs" for existing database ecosystems.^[1][8]

Microsoft's API pages for MongoDB, Cassandra, Gremlin and Table each now point customers who need the 99.999% SLA and instant autoscale toward the API for NoSQL.^{[26][27][28][29]}

Partitioning and request units

A request unit is the currency Cosmos DB uses to abstract CPU, IOPS and memory. Every operation, whether a write, a point read or a query, is charged in RUs; Microsoft's reference point is that reading a single item of about 1 KB by its ID and partition key costs one RU.^[9]

Containers scale out by partitioning. Each item's partition key determines its logical partition; there is no limit on the number of logical partitions, but each can hold at most 20 GB, and hierarchical partition keys of up to three levels can be used when one key value would exceed that.^[10] Logical partitions are mapped onto physical partitions that the service manages internally. Each physical partition can serve up to 10,000 RU/s and store up to 50 GB, and Cosmos DB splits physical partitions automatically as throughput or storage grows. Provisioned throughput is divided evenly across a container's physical partitions, so a partition key that concentrates traffic on a few values creates "hot" partitions and rate limiting.^[10] OpenAI's 2026 account of running Cosmos DB at very large scale describes exactly this effect, and adds that because splits halve each partition's share of throughput, storage growth alone can reduce the throughput available to active workloads.^[35]

Throughput modes

Microsoft also offers a lifetime free tier of 1,000 RU/s and 25 GB of storage.^[1]

Global distribution, consistency and availability

An Azure Cosmos DB account can be associated with any number of Azure regions, and regions can be added or removed without pausing or redeploying the application. With multi-region writes, every region accepts both reads and writes; Microsoft states that this configuration gives 99.999% read and write availability, and it advertises reads and writes served in under 10 milliseconds at the 99th percentile.^[13] In multi-region-write accounts, writes are committed in the local region and then sent to a hub region for asynchronous conflict resolution. Such accounts cannot use strong consistency.^[14]

Microsoft notes that most distributed NoSQL databases offer only strong and eventual consistency. Cosmos DB offers five levels, which Microsoft has specified formally in TLA+ and published in its azure-cosmos-tla repository.^[8][2]

Source: ^[8]

Zone redundancy spreads a region's replicas across availability zones and carries higher availability SLAs than non-zonal configurations.^[40]

Change feed

The change feed is a persistent, ordered record of changes to a container that applications can consume asynchronously and in parallel.^[15] It is commonly used for event-driven and streaming pipelines.^[1] OpenAI uses it to invalidate caches and to drive change data capture pipelines at large scale (see below).^[35]

Vector, full-text and hybrid search

The API for NoSQL can store embedding vectors as properties of ordinary documents and index them, so that vectors sit alongside the data they describe instead of in a separate vector database.^[16] A container vector policy declares each vector path, its data type (float32, float16, int8 or uint8), its dimensions (1,536 by default) and its distance function (cosine by default, dot product or Euclidean). Queries use the VectorDistance system function and can be combined with ordinary WHERE filters.^[16]

Source: ^[16]. The quantizedFlat and diskANN indexes require at least 1,000 vectors; below that, a full scan is used.^[16]

DiskANN is a suite of approximate nearest-neighbor search algorithms developed by Microsoft Research that uses disk-based storage to handle large datasets.^[16][20] Microsoft announced DiskANN-powered vector indexing in public preview at Build 2024 and made vector search and the DiskANN index generally available on 19 November 2024.^[19][20]

The engineering behind the integration is described in "Cost-Effective, Low Latency Vector Search with Azure Cosmos DB" (arXiv:2505.05885; PVLDB volume 18, 2025), by 36 authors led by Nitish Upreti and Harsha Vardhan Simhadri.^[21] The paper keeps a single vector index per partition inside Cosmos DB's existing Bw-Tree index, kept in sync with the data, and required rewriting the DiskANN library to separate its algorithms from the physical index layout. The authors report query latency under 20 ms over an index of 10 million vectors, stable recall over updates, scaling to billions of vectors through automatic partitioning, and roughly 43 times and 12 times lower query cost than the serverless enterprise products of Pinecone and Zilliz respectively. These comparisons are the authors' own.^[21]

Full-text search adds a full-text policy and index with stemming, stop-word removal and tokenization, and ranks results with BM25.^[17] Hybrid search combines vector similarity and full-text scores using Reciprocal Rank Fusion (RRF).^[18]

AI framework integrations

Microsoft positions Cosmos DB as a store for retrieval-augmented generation, AI agents and LLM caching, alongside embeddings produced by services such as Azure OpenAI.^[1][16]

• Semantic Kernel: Microsoft's SDK ships an Azure Cosmos DB NoSQL vector store connector, in preview as of October 2026.^[30]
• LangChain and LangGraph: in May 2026 Microsoft released langchain-azure-cosmosdb, a Python package that uses the API for NoSQL as a vector store, chat-history store, LangGraph checkpointer, semantic cache and long-term memory layer.^[31]
• Python SDK: version 4.14.0 (October 2025) added a semantic reranking API (private preview), a batched read_items call and automatic write retries.^[33]

Cosmos DB in Microsoft Fabric

Cosmos DB in Microsoft Fabric is a version of the database built into Microsoft's Fabric analytics platform. Microsoft says it uses the same engine and infrastructure as Azure Cosmos DB for NoSQL, includes vector, full-text and hybrid search, and automatically makes its data available in Fabric OneLake in Delta Parquet format for analytics in notebooks, lakehouses and Power BI.^[22] A demo was shown at Microsoft Build 2025, and the service and Cosmos DB mirroring into Fabric reached general availability at Ignite 2025.^[22][23]

Use at OpenAI and ChatGPT

Early statements (2023 to 2026)

In March 2023, a post on Microsoft's Cosmos DB blog opened with a quote attributed to Satya Nadella saying that OpenAI relies on Cosmos DB to "dynamically scale their ChatGPT service", which he called one of the fastest-growing consumer apps ever, "enabling high reliability and low maintenance". The same quote heads Microsoft's Cosmos DB product overview on Microsoft Learn.^[32][1] In October 2025 Microsoft said many features in its Python SDK 4.14.0 were developed "in close partnership with OpenAI, who rely heavily on Cosmos DB to store chat data for ChatGPT at massive scale".^[33] A May 2026 Microsoft post described Cosmos DB as "the database powering scenarios across OpenAI including ChatGPT histories and memories".^[31]

Habitat

OpenAI's own account came in a two-part engineering series, "Rapidly scaling online storage to serve over 1 billion ChatGPT users", by members of its technical staff Jon Lee, Chaomin Yu, Ben Ries and Hrachya Hakobyan.^[34][35] OpenAI's storage platform, Habitat, sits between products (ChatGPT, the API, Codex and internal services) and the underlying stores, handling routing, authorization, encryption, caching, rate limiting and data residency. Azure Cosmos DB is its underlying online storage component.^[34][35]

According to OpenAI, Habitat first launched to support GPTs at DevDay 2023 as a small Python client library that mapped operations onto Azure Cosmos DB. Before that, most of OpenAI's online data was stored on PostgreSQL, where single expensive queries on hot paths had caused outages.^[34] Habitat exposes a narrow NoSQL API of client-defined object and edge types, inspired by Meta's TAO; it does not support general graph traversals, and complex queries go instead to an offline secondary view in Rockset fed by change data capture.^[34] By mid-2025 the client library had reached its limits, and OpenAI moved Habitat into its own service. In the second quarter of 2026, OpenAI says, two engineers using Codex and GPT-5.5 rewrote the service from Python into Rust, which it reports is 6 times more CPU-efficient and 15 times more memory-efficient than the Python version.^[34]

OpenAI said that "to our knowledge, we are the only company which has deployed Azure Cosmos DB at this scale".^[35]

Multi-tenancy

Part II describes three phases. OpenAI first put all workloads in one default Cosmos DB container per account and relied on the documented 10,000 RU/s limit per logical partition key for back pressure. Because clients chose partition keys directly, unrelated product features often shared keys such as user_id, and in practice the limit was enforced per physical partition, so noisy workloads could disrupt core login and signup flows.^[35] OpenAI then moved critical workloads into separate single-tenant containers, which limited outages to individual features but pushed capacity planning onto every product team, and teams that forgot to pre-scale their containers caused launch-day incidents.^[35]

In the third phase OpenAI returned to a shared container, after working with the Azure team on hot-key throttling, request-priority quality of service (so offline bulk work is throttled before user-facing requests) and a new serverless mode built for OpenAI. Standard serverless capped each physical partition at 5,000 RU/s and split only on storage; the new mode lets each physical partition reach 10,000 RU/s and adds splits triggered by RU usage. OpenAI says it has had no provisioning or noisy-neighbor incidents in the months since.^[35][11]

Outages and co-engineering

OpenAI disclosed two incidents. In late 2024 a power outage hit the primary Azure region of one of its largest Cosmos DB accounts, and the failover showed that the largest accounts would take several hours to move to a secondary region, something neither team had tested at that scale. About a month later, failing back triggered another serious outage: a quirk in the gateway-mode Cosmos DB client sent all clients' connections to the original region at once and overwhelmed its gateway.^[35] Since then, OpenAI says, the two companies jointly monitor every layer down to Cosmos DB's physical storage, test changes on a production-sized non-production account, iterate on SDKs together, and can page each other directly. Joint work included load protection and metadata caching in the Cosmos DB gateway.^[35]

Latency and caching

For global performance OpenAI uses three tactics: storing data near users where capacity allows (done mainly for users with data-residency requirements, because placement is constrained by datacenter capacity), heavily replicating only small latency-critical datasets such as authentication metadata (some accounts replicate to all 40 Azure regions where OpenAI serves traffic), and a lookaside cache on Valkey servers near compute. Cache invalidations are driven by Cosmos DB's change feed and propagate globally within seconds. Combined with the Rust rewrite, these changes cut global p50 read latency from about 120 ms to about 10 ms.^[35] OpenAI's change data capture system processes millions of changes per second from thousands of containers into more than a hundred Kafka topics, and OpenAI modified Azure's CDC SDK to batch checkpoints in memory.^[35]

Plans beyond Cosmos DB

OpenAI wrote that, to scale Habitat and conversation storage "for the next billion users", it is building a custom in-house exabyte-scale online storage system for bulky Habitat data. The system combines the RocksDB Cloud core from Rockset with Raft-based replication and implements the same document model as Azure Cosmos DB, using disaggregated storage to reduce flash capacity in exchange for lower peak QPS. As of 27 October 2026 it was described as in development.^[35]

Security: ChaosDB (2021)

In August 2021 researchers Nir Ohfeld and Sagi Tzadik of the cloud security firm Wiz disclosed "ChaosDB", a flaw in Cosmos DB's Jupyter Notebook feature. Wiz said the feature, added in 2019, had been turned on automatically for all Cosmos DB accounts in February 2021, and that misconfigurations in the notebook container allowed privilege escalation into other customers' notebooks and theft of their accounts' primary keys, which grant full read, write and delete access.^[37] Wiz said Microsoft disabled the feature within 48 hours of the report.^[37]

Microsoft's Security Response Center said the issue was reported on 12 August 2021 and mitigated immediately, that its investigation found no customer data was accessed by third parties or researchers, and that only the primary read-write key was exposed. It notified the customers it identified, those actively using the notebook feature or who created a Cosmos DB account between 7 and 13 August 2021, and advised them to regenerate their primary keys.^[38] Wiz argued that more customers than those notified might be at risk, saying the flaw had been exploitable for at least several months.^[37]

Azure DocumentDB naming

The DocumentDB name has been used for two different Microsoft products. The original Azure DocumentDB (2014 to 2017) became Azure Cosmos DB.^[3][6] In November 2025 Microsoft gave the name Azure DocumentDB to a separate service, formerly the vCore-based Azure Cosmos DB for MongoDB, which runs the open-source, MIT-licensed DocumentDB engine (built on PostgreSQL and governed by the Linux Foundation) with MongoDB compatibility and vCore-based pricing.^[24][1] Microsoft recommends Cosmos DB for NoSQL for scale-out, multi-region workloads and Azure DocumentDB for MongoDB migrations and complex aggregation queries.^[1]

See also

• Vector database
• HNSW
• Embeddings
• Semantic search
• Cloud computing
• Knowledge graph

References

1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9Microsoft Learn. "Azure Cosmos DB - Database for the AI Era" (overview). Last updated 8 June 2026. learn.microsoft.com/...overview
2. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8Dharma Shukla. "A technical overview of Azure Cosmos DB." Microsoft Azure Blog, 10 May 2017. azure.microsoft.com/...overview-of-azure-cosmos-db
3. ^1 ^2Microsoft Azure Blog. "New Azure services and updates expand openness, choice and flexibility." 21 August 2014. azure.microsoft.com/...ness-choice-and-flexibility
4. ^Microsoft Azure Blog. "DocumentDB moving to general availability." 5 March 2015. azure.microsoft.com/...ing-to-general-availability
5. ^1 ^2Microsoft Azure Blog. "NoSQL database service Azure DocumentDB now Generally Available." 8 April 2015. azure.microsoft.com/...tdb-now-generally-available
6. ^1 ^2 ^3 ^4Microsoft Azure Blog. "Azure Cosmos DB: The industry's first globally-distributed, multi-model database service." 10 May 2017. azure.microsoft.com/...ulti-model-database-service
7. ^Microsoft Azure Blog. "New innovations at Microsoft Build 2017: Helping developers achieve more." 10 May 2017. azure.microsoft.com/...ing-developers-achieve-more
8. ^1 ^2 ^3 ^4Microsoft Learn. "Consistency levels in Azure Cosmos DB." learn.microsoft.com/...consistency-levels
9. ^1 ^2 ^3Microsoft Learn. "Request Units in Azure Cosmos DB." learn.microsoft.com/...request-units
10. ^1 ^2 ^3 ^4Microsoft Learn. "Partitioning and horizontal scaling in Azure Cosmos DB." learn.microsoft.com/...partitioning
11. ^1 ^2Microsoft Learn. "Azure Cosmos DB serverless account type." learn.microsoft.com/...serverless
12. ^Microsoft Learn. "Create Azure Cosmos DB containers and databases with autoscale throughput." learn.microsoft.com/...vision-throughput-autoscale
13. ^Microsoft Learn. "Distribute your data globally with Azure Cosmos DB." learn.microsoft.com/...distribute-data-globally
14. ^Microsoft Learn. "Multi-region writes in Azure Cosmos DB." learn.microsoft.com/...multi-region-writes
15. ^Microsoft Learn. "Change feed in Azure Cosmos DB." learn.microsoft.com/...change-feed
16. ^1 ^2 ^3 ^4 ^5 ^6 ^7Microsoft Learn. "Vector search in Azure Cosmos DB for NoSQL." learn.microsoft.com/...vector-search
17. ^Microsoft Learn. "Full-text search in Azure Cosmos DB for NoSQL." learn.microsoft.com/...full-text-search
18. ^Microsoft Learn. "Hybrid search in Azure Cosmos DB for NoSQL." learn.microsoft.com/...hybrid-search
19. ^1 ^2 ^3Azure Cosmos DB Blog. "Announced at Build 2024: DiskANN-powered vector database, cross-region replication in vCore, and a new era of databases." 21 May 2024. devblogs.microsoft.com/...build2024-cosmos-roundup
20. ^1 ^2 ^3 ^4Azure Cosmos DB Blog. "New Vector Search, Full Text Search, and Hybrid Search Features in Azure Cosmos DB for NoSQL." 19 November 2024. devblogs.microsoft.com/...zure-cosmos-db-for-nosql
21. ^1 ^2 ^3 ^4Nitish Upreti, Harsha Vardhan Simhadri, et al. "Cost-Effective, Low Latency Vector Search with Azure Cosmos DB." arXiv:2505.05885 (v1 9 May 2025, v2 31 July 2025); PVLDB 18(12): 5166-5183, 2025. arxiv.org/...2505.05885
22. ^1 ^2Microsoft Learn. "What is Cosmos DB in Microsoft Fabric?" learn.microsoft.com/...overview
23. ^1 ^2Azure Cosmos DB Blog. "Announced at Ignite 2025: Azure DocumentDB, MCP Toolkit, Fleet Analytics, and more!" 19 November 2025. devblogs.microsoft.com/...fleet-analytics-and-more
24. ^1 ^2Azure Cosmos DB Blog. "Azure DocumentDB is Now Generally Available." 18 November 2025. devblogs.microsoft.com/...-now-generally-available
25. ^Microsoft Learn. "What is Azure Cosmos DB for PostgreSQL?" learn.microsoft.com/...introduction
26. ^1 ^2Microsoft Learn. "What is Azure Cosmos DB for MongoDB?" learn.microsoft.com/...overview
27. ^1 ^2Microsoft Learn. "What is Azure Cosmos DB for Apache Cassandra?" learn.microsoft.com/...overview
28. ^1 ^2Microsoft Learn. "What is Azure Cosmos DB for Apache Gremlin?" learn.microsoft.com/...overview
29. ^1 ^2Microsoft Learn. "What is Azure Cosmos DB for Table?" learn.microsoft.com/...overview
30. ^Microsoft Learn. "Using the Azure CosmosDB NoSQL Vector Store connector (Preview)." Semantic Kernel documentation. learn.microsoft.com/...re-cosmosdb-nosql-connector
31. ^1 ^2James Codella, Aayush Kataria. "Introducing langchain-azure-cosmosdb: Build Agentic Apps and RAG with One Database." Azure Cosmos DB Blog, 1 May 2026. devblogs.microsoft.com/...ure-cosmos-db-agents-rag
32. ^1 ^2 ^3Mark Brown, Sandeep S. Nair. "Build Intelligent Applications using ChatGPT & Azure Cosmos DB." Azure Cosmos DB Blog, 22 March 2023. devblogs.microsoft.com/...chatgpt-azure-cosmos-db
33. ^1 ^2 ^3Theo van Kraay. "Announcing latest Azure Cosmos DB Python SDK: Powering the Future of AI with OpenAI." Azure Cosmos DB Blog, 20 October 2025. devblogs.microsoft.com/...future-of-ai-with-openai
34. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11Jon Lee, Chaomin Yu, Ben Ries. "Rapidly scaling online storage to serve over 1 billion ChatGPT users." OpenAI, 11 September 2026. openai.com/...g-storage-one-billion-users-part-one
35. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19Jon Lee, Hrachya Hakobyan, Chaomin Yu. "Rapidly scaling online storage to serve over 1 billion ChatGPT users (Part II)." OpenAI, 23 October 2026. openai.com/...g-storage-one-billion-users-part-two
36. ^OpenAI Developers (@OpenAIDevs). Post on X, 23 October 2026. x.com/...2113682848607780912
37. ^1 ^2 ^3Nir Ohfeld, Sagi Tzadik. "ChaosDB: How we hacked thousands of Azure customers' databases." Wiz Blog, 26 August 2021. wiz.io/...d-thousands-of-azure-customers-databases
38. ^Microsoft Security Response Center. "Update on the vulnerability in the Azure Cosmos DB Jupyter Notebook Feature." 27 August 2021. microsoft.com/...osmos-db-jupyter-notebook-feature
39. ^Microsoft Learn. "Azure Cosmos DB service quotas." learn.microsoft.com/...concepts-limits
40. ^Microsoft Learn. "Reliability in Azure Cosmos DB for NoSQL." learn.microsoft.com/...reliability-cosmos-db