Databricks
Last reviewed
Sources
22 citations
Review status
Source-backed
Revision
v8 ยท 4,965 words
Improve this article
Add missing citations, update stale details, or suggest a clearer explanation.
Suggest edit
CtrlK
Last reviewed
Sources
22 citations
Review status
Source-backed
Revision
v8 ยท 4,965 words
Add missing citations, update stale details, or suggest a clearer explanation.
| Databricks | |
|---|---|
| Type | Private company |
| Industry | Data, artificial intelligence |
| Founded | 2013 |
| Founders | Ali Ghodsi, Matei Zaharia, Ion Stoica, Andy Konwinski, Patrick Wendell, Reynold Xin, Arsalan Tavakoli-Shiraji |
| Headquarters | San Francisco, California, US |
| Key people | Ali Ghodsi (CEO), Matei Zaharia (CTO) |
| Products | Data Intelligence Platform (lakehouse), Mosaic AI, Unity Catalog, Delta Lake, MLflow, Lakebase, DBRX |
| Funding | Series L: ~$5B equity plus ~$2B debt at $134B valuation (closed February 2026) [4] |
| Revenue run-rate | $5.4 billion, growing >65% YoY (Q4, reported February 9, 2026) [17] |
| Website | databricks.com |
Databricks is an American enterprise software company that provides a unified data analytics and artificial intelligence platform built around the data lakehouse architecture. Founded in 2013 by the creators of Apache Spark, Databricks was valued at $134 billion in its February 2026 Series L financing, making it one of the most valuable private technology companies in the world, and it crossed a $5.4 billion revenue run-rate growing more than 65% year over year in the quarter reported on February 9, 2026 [4][17]. The company's Data Intelligence Platform combines data engineering, data warehousing, and machine learning into a single environment, and its expansion into AI, through the $1.3 billion acquisition of MosaicML and the release of the open-source DBRX model, has positioned it as a major player in the enterprise AI market [18]. More than 20,000 organizations worldwide, including over 60% of the Fortune 500, run on the platform, and the company is widely expected to be a candidate for one of the largest technology IPOs in history [17].
Databricks grew directly out of the Apache Spark open-source project and the data lakehouse concept its founders defined in a 2021 research paper. Ali Ghodsi serves as CEO and Matei Zaharia, who created Apache Spark during his PhD at UC Berkeley, serves as Chief Technologist.
Databricks is a data and AI company whose flagship product, the Data Intelligence Platform, lets organizations store, govern, analyze, and build AI applications on all of their data in one place using the open data lakehouse architecture. The lakehouse unifies the low-cost, flexible storage of a data lake with the reliability, performance, and governance of a data warehouse, so that data engineering, business intelligence, and machine learning workloads can run against a single copy of the data rather than across separate, siloed systems. The platform spans Delta Lake (open storage), Unity Catalog (governance), Mosaic AI (model training, serving, and agents), Lakebase (a transactional database), and Genie (natural language analytics).
Databricks was founded in 2013 by seven co-founders, all of whom were connected through the AMPLab (Algorithms, Machines, and People Lab) at the University of California, Berkeley:
| Co-founder | Role/Background |
|---|---|
| Ali Ghodsi | CEO; PhD from KTH Royal Institute of Technology; UC Berkeley researcher |
| Matei Zaharia | Chief Technologist; created Apache Spark during his PhD at UC Berkeley |
| Andy Konwinski | Co-creator of Apache Mesos |
| Arsalan Tavakoli-Shiraji | Former VP of Engineering |
| Ion Stoica | UC Berkeley professor; co-founder of Conviva |
| Patrick Wendell | Apache Spark release manager |
| Reynold Xin | Apache Spark contributor |
The company grew directly out of the Apache Spark project, which Matei Zaharia created during his doctoral research at Berkeley. Spark was designed as a fast, general-purpose cluster computing system that could handle both batch and streaming data processing. It quickly became one of the most popular open-source data processing frameworks in the world, and Databricks was founded to build a commercial platform and managed service around it.
From the beginning, Databricks embraced an open-source-first strategy, contributing heavily to Apache Spark and later creating additional open-source projects including Delta Lake (a storage layer for data lakes), MLflow (a platform for the machine learning lifecycle), and Apache Spark Structured Streaming.
Databricks is most closely associated with the data lakehouse concept, which it helped popularize. The data lakehouse combines the flexibility and low cost of a data lake with the data management and performance features of a traditional data warehouse.
The term and the architecture were formalized by Databricks co-founders Matei Zaharia, Ali Ghodsi, and Reynold Xin, together with Michael Armbrust, in the paper "Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics," presented at the 11th Conference on Innovative Data Systems Research (CIDR) in January 2021 [19]. The paper argued that the lakehouse would (i) be based on open direct-access data formats such as Apache Parquet, (ii) have first-class support for machine learning and data science, and (iii) offer state-of-the-art performance. The authors predicted that "the data warehouse architecture as we know it today will wither in the coming years and be replaced by a new architectural pattern, the Lakehouse," addressing data warehouse problems such as staleness, reliability, total cost of ownership, and data lock-in [19].
Historically, organizations maintained separate systems for different data workloads:
| System | Strengths | Weaknesses |
|---|---|---|
| Data warehouse | Structured queries, ACID transactions, governance | Expensive, limited to structured data |
| Data lake | Cheap storage, supports all data types | Poor performance, no transactions, "data swamp" risk |
The lakehouse architecture merges these approaches by adding warehouse-like features (ACID transactions, schema enforcement, indexing) directly on top of data lake storage (typically cloud object storage like Amazon S3 or Azure Blob Storage). Databricks implemented this through Delta Lake, an open-source storage layer that brings reliability and performance to data lakes.
Delta Lake is the foundation of Databricks' lakehouse architecture. It provides:
Delta Lake uses the open Parquet file format underneath, which means data stored in Delta Lake can be read by any tool that supports Parquet, avoiding vendor lock-in.
Delta Lake extends the Parquet format with a file-based transaction log that records every change to the data. This log enables several critical capabilities [11]:
| Feature | Description | Benefit |
|---|---|---|
| ACID transactions | Serializable isolation level via optimistic concurrency control | Multiple concurrent writers and readers without corruption |
| Schema enforcement | Validates data against the table schema on write | Prevents silent data quality degradation |
| Schema evolution | Supports adding, renaming, and dropping columns | Adapts to changing data requirements without downtime |
| Time travel | Query data at any point in its history using version numbers or timestamps | Audit trails, reproducibility, rollback capability |
| Data skipping | Maintains statistics (min, max, count) for each file | Queries skip irrelevant files, reducing I/O |
| Z-ordering | Co-locates related data within files based on specified columns | Dramatically improves query performance for filtered reads |
| Change data feed | Tracks row-level changes (inserts, updates, deletes) between versions | Efficient incremental processing |
Delta Lake is fully compatible with Apache Spark APIs and was developed for tight integration with Structured Streaming, allowing a single copy of data to serve both batch and streaming use cases. As of 2025, Delta Lake also supports interoperability with Apache Iceberg, allowing data stored in Delta format to be read by Iceberg-compatible tools and vice versa [11].
In 2025, Databricks unveiled Lakebase, a Postgres-compatible transactional database engine built for the lakehouse, allowing teams to run OLTP-style applications directly on the same data infrastructure used for analytics and AI workloads.
Lakebase entered public preview at the 2025 Data + AI Summit and reached general availability on February 3, 2026. It represents Databricks' entry into the online transaction processing (OLTP) market, traditionally the domain of dedicated database systems like PostgreSQL, MySQL, and cloud-native databases [12]. Databricks describes Lakebase as "a serverless Postgres database built for AI agents," and CEO Ali Ghodsi said in February 2026 that the company would "double down on Lakebase so developers can create operational databases built for AI agents" with the proceeds of its Series L round [17].
Key Lakebase features include:
| Feature | Description |
|---|---|
| Postgres compatibility | Standard PostgreSQL wire protocol and SQL dialect |
| Serverless compute | Auto-scaling with scale-to-zero capability |
| Instant branching | Create database branches for development, testing, or experimentation |
| Point-in-time restore | Recover data to any previous point in time |
| Delta table sync | Managed synchronization between OLTP tables and Delta Lake analytics tables |
| Unity Catalog integration | Governance and access control through the same catalog as all other lakehouse assets |
| Postgres extension support | Compatible with PostgreSQL extensions for specialized functionality |
Lakebase bridges the gap between operational applications and analytical workloads. Instead of maintaining separate OLTP and analytics databases with complex ETL pipelines between them, organizations can use Lakebase for transactional workloads while the built-in sync keeps Delta Lake tables updated for analytics and AI [12].
Databricks announced on June 26, 2023, that it would acquire MosaicML for approximately $1.3 billion, inclusive of retention packages, marking its most significant move into the generative AI space; the deal closed on July 19, 2023, bringing MosaicML's roughly 62 employees and its MPT family of open large language models under Databricks [18]. MosaicML had built tools and infrastructure that simplified and reduced the cost of training large language models, making it possible for enterprises to train custom models without the massive engineering teams that organizations like OpenAI or Google maintained.
Following the acquisition, MosaicML's technology was integrated into Databricks as Mosaic AI, which covers the full machine learning lifecycle from feature engineering and model training to deployment and monitoring. The acquisition brought key talent, including MosaicML's expertise in efficient training techniques, and gave Databricks the capability to offer foundation model training as a service to its enterprise customers.
The Mosaic AI Agent Framework is Databricks' solution for building production-quality AI agent systems, including retrieval-augmented generation (RAG) applications. The framework provides a suite of tooling for developing, evaluating, and deploying compound AI systems that leverage multiple components such as tuned models, retrieval, tool use, and reasoning agents [13].
Key capabilities include:
| Component | Function |
|---|---|
| Agent Bricks | Auto-optimized agent templates for common industry use cases (information extraction, knowledge assistance, text transformation) |
| Agent evaluation | Built-in tools for measuring agent accuracy, safety, and performance |
| Vector Search | Storage-optimized vector search supporting billions of vectors at 7x lower cost |
| Mosaic AI Gateway | Unified entry point for all AI services with centralized governance, usage logging, and control |
| Multi-agent systems | Support for building systems where multiple specialized agents collaborate |
Agent Bricks, announced at the 2025 Data + AI Summit, simplifies agent development by allowing users to provide a high-level description of the agent's task and connect enterprise data, with the system handling optimization automatically [13].
Databricks' "2026 State of AI Agents" report, published in January 2026, found that usage of multi-agent workflows on the Databricks platform grew 327% in less than four months, from June to October 2025, as enterprises moved from single chatbots to systems of collaborating agents [20]. The report drew on telemetry from the more than 20,000 organizations on the Data Intelligence Platform, including over 60% of the Fortune 500. It also reported that companies using evaluation tooling get nearly 6x more AI projects into production, and that those using AI governance get over 12x more, underscoring evaluation and governance as the main bottlenecks to scaling agents [20].
In March 2024, Databricks released DBRX, its first foundation model, under the Databricks Open Model License. DBRX uses a fine-grained mixture-of-experts (MoE) architecture built on the MegaBlocks open-source project. Verified details of DBRX include [21]:
| Specification | Details |
|---|---|
| Release date | March 27, 2024 |
| Architecture | Fine-grained mixture-of-experts (MoE) |
| Total parameters | 132 billion |
| Active parameters per input | 36 billion |
| Experts | 16 experts, 4 chosen per input (vs. 8 choose 2 in Mixtral and Grok-1) |
| Training data | 12 trillion tokens of text and code |
| Training hardware | 3,072 NVIDIA H100 GPUs over ~2.5 months |
| Training cost | ~$10 million |
| License | Databricks Open Model License |
| Foundation | MegaBlocks open-source project |
| Serving | Available via pay-per-token and provisioned throughput endpoints |
DBRX's fine-grained MoE design gives 65x more possible combinations of experts than the 8-choose-2 designs used by Mixtral and Grok-1, which Databricks said improved model quality [21]. At release, Databricks reported that DBRX outperformed Meta's Llama 2-70B, Mistral AI's Mixtral, and xAI's Grok-1 on several language understanding, programming, and mathematics benchmarks, with inference up to 2x faster than Llama 2-70B [21]. DBRX was designed to demonstrate that enterprises could build competitive foundation models at a fraction of the cost of frontier models from major AI labs, reinforcing Databricks' pitch that companies should own and customize their AI models rather than relying entirely on third-party APIs.
In March 2023, shortly before the MosaicML acquisition, Databricks released Dolly, an open-source language model named after Dolly the sheep (the first cloned mammal). Dolly was a 6 billion parameter model based on EleutherAI's GPT-J, fine-tuned on a dataset of instruction-following examples generated by Databricks employees. Dolly was notable as one of the first demonstrations that a relatively small model could exhibit instruction-following capabilities similar to much larger models when fine-tuned on high-quality data.
Databricks later released Dolly 2.0, which used a commercially permissive training dataset created by Databricks employees, making it one of the first instruction-following LLMs that could be used for commercial purposes without licensing restrictions.
MLflow is an open-source platform for managing the end-to-end machine learning lifecycle. Originally created by Databricks and later donated to the Linux Foundation, MLflow provides:
| Component | Function |
|---|---|
| MLflow Tracking | Logging experiments, parameters, metrics, and artifacts |
| MLflow Projects | Packaging ML code for reproducible runs |
| MLflow Models | Deploying models in diverse serving environments |
| MLflow Model Registry | Centralized model store with versioning and staging |
MLflow has become one of the most widely adopted ML lifecycle management tools in the industry, with integration support across major cloud platforms and ML frameworks. Its open-source nature and broad compatibility have helped Databricks build mindshare in the data science community.
MLflow 3, released in 2025, introduces significant enhancements to experiment tracking, observability, and performance evaluation for both traditional ML models and generative AI applications [14].
Key new concepts in MLflow 3 include:
| Feature | Description |
|---|---|
| Logged Models | Persistent model objects that track a model's progress throughout its lifecycle, across environments and runs |
| Deployment Jobs | First-class tracking of model deployment status and configuration |
| Enhanced Model Registry | Direct capture of parameters, metrics, and metadata available across all workspaces |
| GenAI observability | Tracing and evaluation capabilities designed for LLM-powered applications |
| Agent evaluation | Purpose-built metrics and evaluation frameworks for AI agents |
MLflow 3's integration with Unity Catalog means that models tracked in MLflow automatically benefit from centralized governance, lineage tracking, and access control across the Databricks platform [14].
Databricks Model Serving provides real-time and batch inference capabilities integrated with the lakehouse platform. It supports serving custom models trained on Databricks, foundation models accessed via APIs, and external models from providers like OpenAI and Anthropic. Model Serving integrates with MLflow for model versioning and lifecycle management, and with Unity Catalog for governance and access control.
The platform also includes Mosaic AI Model Serving for batch inferencing, which simplifies the infrastructure needed to process unstructured data at scale using large language models.
Mosaic AI Model Serving deploys models to REST API endpoints with automatic monitoring of requests and responses. The serving infrastructure supports several deployment patterns [14]:
| Deployment Type | Description | Billing |
|---|---|---|
| Pay-per-token endpoints | Serverless endpoints billed by tokens processed | Per-token pricing |
| Provisioned throughput | Dedicated compute with guaranteed capacity | Per-compute-hour |
| Custom model endpoints | Serve models trained on Databricks | Per-compute-hour |
| External model endpoints | Proxy to external providers (OpenAI, Anthropic) with governance | Per-token (pass-through + gateway fee) |
All served models are automatically registered in Unity Catalog, ensuring consistent governance and access control regardless of deployment type.
Genie (officially AI/BI Genie) is Databricks' natural language interface for data analysis. Generally available as of 2025, Genie allows business users to query data, build visualizations, and receive AI-generated insights using conversational language, without writing SQL or code. Genie represents Databricks' push to make its platform accessible to non-technical users and to demonstrate the practical value of AI integration in everyday business analytics. Databricks described Genie in February 2026 as a "conversational AI assistant that lets any employee chat with their data," and named it, alongside Lakebase, as a primary focus for the capital raised in its Series L round [17].
An API in public preview also enables developers to integrate Genie into custom-built applications and productivity tools.
Unity Catalog is Databricks' unified governance solution for data and AI assets. It provides a single place to manage access controls, auditing, lineage, and discovery across all data, ML models, notebooks, and dashboards within a Databricks workspace. Key features include:
Unity Catalog addresses a critical enterprise need: as organizations deploy more AI models and manage more data, they require robust governance to ensure compliance with regulations, protect sensitive data, and maintain data quality.
Unity Catalog organizes assets in a three-level namespace: catalog, schema, and object. This hierarchy maps naturally to organizational structures and allows fine-grained access control [15].
| Level | Description | Example |
|---|---|---|
| Catalog | Top-level container, typically representing a business unit or environment | production, development, marketing |
| Schema | Groups related objects within a catalog | production.sales, production.finance |
| Object | Individual data or AI asset | Tables, views, volumes, models, functions |
Unity Catalog governs the following asset types:
| Asset Type | Description |
|---|---|
| Managed tables | Delta Lake tables with storage managed by Unity Catalog |
| External tables | Tables pointing to data in customer-managed storage |
| Views | Virtual tables defined by SQL queries |
| Volumes | Managed and external file storage (images, documents, raw data) |
| ML models | Models registered through MLflow Model Registry |
| Functions | User-defined functions (UDFs) and AI functions |
| Connections | Metadata for external database connections (federation) |
Introduced in 2025, Unity Catalog Metrics extends governance to business metrics definitions, ensuring that key performance indicators (KPIs) are defined once and used consistently across dashboards, reports, and AI applications. This prevents the common problem of different teams calculating the same metric in different ways [15].
Databricks open-sourced Unity Catalog in 2024, allowing organizations to use its governance capabilities outside the Databricks platform. The open-source version supports Apache Iceberg, Delta Lake, and other table formats, reinforcing Databricks' strategy of building ecosystem adoption through open-source contributions.
Databricks has raised substantial funding across numerous rounds, reflecting its rapid growth. Its Series K closed on September 8, 2025, raising $1 billion at a valuation of more than $100 billion, co-led by Andreessen Horowitz, Insight Partners, MGX, Thrive Capital, and WCM Investment Management [22]. The subsequent Series L, first reported in December 2025 and completed on February 9, 2026, comprised total investments in excess of $7 billion, including roughly $5 billion of equity at a $134 billion valuation plus about $2 billion of additional debt capacity [4][17].
| Round | Date | Amount | Valuation | Key Investors |
|---|---|---|---|---|
| Series A | 2013 | $14M | - | Andreessen Horowitz |
| Series B | 2014 | $33M | - | Andreessen Horowitz, New Enterprise Associates |
| Series C | 2016 | $60M | - | Various |
| Series D | 2017 | $140M | - | Andreessen Horowitz |
| Series E | 2019 | $250M | $2.75B | Andreessen Horowitz, Microsoft |
| Series F | 2020 | $400M | $6.2B | Various |
| Series G | 2021 | $1.6B | $38B | Franklin Templeton, Amazon |
| Series H | 2023 | $500M | $43B | Various |
| Series I | December 2024 | $10B | $62B | Thrive Capital, a16z, various |
| Series K | September 2025 | $1B | >$100B | a16z, Insight Partners, MGX, Thrive Capital, WCM [22] |
| Series L | December 2025 / February 2026 | ~$5B equity + ~$2B debt | $134B | Insight Partners, Fidelity, JP Morgan, Goldman Sachs, Qatar Investment Authority [4][17] |
The jump from $62 billion in December 2024 to $134 billion by early 2026 reflects the accelerating demand for unified data and AI platforms and Databricks' strong revenue growth.
Databricks has demonstrated strong financial metrics, and its reported run-rate accelerated through 2025 and into early 2026:
| Metric | Q3 (reported Dec 16, 2025) [5] | Q4 (reported Feb 9, 2026) [17] |
|---|---|---|
| Revenue run-rate | $4.8 billion | $5.4 billion |
| Year-over-year growth | >55% | >65% |
| AI products revenue run-rate | >$1 billion | $1.4 billion |
| Data warehousing revenue run-rate | >$1 billion | (continued growth) |
| Net retention rate | >140% | >140% |
| Customers at >$1M run-rate | >700 | >800 |
| Customers at >$10M run-rate | (not disclosed) | >70 |
| Free cash flow | Positive (trailing 12 months) | Positive (trailing 12 months) |
More than 20,000 organizations worldwide, including adidas, AT&T, Bayer, Block, Mastercard, Rivian, Unilever, and over 60% of the Fortune 500, rely on Databricks to build and scale data and AI applications, analytics, and agents [17]. The company's transition to positive free cash flow is notable for a company of its size and growth rate, and it has been cited as a key factor in the company's readiness for a potential public listing.
The most frequently discussed competitive rivalry in the data platform market is between Databricks and Snowflake. The two companies approach the market from different directions:
| Dimension | Databricks | Snowflake |
|---|---|---|
| Origin | Open-source data processing (Spark) | Cloud data warehousing |
| Architecture | Data lakehouse | Shared-data cloud warehouse |
| AI/ML capabilities | Deep (Mosaic AI, MLflow, model training) | Growing (Cortex AI, Snowpark) |
| Open source commitment | Strong (Spark, Delta Lake, MLflow, Unity Catalog) | Moderate (Iceberg adoption, Open Catalog) |
| Data engineering | Native strength | Acquired capability |
| Data warehousing | Growing strength | Native strength |
| Pricing model | Consumption-based | Consumption-based |
| Unstructured data | Native support for text, images, files | Optimized for structured and semi-structured |
| Learning curve | Code-centric (Python, SQL, Scala) | SQL-first, analyst-friendly |
Databricks has traditionally been stronger in data engineering and machine learning, while Snowflake has dominated the data warehousing and analytics market. Both companies are now converging on each other's territory, with Databricks investing heavily in its SQL and warehousing capabilities and Snowflake expanding into AI and ML. The introduction of Databricks' Lakebase (Postgres-compatible transactional database) and Snowflake's Cortex AI in 2025 further blurred the lines between the two platforms.
In 2025, Snowflake responded to Databricks' AI advances by doubling down on openness with Open Catalog and native Iceberg support, enabling teams to work with data in open formats. Snowflake also unveiled Openflow, a low-code ingestion and transformation service built on Apache NiFi, aimed at simplifying data pipelines for less technical users [16].
Databricks countered with several innovations:
Industry analysts generally view Databricks as having a deeper AI/ML stack due to the MosaicML acquisition and its extensive open-source ecosystem, while Snowflake retains advantages in ease of use for traditional analytics workloads and a larger installed base of SQL-focused users. The consensus for 2026 is that if the primary need is advanced analytics, machine learning, and unified data engineering, Databricks is the stronger choice; for SQL analytics, BI concurrency, and governed reporting, Snowflake typically fits better [16].
Databricks CEO Ali Ghodsi has repeatedly said he would not rule out a 2026 initial public offering. As of early 2026, the company is generating positive free cash flow, its revenue run-rate has reached $5.4 billion, and its growth rate exceeds 65% year over year [17]. The $134 billion private valuation positions Databricks as potentially one of the largest technology IPOs in history if and when it proceeds. Industry observers expect that a Databricks IPO would be a landmark event for the enterprise AI and data platform market.
Databricks' open-source strategy has been central to its success. The company has consistently developed and contributed to open-source projects that form the foundation of its commercial platform:
| Project | Description | Status |
|---|---|---|
| Apache Spark | Distributed data processing framework | Apache Foundation |
| Delta Lake | ACID-compliant storage layer for data lakes | Linux Foundation |
| MLflow | ML lifecycle management platform | Linux Foundation |
| Unity Catalog | Unified governance for data and AI | Open source (2024) |
| DBRX | Mixture-of-experts language model | Open source |
| Dolly | Instruction-following language model | Open source |
This approach creates a broad ecosystem of users and contributors, many of whom eventually become Databricks customers. It also reduces vendor lock-in concerns, as organizations can use the open-source components independently of Databricks' commercial platform.
As of early 2026, Databricks is one of the most valuable private technology companies in the world at $134 billion, and it crossed a $5.4 billion revenue run-rate growing more than 65% year over year in the quarter reported on February 9, 2026, with $1.4 billion of that from AI products and positive free cash flow over the trailing 12 months [17]. Its platform has expanded well beyond its Apache Spark roots to encompass data warehousing, AI model training and serving, natural language analytics (Genie), transactional databases (Lakebase), and comprehensive governance (Unity Catalog). The MosaicML acquisition, DBRX model, and Mosaic AI Agent Framework have established Databricks as a credible player in the foundation model and agentic AI spaces, while the company's open-source commitments and lakehouse architecture continue to differentiate it in the enterprise market. With a potential IPO on the horizon, Databricks is entering the next phase of its growth as a public-market-ready enterprise AI platform.