Databricks

Scroll sideways for more →

Databricks is an American enterprise software company that provides a unified data analytics and artificial intelligence platform built around the data lakehouse architecture. Founded in 2013 by the creators of Apache Spark, Databricks was valued at $134 billion in its February 2026 Series L financing, making it one of the most valuable private technology companies in the world, and it crossed a $5.4 billion revenue run-rate growing more than 65% year over year in the quarter reported on February 9, 2026 ^[4][17]. The company's Data Intelligence Platform combines data engineering, data warehousing, and machine learning into a single environment, and its expansion into AI, through the $1.3 billion acquisition of MosaicML and the release of the open-source DBRX model, has positioned it as a major player in the enterprise AI market ^[18]. More than 20,000 organizations worldwide, including over 60% of the Fortune 500, run on the platform, and the company is widely expected to be a candidate for one of the largest technology IPOs in history ^[17].

Databricks grew directly out of the Apache Spark open-source project and the data lakehouse concept its founders defined in a 2021 research paper. Ali Ghodsi serves as CEO and Matei Zaharia, who created Apache Spark during his PhD at UC Berkeley, serves as Chief Technologist.

What is Databricks?

Databricks is a data and AI company whose flagship product, the Data Intelligence Platform, lets organizations store, govern, analyze, and build AI applications on all of their data in one place using the open data lakehouse architecture. The lakehouse unifies the low-cost, flexible storage of a data lake with the reliability, performance, and governance of a data warehouse, so that data engineering, business intelligence, and machine learning workloads can run against a single copy of the data rather than across separate, siloed systems. The platform spans Delta Lake (open storage), Unity Catalog (governance), Mosaic AI (model training, serving, and agents), Lakebase (a transactional database), and Genie (natural language analytics).

History and Founding

Databricks was founded in 2013 by seven co-founders, all of whom were connected through the AMPLab (Algorithms, Machines, and People Lab) at the University of California, Berkeley:

Scroll sideways for more →

The company grew directly out of the Apache Spark project, which Matei Zaharia created during his doctoral research at Berkeley. Spark was designed as a fast, general-purpose cluster computing system that could handle both batch and streaming data processing. It quickly became one of the most popular open-source data processing frameworks in the world, and Databricks was founded to build a commercial platform and managed service around it.

From the beginning, Databricks embraced an open-source-first strategy, contributing heavily to Apache Spark and later creating additional open-source projects including Delta Lake (a storage layer for data lakes), MLflow (a platform for the machine learning lifecycle), and Apache Spark Structured Streaming.

Data Lakehouse Architecture

Databricks is most closely associated with the data lakehouse concept, which it helped popularize. The data lakehouse combines the flexibility and low cost of a data lake with the data management and performance features of a traditional data warehouse.

Where did the lakehouse concept come from?

The term and the architecture were formalized by Databricks co-founders Matei Zaharia, Ali Ghodsi, and Reynold Xin, together with Michael Armbrust, in the paper "Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics," presented at the 11th Conference on Innovative Data Systems Research (CIDR) in January 2021 ^[19]. The paper argued that the lakehouse would (i) be based on open direct-access data formats such as Apache Parquet, (ii) have first-class support for machine learning and data science, and (iii) offer state-of-the-art performance. The authors predicted that "the data warehouse architecture as we know it today will wither in the coming years and be replaced by a new architectural pattern, the Lakehouse," addressing data warehouse problems such as staleness, reliability, total cost of ownership, and data lock-in ^[19].

Traditional Approaches

Historically, organizations maintained separate systems for different data workloads:

Scroll sideways for more →

The lakehouse architecture merges these approaches by adding warehouse-like features (ACID transactions, schema enforcement, indexing) directly on top of data lake storage (typically cloud object storage like Amazon S3 or Azure Blob Storage). Databricks implemented this through Delta Lake, an open-source storage layer that brings reliability and performance to data lakes.

Delta Lake

Delta Lake is the foundation of Databricks' lakehouse architecture. It provides:

• ACID transactions on data lake storage, preventing data corruption
• Schema enforcement and evolution, ensuring data quality
• Time travel, allowing queries against historical versions of data
• Unified batch and streaming data processing
• Scalable metadata handling for petabyte-scale datasets

Delta Lake uses the open Parquet file format underneath, which means data stored in Delta Lake can be read by any tool that supports Parquet, avoiding vendor lock-in.

Delta Lake Technical Details

Delta Lake extends the Parquet format with a file-based transaction log that records every change to the data. This log enables several critical capabilities ^[11]:

Scroll sideways for more →

Delta Lake is fully compatible with Apache Spark APIs and was developed for tight integration with Structured Streaming, allowing a single copy of data to serve both batch and streaming use cases. As of 2025, Delta Lake also supports interoperability with Apache Iceberg, allowing data stored in Delta format to be read by Iceberg-compatible tools and vice versa ^[11].

Lakebase

In 2025, Databricks unveiled Lakebase, a Postgres-compatible transactional database engine built for the lakehouse, allowing teams to run OLTP-style applications directly on the same data infrastructure used for analytics and AI workloads.

Lakebase entered public preview at the 2025 Data + AI Summit and reached general availability on February 3, 2026. It represents Databricks' entry into the online transaction processing (OLTP) market, traditionally the domain of dedicated database systems like PostgreSQL, MySQL, and cloud-native databases ^[12]. Databricks describes Lakebase as "a serverless Postgres database built for AI agents," and CEO Ali Ghodsi said in February 2026 that the company would "double down on Lakebase so developers can create operational databases built for AI agents" with the proceeds of its Series L round ^[17].

Key Lakebase features include:

Scroll sideways for more →

Lakebase bridges the gap between operational applications and analytical workloads. Instead of maintaining separate OLTP and analytics databases with complex ETL pipelines between them, organizations can use Lakebase for transactional workloads while the built-in sync keeps Delta Lake tables updated for analytics and AI ^[12].

AI and Machine Learning Products

Mosaic ML Acquisition (2023)

Databricks announced on June 26, 2023, that it would acquire MosaicML for approximately $1.3 billion, inclusive of retention packages, marking its most significant move into the generative AI space; the deal closed on July 19, 2023, bringing MosaicML's roughly 62 employees and its MPT family of open large language models under Databricks ^[18]. MosaicML had built tools and infrastructure that simplified and reduced the cost of training large language models, making it possible for enterprises to train custom models without the massive engineering teams that organizations like OpenAI or Google maintained.

Following the acquisition, MosaicML's technology was integrated into Databricks as Mosaic AI, which covers the full machine learning lifecycle from feature engineering and model training to deployment and monitoring. The acquisition brought key talent, including MosaicML's expertise in efficient training techniques, and gave Databricks the capability to offer foundation model training as a service to its enterprise customers.

Mosaic AI Agent Framework

The Mosaic AI Agent Framework is Databricks' solution for building production-quality AI agent systems, including retrieval-augmented generation (RAG) applications. The framework provides a suite of tooling for developing, evaluating, and deploying compound AI systems that leverage multiple components such as tuned models, retrieval, tool use, and reasoning agents ^[13].

Key capabilities include:

Scroll sideways for more →

Agent Bricks, announced at the 2025 Data + AI Summit, simplifies agent development by allowing users to provide a high-level description of the agent's task and connect enterprise data, with the system handling optimization automatically ^[13].

How fast is enterprise agent adoption on Databricks?

Databricks' "2026 State of AI Agents" report, published in January 2026, found that usage of multi-agent workflows on the Databricks platform grew 327% in less than four months, from June to October 2025, as enterprises moved from single chatbots to systems of collaborating agents ^[20]. The report drew on telemetry from the more than 20,000 organizations on the Data Intelligence Platform, including over 60% of the Fortune 500. It also reported that companies using evaluation tooling get nearly 6x more AI projects into production, and that those using AI governance get over 12x more, underscoring evaluation and governance as the main bottlenecks to scaling agents ^[20].

DBRX Open Model

In March 2024, Databricks released DBRX, its first foundation model, under the Databricks Open Model License. DBRX uses a fine-grained mixture-of-experts (MoE) architecture built on the MegaBlocks open-source project. Verified details of DBRX include ^[21]:

Scroll sideways for more →

DBRX's fine-grained MoE design gives 65x more possible combinations of experts than the 8-choose-2 designs used by Mixtral and Grok-1, which Databricks said improved model quality ^[21]. At release, Databricks reported that DBRX outperformed Meta's Llama 2-70B, Mistral AI's Mixtral, and xAI's Grok-1 on several language understanding, programming, and mathematics benchmarks, with inference up to 2x faster than Llama 2-70B ^[21]. DBRX was designed to demonstrate that enterprises could build competitive foundation models at a fraction of the cost of frontier models from major AI labs, reinforcing Databricks' pitch that companies should own and customize their AI models rather than relying entirely on third-party APIs.

Dolly Models

In March 2023, shortly before the MosaicML acquisition, Databricks released Dolly, an open-source language model named after Dolly the sheep (the first cloned mammal). Dolly was a 6 billion parameter model based on EleutherAI's GPT-J, fine-tuned on a dataset of instruction-following examples generated by Databricks employees. Dolly was notable as one of the first demonstrations that a relatively small model could exhibit instruction-following capabilities similar to much larger models when fine-tuned on high-quality data.

Databricks later released Dolly 2.0, which used a commercially permissive training dataset created by Databricks employees, making it one of the first instruction-following LLMs that could be used for commercial purposes without licensing restrictions.

MLflow

MLflow is an open-source platform for managing the end-to-end machine learning lifecycle. Originally created by Databricks and later donated to the Linux Foundation, MLflow provides:

Scroll sideways for more →

MLflow has become one of the most widely adopted ML lifecycle management tools in the industry, with integration support across major cloud platforms and ML frameworks. Its open-source nature and broad compatibility have helped Databricks build mindshare in the data science community.

MLflow 3

MLflow 3, released in 2025, introduces significant enhancements to experiment tracking, observability, and performance evaluation for both traditional ML models and generative AI applications ^[14].

Key new concepts in MLflow 3 include:

Scroll sideways for more →

MLflow 3's integration with Unity Catalog means that models tracked in MLflow automatically benefit from centralized governance, lineage tracking, and access control across the Databricks platform ^[14].

Model Serving

Databricks Model Serving provides real-time and batch inference capabilities integrated with the lakehouse platform. It supports serving custom models trained on Databricks, foundation models accessed via APIs, and external models from providers like OpenAI and Anthropic. Model Serving integrates with MLflow for model versioning and lifecycle management, and with Unity Catalog for governance and access control.

The platform also includes Mosaic AI Model Serving for batch inferencing, which simplifies the infrastructure needed to process unstructured data at scale using large language models.

Model Serving Architecture

Mosaic AI Model Serving deploys models to REST API endpoints with automatic monitoring of requests and responses. The serving infrastructure supports several deployment patterns ^[14]:

Scroll sideways for more →

All served models are automatically registered in Unity Catalog, ensuring consistent governance and access control regardless of deployment type.

Genie

Genie (officially AI/BI Genie) is Databricks' natural language interface for data analysis. Generally available as of 2025, Genie allows business users to query data, build visualizations, and receive AI-generated insights using conversational language, without writing SQL or code. Genie represents Databricks' push to make its platform accessible to non-technical users and to demonstrate the practical value of AI integration in everyday business analytics. Databricks described Genie in February 2026 as a "conversational AI assistant that lets any employee chat with their data," and named it, alongside Lakebase, as a primary focus for the capital raised in its Series L round ^[17].

An API in public preview also enables developers to integrate Genie into custom-built applications and productivity tools.

Unity Catalog

Unity Catalog is Databricks' unified governance solution for data and AI assets. It provides a single place to manage access controls, auditing, lineage, and discovery across all data, ML models, notebooks, and dashboards within a Databricks workspace. Key features include:

• Centralized access control across all data and AI assets
• Data lineage tracking showing how data flows through pipelines
• Automated auditing for compliance and security
• Cross-workspace governance for organizations with multiple Databricks deployments
• AI model governance integrated with MLflow Model Registry

Unity Catalog addresses a critical enterprise need: as organizations deploy more AI models and manage more data, they require robust governance to ensure compliance with regulations, protect sensitive data, and maintain data quality.

Unity Catalog Deep Dive

Unity Catalog organizes assets in a three-level namespace: catalog, schema, and object. This hierarchy maps naturally to organizational structures and allows fine-grained access control ^[15].

Scroll sideways for more →

Unity Catalog governs the following asset types:

Scroll sideways for more →

Unity Catalog Metrics

Introduced in 2025, Unity Catalog Metrics extends governance to business metrics definitions, ensuring that key performance indicators (KPIs) are defined once and used consistently across dashboards, reports, and AI applications. This prevents the common problem of different teams calculating the same metric in different ways ^[15].

Open-Source Unity Catalog

Databricks open-sourced Unity Catalog in 2024, allowing organizations to use its governance capabilities outside the Databricks platform. The open-source version supports Apache Iceberg, Delta Lake, and other table formats, reinforcing Databricks' strategy of building ecosystem adoption through open-source contributions.

Funding and Valuation

How much has Databricks raised and at what valuation?

Databricks has raised substantial funding across numerous rounds, reflecting its rapid growth. Its Series K closed on September 8, 2025, raising $1 billion at a valuation of more than $100 billion, co-led by Andreessen Horowitz, Insight Partners, MGX, Thrive Capital, and WCM Investment Management ^[22]. The subsequent Series L, first reported in December 2025 and completed on February 9, 2026, comprised total investments in excess of $7 billion, including roughly $5 billion of equity at a $134 billion valuation plus about $2 billion of additional debt capacity ^[4][17].

Scroll sideways for more →

The jump from $62 billion in December 2024 to $134 billion by early 2026 reflects the accelerating demand for unified data and AI platforms and Databricks' strong revenue growth.

Financial Performance

Databricks has demonstrated strong financial metrics, and its reported run-rate accelerated through 2025 and into early 2026:

Scroll sideways for more →

More than 20,000 organizations worldwide, including adidas, AT&T, Bayer, Block, Mastercard, Rivian, Unilever, and over 60% of the Fortune 500, rely on Databricks to build and scale data and AI applications, analytics, and agents ^[17]. The company's transition to positive free cash flow is notable for a company of its size and growth rate, and it has been cited as a key factor in the company's readiness for a potential public listing.

Competition with Snowflake

The most frequently discussed competitive rivalry in the data platform market is between Databricks and Snowflake. The two companies approach the market from different directions:

Scroll sideways for more →

Databricks has traditionally been stronger in data engineering and machine learning, while Snowflake has dominated the data warehousing and analytics market. Both companies are now converging on each other's territory, with Databricks investing heavily in its SQL and warehousing capabilities and Snowflake expanding into AI and ML. The introduction of Databricks' Lakebase (Postgres-compatible transactional database) and Snowflake's Cortex AI in 2025 further blurred the lines between the two platforms.

Recent Competitive Developments (2025-2026)

In 2025, Snowflake responded to Databricks' AI advances by doubling down on openness with Open Catalog and native Iceberg support, enabling teams to work with data in open formats. Snowflake also unveiled Openflow, a low-code ingestion and transformation service built on Apache NiFi, aimed at simplifying data pipelines for less technical users ^[16].

Databricks countered with several innovations:

• Lakebase brought OLTP capabilities to the lakehouse, eliminating the need for separate transactional databases
• Delta Lake's Iceberg interoperability allowed data in Delta format to be read by Iceberg-compatible tools
• Unity Catalog Metrics extended governance to business metrics definitions
• Instructed Retriever delivered up to 70% better performance than traditional RAG for complex, instruction-based questions by using metadata more effectively ^[16]

Industry analysts generally view Databricks as having a deeper AI/ML stack due to the MosaicML acquisition and its extensive open-source ecosystem, while Snowflake retains advantages in ease of use for traditional analytics workloads and a larger installed base of SQL-focused users. The consensus for 2026 is that if the primary need is advanced analytics, machine learning, and unified data engineering, Databricks is the stronger choice; for SQL analytics, BI concurrency, and governed reporting, Snowflake typically fits better ^[16].

IPO Plans

Will Databricks IPO in 2026?

Databricks CEO Ali Ghodsi has repeatedly said he would not rule out a 2026 initial public offering. As of early 2026, the company is generating positive free cash flow, its revenue run-rate has reached $5.4 billion, and its growth rate exceeds 65% year over year ^[17]. The $134 billion private valuation positions Databricks as potentially one of the largest technology IPOs in history if and when it proceeds. Industry observers expect that a Databricks IPO would be a landmark event for the enterprise AI and data platform market.

Open Source Strategy

Databricks' open-source strategy has been central to its success. The company has consistently developed and contributed to open-source projects that form the foundation of its commercial platform:

Scroll sideways for more →

This approach creates a broad ecosystem of users and contributors, many of whom eventually become Databricks customers. It also reduces vendor lock-in concerns, as organizations can use the open-source components independently of Databricks' commercial platform.

Current State

As of early 2026, Databricks is one of the most valuable private technology companies in the world at $134 billion, and it crossed a $5.4 billion revenue run-rate growing more than 65% year over year in the quarter reported on February 9, 2026, with $1.4 billion of that from AI products and positive free cash flow over the trailing 12 months ^[17]. Its platform has expanded well beyond its Apache Spark roots to encompass data warehousing, AI model training and serving, natural language analytics (Genie), transactional databases (Lakebase), and comprehensive governance (Unity Catalog). The MosaicML acquisition, DBRX model, and Mosaic AI Agent Framework have established Databricks as a credible player in the foundation model and agentic AI spaces, while the company's open-source commitments and lakehouse architecture continue to differentiate it in the enterprise market. With a potential IPO on the horizon, Databricks is entering the next phase of its growth as a public-market-ready enterprise AI platform.

References

1. "Databricks." Wikipedia.
2. "Databricks Founders." Databricks.
3. "Databricks raises capital at $134 billion valuation in latest funding round." CNBC, December 16, 2025.
4. "Databricks completes $5 billion funding round at $134 billion valuation." CNBC, February 9, 2026. ↩
5. "Databricks Grows >55% YoY, Surpasses $4.8B Revenue Run-Rate." Databricks Newsroom, December 16, 2025. ↩
6. "From Lakehouse to Intelligence Platform: Databricks Declares a New Era at DAIS 2025." Medium.
7. "Databricks vs Snowflake 2025: The Complete Buyer's Guide." B EYE.
8. "What to Expect from the Databricks IPO in Early 2026." Allied VC.
9. "Databricks revenue, valuation & funding." Sacra.
10. "Ali Ghodsi." Wikipedia.
11. "What is Delta Lake in Databricks?" Databricks Documentation. ↩
12. "Lakebase Explained: Why Databricks Is Blending OLTP, Analytics and AI." Coeo, February 2026. ↩
13. "Mosaic AI Announcements at Data + AI Summit 2025." Databricks Blog. ↩
14. "MLflow for ML model lifecycle." Databricks Documentation. ↩
15. "Manage model lifecycle in Unity Catalog." Databricks Documentation. ↩
16. "Databricks vs. Snowflake in 2026: The Architecture-Level Guide." Bix Tech. ↩
17. "Databricks Grows >65% YoY, Surpasses $5.4 Billion Revenue Run-Rate, Doubles Down on Lakebase and Genie." Databricks Newsroom, February 9, 2026. ↩
18. "Databricks Signs Definitive Agreement to Acquire MosaicML, a Leading Generative AI Platform." Databricks Newsroom, June 26, 2023. ↩
19. Zaharia, Ghodsi, Xin, Armbrust. "Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics." CIDR 2021. ↩
20. "2026 State of AI Agents: Enterprise Insights on Building AI." Databricks. ↩
21. "Introducing DBRX: A New State-of-the-Art Open LLM." Databricks Blog, March 27, 2024. ↩
22. "Databricks is raising a Series K Investment at >$100 billion valuation." Databricks Newsroom, September 8, 2025. ↩