Anyscale

Anyscale, Inc.
Type	Private
Industry	Artificial intelligence, Distributed computing, Cloud infrastructure
Founded	2019; Berkeley, California
Founders	Robert Nishihara, Philipp Moritz, Ion Stoica, Michael I. Jordan
Headquarters	San Francisco, California, United States
CEO	Keerti Melkote (since July 2024)
Key product	Ray (open-source distributed computing framework)
Total funding	~$281 million
Valuation	$1 billion (2021)
Website	anyscale.com

Anyscale is an American technology company, founded in 2019 by the creators of Ray at the University of California, Berkeley, that develops the Anyscale Platform: a fully managed compute service built on Ray, the open-source distributed computing framework for scaling artificial intelligence, machine learning, and Python applications.^[2] The company has raised approximately $281 million across four rounds and reached a $1 billion valuation (unicorn status) at its December 2021 Series C.^[3][6] Its co-founders, Robert Nishihara, Philipp Moritz, and Ion Stoica, built Ray inside Berkeley's RISELab, and Stoica also co-founded Databricks and Conviva.^[2][14] Anyscale provides infrastructure for distributed training, model serving, batch inference, and fine-tuning of large language models, and its technology is used by organizations including OpenAI, Uber, Shopify, Spotify, Instacart, Netflix, and ByteDance.^[8]

What is Anyscale and who founded it?

Anyscale commercializes Ray, the distributed computing framework its founders created at UC Berkeley. The company was officially founded in 2019 and was originally headquartered in Berkeley, California, before moving to San Francisco. Its four co-founders are Robert Nishihara (CEO at launch), Philipp Moritz (CTO), Ion Stoica, and Michael I. Jordan.^[2] The stated mission was to "make scalable computing effortless" by building a commercially supported platform around the open-source Ray framework.^[2]

Origins at UC Berkeley

Anyscale's roots trace back to 2016, when Robert Nishihara and Philipp Moritz, then PhD students at the University of California, Berkeley's RISELab (Real-time Intelligent Secure Execution Laboratory), began developing Ray as a class project. The initial goal was to build a framework for distributed training of machine learning models, particularly for reinforcement learning applications that required both task-parallel and actor-based computations.^[1]

Nishihara and Moritz worked under the guidance of UC Berkeley computer science professors Ion Stoica and Michael I. Jordan. Stoica, who also co-founded Databricks and co-created Apache Spark, brought deep expertise in distributed systems and cloud computing.^[14] Jordan, the Pehong Chen Distinguished Professor in the Department of Electrical Engineering and Computer Science and the Department of Statistics, contributed foundational knowledge in machine learning and optimization theory.^[15]

The Ray framework was formally introduced in the 2017 paper "Ray: A Distributed Framework for Emerging AI Applications" (arXiv:1712.05889), and the work was presented at the USENIX Symposium on Operating Systems Design and Implementation (OSDI) in 2018.^[1] The paper described Ray's unified interface for expressing both task-parallel and actor-based computations, supported by a distributed scheduler and a distributed, fault-tolerant store for managing system control state.^[1]

Company founding and early funding

In December 2019, Anyscale launched publicly with a $20.6 million Series A funding round led by Andreessen Horowitz (a16z).^[2][3] Additional participants included NEA (New Enterprise Associates), Intel Capital, Ant Financial, Amplify Partners, 11.2 Capital, and The House Fund.^[2] The funding was announced alongside the company's goal of democratizing distributed programming for developers who lacked specialized expertise in distributed systems.^[2]

How much funding has Anyscale raised?

Anyscale has raised approximately $281 million in total across four rounds from 14 investors.^[3] The table below summarizes its disclosed financing history.

Scroll sideways for more →

Growth and subsequent funding rounds

Less than a year after its Series A, Anyscale raised $40 million in a Series B round in October 2020, led by existing investor NEA with participation from Andreessen Horowitz, Intel Capital, and Foundation Capital.^[4] This round coincided with the launch of the Anyscale managed platform, which offered a hosted service for running Ray workloads without manual cluster management.^[4]

In December 2021, Anyscale closed the first tranche of its Series C funding, raising $100 million in a round led by NEA. This pushed the company to a $1 billion valuation, making it a unicorn.^[6] The investors in this round included Addition, Andreessen Horowitz, Intel Capital, and Foundation Capital.^[6]

In August 2022, Anyscale raised an additional $99 million in an extension of its Series C round, co-led by existing investors Addition, Intel Capital, and Foundation Capital.^[5][6] This brought total Series C funding to $199 million and total funding to approximately $281 million.^[5] The announcement coincided with the release of Ray 2.0 at the Ray Summit 2022 conference.^[6]

Who is the CEO of Anyscale?

In July 2024, Anyscale appointed Keerti Melkote as its new CEO, succeeding Robert Nishihara.^[7] Melkote brought substantial enterprise experience to the role. He founded Aruba Networks in 2001, led the company through its IPO in 2007, and oversaw its acquisition by Hewlett Packard Enterprise (HPE) in 2015 for approximately $3 billion.^[7] Under his leadership at HPE, Aruba Networks grew to over $5 billion in revenue as he served as President of the Intelligent Edge division.^[7] Melkote's appointment followed a year of 4x revenue growth at Anyscale and reflected the company's pivot toward broader enterprise sales and adoption.^[8]

Ion Stoica transitioned to the role of Executive Chairman, while Robert Nishihara took on customer interactions and product roadmap and strategy, and Philipp Moritz continued in a co-founder capacity.^[7]

What is the Ray framework?

Ray is an open-source distributed computing framework written in Python and C++. It provides a simple, universal API for building distributed applications that can scale from a single laptop to a large cluster.^[16] As of early 2026, the Ray repository on GitHub has over 40,000 stars, more than 1,000 contributors, and orchestrates more than one million clusters per month.^[18]

Architecture

Ray's architecture is organized as a three-layer stack:

Scroll sideways for more →

A Ray cluster consists of a head node responsible for cluster management (including the Global Control Store, or GCS, for metadata) and a set of worker nodes that execute tasks. The system supports dynamic scheduling and automatic resource management across nodes.^[16]

Core primitives

Ray Core provides three fundamental building blocks:

Scroll sideways for more →

Developers convert standard Python functions into distributed tasks by adding the @ray.remote decorator. For stateful computations, Ray actors provide persistent worker processes whose methods can access and modify internal state. Objects serve as shared, immutable data that can be passed between tasks and actors across the cluster.^[16]

Ray AI Libraries

Ray includes a suite of specialized libraries for machine learning workloads:

Scroll sideways for more →

These libraries are designed to work together within a single Ray application, allowing users to build end-to-end ML pipelines that cover data processing, training, tuning, and serving in one unified system.^[16]

Ray 2.0 and beyond

Ray 2.0, released in August 2022, introduced several major improvements.^[12] The Ray AI Runtime (AIR) provided a unified toolkit for combining Ray's libraries into cohesive ML workflows.^[12] Ray 2.0 also added native support for shuffling 100 terabytes or more of data through Ray Datasets, improved Kubernetes support through the KubeRay operator, and enhanced the monitoring dashboard.^[12]

Subsequent releases continued to expand the framework's capabilities. Ray 2.3 (February 2023) added Python 3.11 support and Linux ARM64 builds.^[11] Ray 2.4 introduced dedicated infrastructure for LLM training, tuning, inference, and serving. Ray 2.7 added RayLLM for simplified large language model deployment and introduced major stability improvements to KubeRay. As of early 2026, the latest stable release is Ray 2.54.^[18]

Deployment options

Ray can be deployed across a variety of environments:

Scroll sideways for more →

What is the Anyscale Platform?

The Anyscale Platform is the company's commercial product: a fully managed service for running Ray workloads at scale. It abstracts away the complexity of cluster provisioning, scaling, monitoring, and failure recovery, allowing data scientists and ML engineers to focus on their application code rather than infrastructure.^[17]

Key features

Managed infrastructure. Anyscale handles all aspects of Ray cluster management, including provisioning, configuration, monitoring, and teardown. Users can migrate existing Ray workloads to Anyscale with no code changes.^[17]

Serverless autoscaling. The platform automatically scales clusters up or down based on workload demands, optimizing resource utilization and cost. Idle resources are released when not in use.^[17]

Production jobs and services. Anyscale supports both scheduled batch jobs and low-latency production services. The system automatically creates and manages clusters for job execution and tears them down upon completion.^[17]

Cost tracking. A unified dashboard provides visibility into compute expenses organized by jobs, clusters, and individual users.^[17]

Observability. Built-in Grafana dashboards enable monitoring of cluster health, job progress, and resource utilization. Users can also integrate their own observability tools.^[17]

APIs and SDKs. Anyscale provides programmatic interfaces for automating job submission, cluster management, and integration with CI/CD pipelines.^[17]

Deployment models

Anyscale offers two primary deployment models:

Scroll sideways for more →

The BYOC model involves a platform fee on top of the customer's existing cloud provider costs. Billing for BYOC deployments is typically handled through cloud marketplaces. Anyscale also supports Kubernetes-based deployments on Amazon EKS, Google GKE, Azure AKS, CoreWeave CKS, and Oracle OKE.^[17]

How much does Anyscale cost?

Anyscale uses a pay-as-you-go pricing model with no monthly fixed fees. Users pay per-hour charges for compute instances, with rates varying based on hardware type (especially GPU model and specifications), cloud provider, and region. A $100 credit is provided for new users to get started. Enterprise customers can negotiate committed contracts with volume discounts. Anyscale is also available through the Microsoft Azure Marketplace.^[17]

LLM capabilities

Anyscale has positioned itself as a platform for building, fine-tuning, and serving large language models. The platform provides integrated tools for the full LLM lifecycle.

Fine-tuning

Anyscale supports both full fine-tuning and parameter-efficient methods for adapting pre-trained models to specific tasks:

Scroll sideways for more →

Distributed training is supported through strategies such as FSDP (Fully Sharded Data Parallel), DeepSpeed, and Megatron for efficient scaling across multiple GPUs. Monitoring integrations with Weights & Biases, MLflow, and TensorBoard allow users to track training metrics and debug performance issues.^[17]

Anyscale has published case studies showing that users can fine-tune a 6-billion-parameter LLM for less than $7 using their platform.^[13]

Model serving

Anyscale combines Ray Serve for orchestration with vLLM for high-performance inference. This integration provides features such as continuous batching, PagedAttention for efficient memory management, streaming output, and multi-model serving from a single cluster. The serving layer is compatible with the OpenAI API format, allowing developers to switch from proprietary APIs with minimal code changes.^[17]

Anyscale Endpoints

Anyscale Endpoints is a serverless API service for accessing open-source large language models. Launched in 2023, the service offered hosted access to models from the Llama, Mistral, and CodeLlama families, among others.^[13] Anyscale marketed the service as up to 10x more cost-effective than comparable proprietary solutions, with pricing at $1 per million tokens for the 70-billion-parameter Llama 2 model.^[13]

The service supported the OpenAI API format, enabling straightforward migration from OpenAI's API.^[13] Anyscale also offered fine-tuning through the Endpoints platform, allowing users to customize open-source models with their own data.^[13]

As of August 2024, Anyscale Endpoints was integrated into the Anyscale Platform as a unified offering rather than a standalone service.

Batch inference

Anyscale supports large-scale batch inference through Ray Data, which enables offline processing of large datasets with LLMs. The platform can distribute inference workloads across multiple GPUs and nodes, handling tasks such as text classification, summarization, entity extraction, and embedding generation at scale.^[17]

Who uses Anyscale and Ray?

Ray and the Anyscale Platform are used by a wide range of technology companies and enterprises. Some of the most notable users include:

Scroll sideways for more →

OpenAI's use of Ray received particular attention when it became publicly known that the framework powered much of the training infrastructure behind ChatGPT and other OpenAI models.^[9] At Ray Summit 2022, OpenAI engineers discussed how they used Ray to coordinate thousands of hardware components during model training, managing data distribution, failure recovery, and resource allocation across the cluster.^[10]

Ray Summit

Ray Summit is Anyscale's annual conference, bringing together AI engineers, researchers, and open-source contributors. The conference features keynotes, technical talks, workshops, and community networking.

Scroll sideways for more →

How does Anyscale compare to competitors?

Anyscale operates in the AI infrastructure and MLOps market, competing with several categories of platforms:

Scroll sideways for more →

Anyscale differentiates itself through its tight integration with the open-source Ray ecosystem, its focus on distributed compute as a core strength rather than an add-on, and its cloud-agnostic approach that avoids lock-in to a single cloud provider. Notably, Databricks, one of its competitors, was also co-founded by Anyscale co-founder Ion Stoica, and Ray can itself run on the Databricks platform.

Technical leadership and research

Anyscale's founding team has made significant contributions to systems research and the broader AI ecosystem:

Ion Stoica is a Romanian-American computer scientist, a professor in the EECS Department at UC Berkeley, and the Director of SkyLab. He co-founded both Databricks (with the creators of Apache Spark) and Conviva. He is an ACM Fellow, a member of the National Academy of Sciences, the National Academy of Engineering, and the American Academy of Arts and Sciences. In 2022, Forbes ranked him as one of the wealthiest Romanians with a net worth of $1.6 billion.^[14]

Michael I. Jordan is the Pehong Chen Distinguished Professor at UC Berkeley in both the EECS and Statistics departments. He is widely regarded as one of the most influential figures in machine learning and artificial intelligence. He is a member of the National Academy of Sciences, the National Academy of Engineering, the American Academy of Arts and Sciences, and a Foreign Member of the Royal Society.^[15]

Robert Nishihara completed his PhD at UC Berkeley's RISELab, focusing on machine learning, optimization, and AI. He served as Anyscale's initial CEO and continues as co-founder.

Philipp Moritz completed his PhD in the EECS Department at UC Berkeley, with research interests spanning artificial intelligence, machine learning, and distributed systems. He serves as Anyscale's CTO.

Open-source ecosystem

Ray is released under the Apache License 2.0 and maintains an active open-source community. The project is hosted on GitHub at ray-project/ray.^[18] As of 2024, the project has accumulated over 40,000 stars and more than 1,000 contributors.^[18]

The Ray ecosystem includes several related open-source projects and integrations:

Scroll sideways for more →

Ray's integration with vLLM is particularly significant for LLM deployment. While standalone vLLM handles single-node inference efficiently, Ray Serve adds multi-node scaling, autoscaling, fault tolerance, and distributed observability on top of vLLM, making it suitable for production-grade LLM serving.^[16]

See also

• Distributed computing
• Distributed training
• Machine learning
• Large language model
• GPU computing
• Reinforcement learning
• Fine-tuning
• Databricks
• Apache Spark
• PyTorch
• vLLM

References

1. Moritz, P., Nishihara, R., Wang, S., Tumanov, A., Liaw, R., Liang, E., Elibol, M., Yang, Z., Paul, W., Jordan, M.I. and Stoica, I. (2018). "Ray: A Distributed Framework for Emerging AI Applications." In *Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI '18)*. arXiv:1712.05889. ↩
2. "Founders of Open Source Project Ray Launch Anyscale with $20.6M in Funding to Democratize Distributed Programming." Anyscale, December 2019. https://www.anyscale.com/blog/founders-of-open-source-project-ray-launch-anyscale-with-20-6m-in-funding-to-democratize-distributed-programming ↩
3. "Anyscale - Crunchbase Company Profile & Funding." Crunchbase. https://www.crunchbase.com/organization/anyscale ↩
4. "Anyscale adds $40M to bring its Ray-based distributed computing tech to the enterprise masses." TechCrunch, October 21, 2020. https://techcrunch.com/2020/10/21/anyscale-adds-40m-to-bring-its-ray-based-distributed-computing-tech-to-the-enterprise-masses/ ↩
5. "Anyscale raises $99M to build its distributed AI computing platform." SiliconANGLE, August 23, 2022. https://siliconangle.com/2022/08/23/anyscale-raises-99m-build-distributed-ai-computing-platform/ ↩
6. "Anyscale Secures $100M Series C at $1B Valuation to Radically Simplify Scaling and Productionizing AI Applications." Anyscale, December 2021. https://www.anyscale.com/press/anyscale-secures-100m-series-c-at-1b-valuation-to-radically-simplify-scaling-and-productionizing-ai-applications ↩
7. "Anyscale Names Industry Veteran Keerti Melkote Chief Executive Officer." Anyscale, July 2024. https://www.anyscale.com/press/anyscale-names-industry-veteran-keerti-melkote-chief-executive-officer ↩
8. "Anyscale, software provider for OpenAI and other major firms, hires new CEO to boost sales." Semafor, July 31, 2024. https://www.semafor.com/article/07/31/2024/anyscale-software-provider-for-openai-and-other-major-firms-hires-new-ceo-to-boost-sales ↩
9. "How Ray, a Distributed AI Framework, Helps Power ChatGPT." The New Stack. https://thenewstack.io/how-ray-a-distributed-ai-framework-helps-power-chatgpt/ ↩
10. "OpenAI Chats about Scaling LLMs at Anyscale's Ray Summit." The New Stack. https://thenewstack.io/openai-chats-about-scaling-llms-at-anyscales-ray-summit/ ↩
11. "AnyScale Bolsters Ray, the Super-Scalable Framework Used to Train ChatGPT." BigDATAwire, February 10, 2023. https://www.bigdatawire.com/2023/02/10/anyscale-bolsters-ray-the-super-scalable-framework-used-to-train-chatgpt/ ↩
12. "Announcing Ray 2.0." Anyscale, August 2022. https://www.anyscale.com/blog/announcing-ray-2-0 ↩
13. "Anyscale Endpoints Preview: Fast, Cost-Efficient, and Scalable LLM APIs." Anyscale. https://www.anyscale.com/blog/anyscale-endpoints-fast-and-scalable-llm-apis ↩
14. "Ion Stoica." Wikipedia. https://en.wikipedia.org/wiki/Ion_Stoica ↩
15. "Michael I. Jordan." UC Berkeley EECS. https://www2.eecs.berkeley.edu/Faculty/Homepages/jordan.html ↩
16. Ray documentation. https://docs.ray.io/en/latest/ray-overview/index.html ↩
17. Anyscale documentation. https://docs.anyscale.com ↩
18. Ray GitHub repository. https://github.com/ray-project/ray ↩
19. "Ray Summit 2026." Anyscale. https://www.anyscale.com/ray-summit/2026 ↩