Ollama

Ollama is a free, open-source runtime for downloading, running, and managing open-weight large language models (LLMs) locally on personal computers and servers. Often described as "Docker for LLMs," it lets a user pull and chat with a model in a single command (for example ollama run llama3.2) without cloud services, API keys, or deep machine-learning expertise. Built primarily in Go and powered by llama.cpp under the hood, Ollama is one of the most widely adopted local-inference platforms in the AI ecosystem, with more than 175,000 stars and over 16,700 forks on GitHub as of June 2026, making its repository one of the most-starred AI projects on the platform.^[1][2] It is released under the MIT license, and the current stable line is the v0.30.x series (v0.30.10 was published on June 17, 2026).^[2]

History

When was Ollama released and who created it?

The company behind Ollama was founded in 2021 by Jeffrey Morgan and Michael Chiang in Palo Alto, California, and the tool itself had its first public release in 2023.^[3] The company participated in Y Combinator's Winter 2021 batch^[4] and raised $125,000 in pre-seed funding from investors including Y Combinator, Essence Venture Capital, Rogue Capital, and Sunflower Capital.^[5]

Prior to founding Ollama, Morgan and Chiang, along with Sean Li, created Kitematic, a tool designed to simplify Docker container management on macOS, which was eventually acquired by Docker, Inc.^[3] Jeffrey Morgan and Sean Li graduated from the University of Waterloo (BASc 2013, Software Engineering), while Michael Chiang was an electrical engineering student there at the time of Kitematic's acquisition. This experience in making complex command-line tools accessible through simpler interfaces directly influenced Ollama's design philosophy.

The platform quickly gained traction in the open-source AI community for its ease of use and Docker-like simplicity in managing LLMs. Initial releases focused on core functionality for running models like LLaMA 2, with subsequent updates introducing features such as multimodal support and tool calling.

Key Milestones

Scroll sideways for more →

Development is rapid: the project moved from the v0.18.x line in March 2026 to the v0.30.x line by June 2026, and the GitHub repository's tagline by mid-2026 advertised support for "Kimi-K2.6, GLM-5.1, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models."^[2]

Funding and Revenue

Ollama's only publicly confirmed funding round is the $125,000 pre-seed raised during Y Combinator's W21 batch.^[5] According to third-party estimates, Ollama generated approximately $3.2 million in revenue in 2024 with a team of roughly 21 people, primarily through the Ollama Cloud subscription tiers introduced in September 2025.^[3] The company reportedly received an M&A offer in April 2025, though no details about the acquiring party or terms have been publicly disclosed.^[3]

Architecture and Technical Implementation

How does Ollama work under the hood?

Ollama is built primarily in Go and leverages llama.cpp as its underlying inference engine through CGo bindings.^[2] The llama.cpp project, created by Georgi Gerganov in March 2023, provides an efficient C++ implementation of LLaMA and other language models, enabling them to run on consumer-grade hardware. Because Ollama wraps llama.cpp, it inherits support for a wide range of model architectures and quantization formats while presenting a much simpler interface to the end user.

When a user runs a model, Ollama handles downloading the model weights from its registry, loading them into memory with appropriate quantization, allocating GPU or CPU resources, and exposing a local HTTP server for interaction. The server runs on 127.0.0.1:11434 by default and supports both streaming and non-streaming responses.

Model Format

Ollama primarily uses the GGUF (GPT-Generated Unified Format) file format for storing and loading models. GGUF replaced the earlier GGML format and provides better compatibility, metadata handling, and performance optimization for quantized models. This quantization is what allows massive models (for example 70 billion parameters) to run on machines with limited VRAM.

Ollama can also import models from specific Safetensors directories for supported architectures (for example Llama, Mistral, Gemma, Phi).

Quantization

Quantization is central to Ollama's ability to run large models on consumer hardware. By reducing the precision of model weights from 16-bit floating point to lower bit representations (such as 4-bit or 8-bit integers), quantization dramatically reduces both memory usage and computation time. Ollama supports multiple quantization levels through GGUF:

Scroll sideways for more →

Ollama also added experimental support for NVFP4 and FP8 quantization in late 2025, leveraging NVIDIA hardware for faster token generation at lower precision.^[28]

Installation

System Requirements

Scroll sideways for more →

Scroll sideways for more →

Quick Start

After installation, getting started with Ollama takes a single command:

ollama run llama3.2

This command downloads the model (if not already present) and starts an interactive chat session. The Ollama server launches automatically in the background when any command is run, or it can be started explicitly with ollama serve.

GPU Support

Ollama provides hardware acceleration across multiple GPU vendors:

Scroll sideways for more →

Ollama automatically detects available GPUs and allocates model layers accordingly. For models that exceed GPU VRAM, Ollama splits inference between GPU and CPU, loading as many layers as possible onto the GPU.^[19]

Features

What is Ollama used for?

Ollama is used to run open-weight LLMs entirely on a user's own hardware for chat, coding assistance, retrieval-augmented generation, agentic tool use, multimodal (vision) tasks, and, since 2026, experimental local image generation. Because all processing can occur on the loopback interface by default, it is widely adopted for privacy-sensitive workloads where prompts and data must never leave the machine.

Core Capabilities

Scroll sideways for more →

Command-Line Interface

Scroll sideways for more →

Modelfile

A key component of Ollama is the Modelfile, which serves as a blueprint for creating and sharing models. Similar to a Dockerfile, the Modelfile defines model behavior and configuration.

Scroll sideways for more →

Example Modelfile

# Specify the base model
FROM llama3.2

# Set model parameters
PARAMETER temperature 0.8
PARAMETER num_ctx 4096
PARAMETER stop </s>

# Set the system message
SYSTEM """
You are an expert Python programming assistant.
Always provide clear, concise code examples.
Your responses must be formatted in Markdown.
"""

# Define the chat template
TEMPLATE """
<|start_header_id|>system<|end_header_id|>
{{ .System }}<|eot_id|>
<|start_header_id|>user<|end_header_id|>
{{ .[Prompt](/wiki/prompt) }}<|eot_id|>
<|start_header_id|>assistant<|end_header_id|>
"""

This custom model can be created with: ollama create my-assistant -f ./Modelfile

REST API

Ollama exposes a REST API on port 11434 by default, providing programmatic access to model functionality:

Scroll sideways for more →

OpenAI-Compatible API

Since February 2024, Ollama has provided an OpenAI-compatible API endpoint at /v1/chat/completions.^[6] This allows developers to use Ollama as a drop-in replacement for OpenAI in applications that rely on the OpenAI client library. By simply changing the base URL to http://localhost:11434/v1, existing code written for the OpenAI API works with local Ollama models. This compatibility extends to features such as streaming, tool calling, and structured outputs.^[6]

Starting with v0.14.0 (released in early 2026), Ollama also added compatibility with the Anthropic Messages API, enabling tools like Claude Code to work with local open-source models through Ollama.^[18]

Supported Models

Ollama supports a wide range of open-source language models through its model library at ollama.com/library. The registry hosts over 100 model families with various parameter sizes and quantizations, and new models are added regularly as they are released.^[25]

Popular Models

Scroll sideways for more →

The newest model families surfaced in the GitHub repository's tagline as of mid-2026 include Kimi-K2.6, GLM-5.1, and MiniMax, alongside long-standing families such as DeepSeek, gpt-oss, Qwen, and Gemma.^[2]

Vision Model Support

Ollama supports multimodal vision models that can process both text and images. Users can pass images to supported models through the API or by dragging and dropping files in the desktop app. Vision models available through Ollama include:

• Llama 3.2 Vision (11B and 90B): Meta's multimodal models optimized for visual recognition, image reasoning, captioning, and answering questions about images. The architecture adds a vision adapter (image encoder plus cross-attention layers) on top of the Llama 3.1 text transformer.^[24]
• LLaVA 1.6: An end-to-end trained multimodal model combining a vision encoder with Vicuna for general-purpose visual and language understanding.
• Gemma 3: Google DeepMind's models with built-in image understanding capabilities.
• Qwen3-VL: Alibaba's vision-language model with support for image and video inputs.

Vision models can be used with the same ollama run command by providing an image path:

ollama run llama3.2-vision "Describe this image: ./photo.jpg"

Embedding Model Support

Ollama supports dedicated embedding models for tasks such as retrieval-augmented generation (RAG), semantic search, and text classification. Embeddings are generated through the /api/embeddings endpoint or the client libraries.^[23]

Scroll sideways for more →

Tool Calling and Function Calling

Ollama supports tool calling (also known as function calling), which allows models to invoke external functions and incorporate the results into their responses.^[8] This feature is available through the /api/chat endpoint by specifying a tools parameter containing a list of available functions with their descriptions and parameter schemas.^[20]

Key aspects of Ollama's tool calling support:

• Streaming with tool calls: A parser built into Ollama understands the structure of tool calls, enabling real-time streaming of both text and function invocations without waiting for the entire response to complete.^[9]
• Parallel tool calls: Models can request multiple tool calls in a single response. All tool responses can then be sent back to the model together.
• Supported models: Models with strong tool calling performance include Llama 3.1 and later, Qwen 2.5 and Qwen 3, Mistral and Mixtral variants, GLM-5, and specialized models like FunctionGemma.

Tool calling enables use cases such as data retrieval, calculations, API integration, and agentic workflows where the model plans and executes multi-step tasks.

Structured Outputs

Since version 0.5, Ollama supports structured outputs that constrain a model's response to conform to a specific JSON schema.^[10] By passing a JSON schema to the format parameter, Ollama generates a grammar that forces the output to match the defined structure. This is useful for extracting typed data from model responses, building reliable data pipelines, and integrating LLM outputs with downstream systems.

Best practices for structured outputs include lowering the temperature (for example, setting it to 0) for deterministic completions, defining schemas with Pydantic (Python) or Zod (JavaScript), and including the schema in the system prompt to ground the model's response.^[21]

Thinking Mode

Ollama provides the ability to enable or disable "thinking" for reasoning models such as DeepSeek R1 and Qwen 3. When thinking mode is active, the model generates an explicit chain-of-thought reasoning trace before producing its final answer. This improves accuracy on complex tasks like mathematics, logic puzzles, and multi-step planning, while also providing transparency into the model's reasoning process. Thinking can be toggled on or off through the API, and the thinking level can be controlled in v0.17.7 and later.^[29]

Web Search

Announced on September 24, 2025, Ollama's web search feature provides a REST API that augments models with live information from the web.^[13] The API includes two endpoints:

• Web Search (/api/web_search): Returns search results with titles, URLs, and content snippets for a given query.
• Web Fetch (/api/web_fetch): Retrieves the full content of a specific web page.

Both endpoints require Bearer token authentication. Ollama provides a generous free tier of web searches for individuals, with higher rate limits available through Ollama Cloud subscriptions. The web search and fetch functions also integrate as tools that instruction-following models can call autonomously via function calling.^[22]

Image Generation (Experimental)

In January 2026, Ollama introduced experimental support for local text-to-image generation.^[15] The initial release supports two models:

• Z-Image Turbo (6B parameters): A bilingual (English/Chinese) model from Alibaba's Tongyi Lab that generates photorealistic images.
• FLUX.2 Klein (4B and 9B variants): A fast image-generation model from Black Forest Labs with strong text-rendering capabilities.

Image generation is currently available on macOS, with Windows and Linux support planned for future releases. Users can configure generation parameters including width, height, step count, random seeds, and negative prompts.

Ollama Model Library and Registry

The Ollama model library at ollama.com/library serves as a centralized registry for discovering and downloading models. It functions similarly to Docker Hub, allowing users to browse, search, and pull models with a single command. Each model page shows available tags (representing different parameter sizes and quantization levels), file sizes, and documentation.

Users can also push custom models to the registry after creating them with a Modelfile, making it possible to share fine-tuned or customized models with the community. The registry supports versioning through tags, so users can pin specific model versions for reproducibility.

As of early 2026, the library hosts over 100 model families spanning categories such as general-purpose chat, code generation, vision, embedding, tools/function calling, and image generation.^[25]

Desktop Application

Ollama v0.10.0, released on July 30, 2025, introduced a native desktop application for macOS and Windows.^[11] Previously, Ollama operated exclusively through the command line and API. The desktop app brings a graphical chat interface that lowers the barrier to entry for non-technical users; Ollama described it simply as "An easier way to chat with models."^[11]

Key features of the desktop app include:

• Chat interface: A polished conversation UI for interacting with any installed model.
• File drag-and-drop: Users can drag PDFs, images, and code files directly into the chat window for multimodal processing.
• Context length controls: A slider allowing precise control over how much context the model retains, up to 128K tokens for supported models.
• Model management: Browse, download, and switch between models from within the GUI.
• Thinking mode toggle: Enable or disable chain-of-thought reasoning for supported models.

The desktop app runs the same Ollama server under the hood, so the CLI, API, and third-party integrations continue to work alongside the graphical interface.

Ollama Cloud

Announced on September 19, 2025, Ollama Cloud extends the platform beyond local hardware by allowing users to run larger models on datacenter-grade GPUs while maintaining the same tools and workflows.^[12] Cloud models appear alongside local models in the Ollama interface and work through the same API endpoints, including the OpenAI-compatible API.

Ollama Cloud is designed for cases where a model is too large to fit in local memory (for example, 70B+ parameter models that require 48GB or more of VRAM). The cloud infrastructure uses high-memory GPUs with fast interconnects optimized for LLM inference.

Privacy

Ollama states that its cloud service does not retain user data, maintaining the platform's privacy-first principles even when offloading to remote hardware.

Pricing

How much does Ollama cost?

Ollama itself is free and open source under the MIT license, and running models locally has no usage cost. Ollama Cloud launched with fixed-price subscription tiers, billed against the platform's GPU infrastructure rather than per token:^[12]

Scroll sideways for more →

Coding Tools Integration

In January 2026, Ollama introduced the ollama launch command (v0.15), which provides zero-configuration setup for popular AI coding tools.^[16] Rather than manually setting environment variables and API endpoints, users can run a single command to connect coding agents to local or cloud models.

Supported coding tools include:

• Claude Code: Anthropic's CLI coding agent, connected via Ollama's Anthropic-compatible API.
• OpenAI Codex: OpenAI's code execution tool.
• OpenCode: Open-source coding assistant.
• Droid: Android development coding agent.

Usage examples:

# Interactive picker for all supported tools
ollama launch

# Launch Claude Code directly
ollama launch claude

# Launch with a specific model
ollama launch claude --model qwen3-coder

Popular local models for coding tasks include GLM-4.7-flash, qwen3-coder, and gpt-oss:20b, which require around 23GB of VRAM when running with the recommended 64,000-token context length.

OpenClaw Integration

In February 2026, Ollama announced integration with OpenClaw, an open-source personal AI assistant framework that gained over 113,000 GitHub stars within days of its January 2026 launch.^[17] OpenClaw bridges messaging platforms (WhatsApp, Telegram, Slack, Discord, iMessage) to AI agents, allowing users to interact with their local models from any chat application.

Ollama's integration provides a streamlined setup command:

ollama launch openclaw

This automatically configures the connection between OpenClaw and the user's local Ollama models, enabling tasks such as email management, calendar scheduling, and general assistance through familiar messaging interfaces.

Secure Minions

In June 2025, Ollama partnered with Stanford's Hazy Research lab to introduce Secure Minions, a protocol for private collaboration between local and cloud models.^[14] The protocol allows a small local model (such as Gemma 3 4B running on Ollama) to work together with a larger cloud model (such as GPT-4o) while keeping all raw data encrypted end-to-end.

In the Secure Minions protocol, the raw context stays on the local device and can only be accessed by the local LLM. The cloud model orchestrates the local models and aggregates their outputs, but never sees plaintext data. As Ollama describes it, "The raw context stays local and can only be accessed by the local LLM."^[14] Messages are encrypted before being sent to the cloud and decrypted only inside an NVIDIA Hopper H100 GPU running in confidential-computing (secure enclave) mode, with remote attestation used to verify the GPU's secure state before any encrypted message is processed; according to Ollama, "No plaintext is exposed during transmission or remote LLM inference."^[14]

According to the underlying research (the Minions work, presented around ICML 2025), this approach reduces cloud costs by 5x to 30x while achieving 98% of the accuracy of using the frontier model directly; the paper's MinionS protocol specifically reports a 5.7x cost reduction at 97.9% of frontier accuracy.^[31] The latency overhead is minimal: less than 1% even with long prompts and large local models.^[14]

Privacy and Security

Is Ollama private and offline by default?

Yes. By default, Ollama operates entirely locally:

• Server binds to 127.0.0.1:11434 (loopback interface only)
• No prompts or responses sent to external servers
• Complete data privacy for sensitive information
• Offline operation after model download

To expose on a network, users must explicitly set the OLLAMA_HOST environment variable (for example OLLAMA_HOST=0.0.0.0:11434).

Security Vulnerabilities

Ollama has addressed several security vulnerabilities:

Scroll sideways for more →

Users are advised to keep Ollama updated and configure the server securely, especially when exposing the API to a network.

Integration and Ecosystem

Programming Languages

Ollama provides official client libraries:

Scroll sideways for more →

Third-Party Integrations

Ollama's ecosystem has grown substantially, with integrations spanning user interfaces, development frameworks, databases, and more.

User Interfaces

Scroll sideways for more →

Development Frameworks

Scroll sideways for more →

Database and Infrastructure

• PostgreSQL with pgai extension for AI-powered queries
• ChromaDB and other vector databases for RAG pipelines
• IoT device integrations for edge AI deployments

Ollama for Enterprise and Teams

As of early 2026, Ollama is developing team and enterprise features, though formal plans have not yet been publicly released. According to Ollama's website, team and enterprise plans are "coming soon," and interested organizations can contact hello@ollama.com for details.

Features reported to be under development include:

• Team collaboration with shared model workspaces
• Usage analytics and monitoring
• Centralized team model management
• SSO (single sign-on) integration
• Professional support and SLAs

Ollama Cloud's hybrid approach (local plus cloud inference) positions the platform for enterprise adoption, particularly for organizations that need to process sensitive data locally while offloading larger workloads to private cloud infrastructure. For organizations requiring enterprise-grade autoscaling, multi-tenant throughput, and GPU pooling, alternatives such as vLLM or managed inference services may be more appropriate until Ollama's enterprise features mature.

Comparisons with Similar Tools

Ollama competes in the growing market for local LLM runners and inference tools. Each tool has distinct strengths depending on the user's needs.

Scroll sideways for more →

Ollama vs. LM Studio

Ollama and LM Studio are the two most popular tools for running LLMs locally. Ollama excels at automation, scripting, and integration through its API and CLI, while LM Studio provides a more polished graphical interface. LM Studio also supports Apple's MLX framework for optimized performance on Apple Silicon, which Ollama does not currently support. Ollama's API runs as a system service (always available in the background), whereas LM Studio requires manually starting its server.

How does Ollama differ from llama.cpp?

Since Ollama is built on top of llama.cpp, it inherits the same model support and inference performance. The key difference is that Ollama adds model management (pulling, pushing, listing, creating), a REST API, and a simpler user experience. Advanced users who need direct control over inference parameters or want to avoid the overhead of Ollama's server may prefer using llama.cpp directly.

Community and Reception

Ollama has one of the largest open-source AI communities. As of June 2026, the main repository had surpassed 175,000 GitHub stars and 16,700 forks, up from roughly 166,000 stars and 15,000 forks in March 2026.^[2] The project maintains active development with thousands of commits and frequent releases (it advanced from the v0.18.x line to the v0.30.x line over the first half of 2026). The companion libraries (ollama-python and ollama-js) are also widely used, with nearly 1,000 forks for the Python library alone.^[2]

The project has been praised for advancing local AI accessibility, reducing costs for developers and researchers, and enabling privacy-preserving AI workflows. It is frequently cited as the easiest way to get started with open-source LLMs.

Licensing Controversy

Some criticism has arisen regarding licensing compliance issues with dependencies like llama.cpp, with community members raising concerns about proper attribution. The Ollama team has been working to address these concerns.

Significance and Impact

Ollama has been a key driver in the democratization of large language models by:

• Enabling developers to build and test AI-powered applications locally without cost
• Allowing researchers to experiment with various open-source models easily
• Empowering hobbyists to run state-of-the-art AI on personal computers
• Enhancing privacy for users who can leverage powerful AI without data leaving their machine
• Fostering a community-driven approach to AI development
• Providing an on-ramp for enterprises exploring private AI deployments

The tool is widely used in education, research, and enterprise for privacy-sensitive applications and has become a foundational tool in the open-source AI movement. Its Docker-like experience for AI models has set the standard that competitors measure themselves against, and its growing feature set (cloud models, web search, image generation, coding tool integration) signals a trajectory toward becoming a comprehensive local AI platform.

See Also

• Large language model
• llama.cpp
• GGUF
• LangChain
• LlamaIndex
• Hugging Face
• Docker
• LM Studio
• Mixture of Experts
• Retrieval-augmented generation
• Embeddings

References

1. Ollama Official Website ↩
2. Ollama GitHub Repository ↩
3. Ollama on Crunchbase ↩
4. Ollama on Y Combinator ↩
5. Ollama Pre-Seed Funding Round (Crunchbase) ↩
6. Ollama Blog: OpenAI Compatibility ↩
7. Ollama Blog: Windows Preview
8. Ollama Blog: Tool Support ↩
9. Ollama Blog: Streaming Tool Calls ↩
10. Ollama Blog: Structured Outputs ↩
11. Ollama Blog: New Desktop App ↩
12. Ollama Blog: Cloud Models ↩
13. Ollama Blog: Web Search ↩
14. Ollama Blog: Secure Minions ↩
15. Ollama Blog: Image Generation ↩
16. Ollama Blog: ollama launch ↩
17. Ollama Blog: OpenClaw Integration ↩
18. Ollama Blog: Claude Code Integration ↩
19. Ollama Docs: Hardware Support ↩
20. Ollama Docs: Tool Calling ↩
21. Ollama Docs: Structured Outputs ↩
22. Ollama Docs: Web Search ↩
23. Ollama Blog: Embedding Models ↩
24. Ollama Blog: Llama 3.2 Vision ↩
25. Ollama Model Library ↩
26. Open WebUI GitHub Repository
27. LangChain Ollama Integration
28. Infralovers: Ollama in 2025 Major Updates ↩
29. GitHub Releases: ollama/ollama ↩
30. Phoronix: ollama Experimental Vulkan Support
31. Minions: Cost-efficient Collaboration between On-device and Cloud Language Models (OpenReview, ICML 2025) ↩

External Links

• Official Website
• Ollama on GitHub
• Ollama Docker Hub
• Ollama integration with Hugging Face
• Official Documentation