LocalAI (software)
LocalAI is an open-source server for running AI models on hardware the operator controls. It is written mainly in Go and published under the MIT License.[1][2][3] The server exposes an HTTP API in the request formats of the OpenAI API and the Anthropic API, so a program written for those services can be pointed at a LocalAI address by changing its base URL, and it runs text, vision, speech, image, video, music, embedding and reranking models through separate inference programs that the project calls backends.[1][4] Ettore Di Giacinto, who publishes on GitHub as mudler, created the project in March 2023, and its repository is mudler/LocalAI.[1][3][5]
The project is one implementation of local AI, the general practice of running models on one's own machines, and should not be confused with that term. The models it loads are open-weight models fetched from the project's gallery, from Hugging Face or from container registries. The MIT licence covers LocalAI's own code and not those models: the documentation warns that a model may carry a licence that does not permit commercial use, or no licence at all, and tells users to check it.[1][6]
As of November 2026 the current release is 5.0.0, published on 18 November 2026, which adds server-side chats, saved agents, a files area, add-ons and a desktop application on top of the API.[7] The documentation counts more than 80 backends.[8] On 19 November 2026 the GitHub repository had 50,438 stars and 4,617 forks.[3]
History
llama-cli (March and April 2023)
The repository was created on 18 March 2023.[3][5] At its first tag, v0.1 of 20 March 2023, the program was called llama-cli, and its README described "a straightforward golang CLI interface" for llama.cpp with a simple API; the container image shipped with a model from the alpaca.cpp project (see Alpaca).[9] By tag v0.8 of 12 April 2023 the README described a command-line interface and an API "compatible with OpenAI" that could serve several models.[10]
The project was renamed on 19 April 2023, in a commit titled "Rename project to LocalAI" that carries the tag v1.0.[90] The README at that tag announced the change: "This project has been renamed from llama-cli to LocalAI to reflect the fact that we are focusing on a fast drop-in OpenAI API rather on the CLI interface." It described LocalAI as a replacement API for CPU inference based on llama.cpp, GPT4All and ggml.[11]
On 29 April 2023 Spectro Cloud, a company that sells Kubernetes management software, published a tutorial that used LocalAI as the in-cluster model server for the K8sGPT diagnostic tool. Its author called LocalAI "the brainchild of Ettore Di Giacinto (AKA mudler), creator of Kairos".[12] Spectro Cloud's site lists Di Giacinto as its head of open source and says he was previously a staff software engineer at SUSE and lead engineer of the Rancher Elemental project.[13] The company's logo, linked to its site, appears in the sponsors section of the README.[1]
Versions 1 and 2 (2023 to 2025)
The repository first lived in the go-skynet GitHub organisation. Release notes up to v1.30.0 (September 2023) link to go-skynet/LocalAI, those from v1.40.0 (October 2023) link to mudler/LocalAI, and requests for the old address now resolve to the new one.[14][15][16] The Helm chart for Kubernetes is still published from the go-skynet/helm-charts repository.[17]
Two changes of mid-2023 set the later design. Version 1.20.0 (June 2023) added gallery repositories from which models could be listed and installed, and version 1.21.0 (July 2023) converted the inference engines into backends reached over gRPC.[18][89][19] Version 2.0.0, in December 2023, added LLaVA support with an OpenAI-style vision API, parallel requests for llama.cpp models and an initial watchdog.[20]
Di Giacinto's 2026 history of the project says the first two years were spent widening what sat behind the API (transcription with whisper.cpp, image generation, embeddings, reranking, constrained grammars and function calling) while keeping a rule that every kind of model must have a CPU path. He also describes the cost: every engine was compiled into one binary, so users downloaded GPU libraries and a Python runtime whether or not they needed them.[5]
Distributed inference arrived in 2024. Version 2.16.0 (May 2024) added llama.cpp workers that share one model's computation and a peer-to-peer mode built on libp2p, and version 2.19.0 (July 2024) added federated instances on the same peer-to-peer layer.[21][23] Version 2.17.0 added the install.sh installer and a public model listing at models.localai.io.[22]
Version 3: backends leave the core
Version 3.0.0 (June 2025) introduced a Backend Gallery, in which backends are OCI container images that can be installed and removed at run time, and ended the separate "extras" images that had bundled the Python backends.[24] Version 3.2.0, a month later, moved every backend out of the main binary. The release notes describe the result as a core "separated from all the backends" with backends downloaded automatically to match the detected hardware.[25] Di Giacinto later wrote that everything after it depended on that change, because adding an engine no longer added weight to every installation.[5]
Later 3.x releases added a launcher application for Linux and macOS, labelled alpha (3.5.0), agent-style tool use through the Model Context Protocol (3.7.0), and endpoints in the Anthropic Messages and Open Responses formats (3.10.0).[26][27][28]
Version 4: agents, clusters and recognition
Version 4.0.0 (March 2026) put agent management into the core, opened a community site for sharing agents called Agenthub, rewrote the web interface in React, added WebRTC transport to the realtime audio API and added an experimental distributed backend for Apple's MLX framework. It also removed the Hugging Face backend and the all-in-one container images.[29]
Version 4.1.0 (April 2026) added a distributed cluster mode with routing by available video memory, user accounts with OIDC login and per-user quotas, experimental fine-tuning from the interface and an experimental quantization backend.[30] Version 4.2.0 (May 2026) added voice recognition, face recognition with liveness checks, speaker diarization, video generation and an API compatible with Ollama clients.[31] By mid-2026 the team was also writing its own C and C++ inference engines; the README lists 18 of them.[1][5]
Three releases of the second half of 2026 made the security changes described under Security below: version 4.6.0 (July 2026) added a check on gallery configuration URLs, version 4.9.0 (August 2026) made authentication deny-by-default, and version 4.12.1 (October 2026) was a security-only patch.[91][32][33]
Version 5
Version 5.0.0 was published on 18 November 2026.[7] Its notes list a desktop application built on Electron for macOS on Apple silicon and for Linux, which replaces the earlier launcher; Projects, which group chats around a folder; chats that run on the server and continue when the browser tab is closed; a Files page with undoable history; an Office add-on for Word, Excel, PowerPoint and PDF files; a Sandbox add-on for running Python and shell commands; accounts, roles and sharing for teams; and a library of add-ons.[7][34] The release replaced the previous chat page and agent runtime. The old /api/agents routes now answer 404, and the Studio and Talk pages became add-ons that an administrator installs.[7] The notes count 130 merged pull requests and 1,432 commits since 4.12.1, and a model library that grew from 2,212 to 2,325 entries.[7]
Selected releases
| Version | Date | Notable changes |
|---|---|---|
| v0.1 (tag) | 2023-03-20 | First tag, as llama-cli [9] |
| v1.0 (tag) | 2023-04-19 | Renamed LocalAI [11][90] |
| v1.20.0 | 2023-06-28 | Model gallery repository [18] |
| v1.21.0 | 2023-07-16 | gRPC-based backends [19] |
| v2.0.0 | 2023-12-04 | LLaVA and vision API, parallel requests [20] |
| v2.16.0 | 2024-05-24 | Distributed llama.cpp workers, peer-to-peer mode [21] |
| v2.19.0 | 2024-07-19 | Federated instances [23] |
| v3.0.0 | 2025-06-19 | Backend Gallery of OCI images [24] |
| v3.2.0 | 2025-07-24 | All backends moved out of the core binary [25] |
| v3.7.0 | 2025-10-31 | Model Context Protocol tool use [27] |
| v3.10.0 | 2026-01-18 | Anthropic Messages and Open Responses endpoints [28] |
| v4.0.0 | 2026-03-14 | Agents in the core, React interface [29] |
| v4.1.0 | 2026-04-02 | Distributed cluster mode, accounts and quotas [30] |
| v4.2.0 | 2026-05-11 | Voice and face recognition, Ollama-compatible API [31] |
| v4.6.0 | 2026-07-04 | Address check on gallery configuration URLs [91] |
| v4.9.0 | 2026-08-20 | Deny-by-default authentication [32] |
| v4.12.1 | 2026-10-23 | Security fix for user-supplied URLs [33] |
| v5.0.0 | 2026-11-18 | Desktop app, Projects, Files, add-ons [7] |
Architecture
Core and backends
LocalAI is split into a core and backends. The core is a Go program that serves the API and the web interface. Each model runs in a backend, a gRPC service wrapped around an inference engine such as llama.cpp, vLLM, whisper.cpp, stable-diffusion.cpp or MLX. Backends are distributed as OCI images and run as separate processes; the core downloads one the first time a model needs it, and a backend can be installed, upgraded or removed without touching the core, or run on another machine.[35][4] When a request arrives, the model's configuration selects a backend, the backend process runs the model over gRPC, and tokens stream back to the client.[4]
Backends are written in C++, Go and Python, and because the contract is a small gRPC interface the documentation says a backend can be written in any language or supplied as an external gRPC server.[4][35] The FAQ gives the reason for the split: each backend is built for one engine, and a user installs only the backends their models use.[36]
The compatibility table groups the backends by task. Examples as of November 2026:[37]
| Task | Example backends |
|---|---|
| Text generation | llama.cpp, ik_llama.cpp, vLLM, vllm.cpp, SGLang, Transformers, MLX, tinygrad |
| Speech to text | whisper.cpp, faster-whisper, WhisperX, parakeet.cpp, NeMo |
| Text to speech | piper, Kokoro, Chatterbox, VibeVoice, fish-speech, Qwen3-TTS |
| Music and sound | ACE-Step, acestep.cpp, audio.cpp |
| Images and video | stable-diffusion.cpp, Diffusers, mlx-video, vLLM Omni |
| Detection and recognition | RF-DETR, sam3.cpp, depth-anything.cpp, face-detect.cpp |
Most backends wrap an engine maintained elsewhere. The README lists 18 that the LocalAI team wrote, in C, C++ and GGML with no Python at inference time. They include vllm.cpp, described as a from-scratch C++20 port of vLLM with a paged KV cache and continuous batching, parakeet.cpp for NVIDIA's Parakeet speech recognition models, and face-detect.cpp and voice-detect.cpp, which replaced earlier Python backends.[1] The history post gives three motives for these ports: replacing Python dependencies too heavy to ship, porting models that had no C++ implementation, and filling gaps such as acoustic echo cancellation.[5]
Hardware
The README lists acceleration for NVIDIA GPUs (CUDA 12 and 13), AMD (ROCm), Intel (oneAPI and SYCL), Apple silicon (Metal), Vulkan and NVIDIA Jetson boards, and CPU-only operation.[1] When a model is installed, LocalAI detects the hardware and downloads the matching build of the backend.[38] Not every backend covers every target: in the compatibility table vLLM, for example, is listed for CUDA, ROCm, Intel SYCL and Jetson but not for CPU.[37] The project's standing claim is that no GPU is needed, with the qualification that a GPU makes inference faster; the quickstart notes that on a CPU the first reply takes longer because the model has to load.[1][39]
By default a model stays loaded in memory after first use, which can leave too little video memory for the next model. The documentation offers a cap on the number of loaded models with least-recently-used eviction, per-model concurrency groups and watchdog timeouts as remedies.[40]
Models and the gallery
The model gallery is a curated set of model configurations that can be installed from the Models page of the web interface, from the command line or through the API. The documentation states that the models in it are not maintained by LocalAI.[6] A model can also be named by address. The README gives four forms:[1]
# From the model gallery (list it with `local-ai models list` or on the Models page of the web UI)
local-ai run gemma-4-e2b-it-qat-q4_0
# From Hugging Face
local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
# From the Ollama OCI registry
local-ai run ollama://gemma:2b
# From a standard OCI registry (for example Docker Hub)
local-ai run oci://localai/phi-2:latest
A model's settings, including the backend it uses, are kept in a YAML configuration file.[37][38] The FAQ says most GGUF models work and cautions against downloading models from untrusted sources, because a flaw in llama.cpp or ggml could be exploited by a malicious file.[36]
With 5.0, release builds read models from a model library, a default source named official-models, and the older gallery index became a generated copy kept for earlier releases.[7][6] The documentation says the shipped library has no signature policy of its own, although the SHA-256 checksum of each model file is still checked on installation.[6]
API
The server listens on port 8080 by default and publishes an interactive reference at /swagger/index.html.[39][42] The main documented routes as of November 2026:
| Area | Routes | Notes |
|---|---|---|
| Chat and completion | /v1/chat/completions, /v1/completions | OpenAI format [41] |
| Messages | /v1/messages | Anthropic Messages API, with tools and streaming [41] |
| Responses | /v1/responses | Open Responses specification [41] |
| Embeddings | /v1/embeddings | OpenAI format [43] |
| Reranking | /v1/rerank | Scores documents against a query [44] |
| Images | /v1/images/generations | OpenAI format [45] |
| Video | /video | Text prompts with optional image or audio conditioning [46] |
| Text to speech | /v1/audio/speech, /tts | Compatible with the OpenAI and ElevenLabs APIs [47] |
| Music and sound | /v1/sound-generation | Compatible with the ElevenLabs sound generation API [47] |
| Transcription and diarization | /v1/audio/transcriptions, /v1/audio/diarization | whisper.cpp is the default transcription backend [48] |
| Voice activity detection | /v1/vad | Silero VAD backend [49] |
| Realtime voice | Realtime API over WebSocket or WebRTC | Follows the OpenAI Realtime API [50] |
| Object detection | /v1/detection | Detection and image segmentation [51] |
| Moderation | /v1/moderations | OpenAI-compatible [52] |
| Vector store | /stores/set, /stores/get, /stores/find, /stores/delete | Experimental similarity search [53] |
| Ollama clients | /api/chat, /api/generate, /api/embed, /api/tags, /api/ps | Ollama's API [54][55] |
The text routes support function calling through the OpenAI tools format and output constrained by grammars.[1] The realtime route is not a single model: it is configured as a pipeline of a voice activity detector, a transcription model, a language model and a text-to-speech model.[50] The moderation route uses a local text model with a constrained JSON grammar instead of a dedicated moderation service.[52] For the general pattern of reimplementing OpenAI's interface, see OpenAI-compatible API.
LocalAI works with the Model Context Protocol in both directions. A model's configuration can name remote HTTP or local stdio MCP servers, whose tools are then executed on the server during a chat, and the command local-ai mcp-server exposes LocalAI's own management tools to an MCP host such as Claude Desktop or Cursor.[56][57]
Chats, agents and add-ons
Since 5.0 the web interface is more than a model manager. Chats are stored and run by the server; a chat can use saved agents, tools, MCP servers, files and skills, start sub-agents, and work toward a goal over many turns within configurable limits (20 turns and 400 steps by default).[7] Knowledge collections give agents search over uploaded documents: LocalAI splits each document into chunks, stores their embeddings in a vector engine and returns the closest chunks for a query, a form of retrieval-augmented generation. The default engine searches by meaning only, and a PostgreSQL engine adds keyword search.[71]
Add-ons are separate programs installed by an administrator from a library. The README and the 5.0 notes name Office, Sandbox, Tables, chat applications for Slack and Discord, mail, calendar, Google Drive, GitHub and Home Assistant.[1][7]
The documentation states several limits next to these features. A project organises chats but is not an access boundary: a chat outside the project can still read its files from the user's home folder.[7] The Sandbox add-on runs only on Linux hosts that can use KVM or gVisor, has no network access by default, cuts an answer at 64 KiB and stops a command after 120 seconds by default.[58] Without the Sandbox, a chat can run commands only if an administrator allows it, and those commands then run on the server with no isolation.[58][59] The desktop application and the release binaries are built for Linux and for macOS on Apple silicon; for Windows and Intel Macs the documentation points to a container.[7][72]
Distributed and peer-to-peer inference
The documentation describes three ways to spread work over several machines, plus manually addressed llama.cpp workers.[61]
| Mode | Coordination | What is spread | Documented status and limits |
|---|---|---|---|
| P2P federated | libp2p network joined with a shared token, no central server | Whole requests; each goes to one node, which loads the full model | Described as experimental, "tech preview quality" [61] |
| P2P worker mode | Same network | One model's weights, split across workers in proportion to their memory | llama.cpp-compatible models only; one model; workers cannot join once inference has begun [61] |
| Distributed mode | PostgreSQL for state, NATS for messaging | Models placed on worker nodes by a router; one backend process per model | Needs authentication backed by PostgreSQL; SQLite is not supported [60] |
| MLX distributed | P2P discovery or a host file | One model's layers across Apple silicon Macs | Marked experimental; no container image, because containers cannot use Metal [62] |
In the peer-to-peer modes, starting a server with local-ai run --p2p generates a token, and any instance holding the token joins the same private network. The implementation uses go-libp2p and the author's EdgeVPN library, with discovery over a distributed hash table or, for local networks only, mDNS.[61] A federated entry point is started with local-ai federated; by default it assigns each incoming connection to the online node with the fewest active connections, and the documentation notes that these counts are kept per entry point and do not measure total cluster load or GPU use.[61] The project runs a public directory of shared swarms at explorer.localai.io. Listing a swarm there publishes its token, so anyone who sees the list can use that swarm's workers.[61] Worker mode builds on llama.cpp's own RPC feature and can also be used without the peer-to-peer layer by listing worker addresses in the LLAMACPP_GRPC_SERVERS environment variable.[61][21]
Distributed mode, added in 4.1.0, is the variant the documentation describes as designed for production deployments and Kubernetes. Stateless frontends receive API requests and pass them to a component called the SmartRouter, which prefers a node that already has the model loaded, then nodes with enough free video memory, then idle nodes, and otherwise evicts the least recently used idle model. Workers register themselves and install backends on demand.[60][30] A registration token for workers and authentication of the message bus are off by default, and the documentation tells operators to switch both on in production.[60] In his history post Di Giacinto wrote that distributed mode "is also where most of the painful bugs have lived since", citing fixes in the 4.8.0 cycle.[5]
Companion projects
Two other MIT-licensed repositories by the same author are presented alongside LocalAI; the LocalRecall README calls the group the "Local Stack Family".[64][65][66]
- LocalAGI (repository created in July 2023) is an agent platform. Its README describes agents configured in a web interface, connectors for Discord, Slack, Telegram, GitHub issues and IRC, and an API for each agent that follows OpenAI's Responses format.[63][64]
- LocalRecall (repository created in February 2025) is a REST API for knowledge bases stored in vector databases, with a file-based Chromem engine by default and a PostgreSQL engine for hybrid search.[65][66]
On 19 November 2026 the LocalAGI repository had 2,016 stars and LocalRecall 988.[64][66]
The 3.0.0 release notes of June 2025 announced "LocalAGI rejoining LocalAI alongside LocalRecall" as one stack.[24] The dependency files show how the relationship changed afterwards: LocalAI's Go module file required the LocalAGI module at 4.0.0, when agents entered the core, and still did at 4.12.1; the 5.0.0 file, for the release that replaced that agent runtime, no longer does.[67][68][69][7] The 5.0.0 module still requires LocalRecall, and the migration guide says the collections backend "is built on localrecall".[69][70]
Installation
The documentation gives four routes: an install script, a release binary, a container, and a Helm chart, plus the desktop application from 5.0.[72] The script supports Linux on amd64 and arm64 and macOS on Apple silicon. It downloads the release binary, checks it against the release checksums and does not start the server:[72][39]
curl -sSL https://localai.io/install.sh | sh
local-ai
# open http://localhost:8080 and install a model from there
On Windows, on an Intel Mac, or where a container is preferred, the documented command is:[39]
docker run -p 8080:8080 --name local-ai -ti localai/localai:latest
Separate image tags exist for NVIDIA CUDA 12 and 13, AMD ROCm, Intel GPUs, Vulkan and Jetson boards, and the same images run under Podman.[73][39] Files inside a container are lost when it is removed unless the documented storage paths are mounted.[35] The quickstart's first model is gemma-4-e2b-it-qat-q4_0, a small Gemma 4 build for CPUs that is a 4.3 GB download.[39]
For Kubernetes the documented commands are:[17]
helm repo add go-skynet https://go-skynet.github.io/helm-charts/
helm repo update
helm show values go-skynet/local-ai > values.yaml
helm install local-ai go-skynet/local-ai -f values.yaml
The same page warns that the chart is released separately and its default image can be older than the latest LocalAI release, and that LocalAI listens with authentication off, so the authentication guide should be read before the service is exposed.[17] Broader guidance on running models this way is in How to run LLMs locally and Self-hosting LLMs.
License and governance
The repository's LICENSE file is the MIT License, with a copyright notice for 2023-2025 in the name of Ettore Di Giacinto, and GitHub's licence detection reports the SPDX identifier MIT.[2][3] The MIT License is approved by the Open Source Initiative, so the software is open source in that sense (see open-source license).[74] The licence has been MIT since the early tags; at v1.1.0 the copyright line named the "go-skynet authors".[75]
The name, logo and mascot are excluded. A BRAND-USAGE file in the repository says they are "not part of that license: all rights reserved", permits their unmodified use in articles and talks about LocalAI, forbids their use in the name or logo of another project, and adds that a fork shipped under a different name should use its own name and logo.[76] The file states that nothing in it limits what the MIT License allows for the code.[76]
The README's team section says LocalAI is maintained by a small team together with outside contributors, and names two people: Di Giacinto as original author and project lead and Richard Palethorpe as maintainer. Its sponsors section thanks sponsors who cover continuous-integration costs and shows the logos of Spectro Cloud and solstone; individual sponsorship runs through GitHub Sponsors.[1] The security policy says the project pays no bug bounties.[77]
Adoption
The GitHub API reported 50,438 stars, 4,617 forks and 193 open issues and pull requests for mudler/LocalAI on 19 November 2026.[3] Di Giacinto's history post, dated 29 July 2026, counted 48,042 stars, 224 contributors and 133 releases at that time. Its star history, read from GitHub's stargazer records, passes 4,100 on 22 May 2023, 20,100 on 10 June 2024 and 40,000 on 24 December 2025.[5]
Docker Hub's counter for the localai/localai image repository, registered in January 2024, stood at about 10.8 million pulls (10,805,386) when read on 19 November 2026. The counter records pulls of the image from Docker Hub; it is not a count of users.[78]
Security
LocalAI starts with no authentication unless the operator configures it.[17][79] Two mechanisms exist: shared API keys, set with LOCALAI_API_KEY, and a user system with accounts, roles, sessions, GitHub OAuth, OIDC single sign-on and per-user keys. The documentation notes that the shared keys grant full administrator access with no separation of roles.[79]
Since 4.9.0, once either mechanism is configured every route requires credentials unless it is on an explicit public list (health checks, login and discovery routes, static assets).[79][32] The release notes say the change closed a class of bypass in which unprefixed aliases such as /models, /backends and /moderations fell outside the earlier list of protected prefixes, and credit an outside researcher with the report.[32] Version 4.12.1 hardened the policy for URLs supplied by users, such as image and audio URLs in chat requests, so that redirect targets are validated on every hop.[33] The 4.0.0 release introduced a LOCALAI_DISABLE_MCP setting that switches MCP support off.[29]
The National Vulnerability Database held 13 records for LocalAI on 19 November 2026: 11 filed under the product identifier mudler:localai, and two from 2026 that carry no product identifier and are found by keyword.[101][84][85] In the table, the affected versions are those in the record's description or, where it gives none, in the NVD's product configuration. The score is the CVSS base score in the record: the NVD's own assessment where it has made one (marked NVD), otherwise the score attached to the record by the reporting organisation or another contributor, on CVSS 3.x unless noted.
| CVE | Published | Affected versions | Issue | CVSS base score |
|---|---|---|---|---|
| CVE-2024-3135 | 2024-04-01 | Before 2.17.0 | Cross-site request forgery against a local instance [80] | 6.5 |
| CVE-2024-2029 | 2024-04-10 | Before 2.10.0 | Command injection through file names passed to ffmpeg by the transcription endpoint [92] | 9.8 |
| CVE-2024-5182 | 2024-06-20 | 2.14.0 | Path traversal in model deletion that lets an attacker delete arbitrary files [93] | 9.1 (NVD) |
| CVE-2024-5181 | 2024-06-26 | 2.14.0 | Command injection through the backend parameter of a model configuration file [94] | 9.8 |
| CVE-2024-5616 | 2024-07-06 | 2.15.0 and earlier | Cross-site request forgery that deletes installed models [95] | 4.3 |
| CVE-2024-6095 | 2024-07-06 | 2.15.0; fixed in 2.17 | Server-side request forgery and partial local file inclusion through /models/apply [81] | 5.8 (NVD) |
| CVE-2024-6983 | 2024-09-27 | 2.17.1 | Remote code execution through an uploaded binary [82] | 8.8 |
| CVE-2024-6868 | 2024-10-29 | 2.17.1 | Arbitrary file write through automatic extraction of model archives [96] | 9.8 (NVD) |
| CVE-2024-7010 | 2024-10-29 | 2.17.1 | Timing attack on password handling that can reveal valid credentials [97] | 5.9 (NVD) |
| CVE-2024-48057 | 2024-11-04 | 2.20.1 and earlier | Cross-site scripting through the delete-model API [83] | 6.1 |
| CVE-2024-9900 | 2025-03-20 | 2.21.1 | Cross-site scripting in the search function [98] | 6.1 (NVD) |
| CVE-2026-59707 | 2026-07-07 | Not stated | Unauthenticated server-side request forgery through POST /models/apply [84] | 9.2 (CVSS 4.0) |
| CVE-2026-114493 | 2026-10-21 | 4.6.0 through 4.11.0 | Server-side request forgery through a redirecting gallery configuration URL [85] | 9.2 (CVSS 4.0) |
The records for the two 2026 entries name no fixed version.[84][85] The first refers to a report in the project's issue tracker, which was closed on 3 July 2026 by a commit titled "fix(gallery): block SSRF in gallery config URL fetch". The 4.6.0 release notes of the following day describe the change: configuration URLs fetched by POST /models/apply are checked against private, loopback, link-local and cloud-metadata addresses.[84][99][91] The second record describes a way past that check in versions 4.6.0 through 4.11.0, in which only the first URL was validated and a redirect could lead to an internal address.[85]
The repository's security policy asks for private reports by email and also accepts them through the Huntr platform. As read on 19 November 2026, its table of supported versions still named the 3.x series as the actively supported one.[77]
Comparison with Ollama and vLLM
LocalAI overlaps with Ollama and vLLM but is not a direct substitute for either. Repository metadata on 19 November 2026:
| LocalAI | Ollama | vLLM | |
|---|---|---|---|
| Repository | mudler/LocalAI [3] | ollama/ollama [87] | vllm-project/vllm [88] |
| Created | 2023-03-18 | 2023-06-26 | 2023-02-09 |
| Licence | MIT | MIT | Apache-2.0 |
| Main language | Go | Go | Python |
| GitHub stars | 50,438 | 187,554 | 97,183 |
vLLM is an inference engine, described in its repository as "A high-throughput and memory-efficient inference and serving engine for LLMs".[88] LocalAI can use it as one of its backends, on GPU targets only, and also ships its own C++ port, vllm.cpp, which adds CPU, Metal and Vulkan targets.[37] Ollama is closer to a direct alternative: both projects are servers written mainly in Go, and Ollama's documentation says it supports a subset of the OpenAI API.[3][87][100] LocalAI can pull models from Ollama's registry with the ollama:// prefix and answers Ollama's own API routes, so Ollama client software can be pointed at it.[1][31][55]
A comparison published by the hosting company Contabo in May 2026, and updated that November, summarised the difference this way: "Ollama bets on simplicity and a curated model registry; LocalAI bets on extensibility, multi-modal support, and supporting almost any model format." It judged LocalAI to offer broader coverage of OpenAI endpoints and of image and audio models behind one API, and Ollama to be simpler to install and operate, concluding that Ollama "wins on time-to-first-token" while LocalAI "wins on flexibility once you invest in setup".[86] The article is a vendor blog that recommends its own servers for both, and some of its commands refer to the all-in-one images that LocalAI dropped in 4.0.0.[86][29]
LocalAI's own FAQ records one performance difference from plain llama.cpp: LocalAI applies mirostat sampling by default for llama.cpp models, which the FAQ says gives better results but slows inference, and which can be turned off in the model configuration.[36]
Continue reading
- OllamaOllama is a free, open-source runtime for downloading, running, and managing open-weight large language models (LLMs) locally on personal computers and servers.Mentioned 18 times above
- vLLMvLLM is an open-source inference and serving engine for large language models. It is software for running already trained models, either through a Python interface for offline work or through a network server…Mentioned 14 times above
- llama.cppllama.cpp is an open-source large language model inference engine written in C and C++ by Bulgarian software engineer Georgi Gerganov that runs large language models on consumer-grade hardware without…Mentioned 14 times above
- Local AILocal AI is the practice of running artificial intelligence models on hardware that the user owns or controls, such as a laptop, a desktop PC, a workstation or a home serverListed in See also
- How to Run LLMs LocallyThis guide explains how to run LLMs locally: downloading the weights of a large language model (LLM) and performing inference on hardware you control, instead of sending prompts to a hosted API.Listed in See also
- Self-hosting LLMsSelf-hosting LLMs is the practice of running large language models on infrastructure that an organization operates itself, whether GPU servers it owns, GPU instances it rents or a cluster in its own cloud…Listed in See also
More from See also
References
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14mudler/LocalAI. "README.md". GitHub, retrieved November 19, 2026. github.com/...README.md
- ^1 ^2mudler/LocalAI. "LICENSE" (MIT License). GitHub, retrieved November 19, 2026. github.com/...LICENSE
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8GitHub. "mudler/LocalAI" (repository metadata). GitHub REST API, retrieved November 19, 2026. api.github.com/...LocalAI
- ^1 ^2 ^3 ^4LocalAI. "Architecture". LocalAI documentation (docs/content/concepts/architecture.md), retrieved November 19, 2026. github.com/...architecture.md
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8Di Giacinto, Ettore. "LocalAI, from March 2023 to now". LocalAI blog, 29 July 2026. localai.io/...localai-since-march-2023
- ^1 ^2 ^3 ^4LocalAI. "Model Gallery". LocalAI documentation (docs/content/models/_index.md), retrieved November 19, 2026. github.com/..._index.md
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12mudler/LocalAI. "v5.0.0" (release notes). GitHub, November 18, 2026. github.com/...v5.0.0
- ^LocalAI. "Backends". LocalAI documentation (docs/content/models/backends.md), retrieved November 19, 2026. github.com/...backends.md
- ^1 ^2mudler/LocalAI. "README.md" at tag v0.1 ("llama-cli"). GitHub, March 20, 2023. github.com/...README.md
- ^mudler/LocalAI. "README.md" at tag v0.8 ("llama-cli"). GitHub, April 12, 2023. github.com/...README.md
- ^1 ^2mudler/LocalAI. "README.md" at tag v1.0. GitHub, April 19, 2023. github.com/...README.md
- ^Gillson, Tyler. "K8sGPT + LocalAI: Unlock Kubernetes superpowers for free!". Spectro Cloud blog, April 29, 2023. spectrocloud.com/...ubernetes-superpowers-for-free
- ^Spectro Cloud. "Ettore Di Giacinto: Kubernetes insights, blogs & publications" (author page). spectrocloud.com, retrieved November 19, 2026. spectrocloud.com/...ettore-di-giacinto
- ^mudler/LocalAI. "v1.30.0" (release notes). GitHub, September 25, 2023. github.com/...v1.30.0
- ^mudler/LocalAI. "v1.40.0" (release notes). GitHub, October 30, 2023. github.com/...v1.40.0
- ^GitHub. "go-skynet/LocalAI" (repository metadata; resolves to mudler/LocalAI). GitHub REST API, retrieved November 19, 2026. api.github.com/...LocalAI
- ^1 ^2 ^3 ^4LocalAI. "Run with Kubernetes". LocalAI documentation (docs/content/start/install/kubernetes.md), retrieved November 19, 2026. github.com/...kubernetes.md
- ^1 ^2mudler/LocalAI. "v1.20.0" (release notes). GitHub, June 28, 2023. github.com/...v1.20.0
- ^1 ^2mudler/LocalAI. "v1.21.0" (release notes). GitHub, July 16, 2023. github.com/...v1.21.0
- ^1 ^2mudler/LocalAI. "v2.0.0" (release notes). GitHub, December 4, 2023. github.com/...v2.0.0
- ^1 ^2 ^3mudler/LocalAI. "v2.16.0" (release notes). GitHub, May 24, 2024. github.com/...v2.16.0
- ^mudler/LocalAI. "v2.17.0" (release notes). GitHub, June 17, 2024. github.com/...v2.17.0
- ^1 ^2mudler/LocalAI. "v2.19.0" (release notes). GitHub, July 19, 2024. github.com/...v2.19.0
- ^1 ^2 ^3mudler/LocalAI. "v3.0.0" (release notes). GitHub, June 19, 2025. github.com/...v3.0.0
- ^1 ^2mudler/LocalAI. "v3.2.0" (release notes). GitHub, July 24, 2025. github.com/...v3.2.0
- ^mudler/LocalAI. "v3.5.0" (release notes). GitHub, September 3, 2025. github.com/...v3.5.0
- ^1 ^2mudler/LocalAI. "v3.7.0" (release notes). GitHub, October 31, 2025. github.com/...v3.7.0
- ^1 ^2mudler/LocalAI. "v3.10.0" (release notes). GitHub, January 18, 2026. github.com/...v3.10.0
- ^1 ^2 ^3 ^4mudler/LocalAI. "v4.0.0" (release notes). GitHub, March 14, 2026. github.com/...v4.0.0
- ^1 ^2 ^3mudler/LocalAI. "v4.1.0" (release notes). GitHub, April 2, 2026. github.com/...v4.1.0
- ^1 ^2 ^3mudler/LocalAI. "v4.2.0" (release notes). GitHub, May 11, 2026. github.com/...v4.2.0
- ^1 ^2 ^3 ^4mudler/LocalAI. "v4.9.0" (release notes). GitHub, August 20, 2026. github.com/...v4.9.0
- ^1 ^2 ^3mudler/LocalAI. "v4.12.1" (release notes). GitHub, October 23, 2026. github.com/...v4.12.1
- ^Di Giacinto, Ettore. "LocalAI 5.0: your AI does the work, you keep the files". LocalAI blog, 18 November 2026. localai.io/...localai-5-0
- ^1 ^2 ^3LocalAI. "Overview". LocalAI documentation (docs/content/start/overview.md), retrieved November 19, 2026. github.com/...overview.md
- ^1 ^2 ^3LocalAI. "FAQ". LocalAI documentation (docs/content/reference/faq.md), retrieved November 19, 2026. github.com/...faq.md
- ^1 ^2 ^3 ^4LocalAI. "Model compatibility table". LocalAI documentation (docs/content/models/compatibility-table.md), retrieved November 19, 2026. github.com/...compatibility-table.md
- ^1 ^2LocalAI. "GPU Acceleration". LocalAI documentation (docs/content/models/gpu-acceleration.md), retrieved November 19, 2026. github.com/...gpu-acceleration.md
- ^1 ^2 ^3 ^4 ^5 ^6LocalAI. "Quickstart". LocalAI documentation (docs/content/start/quickstart.md), retrieved November 19, 2026. github.com/...quickstart.md
- ^LocalAI. "VRAM and Memory Management". LocalAI documentation (docs/content/models/vram-management.md), retrieved November 19, 2026. github.com/...vram-management.md
- ^1 ^2 ^3LocalAI. "Text generation". LocalAI documentation (docs/content/models/capabilities/text/text-generation.md), retrieved November 19, 2026. github.com/...text-generation.md
- ^LocalAI. "API". LocalAI documentation (docs/content/api/_index.md), retrieved November 19, 2026. github.com/..._index.md
- ^LocalAI. "Embeddings". LocalAI documentation (docs/content/models/capabilities/text/embeddings.md), retrieved November 19, 2026. github.com/...embeddings.md
- ^LocalAI. "Reranker". LocalAI documentation (docs/content/models/capabilities/text/reranker.md), retrieved November 19, 2026. github.com/...reranker.md
- ^LocalAI. "Image Generation". LocalAI documentation (docs/content/models/capabilities/vision/image-generation.md), retrieved November 19, 2026. github.com/...image-generation.md
- ^LocalAI. "Video Generation". LocalAI documentation (docs/content/models/capabilities/vision/video-generation.md), retrieved November 19, 2026. github.com/...video-generation.md
- ^1 ^2LocalAI. "Text to Audio (TTS)". LocalAI documentation (docs/content/models/capabilities/audio/text-to-audio.md), retrieved November 19, 2026. github.com/...text-to-audio.md
- ^LocalAI. "Audio to Text". LocalAI documentation (docs/content/models/capabilities/audio/audio-to-text.md), retrieved November 19, 2026. github.com/...audio-to-text.md
- ^LocalAI. "Voice Activity Detection (VAD)". LocalAI documentation (docs/content/models/capabilities/audio/voice-activity-detection.md), retrieved November 19, 2026. github.com/...voice-activity-detection.md
- ^1 ^2LocalAI. "Realtime API". LocalAI documentation (docs/content/api/openai-realtime.md), retrieved November 19, 2026. github.com/...openai-realtime.md
- ^LocalAI. "Object Detection". LocalAI documentation (docs/content/models/capabilities/vision/object-detection.md), retrieved November 19, 2026. github.com/...object-detection.md
- ^1 ^2LocalAI. "Moderation". LocalAI documentation (docs/content/models/capabilities/text/moderation.md), retrieved November 19, 2026. github.com/...moderation.md
- ^LocalAI. "Stores". LocalAI documentation (docs/content/api/stores.md), retrieved November 19, 2026. github.com/...stores.md
- ^LocalAI. "Middleware: PII filtering and intelligent routing". LocalAI documentation (docs/content/administration/middleware.md), retrieved November 19, 2026. github.com/...middleware.md
- ^1 ^2LocalAI. "Setting Up Models". LocalAI documentation (docs/content/models/setting-up-models.md), retrieved November 19, 2026. github.com/...setting-up-models.md
- ^LocalAI. "Model Context Protocol (MCP)". LocalAI documentation (docs/content/features/chats-and-agents/mcp.md), retrieved November 19, 2026. github.com/...mcp.md
- ^LocalAI. "LocalAI MCP server". LocalAI documentation (docs/content/features/chats-and-agents/localai-mcp-server.md), retrieved November 19, 2026. github.com/...localai-mcp-server.md
- ^1 ^2LocalAI. "Sandbox: run Python on your files". LocalAI documentation (docs/content/addons/catalogue/work/sandbox.md), retrieved November 19, 2026. github.com/...sandbox.md
- ^LocalAI. "Running commands in a chat". LocalAI documentation (docs/content/features/chats-and-agents/chats-commands.md), retrieved November 19, 2026. github.com/...chats-commands.md
- ^1 ^2 ^3LocalAI. "Distributed Mode". LocalAI documentation (docs/content/administration/distributed-mode.md), retrieved November 19, 2026. github.com/...distributed-mode.md
- ^1 ^2 ^3 ^4 ^5 ^6 ^7LocalAI. "P2P / Federated Inference". LocalAI documentation (docs/content/administration/p2p-federated-inference.md), retrieved November 19, 2026. github.com/...p2p-federated-inference.md
- ^LocalAI. "(experimental) MLX Distributed Inference". LocalAI documentation (docs/content/administration/mlx-distributed.md), retrieved November 19, 2026. github.com/...mlx-distributed.md
- ^mudler/LocalAGI. "README.md". GitHub, retrieved November 19, 2026. github.com/...README.md
- ^1 ^2 ^3GitHub. "mudler/LocalAGI" (repository metadata). GitHub REST API, retrieved November 19, 2026. api.github.com/...LocalAGI
- ^1 ^2mudler/LocalRecall. "README.md". GitHub, retrieved November 19, 2026. github.com/...README.md
- ^1 ^2 ^3GitHub. "mudler/LocalRecall" (repository metadata). GitHub REST API, retrieved November 19, 2026. api.github.com/...LocalRecall
- ^mudler/LocalAI. "go.mod" at tag v4.0.0. GitHub, March 2026. github.com/...go.mod
- ^mudler/LocalAI. "go.mod" at tag v4.12.1. GitHub, October 2026. github.com/...go.mod
- ^1 ^2mudler/LocalAI. "go.mod" at tag v5.0.0. GitHub, November 2026. github.com/...go.mod
- ^LocalAI. "Chats: moving from the old chat, agents and skills". LocalAI documentation (docs/content/features/chats-migration.md), retrieved November 19, 2026. github.com/...chats-migration.md
- ^LocalAI. "Knowledge (collections)". LocalAI documentation (docs/content/features/files-and-documents/collections.md), retrieved November 19, 2026. github.com/...collections.md
- ^1 ^2 ^3LocalAI. "Install LocalAI". LocalAI documentation (docs/content/start/install/_index.md), retrieved November 19, 2026. github.com/..._index.md
- ^LocalAI. "Containers". LocalAI documentation (docs/content/start/install/containers.md), retrieved November 19, 2026. github.com/...containers.md
- ^Open Source Initiative. "The MIT License". opensource.org, retrieved November 19, 2026. opensource.org/...mit
- ^mudler/LocalAI. "LICENSE" at tag v1.1.0. GitHub, April 2023. github.com/...LICENSE
- ^1 ^2mudler/LocalAI. "Using the LocalAI name, logo and mascot" (BRAND-USAGE.md). GitHub, retrieved November 19, 2026. github.com/...BRAND-USAGE.md
- ^1 ^2mudler/LocalAI. "Security Policy" (SECURITY.md). GitHub, retrieved November 19, 2026. github.com/...SECURITY.md
- ^Docker Hub. "localai/localai" (repository metadata). Docker Hub API, retrieved November 19, 2026. hub.docker.com/...localai
- ^1 ^2 ^3LocalAI. "Authentication & Authorization". LocalAI documentation (docs/content/administration/authentication.md), retrieved November 19, 2026. github.com/...authentication.md
- ^National Institute of Standards and Technology. "CVE-2024-3135 Detail". National Vulnerability Database, retrieved November 19, 2026. nvd.nist.gov/...CVE-2024-3135
- ^National Institute of Standards and Technology. "CVE-2024-6095 Detail". National Vulnerability Database, retrieved November 19, 2026. nvd.nist.gov/...CVE-2024-6095
- ^National Institute of Standards and Technology. "CVE-2024-6983 Detail". National Vulnerability Database, retrieved November 19, 2026. nvd.nist.gov/...CVE-2024-6983
- ^National Institute of Standards and Technology. "CVE-2024-48057 Detail". National Vulnerability Database, retrieved November 19, 2026. nvd.nist.gov/...CVE-2024-48057
- ^1 ^2 ^3 ^4National Institute of Standards and Technology. "CVE-2026-59707 Detail". National Vulnerability Database, retrieved November 19, 2026. nvd.nist.gov/...CVE-2026-59707
- ^1 ^2 ^3 ^4National Institute of Standards and Technology. "CVE-2026-114493 Detail". National Vulnerability Database, retrieved November 19, 2026. nvd.nist.gov/...CVE-2026-114493
- ^1 ^2Guo, Jie. "Ollama vs LocalAI: Best Self-Hosted OpenAI-Compatible LLM Server (2026)". Contabo blog, May 29, 2026 (updated November 11, 2026). contabo.com/...hosted-openai-compatible-llm-server
- ^1 ^2GitHub. "ollama/ollama" (repository metadata). GitHub REST API, retrieved November 19, 2026. api.github.com/...ollama
- ^1 ^2GitHub. "vllm-project/vllm" (repository metadata). GitHub REST API, retrieved November 19, 2026. api.github.com/...vllm
- ^mudler. "feat: gallery repository" (pull request #663). GitHub, mudler/LocalAI, merged June 24, 2023. github.com/...663
- ^1 ^2mudler/LocalAI. "Rename project to LocalAI (#35)" (commit 80f50e6). GitHub, April 19, 2023. github.com/...0e6ccd33aaefff6314f57a337ec410d33961
- ^1 ^2 ^3mudler/LocalAI. "v4.6.0" (release notes). GitHub, July 4, 2026. github.com/...v4.6.0
- ^National Institute of Standards and Technology. "CVE-2024-2029 Detail". National Vulnerability Database, retrieved November 19, 2026. nvd.nist.gov/...CVE-2024-2029
- ^National Institute of Standards and Technology. "CVE-2024-5182 Detail". National Vulnerability Database, retrieved November 19, 2026. nvd.nist.gov/...CVE-2024-5182
- ^National Institute of Standards and Technology. "CVE-2024-5181 Detail". National Vulnerability Database, retrieved November 19, 2026. nvd.nist.gov/...CVE-2024-5181
- ^National Institute of Standards and Technology. "CVE-2024-5616 Detail". National Vulnerability Database, retrieved November 19, 2026. nvd.nist.gov/...CVE-2024-5616
- ^National Institute of Standards and Technology. "CVE-2024-6868 Detail". National Vulnerability Database, retrieved November 19, 2026. nvd.nist.gov/...CVE-2024-6868
- ^National Institute of Standards and Technology. "CVE-2024-7010 Detail". National Vulnerability Database, retrieved November 19, 2026. nvd.nist.gov/...CVE-2024-7010
- ^National Institute of Standards and Technology. "CVE-2024-9900 Detail". National Vulnerability Database, retrieved November 19, 2026. nvd.nist.gov/...CVE-2024-9900
- ^mudler/LocalAI. "fix(gallery): block SSRF in gallery config URL fetch (#10665) (#10673)" (commit 2cbb3c9). GitHub, July 3, 2026. github.com/...3c96b34df1ca7b630c39cec2bb1a825d173f
- ^Ollama. "OpenAI compatibility". Ollama documentation (docs/api/openai-compatibility.mdx), retrieved November 19, 2026. github.com/...openai-compatibility.mdx
- ^National Institute of Standards and Technology. "CVE API: records matching the product identifier cpe:2.3:a:mudler:localai". National Vulnerability Database, retrieved November 19, 2026. services.nvd.nist.gov/...2.0
Improve this article
Add missing citations, update stale details, or write a clearer explanation. Anyone can edit: changes go live when you publish, and every edit is checked by an editor.
1 revision · v1 · 6,342 words · full history
Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify
Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here
Reviewer note: Independent fact-check before publication on November 19, 2026: 200 claims checked against 131 cited and independent sources; 2 material and 9 minor corrections applied before publication.
Cite this page: AI Wiki. "LocalAI (software)." aiwiki.ai, updated 19 Nov 2026, fact-checked 19 Nov 2026. CC BY 4.0. https://aiwiki.ai/wiki/localai_software

