Project Maya

Project Maya is open-source software for running large mixture-of-experts language models on local hardware. It combines model-specific inference engines with a server, browser dashboard and APIs compatible with OpenAI and Anthropic clients. Peasant Smith, known as mw00, created the project; needmorevram is a co-maintainer. Its main documented model families are GLM-5.3-Flash, DeepSeek-V4.1-Flash-1017 and MiniMax-M3.1-Flash.^[1]

Memory management

Maya uses quantization and memory tiers to serve models whose complete weights exceed GPU memory. In its GLM and DeepSeek paths, frequently used experts stay in VRAM, other experts can reside in system RAM, and data that does not fit in memory is read from model files on an SSD. The setup checks the machine and configures the model, GPU allocation, context and related engine options.^[2][3]

This makes GPU memory capacity different from the total storage and working-memory requirements. The setup documentation gives an example of a DeepSeek EXL3 configuration running on one RTX 3090 and 64 GB of RAM at about 4-5 output tokens per second, with substantial SSD reads. It describes this as much slower than a configuration that holds more of the experts in RAM and VRAM. These are project-reported results for that configuration, rather than a general speed guarantee.^[2]

The GLM settings also expose the tradeoff between prompt buffers and expert-cache capacity. At startup, the engine estimates memory placement and measures CPU computation against transferring experts over PCIe. Its RAM tier normally leaves about 6 GB free for the system. More concurrent requests and larger contexts consume additional resources.^[3]

Models and quantized files

As documented on November 4, 2026, Maya provides engines and its own quantized files for the following families:^[4]

The sizes describe files, rather than the complete memory footprint of an active service. XS, S, M or L identifies a tier within each family. Setup also offers selected community GGUF and EXL3 conversions. MiniMax's standard engine computes off-GPU experts from RAM, with different requirements from DeepSeek's disk-backed path. Format, image support, drafting and hardware requirements vary between options.^[4]

Dashboard and serving

The dashboard provides chat, a hardware monitor and a model-management interface. It can download models, switch the loaded model and change engine settings. The project's Turbo control changes generation and execution settings, including disabling thinking and enabling expert skipping; the documentation describes a quality-for-speed tradeoff.^[1]

Maya exposes Chat Completions and Anthropic-compatible Messages endpoints. Its Responses endpoint supports text and image input, streaming, function calls, background tasks and optional stored conversation history. These are compatibility interfaces to a local model; they do not make the underlying model an OpenAI or Anthropic model.^[1][5]

Responses are not stored by default. Opt-in Responses history is held in memory and does not survive a server restart. The documented defaults bound it to 128 cached responses, 64 MiB of serialized records and one hour of retention after completion.^[5]

Structured outputs use schema validation before publishing generated content. They do not use token-level constrained decoding: the server buffers the result and rejects invalid output. Strict tool validation and schema output can therefore delay content streaming.^[5]

Platforms and deployment limits

The main documentation covers Linux and Windows with NVIDIA or selected AMD hardware, with experimental support distinguished from validated hardware. A fast NVMe SSD and enough disk space for the selected files are important when experts spill beyond memory.^[1][2]

Apple Silicon support is experimental and has separate engines. The GLM Metal backend supports images and one conversation at a time. MiniMax's Metal engine is newer, text-only and not yet offered by the automated setup. It requires manual building and keeps every expert resident; the documentation identifies the 93 GB Maya-XS set as suitable for a 128 GB Mac. DeepSeek-V4.1-Flash and Qwen-3.8-Flash-Next are not supported by that Metal backend.^[6]

Inference runs locally, but setup downloads source dependencies and model files. The update check contacts GitHub periodically and can be disabled. The setup documentation also describes explicit API-key and allowed-origin configuration when serving clients beyond the local browser.^[2]

Development and licensing

At the inspected November 4, 2026 repository state, the version file and changelog identified v1.1.53. Its changes included redesigned model, monitor and settings pages, experimental MiniMax Metal support, and fixes for third-party DeepSeek conversions and failed engine rebuilds.^[7]

The software's license is MIT. Model weights, conversions and optional drafters retain their respective terms. For example, the model documentation identifies MiniMax's community license and labels the optional GLM DFlash2 drafter CC BY-NC-ND 4.0. The software license alone does not establish permission to use every downloadable component commercially.^[4][8]

Continue reading

• llama.cppllama.cpp is an open-source large language model inference engine written in C and C++ by Bulgarian software engineer Georgi Gerganov that runs large language models on consumer-grade hardware without…Listed in See also
• SSpeculative DecodingSpeculative decoding is a lossless inference acceleration technique for autoregressive transformer models in which a small, fast draft model proposes several future tokens at once and the larger target model…Listed in See also
• vLLMvLLM is an open-source inference and serving engine for large language models. It is software for running already trained models, either through a Python interface for offline work or through a network server…Listed in See also
• Mixture of Experts (MoE)A mixture of experts (MoE) is a machine-learning architecture that contains multiple component functions called experts and a gating or routing mechanism that determines how their outputs contribute to a…Linked in the introduction
• MiniMax M3.1 FlashMiniMax M3.1 Flash is a large language model released by MiniMax on October 19, 2026, for software development, tool-using agents and computer use.Links back to this article
• GLM-5.3-FlashGLM-5.3-Flash is an open-weight, natively multimodal mixture-of-experts large language model released by Z.ai on August 26, 2026.Mentioned twice above

More to read

• Quantization
• DeepSeek V4.1-Flash
• NVIDIA Dynamo
• OpenVINO
• FlashInfer
• Text Generation Inference (TGI)

Topics:AI InferenceDeveloper ToolsOpen Source AI

References

1. ^1 ^2 ^3 ^4Project Maya contributors. Project Maya README. Repository state inspected November 4, 2026.
2. ^1 ^2 ^3 ^4Project Maya contributors. Setting up Maya. Installation, expert offload, model downloads and server configuration. Accessed November 4, 2026.
3. ^1 ^2Project Maya contributors. Settings. Memory placement, calibration, expert caches and concurrency. Accessed November 4, 2026.
4. ^1 ^2 ^3Project Maya contributors. The models. Quantized files, engines and model-specific restrictions. Accessed November 4, 2026.
5. ^1 ^2 ^3Project Maya contributors. Responses API. Supported operations, history retention and output validation. Accessed November 4, 2026.
6. ^Project Maya contributors. Maya on Apple Silicon (Metal). Experimental GLM and MiniMax support. Accessed November 4, 2026.
7. ^Project Maya contributors. Changelog, v1.1.53 and version file. November 4, 2026.
8. ^Project Maya contributors. MIT license. Accessed November 4, 2026.