Splash (inference engine)

Splash is an open-source large language model inference engine for Macs with Apple silicon, released by the inference company Inco AI on 18 September 2026 under the Apache-2.0 license.^[1][2][3] Most local engines, such as Ollama, llama.cpp or servers built on MLX, load many models through one general-purpose runtime. Splash takes the opposite approach: it supports a small set of models (two at launch, Qwen3.8-27B and Qwen3.6-35B-A3B) and ships each one as a package with its own fused Metal kernels, a trained DFlash 2 draft model for speculative decoding, and a precomputed memory plan.^[1][2] Inco describes the idea as "the engine is built around the model."^[1]

In Inco's own tests on a 48 GB M5 Pro, Splash decoded Qwen3.8-27B at 74 tokens per second on short prompts, about 2.0 times the next-fastest engine it measured, and Inco's launch post on X quoted 144 tokens per second for the same model on an M5 Max.^[1][9] LM Studio integrated Splash at launch as an experimental backend in its LM Studio Bionic agent app (version 1.1.5) and in LM Studio 0.4.25.^[6][7][8]

Background

Inco AI's core technology is DFlash, a speculative decoding method that uses a small block-diffusion model to draft a whole block of tokens in one forward pass. DFlash was published in February 2026 by Jian Chen, Yesheng Liang and Zhijian Liu of Z Lab at UC San Diego, and Inco says "our team released DFlash in January."^[11][12] Inco released DFlash 2, a successor drafter with a path selector and short convolutions, on 18 August 2026, and launched a hosted inference platform for data-center GPUs on 3 September 2026.^[11][13] Splash is Inco's attempt to carry the same approach (per-model drafts, generated kernels and a fixed memory budget) to a single Mac.^[1] Zhijian Liu, whose X profile says he is "building something new (@inco_ai)," wrote at launch that DFlash 2 had run Qwen3.8-27B at 70 tokens per second on an M5 Max a month earlier and that Splash ran it at 144 tokens per second "on the same Mac."^[10]

Design

One engine per model

The runtime, scheduler, cache and HTTP API are shared across models. Everything else is rebuilt for each supported model: fused, shape-specific kernels, a trained draft model, a memory plan and pinned performance baselines.^[1] By design Splash has no generic multi-model runtime and no fallback path. --model accepts any Hugging Face repository that holds a Splash package, but plain MLX or Transformers checkpoints do not load.^[1][2] Inco says the only settings are ceilings, such as --max-memory and --max-context, meant to leave room for other applications; the README also lists operational flags such as --port, --kv-format, --api-key and --no-webui.^[1][2]

Scheduling and memory

Splash batches requests as they arrive and balances prefill against decode so that new requests start quickly while running ones keep streaming. Attention state lives in a paged KV cache indexed by prefix, so a request that shares a prefix with an earlier one reuses those pages. For hybrid models it also snapshots the Gated DeltaNet recurrent state at prefix boundaries, so cache reuse covers the recurrent layers too.^[1] The target KV cache is stored as INT8 by default; a --kv-format bf16 option avoids that quantization at roughly twice the KV memory.^[2]

Because the weights, the draft and the per-token cost of each request are known in advance, Splash sets a memory budget at startup: Metal's recommended working-set limit minus those fixed costs. For Qwen3.8-27B, Inco puts this at 15 GiB of weights and a 1.2 GiB draft before any KV cache, and says that is where its 48 GB recommendation comes from: it leaves room for an editor and a browser while a task runs.^[1] (The model card breaks the 15 GiB down as 14.1 GiB for the 4-bit target and 0.9 GiB for the bf16 vision encoder.)^[4] If the model does not fit, startup prints a memory breakdown and stops.^[2]

DFlash 2 in the decode path

Speculative decoding is not optional in Splash: every supported model ships with its own DFlash 2 draft, trained for that model.^[1] The draft proposes a block of tokens in one pass and the target model verifies the whole block, so the output distribution is unchanged.^[1][4] The Qwen3.8-27B draft has five layers, reads the target's hidden states at five layers and proposes seven tokens per step; the Qwen3.6-35B-A3B draft has six layers and reads hidden states at eight layers.^[4][5] Inco lists three engine changes built around the draft:^[1]

• The whole decode step (drafting, verification, acceptance and state updates) is submitted as one unit when output is not schema-constrained, instead of step by step.
• Each request in a batch carries its own draft state, so speculation works alongside concurrency and cache reuse.
• The draft uses sliding-window attention, and on long prefills Splash computes only the draft state that generation and prefix snapshots need, which bounds draft memory as context grows.

Generated kernels

Prefill (large batches) and draft verification (small blocks) get separate 4-bit matrix-multiplication and attention kernels, written for each model's exact dimensions. Inco says these are produced by its in-house "kernel agents" rather than by hand.^[1] Examples it gives are decode kernels that read the 8-bit KV cache directly and reuse it across query heads and verification tokens, Gated DeltaNet prefill kernels that keep recurrent state on-chip, and dedicated kernels for routed experts in mixture-of-experts models.^[1] Kernels ship precompiled with presets for GPU family, core count and workload, and weights are packed for them and mapped zero-copy from disk, so no Xcode or compiler toolchain is needed on the user's machine.^[2]

Supported models

Sources: README and model cards.^[2][4][5] Each package includes a manifest.json with the upstream revisions and a SHA-256 hash of every artifact; Splash pins an immutable commit of the package repository and checks the hashes before installing.^[4] Both packages, and each of their upstream components, are listed as Apache-2.0.^[4][5] The chat templates are upstream's with one change: a system message after the first turn is rendered in place instead of rejected, which Inco says coding agents that inject instructions mid-conversation need.^[4] Context goes up to the models' native 256K window, subject to available memory.^[2]

Requirements and use

Splash requires an Apple M3 or newer (Apple GPU family 9), macOS 26.4 or later, Homebrew and at least 36 GB of unified memory, with 48 GB or more recommended.^[1][2][3] It installs with brew install incoai/tap/splash and starts with splash serve --model incoai/Qwen3.8-27B-Splash; the first run downloads and verifies the package, checks memory and serves on 127.0.0.1:8000, with a browser chat page at the same address.^[2]

The server speaks the OpenAI Chat Completions (/v1/chat/completions) and Responses (/v1/responses) APIs and the Anthropic Messages API (/v1/messages), with streaming, tool calls, JSON Schema output, image input and inline PDFs.^[2] Launcher commands (splash opencode, splash claude, splash codex, splash hermes) start OpenCode, Claude Code, Codex or Hermes against the local server.^[1][2] Reasoning follows the model default: the Qwen3.8-27B card says reasoning is on by default at xhigh, and reasoning_effort also takes low, medium or none; for Qwen3.6-35B-A3B reasoning is a switch rather than a dial.^[4][5] Authentication is off by default and is enabled with SPLASH_API_KEY.^[2]

Performance claims

All published Splash benchmarks are Inco's own measurements. The launch post, README and model cards report results from a single machine, an M5 Pro with a 16-core GPU and 48 GB of memory, serving both models over HTTP at each engine's recommended settings. Prompts were a fixed set of coding tasks from NVIDIA's SPEED-Bench up to 32K tokens, with a 1,024-token output limit and reasoning on (medium effort for the 27B, whose Splash default is xhigh). Decode figures include reasoning tokens, and single-request figures are medians.^[1][2][4] Inco notes that because every engine ran at its own recommended settings, the results are end-to-end comparisons that show the combined effect of specialization rather than the contribution of any single component.^[1] The comparison engines were oMLX (a general-purpose MLX server with batching and cache reuse), Lily and uzu (which serve one request at a time; uzu's Qwen3.8-27B package has no draft) and Ollama.^[1]

Single-request decode throughput in tokens per second, from Inco's Figure 2:^[1]

Other headline results from the same setup, with Inco's ratio against the next-fastest engine it measured:^[1][2][4][5]

Several caveats come from Inco's own text. The prefill leader on uncached prompts varies: on the 27B, uzu (313 tok/s at 32K) and Ollama (199) were both faster than oMLX (110), so the 27B's 1.2x prefill ratio is against uzu, and the 35B's 1.3x is against Lily (1,572 tok/s), not oMLX.^[1] The cached-prompt test replays a prompt exactly, which isolates the cached path; a real agent turn also pays for the tokens it appends.^[1] Inco reports that Ollama's cache did not hit on these long prompts, that Lily took 22 seconds on the 35B replay with no cache hits, and that uzu took 112 seconds on the 27B replay.^[1] In a memory test, 16 concurrent 32K requests sent to the 27B were all accepted and completed by Splash, while "a general-purpose memory policy accepted nine of them on its first pass"; Inco adds that it "did not test whether another engine could be configured to do the same."^[1] Inco also cautions that reading a large repository cold remains expensive "for every engine we tested, ours included."^[1]

The 144 tokens per second figure

The widely repeated figure of 144 tokens per second for Qwen3.8-27B comes from Inco's launch post on X, which cites an M5 Max MacBook Pro, and from LM Studio's post, which says "up to 144 tok/sec on M5 Max."^[9][14] It is a different chip from the 48 GB M5 Pro used for every table in the launch blog, README and model cards, and those documents give no M5 Max methodology.^[1][2][4] The same X post summarizes the gains as "up to 3x the decode speed of Ollama, 2x oMLX, and almost 4x when an agent fans out into sub-agents"; in the M5 Pro tables those correspond to the Qwen3.8-27B short-prompt decode ratios (74 against Ollama's 24 and oMLX's 38) and the four-request concurrency result (3.9x).^[1][9]

LM Studio integration

Inco and LM Studio announced Splash support on the same day. LM Studio's 18 September blog post tells users to install LM Studio Bionic 1.1.5 or newer, download "Splash (Metal)" under Settings, Runtime, Experimental backends, then paste one of the two Hugging Face package links into the model search.^[6] The Bionic 1.1.5 release notes, dated 19 September 2026, list "[Mac only] Ultra fast Qwen3.8 inference using a new engine: Splash (by Inco AI)," and LM Studio 0.4.25, released the same day, added "support for the Splash engine on Apple Silicon M3 or newer running macOS 26.4 or newer."^[7][8] Inco's post calls Splash "a first-class inference engine" in LM Studio and says the two companies plan to ship updates and support new models together.^[1] The Splash README notes that such integrations manage their own runtime and settings.^[2]

Release history

For 1.0.2, Inco reported that on an M3 Max the batch extension raised aggregate decode throughput by 53 to 74 percent for the 27B and 30 to 41 percent for the 35B against the preceding batch-one implementation, and described these as incremental measurements rather than guarantees.^[3]

License

The engine is licensed under Apache-2.0; the README states that model weights keep their own licenses.^[2] According to the two model cards, both launch packages are Apache-2.0 throughout: Qwen's base models, the mlx-community 4-bit conversions and Inco's DFlash 2 drafts. The standalone Qwen3.6-35B-A3B draft repository the card cites is not publicly listed on Hugging Face.^[4][5] The GitHub repository is mostly Python, with topics including apple-silicon, metal and speculative-decoding.^[3]

References

1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20 ^21 ^22 ^23 ^24 ^25 ^26 ^27 ^28 ^29 ^30 ^31Inco AI. "Splash: A Local Engine Built Around the Model." Inco AI blog, dated September 17, 2026. inco.ai/...splash (accessed September 23, 2026; the earliest Wayback Machine capture, 19 September 01:29 UTC, is byte-identical)
2. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19Inco AI. "Splash" README. GitHub, incoai/splash. github.com/...splash (accessed September 23, 2026)
3. ^1 ^2 ^3 ^4 ^5 ^6 ^7Inco AI. incoai/splash repository metadata and releases 1.0 (September 18, 2026), 1.0.1 (September 20, 2026) and 1.0.2 (September 21, 2026). GitHub. github.com/...releases
4. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12Inco AI. "Qwen3.8-27B-Splash" model card. Hugging Face. huggingface.co/...Qwen3.8-27B-Splash
5. ^1 ^2 ^3 ^4 ^5 ^6Inco AI. "Qwen3.6-35B-A3B-Splash" model card. Hugging Face. huggingface.co/...Qwen3.6-35B-A3B-Splash
6. ^1 ^2 ^3LM Studio Team. "Splash Engine - the fastest local Qwen3.8 on Apple Silicon." LM Studio Blog, September 18, 2026. lmstudio.ai/...splash-engine
7. ^1 ^2 ^3LM Studio. "Bionic 1.1.5" release notes, September 19, 2026. lmstudio.ai/...bionic-v1.1.5
8. ^1 ^2 ^3LM Studio. "LM Studio 0.4.25" release notes, September 19, 2026. lmstudio.ai/...lmstudio-v0.4.25
9. ^1 ^2 ^3 ^4Inco AI (@inco_ai). Post on X announcing Splash, September 19, 2026 (00:07 UTC). x.com/...2101100749623341513
10. ^1 ^2Zhijian Liu (@zhijianliu_). Post on X, September 19, 2026 (00:15 UTC). x.com/...2101102949158302001
11. ^1 ^2Inco AI. "DFlash 2: Keep Drafting Parallel." Inco AI blog, August 18, 2026. inco.ai/...dflash2
12. ^Jian Chen, Yesheng Liang, and Zhijian Liu. "DFlash: Block Diffusion for Flash Speculative Decoding." arXiv:2602.06036, February 5, 2026 (v2 May 28, 2026; ICML 2026). arxiv.org/...2602.06036
13. ^Inco AI. "Inco AI Launches Its Inference Platform, Leading Across Four Open Models on Artificial Analysis." Inco AI blog, September 3, 2026, updated September 8, 2026. inco.ai/...inco-platform-aa
14. ^1 ^2LM Studio (@lmstudio). Post on X, September 19, 2026 (00:45 UTC). x.com/...2101110346954207521