Atlas (inference engine)

Atlas is an open-source large language model inference engine written in Rust and CUDA, built for desktop "unified memory" machines such as the NVIDIA DGX Spark (GB10) and AMD Strix Halo (Ryzen AI Max+ 395) systems. It ships as a single binary with no Python or PyTorch in the serving path and exposes an OpenAI-compatible API.^[1][2] Azeez Ishaqui announced the engine publicly in May 2026, and it is licensed under the AGPL-3.0.^[3][7] Under the submitter name Atlas Inference, the project entered the first round of the MLPerf Inference Edge Agentic benchmark (v6.1). MLCommons' announcement of that benchmark lists Ishaqui and Thomas Braun among its authors.^[4][5][6] Since late August 2026 the project has had two rival lines, each calling itself Atlas: the original repository, now branded by Atlas Cybernetics Corp, and a newer Atlas-Inf organization run by Ishaqui. The Atlas-Inf side calls the original website and repository "disputed Atlas assets".^[7][8][9]

The engine is unrelated to Atlas (robot), the Atlas language model, ChatGPT Atlas, and the commercial "Atlas Inference" offering that Atlas Cloud announced in March 2026.^[18]

Overview

Repository dates and licenses come from the GitHub API as of 25 October 2026.^[7][8]

Design

The original repository's README describes Atlas as an engine that "serves an OpenAI-compatible server (plus Anthropic and Responses APIs) from a single ~75 MB binary". It says there is "no Python, no PyTorch, no runtime compilation". The README lists hand-tuned CUDA kernels for each combination of hardware, model and quantization; NVFP4 and FP8 quantization; speculative decoding with multi-token prediction (MTP) draft heads and DFlash block diffusion; radix-tree prefix caching; and expert parallelism across nodes.^[2] It says the KV cache type accepts 16 data types in total, including asymmetric "TurboQuant+" pairings that store keys at higher precision than values.^[2]

In the May 2026 launch post, Ishaqui said the kernels cover attention, mixture-of-experts routing, Gated DeltaNet and Mamba-2 paths for Blackwell SM120/121 and avoid generic fallbacks. He said the team had "observed Blackwell tensor cores sitting underutilized while the Python serving stack interpreted itself" and judged a clean rewrite better than more layers of optimization. He gave the container image as about 2.5 GB and the cold start as under two minutes, against "roughly ten minutes for a typical PyTorch-based stack going through torch.compile". The same API port speaks both OpenAI and Anthropic schemas, so clients such as Claude Code work unchanged.^[3]

Both lines describe a release "gate": an image ships only if every model in the serve matrix boots, stays coherent (greedy determinism, no token leakage, reliable tool calls), and holds throughput within 10% of its committed baseline.^[1][9] The original README says AI-authored pull requests are "the default, and the target" of the contribution process.^[2] In an August 2026 blog post, Braun said the team started the engine "earlier this year" and left the Python vLLM stack to do it.^[10]

AMD support works differently depending on the source. The original README and the Atlas-Inf website say the same CUDA source compiles for AMD gfx1151 through SCALE, a CUDA compiler from Spectral Compute.^[2][1] The MLPerf submission describes its Strix Halo build as a "native-HIP NVFP4 build via a CUDA→HIP shim" running on ROCm 7.13.^[6][11]

History

Launch (May 2026)

The repository under the Avarok-Cybersecurity GitHub organization was created on 5 May 2026.^[7] On 7 May 2026 Ishaqui posted "Atlas: Open-source inference engine for DGX Spark" on the NVIDIA Developer Forums. The post linked that repository and the atlasinference.io site, and gave Docker images under avarok/atlas-gb10.^[3] It listed these single-Spark figures, all self-reported:^[3]

Hugging Face Transformers kernel (June 2026)

Ishaqui opened a pull request to Hugging Face Transformers on 4 June 2026, and it was merged on 19 June. It registers a Hub kernel, Atlas-Inference/gdn, for the Gated DeltaNet layers of Qwen3.6 27B and 35B-A3B. The kernel is gated to compute capability 12.1, so it only activates on GB10. The PR says the fla and causal_conv1d fast paths had no SM121 build, so Qwen3.6 fell back to a slow pure-PyTorch Gated DeltaNet on DGX Spark. It reports layer output parity against the PyTorch fallback of cosine 0.999970 (27B) and 0.999967 (35B-A3B), and identical greedy-generation token IDs in a full-model test on the 35B-A3B model.^[12]

MLPerf Edge Agentic benchmark (July 2026)

On 9 July 2026, MLCommons' Edge LLM Taskforce issued a call for submissions for a new Edge Agentic Inference benchmark in MLPerf Inference v6.1. It tests multi-turn tool calling with Qwen3.6-27B on a single edge accelerator. Accuracy is gated by the Berkeley Function Calling Leaderboard v4, and performance is measured on a recorded replay of agentic coding sessions. The reference implementation used llama.cpp, and submissions were due by 31 July 2026. The post lists six authors: four from NVIDIA, including the taskforce chair, plus "Azeez Ishaqui (Atlas Inference)" and "Thomas Braun (Atlas Inference)".^[5]

Two Atlas projects (August 2026 onward)

The Atlas-Inf GitHub organization and its atlas repository were created on 24 August 2026. The repository's README ends with a "continuity notice" (the atlasinference.dev footer carries a near-identical version): "Atlas is continuing. This repository, the Atlas-Inf GitHub organization, and atlasinference.dev are the replacement official Atlas channels. The existing website and GitHub repository remain disputed Atlas assets that have not been relinquished."^[8][1] The X account @AtlasInference, whose display name is "Azeez", links to atlasinference.dev.^[13]

The original repository is still active. Its README is branded "Atlas Cybernetics Corp" and points to atlascybernetics.ai, and its listed homepage, atlasinference.io, now serves the Atlas Cybernetics site.^[2][7][19] Braun published the first post on the Atlas Cybernetics blog on 31 August 2026; the blog footer says the engine is "built in North Carolina".^[10] As of 25 October 2026, the original repository had about 712 GitHub stars and 106 forks, and the Atlas-Inf repository had 36 stars and 12 forks.^[7][8] The sources cited here do not explain the causes of the split.

The two lines have since moved apart. The Atlas-Inf site installs models through the sparkrun tool (sparkrun run @atlas/<recipe>) and lists Qwen3.8 27B as its flagship, along with Nemotron 3.5 Lightning.^[1] The Atlas Cybernetics site says "Sparkrun has been retired" in favour of its own atlasctl control plane, and lists Qwen3.6 as its flagship family.^[9] Both list recipes for Qwen, Gemma, Nemotron, Mistral, MiniMax and DeepSeek models, and both say they offer EP=2 expert parallelism across two DGX Sparks.^[1][9]

MLPerf Inference v6.1 results

MLCommons published the v6.1 results on 16 September 2026. Atlas Inference was one of six first-time submitters.^[4] Its two entries are in the closed edge division, and both ran Qwen3.6-27B quantized to NVFP4 in the SingleStream scenario. The submission's source tree is vendored from the Avarok-Cybersecurity/atlas repository at fixed commits.^[6][14] The benchmark's headline metric is mean per-turn latency in milliseconds (lower is better). Accuracy is BFCL v4 against a floor of 83.64.^[6][11]

These are all closed-division edge agentic entries in MLCommons' published summary file, rounded. Accuracy is shown as a percentage.^[11]

According to the submission README, the DGX Spark run completed all 1,007 turns in 3,834.4 seconds with a median time per output token of 32.90 ms and 20.08 tokens per second. The Strix Halo run took 7,108.6 seconds, with a median time per output token of 47.3 ms.^[6] In its supplemental statement, Atlas Inference said that on DGX Spark "1007 agentic coding turns completed in under 64 minutes at 20.1 tokens per second with time taken to serve just 40 seconds after a cold start", and that on Strix Halo "the same engine and recipe delivered 19.63 tokens per second".^[15] StorageReview attributed the same two figures to Atlas and called the result "a narrower gap than we measured between the two platforms with off-the-shelf runtimes".^[4] The 19.63 figure does not appear in the published Strix Halo result files. Their throughput field reads 10.22 tokens per second, and the Strix Halo run's mean latency is about 1.85 times the DGX Spark run's.^[16][11]

The MLCommons working-group chairs' analysis of the round quoted the opening of Atlas's statement: "For the past few years, serious agentic work meant a datacenter round trip. That assumption is what this submission is meant to retire."^[17]

Self-reported benchmarks against vLLM

The Atlas Cybernetics site publishes a concurrency ladder comparing Atlas with vLLM 0.27.1 on one GB10. The model is the unsloth Qwen3.8-27B-NVFP4 checkpoint, with 128 input and 1,024 output tokens, three timed repetitions, and MTP speculation at K=4 on both engines.^[9] These are the vendor's own measurements:

The page also shows a vLLM run without speculation. It is configured differently (BF16 KV cache, 4,096-token context), and the page leaves it out of the ratio. At 128 concurrent requests that run reached 390.42 tok/s, more than the vLLM + MTP run.^[9] The Atlas-Inf README cites the same 23.59 tok/s single-stream figure for Qwen3.8 27B.^[8]

Reception

The May 2026 NVIDIA forum thread grew to more than 100 posts, and reactions were mixed.^[3] One user ran RedHatAI/Qwen3.6-35B-A3B-NVFP4 and posted single-request results of 90.9 to 140.1 tok/s across five prompt types over two runs, plus a combined 98.9 tok/s for four parallel requests. Another user said the published quick-start command for Qwen3.5-122B would not run. The same user later reported that on his own agentic test Atlas was "6x slower" than vLLM with Qwen3.6-35B-A3B-FP8 and finished 8 of 22 tasks. Others asked for tested per-model recipes and documentation that matched the CLI, and the maintainers said they would improve the README.^[3] In June 2026 another user reported 43.2 to 55.8 tok/s for Qwen3.6-35B-A3B-FP8, a cold start of about 44 seconds, and 21 missing CUDA kernels on SM121.^[3]

Vision-language-action claim (October 2026)

On 23 October 2026 the @AtlasInference account posted that "VLA's like @physical_int Pi05 are coming to Atlas", meaning vision-language-action models such as Physical Intelligence's π0.5. The post included a 4.5-second video and said it showed Atlas controlling "its first robot on GB10 solely through the model itself and the vision feeds". It added that "AGX Thors aren't the only physical edge devices", a reference to NVIDIA's Jetson AGX Thor. This is the account's own description of a short video, not a published benchmark or technical report.^[13]

Licensing

Both repositories are licensed AGPL-3.0.^[7][8] The original README describes a dual-license model: an AGPLv3 Community Edition and a commercial Enterprise Edition for closed-source or SaaS use without AGPL source-disclosure obligations. Contributions are covered by a contributor license agreement that permits re-licensing for the Enterprise Edition.^[2] The atlasinference.dev site describes the same split: "Community Edition under AGPLv3, Enterprise Edition commercial."^[1]

References

1. ^1 ^2 ^3 ^4 ^5 ^6 ^7Atlas Inference. "Atlas, pure Rust inference for DGX Spark." atlasinference.dev. Accessed 25 October 2026. atlasinference.dev
2. ^1 ^2 ^3 ^4 ^5 ^6 ^7Avarok-Cybersecurity/atlas. README.md. GitHub. Accessed 25 October 2026. github.com/...atlas
3. ^1 ^2 ^3 ^4 ^5 ^6 ^7AzeezIsh. "Atlas: Open-source inference engine for DGX Spark <2minute cold start, 100+ tok/s on Qwen3.6-35B-FP8, 13+ supported models." NVIDIA Developer Forums, 7 May 2026. forums.developer.nvidia.com/...369263
4. ^1 ^2 ^3Harold Fritts. "MLPerf Inference v6.1: 5.7x Per-Accelerator Gains, a 512-GPU Run, and Vera Rubin's First Peer-Reviewed Numbers." StorageReview, 16 September 2026. storagereview.com/...s-first-peer-reviewed-numbers
5. ^1 ^2MLCommons. "Call for Submission: Edge Agentic Inference Benchmark for MLPerf Inference v6.1." 9 July 2026. mlcommons.org/...mlperf-inference-v61-edge-agentic
6. ^1 ^2 ^3 ^4 ^5MLCommons. "Atlas_Inference: MLPerf Inference v6.1 submission" (README and per-system result READMEs). mlcommons/inference_results_v6.1, GitHub. github.com/...Atlas_Inference
7. ^1 ^2 ^3 ^4 ^5 ^6 ^7GitHub API. Repository metadata for Avarok-Cybersecurity/atlas. Accessed 25 October 2026. github.com/...atlas
8. ^1 ^2 ^3 ^4 ^5 ^6Atlas-Inf/atlas. README.md and repository metadata. GitHub. Accessed 25 October 2026. github.com/...atlas
9. ^1 ^2 ^3 ^4 ^5 ^6Atlas Cybernetics Corp. "Atlas engine, pure Rust inference for DGX Spark." atlascybernetics.ai. Accessed 25 October 2026. atlascybernetics.ai/engine
10. ^1 ^2Thomas Braun. "Seven Tenets Powering Atlas Inference Accelerated Workloads." Atlas blog, 31 August 2026. blog.atlascybernetics.ai/...wering-atlas-inference
11. ^1 ^2 ^3 ^4MLCommons. MLPerf Inference v6.1 results summary (summary.csv) and Atlas_Inference system descriptions. mlcommons/inference_results_v6.1, GitHub. github.com/...summary.csv
12. ^AzeezIsh. "Add GB10/SM121 Hub-kernel path for Qwen3.6 Gated DeltaNet." huggingface/transformers pull request #46423, opened 4 June 2026, merged 19 June 2026. github.com/...46423
13. ^1 ^2@AtlasInference. Post on X, 23 October 2026. x.com/...2113489792801366517
14. ^MLCommons. Atlas_Inference submission source README (provenance). mlcommons/inference_results_v6.1, GitHub. github.com/...README.md
15. ^MLCommons. "MLPerf Inference v6.1 Supplemental Discussion" (Atlas Inference statement). September 2026. mlcommons.org/...mmons-MLPerf-Inference-v6.1-1.pdf
16. ^MLCommons. Atlas_Inference Strix Halo result_summary.json. mlcommons/inference_results_v6.1, GitHub. github.com/...result_summary.json
17. ^MLCommons. "Where the Industry Is Investing: A Look at MLPerf Inference v6.1." 17 September 2026. mlcommons.org/...chairs-mlperf-inference-v6-1
18. ^Atlas Cloud. "Atlas Inference: Doing DeepSeek Better Than DeepSeek." Atlas Cloud blog, 18 March 2026. atlascloud.ai/...atlas-inference
19. ^atlasinference.io (serving the Atlas Cybernetics Corp home page). Accessed 25 October 2026. atlasinference.io