Muse Spark Open

Muse Spark Open is an open-weight multimodal reasoning model released by Meta Superintelligence Labs on October 15, 2026. Meta specifies 816 billion total parameters, 42 billion active parameters per token, and a 1 million-token context window.^[1] It accepts text, images, video, and audio and produces text, with training focused on coding and extended agent workflows.^[7] Meta calls it "the most capable US open-weight model to date", a description repeated by launch partners such as vLLM and Baseten.^[1][5][4]

The release provides downloadable checkpoints rather than only a hosted service. It belongs to the Muse Spark family and has its own model card, evaluation results, and license.^[1] The PyTorch team at Meta says the open model shares the same base model as Muse Spark 1.3.^[11] The vLLM team describes it as the model from which Muse Glimmer 30B was distilled.^[5] Inference engines, chip vendors, and cloud providers announced support on the day of release, including vLLM, SGLang, NVIDIA TensorRT-LLM, MLX, and llama.cpp.^[7]

Background

Meta launched the original Muse Spark in April 2026 and released several refinements afterwards.^[21] On September 2, 2026, The Register reported that Mark Zuckerberg had promised in an X post that an open-weights version of Muse Spark would arrive "soon", while Muse Spark 1.3 was live on Meta's API and its Muse Code command-line tool.^[21] Meta published the weights on Hugging Face on October 15, 2026.^[1][2] The PyTorch team says the open version builds on Meta's August release of Muse Spark and adds safety post-training intended to make it more robust as a standalone model without system guardrails.^[11]

Model overview

Meta's model card specifies:^[2]

The chat template's default system block gives a knowledge cutoff of January 4, 2026.^[8] Baseten's launch post lists 42.7B parameters active during inference, a more precise figure than Meta's rounded 42B.^[4]

Architecture

Muse Spark Open uses a sparse mixture of experts. A learned projection compresses each token's representation before the selected experts process it; a second shared projection restores the original width. The router still receives the full-width representation. Baseten describes this arrangement as reducing expert parameters and communication between GPUs compared with an uncompressed design.^[4] According to the vLLM team, the shared "pooler" is an 8192 x 4096 matrix, so the experts work at a width of 4096 rather than 8192, which halves the parameters in the two expert projection layers.^[5] The SGLang team adds that each selected expert's output passes through its own RMSNorm before it is multiplied by its routing weight and summed.^[15] Every block is normalized both before and after the sublayer ("sandwich normalization").^[5]

vLLM's breakdown of where the weights sit:^[5]

The vision tower figure differs slightly from the model card, which gives 1.8 billion parameters for the Perception Encoder.^[2][5]

The decoder interleaves 46 local-attention layers with 16 full-attention layers. Local layers use a 2048-token sliding window; full-attention layers retain access to the complete sequence. Baseten reports a reduction in key-value cache memory from 31 GiB to 8.4 GiB per sequence at 128K context relative to an all-global-attention configuration. That is a configuration-specific cache comparison, not a reduction of the model's entire memory requirement.^[4] The layers follow a repeating cycle of four (three sliding-window layers and one full-attention layer), anchored so that the final layer is always full attention. vLLM says that at 1 million tokens this brings the KV cache to just over a quarter of what all-full attention would need.^[5] Baseten adds that the local layers use rotary position embeddings while the global layers use no positional encoding.^[4]

The vLLM implementation uses a Perception Encoder for images and sampled video frames, together with a separate audio encoder. Its audio path takes 24 kHz waveforms and turns every 1,920 samples (80 ms) into one token; vLLM puts the whole audio encoder at 118M parameters. The vLLM team states that a video's audio track is not processed automatically: applications must supply video and extracted audio as separate input parts when both are needed.^[5] Hugging Face's Transformers integration adds a load_audio_from_video option for passing video with its audio, and samples video at 2 frames per second by default, capped at 96 frames.^[16]

Checkpoints and local deployment

Meta distributes two reference checkpoint formats through Hugging Face. Its download guide gives the following approximate weight footprints and example serving configurations.^[6]

NVFP4 stores most weights at 4-bit precision while retaining some layers in BF16. These figures exclude the key-value cache, activations, and runtime overhead. Hardware requirements also depend on the context length, interconnect, parallelism, and serving software.^[6] Baseten says the BF16 parts of the NVFP4 checkpoint are the attention weights and the experts in the first and last layers.^[4] The model card states a requirement of at least 700 GB of GPU memory for NVFP4 and 2 TB for BF16.^[2] The Hugging Face repository is gated: users must accept the license and submit identifying details before downloading.^[2]

Partners published additional quantized formats at launch:^[7][15]

Meta's developer launch guide lists SGLang, vLLM, TensorRT-LLM, Transformers, llama.cpp, MLX, and Unsloth integrations.^[7]

Its Mac recipe requires two M5 Ultra Mac Studios with 512 GB of unified memory each, or four machines with 256 GB each, connected directly with Thunderbolt 5 and RDMA. It also specifies macOS 26.2 or later, MLX 0.30.1 or later, and disk space for the full NVFP4 checkpoint on every machine. The recipe estimates about four hours to complete and says four Macs serve fastest. This is a multi-machine setup, not an ordinary laptop deployment.^[13] Meta's research blog says the M5 Ultra's GPU can use up to 512 GB per Mac Studio and that Meta tested agentic workflows running fully on-device with Muse Code.^[1]

Day-0 inference support

vLLM

On October 15, 2026, the vLLM project announced that day-0 support was live and "verified on NVIDIA and AMD GPUs as well as on TPUs", and said it had worked with Meta Superintelligence Labs on the integration. The announcement listed BF16, NVFP4, and MXFP4 support and DSpark speculative decoding on both CUDA and ROCm.^[14] The accompanying blog post gives further detail:^[5]

Launch-day support required release-specific software. The vLLM team supplied the dedicated NVIDIA and AMD container images and said the model would reach nightly PyPI wheels only after its pull request (#61879) was reviewed and merged, which it said usually takes about a week.^[5] NVIDIA engineers contributed fused normalization, gating, and residual kernels and other optimizations across FlashInfer and vLLM. On TPU, vLLM reported megakernels reaching over 650 tokens per second on 4 TPU v7 chips and over 960 on 8 chips with DSpark speculative decoding (acceptance length 6), and over 270 and 350 tokens per second without it.^[5]

vLLM also supported LoRA reinforcement learning from launch, acting as the rollout engine with Verl FSDP2 as the trainer. In the team's run, GRPO training on DAPO-Math-17K for 150 steps on 16 GB300 GPUs raised AIME-2024 accuracy (mean@8) from 0.56 to 0.85.^[5]

SGLang and Miles

The SGLang team supports both Meta checkpoints and AMD's MXFP4 model, with target platforms including NVIDIA Blackwell GPUs (GB300, B300, GB200, B200), NVIDIA H200, AMD MI350X and MI355X, and Intel Xeon 6 with P-cores.^[15] Its features at launch include sequence parallelism combined with an NVFP4 MegaMoE kernel for faster prefill, separate encoder, prefill, and decode workers, HiCache, and DFlash V2 speculative decoding. In the team's measurements on four GB300 GPUs, sequence parallelism plus MegaMoE cut p90 time to first token by 28.3-41.9% compared with plain tensor parallelism. On an OpenHands multi-turn coding replay with the NVFP4 checkpoint on 8 B300 GPUs, it reported 14.8K total tokens per second per GPU at 543 tokens per second per user with one request, and 80.3K tokens per second per GPU at 61 tokens per second per user with 128 concurrent requests.^[15] The PyTorch team also shipped Helion megakernels for the model in SGLang.^[11]

RadixArk's Miles framework added full-parameter reinforcement learning. The SGLang team reported GRPO training of the full language model on DAPO-Math-17K on 64 H200 GPUs and on 32 B300 GPUs, and a Geometry3K run (vision encoder frozen) in which test accuracy rose from 88.2% to 94.9% over 30 updates.^[15]

TensorRT-LLM, Dynamo, and other runtimes

Meta's guide says TensorRT-LLM serves the NVFP4 checkpoint on a single four-GPU Blackwell node, and NVIDIA Dynamo handles multi-node deployments with separate prefill and decode workers.^[7] NVIDIA says the model was trained on GB300 NVL72 systems and reports up to 50K tokens per second per GPU and up to 600 tokens per second per user for agentic workloads with speculative decoding; these are NVIDIA's own figures. NVIDIA also offers a downloadable NIM container.^[17] Hugging Face added Transformers support for the architecture, and llama.cpp runs GGUF quantizations on CPU, GPU, or both.^[7][16]

Hosted access

Launch partners include Baseten, Fireworks AI, Groq (limited preview), Modal, Nebius, OpenRouter, and Together AI. Meta also lists Amazon SageMaker AI, Google Cloud Model Garden, and Microsoft Foundry Models as distribution catalogs.^[7] AWS said the model was available in SageMaker JumpStart at launch in US East (N. Virginia) and US East (Ohio), using ml.p5e.48xlarge instances for the NVFP4 variant and ml.p6-b300.48xlarge instances for BF16.^[18]

Prompting and tool use

Muse Spark Open always reasons before answering. Meta's prompting guide exposes four settings through reasoning_effort; xhigh is the default. Values such as none and max are rejected.^[8]

Hugging Face's launch post gives different integers for the two lower levels (16 for low and 64 for medium); the values above are from Meta's guide.^[8][16] An explicit numeric Reasoning strength line in the system prompt takes precedence over the request parameter. Servers configured with the muse reasoning parser separate reasoning from the final answer. Meta recommends streaming at higher strengths to avoid timeouts.^[8] The model addresses each of its messages to a recipient: itself for reasoning, commentary for progress updates, a tool name for a tool call, or the user for the final answer.^[8]

For tools, the application supplies function definitions, executes requested calls, and returns their results as tool messages. The model can request multiple calls in one response, but the surrounding application supplies the permissions and execution loop.^[8]

Evaluation

Meta's release table mixes internal runs, benchmark-provider results, and other vendors' self-reports, using xhigh for Muse Spark Open. Meta cautions that its setup may not be optimal for competing models.^[9] All Muse Spark Open figures below are vendor-reported.

Selected release-card results, with Kimi K3 (max reasoning) for comparison:^[2][9]

In the same table, GLM 5.3 has the highest Intelligence Index score (45), and Muse Spark Open trails several open models on Terminal-Bench 2.1, DeepSWE 1.1, and GPQA Diamond while leading the listed models on JobBench, MCP-Atlas, and HLE-Diamond.^[2]

Artificial Analysis' own leaderboard showed Muse Spark Open (xhigh) at 39 under Intelligence Index v4.3.2 in an Internet Archive capture taken at 01:04 UTC on October 27, 2026.^[25] The same day the firm moved to Intelligence Index v5.0, which uses a different set of evaluations, and it says that scores across index versions "are not intended for long-run comparison."^[23][24] On the v5.0 leaderboard read on October 29, 2026, Muse Spark Open (xhigh) scored 25.^[26]

Meta attributes DeepSWE to Artificial Analysis runs with Muse Code, while Terminal-Bench uses Terminus 2.^[9]

OSWorld checks task completion in real computer environments.^[10] Meta used a 361-task Ubuntu split with GUI-only tools, a 200-step limit, and four attempts per task, reporting mean reward.^[9]

Customization and inference optimization

The release includes training integrations as well as inference checkpoints. Meta's developer guide lists supervised fine-tuning, low-rank adaptation, and reinforcement-learning paths through TorchTitan, SGLang with Miles, NVIDIA NeMo AutoModel, Hugging Face tools, and managed training APIs from Fireworks AI and Together AI.^[7]

The PyTorch team released Helion implementations of operations including RoPE, RMSNorm, and quantization. It reported 1.03-3.95x speedups for individual kernels compared with custom kernels written in hardware-specific languages on NVIDIA GB300, and a 3% end-to-end throughput improvement on AMD MI350. Kernel speedups and complete-system throughput measure different things.^[11]

Speculative decoding

Speculative decoding is another supported path, and the launch shipped two separately trained draft models, one for each major open-source runtime.^[7]

DSpark (vLLM). DSpark was introduced in a July 2026 paper by researchers from Peking University and DeepSeek. It pairs a parallel drafting backbone (a DFlash network in the authors' implementation) with a lightweight sequential module that models dependencies between tokens in a draft block, and it schedules verification by confidence, choosing how many drafted tokens to verify per request based on estimated acceptance probability and the serving engine's throughput profile. Deployed in DeepSeek's DeepSeek-V4 serving system, the authors report 60-85% faster per-user generation than their MTP-1 baseline at matched throughput.^[19][20] Meta's developer blog instead describes DSpark as "a method NVIDIA and the vLLM team developed together".^[7] The Muse Spark Open drafter (Inferact/Muse-Spark-Open-DSpark) was trained by Inferact; its Hugging Face metadata lists about 5.3 billion parameters.^[5][22] It proposes up to seven tokens per pass for verification by Muse Spark Open, uses the target model's sliding-window attention to keep drafting cheap at long context, and can verify fewer drafted tokens when the server is busy (adaptive verification).^[5][7]

DFlash V2 (SGLang). RadixArk built a DFlash V2 drafter (RadixArk/Muse-Spark-Open-DFlash2) that adds dynamic convolutions within the draft block and a lightweight candidate selector.^[15] Meta reports 3.7x throughput for a single request on AMD MI355X and 2.9x at 16 concurrent requests, with an average of 6.4 accepted draft tokens per step.^[7]

License and access conditions

The Muse Model License grants rights to use, modify, fine-tune, distill, merge, distribute, and sell the software subject to its conditions. Copies and derivatives must retain the copyright notice, license, and usage policy. Products using the model or derivatives must display "Built with Muse Spark Open" in an allowed location such as the interface, documentation, website, or settings.^[3]

Section 3 requires the licensee to notify Meta and sign a separate written agreement before certain Model-as-a-Service offerings. It applies when the licensee or affiliates provide third-party access to AI models through an API or similar interface and their aggregate gross revenue exceeds USD 20 million during any consecutive 12-month period. The clause excludes internal use, research, commercial activities that are not Model as a Service, and use as a component of a product that is not such a service. It is not a blanket USD 20 million limit on every commercial use.^[3]

The license incorporates a usage policy and says Meta claims no ownership of generated outputs.^[3] The usage policy states that the model is not intended for individuals under 18.^[3]

Safety and limitations

Meta applied additional safety post-training for the open-weight release and says it evaluated the model under its Meta Superintelligence Scaling Framework, with a full evaluation report to follow.^[1] Its model card warns about inaccurate or biased responses, reasoning errors, language differences, and quantization effects.^[2]

The model card reports these results, from Meta's testing of the released checkpoint, except FORTRESS, which Meta took from Scale AI's leaderboard (the cyber tests were run with no system-level safeguards in place):^[2]

On FORTRESS, the low risk score comes with a higher over-refusal score than the other open models in Meta's table, which range from 2.7 (Kimi K3) to 13.3.^[2]

The prompting guide recommends video clips longer than a few seconds at 448 pixels or more on the short side; very short or low-resolution clips can trigger a response that the model cannot view them.^[8]

Meta's StrongREJECT v2 test adapts the original benchmark with 202 disallowed behaviors and multiple attack conditions.^[9] The original StrongREJECT research measures whether responses provide useful harmful information, rather than only whether a refusal was bypassed.^[12]

The card says results do not automatically transfer to modified models and recommends human confirmation for irreversible actions.^[2]

Continue reading

• OOpen weightsOpen weights is the practice of publishing the trained parameter tensors of a neural network so that anyone can download, run, fine-tune, and redistribute the model without obtaining a separate inference…Mentioned 4 times above
• MMeta Superintelligence LabsMeta Superintelligence Labs (MSL) is the artificial intelligence division that Meta created on June 30, 2025 to consolidate its research, foundation models, and AI products under a single organization aimed at…Mentioned twice above
• Muse SparkMuse Spark is a multimodal reasoning model developed by Meta Superintelligence Labs (MSL), the artificial intelligence division Meta reorganized in 2025.Mentioned 18 times above
• Artificial AnalysisArtificial Analysis is an independent benchmarking and analytics platform that evaluates artificial intelligence models and API providers across intelligence, speed, price, and latency, and it is best known…Mentioned 4 times above
• TTorchTitanTorchTitan (styled torchtitan in its repository) is an open-source, PyTorch-native platform for training large generative AI modelsLinks back to this article
• MuseMuse is a hosted personal AI agent developed by Meta Superintelligence Labs. Meta launched the service on September 8, 2026, powered by Muse Spark 1.3.Mentioned 35 times above

More to read

• Hugging Face
• Kimi K3
• DFlash
• Muse Code
• Perception Encoder
• DeepSeek

Topics:AI ModelsLarge Language ModelsMeta AIOpen Source AI

References

1. ^1 ^2 ^3 ^4 ^5 ^6Meta Superintelligence Labs. "Introducing Muse Spark Open, The Most Capable US Open-Weight Model to Date." October 15, 2026.
2. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11Meta Superintelligence Labs. "Muse Spark Open Model Card." Released October 15, 2026; accessed October 17, 2026.
3. ^1 ^2 ^3 ^4Meta Platforms. "Muse Model License," version Muse Spark Open, including Exhibit A usage policy. October 15, 2026.
4. ^1 ^2 ^3 ^4 ^5 ^6Albert Lee, Pavel Belevich, and Marylise Tauzia. "Muse Spark Open, an open-weight model for coding agents." Baseten, October 15, 2026.
5. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14vLLM Team. "Day 0 support for Muse Spark Open on vLLM." October 15, 2026.
6. ^1 ^2Meta. "Get the model." Muse Spark Open developer documentation. Accessed October 17, 2026.
7. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12Adam Loving et al. "Build with Muse Spark Open." Meta for Developers, October 15, 2026.
8. ^1 ^2 ^3 ^4 ^5 ^6 ^7Meta. "Prompting guide." Muse Spark Open developer documentation. Accessed October 17, 2026.
9. ^1 ^2 ^3 ^4 ^5Meta Superintelligence Labs. "Muse Spark Open Evaluation Methodology." October 2026.
10. ^Tianbao Xie et al. "OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments." arXiv:2404.07972.
11. ^1 ^2 ^3 ^4PyTorch Team at Meta. "Muse Spark Is Now an Open-Weights Model." PyTorch blog, October 15, 2026.
12. ^Alexandra Souly et al. "A StrongREJECT for Empty Jailbreaks." arXiv:2402.10260.
13. ^Meta. "Muse Spark Open on Mac." Developer cookbook, accessed October 17, 2026.
14. ^vLLM (@vllm_project). "Day-0 support for Meta's open-weight Muse Spark Open is live on vLLM." X, October 15, 2026.
15. ^1 ^2 ^3 ^4 ^5 ^6SGLang Team. "SGLang and Miles Add Day-0 Support for Muse Spark Open." LMSYS Org, October 15, 2026.
16. ^1 ^2 ^3Hugging Face. "Welcome Muse Spark Open: open-weight frontier multimodal mixture-of-experts model from Meta." Hugging Face blog, October 15, 2026.
17. ^Rajath Narasimha and Esther Kim. "Scale Agentic Coding Workflows with Meta's Muse Spark Open on NVIDIA GB300 NVL72." NVIDIA Technical Blog, October 15, 2026.
18. ^Aris Tsakpinis and Liam Neal Reilly. "Meta's Muse Spark Open now available in Amazon SageMaker JumpStart." AWS Machine Learning Blog, October 15, 2026.
19. ^Xin Cheng et al. "DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation." arXiv:2607.05147, July 6, 2026.
20. ^vLLM Team. "Parallel All the Way Down: Beyond Single-Token Generation with Speculative Decoding." vLLM blog, July 28, 2026.
21. ^1 ^2The Register. "Zuck's Muse to Spark joy with open weights release 'soon'." September 2, 2026.
22. ^Inferact. "Muse-Spark-Open-DSpark." Hugging Face model repository. Accessed October 17, 2026.
23. ^Artificial Analysis. "Announcing Artificial Analysis Intelligence Index v5." October 27, 2026.
24. ^Artificial Analysis. "Artificial Analysis Progress Index v1 Methodology." Accessed October 29, 2026.
25. ^Artificial Analysis. "LLM Leaderboard" (Intelligence Index v4.3.2 scores). Internet Archive capture of October 27, 2026, 01:04 UTC.
26. ^Artificial Analysis. "LLM Leaderboard" (Intelligence Index v5.0 scores). Accessed October 29, 2026.

Show all 26 references