# Spark-X2.6-4B

> Source: https://aiwiki.ai/wiki/spark_x2_6_4b
> Updated: 2026-10-22
> Categories: Large Language Models, Reasoning Models, Small Language Models
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "Spark-X2.6-4B." aiwiki.ai, 22 Oct 2026. https://aiwiki.ai/wiki/spark_x2_6_4b
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

**Spark-X2.6-4B** is a [large language model](https://aiwiki.ai/wiki/large_language_model) released by SparkLLM under the XHToken organization for local text generation and [AI agent](https://aiwiki.ai/wiki/ai_agents) applications. The project's release log dates its global launch to October 21, 2026, with downloads through [Hugging Face](https://aiwiki.ai/wiki/hugging_face), ModelScope and Ollama. Its repository provides the post-trained checkpoint, rather than only an announcement or a hosted demonstration.[1][8]

The model follows Spark-X2.5-4B. Its developers attribute the changes to improved post-training while retaining the predecessor's architecture, and identify coding, agent tasks and long-context work as areas of improvement. These are developer claims supported by the team's own evaluations. The release also provides three thinking settings, described as `none`, `low` and `high`.[1][2]

## Architecture and context

Although branded as a 4B model, the published Safetensors metadata records 4,112,079,360 parameters in bfloat16. The repository identifies the checkpoint as `Spark2_5ForCausalLM`, preserving the architecture identifier used by the preceding generation.[3][4]

| Property | Published value |
| --- | --- |
| Parameter count | 4,112,079,360[3] |
| Weight data type | bfloat16[3][4] |
| Hidden layers | 36[4] |
| Hidden dimension | 2,560[4] |
| Attention heads / key-value heads | 16 / 4[4] |
| Sliding-attention window | 512 tokens[4] |
| Maximum position embeddings | 1,048,576[4] |
| Vocabulary size | 131,072[4] |
| License | Apache License 2.0[9] |

The configuration alternates three [sliding-window attention](https://aiwiki.ai/wiki/sliding_window_attention) layers with one full-attention layer, repeating that pattern across all 36 layers. It enables a sigmoid gate on each attention head's output and uses tied word embeddings. Full-attention and sliding-attention layers have separate rotary-position settings, including different frequency bases and different proportions of each head subject to rotation.[4]

The 1,048,576-position setting is the configured [context window](https://aiwiki.ai/wiki/context_window). It should be distinguished from a measured level of accuracy at that length. The serving documentation uses the same maximum and explicitly says that sufficient device memory is required, with a smaller context setting available when necessary. The example's 131,072-token generation budget is an example setting, rather than a separately established universal output limit.[4][8]

## Thinking controls and tool format

The distributed chat template implements the advertised settings through two arguments. Thinking is enabled by default; when enabled, an omitted reasoning effort resolves to `high`.[5]

| Advertised setting | Chat-template arguments |
| --- | --- |
| `none` | `enable_thinking=false` |
| `low` | `enable_thinking=true`, `reasoning_effort="low"` |
| `high` | `enable_thinking=true`, `reasoning_effort="high"`, or omit the effort |

The template accepts only `low` and `high` as effort values when thinking is enabled. Thus, disabling thinking uses a Boolean switch; `none` is not a third accepted value for `reasoning_effort`. The template adds an effort instruction to the system message and opens or closes the thinking channel in the generation prompt. These controls do not assign a fixed number of reasoning tokens.[5]

For [tool use](https://aiwiki.ai/wiki/tool_use), the template serializes function definitions, assistant tool calls and tool responses. Function arguments must be dictionaries before template application; JSON strings require normalization. The surrounding application still has to execute the requested functions and return their results. The template defines a conversation format, rather than supplying a complete agent environment.[5]

## Reported evaluations

The team's benchmark chart evaluates Spark-X2.6-4B with thinking set to `high`. It recommends `temperature=1.0`, `top_p=0.95` and `top_k=-1`. Selected scores from the chart are shown below; they are developer-reported results.[6]

| Benchmark | Spark-X2.6-4B | Spark-X2.5-4B |
| --- | --- | --- |
| BFCL-V4 | 71.8 | 65.1 |
| SWE Verified | 62.8 | 41.6 |
| LongBench v2 | 53.3 | 48.7 |
| GPQA Diamond | 77.1 | 73.7 |

The full comparison combines the team's evaluations with some competitor scores taken from published model cards or papers, marked with an asterisk. It also includes SWebBench and PAgentBench, which the chart describes as in-house benchmarks that would be released later. The table therefore should not be read as an independently reproduced ranking, or as evidence for the performance of the `low` and `none` settings.[6]

## Local deployment and measured speed

The project documents Transformers, [SGLang](https://aiwiki.ai/wiki/sglang), [vLLM](https://aiwiki.ai/wiki/vllm), [llama.cpp](https://aiwiki.ai/wiki/llama_cpp), [Ollama](https://aiwiki.ai/wiki/ollama) and MLX-based deployment routes. Its Transformers example uses repository-specific model code, and the configuration maps the automatic model classes to Python files distributed with the checkpoint. The runtime and model format therefore matter when interpreting compatibility claims.[4][8]

The team's inference chart separates prompt processing from token decoding and identifies the hardware, runtime and [quantization](https://aiwiki.ai/wiki/quantization) for each series. On an RTX 4090, its llama.cpp Q4_K_M measurements report 219.34 decoded tokens per second at 4K context and 67.85 at 256K. On a 16-core Mac M5 Pro, the corresponding llama.cpp measurements are 76.14 and 23.43 tokens per second. Separate TensorRT INT4 and MLX INT4 series use different runtimes. These measurements describe the tested configurations, rather than a device-independent speed for the model.[7]

## References

1. SparkLLM Team. [Spark-X2.6 official repository, introduction and release log](https://github.com/XHToken/Spark-X2.6). GitHub, accessed October 23, 2026.
2. SparkLLM (@SparkLLM). [Spark-X2.6-4B release announcement](https://x.com/SparkLLM/status/2112614036227408363). X, October 20, 2026 UTC.
3. Hugging Face. [XHToken/Spark-X2.6-4B repository metadata](https://huggingface.co/api/models/XHToken/Spark-X2.6-4B). Parameter count, data type and repository identity, accessed October 23, 2026.
4. XHToken. [Spark-X2.6-4B config.json](https://huggingface.co/XHToken/Spark-X2.6-4B/blob/main/config.json). Hugging Face, accessed October 23, 2026.
5. XHToken. [Spark-X2.6-4B chat_template.jinja](https://huggingface.co/XHToken/Spark-X2.6-4B/blob/main/chat_template.jinja). Hugging Face, accessed October 23, 2026.
6. XHToken. [Spark-X2.6 benchmark results and evaluation notes](https://huggingface.co/XHToken/Spark-X2.6-4B/resolve/main/images/all_benchmark.svg). Developer evaluation chart, accessed October 23, 2026.
7. XHToken. [Spark-X2.6-4B inference speed](https://huggingface.co/XHToken/Spark-X2.6-4B/resolve/main/images/inference.svg). Developer throughput chart, accessed October 23, 2026.
8. SparkLLM Team. [Spark-X2.6-4B model card](https://huggingface.co/XHToken/Spark-X2.6-4B). Hugging Face, accessed October 23, 2026.
9. XHToken. [Spark-X2.6-4B LICENSE](https://huggingface.co/XHToken/Spark-X2.6-4B/blob/main/LICENSE). Apache License, Version 2.0, accessed October 23, 2026.