# ERNIE 5.5 Flash Base

> Source: https://aiwiki.ai/wiki/ernie_5_5_flash_base
> Updated: 2026-11-05
> Categories: Chinese AI, Large Language Models, Mixture of Experts, Model Architecture
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "ERNIE 5.5 Flash Base." aiwiki.ai, 5 Nov 2026. https://aiwiki.ai/wiki/ernie_5_5_flash_base
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

**ERNIE 5.5 Flash Base** is a text-only [large language model](https://aiwiki.ai/wiki/large_language_model) released by [Baidu](https://aiwiki.ai/wiki/baidu) on November 5, 2026. It is a sparse [mixture of experts](https://aiwiki.ai/wiki/mixture_of_experts) (MoE) checkpoint taken directly after pre-training, with 280 billion advertised total parameters and 14 billion activated per token. Baidu released FP8 and BF16 weights for further training and text completion.[1][2][11]

The Base checkpoint has no mid-training, instruction tuning or post-training. Its serving documentation distinguishes it from a forthcoming ERNIE 5.5 Flash chat model with reasoning, tool calling and a separate draft model. Those announced chat capabilities should not be attributed to the released Base weights.[5][6]

## Release and specifications

The model belongs to the [Baidu ERNIE](https://aiwiki.ai/wiki/baidu_ernie) family. Its two official repositories contain the FP8 and BF16 versions of the same Base model.[2]

| Property | Published value |
| --- | --- |
| FP8 repository | `baidu/ERNIE-5.5-Flash-Base` |
| BF16 repository | `baidu/ERNIE-5.5-Flash-Base-BF16` |
| Training corpus | 33.5 trillion tokens |
| Maximum pre-training context | 131,072 tokens |
| Fine-tuning and inference context | Up to 1,048,576 tokens |
| Backbone layers | 43: one dense feed-forward layer and 42 MoE layers |
| Hidden size | 4,096 |
| Attention heads | 64 |
| Routed experts per MoE layer | 512; ten selected per token |
| Shared experts per MoE layer | One |
| MoE latent and expert intermediate dimensions | 2,048 |
| Vocabulary | 201,216 entries |
| Model class | `Ernie5_5_MoeForCausalLM` |

Sources: official model card and released configuration.[2][3] The card's parameter convention includes input embeddings and the output head in the total, includes the output head but excludes embedding lookups in the activated count, and excludes an additional 0.36 billion ECHO parameters from both.[2]

The Hub's Safetensors metadata at revision `028544b2ffceea189b2d8768ed2b9d09b06f2343` reports 286,245,697,882 stored tensor elements, differing from the advertised 280-billion model figure. These are different published counts; the repository metadata does not explain the discrepancy. Its 128 FP8 Safetensors shards total 288,434,400,632 bytes, about 288.4 GB in decimal units. File storage is separate from runtime memory, which also has to accommodate cached context and temporary allocations.[10]

## Hybrid attention

The backbone combines 37 Heavily Compressed Attention (HCA) layers with six layers using [Multi-head Latent Attention](https://aiwiki.ai/wiki/multi-head_latent_attention) and [DeepSeek Sparse Attention](https://aiwiki.ai/wiki/deepseek_sparse_attention). HCA maintains a compressed history with one cached entry for each 128 tokens, alongside a window of recent uncompressed tokens. The sparse MLA layers retain token-level history and select positions through an indexer.[1][3]

The configuration sets the sparse selection to 2,048 positions. HCA uses 512-dimensional heads; the MLA branch uses 256-dimensional query/key and value heads. Fixed top-k selection limits the core attention computation over selected positions, but does not establish constant cost for the entire model or all processing of a growing context.[1][3]

Each attention layer also has a gate computed from a low-rank query representation and a Virtual Head Attention (VHA) mixing operation. The VHA paper describes linear mixing across heads that can be folded into inference operations. Its experiments concern other model scales and training runs; their quality and cache improvements are not independent measurements of this ERNIE checkpoint.[1][7]

## Experts, residual connections and prediction

Multi-Router Latent MoE uses several independently initialized sigmoid routers to select experts. Tokens move into a 2,048-dimensional latent representation for expert computation, with a shared projection returning results to the 4,096-dimensional hidden representation. The architecture uses auxiliary-loss-free load balancing, SiTU-GLU activations and normalization around the latent expert path.[2][3]

The related Multi-View Latent MoE paper separately studies a Multi-Router submodule and additional learned latent views. The release identifies ERNIE's component as Multi-Router Latent MoE; the paper's full Multi-View system and its smaller-model results should not be assumed to describe this checkpoint in every respect.[2][8]

Manifold-constrained hyper-connections replace the ordinary residual stream with four parallel streams and constrained mixing. A MultiMax output head uses a learned piecewise temperature to rescale the vocabulary logits.[1][3]

ECHO reuses the backbone's final layer for future-token prediction, instead of relying on a separate conventional [multi-token prediction](https://aiwiki.ai/wiki/multi_token_prediction) module. The released model was trained to predict one additional token. Its serving implementations can reuse that layer as a draft for [speculative decoding](https://aiwiki.ai/wiki/speculative_decoding), including several draft steps. The ECHO research paper explains the objective and depth-sampling method, while runtime-specific behavior remains documented separately.[1][6][9]

## Pre-training and intended use

Baidu says it trained the checkpoint from scratch using the [Muon optimizer](https://aiwiki.ai/wiki/muon_optimizer) and mixed FP8 precision. The corpus combines multilingual web material, code repositories and long-context data. Code examples include software changes and execution-related material, while long-context data includes natural documents, distractor passages and synthetic retrieval or question-answering tasks.[1]

The release is intended as a starting point for domain adaptation, [fine-tuning](https://aiwiki.ai/wiki/fine_tuning) and reinforcement-learning research. A pre-training checkpoint can continue text without having the conversational behavior or tool-use conventions of an instruction-tuned assistant. Having a million-token configuration limit also does not establish reliable retrieval or reasoning over every input of that length.[1][5]

## Reported evaluations

Baidu evaluated the released Base checkpoint against three other base models: DeepSeek-V4-Flash-Base, Hy3-preview-Base and MiMo-V2.5-Base. It reports using a common evaluation harness, prompts and few-shot examples, with the same model judge for free-form questions. The following are selected developer-reported results, not an independently reproduced leaderboard.[1][2]

| Benchmark and metric | Setting | ERNIE | DeepSeek | Hy3 | MiMo |
| --- | --- | --- | --- | --- | --- |
| MMLU, exact match | Five-shot | 93.3 | 89.4 | 87.5 | 87.6 |
| MMLU-Pro, exact match | Five-shot | 78.4 | 69.2 | 66.9 | 65.7 |
| ChineseSimpleQA, model judge | Ten-shot | 73.0 | 75.3 | 73.2 | 68.8 |
| GPQA, exact match | Five-shot | 60.2 | 55.9 | 50.2 | 51.4 |
| BBH, exact match with chain of thought | Three-shot | 89.2 | 87.5 | 90.0 | 89.8 |
| HumanEval+, pass@1 | One-shot | 82.3 | 69.5 | 78.0 | 74.4 |
| FullStackBench, pass@1 | One-shot | 54.6 | 43.3 | 56.7 | 54.1 |
| HELMET-Rerank, NDCG@10 | Zero-shot | 56.2 | 60.0 | 55.7 | 56.6 |
| LongBench-v2, exact match | Zero-shot | 48.7 | 44.5 | 39.8 | 44.6 |
| InfiniteBench, average | Zero-shot | 46.5 | 48.0 | 33.0 | 46.7 |

The results vary by task. ERNIE's reported lead on several knowledge and coding tests does not extend to every row, and base-model few-shot scores should remain separate from post-trained assistant or agent evaluations.[2]

## Serving the released model

The official [SGLang](https://aiwiki.ai/wiki/sglang) cookbook provides hardware-specific FP8 and BF16 recipes. It identifies the single-node FP8 recipes for B300, GB300, B200, GB200 and H100, and BF16 on B300, as verified. Other BF16 configurations follow estimated memory budgets and were not yet verified. Long-context operation requires additional activation and cache memory, along with reduced concurrency.[5]

Baidu's `vLLM-ERNIE` plugin runs with [vLLM](https://aiwiki.ai/wiki/vllm) 0.32.0 on NVIDIA GPUs. Its instructions use the completions endpoint, `/v1/completions`, for the Base model. The repository reserves reasoning and tool-call parsers and DSpark examples for the forthcoming chat model; Base has ECHO but no bundled DSpark draft.[6]

The vLLM recipe also distinguishes fitting the weights from serving the maximum context. It says 80 GB GPUs can support an eight-GPU FP8 configuration with a shorter context, while its million-token example requires larger-memory hardware and leaves room for temporary allocations. These instructions describe particular configurations, rather than a universal minimum GPU count.[6]

## License

The weights use the custom **ERNIE 5.5 Flash Base Model License Agreement v1.0**. It excludes the European Union, United Kingdom and South Korea from its permitted territory.[4]

Noncommercial use has no model-license charge. Commercial use with related rolling annual revenue at or below US$50 million requires a one-time notice and the attribution `Powered by ERNIE`. Above that threshold, prior written commercial authorization is required. Related revenue includes the whole revenue of a product using the model, with aggregation across entities under common control. The license also restricts use of the materials and outputs to improve other models, except its permitted derivative models.[4]

These conditions apply to the weights independently of serving software. The SGLang cookbook's Apache-2.0 description conflicts with Baidu's model-specific license; the official model card points to the custom agreement. The `vLLM-ERNIE` plugin's own Apache-2.0 license covers its code.[2][4][5][6]

## References

1. Baidu ERNIE Team. [ERNIE 5.5 Flash Base](https://ernie.baidu.com/release/ernie-5-5-flash-base/). Official technical release, November 5, 2026.
2. Baidu. [ERNIE-5.5-Flash-Base model card](https://huggingface.co/baidu/ERNIE-5.5-Flash-Base/blob/028544b2ffceea189b2d8768ed2b9d09b06f2343/README.md). Hugging Face, revision inspected November 5, 2026.
3. Baidu. [Released model configuration](https://huggingface.co/baidu/ERNIE-5.5-Flash-Base/blob/028544b2ffceea189b2d8768ed2b9d09b06f2343/config.json). Hugging Face, inspected November 5, 2026.
4. Baidu. [ERNIE 5.5 Flash Base Model License Agreement v1.0](https://huggingface.co/baidu/ERNIE-5.5-Flash-Base/blob/028544b2ffceea189b2d8768ed2b9d09b06f2343/LICENSE). Inspected November 5, 2026.
5. SGLang. [ERNIE-5.5-Flash deployment cookbook](https://docs.sglang.io/cookbook/autoregressive/Baidu/ERNIE-5.5-Flash). Inspected November 5, 2026.
6. Baidu. [vLLM-ERNIE](https://github.com/baidu/vLLM-ERNIE). Official inference plugin and deployment guidance, inspected November 5, 2026.
7. Xibo Ding et al. [Virtual Head Attention](https://arxiv.org/abs/2610.17959). arXiv:2610.17959, October 13, 2026.
8. Zelin Zhao et al. [Multi-View Latent MoE: Letting Every Router Shape Expert Computation](https://arxiv.org/abs/2610.40577). arXiv:2610.40577, October 29, 2026; revised November 2, 2026.
9. Yuxuan Zhou et al. [ECHO: Efficient Causal Self-Recursion Beyond Auxiliary Multi-Token Prediction](https://arxiv.org/abs/2610.24766). arXiv:2610.24766, October 17, 2026.
10. Hugging Face. [Official repository metadata and file sizes](https://huggingface.co/api/models/baidu/ERNIE-5.5-Flash-Base?blobs=true). Revision `028544b2ffceea189b2d8768ed2b9d09b06f2343`, retrieved November 5, 2026.
11. Baidu Inc. [Base-model release announcement](https://x.com/Baidu_Inc/status/2118265920665587935). X, November 5, 2026.