ERNIE 5.5 Flash Base

ERNIE 5.5 Flash Base is a text-only large language model released by Baidu on November 5, 2026. It is a sparse mixture of experts (MoE) checkpoint taken directly after pre-training, with 280 billion advertised total parameters and 14 billion activated per token. Baidu released FP8 and BF16 weights for further training and text completion.^[1][2][11]

The Base checkpoint has no mid-training, instruction tuning or post-training. Its serving documentation distinguishes it from a forthcoming ERNIE 5.5 Flash chat model with reasoning, tool calling and a separate draft model. Those announced chat capabilities should not be attributed to the released Base weights.^[5][6]

Release and specifications

The model belongs to the Baidu ERNIE family. Its two official repositories contain the FP8 and BF16 versions of the same Base model.^[2]

Sources: official model card and released configuration.^[2][3] The card's parameter convention includes input embeddings and the output head in the total, includes the output head but excludes embedding lookups in the activated count, and excludes an additional 0.36 billion ECHO parameters from both.^[2]

The Hub's Safetensors metadata at revision 028544b2ffceea189b2d8768ed2b9d09b06f2343 reports 286,245,697,882 stored tensor elements, differing from the advertised 280-billion model figure. These are different published counts; the repository metadata does not explain the discrepancy. Its 128 FP8 Safetensors shards total 288,434,400,632 bytes, about 288.4 GB in decimal units. File storage is separate from runtime memory, which also has to accommodate cached context and temporary allocations.^[10]

Hybrid attention

The backbone combines 37 Heavily Compressed Attention (HCA) layers with six layers using Multi-head Latent Attention and DeepSeek Sparse Attention. HCA maintains a compressed history with one cached entry for each 128 tokens, alongside a window of recent uncompressed tokens. The sparse MLA layers retain token-level history and select positions through an indexer.^[1][3]

The configuration sets the sparse selection to 2,048 positions. HCA uses 512-dimensional heads; the MLA branch uses 256-dimensional query/key and value heads. Fixed top-k selection limits the core attention computation over selected positions, but does not establish constant cost for the entire model or all processing of a growing context.^[1][3]

Each attention layer also has a gate computed from a low-rank query representation and a Virtual Head Attention (VHA) mixing operation. The VHA paper describes linear mixing across heads that can be folded into inference operations. Its experiments concern other model scales and training runs; their quality and cache improvements are not independent measurements of this ERNIE checkpoint.^[1][7]

Experts, residual connections and prediction

Multi-Router Latent MoE uses several independently initialized sigmoid routers to select experts. Tokens move into a 2,048-dimensional latent representation for expert computation, with a shared projection returning results to the 4,096-dimensional hidden representation. The architecture uses auxiliary-loss-free load balancing, SiTU-GLU activations and normalization around the latent expert path.^[2][3]

The related Multi-View Latent MoE paper separately studies a Multi-Router submodule and additional learned latent views. The release identifies ERNIE's component as Multi-Router Latent MoE; the paper's full Multi-View system and its smaller-model results should not be assumed to describe this checkpoint in every respect.^[2][8]

Manifold-constrained hyper-connections replace the ordinary residual stream with four parallel streams and constrained mixing. A MultiMax output head uses a learned piecewise temperature to rescale the vocabulary logits.^[1][3]

ECHO reuses the backbone's final layer for future-token prediction, instead of relying on a separate conventional multi-token prediction module. The released model was trained to predict one additional token. Its serving implementations can reuse that layer as a draft for speculative decoding, including several draft steps. The ECHO research paper explains the objective and depth-sampling method, while runtime-specific behavior remains documented separately.^[1][6][9]

Pre-training and intended use

Baidu says it trained the checkpoint from scratch using the Muon optimizer and mixed FP8 precision. The corpus combines multilingual web material, code repositories and long-context data. Code examples include software changes and execution-related material, while long-context data includes natural documents, distractor passages and synthetic retrieval or question-answering tasks.^[1]

The release is intended as a starting point for domain adaptation, fine-tuning and reinforcement-learning research. A pre-training checkpoint can continue text without having the conversational behavior or tool-use conventions of an instruction-tuned assistant. Having a million-token configuration limit also does not establish reliable retrieval or reasoning over every input of that length.^[1][5]

Reported evaluations

Baidu evaluated the released Base checkpoint against three other base models: DeepSeek-V4-Flash-Base, Hy3-preview-Base and MiMo-V2.5-Base. It reports using a common evaluation harness, prompts and few-shot examples, with the same model judge for free-form questions. The following are selected developer-reported results, not an independently reproduced leaderboard.^[1][2]

The results vary by task. ERNIE's reported lead on several knowledge and coding tests does not extend to every row, and base-model few-shot scores should remain separate from post-trained assistant or agent evaluations.^[2]

Serving the released model

The official SGLang cookbook provides hardware-specific FP8 and BF16 recipes. It identifies the single-node FP8 recipes for B300, GB300, B200, GB200 and H100, and BF16 on B300, as verified. Other BF16 configurations follow estimated memory budgets and were not yet verified. Long-context operation requires additional activation and cache memory, along with reduced concurrency.^[5]

Baidu's vLLM-ERNIE plugin runs with vLLM 0.32.0 on NVIDIA GPUs. Its instructions use the completions endpoint, /v1/completions, for the Base model. The repository reserves reasoning and tool-call parsers and DSpark examples for the forthcoming chat model; Base has ECHO but no bundled DSpark draft.^[6]

The vLLM recipe also distinguishes fitting the weights from serving the maximum context. It says 80 GB GPUs can support an eight-GPU FP8 configuration with a shorter context, while its million-token example requires larger-memory hardware and leaves room for temporary allocations. These instructions describe particular configurations, rather than a universal minimum GPU count.^[6]

License

The weights use the custom ERNIE 5.5 Flash Base Model License Agreement v1.0. It excludes the European Union, United Kingdom and South Korea from its permitted territory.^[4]

Noncommercial use has no model-license charge. Commercial use with related rolling annual revenue at or below US$50 million requires a one-time notice and the attribution Powered by ERNIE. Above that threshold, prior written commercial authorization is required. Related revenue includes the whole revenue of a product using the model, with aggregation across entities under common control. The license also restricts use of the materials and outputs to improve other models, except its permitted derivative models.^[4]

These conditions apply to the weights independently of serving software. The SGLang cookbook's Apache-2.0 description conflicts with Baidu's model-specific license; the official model card points to the custom agreement. The vLLM-ERNIE plugin's own Apache-2.0 license covers its code.^[2][4][5][6]

Continue reading

• BaiduBaidu, Inc. (NASDAQ: BIDU; HKEX: 9888) is a Chinese technology company headquartered in Beijing that operates China's leading search engine and has rebuilt itself around artificial intelligence across four…Mentioned 9 times above
• Large Language ModelA large language model (LLM) is an artificial intelligence system built on a transformer neural network with billions to trillions of parameters, trained on massive text corpora to predict the next token in a…Linked in the introduction
• Mixture of Experts (MoE)A mixture of experts (MoE) is a machine-learning architecture that contains multiple component functions called experts and a gating or routing mechanism that determines how their outputs contribute to a…Linked in the introduction
• Baidu ERNIEBaidu ERNIE (Enhanced Representation through Knowledge Integration) is the family of large language and multimodal foundation models built by the Chinese technology company Baidu, spanning the original 2019…Links back to this article
• vLLMvLLM is an open-source inference and serving engine for large language models. It is software for running already trained models, either through a Python interface for offline work or through a network server…Mentioned 4 times above
• SGLangSGLang (short for Structured Generation Language) is a high-performance, open-source serving framework for large language models and multimodal models, originally developed at UC Berkeley's Sky Computing Lab…Mentioned twice above

More to read

• Multi-head Latent Attention
• Speculative Decoding
• Multi-token prediction
• Jamba
• Jamba2
• DeepSeek V3

Topics:Chinese AILarge Language ModelsMixture of ExpertsModel Architecture

References

1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9Baidu ERNIE Team. ERNIE 5.5 Flash Base. Official technical release, November 5, 2026.
2. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9Baidu. ERNIE-5.5-Flash-Base model card. Hugging Face, revision inspected November 5, 2026.
3. ^1 ^2 ^3 ^4 ^5Baidu. Released model configuration. Hugging Face, inspected November 5, 2026.
4. ^1 ^2 ^3Baidu. ERNIE 5.5 Flash Base Model License Agreement v1.0. Inspected November 5, 2026.
5. ^1 ^2 ^3 ^4SGLang. ERNIE-5.5-Flash deployment cookbook. Inspected November 5, 2026.
6. ^1 ^2 ^3 ^4 ^5Baidu. vLLM-ERNIE. Official inference plugin and deployment guidance, inspected November 5, 2026.
7. ^Xibo Ding et al. Virtual Head Attention. arXiv:2610.17959, October 13, 2026.
8. ^Zelin Zhao et al. Multi-View Latent MoE: Letting Every Router Shape Expert Computation. arXiv:2610.40577, October 29, 2026; revised November 2, 2026.
9. ^Yuxuan Zhou et al. ECHO: Efficient Causal Self-Recursion Beyond Auxiliary Multi-Token Prediction. arXiv:2610.24766, October 17, 2026.
10. ^Hugging Face. Official repository metadata and file sizes. Revision `028544b2ffceea189b2d8768ed2b9d09b06f2343`, retrieved November 5, 2026.
11. ^Baidu Inc. Base-model release announcement. X, November 5, 2026.

Show all 11 references