AliceAI-Foundation-80B-A3B-Base (announced in English as "Alice AI Foundation 80B-A3B Base") is an open-weights base large language model from Yandex, published on Hugging Face and announced on 21 September 2026 under the Apache License 2.0.[1][3][4] It is a sparse mixture-of-experts (MoE) model with 80 billion total parameters, of which about 3 billion are active for each token, and a maximum context length of 262,144 tokens.[1] Yandex describes it as a pretrained model that was "trained entirely from scratch", and its Russian-language launch post adds that no weights from third-party open-source models were used.[1][3] That distinguishes it from Yandex's previous closed Alice AI LLM 235B, whose training started from the weights of Alibaba's Qwen model Qwen3-235B-A22B.[3]

The model uses a hybrid attention design in which three linear-attention layers based on Kimi Delta Attention alternate with one gated full-attention layer, and every block ends in a 512-expert MoE layer.[1][2] Alongside the weights, Yandex published a long technical write-up on Habr, two Russian factual-knowledge benchmarks (WikiWebFacts and HardMultiQA), a fine-tuning chat template and a LoRA example script.[1][3][6][7]

Background

Alice AI is the name of Yandex's AI assistant; Yandex calls the fast answers Alice AI gives in Yandex Search its most widely used generative product.[9] According to the Foundation launch post, the company's previous large model, Alice AI LLM 235B (base released in October 2025 and described in a December technical report), was built by continuing training from Qwen3-235B-A22B.[3] For the new model the team rebuilt the training corpus, re-chose the architecture, fitted hyperparameters with its own scaling laws and added reasoning and agentic data; Yandex says the whole path to release took about six months.[3]

Yandex frames the release as "another step" toward what it calls a unified reasoning model (in Russian, единая рассуждающая модель), on which the agentic capabilities of Alice AI are to be developed.[3] Yandex has not described the published checkpoint as the model that serves Alice AI users; it is a base model and the repository deliberately ships no default chat template.[1]

The release followed a related one. On 11 September 2026 the Alice AI Search team open-sourced AliceAI-T5-35B-A0.6B, an encoder-decoder MoE pretrain behind those fast answers in Yandex Search (see below). The Foundation post says an updated pretraining dataset underlies both models.[3][9]

Release

Item	Detail
Developer	Yandex (copyright holder: YANDEX LLC) [5]
Repository	`yandex/AliceAI-Foundation-80B-A3B-Base` [1]
Public announcement	21 September 2026 (Habr post and @yandexcom on X) [3][4]
Model type	Autoregressive base (pretrained) language model; custom `alice_ai` architecture loaded with `trust_remote_code` [1][2]
Parameters	80B total, 3B active per token [1]; 81,286,433,408 BF16 parameters in the safetensors files [15]
Context length	262,144 tokens [1][2]
Languages	Russian and English [1]
License	Apache 2.0 [5]
Weights	BF16, 49 safetensors shards [10]

The repository also contains an export-control notice asking users to comply with applicable export-control law, and a contributor notice requiring external contributors to accept the Yandex Contributor License Agreement.[10] The model card, written in Russian with an English translation, lists the tags custom_code, mixture-of-experts and vllm.[1]

Architecture

The model card and config.json give the following configuration.[1][2]

Component	Value
Layers	48
Layer layout	12 x (3 x (KDA, then MoE) followed by 1 x (Gated Attention, then MoE))
Hidden size	2,048
Vocabulary	129,024 tokens
KDA (linear attention) layers	36 of 48; 32 query heads and 32 KV heads, head dimension 128, short convolution kernel size 4
Gated Attention (full attention) layers	12 of 48; 16 query heads, 2 KV heads, head dimension 256
Positional encoding	RoPE with theta 1,000,000 and `partial_rotary_factor` 0.25
MoE experts	512 routed experts per MoE layer, top-10 routing, plus 1 shared expert
Expert intermediate size	512 (shared expert also 512)
Router	Sigmoid scoring with bias correction
Multi-token prediction	1 MTP layer
Activation	SiLU
Embeddings	Not tied to the output head

Hybrid attention. Kimi Delta Attention (KDA) is the linear-attention mechanism introduced by Moonshot AI in Kimi Linear.[3][11] The Habr post explains the motivation: in full attention the KV cache grows with sequence length, while KDA folds history into a fixed-size state, so its memory use does not grow with context. Unlike Gated DeltaNet in Qwen3-Next, which applies one decay coefficient per attention head, KDA uses a separate coefficient for each row of the state matrix.[3] Yandex reports that in its experiments on 5-15B models, replacing Gated DeltaNet with KDA lowered loss by about 0.02 and improved several task groups, including roughly 5 points on MMLU-Pro CoT.[3]

Attention Residuals. Instead of summing the outputs of earlier layers in the residual stream, the model uses Attention Residuals (AttnRes), a method from a Kimi Team paper that computes a learned, per-token softmax weighting over depth.[3][12] Yandex uses the block variant with a block of eight layers (four transformer blocks, counting attention and MoE sublayers separately) and aligned activation-checkpoint boundaries with the AttnRes operations to limit saved activations. On 5-15B test models, Block AttnRes lowered loss by about 0.017 and added about 3 points on MMLU-Pro CoT.[3]

Routing. MoE routing follows the auxiliary-loss-free, bias-based load balancing of DeepSeek-V3.[3]

Training

The launch post describes four sequential training stages.[3]

Stage	Tokens	Context length	Learning rate	Batch size
Stage 1: main pretraining	17.5T	8,192	8.2e-4	33M
Stage 2: context extension	50B	32,768	4.1e-5	16M
Stage 2: long context	240B	262,144	4.1e-5	16M
Stage 3: reasoning and agentic data	280B	131,072	4.1e-5	16M

Yandex put most of the long-context compute into the 256K stage because extending the 32K stage did not noticeably help long-context benchmarks. The final stage was shortened to 128K context to save compute, on the grounds that the longest reasoning trajectories in its data were often noisy.[3] Only the checkpoint after the reasoning stage is released.[3]

Ablations. Yandex validated its changes in a series of from-scratch runs of 2 trillion tokens each.[1] It compared four configurations in sequence: the previous corpus with a Qwen3-Next architecture, the new corpus with that architecture, the new corpus with Yandex's own architecture, and finally hyperparameters predicted by its scaling laws. The post reports that the new corpus improved all 13 benchmarks in the comparison and that the full configuration had the best macro-average and the best score on 7 of 13.[3] The scaling-law fits used models of about 5B and 10B total parameters trained on 100B to 600B tokens.[3]

Optimizer and stability. Training used the Muon optimizer with a linear learning-rate schedule; Yandex wrote a distributed, pipelined Muon step that it says roughly halved optimizer-step time in the final run.[3] A test run at the learning rate chosen by the scaling law produced a sharp loss spike together with fast growth of the activations at the output of the MoE down-projection. The team tried a router Z-loss, DeepSeek-style routing (kept because it improved quality), a hard-clipped SwiGLU of the kind used in gpt-oss (rejected because it hurt quality) and Gated RMSNorm (stable together with Attention Residuals but about 5% slower). The final fix moved the Z-loss idea from router logits to the MoE output activations, which Yandex says kept training stable without a quality loss or slowdown.[3]

Training data

The corpus mixes natural data (web pages, public code, books, scientific papers, news, multilingual material and open datasets) with semi-synthetic data (generated or corrected solutions and augmented versions of the most useful web pages) and fully synthetic tasks and solutions.[3] It started from the corpus of YandexGPT 5 Lite Base plus datasets prepared for Alice AI LLM 235B. Yandex reports that augmentation recipes that worked when continuing training from open models had to be reworked for a longer from-scratch pretrain, and it describes a pipeline for collecting and paraphrasing factual documents, comparing the approach to one described in the Kimi K2 technical report.[3]

The final stage added reasoning traces and agent data. Yandex's stated finding was that hard-reasoning data alone did not produce agentic skills: a reasoning-only checkpoint trailed a Qwen base model on most agent benchmarks after the same supervised fine-tuning, and adding tool-interaction data to pretraining closed much of the gap and made later RLVR training more stable.[3] The agent data included about 7 billion tokens of trajectories in synthetic CRUD environments built with a pipeline modelled on the Agent World Model paper, about 1 billion tokens of multi-hop search trajectories generated from 10,000 questions, and software-engineering data following the daVinci-Dev approach.[3]

Benchmarks

All figures below are Yandex's own measurements of base (pretrained) checkpoints, run on its internal evaluation infrastructure with vLLM.[1][3]

Knowledge and core skills

This table was run at temperature 0 for every model. Most tasks are 5-shot; CultCat is 4-shot and the two coding tasks are 1-shot. TriviaQA and MATH-500 are graded with an LLM-as-a-judge instead of exact match.[1] The Alice AI LLM 235B column appears only in the Habr post, not on the model card.[3]

Benchmark	AliceAI-Foundation-80B-A3B-Base	Alice AI LLM 235B (Oct 2025) Base	Qwen3.5-35B-A3B-Base	GLM-4.5-Air-Base (106B-A12B)	Nemotron-3-Super-120B-A12B-Base	DeepSeek-V4-Flash-Base (284B-A13B)
WikiWebFacts (RU)	86.5	86.2	62.4	70.2	72.8	83.2
HardMultiQA (RU)	67.9	65.9	47.2	48.6	54.5	65.4
CultCat (RU)	86.5	81.2	59.2	59.1	66.3	80.7
TriviaQA (EN)	79.0	84.0	71.4	83.5	89.8	89.4
EduBench Russian	74.2	71.1	42.9	39.0	44.0	67.7
EduBench Literature	73.8	72.0	51.8	51.4	55.8	69.1
EduBench History	82.0	76.7	65.9	62.8	70.2	76.9
EduBench English	76.1	74.9	71.7	67.2	71.3	82.9
ExpertFactsQA Medicine	63.6	59.9	59.0	50.6	42.3	60.7
ExpertFactsQA Law	49.6	46.5	27.9	22.5	24.3	40.5
EGE CoT (RU)	90.5	93.3	84.7	77.8	84.3	90.3
MMLU-Pro CoT	66.8	68.2	63.2	58.4	69.9	66.5
SuperGPQA CoT	44.3	43.5	43.6	35.4	46.6	46.1
MATH-500	91.1	72.6	81.9	60.2	84.8	80.7
EduBench Math	79.3	75.3	80.0	56.9	69.7	76.3
EduBench Math University	70.1	63.0	69.9	51.4	67.4	68.6
BigCodeBench 1-shot pass@1	48.3	49.8	43.5	44.5	48.8	49.1
LiveCodeBench v5-6 CoT 1-shot pass@1	50.5	15.4	50.4	22.6	50.4	38.1
FinQA 128k	74.1	44.6	73.5	35.5	71.7	74.1
LongMemEval 128k	64.6	31.4	55.6	50.6	64.8	68.0

Bold marks the best score in each row across all six columns. Among the five open base models on the model card, Yandex's model has the top score (or a tie) on 13 of the 20 rows. The larger models lead on English TriviaQA, EduBench English, MMLU-Pro, SuperGPQA, BigCodeBench and LongMemEval, and Qwen3.5-35B-A3B leads on EduBench Math.[1] The Habr post itself says the model is first on 8 of the 10 factual, educational and expert-knowledge benchmarks.[3] The EduBench tasks are built from queries submitted to Alice, ExpertFactsQA was written by domain experts, EGE is based on multiple-choice tasks from Russia's Unified State Exam, and FinQA 128k is Yandex's long-context version of the FinQA financial-report benchmark.[1][3] CultCat is an earlier Yandex benchmark of cultural knowledge.[1]

Yandex notes that every model in the comparison except Qwen3.5-35B-A3B has more total and more active parameters than its own; the Qwen model has fewer total parameters and the same number of active ones.[3] Against its own previous closed model, the post says the new model is ahead on factual knowledge, mathematics, coding and long-context work while having almost three times fewer total parameters and about seven times fewer active parameters.[3] The 235B model still scores higher on EGE, TriviaQA, MMLU-Pro and BigCodeBench in the same table.[3]

Reasoning benchmarks for base models

For harder math and programming tasks, Yandex scores base models zero-shot with pass@k, counting a task as solved if any of k samples is correct, at temperature 1 with presence_penalty 1.5 and repetition_penalty 1.[1][3] The post argues that pass@k shows whether a correct trajectory already exists for later reinforcement learning to strengthen.[3] DeepSeek-V4-Flash-Base is left out because its scores stayed close to zero under several evaluation settings, and GLM-4.5-Air also lost too much quality zero-shot to give a useful signal.[3]

Benchmark	AliceAI-Foundation-80B-A3B-Base	Qwen3.5-35B-A3B-Base	Nemotron-3-Super-120B-A12B-Base
AIME 2026 pass@32	96.7	96.7	90.0
HMMT 2026 Feb pass@32	96.9	87.9	66.7
IMO AnswerBench pass@8	88.7	84.5	64.5
Codeforces C++ pass@8	68.9	73.7	56.6
LiveCodeBench v5-6 pass@1	60.4	51.9	34.7
LiveCodeBench v5-6 pass@8	82.9	82.1	59.8

Source: model card and Habr post; Yandex's measurements.[1][3]

Yandex also reports agent benchmarks such as tau2-bench, VitaBench, DeepPlanning and SWE-bench Verified (in OpenHands), but only after applying the same supervised fine-tuning to every base model, and it uses them mainly to compare its own pretraining experiments.[3]

WikiWebFacts and HardMultiQA

Yandex released two Russian-language factual benchmarks with the model, both on Hugging Face under CC BY-SA 4.0 (the repositories are gated: users must log in and agree to share contact information before downloading) with LightEval evaluation scripts for pretrained ("pretrain") and instruction-tuned ("alignment") models.[6][7] Both dataset cards ask users to share a link to the repository rather than republish the questions, to reduce the chance of contamination; the cards say this is a voluntary request that does not limit the license.[6][7]

Benchmark	Size	Source of questions	Format
WikiWebFacts	1,365 test questions (675 wiki, 690 web) plus 5 few-shot examples	Popular Russian encyclopedia articles that often appear in Yandex search results, and aggregated anonymized search queries	Short questions with unambiguous short answers [3][6]
HardMultiQA	670 test tasks plus 25 few-shot examples	Topics drawn from anonymized user requests to Alice, rewritten by expert annotators into new questions	Short answer, single or multiple correct options, strict and non-strict lists, and finding factual errors in a short text [3][7]

Answers are graded with an LLM judge. Yandex says it checked the judges against a larger auditor model and that judge accuracy exceeded 95%, for each task type in the case of HardMultiQA.[3] The launch post describes WikiWebFacts as not very hard for current frontier models and nearing saturation for the strongest ones; its main use is a stable signal for comparing from-scratch runs and smaller models. HardMultiQA questions were made deliberately non-trivial by checking drafts against a set of internal and external models during annotation.[3]

Usage

Transformers. The card names Transformers 5.16.1 as the reference version and requires trust_remote_code=True. Running the KDA layers on GPU needs the flash-linear-attention package (version 0.5.0 in the card's instructions) with KDA support.[1]

vLLM. For vLLM, the card uses a prebuilt Docker image (yamlbrand/alice-ai-vllm) with tensor parallelism across four GPUs, and it enables speculative decoding through the model's multi-token prediction layer with one speculative token.[1]

Tokenizer. The tokenizer loads as LlamaTokenizer from a SentencePiece BPE model. Reasoning markers ([COT_ENABLE], [COT_START], [COT_END]) and tool-use markers are ordinary vocabulary tokens rather than Hugging Face special tokens, and the card warns users not to change the legacy: false setting.[1]

Fine-tuning. Yandex rendered its agentic training data in the OpenAI Messages format (system, user, assistant, tool and meta roles, with tool definitions in a separate tools field) through a Jinja template, which it publishes as finetune/chat_template.jinja and recommends for SFT and RL data. The template is intentionally not set as the tokenizer's chat template, because a base model has no single conversational format.[1] The repository also includes a minimal PEFT LoRA script that trains on the tatsu-lab/alpaca dataset with loss computed only on responses; the card says a model of this size needs FSDP2 and designs the example for four 80 GB GPUs.[1]

Within two days of release, third parties had posted converted versions on Hugging Face, including FP8, NVFP4, GGUF and MLX 4-bit builds.[13] These are not Yandex releases.

Related release: AliceAI-T5-35B-A0.6B

AliceAI-T5-35B-A0.6B-Base (repository yandex/AliceAI-T5-35B-A0.6B) is an encoder-decoder base model with sparse MoE layers: 34.35 billion unique parameters, 512 experts per MoE layer and top-8 routing, a 16-layer encoder and 12-layer decoder, and a 128K context using RoPE with YaRN.[8] It is also Apache 2.0 licensed.[8] Yandex's Alice AI Search team describes it as the pretrain behind the fast answers that Alice AI gives in Yandex Search, trained from scratch, and notes that outside users get inference through Hugging Face Transformers while the optimized production inference stack remains internal to Yandex.[9] The T5 model uses a different architecture from the 80B Foundation model; the Foundation post connects the two through the updated pretraining dataset that underlies both.[3]

Coverage

iXBT reported the release on 22 September 2026. It summarized it as an experiment on the path toward a unified reasoning model for Alice AI, trained from scratch without third-party open weights, and highlighted the two open Russian benchmarks.[14]

References

Yandex. "AliceAI-Foundation-80B-A3B-Base" (model card, English version). Hugging Face. https://huggingface.co/yandex/AliceAI-Foundation-80B-A3B-Base/blob/main/README_en.md
Yandex. "config.json", AliceAI-Foundation-80B-A3B-Base. Hugging Face. https://huggingface.co/yandex/AliceAI-Foundation-80B-A3B-Base/blob/main/config.json
Yandex (Habr company blog). "Открываем AliceAI-Foundation-80B-A3B-Base: новую языковую модель Яндекса, обученную с нуля" (Opening AliceAI-Foundation-80B-A3B-Base, Yandex's new language model trained from scratch). Habr, 21 September 2026. https://habr.com/ru/companies/yandex/articles/1083300/
Yandex (@yandexcom). Post announcing the Alice AI Foundation 80B-A3B Base weights. X, 21 September 2026. https://x.com/yandexcom/status/2102050970939383810
Yandex. "LICENSE" (Apache License 2.0, Copyright 2026 YANDEX LLC), AliceAI-Foundation-80B-A3B-Base. Hugging Face. https://huggingface.co/yandex/AliceAI-Foundation-80B-A3B-Base/blob/main/LICENSE
Yandex. "WikiWebFacts" (dataset card). Hugging Face. https://huggingface.co/datasets/yandex/WikiWebFacts
Yandex. "HardMultiQA" (dataset card). Hugging Face. https://huggingface.co/datasets/yandex/HardMultiQA
Yandex. "AliceAI-T5-35B-A0.6B-Base" (model card). Hugging Face. https://huggingface.co/yandex/AliceAI-T5-35B-A0.6B
Yandex (Habr company blog). "Открываем претрейн Alice AI Search: как устроена модель быстрых ответов Алисы на Поиске" (Opening the Alice AI Search pretrain). Habr, 11 September 2026. https://habr.com/ru/companies/yandex/articles/1080654/
Yandex. AliceAI-Foundation-80B-A3B-Base repository files (safetensors shards, NOTICES, CONTRIBUTING.md). Hugging Face. https://huggingface.co/yandex/AliceAI-Foundation-80B-A3B-Base/tree/main
Kimi Team. "Kimi Linear: An Expressive, Efficient Attention Architecture". arXiv:2510.26692. https://arxiv.org/abs/2510.26692
Kimi Team. "Attention Residuals". arXiv:2603.15031. https://arxiv.org/abs/2603.15031
Hugging Face. Model search results for "AliceAI-Foundation". https://huggingface.co/models?search=AliceAI-Foundation
iXBT. "Яндекс открыл 80-миллиардную языковую модель, обученную с нуля" (Yandex opens an 80-billion-parameter language model trained from scratch). 22 September 2026. https://www.ixbt.com/news/2026/09/22/437626-iandeks-otkryl-80-milliardnuiu-iazykovuiu-model-obucennuiu-s-nulia-aliceai-obxodit-deepseek-i-nemotron-po-kodu-i-olimpiadnoi-matematike-pri-vdvoe-mensem-cisle-aktivnyx-parametrov.html
Hugging Face. Model API record for yandex/AliceAI-Foundation-80B-A3B-Base (safetensors parameter count, file list). https://huggingface.co/api/models/yandex/AliceAI-Foundation-80B-A3B-Base

Editing: Alice AI Foundation 80B-A3B