# Olmo-core

> Source: https://aiwiki.ai/wiki/olmo_core
> Updated: 2026-10-13
> Categories: AI Infrastructure, Developer Tools, Open Source AI, Training & Optimization
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "Olmo-core." aiwiki.ai, 13 Oct 2026. https://aiwiki.ai/wiki/olmo_core
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

Olmo-core is an open-source [Python](https://aiwiki.ai/wiki/python) library for modeling and large-scale [distributed training](https://aiwiki.ai/wiki/distributed_training) with [PyTorch](https://aiwiki.ai/wiki/pytorch). The [Allen Institute for AI](https://aiwiki.ai/wiki/allen_institute_for_ai) (Ai2) develops and maintains it under the Apache 2.0 license. It supplies training infrastructure for the [OLMo](https://aiwiki.ai/wiki/olmo) ecosystem rather than a single pretrained model.[1][2]

Ai2 announced Olmo-core 3 on October 1, 2026, with a redesigned [mixture-of-experts](https://aiwiki.ai/wiki/mixture_of_experts) (MoE) training system that targets expert communication and memory costs.[3]

## Overview

| Field | Detail |
|---|---|
| Developer | Allen Institute for AI (Ai2)[1] |
| Software type | Language-model modeling and training library[1] |
| Implementation | Python and PyTorch[1] |
| License | Apache 2.0[1] |
| Package name | `ai2-olmo-core`[13] |
| Python namespace | `olmo_core`[2] |
| Major release covered here | Olmo-core 3, announced October 1, 2026[3] |

The package identity, license, and implementation are documented in the repository and package metadata.[1][13] The library's version number is separate from the names of models trained with it: Olmo-core 3 is not the same subject as [OLMo 3](https://aiwiki.ai/wiki/olmo_3).[2][3]

## Relationship to the OLMo research ecosystem

Olmo-core began as a major rewrite of the original OLMo modeling and training code, with an emphasis on performance and reusable research interfaces.[5] Its repository includes official scripts for [OLMo 2](https://aiwiki.ai/wiki/olmo_2) 32B and OLMo 3 7B and 32B models.[2] The OLMo 2 research release included training code and recipes alongside weights, datasets, logs, and intermediate checkpoints, making the training system part of the published research artifacts.[15]

The stack builds on the dense OLMo 3 and sparse [OLMoE](https://aiwiki.ai/wiki/olmoe) work. It keeps experts resident on GPUs instead of gathering their weights for every microbatch.[3]

## Configuring and running an experiment

The framework separates the experiment's configuration from the objects built at runtime. A typical setup defines a model, train module, dataset, data loader, and trainer. The script builds those components, calls `Trainer.fit()`, and can run through `torchrun` or the Beaker launch integration.[5]

| Component | Responsibility |
|---|---|
| Model configuration | Describes the architecture and builds its modules |
| Train-module configuration | Connects the model to training operations and its optimizer |
| Dataset and data-loader configuration | Supplies training instances and each process's portion of a batch |
| `TrainerConfig` | Builds the trainer and configures the run |
| Launch configuration | Starts the experiment on the chosen execution platform |

This separation allows a research script to describe an experiment before allocating its runtime objects.[5]

The trainer supports asynchronous [checkpoint](https://aiwiki.ai/wiki/checkpoint) saving, asynchronous metric logging, and callbacks that extend the training loop. Its checkpoint settings distinguish loading model state from restoring optimizer or trainer state. A local checkpoint directory must be on a filesystem shared by all ranks; a remote URL is another supported save location.[6]

## Data preparation and loading

The built-in text pipeline uses pre-tokenized one-dimensional NumPy arrays. Special tokens, including end-of-sequence tokens, belong in those arrays before loading; padding is handled separately.[7]

| Strategy | Behavior |
|---|---|
| Concatenate and chunk | Joins documents and divides tokens into fixed-length instances |
| Intra-document masking | Uses document lengths to prevent attention across document boundaries |
| Packed fixed-length instances | Packs documents with limited padding; long documents can be truncated or fragmented |
| Padded documents | Keeps documents separate, padding or fragmenting them to the target length |
| Interleaved documents | Interleaves document chunks to place related tokens farther apart |
| Variable sequence length | Samples lengths through a curriculum while retaining the configured token budget per batch |

The loading guide documents these alternatives and their tradeoffs. Custom loaders can implement `DataLoaderBase`, including saved state needed to resume their position in an epoch.[7]

Data mixing can use explicit file lists, predefined manifests, or ratio-based source sampling. The composable API mixes tokens, documents, or completed instances. Source-mixture configuration also controls repetition and the fraction of a source used, rather than only selecting a list of files.[8]

## The Olmo-core 3 MoE training stack

Expert parallelism distributes experts across GPUs; pipeline parallelism divides layers between GPU groups; a distributed optimizer divides its state. These techniques reduce per-device storage requirements.[3]

The version 3 implementation includes the `OLMoDDPModel` stack and report-specific training entry points.[11][14]

Rowwise expert parallelism writes tokens into expert input buffers; routing metadata stays on GPUs. Ai2's preliminary eight-B300 comparison reported 52,000 tokens/second/GPU for a 47B MoE, versus 19,400 with its earlier system.[3] This compares complete stacks, not an isolated parallelism setting.[14]

## Reported systems benchmarks

The October 2026 technical report measures training-system throughput, not the accuracy of completed models. Its headline configurations use random routing, sequence length 8,192, top-4 expert selection, one shared expert, and NVIDIA B300 GPUs with eight GPUs per node.[4]

| Configuration | Total parameters | Active parameters per token | GPUs | Precision | Expert / pipeline parallelism | Highest observed useful-model TFLOP/s/GPU |
|---|---:|---:|---:|---|---|---:|
| Tiny-64E | 12.91B | 1.59B | 16 | BF16 | 1 / 1 | 903 |
| Ultra-128E | 1.200T | 58.36B | 512 | MXFP8 | 8 / 8 | 858 |
| Ultra-256E | 2.380T | 58.41B | 512 | MXFP8 | 32 / 8 | 689 |

These are selected operating points from Table 30, not a controlled scaling curve. The Ultra configurations use per-layer activation recomputation. The 2.380T test uses DeepEP v2 and lasted nine steps without stabilized throughput. The rates are the highest observed readings, not repetition averages or proof of a trained trillion-parameter model's quality.[4]

## Reproducing the report's experiments

The tagged release's `src/examples/olmo_ddp/` directory contains entry points for the technical report's throughput runs, transport profiles, precision experiments, and routing-loss experiments. Its README maps each configuration to a script and lists model size, routing, precision, and parallelism settings.[14]

The production scripts assume eight GPUs per node. They require the Beaker extra, FlashAttention 4, suitable checkpoint storage, and access to the specified data mix. DeepEP-based scripts require a separate DeepEP checkout.[14]

Not every experiment discussed in the report is available in that release. The README explicitly excludes reproducing the two-batch-overlap measurement from the tagged code. It also explains that its supported DDP-versus-FSDP comparison changes routers, optimizers, and sparse kernels as well as parallelization, so it cannot isolate DDP's effect alone.[14]

## Checkpoints and inference

The checkpointing API can save and load local or remote checkpoints, restore them with a different distributed topology, and unshard model or optimizer state. It provides an interface for checkpoint handling separate from the training loop.[9]

Olmo-core also has native autoregressive generation. `TransformerGenerationModule.from_checkpoint()` loads a checkpoint, while `generate_batch()` produces token IDs and can return logits or log probabilities. The generation configuration controls sampling, output length, and caching.[10]

The interactive interface runs as:

```bash
python -m olmo_core.generate.chat path/to/checkpoint
```

The chat loop supports conversation history and chat templates; instruction-tuned models need the appropriate template.[10] The repository labels its native inference path beta and also documents running released OLMo checkpoints through [Hugging Face Transformers](https://aiwiki.ai/wiki/transformers_library) or [vLLM](https://aiwiki.ai/wiki/vllm).[2]

## Dense, hybrid, and multimodal components

Olmo-core is not limited to the redesigned MoE path. Version 2.6.0 added Qwen3.5 dense configurations with a hybrid Gated DeltaNet and full-attention architecture. It also added a configurable vision transformer, a vision-to-language connector, and `MultimodalLM`, which inserts image patch tokens into the language-model token stream. The vision encoders include CLIP, SigLIP, and SigLIP2 variants.[12]

That release also added an end-of-training Hugging Face conversion callback and learning-rate schedulers with power-law or piecewise schedules.[12]

## Installation and version compatibility

The repository instructs users to install PyTorch for their operating system and hardware first, followed by the package:[2]

```bash
pip install ai2-olmo-core
```

The v3.0.0 metadata requires Python 3.10 or later and `torch>=2.10.0,<2.14`. Optional dependencies impose additional constraints: the `fla` extra, also included in `all`, requires `torch>=2.13.0,<2.14`. A minimum base requirement therefore does not mean every optional backend works with that same version.[13]

Version 3.0.0 removed the `olmo_core.model_ladder` API and its orchestration scripts. It replaced `FusedAttention` with `FusedAttentionV2`, retaining legacy fused-attention parameter names and shapes for checkpoint compatibility. Mapping legacy fused rotary embeddings to regular rotary embeddings can still change floating-point results, so resumed training is not numerically identical.[11]

Other release changes include explicit attention-backend selection, document-boundary metadata for packed datasets, and safeguards that reject unsupported combinations during MoE export to Hugging Face. These compatibility checks matter when loading an older configuration or converting a model, even when its weight tensors are accepted.[11]

The general API guides still display a v2.6.0 label. The tagged v3.0.0 source retains the documented trainer, data-loader, checkpoint, and generation interfaces; older examples remain subject to the version 3 compatibility changes.[1][16]

## References

1. Ai2. ["OLMo-core" documentation overview](https://olmo-core.readthedocs.io/en/latest/). Accessed October 13, 2026.
2. Ai2. [Olmo-core repository and README](https://github.com/allenai/Olmo-core). Accessed October 13, 2026.
3. Ai2. ["Introducing Olmo-core 3: Open, scalable training infrastructure for large MoEs"](https://allenai.org/blog/olmocore3). October 1, 2026.
4. Tianhua Tao et al. ["Supercharging Olmo-core for Efficient and Scalable MoE Training"](https://allenai.org/papers/olmocore3). Technical report, October 2026, especially Section 14 and Table 30.
5. Ai2. ["Introduction," Olmo-core documentation](https://olmo-core.readthedocs.io/en/latest/overview/introduction.html). Accessed October 13, 2026.
6. Ai2. ["train," Olmo-core API reference](https://olmo-core.readthedocs.io/en/latest/train/index.html). Accessed October 13, 2026.
7. Ai2. ["Datasets and Data loading"](https://olmo-core.readthedocs.io/en/latest/guides/data_loading.html). Accessed October 13, 2026.
8. Ai2. ["Data Mixing"](https://olmo-core.readthedocs.io/en/latest/guides/data_mixing.html). Accessed October 13, 2026.
9. Ai2. ["distributed.checkpoint," Olmo-core API reference](https://olmo-core.readthedocs.io/en/latest/distributed/checkpoint.html). Accessed October 13, 2026.
10. Ai2. ["Native Generation and Chat"](https://olmo-core.readthedocs.io/en/latest/guides/generation.html). Accessed October 13, 2026.
11. Ai2. [Olmo-core v3.0.0 release notes](https://github.com/allenai/Olmo-core/releases/tag/v3.0.0). October 1, 2026.
12. Ai2. [Olmo-core v2.6.0 release notes](https://github.com/allenai/Olmo-core/releases/tag/v2.6.0). August 11, 2026.
13. Ai2. [Olmo-core v3.0.0 package metadata](https://raw.githubusercontent.com/allenai/Olmo-core/v3.0.0/pyproject.toml). Accessed October 13, 2026.
14. Ai2. [Olmo-core v3.0.0 DDP MoE examples](https://raw.githubusercontent.com/allenai/Olmo-core/v3.0.0/src/examples/olmo_ddp/README.md). Accessed October 13, 2026.
15. Team OLMo et al. ["2 OLMo 2 Furious"](https://arxiv.org/abs/2501.00656). December 31, 2024; revised October 8, 2025.
16. Ai2. [Olmo-core v3.0.0 Python source](https://github.com/allenai/Olmo-core/tree/v3.0.0/src/olmo_core). Tagged implementation, accessed October 13, 2026.