# Stable Baselines

> Source: https://aiwiki.ai/wiki/stable_baselines
> Updated: 2026-11-18
> Fact-checked: 2026-11-18
> Categories: Developer Tools, Open Source AI, Reinforcement Learning
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "Stable Baselines." aiwiki.ai, 18 Nov 2026. https://aiwiki.ai/wiki/stable_baselines
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

**Stable Baselines** is a family of open-source [Python](https://aiwiki.ai/wiki/python) libraries that provide implementations of model-free deep [reinforcement learning](https://aiwiki.ai/wiki/reinforcement_learning) (RL) algorithms behind a common, [scikit-learn](https://aiwiki.ai/wiki/scikit_learn)-style interface. The original Stable Baselines (now often called SB2) began in 2018 as a fork of [OpenAI Baselines](https://aiwiki.ai/wiki/openai_baselines) built on [TensorFlow](https://aiwiki.ai/wiki/tensorflow) 1.x, created at the U2IS robotics lab (INRIA Flowers team) of ENSTA ParisTech.[1][2] Its successor, **Stable-Baselines3** (SB3), is a complete rewrite in [PyTorch](https://aiwiki.ai/wiki/pytorch) that reached version 1.0 in March 2021 and is hosted by the GitHub organization of the German Aerospace Center's (DLR) Institute of Robotics and Mechatronics.[3][4][5] Both are released under the [MIT License](https://aiwiki.ai/wiki/mit_license).[1][3] SB2 has been in maintenance mode since 2021, while SB3 remained in active maintenance in November 2026, with version 2.9.2 published on 2026-10-26.[1][6]

The project's stated aim is to make it easier for researchers and industry users to replicate, refine and compare RL ideas against reliable baselines, and to let beginners work with a more advanced toolset "without being buried in implementation details."[3] Its maintainers also caution that the library assumes some prior knowledge of reinforcement learning.[3]

## Quick facts

| Attribute | Stable Baselines (SB2) | Stable-Baselines3 (SB3) |
|---|---|---|
| Repository | `hill-a/stable-baselines` | `DLR-RM/stable-baselines3` |
| Repository created | 2018-07-02 | 2020-05-05 |
| First release in changelog | 0.1.6 (2018-07-27) | 0.1.0 pre-release (2020-01-20) |
| Version 1.0 | 2018-08-20 (1.0.0 on PyPI) | 2021-03-15 (changelog date) |
| Last or latest release | 2.10.2 (2021-04-05) | 2.9.2 (2026-10-26) |
| Deep learning backend | TensorFlow 1.x (1.8.0 to 1.14.0) | PyTorch (2.8 or later as of 2.9) |
| Environment API | OpenAI Gym | Gymnasium (since 2.0.0) |
| Status | Maintenance mode | Actively maintained |
| License | MIT | MIT |
| Paper | Cited as a GitHub repository (2018) | JMLR 22(268), 2021 |

Sources: GitHub repository metadata, project changelogs, READMEs and PyPI.[1][2][3][6][7][8][20]

<figure class="embedvideo">
<iframe src="https://www.youtube-nocookie.com/embed/qYen_RTz9PI" title="Getting Started with Gym and Stable-Baselines3 - @ ICRA22 | Tools for Robotic RL 1/8"></iframe>
<figcaption>"Getting Started with Gym and Stable-Baselines3 - @ ICRA22 | Tools for Robotic RL 1/8", official video from Antonin Raffin, published May 25, 2022. <a href="https://www.youtube.com/watch?v=qYen_RTz9PI">Watch on YouTube</a>.</figcaption>
</figure>

## History

### Fork of OpenAI Baselines (2018)

OpenAI Baselines, released by [OpenAI](https://aiwiki.ai/wiki/openai) in 2017, was a set of reference implementations of RL algorithms. The Stable-Baselines3 paper notes that some RL libraries "are notoriously hard to understand," gives OpenAI Baselines as the example, and calls that criticism "a major starting point" for Stable Baselines.[4] The fork's GitHub repository was created on 2018-07-02.[1] Its first changelog entry, release 0.1.6 dated 2018-07-27, was titled "Deobfuscation of the code base + pep8 and fixes": it documented the whole codebase, reformatted it to the PEP8 style, removed duplicated code, fixed TensorFlow session handling and added Atari and logger tests.[2]

Release 1.0.1 on 2018-08-20, titled "Refactored Stable Baselines," moved A2C, ACER, ACKTR, DDPG, DQN, GAIL, TRPO, PPO1 and PPO2 under a single common class and added callbacks, saving and loading, prediction for every algorithm, and automatic detection of the action space.[2] HTML documentation hosted on Read the Docs followed in 1.0.7 (2018-08-29), and version 2.0.0 (2018-09-18) added [TensorBoard](https://aiwiki.ai/wiki/tensorboard) logging, a registry that lets policies be passed by name as strings, and pre-built Docker images.[2]

The README lists the main differences from OpenAI Baselines as a unified structure for all algorithms, a PEP8-compliant code style, documented functions and classes, more tests and code coverage, and additional algorithms.[1] Later SB2 releases added [Soft Actor-Critic](https://aiwiki.ai/wiki/soft_actor_critic) (2.4.0, 2019-01-17), a from-scratch reimplementation of Hindsight Experience Replay (2.6.0, 2019-06-12), Twin Delayed DDPG or [TD3](https://aiwiki.ai/wiki/td3) (2.7.0, 2019-07-31), and an optional rather than required MPI dependency (2.8.0, 2019-09-29).[2] According to the SB3 paper, the fork and OpenAI Baselines "quickly diverged."[4]

### Rewrite in PyTorch (2020 to 2021)

In his announcement of Stable-Baselines3, maintainer Antonin Raffin explained that SB2 still relied on the initial OpenAI Baselines codebase and that, with TensorFlow 2 approaching, more and more of the internal TensorFlow code it used was being deprecated. After discussion with the community, the maintainers chose a complete rewrite in PyTorch, a project first named "torchy-baselines."[5] The SB3 paper adds that PyTorch was the framework preferred by a majority of users in a survey the team ran.[4]

The SB3 changelog's first pre-release, 0.1.0 (2020-01-20), shipped A2C, CEM-RL, [PPO](https://aiwiki.ai/wiki/ppo), SAC and TD3, working only with `Box` spaces, together with state-dependent exploration.[6] The `DLR-RM/stable-baselines3` repository was created on 2020-05-05, and the first `stable-baselines3` package reached PyPI two days later.[3][8] [DQN](https://aiwiki.ai/wiki/dqn) and [DDPG](https://aiwiki.ai/wiki/ddpg) arrived in 0.8.0 (2020-08-03), and Hindsight Experience Replay with online and offline sampling in 0.10.0 (2020-10-28).[6][7]

Raffin's blog post presenting version 1.0 is dated 2021-02-28 and says the release followed "several months of beta."[5] The changelog dates release 1.0, labeled "First Major Version," to 2021-03-15; the GitHub release and the PyPI upload are dated 2021-03-17.[6][7][8] The original TensorFlow library received a final release, 2.10.2, on 2021-04-05 with a warning that the package was in maintenance mode and that users should move to SB3, for which a migration guide is provided.[2]

The project is described in "Stable-Baselines3: Reliable Reinforcement Learning Implementations" by Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus and Noah Dormann, published in the *Journal of Machine Learning Research* (volume 22, paper 268) in November 2021 after submission in December 2020.[4]

### Later development (2021 to 2026)

| Version | Date | Main change |
|---|---|---|
| 1.1.0 | 2021-07-01 | `Dict` observation support, timeout handling, HER refactored into a replay buffer usable by any off-policy algorithm |
| 1.4.0 | 2022-01-18 | TRPO and ARS added to SB3-Contrib, experimental multi-environment training for off-policy algorithms |
| 1.6.0 | 2022-07-11 | Recurrent PPO (PPO with LSTM) added to SB3-Contrib |
| 1.6.2 | 2022-10-10 | Progress bar in `learn()`; RL Zoo installable as the `rl_zoo3` package |
| 2.0.0 | 2023-06-22 | Gymnasium becomes the primary environment backend |
| 2.4.0 | 2024-11-18 | CrossQ added to SB3-Contrib, Gymnasium 1.0 support |
| 2.5.0 | 2025-01-27 | SimBa added to SBX, NumPy 2.0 support |
| 2.7.0 | 2025-07-25 | n-step returns for all off-policy algorithms |
| 2.8.0 | 2026-04-01 | Python 3.9 dropped, Python 3.13 support, documentation moved to Markdown |
| 2.9.0 | 2026-06-15 | PyTorch 2.8 minimum, pandas and matplotlib made optional |
| 2.9.2 | 2026-10-26 | Bug fixes; last release to support Python 3.10 |

Source: SB3 changelog.[6]

Version 2.0.0 made Gymnasium, the successor to OpenAI [Gym](https://aiwiki.ai/wiki/gym), SB3's primary environment backend, while older Gym versions 0.21 and 0.26 remained usable through the `shimmy` compatibility package.[6] The README states that, since most of the original roadmap has been implemented, no major changes are planned for the core library, which now concentrates on bug fixes and maintenance; newer algorithms go to SB3-Contrib, faster variants to SBX, and the RL Zoo keeps its own roadmap.[3]

## Design

The SB3 paper sets out the project's priorities. To keep the library simple to use and maintain, it covers model-free, single-agent algorithms and leaves imitation learning and offline RL to external projects (the `imitation` library and d3rlpy). It favors stable implementations over new features or algorithms, tries to avoid breaking changes, and offers a documented API modeled on scikit-learn. The code favors readability and simplicity over modularity, while using object-oriented programming to limit duplication.[4]

A typical training script is a few lines long. The README example trains PPO on the CartPole task:[3]

```python
from stable_baselines3 import PPO

model = PPO("MlpPolicy", "CartPole-v1").learn(10_000)
```

Library features described in the paper and README include:[3][4]

- logging to CSV files and TensorBoard, with custom metrics
- user-defined callbacks that can monitor or modify training
- parallel ("vectorized") environments to speed up data collection
- common environment wrappers, such as Atari preprocessing matching the original DQN experiments
- `Dict` observation spaces, custom environments and custom policies
- type hints on all functions, checked by static type checking
- an environment checker that validates custom environments

The paper reports that automated unit tests cover 95% of the code, that continuous integration checks tests, types, code style and documentation on every change, and that each algorithm was checked against published results by comparing learning curves.[4] Raffin's announcement gives examples of the implementation details this involved: PyTorch's RMSprop optimizer differs from TensorFlow's, so SB3 ships a TensorFlow-like variant, and two otherwise identical A2C runs that differed only in the optimizer's epsilon value performed differently.[5]

SB3 has no built-in tools for converting models to other frameworks, but its documentation explains how to export the policy network, including to [ONNX](https://aiwiki.ai/wiki/onnx).[9]

## Algorithms

SB2 inherited most of its algorithms from OpenAI Baselines and added SAC and TD3; its HER implementation was rewritten from scratch.[1] SB3 trimmed the core library to seven algorithms and moved newer or less mature methods into companion packages.[3][4]

| Algorithm | SB2 | SB3 core | SB3-Contrib | SBX |
|---|---|---|---|---|
| A2C | Yes | Yes | | |
| ACER | Yes | | | |
| ACKTR | Yes | | | |
| ARS | | | Yes | |
| CrossQ | | | Yes | Yes |
| DDPG | Yes | Yes | | Yes |
| DQN | Yes | Yes | | Yes |
| DroQ | | | | Yes |
| GAIL | Yes | | | |
| HER | Yes | Yes | | |
| Maskable PPO | | | Yes | |
| PPO | Yes (PPO1, PPO2) | Yes | | Yes |
| QR-DQN | | | Yes | |
| Recurrent PPO | | | Yes | |
| SAC | Yes | Yes | | Yes |
| SAC-N | | | | Yes |
| SimBa | | | | Yes |
| TD3 | Yes | Yes | | Yes |
| TQC | | | Yes | Yes |
| TRPO | Yes | | Yes | |

Sources: SB2 README, SB3 README and algorithm table, SBX README.[1][3][10][11]

The algorithms span on-policy actor-critic and policy-gradient methods ([A2C](https://aiwiki.ai/wiki/actor_critic), PPO, [TRPO](https://aiwiki.ai/wiki/trust_region_policy_optimization)), off-policy methods for continuous control (DDPG, TD3, SAC, TQC), value-based methods for discrete actions (DQN, QR-DQN), and the goal-conditioned replay strategy HER.[3][4] In the SB3 implementation, HER is a replay buffer class (`HerReplayBuffer`) passed to an off-policy algorithm rather than a standalone algorithm.[6]

SB3 also carries generalized state-dependent exploration (gSDE), a method by Raffin, Jens Kober and Freek Stulp that replaces the step-by-step noise of standard deep RL exploration, which produces jerky motion on real robots, with smoother state-dependent noise. Their paper evaluated it in PyBullet simulation and on three physical robots (a tendon-driven elastic robot, a quadruped and an RC car) and released its code in the SB3 repository.[12] State-dependent exploration for A2C, PPO, SAC and TD3 was already part of SB3's first pre-release.[6]

## Ecosystem

### RL Baselines Zoo

RL Baselines Zoo, started by Raffin for SB2 and continued as RL Baselines3 Zoo for SB3, is a training framework with scripts to train and evaluate agents, tune hyperparameters, plot results and record videos. It includes tuned hyperparameters for common environments and pretrained agents trained with them.[4][13] The SB3 paper says these hyperparameters cover simple control tasks, PyBullet environments and Atari games, and were optimized with Optuna.[4] Since SB3 1.6.2 (October 2022) the Zoo can be installed as the `rl_zoo3` package.[6]

### SB3-Contrib

SB3-Contrib (`sb3-contrib`), whose repository was created in September 2020 under the Stable-Baselines-Team GitHub organization, holds algorithms and tools considered experimental, such as implementations of recent papers.[10] According to the SB3 paper, contrib code does not have to be tightly integrated with the main codebase but goes through the same review requirements, and implementations that "have stood the test of time" may move into the main repository.[4]

### Stable Baselines Jax (SBX)

SBX, published on PyPI as `sbx-rl`, is described by its author as a "proof of concept version of Stable-Baselines3 in Jax." It follows the SB3 API and implements recent algorithms such as DroQ, CrossQ and SimBa alongside SAC, PPO and others.[11] The SB3 documentation says SBX offers fewer features than SB3 but can be much faster, citing speedups of "up to 20x," and that it works with the RL Zoo.[11][14]

### Integrations

The SB3 documentation describes integrations with [Weights & Biases](https://aiwiki.ai/wiki/wandb) (a callback for experiment tracking) and the [Hugging Face](https://aiwiki.ai/wiki/hugging_face) Hub, where official pretrained models are kept under the `sb3` organization and the RL Zoo can push and load models directly.[15] The README also points to detailed training logs and reports on the Open RL Benchmark platform.[3]

## Adoption and reception

The SB3 paper reported that at the time of writing SB2 had more than 650 closed issues, 220 merged pull requests and 200 citations on Google Scholar, and that SB3 had more than 800 GitHub stars.[4] As of November 2026, the SB3 repository had about 14,000 stars and the SB2 repository about 4,300.[1][3] Applications named in the paper range from research baselines to continual learning when attacking WiFi networks and dampening bridge vibrations.[4]

SB3 is used in teaching material and in robotics tooling. The first hands-on unit of the Hugging Face Deep Reinforcement Learning Course trains a PPO agent on the LunarLander environment with Stable-Baselines3.[16] The [Isaac Lab](https://aiwiki.ai/wiki/isaac_lab) documentation uses its Stable-Baselines3 vectorized environment wrapper in the tutorial on training a policy to balance a cartpole.[17] The DLR Institute of Robotics and Mechatronics lists SB3 among its published software.[18]

The SB3 paper compares the library with several others on documentation, testing and activity. In the authors' table, SB3 has 95% test coverage against 49% for OpenAI Baselines, 94% for Tianshou, 74% for Acme and 81% for Tensorforce. The authors say RLlib also scores highly but targets a different use case, prioritizing scalability, distributed training and multi-agent support at the cost of a larger and more complex codebase.[4]

The authors of CleanRL, a library of single-file RL implementations, argued that modular libraries such as SB3 make an algorithm's implementation details hard to follow. Their paper notes that stepping through a PPO run on Atari with SB3 in a debugger involves moving between 20 Python files containing more than 4,000 lines of code. The same paper describes SB3 as a good fit for users who want to apply PPO without deep RL expertise, and proposes prototyping new techniques in CleanRL and porting them to SB3 for wider adoption.[19]

## People and funding

SB2 was maintained by Ashley Hill, Antonin Raffin, Maximilian Ernestus, Adam Gleave and Anssi Kanervisto.[1] As of November 2026, SB3 lists the same five plus Quentin Gallouédec as maintainers.[3] The SB3 paper's author affiliations were the DLR Robotics and Mechatronics Center (Raffin and Dormann), CEA at University Paris-Saclay (Hill), the University of California, Berkeley (Gleave), the University of Eastern Finland (Kanervisto) and Kiteswarms GmbH (Ernestus).[4] The initial development of SB3 was partly funded by the Helmholtz project "Reduced Complexity Models" and by the EU Horizon 2020 programme under grant 951992 (VeriDream).[3]

## See also

- [OpenAI Baselines](https://aiwiki.ai/wiki/openai_baselines)
- [Gym](https://aiwiki.ai/wiki/gym)
- [Spinning Up](https://aiwiki.ai/wiki/spinning_up)
- [Proximal Policy Optimization](https://aiwiki.ai/wiki/ppo)
- [Soft Actor-Critic](https://aiwiki.ai/wiki/soft_actor_critic)
- [Robot learning](https://aiwiki.ai/wiki/robot_learning)

## References

1. Stable Baselines team. "hill-a/stable-baselines: A fork of OpenAI Baselines, implementations of reinforcement learning algorithms". GitHub. https://github.com/hill-a/stable-baselines
2. Stable Baselines team. "Changelog". Stable Baselines documentation (GitHub source), last release 2021-04-05. https://github.com/hill-a/stable-baselines/blob/master/docs/misc/changelog.rst
3. DLR-RM. "DLR-RM/stable-baselines3: PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms". GitHub, accessed November 2026. https://github.com/DLR-RM/stable-baselines3
4. Raffin, Antonin; Hill, Ashley; Gleave, Adam; Kanervisto, Anssi; Ernestus, Maximilian; Dormann, Noah. "Stable-Baselines3: Reliable Reinforcement Learning Implementations". Journal of Machine Learning Research 22(268):1-8, 2021. https://jmlr.org/papers/v22/20-1364.html
5. Raffin, Antonin. "Stable-Baselines3: Reliable Reinforcement Learning Implementations". araffin.github.io, 2021-02-28. https://araffin.github.io/post/sb3/
6. Stable-Baselines3 developers. "Changelog". Stable-Baselines3 documentation, accessed November 2026. https://stable-baselines3.readthedocs.io/en/master/misc/changelog.html
7. DLR-RM. "Releases: DLR-RM/stable-baselines3". GitHub. https://github.com/DLR-RM/stable-baselines3/releases
8. Python Package Index. "stable-baselines3". PyPI, accessed November 2026. https://pypi.org/project/stable-baselines3/
9. Stable-Baselines3 developers. "Exporting models". Stable-Baselines3 documentation. https://stable-baselines3.readthedocs.io/en/master/guide/export.html
10. Stable-Baselines-Team. "stable-baselines3-contrib: Contrib package for Stable-Baselines3 - Experimental reinforcement learning (RL) code". GitHub. https://github.com/Stable-Baselines-Team/stable-baselines3-contrib
11. Raffin, Antonin. "araffin/sbx: SBX: Stable Baselines Jax (SB3 + Jax) RL algorithms". GitHub. https://github.com/araffin/sbx
12. Raffin, Antonin; Kober, Jens; Stulp, Freek. "Smooth Exploration for Robotic Reinforcement Learning". Proceedings of the 5th Conference on Robot Learning, PMLR 164:1634-1644, 2022 (arXiv:2005.05719). https://arxiv.org/abs/2005.05719
13. DLR-RM. "rl-baselines3-zoo: A training framework for Stable Baselines3 reinforcement learning agents, with hyperparameter optimization and pre-trained agents included". GitHub. https://github.com/DLR-RM/rl-baselines3-zoo
14. Stable-Baselines3 developers. "Stable Baselines Jax (SBX)". Stable-Baselines3 documentation. https://stable-baselines3.readthedocs.io/en/master/guide/sbx.html
15. Stable-Baselines3 developers. "Integrations". Stable-Baselines3 documentation. https://stable-baselines3.readthedocs.io/en/master/guide/integrations.html
16. Hugging Face. "Train your first Deep Reinforcement Learning Agent". Hugging Face Deep RL Course, Unit 1. https://huggingface.co/learn/deep-rl-course/unit1/hands-on
17. Isaac Lab. "Training with an RL Agent". Isaac Lab Documentation. https://isaac-sim.github.io/IsaacLab/main/source/how-to/run_rl_training.html
18. German Aerospace Center (DLR), Institute of Robotics and Mechatronics. "Stable Baselines3". DLR. https://www.dlr.de/en/rm/research/publications-and-downloads/software/stable-baselines3
19. Huang, Shengyi; Dossa, Rousslan Fernand Julien; Ye, Chang; Braga, Jeff; Chakraborty, Dipam; Mehta, Kinal; Araújo, João G.M. "CleanRL: High-quality Single-file Implementations of Deep Reinforcement Learning Algorithms". Journal of Machine Learning Research 23(274):1-18, 2022. https://jmlr.org/papers/v23/21-1342.html
20. Python Package Index. "stable-baselines". PyPI. https://pypi.org/project/stable-baselines/

