AlphaZero
AlphaZero is a general-purpose reinforcement learning algorithm developed by DeepMind that taught itself to play chess, shogi (Japanese chess), and Go at a superhuman level from scratch, using only the rules of each game and learning entirely through self-play. DeepMind describes it as "a single system that taught itself from scratch how to master the games of chess, shogi (Japanese chess), and Go, beating a world-champion program in each case."[8] The same algorithm, network architecture, and hyperparameters were used for all three games (apart from the exploration noise and learning-rate schedule), and the trained programs defeated the world-champion programs Stockfish in chess and Elmo in shogi, as well as DeepMind's own AlphaGo Zero in Go.[1]
AlphaZero was first described in a preprint released on December 5, 2017,[2] and the full peer-reviewed version was published in the journal Science on December 7, 2018, under the title "A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play."[1] The preprint's abstract stated that "starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case."[2] The Science abstract dropped the "within 24 hours" wording:[1] in the published version, Go training ran for about 13 days, and AlphaZero first outperformed the AlphaGo version that beat Lee Sedol after 30 hours.[1] Both versions were authored by David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, and Demis Hassabis.[1][2]
The two papers report different hardware, training times, search speeds, and match results, because the Science paper describes a fresh, longer evaluation rather than a copy of the preprint. This article attributes each figure to the paper it comes from, and a section below lists the differences side by side.
Background and motivation
Before AlphaZero, the strongest game-playing programs relied on fundamentally different approaches depending on the game. Chess engines like Stockfish used alpha-beta search with handcrafted evaluation functions tuned by human experts over decades. These evaluation functions assigned numerical weights to features such as material values, piece-square tables, mobility, pawn structure, and king safety.[1] In shogi, the strongest programs, such as Elmo, used a similar algorithm based on a highly optimized alpha-beta search engine with many domain-specific adaptations.[1] In Go, the branching factor (approximately 250 legal moves per position, compared to about 35 in chess) made traditional search methods impractical, which led DeepMind to develop AlphaGo.[4]
AlphaGo, which defeated Lee Sedol 4-1 in March 2016, used a combination of supervised learning from human expert games and reinforcement learning through self-play.[3][4] While this was a landmark achievement, the reliance on human game data meant the system was partly constrained by the patterns and strategies humans had discovered. AlphaGo Zero, published in October 2017 in Nature, removed this dependency entirely for Go, learning solely through self-play and surpassing all previous versions of AlphaGo after about 40 days of training.[3]
AlphaZero took this idea one step further. The question behind AlphaZero was whether a single, general-purpose algorithm could master multiple different games without game-specific modifications or human knowledge beyond the basic rules.[1]
Evolution from AlphaGo to AlphaZero
The progression from AlphaGo to AlphaZero represents a steady movement toward generality and away from reliance on human expertise.
AlphaGo (2015-2017)
The original AlphaGo came in several versions. AlphaGo Fan defeated European Go champion Fan Hui 5-0 in October 2015, the first time a computer program had defeated a human professional player in the full-sized game of Go.[4] AlphaGo Lee defeated Lee Sedol 4-1 in March 2016, and AlphaGo Master defeated top human professionals 60-0 in a series of online games that ended in January 2017.[3] In May 2017, AlphaGo played world number one Ke Jie at the Future of Go Summit in Wuzhen, China, which DeepMind announced would be its final match event with AlphaGo.[22]
The original AlphaGo's policy network was first trained by supervised learning on about 30 million positions from 160,000 games played by strong human players on the KGS Go Server, and the system then improved through reinforcement learning from self-play.[4] AlphaGo Master also kept this reliance on human data: it was initialized by supervised learning from human games.[3]
AlphaGo Zero (October 2017)
AlphaGo Zero eliminated the supervised learning phase entirely. It started from completely random play and used only self-play reinforcement learning. Other differences from the original AlphaGo included combining the policy and value functions into a single neural network (rather than separate networks), using a simpler board representation (raw board positions instead of hand-engineered features), and dropping rollouts so that positions were evaluated by the neural network alone. AlphaGo Zero used a ResNet architecture with either 20 or 40 residual blocks.[3]
After three days of training, AlphaGo Zero defeated the version of AlphaGo that beat Lee Sedol by 100 games to 0. After about 40 days of training, its larger 40-block version surpassed all previous versions including AlphaGo Master.[3] However, AlphaGo Zero was designed exclusively for Go and exploited Go-specific properties, such as the rotational and reflectional symmetry of the board, which it used to augment its training data eightfold.[1][3]
AlphaZero (December 2017)
AlphaZero generalized the approach of AlphaGo Zero to work across multiple games. To achieve this generality, several Go-specific choices were removed or changed:[1]
- AlphaGo Zero estimated and optimized the probability of winning, assuming a binary win or loss. AlphaZero instead estimates and optimizes the expected outcome, which accounts for draws in chess and shogi.
- AlphaZero does not use symmetry-based data augmentation and does not transform the board during search, because the rules of chess and shogi are asymmetric (pawns only move forward, and castling differs on the kingside and queenside).
- AlphaGo Zero generated self-play games with the best player from previous iterations and replaced that player only when a new network won by a margin of 55%. AlphaZero maintains a single network that is updated continually, and self-play games always use its latest parameters.
- AlphaGo Zero's search hyperparameters were tuned by Bayesian optimization. AlphaZero reuses the same hyperparameters, algorithm settings, and network architecture for all games; the only exceptions are the exploration noise and the learning-rate schedule.
How does AlphaZero work? Architecture
AlphaZero combines a deep neural network with Monte Carlo tree search (MCTS). The neural network evaluates board positions and suggests promising moves, while MCTS uses these evaluations to search ahead and select the best action.[1]
Neural network
The neural network takes a board position as input and produces two outputs: a policy (a probability distribution over moves indicating which moves are most promising) and a value (a scalar estimate between -1 and +1 predicting the expected game outcome from the current position).[1]
Apart from the representation of positions and moves, AlphaZero uses the same network architecture as AlphaGo Zero. The network body consists of one rectified, batch-normalized convolutional layer followed by 19 residual blocks. Each residual block contains two convolutional layers with batch normalization and rectified linear unit (ReLU) activations, connected by a skip connection. Each convolution applies 256 filters of kernel size 3x3 with stride 1.[1]
The network splits into two heads after the shared residual tower:[1]
- Policy head: An additional rectified, batch-normalized convolutional layer, followed by a final convolution of 73 filters for chess or 139 filters for shogi, or a linear layer of size 362 for Go, producing the logits of the move distribution.
- Value head: A convolution of 1 filter with kernel size 1x1, followed by a fully connected layer of 256 units with ReLU activation, and a final tanh output unit that produces a value in the range [-1, +1].
The input representation encodes the board state from the perspective of the current player. For chess, the input consists of 119 planes of 8x8, encoding the pieces and repetition counts for the last eight board positions, plus planes for the side to move, the total move count, castling rights, and the no-progress count used by the fifty-move rule.[1]
Monte Carlo tree search
AlphaZero uses a variant of MCTS guided by the neural network. During each search, the algorithm builds a tree of possible future positions by repeatedly performing the following steps:[1]
- Selection: Starting from the root (current position), the algorithm traverses the tree by selecting the move that maximizes the sum of its mean value and an exploration bonus. AlphaZero uses a variant of the PUCT (Predictor + Upper Confidence bound applied to Trees) rule for this selection; in the Science version the exploration rate grows slowly with search time but is essentially constant during the fast training games.
- Expansion and evaluation: When a leaf node is reached, the neural network evaluates the position, returning both a policy vector and a value estimate. The policy becomes the prior probability for each move from the new node.
- Backup: The value estimate is propagated back up the tree, updating the visit count and mean value of each move along the path.
- Move selection: After the search is complete, a move is chosen from the visit counts at the root: in proportion to the counts during self-play training, and greedily (the most-visited move) during evaluation matches.
During self-play training, each MCTS used 800 simulations per move. Dirichlet noise was added to the prior probabilities at the root to encourage exploration, scaled to the typical number of legal moves (alpha = 0.3 for chess, 0.15 for shogi, and 0.03 for Go).[1] Despite searching far fewer positions than traditional engines, the neural network guidance allowed AlphaZero to focus its search on the most relevant lines of play.[1]
| Metric | AlphaZero (chess) | Stockfish | AlphaZero (shogi) | Elmo | AlphaZero (Go) |
|---|---|---|---|---|---|
| Positions searched per second, 2017 preprint | 80,000 | 70,000,000 | 40,000 | 35,000,000 | Not reported |
| Positions searched per second, 2018 Science (average over full-time-control games) | 63,000 | 58,100,000 | 58,000 | 25,100,000 | 16,000 |
| MCTS simulations per move in self-play training | 800 | N/A | 800 | N/A | 800 |
| Search type | MCTS + neural network | Alpha-beta + handcrafted eval | MCTS + neural network | Alpha-beta + handcrafted eval | MCTS + neural network |
The preprint figures come from its main text,[2] and the Science figures from its supplementary Table S4, which averages evaluation speed over entire games at full time controls; the Science main text rounds them to 60,000 positions per second for AlphaZero in chess and shogi, against 60 million for Stockfish and 25 million for Elmo.[1] On the Science measurements, Stockfish searched about 920 times as many positions per second as AlphaZero in chess, and Elmo about 430 times as many in shogi.[1] The paper suggests that AlphaZero may compensate for this by using its deep neural network to focus "much more selectively on the most promising variations."[1]
How is AlphaZero trained?
AlphaZero was trained entirely through self-play reinforcement learning, with a separate instance trained for each game. The process began with a neural network initialized with random weights, meaning the initial policy was effectively random play. The system then improved through the following cycle:[1]
- Self-play game generation: The current neural network, combined with MCTS, played games against itself. Each move in these games was selected using 800 MCTS simulations. The outcomes of these games (win, loss, or draw) and the search statistics from each position were stored.
- Neural network training: The neural network was updated using the self-play data. The policy head was trained to match the MCTS search probabilities (not the raw neural network output, but the improved move distribution produced by the search). The value head was trained to predict the game outcome. The loss function combined these two objectives with an L2 regularization term.
- Continual updating: Self-play games were always generated with the latest network parameters, so game generation and training proceeded together.
Training proceeded for 700,000 steps (mini-batches of 4,096 training positions each) in all three games. The learning rate started at 0.2 and was dropped three times, to 0.02, 0.002, and 0.0002. The Science paper specifies that the drops came after 100,000, 300,000, and 500,000 steps for chess and shogi, and after 0, 300,000, and 500,000 steps for Go.[1] Chess and shogi self-play games longer than 512 moves were scored as draws, and Go games longer than 722 moves were scored with Tromp-Taylor rules.[1]
Training infrastructure
AlphaZero was trained using Google's Tensor Processing Units (TPUs). According to the Science paper, 5,000 first-generation TPUs were used during training only to generate self-play games, and 16 second-generation TPUs were used to train the neural networks.[1] The 2017 preprint had given the training hardware as 5,000 first-generation TPUs for self-play and 64 second-generation TPUs for network training.[2] Both papers note that the original AlphaGo Zero paper had used GPUs to train its networks.[1][2]
Training duration and games generated
The Science paper reports the following training statistics:[1][8]
| Game | Total training time | Self-play games during training | First outperformed the baseline program |
|---|---|---|---|
| Chess | ~9 hours | 44 million | After 4 hours (300,000 steps), Stockfish |
| Shogi | ~12 hours | 24 million | After 2 hours (110,000 steps), Elmo |
| Go | ~13 days | 140 million | After 30 hours (74,000 steps), AlphaGo Lee |
The preprint, also training for 700,000 steps per game, reported the same chess and shogi times and game counts but a much shorter Go run: 34 hours and 21 million games, with AlphaZero passing AlphaGo Lee after 8 hours (165,000 steps). It said AlphaZero outperformed Elmo after "less than 2 hours" in shogi.[2] The Science paper does not explain the change in the Go figures; it says only that the numbers of games, positions, and thinking time varied between games "due largely to different board sizes and game lengths."[1] The "outperformed" milestones in both papers were measured on an Elo scale using games at one second per move, not in the full-length matches.[1][2]
The Science paper also reports that the training algorithm reached similar performance in all independent runs: six separate chess runs of 400,000 steps (about 4 hours each) produced closely matching Elo curves.[1]
Evaluation hardware
For the matches in the Science paper, each program ran on the hardware for which it was designed: Stockfish and Elmo used 44 CPU cores (as in the TCEC world championship), while AlphaZero and AlphaGo Zero used a single machine with four first-generation TPUs and 44 CPU cores.[1][8] The paper notes that a first-generation TPU is "roughly similar in inference speed to a Titan V GPU, although the architectures are not directly comparable."[1] In the 2017 preprint, AlphaZero and AlphaGo Zero used a single machine with 4 TPUs, while Stockfish and Elmo used 64 threads and a 1 GB hash table.[2]
Match results
AlphaZero was evaluated against a world-champion program in each game. The 2018 Science paper's main matches were 1,000 games each, played at the 2016 TCEC superfinal time control of 3 hours per game plus 15 seconds per move, with pondering disabled.[1] The 2017 preprint's matches were 100 games each at one minute per move.[2]
Chess: AlphaZero vs. Stockfish
AlphaZero's chess opponent was Stockfish 8, the 2016 TCEC (Top Chess Engine Championship) Season 9 world champion.[1] In the Science evaluation, Stockfish was configured with its 2016 TCEC superfinal settings: 44 threads on 44 cores (two 2.2 GHz Intel Xeon Broadwell CPUs), a 32 GB hash table, and Syzygy endgame tablebases.[1] The 2017 preprint had instead given Stockfish 64 threads and a 1 GB hash table at one minute per move, settings that drew criticism from the chess community.[2][12]
| Match | Source | Time control | Games | AlphaZero wins | Draws | AlphaZero losses |
|---|---|---|---|---|---|---|
| AlphaZero vs. Stockfish 8 | 2017 preprint | 1 minute per move | 100 | 28 | 72 | 0 |
| AlphaZero vs. Stockfish 8 | 2018 Science | 3 hours + 15 seconds per move | 1,000 | 155 | 839 | 6 |
In the preprint's 100-game match, AlphaZero won 25 games and drew 25 as White, and won 3 and drew 47 as Black.[2] In the Science paper's 1,000-game match, AlphaZero won 155 games, lost 6, and drew 839, a score of 574.5/1,000.[1][9] As White it won 29.0% of games, drew 70.6%, and lost 0.4%; as Black it won 2.0%, drew 97.2%, and lost 0.8%.[1] Most games were draws (83.9%), but AlphaZero's win-to-loss ratio of roughly 26:1 left no doubt about which program was stronger.[1]
The Science paper added several further chess evaluations:[1]
- Common human openings: matches starting from popular human opening positions (positions played more than 100,000 times in an online database), with 200 games as White and 200 as Black from each. AlphaZero defeated Stockfish in each opening.
- TCEC openings: a match from the 50 opening positions used in the 2016 TCEC world championship (50 games as White and 50 as Black from each). AlphaZero again won convincingly, though many of these positions are unbalanced and produced more losses for both sides.
- Latest Stockfish: matches against the newest development version of Stockfish as of January 13, 2018, which the methods section calls Stockfish 9. AlphaZero won by a large margin.
- Opening book: matches in which Stockfish's opening moves came from the Brainfish program's Cerebellum opening book. These games were low in diversity and more than 90% were draws; when AlphaZero was made to vary its opening moves slightly, its winning rate rose from 5.8% to 14%. AlphaZero still won the matches.
- Time odds: AlphaZero defeated Stockfish even when given one-tenth as much thinking time as its opponent.
Chess.com's report on the Science results described AlphaZero as "dominant up to 10-to-1 odds" and said that "Stockfish only began to outscore AlphaZero when the odds reached 30-to-1."[9]
The 2017 preprint had also played 100-game matches from each of the 12 most popular human openings, 50 games with each color. Across those 1,200 games AlphaZero won 242, drew 353, and lost 5 as White, and won 48, drew 533, and lost 19 as Black.[2]
Shogi: AlphaZero vs. Elmo
In shogi, AlphaZero faced the version of Elmo that won the 2017 Computer Shogi Association (CSA) world championship (WCSC27), combined with the YaneuraOu search engine. In the Science evaluation, Elmo ran with 44 CPU threads and a 32 GB hash table, and the main match used the same 3 hours plus 15 seconds time control as the chess match.[1]
AlphaZero won 91.2% of games against Elmo overall and 98.2% of games when playing Black (sente, the player who moves first in shogi).[1][8] As White it won 84.2%, drew 2.2%, and lost 13.6%.[1] AlphaZero also won by a wide margin in a separate match at the faster 2017 CSA championship time control (10 minutes per game plus 10 seconds per move) and in a match against another strong shogi program, Aperyqhapaq. With one-hundredth of Elmo's thinking time, AlphaZero still won 46% of games.[1] In the 2017 preprint's 100-game match at one minute per move, AlphaZero won 90 games, drew 2, and lost 8.[2]
Shogi is in some ways more complex than Western chess: it is played on a larger board with a wider variety of pieces, and captured pieces change sides and can be dropped back onto the board.[1] AlphaZero handled these rules through its input and move encodings, without other game-specific modifications.[1]
Go: AlphaZero vs. AlphaGo Zero
In Go, AlphaZero played the previously published version of AlphaGo Zero; the Science paper describes it as the version trained for 3 days, and its main text adds that it, too, was trained for 700,000 steps.[1] This 3-day AlphaGo Zero had defeated AlphaGo Lee 100-0, although the final 40-day AlphaGo Zero was stronger.[3]
| Source | Time control | Games | AlphaZero result |
|---|---|---|---|
| 2017 preprint | 1 minute per move | 100 | 60 wins, 40 losses (31-19 as White, 29-21 as Black) |
| 2018 Science | 3 hours + 15 seconds per move | 1,000 | Won 61% of games (68.9% as White, 53.7% as Black) |
The Science paper presents this result as showing that "a general approach can recover the performance of an algorithm that exploited board symmetries to generate eight times as much data."[1] AlphaGo Zero had augmented its training data eightfold through rotations and reflections of the board, while AlphaZero did not use any such augmentation.[1][3]
Playing style
AlphaZero's playing style attracted intense interest from the chess community because it was so different from conventional computer chess. Presenting further details of the system at the Neural Information Processing Systems (NIPS) conference in Long Beach in December 2017, Demis Hassabis, an expert chess player himself, said: "It doesn't play like a human, and it doesn't play like a program. It plays in a third, almost alien, way."[11]
Material sacrifice and piece activity
The Science paper notes that "in several games AlphaZero sacrificed pieces for long-term strategic advantage, suggesting that it has a more fluid, context-dependent positional evaluation than the rule-based evaluations used by previous chess programs."[1] According to DeepMind, Grandmaster Matthew Sadler described AlphaZero's highly dynamic play as maximizing the activity and mobility of its own pieces while minimizing those of its opponent, and observed that it seems to place less value on material, willingly sacrificing material early for gains recouped only in the long term.[8]
Sadler and Women's International Master Natasha Regan, who examined more than two thousand previously unpublished AlphaZero games for their book Game Changer,[6] said its style was unlike any traditional chess engine. Sadler said the first thing players would notice is "the way its pieces swarm around the opponent's king with purpose and power," and compared the experience to "discovering the secret notebooks of some great player from the past."[8]
Grandmaster and expert reactions
In an editorial published in Science alongside the 2018 paper, former World Chess Champion Garry Kasparov wrote: "I admit that I was pleased to see that AlphaZero had a dynamic, open style like my own." He added that "AlphaZero prioritizes piece activity over material, preferring positions that to my eye looked risky and aggressive."[14] DeepMind's announcement of the Science paper quoted him as saying: "I can't disguise my satisfaction that it plays with a very dynamic style, much like my own!"[8]
After the 2017 preprint, Danish Grandmaster Peter Heine Nielsen told the BBC: "I always wondered how it would be if a superior species landed on earth and showed us how they played chess. Now I know."[13] Yoshiharu Habu, who in 1996 became the first player to hold all seven major shogi titles at the same time, said of AlphaZero's shogi: "Its unique playing style shows us that there are new possibilities for the game."[8]
Specific chess discoveries
Both papers analyzed how AlphaZero's opening choices developed during training. The most common human openings were independently discovered and played frequently by AlphaZero during self-play, and the Science paper includes plots and a timeline showing how often it chose each opening as training progressed.[1][2] In shogi, AlphaZero frequently played one of the two most common human openings but rarely played the second, deviating on the very first move.[1] DeepMind wrote that AlphaZero independently discovered common human chess motifs such as openings, king safety, and pawn structure, while also developing "its own intuitions and strategies."[8]
Impact on chess theory and practice
AlphaZero's games and approach have had a lasting effect on computer chess and on how strong players think about the game.
Influence on human players
The publisher of Game Changer quotes Magnus Carlsen, then the World Chess Champion: "Quite inspirational. I was thinking at several points during the game [in Round 11 of the 2019 Tata Steel Chess Tournament]: how would AlphaZero have approached this?"[6] Natasha Regan told DeepMind that AlphaZero "could be a powerful teaching tool for the whole community."[8]
Neural network chess engines
AlphaZero's success inspired open-source neural network chess engines, most notably Leela Chess Zero (Lc0). Stockfish co-author Gary Linscott announced the project on the Computer Chess Club forum on January 9, 2018, about a month after the AlphaZero preprint. Lc0 follows the AlphaZero approach of deep learning with MCTS, but trains its network weights through distributed computing contributed by volunteers.[15] It first entered the Top Chess Engine Championship in April 2018, reached the TCEC Season 14 superfinal (losing narrowly to Stockfish), and won the Season 15 superfinal against Stockfish in May 2019 and the Season 17 superfinal in April 2020.[15]
Stockfish itself adopted a neural network evaluation with Stockfish 12, released on September 2, 2020, which introduced an efficiently updatable neural network (NNUE) and used it by default. The Stockfish team said the NNUE concept "was first introduced in shogi, and ported to Stockfish afterward," and that Stockfish remained a CPU-only engine that uses its evaluation inside an alpha-beta search.[16]
Reimagining chess variants
In a 2020 study with former World Chess Champion Vladimir Kramnik, DeepMind researchers used AlphaZero to evaluate alternative chess rule sets. The paper, "Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess," compared nine variants that each change one rule of chess, including No-castling chess, Torpedo chess (in which pawns can move one or two squares anywhere on the board), Self-capture chess, and Stalemate=win. The authors wrote that AlphaZero let them "simulate the equivalent of decades of human play within a day." They found that pieces are valued differently between variants and that some variants are more decisive than classical chess.[7]
What AlphaZero learned
A 2022 study in the Proceedings of the National Academy of Sciences, "Acquisition of chess knowledge in AlphaZero," by Thomas McGrath and colleagues including Demis Hassabis and Vladimir Kramnik, examined what the network had learned. By probing the network for a broad range of human chess concepts, the authors provided evidence that human chess knowledge is acquired by AlphaZero's network as it trains, and showed when and where those concepts are represented. The study also analyzed AlphaZero's opening play, with qualitative analysis from Kramnik.[10]
How did the 2017 preprint differ from the 2018 Science paper?
The initial preprint released in December 2017 attracted both excitement and criticism. Stockfish co-author Tord Romstad told Chess.com that the preprint's match results were "not particularly meaningful because of the rather strange choice of time controls and Stockfish parameter settings," citing the fixed one minute per move, the year-old Stockfish version, the large number of search threads, and a hash table that was "way too small" for that many threads.[12] The Science paper ran a new, larger evaluation under TCEC-style conditions.[1]
| Aspect | 2017 preprint | 2018 Science paper |
|---|---|---|
| Network training hardware | 64 second-generation TPUs | 16 second-generation TPUs |
| Self-play hardware | 5,000 first-generation TPUs | 5,000 first-generation TPUs |
| Go training time and games | 34 hours, 21 million games | ~13 days, 140 million games |
| Go: first passed AlphaGo Lee | 8 hours (165,000 steps) | 30 hours (74,000 steps) |
| Time control | 1 minute per move | 3 hours + 15 seconds per move |
| Games per match | 100 | 1,000 |
| Stockfish version | Stockfish 8 | Stockfish 8; also the January 2018 development version (Stockfish 9) |
| Stockfish threads and hash | 64 threads, 1 GB | 44 threads on 44 cores, 32 GB |
| Endgame tablebases for Stockfish | Not mentioned | Syzygy tablebases |
| Elmo settings | 64 threads, 1 GB hash (YaneuraOu 4.73) | 44 threads, 32 GB hash (YaneuraOu 4.79) |
| Chess result | +28 =72 -0 | +155 =839 -6 |
| Shogi result | +90 =2 -8 | 91.2% wins |
| Go result | 60-40 | 61% wins |
| AlphaZero positions per second (chess / shogi) | 80,000 / 40,000 | 63,000 / 58,000 |
| Additional chess matches | 12 popular human openings | Human openings, TCEC openings, latest Stockfish, opening book, time odds |
| PUCT exploration rate | Same as AlphaGo Zero | Grows slowly with search time |
Sources for the table: the preprint[2] and the Science paper.[1] The 2018 paper gave Stockfish much better conditions, including endgame tablebases and a far larger hash table, and AlphaZero's margin remained clear.[1]
How does AlphaZero differ from AlphaGo, AlphaGo Zero, and MuZero?
The following table compares the four major iterations of DeepMind's game-playing AI systems.
| Feature | AlphaGo | AlphaGo Zero | AlphaZero | MuZero |
|---|---|---|---|---|
| Year | 2015-2017 | 2017 | 2017-2018 | 2019-2020 |
| Publication venue | Nature (2016) | Nature (2017) | Science (2018) | Nature (2020) |
| Games played | Go only | Go only | Chess, shogi, Go | Chess, shogi, Go, Atari (57 games) |
| Human data required | Yes (160,000 expert games) | No | No | No |
| Game rules required | Yes | Yes | Yes | No (learns a model of the environment) |
| Network architecture | Separate policy and value networks | Single dual-headed ResNet (20 or 40 blocks) | Single dual-headed ResNet (19 residual blocks, 256 filters) | Representation, dynamics, and prediction networks |
| Training method | Supervised learning + RL self-play | RL self-play only | RL self-play only | RL self-play only |
| Board symmetry exploitation | Yes | Yes (8x augmentation) | No | No |
| Search algorithm | MCTS combining a value network with rollouts | MCTS with neural network evaluation | MCTS with neural network evaluation | MCTS with learned model |
| Key advancement | First to beat a professional Go player | Removed need for human data in Go | Generalized to multiple games | Removed need for known game rules |
MuZero: the successor
MuZero, released by DeepMind as a preprint in November 2019 and published in Nature in December 2020, extended AlphaZero's approach by removing the requirement for a known game model. While AlphaZero needed the exact rules of each game to simulate future positions during MCTS, MuZero learned its own internal model of the environment. This model did not attempt to reconstruct the full game state; instead, it learned to predict only the quantities most relevant to planning: the reward, the action-selection policy, and the value function.[5]
MuZero achieved this through three learned functions working together:[5]
- Representation function: Converts an observation (e.g., the current board state) into a hidden state.
- Dynamics function: Given a hidden state and an action, predicts the next hidden state and the immediate reward.
- Prediction function: Given a hidden state, outputs a policy and value estimate (similar to AlphaZero's dual-headed network).
Without any knowledge of the game rules, MuZero matched AlphaZero's superhuman performance in chess and shogi and slightly exceeded it in Go, despite using less computation per node in the search tree (16 residual blocks per evaluation, compared with what the MuZero paper counts as 20 blocks in AlphaZero). It also set a new state of the art for mean and median human-normalized score across 57 Atari games, outperforming the previous best method, R2D2 (Recurrent Replay Distributed DQN).[5]
Technical details
Input representation
The input to AlphaZero's neural network varies by game. Each input is a stack of planes covering the current position and the seven preceding positions, plus a few constant-valued planes:[1]
- Chess: 119 input planes of 8x8. These encode each player's six piece types and two repetition planes for each of the eight positions, plus planes for the side to move, the total move count, each side's castling rights, and the no-progress count.
- Shogi: 362 input planes of 9x9. Shogi requires more input planes because there are more piece types (including promoted pieces), and captured pieces ("prisoners") must be counted for each player.
- Go: 17 input planes of 19x19: each player's stones for the last eight positions, plus one plane for the side to move.
Move representation
Moves are encoded as outputs of the neural network's policy head:[1]
- Chess: 4,672 possible moves, encoded as 73 planes of 8x8. Each square identifies the piece to "pick up"; 56 planes encode "queen moves" of one to seven squares in eight directions, 8 planes encode knight moves, and 9 planes encode underpromotions to a knight, bishop, or rook.
- Shogi: 11,259 possible moves, encoded as 139 planes of 9x9 (queen moves, knight moves, their promoting versions, and 7 planes for dropping a captured piece).
- Go: 362 possible moves (one for each intersection plus a pass move), as a flat distribution.
Illegal moves are masked out by setting their probabilities to zero and renormalizing. The authors reported that a flat move distribution for chess and shogi gave almost identical final results, although training was slightly slower.[1]
Loss function
The neural network is trained to minimize a combined loss:[1]
where is the actual game outcome (+1 for win, -1 for loss, 0 for draw), is the predicted value, is the MCTS search probability vector, is the predicted policy, and is an L2 regularization term that helps prevent overfitting. The first term trains the value head, the second term trains the policy head, and the regularization term keeps the weights small.[1]
Domain knowledge provided
The Science paper lists the only domain knowledge given to AlphaZero: the grid structure of the board used in the input and output planes; perfect knowledge of the game rules, used during MCTS to simulate positions, detect the end of the game, and score terminal positions; the encodings of special rules such as castling, repetition, promotions, and shogi piece drops; the typical number of legal moves, used to scale exploration noise; and the maximum game lengths. It did not use an opening book, endgame tablebases, or domain-specific heuristics.[1]
Limitations and criticisms
AlphaZero, despite its achievements, has several notable limitations.
Hardware requirements
Training AlphaZero used 5,000 first-generation TPUs for self-play generation and, according to the Science paper, 16 second-generation TPUs for neural network training (64 in the preprint).[1][2] Romstad argued that the preprint's "4 hours of learning" was "highly misleading when you take into account the massive hardware resources used during those 4 hours," and predicted that the resources needed to create such programs would remain beyond the budget of hobbyists.[12] Leela Chess Zero later pursued the approach with a distributed network of volunteer computers, taking more than a year from its January 2018 announcement to its first TCEC superfinal win in May 2019.[15]
Closed-source
AlphaZero was never released as open-source software, and the trained networks were not made publicly available. The Science paper's supplementary materials did include a pseudocode description of the algorithm, 210 chess games against Stockfish, and 100 shogi games against Elmo.[1][8] The lack of public access motivated open-source re-implementations such as Leela Chess Zero.[15]
Perfect information games only
AlphaZero was designed for and tested on two-player games of perfect information, and its search relies on perfect knowledge of the game rules to simulate the positions that follow a sequence of moves.[1] The MuZero paper notes that tree-based planning methods like AlphaZero's had succeeded in domains "where a perfect simulator is available," whereas in real-world problems the dynamics are often complex and unknown. MuZero addressed this by learning a model of the environment rather than requiring the rules.[5]
Match conditions debate
The fairness of the Stockfish comparison was debated. Reacting to the 2017 preprint, Grandmaster Hikaru Nakamura said: "I don't necessarily put a lot of credibility in the results simply because my understanding is that AlphaZero is basically using the Google super computer and Stockfish doesn't run on that hardware."[12] Romstad called the comparison one of "apples to orangutans," between a conventional program on ordinary computers and one running on custom hardware "that is not available for purchase."[12] The Science paper responded by running Stockfish under its 2016 TCEC superfinal settings and noting that each program ran on the hardware for which it was designed: Stockfish is built to exploit CPUs and cannot use GPUs or TPUs, while AlphaZero is built for GPU or TPU hardware.[1]
Legacy and broader significance
AlphaZero demonstrated that a single algorithm (self-play reinforcement learning with MCTS and a deep neural network) could achieve superhuman performance across multiple board games without domain-specific human knowledge or data.[1] This was a significant result for artificial intelligence and machine learning because it showed that handcrafted evaluation functions and domain-specific search enhancements, which had been the foundation of game-playing AI for decades, could be replaced by learned representations in these games.[1]
DeepMind wrote that AlphaZero, being self-taught and "unconstrained by conventional wisdom about the game," developed its own intuitions and strategies, "adding a new and expansive set of exciting and novel ideas that augment centuries of thinking about chess strategy."[8] Kasparov wrote in his Science editorial that "AlphaZero shows us that machines can be the experts, not merely expert tools," and that the knowledge it generates "is information we can all learn from."[14]
Matthew Sadler and Natasha Regan documented AlphaZero's chess strategies in Game Changer: AlphaZero's Groundbreaking Chess Strategies and the Promise of AI, published by New in Chess in early 2019, with a foreword by Garry Kasparov and an introduction by Demis Hassabis.[6] The book won the English Chess Federation's Book of the Year award for 2019[17] and FIDE's Averbakh-Boleslavsky Award for the best book of 2019.[18]
The AlphaZero algorithm has also been carried beyond games. DeepMind's AlphaTensor, which searched for faster matrix multiplication algorithms, is built on AlphaZero;[19] the primary learning algorithm of AlphaDev, which discovered faster sorting routines, is an extension of AlphaZero;[20] and AlphaProof, a system for formal mathematical proof, is described as "AlphaZero-inspired."[21]
Continue reading
- AlphaGoAlphaGo is a computer program developed by DeepMind that plays the board game Go, and it was the first program to defeat a human professional Go player on a full 19x19 board without a handicap.Mentioned 47 times above
- AlphaGo ZeroAlphaGo Zero is a Go-playing computer program developed by DeepMind that reached a superhuman level entirely through self-play reinforcement learning, starting from random play with no human game data.Mentioned 23 times above
- Google DeepMindGoogle DeepMind is an artificial intelligence research and model-development organization at Google.Mentioned 15 times above
- MuZeroMuZero is a model-based reinforcement learning algorithm developed by DeepMind that reached superhuman play in Go, chess, and shogi and set a new state of the art on the 57-game Atari benchmark without being…Mentioned 10 times above
- Reinforcement learningReinforcement learning (RL) is a branch of machine learning in which an agent learns to make decisions by taking actions in an environment to maximize a cumulative reward signal, discovering good behavior…Mentioned 7 times above
- Monte Carlo Tree SearchMonte Carlo Tree Search (MCTS) is a heuristic search algorithm for sequential decision-making that finds strong actions by running many simulated playthroughs of a problem and aggregating their outcomes into a…Mentioned twice above
More from See also
More to read
Topics:AI in GamingArtificial IntelligenceGoogle DeepMindReinforcement Learning
References
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20 ^21 ^22 ^23 ^24 ^25 ^26 ^27 ^28 ^29 ^30 ^31 ^32 ^33 ^34 ^35 ^36 ^37 ^38 ^39 ^40 ^41 ^42 ^43 ^44 ^45 ^46 ^47 ^48 ^49 ^50 ^51 ^52 ^53 ^54 ^55 ^56 ^57 ^58 ^59 ^60 ^61 ^62 ^63 ^64 ^65 ^66Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., Lillicrap, T., Simonyan, K., & Hassabis, D. (2018). "A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play." *Science*, 362(6419), 1140-1144. doi.org/...science.aar6404 (open-access author version with supplementary materials: storage.googleapis.com/...alphazero_preprint.pdf)
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., Lillicrap, T., Simonyan, K., & Hassabis, D. (2017). "Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm." *arXiv preprint arXiv:1712.01815*. arxiv.org/...1712.01815
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., Chen, Y., Lillicrap, T., Hui, F., Sifre, L., van den Driessche, G., Graepel, T., & Hassabis, D. (2017). "Mastering the game of Go without human knowledge." *Nature*, 550(7676), 354-359. doi.org/...nature24270
- ^1 ^2 ^3 ^4Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., & Hassabis, D. (2016). "Mastering the game of Go with deep neural networks and tree search." *Nature*, 529(7587), 484-489. doi.org/...nature16961
- ^1 ^2 ^3 ^4Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., Lillicrap, T., & Silver, D. (2020). "Mastering Atari, Go, chess and shogi by planning with a learned model." *Nature*, 588(7839), 604-609. doi.org/...s41586-020-03051-4 (preprint: arXiv:1911.08265, arxiv.org/...1911.08265)
- ^1 ^2 ^3Sadler, M., & Regan, N. (2019). *Game Changer: AlphaZero's Groundbreaking Chess Strategies and the Promise of AI*. New in Chess. Publisher's description: newinchess.com/game-changer (archived: web.archive.org/...game-changer)
- ^Tomašev, N., Paquet, U., Hassabis, D., & Kramnik, V. (2020). "Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess." *arXiv preprint arXiv:2009.04374*. arxiv.org/...2009.04374
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12Silver, D., Hubert, T., Schrittwieser, J., & Hassabis, D. (2018, December 6). "AlphaZero: Shedding new light on chess, shogi, and Go." Google DeepMind Blog. deepmind.google/...new-light-on-chess-shogi-and-go
- ^1 ^2"AlphaZero Crushes Stockfish In New 1,000-Game Match." Chess.com News, December 2018 (updated April 17, 2019). chess.com/...hes-stockfish-in-new-1-000-game-match
- ^McGrath, T., Kapishnikov, A., Tomašev, N., Pearce, A., Wattenberg, M., Hassabis, D., Kim, B., Paquet, U., & Kramnik, V. (2022). "Acquisition of chess knowledge in AlphaZero." *Proceedings of the National Academy of Sciences*, 119(47), e2206625119. doi.org/...pnas.2206625119
- ^Knight, W. (2017, December 8). "Alpha Zero's 'Alien' Chess Shows the Power, and the Peculiarity, of AI." *MIT Technology Review*. technologyreview.com/...-and-the-peculiarity-of-ai
- ^1 ^2 ^3 ^4 ^5Doggers, P. (2017, December). "AlphaZero Chess: Reactions From Top GMs, Stockfish Author." Chess.com News. chess.com/...actions-from-top-gms-stockfish-author
- ^"Google's 'superhuman' DeepMind AI claims chess crown." BBC News, December 6, 2017. bbc.com/...technology-42251535
- ^1 ^2Kasparov, G. (2018). "Chess, a Drosophila of reasoning." *Science*, 362(6419), 1087. doi.org/...science.aaw2221
- ^1 ^2 ^3 ^4"Leela Chess Zero." Chess Programming Wiki. chessprogramming.org/Leela_Chess_Zero
- ^The Stockfish team (2020, September 2). "Stockfish 12." Stockfish blog. stockfishchess.org/...stockfish-12
- ^English Chess Federation. "BOCC Commentary" (biographies of Matthew Sadler and Natasha Regan). englishchess.org.uk
- ^"'Game Changer' wins the Averbakh-Boleslavsky Award 2019." ChessBase, 2020. en.chessbase.com/...verbakh-boleslavsky-award-2019
- ^Fawzi, A., Balog, M., Huang, A., et al. (2022). "Discovering faster matrix multiplication algorithms with reinforcement learning." *Nature*, 610(7930), 47-53. doi.org/...s41586-022-05172-4
- ^Mankowitz, D. J., Michi, A., Zhernov, A., et al. (2023). "Faster sorting algorithms discovered using deep reinforcement learning." *Nature*, 618(7964), 257-263. doi.org/...s41586-023-06004-9
- ^Hubert, T., Mehta, R., Sartran, L., et al. (2025). "Olympiad-level formal mathematical reasoning with reinforcement learning." *Nature*. doi.org/...s41586-025-09833-y
- ^Hassabis, D., & Silver, D. (2017, May 27). "AlphaGo's next move." Google DeepMind Blog. deepmind.google/...alphagos-next-move
Improve this article
Add missing citations, update stale details, or write a clearer explanation. Anyone can edit: changes go live when you publish, and every edit is checked by an editor.
12 revisions · v12 · 7,001 words · full history
Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify
Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here
Reviewer note: Fact-checked on 2026-10-30 against the 2017 arXiv preprint and the 2018 Science paper with its supplementary tables, DeepMind's blog posts, Kasparov's Science editorial, contemporaneous chess-press reactions, and the follow-up papers cited, with an independent second review.
Cite this page: AI Wiki. "AlphaZero." aiwiki.ai, updated 30 Oct 2026, fact-checked 30 Oct 2026. CC BY 4.0. https://aiwiki.ai/wiki/alphazero



