# NASNet

> Source: https://aiwiki.ai/wiki/nasnet
> Updated: 2026-11-18
> Fact-checked: 2026-11-18
> Categories: Computer Vision, Deep Learning, Google, Model Architecture
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "NASNet." aiwiki.ai, 18 Nov 2026. https://aiwiki.ai/wiki/nasnet
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

**NASNet** is a family of [convolutional neural network](https://aiwiki.ai/wiki/convolutional_neural_network) architectures for [image classification](https://aiwiki.ai/wiki/image_classification) whose building blocks were found by [neural architecture search](https://aiwiki.ai/wiki/neural_architecture_search) rather than designed by hand. It was introduced by [Barret Zoph](https://aiwiki.ai/wiki/barret_zoph), Vijay Vasudevan, Jonathon Shlens and [Quoc V. Le](https://aiwiki.ai/wiki/quoc_le) of [Google Brain](https://aiwiki.ai/wiki/google_brain) in the paper "Learning Transferable Architectures for Scalable Image Recognition", first posted to arXiv on 21 July 2017 and published at the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition ([CVPR](https://aiwiki.ai/wiki/cvpr) 2018).[1][2] Instead of searching for a whole network, the authors searched for a small repeating unit, a convolutional "cell", on the small [CIFAR-10](https://aiwiki.ai/wiki/cifar_10) dataset and then stacked copies of that cell to build networks for the much larger [ImageNet](https://aiwiki.ai/wiki/imagenet) dataset.[1] The paper calls the set of possible cells the "NASNet search space" and the resulting networks "NASNet architectures".[1]

The largest model, NASNet-A (6 @ 4032), reached 82.7% top-1 and 96.2% top-5 accuracy on ImageNet, which the authors reported as the best result among published works at the time, while a small version reached 74.0% top-1 in a mobile-sized compute budget.[1] Features learned by NASNet also improved [object detection](https://aiwiki.ai/wiki/object_detection) on the [COCO](https://aiwiki.ai/wiki/coco_dataset) benchmark when used inside the [Faster R-CNN](https://aiwiki.ai/wiki/faster_r_cnn) framework.[1] Google released pretrained NASNet models in November 2017, and the cell-based search space became a common starting point for later architecture search methods such as AmoebaNet, PNAS, ENAS and DARTS.[3][9][10][11][12]

## Background

NASNet builds on "Neural Architecture Search with Reinforcement Learning" by Zoph and Le, posted in November 2016 and presented at ICLR 2017.[1][4] In that earlier method, a [recurrent neural network](https://aiwiki.ai/wiki/recurrent_neural_network) controller generates descriptions of candidate networks and is trained with [reinforcement learning](https://aiwiki.ai/wiki/reinforcement_learning) to maximize the validation accuracy of the networks it generates; its best CIFAR-10 model reached a test error rate of 3.65.[4] Each sampled "child" network has to be trained before the controller receives its reward, so the search is expensive: the NASNet paper notes that the earlier search used 800 GPUs for 28 days.[1]

Zoph and colleagues observed that running such a search directly on a large dataset like ImageNet would be computationally expensive.[1] In Google's announcement of the work, the authors wrote that applying their method directly to ImageNet would have required many months of training.[3] Their solution was to search on CIFAR-10 as a proxy and to design the search space so that what is learned there carries over to larger images and deeper networks. They noted that successful hand-designed networks such as [Inception](https://aiwiki.ai/wiki/inception) and [ResNet](https://aiwiki.ai/wiki/resnet) are built from repeated motifs, which suggested that a controller could learn one generic cell to be stacked many times.[1]

## The NASNet search space

### Normal and reduction cells

Every network in the NASNet search space has a fixed outer structure chosen by hand: a stack of convolutional cells that share the same architecture but have their own weights.[1] There are two kinds of cell:[1]

| Cell type | Function |
|---|---|
| Normal Cell | Returns a feature map of the same height and width as its input |
| Reduction Cell | Returns a feature map whose height and width are halved; its first operations use a stride of 2 |

The authors found it worked better to learn the two cell types as separate architectures, although they could have been the same.[1] Following a common heuristic, the number of filters is doubled whenever the spatial size is reduced.[1] Two numbers stay free and are tuned to the problem: the number of Normal Cells stacked between Reduction Cells, $N$, and the number of filters. The paper writes a configuration as $N$ @ $F$; for example, 4 @ 64 means 4 cell repeats and 64 filters in the penultimate layer.[1] Because ImageNet inputs are 299x299 rather than CIFAR-10's 32x32, the ImageNet networks contain more Reduction Cells than the CIFAR-10 ones.[1]

The authors did not add residual connections between cells. They wrote that the models learn skip connections on their own and that manually inserting [residual connections](https://aiwiki.ai/wiki/residual_connection) between cells did not improve performance.[1]

### How a cell is constructed

Each cell takes two inputs, $h_i$ and $h_{i-1}$: the outputs of the two previous cells, or the input image.[1] The controller builds the cell from $B$ blocks. For each block it makes five decisions, each produced by its own softmax classifier:[1]

1. Choose a hidden state from $h_i$, $h_{i-1}$ or the states created by earlier blocks.
2. Choose a second hidden state from the same options.
3. Choose an operation to apply to the first hidden state.
4. Choose an operation to apply to the second hidden state.
5. Choose how to combine the two results: element-wise addition or concatenation along the filter dimension.

The new hidden state is added to the pool that later blocks can select from. After the last block, all hidden states that were not used as inputs inside the cell are concatenated to form the cell's output.[1] The paper used $B = 5$, noting that this gave good results but that the choice had not been searched exhaustively because of computational limits.[1] Since the controller predicts both a Normal Cell and a Reduction Cell, it makes $2 \times 5B$ predictions per architecture, or 50 when $B = 5$.[1]

The candidate operations were collected, in the authors' words, "based on their prevalence in the CNN literature":[1]

| Category | Operations |
|---|---|
| Identity | identity |
| Standard [convolutions](https://aiwiki.ai/wiki/convolution) | 1x1, 3x3 |
| Factorized convolutions | 1x3 then 3x1, 1x7 then 7x1 |
| Dilated convolution | 3x3 dilated |
| Depthwise-separable convolutions | 3x3, 5x5, 7x7 |
| [Pooling](https://aiwiki.ai/wiki/pooling) | 3x3 average; 3x3, 5x5 and 7x7 max |

Several implementation details were fixed by hand after preliminary experiments. All convolutions use [ReLU](https://aiwiki.ai/wiki/relu) and follow the order ReLU, convolution, [batch normalization](https://aiwiki.ai/wiki/batch_normalization). Unlike [MobileNet](https://aiwiki.ai/wiki/mobilenet), the depthwise-separable convolutions have no batch normalization or ReLU between the depthwise and pointwise steps, and any separable convolution the controller selects is applied twice, which the authors found improved performance. 1x1 convolutions are inserted where needed so that tensor shapes match.[1]

## Search procedure

The controller is a one-layer [LSTM](https://aiwiki.ai/wiki/lstm) with 100 hidden units.[1] The probability of a child network is the product of the probabilities of all the controller's choices, and the gradient of that probability is scaled by the child's validation accuracy, so the controller learns to assign high probability to good architectures and low probability to bad ones.[1] Where the original NAS work updated the controller with the REINFORCE rule, NASNet used [Proximal Policy Optimization](https://aiwiki.ai/wiki/ppo) (PPO) with a learning rate of 0.00035, which the authors found faster and more stable, plus an entropy penalty (weight 0.00001) to encourage exploration.[1]

Training was distributed through a global work queue: the controller's samples were placed on the queue and free workers trained them. The appendix describes a pool of 450 child workers training 450 networks on 450 GPUs at a time, while the main text describes the worker pool as 500 GPUs.[1] Key settings of the search were:[1]

| Setting | Value |
|---|---|
| Search dataset | CIFAR-10, with 5,000 training images held out as the controller's validation set |
| Child network training | 20 epochs, cosine learning-rate decay, small stacks ($N = 2$) |
| Controller update | Minibatches of 20 architectures |
| Stopping point | 20,000 child models trained |
| Final selection | Top 250 architectures trained to convergence on CIFAR-10 |
| Duration | 4 days on 500 GPUs |

The authors described the search as almost 7 times faster than the original NAS search, which took 28 days. In a footnote they put the original search at 800 GPUs for 28 days and the new one at 500 GPUs for 4 days; the footnote labels the totals 22,400 and 2,000 "GPU-hours", although the GPU counts multiplied by the days give GPU-days. They also noted that the original search used Nvidia K40 GPUs while NASNet used faster Nvidia P100s, and estimated that, discounting the faster hardware, the new procedure was roughly 7 times more efficient.[1] Later papers counted the NASNet search differently: DARTS cites it as 2,000 GPU days, and the ENAS paper describes it as 450 GPUs for 3 to 4 days, or 32,400 to 43,200 GPU hours.[11][12]

The three networks built from the best three searches were named NASNet-A, NASNet-B and NASNet-C. NASNet-A, built from the top-performing cells, is the one the rest of the paper focuses on.[1] NASNet-B and NASNet-C came from slightly different search spaces: NASNet-B uses $B = 4$, passes every hidden state to the following layers instead of concatenating unused ones, and allows addition followed by layer normalization or instance normalization as a combination step; NASNet-C also uses $B = 4$ and keeps NASNet-A's concatenation of unused hidden states, but also allows those normalized additions.[1] A later paper by Liu, Zoph and colleagues remarked that the NASNet-A cell had originally ranked 70th among the top 250 before the final round of full training.[10]

### Reinforcement learning compared with random search

The paper notes that the same search space can be explored with random search by sampling each decision uniformly instead of from the controller.[1] Random search did slightly worse than reinforcement learning on CIFAR-10, and the gap was smaller than in the original NAS work. The authors read this as evidence that the search space was well constructed and that random search is a hard baseline to beat.[1] In a comparison with equal computational resources, measured by validation accuracy after 20 epochs of training, the best model found by reinforcement learning beat the best model found by random search by more than 1%, and reinforcement learning also found more high-quality models overall.[1]

## ScheduledDropPath

While training NASNets, the authors found that applying ordinary [dropout](https://aiwiki.ai/wiki/dropout) across convolutional filters hurt performance.[1] They instead modified DropPath, a technique from the FractalNet paper in which each path through a cell (each operation edge) is randomly dropped with a fixed probability during training. Plain DropPath did not help NASNet much. In the new version, called **ScheduledDropPath**, the drop probability increases linearly over the course of training, and the authors reported that this significantly improved final performance on both CIFAR-10 and ImageNet.[1]

## Results

### CIFAR-10

For CIFAR-10 the cells were stacked with $N = 4$ or $6$, and all NASNet results are averages over five runs.[1] The large NASNet-A trained with cutout [data augmentation](https://aiwiki.ai/wiki/data_augmentation) reached a 2.40% error rate, slightly better than the previous best of 2.56%; its best single run reached 2.19%.[1]

| Model | Parameters | Error rate (%) |
|---|---|---|
| NAS v3 (Zoph and Le) | 37.4M | 3.65 |
| Shake-Shake 26 2x96d + cutout | 26.2M | 2.56 |
| NASNet-A (6 @ 768) | 3.3M | 3.41 |
| NASNet-A (6 @ 768) + cutout | 3.3M | 2.65 |
| NASNet-A (7 @ 2304) | 27.6M | 2.97 |
| NASNet-A (7 @ 2304) + cutout | 27.6M | 2.40 |
| NASNet-B (4 @ 1152) | 2.6M | 3.73 |
| NASNet-C (4 @ 640) | 3.1M | 3.59 |

Source: Table 1 of the paper (selected rows).[1]

### ImageNet

The cell architectures were transferred from CIFAR-10, but every ImageNet model was trained from scratch.[1] Training used the ImageNet 2012 ILSVRC data (about 1.2 million images in 1,000 classes), synchronous SGD across 50 workers plus 3 backup workers on Tesla K40 GPUs, [RMSProp](https://aiwiki.ai/wiki/rmsprop), [label smoothing](https://aiwiki.ai/wiki/label_smoothing) of 0.1, an auxiliary classifier whose loss was weighted 0.4, and dropout of 0.5 on the final softmax layer.[1]

| Model | Image size | Parameters | Mult-adds | Top-1 (%) | Top-5 (%) |
|---|---|---|---|---|---|
| Inception V2 | 224x224 | 11.2M | 1.94B | 74.8 | 92.2 |
| NASNet-A (5 @ 1538) | 299x299 | 10.9M | 2.35B | 78.6 | 94.2 |
| Inception V3 | 299x299 | 23.8M | 5.72B | 78.8 | 94.4 |
| [Xception](https://aiwiki.ai/wiki/xception) | 299x299 | 22.8M | 8.38B | 79.0 | 94.5 |
| Inception ResNet V2 | 299x299 | 55.8M | 13.2B | 80.1 | 95.1 |
| NASNet-A (7 @ 1920) | 299x299 | 22.6M | 4.93B | 80.8 | 95.3 |
| ResNeXt-101 (64 x 4d) | 320x320 | 83.6M | 31.5B | 80.9 | 95.6 |
| PolyNet | 331x331 | 92M | 34.7B | 81.3 | 95.8 |
| DPN-131 | 320x320 | 79.5M | 32.0B | 81.5 | 95.8 |
| SENet | 320x320 | 145.8M | 42.3B | 82.7 | 96.2 |
| NASNet-A (6 @ 4032) | 331x331 | 88.9M | 23.8B | 82.7 | 96.2 |

Source: Table 2 of the paper.[1]

The largest NASNet-A beat the best previously published single-model result by about 1.2 percentage points and matched SENet, then the best unpublished arXiv result, with far fewer floating-point operations.[1] Google's announcement described this as achieving state-of-the-art accuracy while halving SENet's computational cost.[3] The authors also reported that each NASNet model outperformed the hand-designed model of comparable cost.[1]

In the mobile setting (224x224 images and fewer than 1.5 billion mult-adds per image), NASNet-A beat the hand-designed MobileNet and ShuffleNet by about 3 points of top-1 accuracy at similar cost:[1]

| Model | Parameters | Mult-adds | Top-1 (%) | Top-5 (%) |
|---|---|---|---|---|
| Inception V1 | 6.6M | 1,448M | 69.8 | 89.9 |
| MobileNet-224 | 4.2M | 569M | 70.6 | 89.5 |
| ShuffleNet (2x) | about 5M | 524M | 70.9 | 89.8 |
| NASNet-A (4 @ 1056) | 5.3M | 564M | 74.0 | 91.6 |
| NASNet-B (4 @ 1536) | 5.3M | 488M | 72.8 | 91.3 |
| NASNet-C (3 @ 960) | 4.9M | 558M | 72.5 | 91.0 |

Source: Table 3 of the paper.[1]

### Object detection on COCO

To test whether NASNet features [transfer](https://aiwiki.ai/wiki/transfer_learning) to other vision tasks, the authors plugged ImageNet-pretrained NASNet-A networks into the Faster R-CNN detection pipeline and retrained on COCO.[1] The mobile NASNet-A reached 29.6% mAP on the COCO mini-validation set, more than 5 points above earlier mobile-optimized Faster R-CNN systems. The large NASNet-A reached 40.7% mAP on test-dev at 800x800 input, 4.0 points above comparable Inception-ResNet-v2 systems, and 43.1% at 1200x1200, which the authors reported as the best single-model detection result, more than 4.0 points above the previous best.[1]

| Featurizer | Resolution | mAP mini-val (%) | mAP test-dev (%) |
|---|---|---|---|
| MobileNet-224 | 600x600 | 19.8 | - |
| ShuffleNet (2x) | 600x600 | 24.5 | - |
| NASNet-A (4 @ 1056) | 600x600 | 29.6 | - |
| Inception-ResNet-v2 (G-RMI) | 600x600 | 35.7 | 35.6 |
| ResNet-101-FPN (RetinaNet) | 800 (short side) | - | 39.1 |
| NASNet-A (6 @ 4032) | 800x800 | 41.3 | 40.7 |
| NASNet-A (6 @ 4032) | 1200x1200 | 43.2 | 43.1 |

Source: Table 4 of the paper (selected rows).[1] The authors noted that the RetinaNet result owed much to a new loss function and suggested that combining that loss with NASNet features might give further gains.[1]

## Release and implementations

Google announced NASNet on its research blog on 2 November 2017, in a post by the four authors titled "AutoML for large scale image classification and object detection".[3] The post said Google had open-sourced NASNet for inference on image classification and for object detection, through the TensorFlow-Slim and Object Detection repositories of the [TensorFlow](https://aiwiki.ai/wiki/tensorflow) models project.[3] The search code itself was not part of the release.[3][14]

The TensorFlow-Slim directory provides three NASNet-A configurations: one built for CIFAR-10 and two ImageNet variants with downloadable checkpoints.[5] The same directory also holds the code for PNASNet, a later model from the follow-up PNAS paper.[6]

| Checkpoint | Million mult-adds | Million parameters | Top-1 (%) | Top-5 (%) |
|---|---|---|---|---|
| NASNet-A_Mobile_224 | 564 | 5.3 | 74.0 | 91.6 |
| NASNet-A_Large_331 | 23,800 | 88.9 | 82.7 | 96.2 |

Source: TensorFlow-Slim NASNet README.[5]

[Keras](https://aiwiki.ai/wiki/keras) includes the two ImageNet models as `keras.applications.NASNetLarge` and `keras.applications.NASNetMobile`, with pretrained ImageNet weights that expect 331x331 and 224x224 inputs respectively.[7] The Keras applications table lists NASNetLarge at 88.9M parameters, 82.5% top-1 and 96.0% top-5, and NASNetMobile at 5.3M parameters, 74.4% top-1 and 91.9% top-5.[8]

## Influence and later work

NASNet's cell-based design became a shared testbed for comparing search methods:

| Work | Relationship to NASNet |
|---|---|
| Regularized evolution / AmoebaNet (Real, Aggarwal, Huang and Le; AAAI 2019) | Ran an evolutionary algorithm in the NASNet search space so it could be compared directly with the reinforcement learning search; a scaled-up AmoebaNet-A reached 83.9% top-1 and 96.6% top-5 on ImageNet[9] |
| PNAS (Liu, Zoph and others; ECCV 2018) | Searched cells in order of increasing complexity with a learned surrogate model; under the same search space it reported being up to 5 times more efficient than the NASNet method in models evaluated and 8 times faster in total compute. PNASNet-5 reached 82.9% top-1 (large) and 74.2% (mobile)[10] |
| ENAS (Pham, Guan, Zoph, Le and Dean; 2018) | Shared weights among child models and designed NASNet-style convolutional and reduction cells; reported a cost about 1,000 times lower than standard NAS and a CIFAR-10 error of 2.89%, compared with NASNet's 2.65%[11] |
| DARTS (Liu, Simonyan and Yang; ICLR 2019) | Followed Zoph et al. (2018) in searching for a cell whose two inputs are the outputs of the previous two layers, but relaxed the choice of operation so it could be optimized by gradient descent; reported 1.5 or 4 GPU days of search against 2,000 for NASNet[12] |
| MnasNet (Tan and others; CVPR 2019) | Measured real latency on phones during the search; reported 75.2% ImageNet top-1, 2.3 times faster than NASNet with 1.2 points higher accuracy[13] |

## Criticism and limitations

The main limitation was cost. Although the proxy-dataset approach was cheaper than the original NAS method, it still required hundreds of GPUs for several days, and later methods such as ENAS and DARTS were motivated by the expense of training thousands of child networks from scratch.[11][12]

Reproducibility has also been questioned. Liam Li and Ameet Talwalkar (UAI 2019) wrote that the earliest NAS results, including NASNet, lacked exact and broad reproducibility because of the computation they required, and that although the final architectures were eventually released, the code for the search methods had not been, which meant researchers could not reproduce the results even with enough compute.[14]

Other work has questioned how much the search itself contributes. The NASNet paper already found random search only slightly worse than reinforcement learning in its own search space.[1] Antoine Yang, Pedro Esperança and Fabio Carlucci (ICLR 2020) benchmarked eight fast NAS methods (not including NASNet) on five datasets and found that many of them struggled to significantly beat the average randomly sampled architecture. In further experiments on the widely used DARTS search space, a cell-based space that follows the NASNet design of two-input cells,[12] they found that 200+ sampled architectures all fell within one percentage point of top-1 accuracy on CIFAR-10 after full training, that training tricks had a predominant effect on reported performance, and that the hand-designed macro-structure mattered more than the searched operations.[15]

## See also

- [Neural architecture search](https://aiwiki.ai/wiki/neural_architecture_search)
- [EfficientNet](https://aiwiki.ai/wiki/efficientnet)
- [AutoML](https://aiwiki.ai/wiki/automl)
- [Inception](https://aiwiki.ai/wiki/inception)
- [MobileNet](https://aiwiki.ai/wiki/mobilenet)
- [Hyperparameter tuning](https://aiwiki.ai/wiki/hyperparameter_tuning)

## References

1. Zoph, Barret; Vasudevan, Vijay; Shlens, Jonathon; Le, Quoc V. "Learning Transferable Architectures for Scalable Image Recognition". arXiv:1707.07012, submitted 21 July 2017, revised 11 April 2018 (v4). https://arxiv.org/abs/1707.07012
2. Crossref. "Learning Transferable Architectures for Scalable Image Recognition" (Zoph, Vasudevan, Shlens, Le). 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8697-8710, June 2018. DOI 10.1109/CVPR.2018.00907. https://api.crossref.org/works/10.1109/cvpr.2018.00907
3. Zoph, Barret; Vasudevan, Vijay; Shlens, Jonathon; Le, Quoc. "AutoML for large scale image classification and object detection". Google Research Blog, 2 November 2017. https://research.google/blog/automl-for-large-scale-image-classification-and-object-detection/
4. Zoph, Barret; Le, Quoc V. "Neural Architecture Search with Reinforcement Learning". arXiv:1611.01578, 5 November 2016 (ICLR 2017). https://arxiv.org/abs/1611.01578
5. TensorFlow. "TensorFlow-Slim NASNet-A Implementation/Checkpoints". tensorflow/models repository, GitHub. https://github.com/tensorflow/models/blob/master/research/slim/nets/nasnet/README.md
6. TensorFlow. "TensorFlow-Slim image classification model library" (pre-trained models table). tensorflow/models repository, GitHub. https://github.com/tensorflow/models/blob/master/research/slim/README.md
7. Keras. "NASNetLarge and NASNetMobile". Keras API documentation. https://keras.io/api/applications/nasnet/nasnet_models/
8. Keras. "Keras Applications". Keras API documentation. https://keras.io/api/applications/
9. Real, Esteban; Aggarwal, Alok; Huang, Yanping; Le, Quoc V. "Regularized Evolution for Image Classifier Architecture Search". arXiv:1802.01548, 5 February 2018 (AAAI 2019). https://arxiv.org/abs/1802.01548
10. Liu, Chenxi; Zoph, Barret; Neumann, Maxim; Shlens, Jonathon; Hua, Wei; Li, Li-Jia; Fei-Fei, Li; Yuille, Alan; Huang, Jonathan; Murphy, Kevin. "Progressive Neural Architecture Search". arXiv:1712.00559, 2 December 2017 (ECCV 2018). https://arxiv.org/abs/1712.00559
11. Pham, Hieu; Guan, Melody Y.; Zoph, Barret; Le, Quoc V.; Dean, Jeff. "Efficient Neural Architecture Search via Parameter Sharing". arXiv:1802.03268, 9 February 2018. https://arxiv.org/abs/1802.03268
12. Liu, Hanxiao; Simonyan, Karen; Yang, Yiming. "DARTS: Differentiable Architecture Search". arXiv:1806.09055, 24 June 2018 (ICLR 2019). https://arxiv.org/abs/1806.09055
13. Tan, Mingxing; Chen, Bo; Pang, Ruoming; Vasudevan, Vijay; Sandler, Mark; Howard, Andrew; Le, Quoc V. "MnasNet: Platform-Aware Neural Architecture Search for Mobile". arXiv:1807.11626, 31 July 2018 (CVPR 2019). https://arxiv.org/abs/1807.11626
14. Li, Liam; Talwalkar, Ameet. "Random Search and Reproducibility for Neural Architecture Search". arXiv:1902.07638, 20 February 2019 (Conference on Uncertainty in Artificial Intelligence, UAI 2019). https://arxiv.org/abs/1902.07638
15. Yang, Antoine; Esperança, Pedro M.; Carlucci, Fabio M. "NAS evaluation is frustratingly hard". arXiv:1912.12522, 28 December 2019 (ICLR 2020). https://arxiv.org/abs/1912.12522

