ResNet

ResNet, short for residual network, is a family of deep convolutional neural networks introduced by Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun at CVPR 2016. Its defining operation is the residual block: a learned transformation is added to a shortcut carrying the block input. When the two tensors have the same shape, the shortcut can be the identity and adds no learned parameters. The resulting network learns changes to a representation rather than requiring every stack of layers to reconstruct the complete desired mapping.^[1]

The original paper used this design to train image classifiers with as many as 152 weighted layers. A six-model ensemble achieved 3.57% top-5 error on the ImageNet test set and won the ILSVRC 2015 classification task. ResNet-based systems also won the ImageNet detection and localization tracks and the COCO 2015 detection and segmentation tracks reported by the authors. The paper received the CVPR 2016 Best Paper Award and, ten years later, the 2026 Longuet-Higgins Prize.^[1][2]

ResNet did not invent all forms of skip connection, and it was not the first published system to exceed the often-cited 5.1% human top-5 reference on ImageNet. Its historical importance lies instead in the unusually simple additive identity shortcut, the evidence that it addressed a concrete optimization failure in deep plain networks, and the large family of models and systems built from that template. A 2025 Nature analysis ranked the ResNet paper first among twenty-first-century papers using the final order derived from median ranks across five citation databases; its supplementary data record database ranks of 1, 1, 2, 3, and 2 for the paper.^[3]

Historical context

Before ResNet, increasing the depth of an image classifier had produced major gains. AlexNet had eight learned layers, while VGG networks used 16 or 19 learned layers and GoogLeNet used a 22-layer design. Better initialization, rectified activations, and batch normalization made networks with tens of layers trainable, but depth still created an optimization problem that was separate from the classic vanishing gradient problem.^[1]

The degradation problem

The ResNet paper compared plain networks that differed mainly in depth. On CIFAR-10, its 56-layer plain network had higher training error than its 20-layer plain network. On ImageNet, a 34-layer plain network also had higher training and validation error than an 18-layer counterpart. Because the deeper models fit the training data worse, the failure could not be explained as ordinary overfitting. The authors called it the degradation problem.^[1]

There is a simple representational argument for why this result was surprising. A deeper network should be able to reproduce a shallower one by copying the shallower layers and making the added layers implement identity mappings. The existence of that constructed solution does not mean that gradient-based training will find it. The experiments suggested that the parameterization of a conventional nonlinear stack made such solutions difficult to reach. Residual learning changed the parameterization so that a zero residual branch corresponds to an identity mapping whenever the shortcut itself is an identity.^[1]

Earlier and contemporary shortcut networks

Connections that bypassed layers predated ResNet. Highway networks, published in 2015, combined a transformed signal with a carried signal through learned gates inspired by recurrent networks. Their authors trained networks with hundreds of layers using ordinary gradient descent. ResNet removed the learned transform and carry gates from its principal shortcut and instead used an unmodulated addition. The original ResNet paper explicitly discussed Highway networks and other shortcut-based systems, so it is inaccurate to describe residual learning as the first use of deep skip connections.^[4]

The same four ResNet authors had also developed the variance-preserving initialization commonly called He initialization. Their earlier rectifier paper reported a 4.94% top-5 ImageNet test error for a multi-model PReLU system and described that result as the first they knew to exceed the reported 5.1% human reference. That paper was reported in February 2015, before the ResNet result. ResNet's 3.57% ensemble result was lower, but it was not the first published crossing of that particular reference value.^[5]

Residual formulation

For an input tensor x, a residual block can be written in simplified form as:

z = F(x; W) + s(x)
y = f(z)

F is the residual branch, W denotes its learned weights, s is the shortcut, and f is the operation after addition. In the original post-activation design, f is a ReLU. If input and output shapes match, s(x) = x. The learned branch then models a residual with respect to the input. If the desired mapping is H(x), the branch is parameterized to learn F(x) = H(x) - x rather than H(x) directly.^[1]

This formulation does not prove that residual networks will optimize successfully, nor does it make every gradient exactly equal to one. The derivative through a block contains a contribution from the shortcut in addition to the derivative through F. A truly unobstructed additive path across many blocks requires both identity shortcuts and an identity operation after each addition. Original ResNet v1 applies a ReLU after addition, so its long path is not mathematically identical to the later full pre-activation construction.^[6]

Shape changes and shortcut options

An identity addition requires the two branches to have the same spatial dimensions and channel count. The original paper evaluated three ways to handle this constraint:

• Option A used identity shortcuts and zero-padded extra channels when dimensions increased. Downsampling occurred by striding.
• Option B used a learned 1x1 projection only when the dimensions changed and used identity shortcuts elsewhere.
• Option C used learned projections for all shortcuts.

In the authors' 34-layer ImageNet comparison, option B was slightly more accurate than option A, while option C was only marginally more accurate than B despite adding more projection parameters. The 50-, 101-, and 152-layer experiments used option B when dimensions increased. This history is more precise than calling option B the only standard ResNet shortcut.^[1]

Basic and bottleneck blocks

ResNet-18 and ResNet-34 use a basic block with two 3x3 convolutions on the residual branch. ResNet-50, ResNet-101, and ResNet-152 use a bottleneck block with a 1x1 convolution, a 3x3 convolution, and another 1x1 convolution. In the original bottleneck, the first 1x1 layer reduces the channel width, the 3x3 layer processes the narrower representation, and the last 1x1 layer expands it. This arrangement allows substantially greater depth without making every 3x3 convolution operate at the full output width.^[1]

The architecture name counts weighted layers: the initial convolution, convolutions inside every block, and the final fully connected classifier. Batch-normalization layers, activation functions, pooling operations, and shortcut additions are not included in the number. A ResNet-50 therefore has 49 convolutional layers plus one fully connected layer, not 50 residual blocks.^[1]

Original architectures and training

ImageNet model family

All five original ImageNet models begin with a 7x7 convolution at stride 2 and a 3x3 max-pooling layer at stride 2. Four residual stages then operate at nominal spatial sizes of 56x56, 28x28, 14x14, and 7x7 for a 224x224 crop. The number of blocks in each stage and the paper's reported multiply-add counts were:^[1]

Scroll sideways for more →

The paper compared its 152-layer model's 11.3 billion FLOPs with 15.3 billion for VGG-16 and 19.6 billion for VGG-19 under its counting convention. FLOP totals depend on conventions such as whether one multiply-add is counted as one or two operations, so values from different libraries should not be mixed without checking the definition.^[1]

After the final residual stage, the original classifiers use global average pooling and a 1,000-way fully connected layer. The stage structure made it possible to replace the classifier with task-specific heads while reusing the convolutional representation, a property that became important for detection and segmentation.^[1]

Original training procedure

The ImageNet models were trained from scratch using stochastic gradient descent with mini-batches of 256, momentum of 0.9, weight decay of 0.0001, and an initial learning rate of 0.1 divided by ten when the error plateaued. The paper says training continued for up to 60 x 10^4, or 600,000, iterations. It did not describe this run as a fixed 90-epoch recipe.^[1]

Training augmentation resized each image so that its shorter side was randomly sampled between 256 and 480 pixels, then sampled a 224x224 crop from the image or its horizontal reflection. The authors subtracted a per-pixel mean and used color augmentation. They applied batch normalization after each convolution and before activation, used He initialization, and did not use dropout in the ImageNet ResNets.^[1][5]

Evaluation protocol materially affects reported accuracy. The paper used 10-crop testing for controlled architecture comparisons and a fully convolutional, multi-scale procedure for its stronger single-model results. Modern libraries often report one center crop from a separately specified resize. Those results answer related but different questions and should be labeled rather than placed in a single table as interchangeable measurements.^[1]

Evaluation results

ImageNet classification

The original paper's controlled 10-crop validation results are shown below. ResNet-18 used option A; the deeper entries shown here used the shortcut setting reported for their rows. A dash means the paper did not publish a top-5 value for that row, not that the model lacks such a result.^[1]

Scroll sideways for more →

Under the paper's stronger single-model validation procedure, ResNet-50, ResNet-101, and ResNet-152 obtained top-1 errors of 20.74%, 19.87%, and 19.38%, and top-5 errors of 5.25%, 4.60%, and 4.49%, respectively. The competition entry combined six models of different depths, including two 152-layer models at submission time, and obtained 3.57% top-5 test error.^[1]

The experiment established two distinct points. First, adding depth to a plain 18- or 34-layer network could worsen optimization. Second, replacing those stacks with residual blocks reversed the trend in the tested settings: the 34-layer residual network trained better and was more accurate than the 18-layer residual network. The results do not imply that accuracy improves monotonically with arbitrary depth. On CIFAR-10, the paper's 1,202-layer network achieved training error below 0.1% but test error of 7.93%, worse than its 110-layer network. The authors attributed the gap to overfitting and explicitly described very deep models as an open problem.^[1]

Modern implementation numbers

TorchVision documents a widely used ResNet-50 that differs slightly from the original bottleneck: it places the downsampling stride in the 3x3 convolution rather than the first 1x1 convolution. The documentation calls this ResNet v1.5. Its current model entry lists 25,557,032 parameters and 4.09 GFLOPs under TorchVision's convention. The IMAGENET1K_V1 weights obtain 76.13% top-1 and 92.862% top-5 accuracy, while the default IMAGENET1K_V2 weights, trained with a newer recipe, obtain 80.858% and 95.434%.^[7]

The V2 in IMAGENET1K_V2 names a weights and training-recipe revision; it does not mean that the network uses the pre-activation architecture often called ResNet v2. Similarly, TorchVision's v1.5 stride placement is an implementation variant, not the full pre-activation redesign. Reporting a modern checkpoint therefore requires naming the library, architecture variant, weights, preprocessing, and evaluation protocol.^[7]

Detection and segmentation

The original paper replaced a VGG-16 backbone with ResNet-101 in a baseline Faster R-CNN detector while holding the detection method otherwise fixed. On the COCO validation set, VGG-16 obtained 21.2% under the standard AP metric averaged from intersection-over-union thresholds 0.50 through 0.95, while ResNet-101 obtained 27.2%. The absolute increase was 6.0 percentage points, or about 28% relative. At the older AP@0.50 metric, the corresponding values were 41.5% and 48.4%.^[1][8]

The supplement's higher 37.4% standard-AP result belongs to an ensemble competition system after box refinement, contextual features, multi-scale testing, and additional training data. It is not the baseline backbone comparison that produced the cited 28% relative gain.^[8]

The authors' ResNet-based entries won the ILSVRC 2015 detection and localization tasks and the COCO 2015 detection and segmentation tasks. These results supported a narrower and well-evidenced conclusion: features learned by a residual classifier transferred effectively to the detection systems tested by the team. They do not by themselves establish that ResNet was best for every subsequent vision task or deployment constraint.^[1][8]

Follow-up refinements and explanations

Identity mappings and pre-activation

The same authors' 2016 follow-up, "Identity Mappings in Deep Residual Networks," separated two parts of a residual unit: the shortcut function and the operation after addition. It derived a direct additive propagation expression when both are identities. The proposed full pre-activation unit moves batch normalization and ReLU before each weight layer, leaving the addition followed by no activation. This differs from the original post-activation block, where a ReLU follows the addition.^[6]

In the follow-up's CIFAR-10 experiments, a 1,001-layer full pre-activation ResNet reached 4.92% test error under its standard batch setting; the abstract's 4.62% result used a smaller mini-batch of 64. On ImageNet, the paper compared 152- and 200-layer models and reported that a pre-activation ResNet-200 reduced overfitting relative to its original-unit counterpart. Projection shortcuts at stage transitions still break a strictly identity-only chain, so the mathematical derivation is best read as an explanation of the dominant same-shape path rather than a literal description of every unit.^[6]

Pre-activation became an important ResNet variant, but it did not replace post-activation ResNet in every framework. A model labeled only "ResNet-50" may refer to original v1, a stride-modified v1.5, a full pre-activation v2, or a further library-specific recipe. Architecture and checkpoint metadata are necessary to disambiguate them.^[6][7]

Paths through a residual network

Veit, Wilber, and Belongie gave an "unraveled" interpretation in which a network of residual modules represents many computational paths of different lengths. Algebraically, each module offers a residual branch and a skip branch, producing an exponential number of path combinations. Their lesion experiments found ensemble-like behavior in the tested models: removing individual modules generally caused gradual rather than catastrophic degradation.^[9]

The paper also measured gradient contributions by path length in a 54-module network. In that experiment, nearly all aggregate gradient came through paths containing between 5 and 17 residual modules, even though those paths represented only 0.45% of all possible paths. A network trained using the corresponding effective-path distribution reached 5.96% error compared with 6.10% for the full-path training setup, a difference the authors said was not statistically significant. These are results for a particular architecture and experiment, not a theorem that all ResNets behave as literal ensembles or that long paths never matter.^[9]

Loss-landscape evidence

Li and colleagues introduced filter normalization for comparing two-dimensional slices through neural-network loss functions. On CIFAR-10, their visualizations compared ResNet-20, ResNet-56, and ResNet-110 with versions formed by removing the shortcuts. In the displayed slices, the deeper no-shortcut networks developed irregular, poorly conditioned surfaces, while the shortcut versions retained smoother basins. The authors interpreted this as empirical evidence that residual connections prevent the observed transition to chaotic landscapes as depth increases.^[10]

The paper explicitly cautions that a two-dimensional slice is a drastic reduction of a high-dimensional function. Its plots are useful diagnostic evidence but do not prove global convexity, eliminate dependence on initialization and optimizer choice, or provide a complete theory of residual learning. Both the path analysis and the landscape study help explain particular observations; neither should be promoted to a universal causal law.^[9][10]

ResNet as a vision backbone

ResNet's staged feature hierarchy and pretrained checkpoints made it a common backbone for transfer learning, object detection, and segmentation. A backbone supplies feature maps; the rest of the system adds task-specific structures such as proposal networks, pyramid fusion, detection heads, or mask prediction. Results for a complete detector should not be attributed to the backbone alone.^[1]

Feature Pyramid Networks used the bottom-up hierarchy of a ResNet, a top-down pathway, and lateral connections to construct semantically strong feature maps at multiple resolutions. The FPN paper evaluated ResNet-50 and ResNet-101 backbones and showed that the pyramid could support region proposal and Faster R-CNN systems without separately running the backbone at every image scale.^[11]

Mask R-CNN extended Faster R-CNN with a parallel mask-prediction branch and introduced RoIAlign to avoid quantization in feature extraction. Its experiments used ResNet or ResNet-FPN backbones. The resulting performance was a property of the whole Mask R-CNN system, including the detector, alignment operation, mask head, schedule, and backbone.^[12]

This modular role explains why "ResNet backbone" can describe systems with different outputs and training objectives. Image classification uses the global pooled representation and classifier. Detection retains spatial maps and adds region or dense prediction machinery. Segmentation produces pixel- or instance-level outputs. The shared ResNet weights can initialize these systems, but fine-tuning procedure, input resolution, normalization behavior, and head design can materially change results.^[11][12]

Variants and later convolutional designs

Many later architectures retained an additive residual path while changing the transformation inside a block, the connection pattern, or the training recipe.

Scroll sideways for more →

ResNeXt implemented its aggregated transformations efficiently with grouped convolutions. The paper's ResNeXt-50 32x4d example had 25.0 million parameters and 4.2 billion FLOPs, close to the compared ResNet-50's 25.5 million and 4.1 billion under that paper's convention. The key controlled variable was cardinality, the number of parallel transformations, rather than simply greater depth or width.^[13]

Wide Residual Networks argued that very thin, extremely deep networks were not always the best use of computation. In its CIFAR experiments, WRN-40-4 and the compared 1,001-layer pre-activation ResNet had roughly similar parameter counts, while WRN-40-4 trained about eight times faster and obtained lower test error under the paper's reported setups. That experiment supports a depth-width tradeoff; it does not establish one widening factor as optimal for every dataset or hardware platform.^[14]

DenseNet used a different connectivity rule: each layer received the concatenated feature maps of all preceding layers within a dense block. Unlike a ResNet addition, concatenation preserves each contributing feature map as a separate channel. The DenseNet paper reported strong parameter efficiency in its controlled ImageNet comparison, but the architecture's growing concatenated state creates different memory and implementation costs from an additive residual stream.^[15]

Squeeze-and-Excitation networks learned channel-wise gates from globally pooled features and could insert those blocks into ResNet and ResNeXt models. In the paper's basic COCO detector comparison, SE-ResNet-50 improved standard AP from 25.1 to 26.4 and SE-ResNet-101 improved it from 27.2 to 27.9. Its ILSVRC 2017 competition system was an ensemble based on SENets and reported 2.251% top-5 test error, so that number should not be presented as a plain single ResNet result.^[16]

ResNet-RS showed how much performance could come from modern training and scaling rather than a completely new backbone. Its additive study began from a ResNet-200 at 79.0% ImageNet top-1 accuracy. Training and regularization changes increased that to 82.2%; squeeze-and-excitation raised it to 82.9%; and the ResNet-D stem and downsampling changes raised it to 83.4%. The 83.4% row is therefore a ResNet-200 result, not a ResNet-50 result. The paper also reported 86.2% for a larger semi-supervised ResNet-RS trained with 130 million pseudo-labeled images, which is a different data regime.^[17]

ConvNeXt used ResNet-50 and ResNet-200 as starting points for a controlled modernization sequence. A Transformer-style training recipe alone raised the paper's ResNet-50 baseline from 76.1% to 78.8% ImageNet top-1 accuracy. Later changes included a patch-like stem, a revised stage ratio, depthwise convolution, an inverted bottleneck, a 7x7 kernel, fewer activation and normalization layers, LayerNorm, and separate downsampling layers. The final ConvNeXt blocks retained residual additions. The paper's 87.8% result belongs to ConvNeXt-XL after ImageNet-22K pretraining and 384x384 fine-tuning, not to a standard ResNet-50.^[18]

These variants also show why architecture comparisons require controls. Data scale, augmentation, optimizer, training duration, resolution, pretraining, parameter count, FLOP convention, and accelerator implementation can all move the reported frontier. Calling every improvement a consequence of the block topology alone overstates the evidence.^[17][18]

Residual connections beyond convolutional networks

Additive residual connections became a general neural-network design pattern. The original Transformer placed a residual connection around each attention and feed-forward sublayer, followed by layer normalization in its post-normalization formulation:

LayerNorm(x + Sublayer(x))

That paper cites ResNet when introducing the residual connection. The similarity is structural: both add a sublayer transformation to a carried representation. The sublayers, normalization order, tensor semantics, and training behavior are otherwise different. Later pre-normalization Transformers also changed where normalization occurs, much as residual CNNs have multiple activation and normalization orders.^[19]

It is reasonable to describe ResNet as a major route through which additive residual blocks became standard, but claims that every language model, diffusion model, or modern neural network directly inherits an unchanged 2015 block are too broad. Some systems use gated residuals, scaled residuals, learned projections, concatenative skips, cross-stage connections, or other pathways. Historical influence should be supported model by model rather than inferred from the word "skip."

Limitations and implementation distinctions

Residual connections address an optimization problem; they do not remove the ordinary limits of supervised learning. A ResNet can overfit, inherit biases from its data, fail under distribution shift, and be miscalibrated. Greater depth can increase memory, training time, inference latency, and implementation complexity even when it remains mathematically trainable. The original 1,202-layer CIFAR experiment is direct evidence that near-zero training error and successful optimization do not guarantee improved test accuracy.^[1]

Several names that look interchangeable refer to meaningfully different objects:

• ResNet v1 usually means the original post-activation unit.
• ResNet v1.5 commonly means the bottleneck stride is moved from the first 1x1 convolution to the 3x3 convolution.
• ResNet v2 commonly means a full pre-activation unit with normalization and activation before the weight layers.
• A library's V2 weights can mean a newer training recipe for the same architecture rather than ResNet v2.
• ResNet-D changes the stem and downsampling path; SE-ResNet adds channel recalibration; ResNet-RS combines multiple recipe and architecture changes.

Checkpoint accuracy is therefore not a property of the name alone. Reproducible reporting should include the exact variant, dataset split, crop and resize policy, weights identifier, precision, software version, and metric protocol. Parameter and FLOP counts should state the input size and counting convention.^[6][7][17]

ResNet-50 remained an MLPerf Training image-classification reference through the benchmark's v4.1 entry, with a 75.90% target on ImageNet. On the MLCommons page available at the research cutoff, later suite versions feature newer language, recommendation, and image-generation workloads, while the ResNet-50 row is still listed as the latest available version for that particular image-classification benchmark. This makes it a durable systems reference, but not evidence that it is the current accuracy frontier in vision.^[20]

Recognition and historical impact

The Computer Vision Foundation lists "Deep Residual Learning for Image Recognition" as both the CVPR 2016 Best Paper and a 2026 Longuet-Higgins Prize recipient. The latter prize recognizes CVPR papers from ten years earlier that made a significant impact. Those two awards bracket the paper's immediate reception and its later standing in the computer-vision community.^[2]

The 2025 Nature ranking used five databases rather than a single live citation counter. For ResNet, the supplementary spreadsheet lists ranks of 1 in Dimensions, 1 in Scopus, 2 in OpenAlex, 3 in Web of Science, and 2 in Google Scholar, with a median rank of 2 and final rank 1 among the twenty-first-century papers considered. The methodology matters because citation databases merge versions and index conference papers differently; the same spreadsheet documents such discrepancies for several other machine-learning papers.^[3]

ResNet's lasting contribution is consequently best stated without exclusivity claims. It supplied a compact residual formulation, controlled evidence against degradation in deep plain networks, successful ImageNet and COCO systems, and an architecture that researchers could adapt and compare. Its descendants changed depth, width, cardinality, connectivity, channel weighting, training, and scaling, while residual addition also appeared in non-convolutional architectures. The original configuration is no longer the universal state of the art, but it remains a central reference point for understanding deep-network optimization and vision backbones.^{[1][6][17][18]}

See also

• AlexNet
• VGG
• ConvNeXt
• DenseNet
• ImageNet
• Computer vision
• Residual connection

References

1. ^Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Deep Residual Learning for Image Recognition." CVPR 2016. openaccess.thecvf.com/...rning_CVPR_2016_paper.pdf
2. ^Computer Vision Foundation. "Computer Vision Awards." thecvf.com
3. ^Helen Pearson, Heidi Ledford, Matthew Hutson, and Richard Van Noorden. "Exclusive: the most-cited papers of the twenty-first century." Nature, 2025. nature.com/...d41586-025-01125-9
4. ^Rupesh Kumar Srivastava, Klaus Greff, and Jurgen Schmidhuber. "Training Very Deep Networks." 2015. arxiv.org/...1507.06228
5. ^Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification." ICCV 2015. openaccess.thecvf.com/..._into_ICCV_2015_paper.pdf
6. ^Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Identity Mappings in Deep Residual Networks." ECCV 2016. arxiv.org/...1603.05027
7. ^TorchVision. "resnet50." docs.pytorch.org/...torchvision.models.resnet50
8. ^Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Deep Residual Learning for Image Recognition: Supplementary Materials." CVPR 2016. openaccess.thecvf.com/...016_CVPR_supplemental.pdf
9. ^Andreas Veit, Michael Wilber, and Serge Belongie. "Residual Networks Behave Like Ensembles of Relatively Shallow Networks." NeurIPS 2016. proceedings.neurips.cc/...0a1a41c200364c-Paper.pdf
10. ^Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. "Visualizing the Loss Landscape of Neural Nets." NeurIPS 2018. proceedings.neurips.cc/...067c67f663b915-Paper.pdf
11. ^Tsung-Yi Lin, Piotr Dollar, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. "Feature Pyramid Networks for Object Detection." CVPR 2017. openaccess.thecvf.com/...works_CVPR_2017_paper.pdf
12. ^Kaiming He, Georgia Gkioxari, Piotr Dollar, and Ross Girshick. "Mask R-CNN." ICCV 2017. openaccess.thecvf.com/...R-CNN_ICCV_2017_paper.pdf
13. ^Saining Xie, Ross Girshick, Piotr Dollar, Zhuowen Tu, and Kaiming He. "Aggregated Residual Transformations for Deep Neural Networks." CVPR 2017. openaccess.thecvf.com/...tions_CVPR_2017_paper.pdf
14. ^Sergey Zagoruyko and Nikos Komodakis. "Wide Residual Networks." BMVC 2016. bmva-archive.org.uk/...paper087.pdf
15. ^Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. "Densely Connected Convolutional Networks." CVPR 2017. openaccess.thecvf.com/...ional_CVPR_2017_paper.pdf
16. ^Jie Hu, Li Shen, and Gang Sun. "Squeeze-and-Excitation Networks." CVPR 2018. openaccess.thecvf.com/...works_CVPR_2018_paper.pdf
17. ^Irwan Bello, William Fedus, Xianzhi Du, Ekin D. Cubuk, Aravind Srinivas, Tsung-Yi Lin, Jonathon Shlens, and Barret Zoph. "Revisiting ResNets: Improved Training and Scaling Strategies." 2021. arxiv.org/...2103.07579
18. ^Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. "A ConvNet for the 2020s." CVPR 2022. openaccess.thecvf.com/...2020s_CVPR_2022_paper.pdf
19. ^Ashish Vaswani and others. "Attention Is All You Need." NeurIPS 2017. proceedings.neurips.cc/...d053c1c4a845aa-Paper.pdf
20. ^MLCommons. "MLPerf Training." mlcommons.org/...training