Convolutional Neural Network

A convolutional neural network (CNN, or ConvNet) is a neural network that uses convolution-like linear operators in at least some layers. CNNs are especially common in computer vision, but the defining idea applies to any input with an ordered grid or neighborhood structure, including one-dimensional signals, images, video, and volumetric data. A convolutional layer applies a bank of learned kernels at many locations. This creates local connectivity and shares the same kernel parameters across locations, two architectural choices that distinguish a conventional CNN from a network made only of fully connected layers.^[1]

In most deep learning libraries, the operation called convolution is technically cross-correlation because the kernel is not reversed before it is applied. The distinction matters when comparing the operator with its mathematical definition, but it does not change the role of a learned kernel: training can learn either orientation.^[1] A CNN normally combines these linear operators with nonlinear activation functions, normalization, residual paths, downsampling or upsampling, and a task-specific output head.

Convolution supplies an architectural prior, not a complete theory of vision. Weight sharing makes a stride-one convolution equivariant to translations on an idealized grid, apart from boundary effects. It does not by itself make the final prediction invariant to position, rotation, scale, or deformation. Striding, padding, pooling, finite image boundaries, preprocessing, and the learned head all affect the resulting behavior.^[9][10] CNN accuracy also depends on the data, objective, optimization procedure, evaluation protocol, and deployment implementation. This article therefore focuses on the operation, the architectural consequences, the major design patterns, and the limits of claims that can be made from the name CNN alone.

Definition and scope

The broadest practical definition is a network that contains at least one convolutional or cross-correlation layer. That definition includes architectures whose later blocks use attention, recurrence, or fully connected computation. In ordinary usage, however, "CNN" usually means that spatially shared local operators provide a substantial part of the feature extractor or prediction network.

An input to a CNN is usually represented as a tensor. For a two-dimensional image model, an implementation commonly stores a batch, a channel axis, and two spatial axes. The exact memory order varies. Some systems use batch-channel-height-width, while others use batch-height-width-channel. The semantic operation can be the same even when layouts differ, but layout conversions may affect performance.

A feature map is the activation tensor produced for one or more channels at a layer. A kernel, also called a filter in this context, is a small array of learned coefficients. A single output channel generally has one kernel slice for every connected input channel, not merely one two-dimensional filter. The network learns these coefficients from data through backpropagation or another optimization procedure.

The word convolution can refer to several related objects:

Scroll sideways for more →

These terms should not be treated as interchangeable. Two trained networks can have the same architecture and different learned parameters. Two implementations can store equivalent kernels in different layouts. A model name can identify a family rather than one exact artifact. Conversely, a system described as convolutional can contain substantial non-convolutional computation.

CNNs are not limited to image classification. A one-dimensional CNN can process an audio waveform, time series, or token sequence. A two-dimensional CNN can operate on images, spectrograms, or other rectangular fields. A three-dimensional CNN can process a video volume or medical scan. Convolution can also be defined on graphs, spheres, groups, and manifolds, but those extensions require a domain-specific notion of neighborhood and symmetry. They should not be assumed to behave exactly like a planar image CNN.

The convolutional operation

Consider an input tensor $X$ with $C_{in}$ channels and a learned kernel tensor $K$ that produces $C_{out}$ channels. For a two-dimensional cross-correlation with one group, one output value can be written as:

$$Y[o,i,j] = b[o] + \sum_{c=0}^{C_{in}-1} \sum_{u=0}^{k_h-1}\sum_{v=0}^{k_w-1} K[o,c,u,v]\, X[c,\,i s_h + u d_h - p_h,\,j s_w + v d_w - p_w].$$

Here $o$ indexes an output channel, $i$ and $j$ index output locations, $b$ is an optional bias, $k_h$ and $k_w$ are kernel sizes, $s$ is stride, $d$ is dilation, and $p$ is padding. Values outside the stored input require an explicit boundary rule. Zero padding is common, but reflection, replication, circular padding, valid-only evaluation, and learned boundary handling are different operators.

For one spatial dimension with input length $n$, kernel size $k$, symmetric padding $p$, stride $s$, and dilation $d$, the usual output length is:

$$n_{out} = \left\lfloor \frac{n + 2p - d(k-1) - 1}{s} \right\rfloor + 1.$$

The formula applies independently to height and width in the standard rectangular two-dimensional case. It also shows why an output shape cannot be inferred from kernel size alone. Padding, stride, and dilation are part of the layer definition. Convolution arithmetic guides derive the corresponding cases for pooling and transposed convolution.^[2]

The effective kernel span along one axis is $d(k-1)+1$. A dilation of one uses adjacent samples. A larger dilation inserts gaps between sampled input positions without increasing the number of kernel coefficients. This can expand the theoretical receptive field, but it can also create sparse sampling patterns or grid artifacts if used without suitable surrounding layers.

With one group and a bias per output channel, the number of trainable scalar parameters is:

$$C_{out}(C_{in}k_hk_w + 1).$$

Without bias, the final $C_{out}$ term is omitted. The parameter count does not depend on image height or width because each kernel is shared over all evaluated locations. In contrast, the arithmetic and activation-memory costs normally grow with the number of output locations.

The approximate multiply-accumulate count for a dense two-dimensional layer is proportional to:

$$H_{out}W_{out}C_{out}C_{in}k_hk_w.$$

This expression is useful for comparing layer shapes, but it is not a latency prediction. Kernel implementations, memory traffic, cache behavior, data type, sparsity, batching, hardware, compiler choices, and operator fusion can change runtime. Counting one multiply-accumulate as one operation or two operations also changes reported FLOP totals, so reports need to state the convention.

Cross-correlation and mathematical convolution

Mathematical discrete convolution reverses the kernel indices. Cross-correlation does not. Many machine-learning libraries implement cross-correlation and call it convolution.^[1] For learned unconstrained kernels, reversing the stored coefficient pattern maps one parameterization to the other. The naming convention therefore does not reduce what the layer can learn, but it matters when importing a fixed signal-processing filter, proving an identity, or comparing implementation details.

A convolution is linear in the input when its kernel is fixed, and linear in the kernel when its input is fixed, but a trained CNN as a whole is usually nonlinear. Activations, normalization behavior, pooling, gates, clipping, and task heads break the simple linear form. Stacking linear convolutional layers without nonlinear operations between them would still represent one larger linear operator, subject to boundary and stride details.

Padding and boundaries

Padding determines which input locations contribute near an edge. "Valid" convolution evaluates only locations where the complete kernel lies within the input. "Same" often means choosing padding so that a stride-one output has the same spatial size as the input, but even-sized kernels and strides greater than one require conventions about asymmetric padding and rounding.

Zero padding introduces values that may not resemble the training distribution at the image boundary. Reflection and replication padding encode different assumptions. Circular padding treats opposite edges as neighbors. Cropping an image, translating content relative to its frame, or changing the padding rule can therefore alter activations even when interior convolution is translation equivariant.

Boundary details propagate through depth. A unit whose theoretical receptive field reaches an edge depends on padded values, and the fraction of affected units grows as receptive fields expand. Exact comparisons should include input size, crop policy, padding convention, and alignment, not only the layer names.

Groups, depthwise filters, and pointwise mixing

Grouped convolution divides channels into groups. Each output channel connects only to the input channels in its group. If $g$ equal groups are used and channel counts are divisible by $g$, the dense parameter term is reduced by a factor of $g$:

$$\frac{C_{out}C_{in}k_hk_w}{g}.$$

A depthwise convolution is the limiting case in which each input channel is filtered separately, often with one or a small multiplier of output channels per input channel. A following $1 \times 1$ pointwise convolution can then mix information across channels. The original MobileNet architecture used this depthwise-separable factorization to construct lightweight models and expose width and resolution tradeoffs.^[14] It reduces arithmetic relative to a comparable dense convolution, but realized speed still depends on whether the target runtime and hardware execute the smaller operations efficiently.

A $1 \times 1$ convolution is not spatially connected beyond one location, but it is a learned linear transformation across channels at every location. It can expand, contract, or mix channel representations while retaining shared spatial application. Bottleneck blocks often use pointwise layers around a more expensive spatial operation.

Transposed convolution and upsampling

A transposed convolution is the linear transpose, with respect to a chosen matrix representation, of a corresponding convolution operator. It is sometimes called deconvolution, but it does not generally invert a preceding convolution or recover information discarded by downsampling. Its output size depends on stride, padding, kernel, dilation, and an output-padding convention.^[2]

Transposed convolution can learn spatial upsampling. Alternative designs resize with a fixed interpolation rule and then apply an ordinary convolution. The methods have different sampling behavior and artifact risks. A checkerboard pattern, for example, can arise when uneven overlap causes output positions to receive different numbers of contributions. The appropriate method depends on alignment, output semantics, and evaluation rather than on the label "learned upsampling."

Locality, sharing, and equivariance

CNNs encode two strong structural assumptions. First, nearby input values are connected before distant values in a conventional small-kernel stack. Second, the same local transformation is useful at more than one location. These assumptions are often appropriate for images and signals, but they are priors imposed by the architecture, not facts learned from scratch.

Local connectivity reduces the number of direct connections relative to a dense layer over an entire image. Parameter sharing reduces the number of distinct coefficients further. Together they allow one learned detector to be evaluated across many positions. This can improve statistical efficiency when the task has approximately repeated local structure.

For an infinite or consistently bounded grid, a stride-one cross-correlation commutes with integer translation. If $T_\delta$ shifts the input and $F$ is such a layer, then:

$$F(T_\delta X) = T_\delta F(X).$$

This property is translation equivariance: shifting the input shifts the output feature map. It is different from translation invariance, where the output would remain unchanged. Pointwise activations preserve equivariance. Several other components can preserve it under specified transformations and boundary assumptions, while ordinary CNNs do not automatically equivary to rotations or reflections. Group-equivariant CNNs extend weight sharing to selected transformation groups.^[9]

Finite images make the equality conditional. A shift can expose new boundary values, remove old ones, or change how padding is applied. Stride greater than one retains only a sampling lattice, so a one-pixel input shift need not produce a correspondingly shifted output. Max pooling, average pooling, and strided convolution can all introduce aliasing when they downsample without appropriate low-pass behavior.^[11]

A task-level prediction can become less sensitive to location through several mechanisms:

• pooling or strided aggregation enlarges the area summarized by later units;
• global aggregation removes an explicit output position;
• data augmentation exposes the model to transformed examples;
• the loss rewards equal labels across those examples;
• a task head combines spatial evidence;
• an equivariant architecture can encode a broader symmetry group.

None of these mechanisms guarantees invariance to every transformation. Even global average pooling is only invariant to permutations or translations that preserve the complete finite feature map being averaged. Cropping, padding, occlusion, interpolation, resampling, and boundary changes can alter the values in that map.

Equivariance can also be undesirable if the label depends on position. A network for anatomical imaging, document layout, remote sensing, or instrument readings may need absolute coordinates. Architectures can add coordinate channels, positional features, asymmetric padding, or non-shared components when the task requires them. The useful prior is determined by the data-generating process and objective, not by a universal preference for invariance.

Receptive fields and learned representations

The receptive field of a unit is the region of an earlier representation that can influence it. In a single convolutional layer, the receptive field follows the kernel footprint. In a stack, receptive fields grow according to kernel spans, dilation, and the accumulated stride. A three-by-three stride-one stack grows the theoretical span by two positions per layer, while downsampling makes each later step correspond to a larger jump in input coordinates.

The theoretical receptive field records possible dependency. It does not say that all positions contribute equally. Luo and colleagues distinguished this from the effective receptive field measured through influence or gradient magnitude. In the architectures they analyzed, the effective region occupied only part of the theoretical field and its influence was concentrated near the center.^[8] The distinction matters when a nominally deep network still underuses long-range context.

Skip connections, dilation, pooling, kernel size, depth, and nonlinear behavior affect effective context. So do training data and the objective. A receptive-field calculation from the architecture can diagnose whether a dependency is possible, but it cannot prove that a trained model uses the dependency or represents a particular concept.

Descriptions of early layers as edge detectors, middle layers as textures or parts, and late layers as objects are useful intuition but not a deterministic law. Learned representations are distributed across channels and locations. Individual units can respond to several patterns, and semantically relevant information can be encoded by combinations of units. What emerges depends on initialization, data, regularization, architecture, and task.

The historical relationship with biological vision should also be stated carefully. Hubel and Wiesel reported localized, orientation-sensitive receptive fields in cat striate cortex in 1959.^[3] Fukushima's 1980 neocognitron used alternating cell types and shared local patterns to build a multilayer recognition system intended to be less affected by positional shifts.^[4] These works supplied influential concepts and analogies. A modern CNN is not a detailed model of the retina or visual cortex, and its backpropagation, data requirements, activations, connectivity, and objectives differ substantially from biological learning.

Representation analysis has also challenged a simple progression from local texture to global shape. In cue-conflict experiments, Geirhos and colleagues found that the ImageNet-trained CNNs they tested relied more strongly on texture than human observers, while altered training data could increase shape bias.^[20] A 2025 study argued that this setup confounded feature reliance and, using controlled feature suppression, found that the tested computer-vision CNNs relied predominantly on local shape features.^[29] The methods answer different experimental questions. Together they give no basis for calling every CNN intrinsically texture-based and show why a layer-depth story alone cannot establish what evidence a model uses.

Components and architectural patterns

A CNN is assembled from operations whose interaction matters more than any one component. A common block applies a convolutional layer, a normalization operation, and an activation function. The order, presence of bias, initialization, residual path, and downsampling position differ among families.

Nonlinear activation

Without nonlinear operations, a stack of convolutional layers would remain linear. Rectified linear units became common because they avoid saturation on their positive side and are inexpensive to evaluate. The AlexNet paper used non-saturating neurons as one ingredient in faster training of its large image classifier.^[6] ReLU is not mandatory: sigmoid, tanh, leaky rectifiers, GELU, SiLU, gated functions, and learned activations appear in different designs.

An activation changes optimization and representation behavior. ReLU can produce permanently inactive units if inputs remain negative, while smooth or gated alternatives have different compute costs. No activation is universally best independent of initialization, normalization, depth, precision, and task.

Normalization

Batch normalization normalizes activations using mini-batch statistics during training and stored or estimated statistics during inference. Ioffe and Szegedy introduced it as a method intended to reduce internal covariate shift and reported that it enabled higher learning rates and faster training in their experiments.^[12] Later work found that stabilizing layer-input distributions did not explain its effectiveness in the tested settings and instead associated it with smoother optimization behavior.^[13]

The mechanism should therefore not be reduced to one settled slogan. Batch normalization also makes behavior depend on training mode, batch composition, statistic estimation, and synchronization choices. Small batches, non-independent samples, domain shift, and distributed execution can make those choices consequential. Group, layer, instance, and response normalization use other axes or statistics and encode different behavior.

Downsampling and pooling

Pooling summarizes values over a neighborhood. Max pooling retains the largest value in each window, while average pooling computes a mean. A strided convolution can learn the downsampling filter instead. Each reduces spatial resolution and increases the input spacing represented by later units.

Downsampling saves activation memory and computation at deeper stages, but it discards spatial samples. If high-frequency content is not appropriately filtered, aliasing can make outputs sensitive to small shifts.^[10][11] Dense prediction tasks often retain higher resolution, use dilated layers, maintain multiple scales, or recover resolution through an upsampling path.

Global average pooling reduces each final feature map to one scalar. It can replace a large fully connected layer in a classification head and permits some variation in input size. It does not erase all spatial information earlier in the network or guarantee invariance under transformations that change feature values.

Residual and skip connections

A residual connection adds a block input to a learned residual branch when their shapes agree, or uses a projection when they do not. The original ResNet study framed residual learning as an optimization reformulation and presented empirical evidence that its substantially deeper residual networks were easier to optimize than comparable plain networks.^[7]

It is common to say that a residual path lets gradients flow, but that shorthand is incomplete. Gradient propagation still depends on nonlinearities, normalization, parameter values, scaling, precision, and the loss. Residual connections change both the forward function class and the backward paths. They reduce some optimization difficulties but do not guarantee stable or successful training at arbitrary depth.

Skip connections also join representations at different resolutions. A segmentation decoder may combine coarse semantic features with finer spatial features. A feature-pyramid design can expose several scales to a detection head. Addition, concatenation, attention, and gated fusion have different parameter and memory implications.

Width, depth, resolution, and stages

Depth counts sequential transformations, but published depth labels do not always count operations the same way. Width refers to channel counts or another hidden dimension. Input resolution controls the number of spatial positions. Increasing any of these can raise capacity and computation, yet the scaling relationships differ.

Many image CNNs use stages. Within a stage, spatial resolution is constant and several blocks operate at one channel width. Between stages, resolution decreases and channel width often increases. This concentrates high-resolution computation near the input and allocates more channels to coarser features.

EfficientNet studied compound scaling of depth, width, and input resolution under a fixed family and resource-oriented search process.^[15] Its reported comparisons are tied to the architectures, training recipes, devices, and metrics in that paper. Compound scaling is a design method, not a proof that one fixed ratio is optimal for every CNN, dataset, or hardware target.

Multi-branch and separable designs

A multi-branch block applies different transformations to the same input and merges their outputs. The Inception family used parallel paths and pointwise projections to combine multiple receptive-field patterns while managing cost.^[27] Residual blocks instead emphasize additive identity paths, and dense connectivity concatenates outputs from earlier blocks. These patterns can be combined.

Depthwise-separable and grouped operations reduce formal arithmetic or parameter count, but they restrict cross-channel connectivity inside a layer. Pointwise mixing, channel shuffling, expansion layers, or later dense operations restore communication. The distribution of cost across operations is important: a model with fewer FLOPs can still be slower if it creates many small kernels, large intermediate tensors, layout conversions, or synchronization points.

Heads and output semantics

The feature extractor, often called a backbone, produces representations. A task head converts them to outputs such as class logits, bounding boxes, masks, keypoints, depth values, or embeddings. The boundary is conventional. A trained detector includes more than a classification backbone, and a segmentation network can make spatial predictions throughout a decoder.

A softmax head converts logits into normalized class scores for a mutually exclusive label set. Independent sigmoid outputs suit some multilabel problems. A regression head may predict continuous values or parameters of a probability distribution. Non-maximum suppression, thresholding, resizing, and calibration can live outside the learned network but still change system behavior. The phrase "CNN output" is ambiguous unless this pipeline is specified.

Training and evaluation

CNN training is not a separate learning theory. The network defines a parameterized function, and an optimizer updates its parameters to reduce an objective over training examples. The same broad procedures used for other neural networks apply: minibatch optimization, regularization, learning-rate schedules, validation-based selection, and checkpointing.

The architecture does influence optimization. Shared kernels accumulate gradient contributions from many locations. Activation scale can vary across channels and stages. Very deep plain networks can exhibit degradation even when vanishing gradients are not the only issue, which motivated residual reformulations.^[7] Normalization, initialization, residual scaling, optimizer settings, batch size, precision, and data order interact, so a recipe successful for one family may not transfer unchanged to another.

Data augmentation is particularly important in vision training. Crops, flips, color changes, geometric transformations, mixing methods, and synthetic corruptions can encode desired robustness and expand the observed input distribution. An augmentation is valid only when it preserves or deliberately transforms the target. Horizontal flipping may preserve an object class but reverse text, handedness, traffic orientation, or anatomy. Aggressive crops can remove the labeled object.

Augmentation does not guarantee invariance. Azulay and Weiss found that the tested CNNs could change predictions under small translations and rescalings even when trained with augmentation, and that their remedies were partial.^[10] Training examples constrain behavior near observed transformations; they do not prove equal outputs for every transformed input.

Transfer learning commonly initializes a CNN from a model trained on a larger or related dataset and adapts it to a target task. The benefit depends on source data, target data, layer choice, objective, and domain distance. Freezing a backbone, updating all layers, or training adapters produces different models. A pretrained checkpoint can also carry unwanted correlations or preprocessing assumptions.

Evaluation units

An evaluation result belongs to an exact model and protocol. For image classification, a report should identify at least:

• dataset version and split;
• input resolution and color convention;
• resize and crop policy;
• normalization and test-time augmentation;
• checkpoint and parameter averaging;
• class mapping;
• top-1, top-k, multilabel, or other metric definition;
• treatment of rejected, corrupted, or missing inputs;
• precision, device, runtime, and batch size for efficiency measurements.

The ImageNet benchmark accelerated comparison among image classifiers, but a number quoted as "ImageNet accuracy" is incomplete without protocol and dataset details. Top-1 accuracy and top-5 accuracy are different metrics. Validation, public test, and newly collected test sets are not interchangeable.

Recht and colleagues built new CIFAR-10 and ImageNet test sets using procedures intended to follow the original data-collection processes. They observed accuracy drops across the evaluated classifiers while the relative ordering was largely preserved.^[18] The result illustrates that benchmark performance is conditional on a sampled distribution and curation process. It does not imply that evaluation is futile; it supports reporting uncertainty and testing the conditions that matter in use.

Accuracy, cost, and reproducibility

Parameter count, FLOPs, throughput, latency, peak memory, energy, binary size, and hardware utilization measure different things. A meaningful efficiency comparison holds as many conditions constant as possible and states those that differ. Batch-one latency on a CPU does not predict high-throughput accelerator performance. Training cost does not equal inference cost. Sparse theoretical operations do not guarantee sparse hardware execution.

Random initialization, data order, nondeterministic kernels, augmentation, mixed precision, and distributed reduction can change a run. Reporting a mean and variation over independent runs can be more informative than one favorable checkpoint. Releasing code and weights helps, but reproduction also needs exact preprocessing, dependency versions, data construction, and evaluation scripts.

Selection can create optimistic estimates when many architectures or checkpoints are compared on the same validation signal. A final test set should be protected from repeated design decisions where practical. For deployment, evaluation should also include representative subgroups, operational thresholds, input failures, drift, and the cost of different error types.

Historical development

CNN history is a sequence of related ideas rather than one moment of invention. Biological receptive-field studies, signal-processing convolution, pattern-recognition systems, backpropagation, benchmark datasets, accelerators, and software frameworks all contributed. The following milestones are selective and describe architectural changes, not a leaderboard.

Scroll sideways for more →

The 1998 document-recognition paper described systems that combined local receptive fields, shared weights, subsampling, and gradient-based learning.^[5] It is a stronger historical basis for modern trainable CNNs than a claim that one later benchmark model invented the architecture. Likewise, AlexNet did not create convolution, backpropagation, GPUs, or rectified units. Its significance lies in the scale and combination of those elements and the reported ImageNet result.^[6]

The original AlexNet paper described five convolutional layers followed by three fully connected layers, about 60 million parameters, and training split across two GPUs.^[6] Those facts are specific to that model. Later CNNs used many more layers, replaced large dense heads with global pooling, introduced residual connections, reduced parameter counts, and changed training procedures. The historical label CNN therefore covers substantially different computation graphs.

Libraries such as PyTorch and TensorFlow expose convolution operators, automatic differentiation, accelerator backends, and pretrained-model interfaces. A framework operator name still does not completely specify behavior. Defaults for padding, data layout, bias, initialization, determinism, interpolation, and serialization can differ by version and API.

History should also avoid a single "human-level" threshold. Human comparison depends on the annotation task, observer expertise, allowed time, class taxonomy, preprocessing, and error metric. A model can exceed a specified human baseline on one benchmark while remaining brittle under distribution shift or unsuitable for a real decision process.

Task adaptations and dimensional variants

CNNs provide reusable spatial or temporal features, but task architecture determines how those features become predictions.

Classification

An image classification CNN maps an image or crop to one or more labels. A conventional design reduces spatial resolution through a backbone, aggregates the last feature map, and applies a classification head. The output covers the supplied crop, not necessarily every object in the original scene. Multilabel classification, hierarchical labels, open-set rejection, and localization require additional decisions beyond the backbone.

Classification pretraining has often supplied backbones for other tasks. This reuse can help when early and intermediate features transfer, but a classifier's downsampling and global aggregation can discard spatial details that dense tasks require. Fine-tuning, feature pyramids, dilated stages, or redesigned stems can address that mismatch.

Detection

Object detection predicts both categories and locations. Two-stage systems first generate candidate regions and then classify or refine them. Faster R-CNN introduced a region-proposal network that shared full-image convolutional features with its detection network in the studied system.^[17] One-stage systems predict dense boxes or related representations directly.

Detection metrics depend on localization overlap, confidence ranking, class definitions, object sizes, and duplicate handling. A backbone classification score does not predict detector quality by itself. Input resizing, proposal settings, anchor design or anchor-free assignment, and postprocessing can materially affect results.

Segmentation and dense prediction

Image segmentation assigns labels or values at pixel or region level. Fully convolutional networks showed how classification networks could be converted into dense predictors and combined coarse semantic information with finer appearance information through skip connections.^[16] Encoder-decoder designs such as U-Net use a contracting path for context and an expanding path for localization.^[28]

Downsampling creates a resolution tradeoff. A large receptive field helps contextual classification, while fine boundaries require spatial detail. Dilated convolution, multiscale features, skip fusion, and learned upsampling address different parts of this tradeoff. Output interpolation and alignment conventions must be included in evaluation because a one-pixel shift can change boundary metrics.

Other two-dimensional tasks include keypoint estimation, optical flow, depth prediction, image restoration, super-resolution, and generative modeling. The same convolutional operator can serve each task, but losses and output semantics differ. A convolutional generator or discriminator is still a CNN even when its purpose is not recognition.

One-dimensional and three-dimensional CNNs

A one-dimensional CNN shares kernels along one ordered axis. It can process waveform samples, time windows, spectra, or sequences. Causality is separate from dimensionality: a causal convolution excludes future positions, while a centered convolution uses both directions. Dilation can expand temporal context without proportionally increasing parameters.

A three-dimensional CNN shares kernels across three axes. For video, those axes can be time, height, and width. For medical or scientific volumes, all three can be spatial. Full three-dimensional kernels are expensive because activation and kernel volumes grow with an extra dimension. Factorized spatial-temporal layers, slice-based models, sparse representations, or mixed two-dimensional and three-dimensional processing offer different compromises.

Axis meaning matters. Translation along time is not necessarily equivalent to translation across image space, and voxel spacing may differ among medical-scan axes. Kernel size expressed in samples does not equal a fixed physical extent unless sampling resolution is controlled.

Beyond regular grids

Graph convolution, spherical convolution, and group convolution generalize some CNN principles, especially local aggregation and shared transformation rules. They require different definitions of translation, neighborhood, orientation, and measure. A graph neural network is not simply a two-dimensional CNN with irregular padding, and a spherical model must account for the geometry and sampling of the sphere.

Calling every local shared operator a convolution can obscure these differences. A precise description should identify the domain, transformation group, discretization, and equivariance property that the operator is intended to preserve.

Efficiency and deployment

CNN deployment begins with a target, not a parameter-count goal. A server accelerator, desktop GPU, mobile neural processor, microcontroller, and browser runtime expose different arithmetic, memory, precision, and operator capabilities. An architecture efficient on one target can perform poorly on another.

Sources of cost

Convolutional cost can be divided into several interacting resources:

• kernel arithmetic, including multiply-accumulate operations;
• reading weights and feature maps;
• writing intermediate activations;
• layout conversion and padding;
• launch or dispatch overhead;
• synchronization across devices;
• temporary workspace used by an optimized algorithm;
• postprocessing and data transfer around the network.

Early high-resolution layers can dominate activation traffic even when they contain relatively few parameters. Pointwise convolutions can dominate arithmetic in depthwise-separable models because channel mixing is dense. A large classifier head can dominate parameter storage in an older architecture while contributing less spatial computation.

Peak memory is not the sum of model parameters alone. Training stores activations, gradients, optimizer state, and sometimes master-precision weights. Inference may reuse buffers, but dynamic shapes, branch lifetimes, and workspace can raise the peak. Quantization reduces storage or arithmetic only when the runtime supports the relevant operators and maintains acceptable accuracy.

Architecture and hardware co-design

MobileNet is an example of architecture designed around a reduced-cost factorization rather than only maximum benchmark accuracy.^[14] EfficientNet illustrates joint scaling under a resource-oriented search and training setup.^[15] ConvNeXt later revisited standard CNN design using training and architectural choices informed by the transformer era.^[25]

These examples do not define a universal efficiency ordering. Depthwise convolution can be memory-bound. A wider dense kernel can use an accelerator more effectively. Larger kernels can reduce sequential depth while increasing weight access. Operator fusion can change the balance. Direct latency and energy measurements on the intended device remain necessary.

Numerical precision and conversion

CNN inference commonly uses floating-point or integer formats with lower precision than training. Calibration or quantization-aware training determines scales and clipping behavior in many integer pipelines. Per-tensor and per-channel scales make different tradeoffs. Accumulation precision can be higher than operand precision.

Conversion can change padding, interpolation, activation approximation, normalization folding, and operation order. A small numerical difference can change a thresholded or ranked output even when aggregate accuracy appears stable. Validation should compare intermediate shapes and final outputs on representative and edge-case inputs, then rerun task metrics after conversion.

Batch normalization is often folded into an adjacent convolution for inference by modifying weights and biases using fixed statistics. That algebra is valid only for the specified inference-mode parameters and numerical conventions. Accidentally leaving training mode active, using stale statistics, or folding with a different epsilon changes behavior.

Dynamic input sizes

Convolutional backbones can often accept several input sizes because their kernels are shared spatially. A fixed dense head, fixed positional assumption, minimum stage size, alignment constraint, or compiled shape can still impose limits. Variable resolution also changes receptive-field coverage, object scale, arithmetic, and memory.

A model trained on one crop size is not automatically validated at every larger size. Resizing may change aspect ratio or pixel density. Batch composition can require padding to a common size. Deployment documentation should distinguish mathematically accepted shapes from evaluated shapes.

Operational measurement

A reproducible latency report states the hardware, software, model artifact, precision, input shape, batch size, warmup, sample count, synchronization, threading, and whether preprocessing and transfer are included. Percentiles can reveal jitter hidden by a mean. Throughput tests should state concurrency and queueing.

Energy and thermal behavior require sustained measurements. A mobile device can throttle after repeated inference. A cloud accelerator can share resources. Model-level FLOPs cannot capture these effects. Production monitoring should track input shape, error rates, latency, resource use, data drift, and version identifiers.

Limits and failure modes

The CNN label establishes architecture, not reliability. Failures can arise from data, objectives, sampling, optimization, numerical execution, and the surrounding system.

Translation sensitivity and aliasing

A stride-one convolution has a clean equivariance property only under defined boundary conditions. Real classifiers use striding, pooling, cropping, and finite frames. Azulay and Weiss showed large prediction changes under small image transformations for the CNNs they studied.^[10] Zhang linked shift sensitivity to common downsampling methods that ignore anti-aliasing and showed that low-pass filtering before subsampling improved stability in the evaluated models.^[11]

Anti-aliasing is not a universal guarantee. The filter choice can remove task-relevant detail, and nonlinear operations can generate new high-frequency components. Integer-pixel translation tests do not cover rotations, rescaling, perspective changes, or continuous subpixel motion. Robustness claims should name the transformations and ranges tested.

Distribution shift

Training and test examples are samples from particular collection and labeling processes. A model can exploit correlations that are stable inside a benchmark but absent in deployment. Background, camera, compression, geography, season, institution, demographic composition, and annotation policy can all shift.

The newly collected test sets studied by Recht and colleagues produced lower accuracy for a wide range of classifiers despite an effort to reproduce the original dataset process.^[18] ImageNet-C and ImageNet-P were introduced to measure common corruptions and perturbations separately from worst-case adversarial attacks.^[19] These benchmarks reveal specified weaknesses, but passing them does not establish robustness to an unknown deployment distribution.

Evaluation should separate in-distribution accuracy, corruption robustness, transformation stability, calibration, subgroup behavior, and out-of-distribution detection. Combining them into one claim such as "robust CNN" hides the threat model and operational cost.

Texture, shape, and shortcut learning

CNNs can use whichever predictive features the training setup rewards. In the cue-conflict experiments of Geirhos and colleagues, selected ImageNet-trained CNNs favored texture cues, while training on stylized images shifted behavior toward shape and improved some tested robustness measures.^[20] Burgert and colleagues later criticized confounds in that test and reported predominantly local-shape reliance under their suppression-based evaluation, with different reliance patterns across application domains.^[29] Neither result should be converted into the claim that convolution mathematically requires one feature preference. Architecture, data, objective, domain, optimization, and the measurement method all influence the conclusion.

Shortcut learning is broader than texture. A medical model might use scanner marks, a wildlife model might use background, and a product classifier might use studio style. High benchmark accuracy can coexist with the wrong causal basis. Counterfactual tests, stratified data, acquisition audits, and external validation can expose some shortcuts, but no single explanation visualization proves their absence.

Adversarial examples

Szegedy and colleagues found that small optimized input perturbations could cause misclassification and could transfer between networks in their experiments.^[21] Later work developed many attacks and defenses, but adversarial robustness remains conditional on the threat model: allowed norm or transformation, perturbation budget, attacker knowledge, query access, and evaluation method.

Natural corruption and adversarial robustness are not synonyms. A model stable to blur may still be vulnerable to a targeted optimization attack. A defense evaluated against one attack can fail against a stronger adaptive attack. A deployment assessment should include the attacker capabilities and the consequences of abstention, rejection, or false alarms.

Uncertainty and confidence

A softmax score is not automatically a calibrated probability of correctness. Neural classifiers can be confidently wrong on shifted, ambiguous, or adversarial inputs. Calibration measured on one validation distribution can degrade under another. Ensembles, temperature scaling, Bayesian approximations, and selective prediction can be useful, but each requires its own assumptions and evaluation.

An out-of-distribution score also cannot identify every novel input. "Unknown" is not one coherent distribution. Monitoring should combine model signals with input validation, data provenance, operational constraints, and human escalation where consequences justify it.

Resolution and information loss

Striding and pooling discard samples. A classifier may tolerate this when the target is global, but small objects, thin structures, text, and precise boundaries can disappear. Increasing input resolution helps only if acquisition contains the information and the architecture preserves it. Interpolation cannot recover detail that was never captured.

The theoretical receptive field can cover the whole input while the effective influence remains concentrated.^[8] Conversely, a network can use a large context that contains spurious background cues. Receptive-field size is therefore neither a direct quality metric nor proof of local or global reasoning.

Dataset and label limits

CNNs learn the distinctions represented by labels and examples. Ambiguous labels, annotator disagreement, missing classes, duplicate images, leakage, and class imbalance constrain the resulting model. A closed-set classifier must assign among its known classes unless the system adds rejection behavior.

Reported performance may average away rare but costly errors. For detection and segmentation, small structures or underrepresented conditions can have much lower quality than the headline metric. For medical, industrial, or scientific use, an internal random split may share acquisition artifacts with training and overstate external performance.

Interpretability limits

Feature visualization, saliency, attribution, activation maximization, and example retrieval answer different questions. They can help form hypotheses about model behavior, but they may be unstable, insensitive to relevant changes, or difficult to validate. An appealing heat map is not evidence that the highlighted region caused the decision in the deployment sense.

CNNs are deterministic functions at inference when the artifact and execution are fixed, except for intentional stochastic operations or nondeterministic implementations. Determinism does not imply interpretability, fairness, causality, or safety. Those properties require separate definitions and tests.

System-level limits

Preprocessing can dominate failure. Color-channel order, value range, orientation metadata, alpha handling, crop alignment, and normalization must match training. A correct network fed the wrong representation is a wrong system. Postprocessing thresholds and coordinate transforms can similarly corrupt correct internal predictions.

Feedback loops, user behavior, sensor aging, changing data sources, and software updates occur outside the CNN. A model card or benchmark table describes an artifact under conditions; it does not substitute for monitoring, access control, incident response, rollback, and domain review.

CNNs and vision transformers

A vision transformer processes image-derived tokens with attention rather than relying primarily on spatial convolution. The original ViT study split images into patches and found that pure transformers could perform strongly when pretrained on large datasets and transferred to image-recognition benchmarks.^[22] This established a practical alternative, not a universal replacement for CNNs.

The architectures encode different default interaction patterns:

Scroll sideways for more →

The table describes conventional forms, not all members of either family. Windowed, sparse, linear, or hierarchical attention changes scaling. Large-kernel and dynamic convolutions change CNN interaction patterns. Hybrid networks use convolutional stems, attention blocks, or convolution inside token-processing stages.

Data efficiency is also not a fixed architecture ranking. The original ViT results emphasized large-scale pretraining.^[22] DeiT subsequently showed competitive transformer training on ImageNet alone using a carefully designed recipe and a distillation method in the reported setting.^[23] It is therefore inaccurate to say that every transformer requires hundreds of millions of images or that every CNN trains well with little data.

Representation studies have found differences beyond benchmark scores. Raghu and colleagues reported more uniform layer representations and earlier aggregation of global information in the ViTs they analyzed, while CNNs showed a more gradual representational progression.^[24] These are empirical patterns for specified models and analyses, not guarantees about every layer in every architecture.

Comparisons are highly protocol-sensitive. Training augmentation, optimizer, regularization, epoch count, input resolution, pretraining data, parameter count, FLOPs, and evaluation crop can produce larger differences than the family label. A fair experiment should match the resource or quality constraint relevant to the application rather than compare conveniently chosen published numbers.

ConvNeXt modernized a ResNet-like CNN with design and training choices associated with contemporary vision transformers and reported competitive results on image classification, detection, and segmentation under its protocols.^[25] Its significance is not that convolution always wins. It demonstrates that older CNN baselines and older training recipes are not sufficient evidence for an architectural conclusion.

CNNs and attention can solve complementary parts of a system. Convolution offers local shared processing and efficient multiscale feature pyramids on many targets. Attention offers data-dependent interactions and direct long-range aggregation. A hybrid can use a convolutional stem for stable local processing, an attention stage for global context, and a convolutional decoder for dense output. Whether that is worthwhile depends on measured accuracy, robustness, memory, latency, data, and implementation support.

The durable conclusion is narrower than a forecast about which family will dominate. "CNN" identifies a useful set of operators and priors, not a fixed 2012 architecture. "Transformer" likewise covers many tokenizations and attention patterns. Model selection should be made at the level of an exact architecture, training procedure, artifact, and deployment target.

References

1. ^Goodfellow, I., Y. Bengio, and A. Courville. "Convolutional Networks." In Deep Learning, Chapter 9, MIT Press, 2016. deeplearningbook.org/...convnets
2. ^Dumoulin, V., and F. Visin. "A Guide to Convolution Arithmetic for Deep Learning." arXiv:1603.07285, revised 2018. arxiv.org/...1603.07285
3. ^Hubel, D. H., and T. N. Wiesel. "Receptive Fields of Single Neurones in the Cat's Striate Cortex." Journal of Physiology 148(3), 1959. pmc.ncbi.nlm.nih.gov/...PMC1363130
4. ^Fukushima, K. "Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position." Biological Cybernetics 36, 1980. doi.org/...BF00344251
5. ^LeCun, Y., L. Bottou, Y. Bengio, and P. Haffner. "Gradient-Based Learning Applied to Document Recognition." Proceedings of the IEEE 86(11), 1998. bottou.org/...lecun-98h
6. ^Krizhevsky, A., I. Sutskever, and G. E. Hinton. "ImageNet Classification with Deep Convolutional Neural Networks." Advances in Neural Information Processing Systems 25, 2012. proceedings.neurips.cc/...8436e924a68c45b-Abstract
7. ^He, K., X. Zhang, S. Ren, and J. Sun. "Deep Residual Learning for Image Recognition." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016. openaccess.thecvf.com/..._Learning_CVPR_2016_paper
8. ^Luo, W., Y. Li, R. Urtasun, and R. Zemel. "Understanding the Effective Receptive Field in Deep Convolutional Neural Networks." Advances in Neural Information Processing Systems 29, 2016. proceedings.neurips.cc/...1288b3eb986afaa-Abstract
9. ^Cohen, T., and M. Welling. "Group Equivariant Convolutional Networks." Proceedings of the 33rd International Conference on Machine Learning, 2016. proceedings.mlr.press/...cohenc16
10. ^Azulay, A., and Y. Weiss. "Why Do Deep Convolutional Networks Generalize So Poorly to Small Image Transformations?" Journal of Machine Learning Research 20, 2019. jmlr.csail.mit.edu/...19-519
11. ^Zhang, R. "Making Convolutional Networks Shift-Invariant Again." Proceedings of the 36th International Conference on Machine Learning, 2019. proceedings.mlr.press/...zhang19a
12. ^Ioffe, S., and C. Szegedy. "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift." Proceedings of the 32nd International Conference on Machine Learning, 2015. proceedings.mlr.press/...ioffe15
13. ^Santurkar, S., D. Tsipras, A. Ilyas, and A. Madry. "How Does Batch Normalization Help Optimization?" Advances in Neural Information Processing Systems 31, 2018. proceedings.neurips.cc/...560467e0a99e1cf-Abstract
14. ^Howard, A. G. et al. "MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications." arXiv:1704.04861, 2017. arxiv.org/...1704.04861
15. ^Tan, M., and Q. Le. "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks." Proceedings of the 36th International Conference on Machine Learning, 2019. proceedings.mlr.press/...tan19a
16. ^Long, J., E. Shelhamer, and T. Darrell. "Fully Convolutional Networks for Semantic Segmentation." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015. openaccess.thecvf.com/..._Networks_2015_CVPR_paper
17. ^Ren, S., K. He, R. Girshick, and J. Sun. "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks." Advances in Neural Information Processing Systems 28, 2015. proceedings.neurips.cc/...ba028a21ed38046-Abstract
18. ^Recht, B., R. Roelofs, L. Schmidt, and V. Shankar. "Do ImageNet Classifiers Generalize to ImageNet?" Proceedings of the 36th International Conference on Machine Learning, 2019. proceedings.mlr.press/...recht19a
19. ^Hendrycks, D., and T. Dietterich. "Benchmarking Neural Network Robustness to Common Corruptions and Perturbations." International Conference on Learning Representations, 2019. iclr.cc/...731
20. ^Geirhos, R. et al. "ImageNet-trained CNNs Are Biased Towards Texture; Increasing Shape Bias Improves Accuracy and Robustness." International Conference on Learning Representations, 2019. openreview.net/forum
21. ^Szegedy, C. et al. "Intriguing Properties of Neural Networks." International Conference on Learning Representations, 2014. arxiv.org/...1312.6199
22. ^Dosovitskiy, A. et al. "An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale." International Conference on Learning Representations, 2021. openreview.net/forum
23. ^Touvron, H. et al. "Training Data-Efficient Image Transformers and Distillation Through Attention." Proceedings of the 38th International Conference on Machine Learning, 2021. proceedings.mlr.press/...touvron21a
24. ^Raghu, M., T. Unterthiner, S. Kornblith, C. Zhang, and A. Dosovitskiy. "Do Vision Transformers See Like Convolutional Neural Networks?" Advances in Neural Information Processing Systems 34, 2021. proceedings.neurips.cc/...302ba2b8b7f51e0-Abstract
25. ^Liu, Z. et al. "A ConvNet for the 2020s." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022. openaccess.thecvf.com/...the_2020s_CVPR_2022_paper
26. ^Simonyan, K., and A. Zisserman. "Very Deep Convolutional Networks for Large-Scale Image Recognition." International Conference on Learning Representations, 2015. arxiv.org/...1409.1556
27. ^Szegedy, C. et al. "Going Deeper with Convolutions." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015. openaccess.thecvf.com/...eper_With_2015_CVPR_paper
28. ^Ronneberger, O., P. Fischer, and T. Brox. "U-Net: Convolutional Networks for Biomedical Image Segmentation." Medical Image Computing and Computer-Assisted Intervention, 2015. arxiv.org/...1505.04597
29. ^Burgert, T., O. Stoll, P. Rota, and B. Demir. "ImageNet-trained CNNs Are Not Biased Towards Texture: Revisiting Feature Reliance Through Controlled Suppression." Advances in Neural Information Processing Systems 38, 2025. proceedings.neurips.cc/...7b0a-Abstract-Conference