Quantization

Quantization is the representation of values from a large or continuous set by values from a smaller, usually finite set. A quantizer assigns an input to an index or reconstruction level, so different inputs can produce the same output. The resulting loss of detail is intentional: it permits a signal, vector, or numerical array to be stored, transmitted, or processed at a lower rate or in a more convenient representation. Quantization is therefore a foundational operation in digital signal representation and lossy source coding, not only a technique for machine learning.^[1]

In a neural network, the term usually means replacing some high-precision weights, activations, gradients, or cached states with lower-precision values. That can reduce storage and memory traffic, and it can accelerate computation when the deployment hardware and software have efficient kernels for the chosen representation. Those benefits are not automatic. Accuracy depends on what is quantized, the mapping and granularity used, the data seen during calibration or training, and the model and task. Latency depends additionally on the workload, batch size, operator coverage, data movement, conversions, and target processor.^[6][7][8]

Quantization is generally lossy. A dequantized value is an approximation to the original, not a recovery of every discarded bit. The relevant question is therefore not whether the output changed, but whether the chosen rate, distortion measure, task metric, and deployment cost produce an acceptable trade-off. A claim such as "4-bit" or "INT8" describes only part of a system. It does not by itself specify the quantizer, scale granularity, clipping rule, accumulator, unquantized components, file size, accuracy, or speed.

Mathematical model

Encoder, cells, and reconstruction values

A quantizer can be described as an encoder followed by a decoder. Let an input $x$ belong to a source alphabet $\mathcal{X}$. The encoder $E$ maps $x$ to an index $i$, and the decoder $D$ maps that index to a reconstruction value $y_i$:

$$Q(x) = D(E(x)) = y_i.$$

All inputs assigned to the same index form a quantization cell. For a scalar quantizer, the input and reconstruction values are scalar numbers. For a vector quantizer, they are vectors. If there are $N$ possible indices and every index receives a fixed-length binary codeword, an index requires at least $\lceil \log_2 N \rceil$ bits. A variable-rate system may give more probable indices shorter codewords and less probable indices longer ones, so its average rate is not determined solely by the number of reconstruction values.^[1]

The reconstruction set is often called a codebook, and its members are also called levels, codewords, reproduction points, or centroids. The words "encode" and "decode" in this setting refer to the quantizer stages; they do not imply that a lossy mapping becomes reversible. Once two source values have been assigned the same index, the decoder cannot distinguish them without additional information.

Distortion

A distortion measure $d(x,y)$ states the cost of reconstructing $x$ as $y$. A common scalar choice is squared error,

$$d(x, y) = (x - y)^2,$$

with mean-squared error $D = \mathbb{E}[(X-Q(X))^2]$. Absolute error, weighted quadratic error, perceptual measures, task loss, and other criteria can be more appropriate in particular applications. A quantizer that is optimal for one source distribution and distortion measure need not be optimal for another.^[1]

Quantization design is commonly posed in one of two equivalent directions: minimize expected distortion subject to a rate constraint, or minimize rate subject to a distortion constraint. Operational distortion-rate functions describe the best trade-off achievable within a specified class of quantizers. Shannon rate-distortion theory supplies a broader information-theoretic lower bound, whereas practical scalar and structured vector quantizers impose restrictions that make implementation possible.^[1]

Uniform scalar quantization

A uniform scalar quantizer uses equally spaced reconstruction levels over its granular region. With step size $\Delta$, a simple rounding quantizer is

$$Q(x) = \Delta \operatorname{round}(x / \Delta)$$

within its supported range. A mid-tread design has a reconstruction level at zero; a mid-rise design has a decision threshold at zero. Actual integer formats also specify their minimum and maximum codes, rounding behavior, and the treatment of values outside the representable range.

Two kinds of error should be distinguished:

• Granular error occurs when an input falls inside the covered range but is replaced by a nearby reconstruction level.
• Overload or clipping error occurs when an input lies outside that range and is saturated or otherwise mapped to a boundary value.

A smaller step size reduces granular error for values inside a fixed neighborhood, but, with a fixed number of levels, it also narrows the covered range and can increase clipping. Choosing the range is therefore part of the quantizer design. It is not correct to infer quality from the bit width alone.

For a sufficiently fine uniform quantizer under appropriate regularity assumptions, its error is sometimes approximated as uniformly distributed over an interval of width $\Delta$, giving mean-squared error near $\Delta^2/12$. The familiar model of additive, input-independent "quantization noise" is an approximation, not an identity. Quantization error is a deterministic function of the input unless randomness such as dither is introduced, and it can be correlated with the signal. High-resolution formulas can be inaccurate with few levels, strong clipping, discrete inputs, or irregular source densities.^[1]

Nonuniform scalar quantization

A nonuniform scalar quantizer uses cells of different widths. Narrower cells can be placed where the source is common or where errors are costly, while wider cells cover less important regions. Companding implements one such design by applying a nonlinear transformation, uniformly quantizing in the transformed domain, and applying an inverse transformation at reconstruction.

For scalar quantization with squared-error distortion, the Lloyd-Max conditions give two necessary stationarity conditions. Given reconstruction values, decision boundaries lie at nearest-neighbor boundaries. Given cells, each reconstruction value is the conditional mean, or centroid, of the source values in its cell. Alternating these updates cannot increase the distortion, but the result may depend on initialization and need not be the global optimum. Max published minimum-distortion calculations in 1960, and Lloyd's independently developed least-squares method, circulated at Bell Laboratories in 1957, appeared in the open literature in 1982.^[2][3]

Vector quantization

Vector quantization maps a block $x \in \mathbb{R}^k$ to one of a finite set of codevectors. Its cells are regions in a multidimensional space, rather than intervals on a line. By representing correlations among components and optimizing the codebook for a joint distribution, vector quantization can obtain rate-distortion behavior unavailable to independent scalar quantizers. Its direct form can be expensive: storage and nearest-neighbor search grow quickly with vector dimension and codebook size.^[1]

The Linde-Buzo-Gray algorithm designs a codebook from training vectors by alternating assignments to minimum-distortion cells and replacement of codevectors by centroids. It generalizes the Lloyd procedure to vectors and more general distortion measures. The distortion decreases across iterations, but ordinary runs can converge to a local optimum; initialization and the training sample matter.^[4] Structured forms reduce cost. For example, product quantization splits a vector into subspaces and quantizes the sub-vectors separately, trading some modeling freedom for compact codebooks and faster search.

Rate, precision, and representation

Fixed rate and variable rate

The number of quantizer levels and the number of stored bits are related but not interchangeable. A fixed-rate $b$-bit scalar code can address at most $2^b$ indices. A variable-rate encoder can use entropy coding so that frequent indices consume fewer bits on average. Conversely, a practical file or in-memory object may store scale factors, zero-points, codebooks, grouping metadata, padding, sparse indices, and alignment bytes in addition to the low-bit codes.

For this reason, converting a 32-bit weight payload to $b$-bit codes has an ideal raw-code ratio of $32/b$, but it does not guarantee that the entire model file or runtime allocation shrinks by that ratio. Embeddings, biases, normalization parameters, output heads, caches, and unsupported operators may remain at higher precision. Small groups improve local fit but require more metadata. Packing 4-bit or smaller codes into machine words can also add padding or unpacking cost.

Quantization versus reduced floating-point precision

Every finite digital number format represents only a finite subset of real numbers, so conversion from a richer format to a narrower floating-point format is a form of quantization in the broad mathematical sense. In engineering discussions, however, "integer quantization" and "reduced-precision floating point" usually describe different deployment paths.

An integer affine quantizer uses an explicit scale and often a zero-point to interpret integer codes. Floating-point formats distribute their levels nonuniformly through exponent and significand fields. They can cover a wider dynamic range for the same number of bits, but their spacing grows with magnitude. Fixed-point and integer schemes have uniform spacing within a scale group, while learned codebooks or logarithmic formats create other nonuniform level sets.

IEEE 754-2019 specifies arithmetic and interchange formats for binary and decimal floating-point computation, including rounding and exceptional values.^[5] The E4M3 and E5M2 8-bit formats proposed for deep learning in 2022 are not simply the IEEE binary interchange formats shortened to eight bits. The proposal gives E5M2 IEEE-like special-value conventions, while E4M3 extends finite range by omitting infinities and reducing NaN encodings.^[17] The article on FP8 covers those formats in more detail.

Rounding, saturation, and exceptional cases

Rounding-to-nearest is common because it minimizes the error of each scalar assignment when all other choices are fixed, but it is not always optimal for a network-level objective. Stochastic rounding instead assigns adjacent levels probabilistically. Learned or optimized rounding can use data and a reconstruction objective to coordinate assignments across a layer.^[10][13]

Saturation maps out-of-range finite values to a boundary. Floating-point conversions may instead produce infinities, subnormals, signed zeros, or NaNs according to their format and rounding mode. Neural-network deployment pipelines must state how they handle these cases. A graph that contains quantized matrix multiplication but repeatedly converts intermediate tensors to a higher precision is different from an integer-only graph.

Neural-network quantization

What can be quantized

A trained model contains multiple kinds of numerical state, and they have different distributions and operational roles:

• Weights are learned parameters and are usually fixed during inference. Their values can be inspected and encoded offline.
• Activations are intermediate results that depend on the input. Their ranges may vary across examples, sequence positions, or batches.
• Biases and accumulators often require more range than the input codes. An INT8 multiplication pipeline commonly accumulates products into a wider integer type before requantizing.
• Gradients, optimizer states, and master weights matter during training. They do not have to use the same format as inference weights.
• Attention key-value caches grow with sequence length during autoregressive inference and can be quantized independently of the stored parameters.

The notation W8A8 means 8-bit weights and 8-bit activations. W4A16 usually means 4-bit weights with 16-bit activations or computation, although a specification should identify the exact formats. "Weight-only" says nothing about the activation, accumulator, output, or cache formats.

Uniform affine mapping

A widely used integer mapping represents a real value $x$ by an integer $q$, a positive scale $s$, and an integer zero-point $z$:

$$q = \operatorname{clip}\bigl(\operatorname{round}(x/s)+z,\ q_{\min},\ q_{\max}\bigr),$$ $$\hat{x}=s(q-z).$$

Here $\hat{x}$ is the reconstructed approximation. The zero-point is chosen so that real zero is exactly representable, which matters for operations such as zero padding. The scale establishes the spacing of representable real values. Jacob and colleagues developed integer-only neural inference around this affine relationship, wider accumulators, fixed-point rescaling, operator fusion, and simulated quantization during training.^[6]

If an observed real range $[a,b]$ is mapped to integer codes $[q_{\min},q_{\max}]$, a basic asymmetric choice is $s = (b-a)/(q_{\max}-q_{\min})$, followed by a rounded and clipped zero-point. Implementations often adjust the range so that zero is included. Degenerate ranges and non-finite values require explicit handling.

Symmetric quantization uses a range centered on zero and usually fixes the signed integer zero-point at zero. This simplifies some arithmetic and is often suitable for roughly symmetric weight distributions. Asymmetric quantization permits an offset and can use the available codes more efficiently for one-sided or skewed data, such as some post-activation tensors. The best choice depends on the operator and backend; "symmetric" is not inherently more accurate, and "asymmetric" is not free of arithmetic cost.^[8]

Granularity

Quantization parameters can be shared at different granularities:

Scroll sideways for more →

Granularity must be stated together with the bit width. Two "4-bit" checkpoints can have different accuracy and actual size because one uses groups of 32 values and another groups of 128, or because one retains selected channels in a higher precision. Per-channel weight quantization can be especially helpful when output channels have very different ranges. Finer granularity generally reduces local approximation error but does not guarantee better end-task accuracy.^[7][8][9]

Calibration and range selection

Static activation quantization estimates ranges from a calibration set. Simple min-max calibration records extrema, but a single rare outlier can make the quantization step coarse for most values. Percentile, entropy-based, or reconstruction-error criteria deliberately consider clipping some values to improve resolution elsewhere. The calibration sample should represent the intended deployment distribution, preprocessing, sequence lengths, and operating modes.

Calibration is separate from probabilistic calibration of a classifier's confidence scores. Here it means collecting tensor statistics or optimizing quantizer parameters. A method described as "data-free" may still use model weights, synthetic samples, or other assumptions; the precise information available to the method should be stated.

Static parameters are fixed before inference. Dynamic quantization calculates some activation parameters at runtime, often for each tensor, row, token, or batch. Dynamic ranges can adapt to each input, but range calculation and quantize-dequantize operations add overhead. A hybrid system may use static weight codes and dynamic activation scales.

Workflows

Post-training quantization

Post-training quantization, or PTQ, starts with a trained higher-precision model. It selects quantized tensors, estimates or optimizes quantizer parameters, encodes the model, and validates the result without ordinary end-to-end retraining. PTQ ranges from direct round-to-nearest conversion to data-driven layer reconstruction and approximate second-order optimization.^[7][8]

A typical PTQ sequence is:

1. Freeze a known reference model and evaluation configuration.
2. Decide which operators, tensors, formats, bit widths, and granularities the target runtime supports.
3. Fold or fuse operations only where the mathematical and deployment semantics are understood.
4. Collect representative activation statistics if static calibration is used.
5. Quantize, then compare tensor outputs and end-task results with the reference.
6. Exclude sensitive operations, change granularity, or use an optimized rounding or reconstruction method if the quality target is missed.
7. Benchmark the exported artifact on the actual target hardware.

AdaRound showed that independent rounding-to-nearest is not necessarily optimal for post-training network loss. It replaces each layer's rounding decisions with a data-driven optimization based on local reconstruction and a relaxed objective.^[13] Approximate Hessian or sensitivity information can similarly guide which errors or bit widths a network tolerates.

Static and dynamic PTQ

Scroll sideways for more →

These labels do not determine a unique graph. A static model may use quantize-dequantize boundaries around only a subset of operators. A dynamic system may quantize each activation tensor once per call or at a much finer granularity. Documentation and benchmarks should describe the actual graph.

Quantization-aware training

Quantization-aware training exposes a trainable model to the intended rounding and clipping behavior. A common implementation inserts fake-quantization operations in the forward path: a high-precision tensor is quantized and immediately reconstructed, so subsequent computation observes values on the target grid while trainable parameters remain in a form the optimizer can update. Learned Step Size Quantization makes the quantizer step sizes trainable alongside network parameters and reported low-bit ImageNet results across several architectures.^[14]

Rounding has zero derivative almost everywhere and discontinuities at thresholds, so ordinary backpropagation through a hard quantizer is not useful. Many QAT systems use a straight-through estimator, or STE, that substitutes a surrogate derivative in the backward pass. An STE is not the true derivative of the quantized objective. Theory for a simplified activation-quantized network shows that a suitable estimator can provide a descent-related coarse gradient, while a poor choice can produce instability. That result does not establish that every STE is valid for every architecture.^[15]

QAT usually costs more data and compute than PTQ, and its learned robustness can be tied to a particular graph, format, or backend. It may nevertheless be preferable at low bit widths or when PTQ misses a strict quality target. Deployment validation remains necessary because fake quantization does not prove that the exported integer kernels reproduce the training graph exactly.

Reduced-precision training is related but distinct

Mixed-precision training uses more than one numerical format to make training faster or smaller while retaining numerically sensitive operations in higher precision. The widely used FP16 recipe described by Micikevicius and colleagues stores a single-precision master copy of weights, applies loss scaling, and accumulates selected operations in single precision.^[16] This is a numerical training strategy, not automatically a compressed integer inference model.

The distinction matters. A model trained with BF16 activations may still be deployed with FP32 weights. A QAT model may keep high-precision shadow weights during optimization but export INT8 weights. A paper or benchmark should identify the format of storage, arithmetic inputs, products, accumulation, reductions, parameter updates, and the final artifact rather than labeling all of them "quantized."

Historical development

Quantization predates neural networks by decades. It became central to pulse-code modulation, source coding, and communication theory because continuous or finely valued signals had to be represented by discrete symbols. The history surveyed by Gray and Neuhoff includes early scalar quantizer design, high-resolution analysis, predictive and transform coding, rate-distortion theory, and later vector quantization.^[1]

Minimum-distortion scalar design acquired its standard iterative form through Lloyd's 1957 Bell Laboratories work and Max's 1960 publication.^[2][3] Linde, Buzo, and Gray provided a broadly applicable vector-quantizer design algorithm in 1980.^[4] These foundations remain relevant to learned systems: modern codebook methods, clustering-based representations, and block quantizers still balance reconstruction error, rate, codebook cost, and search complexity.

Neural-network research later adapted low-precision representations to learned parameters and computation:

• BinaryConnect (2015) used binary weights during forward and backward propagation while retaining real-valued weights for parameter updates. Its experiments showed that binary propagation could train selected networks on the reported image benchmarks.^[10]
• Binarized Neural Networks (2016) extended binarization to weights and activations and described bitwise implementations. Its results were tied to the tested architectures and datasets, not a proof that binary networks universally match real-valued models.^[11]
• Deep Compression (2016) combined pruning, trained weight sharing, and Huffman coding. The paper reported reducing AlexNet storage from 240 MB to 6.9 MB and VGG-16 from 552 MB to 11.3 MB without accuracy loss in its ImageNet experiments. Those 35-fold and 49-fold figures describe the full three-stage pipeline, not quantization alone.^[12]
• Integer-only inference (2018) formalized affine 8-bit mappings, wider accumulation, fixed-point rescaling, and training simulation for mobile convolutional networks.^[6]
• PTQ and QAT systems (2020 onward) developed optimized rounding, learned quantizer parameters, mixed bit widths, and architecture-specific calibration.^{[13][14][28][29]}
• Large-model methods (2022 onward) addressed transformer activation outliers, low-bit weight-only encoding, quantized fine-tuning, and the growing attention cache.^{[18][19][20][21][22][23][24]}

This chronology is not a progression toward one universally best bit width. It records different solutions for different sources, models, objectives, and hardware.

Method families for neural networks

Weight sharing and codebooks

Codebook quantization stores a small set of representative weight values and an index for each weight. If a layer with $n$ weights uses a codebook of $K$ high-precision centroids, its raw index payload is approximately $n\lceil\log_2 K\rceil$ bits, plus the codebook and any packing metadata. Repeated indices create weight sharing. The centroids can be selected by clustering and then fine-tuned, as in the quantization stage of Deep Compression.^[12]

The codebook need not be a set of uniformly spaced integers. It may be learned, distribution-aware, vector-valued, or composed additively from several smaller codebooks. Codebook lookup can reduce stored bits without providing a native low-bit multiply instruction. A system may reconstruct codebook entries into a conventional floating-point type before computation, fuse lookup into a kernel, or operate on indices in a specialized implementation. Those choices determine runtime cost.

Vector-quantized latent models apply the same broad idea to learned representations rather than merely compressing a finished network. That is a different research problem from replacing a trained layer's scalar weights. The dedicated VQ-VAE article covers a prominent learned-latent example.

Binary and ternary networks

A binary quantizer has two reconstruction levels; a ternary quantizer has three. In neural networks, binary weights can replace some multiplications with sign changes, and binary weights plus binary activations can permit XNOR and population-count operations. Practical training normally keeps latent or shadow parameters at higher precision, because directly updating a two-value parameter is difficult.^[10][11]

The theoretical code rate of one bit per binary value does not equal the size of a deployable model. Scaling factors, normalization parameters, packing, tensor shapes, and unsupported layers still consume storage. Hardware also must expose efficient bitwise kernels at the relevant tensor shapes. A binary or ternary arithmetic count is therefore not directly comparable with an FP16 operation count without a measured implementation.

Research on low-bit architectures includes BitNet and the ternary-weight BitNet b1.58. These architectures train with restricted weight representations and should not be treated as evidence that an arbitrary existing model can be rounded to one or two bits without loss.

Optimized rounding and reconstruction

Straight round-to-nearest minimizes the scalar difference between a weight and its chosen level when the scale and all other assignments are fixed. A network layer, however, applies many weights jointly to structured inputs. Changing one assignment can compensate for another in the layer output. PTQ methods exploit this by minimizing a reconstruction error using calibration examples, by estimating second-order sensitivity, or by updating quantization parameters layer by layer.^[13]

GPTQ is a one-shot weight-quantization method based on approximate second-order information. In its ICLR 2023 paper, the authors reported quantizing 175-billion-parameter GPT-family models to 3 or 4 bits per weight in about four GPU hours, with small measured degradation in their experiments. They also reported end-to-end speedups for custom kernels on specific NVIDIA A100 and A6000 configurations.^[19] These figures are method-and-system results from that study. They do not imply that every 175-billion-parameter checkpoint, 3-bit container, GPU, or inference framework has the same fit, quality, or speed.

Activation-aware weight quantization uses activation observations to decide how weight error should be allocated. AWQ identifies a small fraction of salient weight channels through activation statistics and searches for per-channel scaling that protects them without backpropagation. Its paper states that protecting about 1% of salient weights substantially reduces error, and its TinyChat system achieved more than threefold speedup over the specified Hugging Face FP16 implementation on tested desktop and mobile GPUs.^[22] The algorithmic result and the TinyChat kernel result are separate claims.

Additive codebook approaches represent a weight vector as a sum of entries selected from several codebooks. They can reach low average rates but require codebook searches during compression and specialized decoding or matrix kernels. AQLM is a dedicated article for additive quantization of language models.

Smoothing, rotations, and outlier handling

Activation distributions can contain a small number of large-magnitude components. With one scale shared across a tensor or channel, an outlier can enlarge the step for all ordinary values. Several methods transform the network or separate the outliers rather than simply increasing the bit width.

LLM.int8() applies vector-wise 8-bit quantization to most transformer matrix-multiplication values and handles identified outlier feature dimensions in 16-bit. Its NeurIPS 2022 experiments reported that more than 99.9% of values were multiplied in 8-bit and that models through 175 billion parameters retained the evaluated full-precision performance.^[18] The mixed-precision path is essential to that result; it is not evidence for uniform per-tensor INT8 conversion.

SmoothQuant moves quantization difficulty from activations to weights by applying an offline, mathematically equivalent channel scaling. The paper evaluated W8A8 matrix multiplication across several large language model families and reported up to 1.56-fold speedup and twofold memory reduction, with negligible loss under its tests. It also demonstrated serving a 530-billion-parameter model within one multi-GPU node.^[20] "One node" is not "one GPU," and the result depends on the paper's implementation and hardware.

QuaRot applies function-preserving orthogonal rotations to reduce hidden-state outliers before quantizing weights, activations, and the attention cache. For its 4-bit Llama 2-70B experiments, the paper reported at most 0.47 additional WikiText-2 perplexity and retention of 99% of its evaluated zero-shot performance.^[24] The method demonstrates an end-to-end 4-bit design for the tested Llama 2 models, not a general guarantee for every architecture.

Outliers are not necessarily an unavoidable consequence of parameter count. Experiments in Intriguing Properties of Quantization at Scale found that the occurrence of large activation outliers and associated quantization cliffs was sensitive to pretraining choices. The authors successfully quantized their tested 410-million to 52-billion-parameter models with limited degradation by using a more quantization-friendly training recipe.^[25] This qualifies, rather than disproves, earlier observations: scale, architecture, training, normalization, and the selected quantizer can interact.

Mixed bit width

Mixed-bit systems allocate different precisions to layers, channels, tensors, or operations. Sensitive components receive more bits, while robust components receive fewer. A sensitivity measure may be based on reconstruction error, a Hessian approximation, gradients, or direct task evaluation. A resource budget can then be treated as a constrained optimization problem.

HAWQ-V3 combined a Hessian-aware mixed-precision assignment with an integer-only computational graph and a hardware cost constraint. In its reported ResNet-50 deployment, uniform 4-bit inference was on average 1.45 times faster than uniform 8-bit on a T4 GPU, and a mixed INT4/INT8 configuration reduced INT8 latency by 23% while reporting 76.73% accuracy.^[28] Those values should remain attached to that model, implementation, and device. Mixed precision can also increase engineering complexity and inhibit generic kernel fusion.

Large-language-model scaling experiments provide another perspective. Dettmers and Zettlemoyer ran more than 35,000 zero-shot experiments over 19-million to 176-billion-parameter models from several named families, using 3- to 8-bit weights and 16-bit inputs. Within that study's comparison of total parameter bits and selected zero-shot tasks, 4-bit weights were almost universally the best tested trade-off, and block sizes of 64 to 128 often worked best.^[27] This is an empirical result over a defined experimental grid, not a law that makes 4-bit optimal for every model, task, activation format, cache, or latency target.

Large language models

Weight-only inference

Autoregressive large language models often decode at small batch sizes where reading a large weight matrix can dominate a token step. Weight-only quantization reduces this traffic. A kernel may unpack low-bit codes and dequantize them into registers or fuse the scale operation with matrix multiplication. It can improve latency when reduced memory traffic outweighs unpacking, scale lookup, and conversion.

Weight-only compression does not shrink activations or the attention cache, and it does not necessarily reduce the arithmetic precision of the matrix multiplication. At large batches or long prompt prefills, computation can become the bottleneck, making joint weight-and-activation quantization more attractive. The balance changes across prefill and decode, model shape, sequence length, and parallelization strategy.

Quantized fine-tuning

QLoRA is primarily a memory-efficient fine-tuning method. It stores a frozen pretrained base model in 4-bit NormalFloat, backpropagates through its reconstructed values, and trains high-precision LoRA adapters. It also quantizes quantization constants through "double quantization" and uses paged optimizers to manage memory spikes. The paper reported fine-tuning a 65-billion-parameter model on one 48 GB GPU while matching its full 16-bit fine-tuning baseline on the evaluated tasks.^[21]

NF4 is a nonuniform 4-bit codebook designed for normally distributed weights. Calling it an "integer" format obscures that the four-bit patterns index reconstruction values. QLoRA does not mean that optimizer states, gradients, adapters, and arithmetic all use four bits. It also does not by itself specify a production inference format for the merged or adapter-equipped model.

Attention-cache quantization

During autoregressive generation, each transformer layer stores key and value vectors from prior tokens. This key-value cache can dominate memory for long contexts even if model weights are already compressed. KV-cache quantization uses a separate quantization scheme for this growing state.

KVQuant combines per-channel key quantization, quantization before rotary positional embedding, learned nonuniform data types, and sparse retention of outliers. Across the paper's tested LLaMA, Llama 2, Llama 3, and Mistral models, its 3-bit configurations produced less than 0.1 perplexity degradation on WikiText-2 and C4. The authors demonstrated a one-million-token cache for LLaMA-7B on one A100 80 GB GPU, a ten-million-token setup on eight such GPUs, and up to about 1.7-fold speedup for their custom LLaMA-7B matrix-vector kernels.^[23] These are controlled system demonstrations, not statements about usable semantic context, model support for those lengths, or performance on unrelated hardware.

End-to-end versus partial quantization

An "end-to-end quantized" claim should identify every remaining high-precision path. Common exceptions include embeddings, output heads, normalization, softmax, nonlinear functions, residual additions, position encodings, sampling, and accumulators. Some systems call a model W4A4 even though selected channels or operations remain in FP16. That may be a sound engineering choice, but it should be disclosed.

The compute graph also changes over the lifetime of a request. Prompt prefill performs matrix-matrix operations over many tokens, while single-token decode often performs matrix-vector operations and repeatedly reads weights and cache data. A quantizer that improves one phase can have little effect on the other. End-to-end request latency includes tokenization, scheduling, communication, cache management, and sampling as well as low-bit matrix kernels.

Data formats and arithmetic

Integer codes and scales

Signed INT8 provides 256 codes, conventionally from -128 through 127. A symmetric implementation may use only -127 through 127 so positive and negative magnitudes have equal ranges, but this is a convention rather than a requirement of the data type. INT4 and sub-byte values usually need packing because common processors address bytes or larger words.

The stored scale can be FP32, FP16, BF16, or another type. Zero-points may be omitted for symmetric groups or stored separately for asymmetric groups. A scale per 32 weights costs more per value than a scale per 128 weights. For a four-bit weight tensor with a 16-bit scale per group of $g$ and no other overhead, the payload is approximately $4 + 16/g$ bits per weight before padding. This illustrative calculation is not a file-size guarantee.

Multiplying two quantized values produces a result with more range. INT8-by-INT8 products are therefore commonly accumulated into 32-bit integers. Biases can be represented at a scale compatible with the product of the input and weight scales. The accumulator is then multiplied by a fixed-point or floating scale, rounded, offset, and clamped into the next tensor's format. Overflow behavior and the order of requantization affect correctness.^[6]

Floating-point formats

FP16 has a five-bit exponent and ten explicitly stored fraction bits. bfloat16 keeps the eight-bit exponent width of binary32 and uses fewer fraction bits, giving it a broad dynamic range but lower precision than FP16 near the same magnitude. Both use 16-bit storage, yet their numerical errors and hardware support differ.^[17]

The E4M3 and E5M2 FP8 encodings trade precision against dynamic range. The 2022 proposal used E4M3 more often for forward tensors and E5M2 where greater range was needed, while demonstrating training across selected convolutional, recurrent, and transformer models through a 175-billion-parameter language model.^[17] These results support the two proposed formats under the paper's scaling and accumulation recipes. They do not establish bitwise interoperability among every later vendor implementation.

Floating-point conversion can be useful without an explicit zero-point, but tensor scaling is still common for FP8 because the format has limited range. Implementations may use a scale for each tensor, channel, or delayed window of observations. The term "FP8 quantization" should therefore include both the encoding and the scaling policy.

Learned and nonuniform formats

Nonuniform low-bit formats place reconstruction values according to a target distribution. NF4 is one example for approximately normal weights.^[21] Learned step sizes adjust the spacing of a uniform quantizer during QAT.^[14] Vector and additive quantizers learn codebooks for groups of values. Logarithmic formats devote more codes to small magnitudes.

These representations can reduce distortion at a given nominal rate, but they may not match a processor's native arithmetic. A lower reconstruction error measured before deployment can be offset by lookup, decoding, or conversion overhead. Comparing formats fairly requires both task metrics and a real implementation.

Accuracy and model behavior

Error propagation

Quantization changes a model at many sites, and local numerical errors can interact through depth, residual connections, attention, and nonlinearities. Mean-squared weight error is useful for diagnosis but is not a complete predictor of end-task quality. A small change in a sensitive direction can matter more than a larger change in an insensitive one. Conversely, errors can cancel.

The appropriate evaluation depends on the model:

• Image classifiers need top-1 or top-5 accuracy, but detection and segmentation require their own metrics.
• Language models often use perplexity plus downstream or generative evaluations.
• Retrieval, speech, recommendation, control, and scientific models need domain-specific measures.
• Generative systems also need checks for output distribution, instruction following, refusal behavior, and other deployment requirements.

Evaluation should use the same tokenizer, preprocessing, decoding settings, prompts, and data split for the reference and quantized models. Reporting only the quantized score without the exact reference hides the degradation. "No accuracy loss" should mean no detected loss under a named test and uncertainty level, not mathematical equivalence.

Distribution shift and calibration

A static range fitted on one distribution can clip or coarsely represent another. Calibration examples should cover relevant languages, image conditions, sequence lengths, audio levels, and operational modes. Leakage from a final test set into quantizer selection can bias the reported quality, just as it can in ordinary model selection.

Vision transformers illustrate why architecture-specific validation matters. Their attention maps and layer sensitivities do not necessarily resemble those of convolutional networks. A 2021 PTQ method for vision transformers added a ranking loss for attention and used feature statistics for mixed precision, reporting 81.29% ImageNet top-1 accuracy for an approximately 8-bit DeiT-B configuration.^[29] That result does not imply that generic INT8 conversion gives the same score.

Scale is not a sufficient predictor

Larger models sometimes absorb weight perturbations, but large transformer activations can also contain outliers that make a shared integer scale ineffective. The observed result depends on the training recipe and model family.^[18][25] It is therefore unsafe to state that quantization becomes easier or harder solely because parameter count increases.

Likewise, a larger low-bit model can outperform a smaller high-precision model under a fixed parameter-bit budget, as the k-bit scaling study investigated.^[27] This is a model-selection comparison, not proof that quantizing one fixed model preserves its behavior. Parameter count, training compute, data, architecture, and bit width are separate variables.

Safety and rare behavior

Aggregate utility metrics can remain stable while rare or safety-relevant behavior changes. An ICML 2025 study evaluated several quantization methods, bit widths, model families, calibration datasets, and safety benchmarks. Within that experimental scope, it found safety degradation after quantization, stronger degradation at lower bit widths, and sensitivity to the content of calibration data; it also proposed a quantization-aware safety patching method.^[26]

The finding should not be generalized into "all quantization makes every model unsafe." It does show that perplexity or average task accuracy alone is not an adequate acceptance test for a safety-sensitive deployment. Teams should rerun the safety, fairness, robustness, privacy, and domain-specific evaluations that apply to the original model, using the quantized artifact and production decoding path.

Performance and memory

Storage

For $n$ values, an ideal densely packed $b$-bit code payload occupies $nb/8$ bytes. A corresponding FP32 payload occupies $4n$ bytes. This yields an ideal raw-value ratio of $32/b$. It is a useful upper-bound calculation for the affected tensor, not a measured whole-model ratio.

Actual storage includes:

• scales, zero-points, codebooks, sparse indices, and group metadata;
• padding and alignment imposed by a file format or accelerator;
• tensors intentionally kept in FP16, BF16, or FP32;
• graph structure, tokenizer or vocabulary data, and other nonparameter assets;
• duplicate or prepacked weights for different kernels.

Runtime memory adds temporary workspaces, activation buffers, attention caches, communication buffers, and allocator fragmentation. QLoRA's 65-billion-parameter fine-tuning result, for example, depended on frozen 4-bit base weights, trainable adapters, double quantization, paged optimizers, and a specified 48 GB GPU.^[21] It cannot be derived from parameter bits alone.

Latency and throughput

Lower-bit arithmetic can increase the number of operations a processor executes per cycle and can reduce bytes transferred from memory. Whether that improves an application depends on its bottleneck. A memory-bound weight-only decode may benefit mainly from weight compression. A compute-bound prefill may require quantized activations and native low-bit matrix instructions. A tiny model can be dominated by launch and conversion overhead.

Useful measurements include:

• end-to-end latency at specified prompt and output lengths;
• time to first token and inter-token latency for autoregressive models;
• throughput at specified batch sizes and concurrency;
• peak and steady-state memory;
• kernel coverage and the fraction of execution that falls back to another precision;
• power or energy measured over the same work, not inferred from bit width.

ONNX Runtime's documentation explicitly warns that performance depends on the model and hardware, that old hardware may lack useful low-bit instructions, and that quantize-dequantize overhead can make a quantized model slower.^[32] Accordingly, statements such as "INT8 is four times faster" or "4-bit cuts energy by 80%" require a cited measurement with a named system. Bit width alone cannot support them.

Hardware and kernel support

The relevant capability is not merely whether a processor can store an INT4 or FP8 value. It must also efficiently load the packed layout, apply scales, perform the intended multiply or lookup, accumulate with sufficient range, and integrate the operation with surrounding graph nodes. An unsupported activation, transpose, or normalization can force conversions that erase the expected gain.

CPU vector instructions, GPU tensor or matrix units, mobile neural accelerators, and custom inference processors support different signedness, group sizes, layouts, accumulators, and operator sets. A checkpoint optimized for one backend can require repacking or requantization for another. Hardware-aware quantization treats those restrictions as part of the optimization rather than as an afterthought.^[7][28]

Model and workload considerations

Convolutional networks

Convolutional models were central to early practical integer inference. Per-output-channel weight scales help when filters have different ranges, and static activation calibration can work well when deployment inputs resemble the calibration data. Batch-normalization parameters are often folded into an adjacent convolution before quantization, but folding changes weight ranges and must occur in the same order used during training or calibration.^[6][8][9]

The input, first layer, final classifier, residual additions, and depthwise convolutions can be more sensitive than ordinary convolutions in some architectures. This is not universal. A conversion tool should provide per-operator diagnostics rather than relying on a fixed list of layers to exclude.

Transformers

Transformer quantization must account for projection matrices, residual streams, normalization, attention scores, softmax, feed-forward activations, positional operations, and the attention cache. Weight-only methods are attractive for memory-bound decoding. W8A8 or W4A4 systems target more of the compute but must handle activation outliers and nonlinear operations.

Sequence length and batch size change both memory pressure and arithmetic intensity. Calibration prompts should reflect expected length and content. A language-model benchmark on short English prompts does not establish quality for long multilingual, code, mathematical, or tool-use workloads.

Edge and mobile deployment

Quantization can help Edge AI by reducing package size and data movement, but mobile and embedded accelerators commonly impose strict operator and tensor-shape constraints. Unsupported nodes may run on a CPU, introducing transfers between processors. End-to-end tests should include model loading, thermal behavior, sustained latency, and battery or power measurements where relevant.

Full integer input and output can avoid boundary conversions when a sensor pipeline already supplies bytes or integers. Hybrid models may retain floating-point inputs and outputs for convenience. Neither design is universally faster; conversion placement and accelerator delegation determine the result.

Quantization among compression methods

Model compression also includes pruning, knowledge distillation, low-rank factorization, and architecture redesign. These methods change different aspects of cost:

• Quantization reduces bits per represented value and may enable cheaper arithmetic.
• Pruning removes parameters or operations, but unstructured sparsity requires suitable sparse kernels to improve speed.
• Distillation trains a separate student to imitate a teacher, potentially changing architecture and behavior.
• Low-rank methods factor matrices or restrict trainable updates.

They can be combined, as Deep Compression combined pruning, quantized weight sharing, and entropy coding.^[12] The contribution of each stage should be measured separately when attributing size or speed gains.

Software and deployment representations

Software APIs evolve, so a reproducible report should record framework and runtime versions as well as quantizer settings.

PyTorch

The PyTorch documentation updated in May 2026 states that quantization development is being centralized in the torchao project. It points users from older eager-mode and FX graph-mode APIs toward maintained torchao flows while documenting compatibility and migration status.^[30] This matters for long-lived deployments: an example written against a legacy namespace may still run, but it should not be presented as the current development path without a version qualifier.

PyTorch-family workflows can perform PTQ, QAT, dynamic quantization, weight-only packing, and export, but support varies by backend. The observer or fake-quantization configuration used during preparation must be compatible with the exported operators and target runtime. A Python model that simulates quantization is not yet proof of a deployable low-bit kernel.

LiteRT

Google's TensorFlow Lite (LiteRT) documentation distinguishes dynamic-range, float16, full-integer, and integer-input/output conversion paths. Full integer quantization requires a representative dataset so the converter can estimate activation ranges.^[31] Whether the resulting model is entirely delegated to a device accelerator depends on its supported operators and data types.

The representative-data generator must supply tensors with the same shape, type, preprocessing, and range as deployment inputs. A model can convert successfully while still losing accuracy because the sample was unrepresentative or because a sensitive operation used an unsuitable range.

ONNX Runtime

ONNX Runtime represents quantized graphs in two principal ways. QOperator graphs replace ordinary operators with quantized operator forms, while QDQ graphs insert QuantizeLinear and DequantizeLinear nodes around otherwise recognizable operators. Static quantization stores activation parameters obtained from calibration; dynamic quantization computes them during execution.^[32]

The documentation recommends separating graph preprocessing from quantization so that weight and activation differences remain easier to debug. It also exposes tools for matching tensors between the reference and quantized graphs. Operator support, signedness, saturation behavior, execution provider, and processor instructions affect both accuracy and speed.^[32]

Core ML Tools

Core ML Tools documents data-free round-to-nearest weight PTQ, calibration-based activation quantization, GPTQ-style layerwise compression, and fine-tuning-based QAT. Apple's documentation warns that default PyTorch quantization settings need not be optimal for the Core ML stack and Apple hardware, and recommends the Core ML Tools defaults for that conversion path.^[33] An encoded model may be smaller on disk without every operation executing at the nominal precision.

Across frameworks, a robust deployment record includes the original model hash, exported artifact hash, tool versions, calibration data description, quantizer configuration, target device, runtime flags, and measured results. Without those details, a reported "INT8 model" is difficult to reproduce.

Evaluation and validation

Reference model

Validation begins with an immutable reference artifact. Run the reference and candidate with identical preprocessing and task settings before attributing any difference to quantization. If graph optimization, operator fusion, or framework conversion happens at the same time, also evaluate an unquantized converted graph so conversion error can be separated from quantization error.

For deterministic components, tensor-level comparisons can identify the first layer whose output diverges beyond an expected bound. For stochastic generation, use fixed seeds where possible, but also evaluate distributions and task outcomes across enough prompts. Exact token equality is often too strict for diagnosis and too weak for safety: one changed early token can produce a different but equally valid continuation, while unchanged benchmark answers can hide altered rare behavior.

Calibration discipline

Calibration data should be held apart from the final evaluation set. It should cover the deployment distribution without containing private or prohibited data. Report the number of examples, sampling rule, preprocessing, sequence length, and whether labels were used. If ranges or methods were selected after looking at test performance, that test set has become part of model selection and a new held-out evaluation is needed.

Outlier clipping, group size, mixed-precision exclusions, and reconstruction iterations are hyperparameters. Searching them over a benchmark can overfit the quantizer even though the original weights remain frozen. A separate acceptance set reduces this risk.

Quality checks

A complete quality report can include:

1. Task metrics for the reference and quantized artifact, with absolute and relative differences.
2. Per-subgroup or per-domain results where aggregate scores can hide failures.
3. Calibration-distribution and shifted-distribution tests.
4. Long-context, high-amplitude, or other boundary cases relevant to range selection.
5. Safety, robustness, fairness, and privacy checks appropriate to the application.
6. Layer or tensor diagnostics for clipping rate, reconstruction error, saturation, and non-finite values.
7. Repeated measurements or uncertainty intervals when the evaluation itself is noisy.

The tolerance should be selected before inspecting the candidate when possible. "Negligible" is an application judgment and should be attached to a numerical threshold.

Systems checks

Benchmark the exact serialized artifact and runtime that will be deployed. Warm up compilation and caches, then report the measurement protocol. Control batch size, sequence length, thread count, clock or power mode, concurrent load, and memory allocator. Separate model-loading time, prefill, decode, and postprocessing where relevant.

Check the runtime trace for fallbacks and conversions. Compare file size, resident memory, peak memory, memory bandwidth if measurable, latency distributions, throughput, and energy over a fixed amount of work. A microkernel speedup and an end-to-end application speedup answer different questions.

Reproducibility checklist

Scroll sideways for more →

Common misconceptions

Scroll sideways for more →

Practical selection guide

The deployment objective should choose the method, not the popularity of a format.

• If disk or transfer size is the main constraint, measure a weight-only or codebook representation first, including metadata.
• If small-batch decode is memory-bound, test low-bit weight kernels on the target device.
• If prefill or large-batch throughput is compute-bound, investigate supported weight-and-activation formats and end-to-end graph coverage.
• If the attention cache dominates long-context memory, evaluate a cache-specific method separately from weight quantization.
• If simple PTQ misses the quality target, try finer granularity, different clipping, optimized rounding, mixed precision, or QAT.
• If training memory is the constraint, distinguish reduced-precision training and quantized-base adapter tuning from inference compression.
• If deployment is safety-sensitive, rerun the relevant behavioral suite rather than accepting average accuracy alone.

No method should be selected from the nominal bit width alone. The smallest acceptable artifact may not be the fastest, and the highest-throughput graph may not have the lowest single-request latency. A defensible choice reports the complete quality-cost frontier on the intended workload.

See also

• Artificial Intelligence for the broader field in which neural-network quantization is used
• Deep Learning for training and deployment of multilayer learned models
• Computer Vision for an application area central to early practical network quantization
• Inference for the execution stage targeted by many quantized deployments
• Tensor for the multidimensional arrays to which scales and granularities are assigned

References

1. ^Gray, R. M., and Neuhoff, D. L. (1998). "Quantization." IEEE Transactions on Information Theory, 44(6), 2325-2383. doi.org/...18.720541
2. ^Max, J. (1960). "Quantizing for Minimum Distortion." IRE Transactions on Information Theory, 6(1), 7-12. doi.org/...TIT.1960.1057548
3. ^Lloyd, S. P. (1982). "Least Squares Quantization in PCM." IEEE Transactions on Information Theory, 28(2), 129-137. doi.org/...TIT.1982.1056489
4. ^Linde, Y., Buzo, A., and Gray, R. M. (1980). "An Algorithm for Vector Quantizer Design." IEEE Transactions on Communications, 28(1), 84-95. doi.org/...TCOM.1980.1094577
5. ^IEEE Standards Association. (2019). "IEEE Standard for Floating-Point Arithmetic, IEEE 754-2019." standards.ieee.org/...6210
6. ^Jacob, B., et al. (2018). "Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference." Proceedings of CVPR, 2704-2713. openaccess.thecvf.com/..._Training_CVPR_2018_paper
7. ^Gholami, A., et al. (2021). "A Survey of Quantization Methods for Efficient Neural Network Inference." arXiv:2103.13630. arxiv.org/...2103.13630
8. ^Nagel, M., et al. (2021). "A White Paper on Neural Network Quantization." arXiv:2106.08295. arxiv.org/...2106.08295
9. ^Krishnamoorthi, R. (2018). "Quantizing Deep Convolutional Networks for Efficient Inference: A Whitepaper." arXiv:1806.08342. arxiv.org/...1806.08342
10. ^Courbariaux, M., Bengio, Y., and David, J.-P. (2015). "BinaryConnect: Training Deep Neural Networks with Binary Weights during Propagations." Advances in Neural Information Processing Systems 28. papers.nips.cc/...979ed25912dff5b0669f2cd-Abstract
11. ^Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., and Bengio, Y. (2016). "Binarized Neural Networks." Advances in Neural Information Processing Systems 29. papers.nips.cc/...a17c53d217014ee776bfd50-Abstract
12. ^Han, S., Mao, H., and Dally, W. J. (2016). "Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding." ICLR 2016. iclr.cc/...2016
13. ^Nagel, M., van Baalen, M., Blankevoort, T., and Welling, M. (2020). "Up or Down? Adaptive Rounding for Post-Training Quantization." Proceedings of ICML, PMLR 119:7197-7206. proceedings.mlr.press/...nagel20a
14. ^Esser, S. K., et al. (2020). "Learned Step Size Quantization." ICLR 2020. openreview.net/forum
15. ^Yin, P., et al. (2019). "Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets." ICLR 2019. iclr.cc/...671
16. ^Micikevicius, P., et al. (2018). "Mixed Precision Training." ICLR 2018. openreview.net/forum
17. ^Micikevicius, P., et al. (2022). "FP8 Formats for Deep Learning." arXiv:2209.05433. arxiv.org/...2209.05433
18. ^Dettmers, T., Lewis, M., Belkada, Y., and Zettlemoyer, L. (2022). "LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale." Advances in Neural Information Processing Systems 35. proceedings.neurips.cc/...9c8a-Abstract-Conference
19. ^Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D. (2023). "GPTQ: Accurate Post-Training Quantization for Generative Pre-Trained Transformers." ICLR 2023. openreview.net/forum
20. ^Xiao, G., et al. (2023). "SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models." Proceedings of ICML, PMLR 202:38087-38099. proceedings.mlr.press/...xiao23c
21. ^Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L. (2023). "QLoRA: Efficient Finetuning of Quantized LLMs." Advances in Neural Information Processing Systems 36. papers.nips.cc/...2beaa62a049b-Abstract-Conference
22. ^Lin, J., et al. (2024). "AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration." Proceedings of Machine Learning and Systems 6. proceedings.mlsys.org/...1ef21-Abstract-Conference
23. ^Hooper, C., et al. (2024). "KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization." Advances in Neural Information Processing Systems 37. papers.nips.cc/...4d61b42c99a4-Abstract-Conference
24. ^Ashkboos, S., et al. (2024). "QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs." Advances in Neural Information Processing Systems 37. papers.nips.cc/...d0da5e81af7c-Abstract-Conference
25. ^Ahmadian, A., et al. (2023). "Intriguing Properties of Quantization at Scale." Advances in Neural Information Processing Systems 36. papers.nips.cc/...f05c7c0ccb82-Abstract-Conference
26. ^Chen, K., et al. (2025). "Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Models." Proceedings of ICML, PMLR 267:9728-9746. proceedings.mlr.press/...chen25ci
27. ^Dettmers, T., and Zettlemoyer, L. (2023). "The Case for 4-bit Precision: k-bit Inference Scaling Laws." Proceedings of ICML, PMLR 202:7750-7774. proceedings.mlr.press/...dettmers23a
28. ^Yao, Z., et al. (2021). "HAWQ-V3: Dyadic Neural Network Quantization." Proceedings of ICML, PMLR 139:11875-11886. proceedings.mlr.press/...yao21a
29. ^Liu, Z., et al. (2021). "Post-Training Quantization for Vision Transformer." Advances in Neural Information Processing Systems 34. papers.nips.cc/...a99787bd197eacd77acce5e-Abstract
30. ^PyTorch. (2026). "Quantization." PyTorch 2.13 documentation, updated May 11, 2026. docs.pytorch.org/...quantization
31. ^Google AI Edge. "Post-training quantization." LiteRT documentation. developers.google.com/...post_training_quantization
32. ^Microsoft. "Quantize ONNX Models." ONNX Runtime documentation. onnxruntime.ai/...quantization
33. ^Apple. "Quantization Algorithms." Core ML Tools documentation. apple.github.io/...opt-quantization-algos