Neural Tangent Kernel

RawGraph

The neural tangent kernel (NTK) is a kernel function built from the parameter gradients of a neural network. For a scalar-output network f(x; theta) with P parameters, it is the inner product of the two gradient vectors obtained by differentiating the network output at two inputs:

Θ(x,x)=p=1Pf(x;θ)θpf(x;θ)θp\Theta(x, x') = \sum_{p=1}^{P} \frac{\partial f(x;\theta)}{\partial \theta_p}\,\frac{\partial f(x';\theta)}{\partial \theta_p}

The object matters because of what happens to it when the network gets wide. Arthur Jacot, Franck Gabriel, and Clement Hongler showed in a paper first posted to arXiv on 20 June 2018 and published at NeurIPS 2018 that as the hidden layers of a fully connected network grow toward infinite width, the NTK converges in probability to a deterministic kernel that depends only on the architecture, the nonlinearity, and the input data, not on the random draw of the weights. More surprisingly, that limiting kernel does not move while the network is being trained by gradient descent [1]. Training a network in that limit is therefore the same as running kernel gradient descent with a fixed kernel on a functional cost that is convex, and for squared loss the resulting dynamics are linear and solvable in closed form.

That result gave deep learning theory something it had lacked: a regime in which the training of a non-convex model can be analyzed exactly. Within a couple of years the NTK had been used to prove global convergence for gradient descent on deep networks [12][13], to predict which components of a target function are fitted first [14], and to design practical fixes for coordinate-based networks [16] and physics-informed solvers [17].

The limitation is built into the same property. A kernel frozen at its initialization value cannot change the features the model uses, so the infinite-width limit describes a network that fits data without learning representations from it. Later work proved this is not an artifact of the analysis but a genuine dichotomy [19], and measured the accuracy cost: on CIFAR-10 the convolutional kernel limit trails the finite architecture it is derived from by 5 to 6 points [8]. The NTK is therefore best read as an exact description of one corner of neural network behavior rather than a theory of deep learning as practiced.

Background: infinite width and Gaussian processes

The NTK sits on top of an older observation. In a 1994 technical report, Radford Neal showed that a one-hidden-layer network with a Gaussian prior on its hidden-to-output weights converges, as the number of hidden units grows without bound, to a Gaussian process prior over functions. The shape of that process (smooth, Brownian, or fractional Brownian) depends on the activation function and on the prior over input-to-hidden weights [2].

Two later papers extended this to arbitrary depth. Jaehoon Lee, Yasaman Bahri, Roman Novak, Samuel Schoenholz, Jeffrey Pennington, and Jascha Sohl-Dickstein derived the exact correspondence between infinitely wide deep networks and Gaussian processes, built an efficient pipeline for computing the layerwise covariance function, and used it to do Bayesian inference on MNIST and CIFAR-10. They found that trained finite networks approached the accuracy of the corresponding Gaussian process as width grew, and that the Gaussian process uncertainty correlated with trained-network error [3]. Alexander Matthews, Mark Rowland, Jiri Hron, Richard Turner, and Zoubin Ghahramani proved convergence in distribution for wide multi-layer networks and measured the rate empirically using maximum mean discrepancy [4]. The kernel from this line of work is now usually called the NNGP kernel.

The gap that Jacot, Gabriel, and Hongler closed was that all of this described the network at initialization, or a Bayesian posterior. It said nothing about what gradient descent does.

The 2018 result

The paper works in a specific parametrization, since the conclusion depends on it. Each layer's pre-activation is written as a 1/sqrt(n) multiple of the weight matrix applied to the previous layer, with weights and biases drawn independently from a standard normal distribution rather than from a variance-scaled distribution [1]. Moving the width factor into the forward pass rather than the initializer changes how gradients scale per layer, and this "NTK parametrization" is what produces a fixed kernel in the limit.

Under that setup the paper establishes three things. First, at initialization the NTK, which is random for any finite network, converges in probability to a deterministic limiting kernel as the hidden widths go to infinity. Second, during training the kernel stays at that limit, uniformly over any finite time interval. Third, once the kernel is fixed, the network function evolves by kernel gradient descent on the functional cost, which for squared loss is a linear differential equation in function space [1].

The limiting kernel is computable by a layer-by-layer recursion that reuses the NNGP kernel. Writing Sigma for the NNGP covariance at each layer and Sigma-dot for the corresponding expectation over the derivative of the nonlinearity, the depth recursion is

Θ(L+1)(x,x)=Θ(L)(x,x)Σ˙(L+1)(x,x)+Σ(L+1)(x,x)\Theta^{(L+1)}_{\infty}(x,x') = \Theta^{(L)}_{\infty}(x,x')\,\dot{\Sigma}^{(L+1)}(x,x') + \Sigma^{(L+1)}(x,x')

with the base case equal to the first-layer NNGP kernel [1]. The paper also proves the limiting kernel is positive definite on the unit sphere for any non-polynomial Lipschitz nonlinearity at depth two or greater, which is what guarantees that gradient descent converges rather than stalling [1].

Because the dynamics are linear, they decompose in the eigenbasis of the kernel. Each kernel principal component is fitted at a rate proportional to its eigenvalue, so large-eigenvalue directions are learned quickly and small-eigenvalue directions slowly. Early stopping then acts as a spectral filter that keeps the fast components and discards the slow ones [1]. The paper's experiments used fully connected ReLU networks with hidden widths between 50 and 10,000 on a two-dimensional synthetic target and on MNIST, and showed the kernel's variance and its drift during training both shrinking as width grew [1].

Lazy training

A companion idea arrived from Lenaic Chizat, Edouard Oyallon, and Francis Bach, who published "On Lazy Training in Differentiable Programming" at NeurIPS 2019. Their point is that the frozen-kernel behavior is not really about neural networks or about overparametrization. It follows from a scaling choice. If a differentiable model h(w) is multiplied by a factor alpha and the loss is divided by alpha^2, then for large alpha the optimization path stays close to the path of the model's linearization around initialization, and the effective problem becomes kernel learning [5].

They give a dimensionless criterion for when this happens, combining the size of the initial residual with the ratio of the model's curvature to the square of its Jacobian norm; when that quantity is much smaller than one, training is lazy. The paper's experimental section is the part usually cited as a caution: on CIFAR-10 with standard convolutional architectures, deliberately pushing the model into the lazy regime made test accuracy worse, which the authors read as evidence that lazy training is unlikely to explain the success of deep networks on hard high-dimensional problems [5].

Jaehoon Lee and coauthors made the linearization concrete in "Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent" (NeurIPS 2019). They showed that in the infinite-width limit the dynamics are exactly those of the first-order Taylor expansion of the network around its initial parameters, that squared-loss training produces test predictions drawn from a Gaussian process, and that the agreement between a real network and its linearization is close even at practical finite widths across architectures, optimizers, and losses [6].

NNGP and NTK compared

NNGP kernelNeural tangent kernel
Built fromCovariance of network outputs at random initializationGram matrix of parameter gradients
Corresponds toBayesian posterior of an infinitely wide networkInfinitely wide network trained by gradient descent
Earliest formNeal, single hidden layer, 1994 [2]Jacot, Gabriel and Hongler, 2018 [1]
Deep recursionLayerwise covariance recursion [3][4]Recursion that reuses the NNGP kernel [1]
Library call in Neural Tangentskernel_fn(x1, x2, 'nngp')kernel_fn(x1, x2, 'ntk') [11]

Both kernels are computed by the same kind of depth recursion, and both are available for a wide range of architectures. Greg Yang's tensor programs framework generalized the Gaussian-process derivations to recurrent networks, convolutions, residual networks, and attention, established NTK convergence at initialization within the same formalism, and clarified when the "gradient independence" assumption used in earlier informal derivations actually holds [7].

Computing NTKs in practice

Sanjeev Arora, Simon Du, Wei Hu, Zhiyuan Li, Ruslan Salakhutdinov, and Ruosong Wang gave the first efficient exact algorithm for the convolutional NTK, with a GPU implementation, and used it to ask what a classic convolutional neural network computes when its channel count goes to infinity [8]. Their CIFAR-10 table is the standard reference point for how far the kernel limit gets on a real vision task:

DepthCNN-VCNTK-VCNN-GAPCNTK-GAP
359.97%64.47%63.81%70.47%
460.20%65.52%80.93%75.93%
664.11%66.03%83.75%76.73%
1169.48%65.90%82.92%77.43%
2175.57%64.09%83.30%77.08%

Source: Arora et al., Table 1 [8]. "V" is a vanilla network, "GAP" adds global average pooling, and depth counts every layer, convolutional and fully connected.

The best kernel result, 77.43%, is described by the authors as 10% higher than previously reported kernel methods and 6% lower than the corresponding finite architecture with batch normalization and similar tricks turned off. The same paper supplied the first non-asymptotic proof that a sufficiently wide trained network matches NTK kernel regression [8].

Kernel methods scale worse than quadratically in the number of training examples, so the natural niche is small data. A follow-up by Arora, Du, Li, Salakhutdinov, Wang, and Dingli Yu reported that an NTK support vector machine beat random forests and the corresponding finite networks on a standard UCI classification and regression testbed, that the convolutional NTK beat ResNet-34 by 1% to 3% on CIFAR-10 with only 10 to 640 training examples, and that swapping a linear SVM for a convolutional NTK SVM improved few-shot ImageNet transfer results on the VOC07 testbed [9].

Google's Neural Tangents library, built on JAX, made both kernels available through a high-level API in which one architecture specification yields a finite network and its infinite-width NNGP and NTK kernels, runnable on CPU, GPU, or TPU [10][11]. The repository is now archived and read-only; its last release, v0.6.5, dates from December 2023 [11].

What the theory accounts for

The most direct payoff was optimization. Simon Du, Xiyu Zhai, Barnabas Poczos, and Aarti Singh proved that randomly initialized gradient descent drives a sufficiently wide two-layer ReLU network to zero training loss at a linear rate, because overparametrization keeps every weight vector near its initialization [12]. Zeyuan Allen-Zhu, Yuanzhi Li, and Zhao Song extended this to deep fully connected networks, convolutional networks, and residual networks with width polynomial in depth and sample count, framing the result as an equivalence between overparametrized networks and the NTK at finite polynomial width [13].

The kernel spectrum also predicts the order in which a network learns. Blake Bordelon, Abdulkadir Canatar, and Cengiz Pehlevan derived analytic learning curves for kernel regression and found that as the training set grows, kernel machines and wide networks fit successively higher spectral modes of the target, which for dot-product kernels including the NTK means learning frequency band by frequency band [14]. Alberto Bietti and Julien Mairal characterized the reproducing kernel Hilbert space of the NTK, including smoothness, approximation, and stability to image deformations for convolutional architectures [15].

That spectral picture produced one of the most widely used practical applications of NTK theory. Matthew Tancik and coauthors showed that a plain multilayer perceptron cannot fit high-frequency detail in low-dimensional input domains, used NTK tools to explain why, and fixed it by passing coordinates through a Fourier feature mapping that turns the effective NTK into a stationary kernel with tunable bandwidth; the sinusoidal positional encoding used in NeRF and related coordinate networks is a special case of that mapping [16].

The analysis carried over to physics-informed neural networks as well. Sifan Wang, Xinling Yu, and Paris Perdikaris derived the NTK for PINNs, proved that under appropriate conditions it converges to a deterministic kernel that stays constant during training in the infinite-width limit, traced PINN training failures to a discrepancy in the convergence rates of the different loss components, and proposed a gradient descent algorithm that uses the kernel eigenvalues to recalibrate those rates adaptively [17].

The name also travels farther than the theory does. The "NTK-aware" interpolation used to extend the context window of rotary position embedding models was proposed under the handle bloc97 in a 2023 online post, and motivated by the Fourier-features result above on the grounds that RoPE can be defined as a one-dimensional special case of a Fourier feature. The YaRN paper describes the method and its provenance [18]. The link back to Jacot's convergence theorem is indirect, running through that Fourier-features analysis rather than through the infinite-width limit itself.

Limitations

The NTK limit removes feature learning by construction. Greg Yang and Edward Hu proved this as a dichotomy: within a natural family of parametrizations that includes the standard, NTK, and mean-field choices, an infinite-width limit either learns features or has training dynamics given by kernel gradient descent, and never both [19]. Their alternative, the maximal update parametrization, does admit feature learning in the limit and outperformed NTK baselines on Word2Vec and few-shot Omniglot [19]. It also turned into a production tool: muP underpins muTransfer, where optimal hyperparameter settings are tuned on a small proxy model and transferred zero-shot to a large one. Yang, Hu, and coauthors reported outperforming published BERT-large numbers by transferring from a 13M-parameter model, and outperforming published GPT-3 6.7B numbers by transferring from a 40M-parameter model at 7% of total pretraining cost [20].

Empirically, the kernel of a real network does not sit still. Stanislav Fort, Gintare Karolina Dziugaite, Mansheej Paul, Sepideh Kharaghani, Daniel Roy, and Surya Ganguli measured the data-dependent NTK across architectures and datasets and found a chaotic transient in the first two to three epochs during which the kernel changes rapidly and picks up useful features, outperforming the initial NTK by a factor of three within three to four epochs; only afterward does it settle into steady drift [21]. Boris Hanin and Mihai Nica showed that at finite depth and width the NTK's standard deviation grows exponentially in the depth-to-width ratio, so a deep network taken to infinite width along a fixed depth-to-width ratio has a kernel that stays random and evolves during training [22].

There are also provable separations. Colin Wei, Jason Lee, Qiang Liu, and Tengyu Ma constructed a distribution in d dimensions that the optimal L2-regularized network learns from O(d) samples but that requires Omega(d^2) samples for the NTK [23]. Zeyuan Allen-Zhu and Yuanzhi Li proved that three-layer residual networks can learn a class of functions with far lower test error than any kernel method, including the NTK, at equal sample size, attributing the gap to hierarchical learning across layers [24]. Behrooz Ghorbani, Song Mei, Theodor Misiakiewicz, and Andrea Montanari argued that when covariates are close to isotropic, kernel methods suffer the curse of dimensionality while networks escape it by finding a low-dimensional representation, and showed that perturbing the training distribution degrades kernels much more than networks [25].

The theory is also narrower than its usual statement. Zixiong Yu, Songtao Tian, and Guhan Chen proved that with cross-entropy loss and unbounded training time, the empirical NTK of fully connected and residual networks diverges from the limiting NTK on the training samples at any finite width, so the lazy-training picture does not carry over from regression to classification as usually posed [26]. A 2026 preprint by Jonathan Plenk, Sergio Calvo-Ordonez, Alvaro Cartea, Yarin Gal, Mark van der Wilk, and Kamil Ciosek identifies conditions that restore it: parameter-space regularization keeps the NTK constant under cross-entropy, and without regularization the lazy regime returns when the targets are non-degenerate, meaning every class carries strictly positive probability [27]. Arkaprabha Ganguli and Emil Constantinescu quantified a separate failure on compositional targets, showing that NTK regression needs a number of samples exponential in depth for iterated sawtooth functions whose minimax rate is polynomial, and reporting four to six orders of magnitude in test error between a two-layer network and the NTK on sparse parity [28].

Haobo Zhang, Jianfa Lai, Yicheng Li, Qian Lin, and Jun Liu summarize the position in a review that treats the fixed-kernel assumption as the boundary of what NTK theory can explain and proposes adaptive feature models as the next step [29].

Work since 2025

Two strands dominate current work: sharpening the theory at finite width, and using the empirical NTK as a measurement instrument on real trained models.

On the theory side, Max Guillen, Philipp Misof, and Jan Gerken introduced Feynman diagram methods for computing finite-width corrections to NTK statistics, deriving layerwise recursions for the higher-derivative tensors needed to predict training dynamics at leading order and matching sampled networks at widths above roughly 20 [30]. William St-Arnaud, Margarida Carvalho, and Golnoosh Farnadi revisited the large-width and large-depth regime and argued that although individual neuron deviations vanish, their aggregate norm does not, so some conclusions about NTK breakdown at depth need revision [31]. Yuqing Wang set out a convergence framework for gradient descent on general architectures, including pre-normalized multi-layer transformers, explicitly outside the NTK regime [32].

On the measurement side, Kaiqi Jiang, Jeremy Cohen, and Yuanzhi Li studied NTK eigenvectors during the edge-of-stability phenomenon and found that larger learning rates make the leading eigenvectors of the final kernel align more strongly with the training target [33]. Konstantin Nikolaou and coauthors used the empirical NTK to define a "spectral position" measure of which eigenmodes are currently driving loss reduction, and reported that learning moves into the spectral tail over training, that larger models reach further into that tail, and that feature learning is what sustains the reach; they offer this as a mechanism behind scaling laws [34]. The empirical NTK has also become a practical tool for uncertainty quantification: Joseph Wilson, Chris van der Heide, Liam Hodgkinson, and Fred Roosta build post-hoc deep ensembles by running gradient-descent sampling on a linearized trained network, which approximates the posterior of a Gaussian process under the empirical NTK at a fraction of the cost of competing Bayesian methods [35].

See also

References

  1. ^Jacot, A., Gabriel, F., Hongler, C. "Neural Tangent Kernel: Convergence and Generalization in Neural Networks." arXiv:1806.07572, 20 June 2018; Advances in Neural Information Processing Systems 31 (NeurIPS 2018). arxiv.org/...1806.07572
  2. ^Neal, R. M. "Priors for Infinite Networks." Technical Report CRG-TR-94-1, Department of Computer Science, University of Toronto, March 1994. glizen.com/...pin.abstract
  3. ^Lee, J., Bahri, Y., Novak, R., Schoenholz, S. S., Pennington, J., Sohl-Dickstein, J. "Deep Neural Networks as Gaussian Processes." arXiv:1711.00165; ICLR 2018. arxiv.org/...1711.00165
  4. ^Matthews, A. G. de G., Rowland, M., Hron, J., Turner, R. E., Ghahramani, Z. "Gaussian Process Behaviour in Wide Deep Neural Networks." arXiv:1804.11271, 30 April 2018. arxiv.org/...1804.11271
  5. ^Chizat, L., Oyallon, E., Bach, F. "On Lazy Training in Differentiable Programming." arXiv:1812.07956, 19 December 2018; NeurIPS 2019. arxiv.org/...1812.07956
  6. ^Lee, J., Xiao, L., Schoenholz, S. S., Bahri, Y., Novak, R., Sohl-Dickstein, J., Pennington, J. "Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent." arXiv:1902.06720, 18 February 2019; NeurIPS 2019. arxiv.org/...1902.06720
  7. ^Yang, G. "Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation." arXiv:1902.04760, 13 February 2019. arxiv.org/...1902.04760
  8. ^Arora, S., Du, S. S., Hu, W., Li, Z., Salakhutdinov, R., Wang, R. "On Exact Computation with an Infinitely Wide Neural Net." arXiv:1904.11955, 26 April 2019; NeurIPS 2019 (CIFAR-10 results, Table 1). arxiv.org/...1904.11955
  9. ^Arora, S., Du, S. S., Li, Z., Salakhutdinov, R., Wang, R., Yu, D. "Harnessing the Power of Infinitely Wide Deep Nets on Small-data Tasks." arXiv:1910.01663, 3 October 2019. arxiv.org/...1910.01663
  10. ^Novak, R., Xiao, L., Hron, J., Lee, J., Alemi, A. A., Sohl-Dickstein, J., Schoenholz, S. S. "Neural Tangents: Fast and Easy Infinite Neural Networks in Python." arXiv:1912.02803, 5 December 2019. arxiv.org/...1912.02803
  11. ^google/neural-tangents repository (archived 6 May 2025; latest release v0.6.5, 11 December 2023). github.com/...neural-tangents
  12. ^Du, S. S., Zhai, X., Poczos, B., Singh, A. "Gradient Descent Provably Optimizes Over-parameterized Neural Networks." arXiv:1810.02054, 4 October 2018; ICLR 2019. arxiv.org/...1810.02054
  13. ^Allen-Zhu, Z., Li, Y., Song, Z. "A Convergence Theory for Deep Learning via Over-Parameterization." arXiv:1811.03962, 9 November 2018. arxiv.org/...1811.03962
  14. ^Bordelon, B., Canatar, A., Pehlevan, C. "Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networks." arXiv:2002.02561, 7 February 2020; ICML 2020, PMLR 119:1024-1034. arxiv.org/...2002.02561
  15. ^Bietti, A., Mairal, J. "On the Inductive Bias of Neural Tangent Kernels." arXiv:1905.12173, 29 May 2019; NeurIPS 2019. arxiv.org/...1905.12173
  16. ^Tancik, M., Srinivasan, P. P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., Ramamoorthi, R., Barron, J. T., Ng, R. "Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains." arXiv:2006.10739, 18 June 2020. arxiv.org/...2006.10739
  17. ^Wang, S., Yu, X., Perdikaris, P. "When and why PINNs fail to train: A neural tangent kernel perspective." arXiv:2007.14527, 28 July 2020. arxiv.org/...2007.14527
  18. ^Peng, B., Quesnelle, J., Fan, H., Shippole, E. "YaRN: Efficient Context Window Extension of Large Language Models." arXiv:2309.00071, 31 August 2023 (describes "NTK-aware" interpolation and its origin). arxiv.org/...2309.00071
  19. ^Yang, G., Hu, E. J. "Tensor Programs IV: Feature Learning in Infinite-Width Neural Networks." arXiv:2011.14522, 30 November 2020; ICML 2021. arxiv.org/...2011.14522
  20. ^Yang, G., Hu, E. J., Babuschkin, I., Sidor, S., Liu, X., Farhi, D., Ryder, N., Pachocki, J., Chen, W., Gao, J. "Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer." arXiv:2203.03466, 7 March 2022; NeurIPS 2021. arxiv.org/...2203.03466
  21. ^Fort, S., Dziugaite, G. K., Paul, M., Kharaghani, S., Roy, D. M., Ganguli, S. "Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel." arXiv:2010.15110, 28 October 2020; NeurIPS 2020. arxiv.org/...2010.15110
  22. ^Hanin, B., Nica, M. "Finite Depth and Width Corrections to the Neural Tangent Kernel." arXiv:1909.05989, 13 September 2019. arxiv.org/...1909.05989
  23. ^Wei, C., Lee, J. D., Liu, Q., Ma, T. "Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel." arXiv:1810.05369, 12 October 2018. arxiv.org/...1810.05369
  24. ^Allen-Zhu, Z., Li, Y. "What Can ResNet Learn Efficiently, Going Beyond Kernels?" arXiv:1905.10337, 24 May 2019. arxiv.org/...1905.10337
  25. ^Ghorbani, B., Mei, S., Misiakiewicz, T., Montanari, A. "When Do Neural Networks Outperform Kernel Methods?" arXiv:2006.13409, 24 June 2020. arxiv.org/...2006.13409
  26. ^Yu, Z., Tian, S., Chen, G. "Divergence of Empirical Neural Tangent Kernel in Classification Problems." arXiv:2504.11130, 15 April 2025; ICLR 2025. arxiv.org/...2504.11130
  27. ^Plenk, J., Calvo-Ordonez, S., Cartea, A., Gal, Y., van der Wilk, M., Ciosek, K. "The Neural Tangent Kernel for Classification." arXiv:2605.17606, 17 May 2026. arxiv.org/...2605.17606
  28. ^Ganguli, A., Constantinescu, E. "A Function-Space Dichotomy for Compositional Learning: Exponential Sub-Optimality of the Neural Tangent Kernel." arXiv:2607.06382, 7 July 2026. arxiv.org/...2607.06382
  29. ^Zhang, H., Lai, J., Li, Y., Lin, Q., Liu, J. S. "Towards a Statistical Understanding of Neural Networks: Beyond the Neural Tangent Kernel Theories." arXiv:2412.18756, 25 December 2024. arxiv.org/...2412.18756
  30. ^Guillen, M., Misof, P., Gerken, J. E. "Finite-Width Neural Tangent Kernels from Feynman Diagrams." arXiv:2508.11522, 15 August 2025. arxiv.org/...2508.11522
  31. ^St-Arnaud, W., Carvalho, M., Farnadi, G. "Revisiting the Neural Tangent Kernel: the role of large width and depth." arXiv:2511.07272, 10 November 2025. arxiv.org/...2511.07272
  32. ^Wang, Y. "Convergence of Gradient Descent for General Neural Network Architectures Beyond the NTK Regime." arXiv:2606.23364, 22 June 2026. arxiv.org/...2606.23364
  33. ^Jiang, K., Cohen, J., Li, Y. "Understanding the Evolution of the Neural Tangent Kernel at the Edge of Stability." arXiv:2507.12837, 17 July 2025. arxiv.org/...2507.12837
  34. ^Nikolaou, K., Scheunemann, J., Krippendorf, S., Tovey, S., Holm, C. "Spectral Reach: Understanding Neural Scaling as Progress into the Spectral Tail." arXiv:2605.31244, 29 May 2026. arxiv.org/...2605.31244
  35. ^Wilson, J., van der Heide, C., Hodgkinson, L., Roosta, F. "Uncertainty Quantification with the Empirical Neural Tangent Kernel." arXiv:2502.02870, 5 February 2025. arxiv.org/...2502.02870

Improve this article

Add missing citations, update stale details, or suggest a clearer explanation. Every suggestion is reviewed for sourcing before it goes live.

v1 · 4,023 words · full history

Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify

Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here

Reviewer note: Independent adversarial fact-check at creation (wanted175 campaign, 2026-07-24): every claim verified against primary sources by a dedicated verification agent; corrections applied before publication.

Cite this page: AI Wiki. "Neural Tangent Kernel." aiwiki.ai, updated 24 Jul 2026, fact-checked 24 Jul 2026. CC BY 4.0. https://aiwiki.ai/wiki/neural_tangent_kernel

Suggest edit