# Hopfield Network

> Source: https://aiwiki.ai/wiki/hopfield_network
> Updated: 2026-07-24
> Fact-checked: 2026-07-24
> Categories: AI History, Deep Learning, Machine Learning, Neural Networks
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "Hopfield Network." aiwiki.ai, 24 Jul 2026. https://aiwiki.ai/wiki/hopfield_network
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

A **Hopfield network** is a form of recurrent [neural network](https://aiwiki.ai/wiki/neural_network) that stores information as stable patterns in a system of interconnected binary units and retrieves it by content rather than by address. Introduced by the physicist [John Hopfield](https://aiwiki.ai/wiki/john_hopfield) in a 1982 paper in the Proceedings of the National Academy of Sciences, the model showed that a large collection of simple two-state "neurons" wired together with symmetric connections can behave as an associative, content-addressable memory: given a corrupted or partial version of a stored pattern, the network settles into the complete stored pattern nearby.[1] The retrieval process is driven downhill along an energy function, so that stored memories sit at the bottoms of energy valleys and recall is the physical relaxation of the system toward one of those minima.[1]

The paper was influential well beyond its original biological framing. By casting memory recall as energy minimization, Hopfield connected neural computation to the statistical mechanics of magnetic systems, and the model became a bridge between physics and what would later be called [machine learning](https://aiwiki.ai/wiki/machine_learning).[1] Its ideas fed directly into the [Boltzmann machine](https://aiwiki.ai/wiki/boltzmann_machine),[11] the study of energy-based models, and, decades later, a line of "modern" Hopfield networks whose update rule turns out to be closely related to the attention mechanism used in transformer models.[6][7] In 2024 Hopfield shared the Nobel Prize in Physics with [Geoffrey Hinton](https://aiwiki.ai/wiki/geoffrey_hinton) for this body of work.[8]

## Background and precursors

The idea of storing memories in the connection strengths of a network of neurons predates Hopfield. Donald Hebb's 1949 proposal that repeated co-activation strengthens the link between two cells, often summarized later as "neurons that fire together wire together," gave a learning rule for writing memories into connection strengths, and this Hebbian principle is the basis for how patterns are written into a Hopfield network.[1][10] In the early 1970s, Shun-Ichi Amari and, independently, W. A. Little studied associative networks built from Ising-like binary units whose couplings are set by correlations between stored states, and Teuvo Kohonen and James Anderson developed linear associative memories in the same period.[1][10] Hopfield cited these lines of work and pointed out where his model departed from them: earlier associative nets were largely linear and had to be paired with an external nonlinear processor to make decisions, while his network used its own strong nonlinearity and feedback to regenerate a full pattern from a partial cue.[1]

Hopfield also distinguished his model from the [perceptron](https://aiwiki.ai/wiki/perceptron) tradition. Perceptrons were usually studied with feed-forward connections and synchronous, clocked updates, and the analysis of networks with strong backward coupling had proved intractable.[1] The interesting collective behavior of a Hopfield network, by contrast, arises precisely from that symmetric back-coupling, and from updating neurons randomly and asynchronously rather than in lockstep, which he argued was more plausible for real nervous systems that have no global clock.[1]

## The 1982 model

In the original network each neuron i has two states, written V_i = 0 ("not firing") and V_i = 1 ("firing at maximum rate"), in the style of the McCulloch-Pitts model.[1] Every ordered pair of neurons is joined by a connection strength T_ij, with no self-connections (T_ii = 0). The instantaneous state of the whole system is a binary word of N bits.[1]

Neurons update asynchronously. Each neuron i has a fixed threshold U_i and, at random times with a mean attempt rate, resets its state according to whether the weighted sum of its inputs exceeds that threshold: V_i becomes 1 if the sum over j of T_ij V_j is greater than U_i, and 0 if it is less.[1] Because updates happen one neuron at a time and at random, there is no global synchrony, which Hopfield treated as a feature rather than a limitation.[1]

Patterns are written into the weights with a Hebbian storage prescription. To store a set of states V^s (indexed by s), the connection between neurons i and j is set to the sum over the stored patterns of the product of their bipolar values, T_ij = sum_s (2V_i^s - 1)(2V_j^s - 1), again with the diagonal removed.[1] Each stored pattern reinforces the couplings between units that share the same sign in that pattern, so that the memory is distributed across all the weights rather than kept in any single location. Recall then works by content: presenting a partial or noisy pattern and letting the asynchronous dynamics run drives the network into the complete stored state it most resembles.[1]

## Energy landscape and the physics connection

The key theoretical result is that when the weight matrix is symmetric (T_ij = T_ji), the dynamics can be described by an energy function,

E = -(1/2) sum over i not equal to j of T_ij V_i V_j,

and every allowed state change lowers E or leaves it unchanged.[1] Because each asynchronous update can only decrease the energy, the network cannot cycle forever; it descends monotonically until it reaches a local minimum and stops.[1] The stored patterns are engineered, through the Hebbian rule, to sit at or near those minima, so recall is the relaxation of the system to the bottom of an energy valley. A corrupted input starts partway up a valley wall and rolls down to the memory at its floor.[1]

Hopfield noted explicitly that this construction "is isomorphic with an Ising model," the canonical physics model of interacting magnetic spins.[1] In that mapping the connection T_ij plays the role of the exchange coupling between spins and the threshold acts like a local magnetic field, while the symmetric but otherwise random and often frustrated couplings produced by storing many patterns give the system the character of a spin glass, which is known to have many local energy minima.[1] This correspondence is what let physicists bring the tools of statistical mechanics to bear on neural memory, and it is a large part of why the model resonated in the physics community.

## Storage capacity

A Hopfield network cannot store an unlimited number of patterns. As more memories are packed into the same weights, their energy valleys start to interfere: spurious minima appear, and stored states drift or merge. In the original paper, Hopfield reported from simulations with N = 30 and N = 100 neurons that "about 0.15 N states can be simultaneously remembered before error in recall is severe."[1] For small loads the memories were essentially perfect (with N = 100 and 5 patterns, states were almost always stable and exactly recallable), while at 15 patterns roughly half the nominal memories had evolved to states with fewer than 5 errors and the rest had drifted substantially.[1]

Later work sharpened this estimate using the spin-glass analysis the model invited. Daniel Amit, Hanoch Gutfreund, and Haim Sompolinsky applied statistical-mechanics methods and found a critical loading of roughly 0.138 patterns per neuron, above which the memory states abruptly lose stability; this figure of about 0.138N is the standard quoted capacity for the Hebbian Hopfield network with a small tolerance for errors.[9] If instead one demands retrieval that is essentially free of errors, the capacity scales more slowly, roughly as N divided by twice the logarithm of N.[7] Either way the practical lesson is the same: a classical Hopfield network stores a number of patterns that grows only linearly (or sub-linearly) with the number of neurons, which is modest compared with the size of the weight matrix.[7][9]

## Continuous neurons and optimization

Two years after the binary model, Hopfield published a continuous version in which neurons have a graded, sigmoid input-output response instead of a hard threshold, and showed that "neurons with a graded response have collective computational properties like those of two-state neurons."[2] This 1984 formulation could be built directly as an analog electrical circuit of amplifiers, resistors, and capacitors, with an energy function whose minima correspond to those of the discrete model, which made the idea concrete for hardware.[2]

The continuous network also suggested a way to attack optimization problems. In 1985 Hopfield and David Tank showed that a suitably wired network of nonlinear analog neurons could compute good solutions to hard combinatorial problems, using the classical traveling-salesman problem as their example, by encoding the cost of a candidate solution as the network's energy so that settling to a low-energy state corresponds to finding a short tour.[3] The approach computed solutions "within an elapsed time of only a few neural time constants," and although it was later found to be sensitive to parameter tuning and easily trapped in poor local minima, it opened a durable line of research into neural and analog approaches to optimization.[3] The connection to physics ran the other way too: adding controlled noise to escape shallow minima links the Hopfield picture to [simulated annealing](https://aiwiki.ai/wiki/simulated_annealing) and to the stochastic Boltzmann machine.

## Limitations

The classical Hopfield network has well-known weaknesses. Its capacity is small relative to the number of connections, and pushing past the roughly 0.138N limit causes the memory to break down rather than degrade gracefully.[9] Correlated patterns, which overlap heavily, interfere much more strongly than random ones, so real-world data that is not close to random is stored poorly.[1] The energy landscape also contains spurious minima, stable states that are not any of the intended memories but mixtures or reversals of them, and the network can settle into these instead of a genuine pattern.[1] Finally, the basic model handles binary vectors of fixed length and is best suited to auto-association (completing a pattern from part of itself) rather than the richer learning tasks that feed-forward networks trained by [backpropagation](https://aiwiki.ai/wiki/backpropagation) came to dominate. For much of the 1990s and 2000s these limits kept Hopfield networks more prominent in physics and neuroscience than in mainstream applied machine learning.

## Modern Hopfield networks

Interest revived when researchers found that the storage limit is not fundamental but a consequence of the specific quadratic energy function. In 2016 Dmitry Krotov and John Hopfield introduced "dense associative memories," replacing the pairwise energy with higher-order polynomial interactions among the units; sharper interaction functions carve deeper, narrower energy wells that can pack in far more patterns, and the same construction connects associative memory to feed-forward networks with unusual activation functions.[4] In 2017 Mete Demircigil and collaborators pushed this to an exponential interaction function and proved that the resulting model has a storage capacity that grows exponentially with the number of neurons, a dramatic jump from the linear classical case.[5]

The best-known modern version came in 2020, when Hubert Ramsauer, Sepp Hochreiter, and colleagues published "Hopfield Networks is All You Need."[6] They generalized the energy to continuous-valued patterns using a log-sum-exp energy function, giving a network that can store a number of continuous patterns that is exponential in the dimension of the associative space and that typically retrieves any of them in a single update step with exponentially small error.[6][7] The retrieval update in this model is a softmax over the stored patterns, and the authors showed that this update rule is mathematically equivalent to the [attention](https://aiwiki.ai/wiki/attention) mechanism at the heart of the transformer architecture introduced in [Attention Is All You Need](https://aiwiki.ai/wiki/attention_is_all_you_need).[6][7] In that mapping, the query, keys, and values of an attention layer correspond to the state and the stored patterns of a modern Hopfield network, which recasts a transformer's attention as associative retrieval and gives a new theoretical lens on why attention works.[7] The authors packaged the result as reusable "Hopfield layers" and applied them to tasks such as immune repertoire classification, multiple-instance learning benchmarks, and drug-design datasets.[6][7]

## The 2024 Nobel Prize in Physics

On 8 October 2024 the Royal Swedish Academy of Sciences awarded the Nobel Prize in Physics jointly to John J. Hopfield and Geoffrey E. Hinton "for foundational discoveries and inventions that enable machine learning with artificial neural networks."[8] The committee highlighted the Hopfield network as a memory that stores and reconstructs patterns from partial or distorted input using physics that describes materials in terms of atomic spins, and Hinton's Boltzmann machine as a network built on the Hopfield foundation, using tools from statistical physics, that can learn to recognize characteristic elements in data and generate new examples of the patterns it was trained on.[8][11] Hopfield, by then the Howard A. Prior Professor in the Life Sciences, Emeritus, at Princeton University, had spent a career moving between physics, chemistry, and biology: he earned his PhD from Cornell in 1958, worked at Bell Laboratories, joined Princeton in 1964, moved to Caltech in 1980, where he led its pioneering Computation and Neural Systems program, and returned to Princeton in 1997.[8] Hinton, at the University of Toronto, was recognized for the Boltzmann machine and the later work he built on it, which the committee credited with helping to initiate the current explosive development of machine learning.[11]

## See also

- [John Hopfield](https://aiwiki.ai/wiki/john_hopfield)
- [Boltzmann machine](https://aiwiki.ai/wiki/boltzmann_machine)
- [Attention](https://aiwiki.ai/wiki/attention)
- [Recurrent neural network](https://aiwiki.ai/wiki/recurrent_neural_network)
- [Neural network](https://aiwiki.ai/wiki/neural_network)
- [AI Nobel Prize](https://aiwiki.ai/wiki/ai_nobel_prize)

## References

1. Hopfield, J. J. "Neural networks and physical systems with emergent collective computational abilities." Proceedings of the National Academy of Sciences, Vol. 79, No. 8, pp. 2554-2558 (15 April 1982). https://www.pnas.org/doi/10.1073/pnas.79.8.2554
2. Hopfield, J. J. "Neurons with graded response have collective computational properties like those of two-state neurons." Proceedings of the National Academy of Sciences, Vol. 81, No. 10, pp. 3088-3092 (May 1984). https://pmc.ncbi.nlm.nih.gov/articles/PMC345226/
3. Hopfield, J. J. and Tank, D. W. "'Neural' computation of decisions in optimization problems." Biological Cybernetics, Vol. 52, pp. 141-152 (1985). https://link.springer.com/article/10.1007/BF00339943
4. Krotov, D. and Hopfield, J. J. "Dense Associative Memory for Pattern Recognition." Advances in Neural Information Processing Systems 29 (NIPS 2016); arXiv:1606.01164. https://arxiv.org/abs/1606.01164
5. Demircigil, M., Heusel, J., Löwe, M., Upgang, S., and Vermet, F. "On a model of associative memory with huge storage capacity." Journal of Statistical Physics, Vol. 168, No. 2, pp. 288-299 (2017); arXiv:1702.01929. https://arxiv.org/abs/1702.01929
6. Ramsauer, H., Schäfl, B., Lehner, J., et al. (Hochreiter, S., senior author). "Hopfield Networks is All You Need." arXiv:2008.02217 (submitted 16 July 2020). https://arxiv.org/abs/2008.02217
7. Brandstetter, J. "Hopfield Networks is All You Need" (project blog, Institute for Machine Learning, JKU Linz, 2020). https://ml-jku.github.io/hopfield-layers/
8. Fuller-Wright, L. "Princeton's John Hopfield receives Nobel Prize in physics." Princeton University, 8 October 2024. https://www.princeton.edu/news/2024/10/08/princetons-john-hopfield-receives-nobel-prize-physics
9. Amit, D. J., Gutfreund, H., and Sompolinsky, H. "Storing infinite numbers of patterns in a spin-glass model of neural networks." Physical Review Letters, Vol. 55, No. 14, p. 1530 (1985). https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.55.1530
10. "Hopfield network." Wikipedia. https://en.wikipedia.org/wiki/Hopfield_network
11. "Press release: The Nobel Prize in Physics 2024." The Royal Swedish Academy of Sciences, 8 October 2024. https://www.nobelprize.org/prizes/physics/2024/press-release/