Connectionism

Connectionism is an approach in cognitive science and artificial intelligence that explains mental phenomena as the collective behavior of large networks of simple, interconnected processing units. Knowledge in a connectionist system is not written down as symbols or rules. It is held in the strengths of the connections between units, and it is acquired by changing those strengths in response to experience. The Internet Encyclopedia of Philosophy describes the field as the study of cognition using "mathematical models, known as connectionist networks or artificial neural networks" built from "highly interconnected, neuron-like processing units" ^[36]. The Stanford Encyclopedia of Philosophy calls it "a movement in cognitive science that hopes to explain intellectual abilities using artificial neural networks" ^[37].

The approach has always defined itself against symbolic AI, which treats thinking as rule-governed manipulation of structured symbolic expressions. Jerry Fodor and Zenon Pylyshyn stated the contrast precisely in 1988: both camps "postulate representational mental states", but only the classical one is "committed to a symbol-level of representation, or to a 'language of thought'" ^[16][17]. Connectionists, on their account, "propose to design systems that can exhibit intelligent behavior without storing, retrieving, or otherwise operating on structured symbolic expressions" ^[17].

Interest in the approach has risen and fallen repeatedly. It started with the formal neuron of the 1940s and the perceptron of the late 1950s, collapsed after 1969, returned in the mid-1980s under the name parallel distributed processing, and became what ACM, announcing the 2018 Turing Award, called "the dominant paradigm in the field" ^[29]. The 2024 Nobel Prize in Physics went to John Hopfield and Geoffrey Hinton "for foundational discoveries and inventions that enable machine learning with artificial neural networks" ^[30]. Whether that also settles the philosophical arguments of the 1980s is a separate question, and it was still being fought over in print in 2026 ^[32][33][34].

Associationist roots and the name

The idea that mental life is built out of associations between simple elements predates any network model. Frank Rosenblatt located his own work in that tradition explicitly. His 1958 paper in Psychological Review sets out two rival answers to the question of how sensory information is stored: one holds that storage takes "the form of coded representations or images, with some sort of one-to-one mapping between the sensory stimulus and the stored pattern", while the other, "which stems from the tradition of British empiricism", holds that "the information is contained in connections or associations rather than topographic representations". Rosenblatt then writes that his theory "takes the empiricist, or 'connectionist' position with regard to these questions" ^[3].

So the word was already in use in 1958. It returned as the name of a research program with Jerome Feldman and Dana Ballard's 1982 Cognitive Science paper "Connectionist Models and Their Properties", which argued that cognitive science had relied on information processing models "patterned after conventional serial computers" and set out to introduce "a general connectionist model" for the field instead ^[7]. Fodor and Pylyshyn were still classifying connectionists as associationists in 1988 and traced the lineage back to Donald Hebb, Charles Osgood and Clark Hull ^[17].

A separate and older use of the same word survives in educational and behavioral psychology, where "connectionism" names Edward Thorndike's stimulus-response theory of learning. It is still taught under that heading in teacher-education texts ^[38], and it is not the subject of this article.

Logical neurons, perceptrons and the first collapse

Warren McCulloch and Walter Pitts opened the technical history in 1943 with "A logical calculus of the ideas immanent in nervous activity", published in the Bulletin of Mathematical Biophysics. Their claim was that "because of the 'all-or-none' character of nervous activity, neural events and the relations among them can be treated by means of propositional logic", and that "for any logical expression satisfying certain conditions, one can find a net behaving in the fashion it describes" ^[1]. The neuron in that paper is a threshold device, not a biological cell, and the result is about what networks of such devices can compute rather than about how brains learn.

Learning arrived with Donald Hebb's 1949 book The Organization of Behavior, which proposed that a connection strengthens when the cells it joins are active together ^[2]. That principle, later compressed into the slogan that neurons which fire together wire together, gave networks a way to set their own weights instead of having them wired by hand ^[36].

Rosenblatt combined the two. Working at the Cornell Aeronautical Laboratory under Office of Naval Research contract Nonr-2381(00), he described the perceptron as "a hypothetical nervous system, or machine" designed "to illustrate some of the fundamental properties of intelligent systems in general" ^[3]. In July 1958 the Navy demonstrated a perceptron running on an IBM 704, and after fifty trials, according to Cornell's own account, "the computer taught itself to distinguish cards marked on the left from cards marked on the right". The New York Times covered it under the headline "NEW NAVY DEVICE LEARNS BY DOING: Psychologist Shows Embryo of Computer Designed to Read and Grow Wiser" ^[4]. Rosenblatt also proved a perceptron convergence theorem, which guarantees that the learning rule finds a solution when one exists ^[36].

Scroll sideways for more →

Marvin Minsky and Seymour Papert's Perceptrons: An Introduction to Computational Geometry, published by MIT Press in 1969, established what a simple perceptron cannot do. The standard illustration is exclusive-or, a function no single-layer network can compute ^[5][36]. Funding and interest fell away afterwards, and Cornell's retrospective states flatly that the book "assailed Rosenblatt's work and, essentially, sealed its fate" ^[4]. That narrative has itself been examined: Mikel Olazaran's 1996 paper in Social Studies of Science is a sociological study of the official history of the perceptrons controversy, treating the standard story as an object of analysis rather than a settled account ^[6]. The stretch between 1969 and the mid-1980s overlaps the wider AI winter chronology. Rosenblatt did not outlive it: in 1971 he drowned while sailing in Chesapeake Bay on his 43rd birthday ^[4].

The parallel distributed processing revival

Feldman and Ballard's 1982 paper restarted the field under a name that stuck ^[7]. One of its arguments was widely reused: computer instructions execute in nanoseconds while neurons take tens of milliseconds to fire, so in the time a person needs to recognize a word or a picture, a serial program running on neural hardware could execute only about a hundred steps. Whatever the brain is doing, it cannot be a long serial program ^[17].

Two physics-flavored models followed. John Hopfield's 1982 paper in the Proceedings of the National Academy of Sciences described a recurrent network whose dynamics minimize an energy function, so that stored patterns sit at energy minima and a corrupted input relaxes toward the nearest stored one ^[8]. The Royal Swedish Academy summarized the mechanism in 2024: the Hopfield network "utilises physics that describes a material's characteristics due to its atomic spin", and "when the Hopfield network is fed a distorted or incomplete image, it methodically works through the nodes and updates their values so the network's energy falls" ^[30]. David Ackley, Geoffrey Hinton and Terrence Sejnowski then built the Boltzmann machine on that foundation in 1985, adding stochastic units and a learning algorithm drawn from statistical physics ^[9][30].

The movement's central publication was the two-volume Parallel Distributed Processing: Explorations in the Microstructure of Cognition, issued by MIT Press in 1986. Volume 1, Foundations, was edited by David Rumelhart and James McClelland; volume 2, Psychological and Biological Models, by McClelland and Rumelhart ^[11][12]. The volumes collected working models of perception, memory and language rather than a manifesto, and they built on earlier work such as McClelland and Rumelhart's 1981 interactive activation model of context effects in letter perception ^[10].

The learning algorithm that made deep networks trainable appeared the same year. Rumelhart, Hinton and Ronald Williams published "Learning representations by back-propagating errors" in Nature on 9 October 1986. Their abstract describes backpropagation as a procedure that "repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector", with the consequence that "internal 'hidden' units which are not part of the input or output come to represent important features of the task domain". They singled out one property as decisive: "The ability to create useful new features distinguishes back-propagation from earlier, simpler methods such as the perceptron-convergence procedure" ^[13]. Hidden units are what a multilayer perceptron has and a single-layer one does not, which is how the 1969 objection was answered in practice.

Applications followed quickly. Terrence Sejnowski and Charles Rosenberg's NETtalk, published in Complex Systems in 1987, learned to convert English text to speech and behaved in ways its authors compared to human learning: "the learning follows a power law", "the more words the network learns, the better it is at generalizing and correctly pronouncing new words", and "the performance of the network degrades very slowly as connections in the network are damaged: no single link or processing unit is essential" ^[14]. Jeffrey Elman's "Finding Structure in Time", in Cognitive Science in 1990, addressed how such networks could handle input that unfolds over time, the problem later taken up by the recurrent neural network and LSTM literature ^[15].

Distributed and subsymbolic representation

What distinguishes connectionist representation is where information sits. In a localist scheme one unit stands for one thing. In a distributed scheme a concept is a pattern of activity spread across many units, and each unit participates in many concepts. The Stanford Encyclopedia notes that distributed representations "remain relatively well preserved when parts of the model are destroyed or overloaded", which is what NETtalk's damage results showed ^[14][37]. The Internet Encyclopedia describes the hidden-unit version of this as coarse coding, in which "particular hidden units often fail to have any particular input feature that they are exclusively sensitive to" and instead "participate in different ways in the processing of many different kinds of input" ^[36]. Fodor and Pylyshyn were describing the same idea when they wrote that in some connectionist architectures "environmental properties are encoded by the pattern of states of entire populations of units" ^[17].

Because the units in such a pattern do not individually correspond to anything nameable, the representation is called subsymbolic. The Stanford Encyclopedia puts it this way: the representations "are sub-symbolic in the sense that analysis into their components leaves the symbolic level behind" ^[37]. Paul Smolensky set out a defense of this level of description in a 1988 target article in Behavioral and Brain Sciences, "On the proper treatment of connectionism" ^[18]. The modern descendants of the idea are the learned vectors of representation learning: a word embedding is a distributed code in exactly this sense, and the geometry of a latent space is where the model's knowledge representation lives.

The systematicity debate

Fodor and Pylyshyn's "Connectionism and Cognitive Architecture: A Critical Analysis", published in Cognition in 1988, is the best known attack on the approach, and on the Internet Encyclopedia's assessment it "did succeed in framing the debate over connectionism for years to come" ^[16][36]. Their argument starts from systematicity, which they define as the observation that "the ability to produce/understand some of the sentences is intrinsically connected to the ability to produce/understand certain of the others". Nobody who understands "John loves the girl" fails to understand "the girl loves John"; and the same holds, they were prepared to bet, of animals without language, which are not in general "capable of representing the state of affairs aRb, but incapable of representing the state of affairs bRa" ^[17].

Classical architectures explain this because their representations have constituent structure: the two sentences are built from the same parts by the same rules, so any process sensitive to that structure applies to both. Connectionist architectures, Fodor and Pylyshyn argued, have no such guarantee. "Because it acknowledges neither syntactic nor semantic structure in mental representations, it perforce treats them not as a generated set but as a list. But lists, qua lists, have no structure; any collection of items is a possible list. And, correspondingly, on Connectionist principles, any collection of (causally connected) representational states is a possible mind" ^[17]. A connectionist can of course build a network that happens to be systematic, but that misses the point: "It's not enough just to stipulate systematicity; one is also required to specify a mechanism that is able to enforce the stipulation" ^[17]. Their conclusion was that connectionism, if it works at all, works as an account of how a classical architecture is implemented in neural hardware, not as a replacement for it ^[16].

Scroll sideways for more →

The Internet Encyclopedia's summary of the exchange is that connectionists offered "functional rather than literal compositionality", and that Fodor and McLaughlin answered that such demonstrations "only show that networks can be forced to exhibit systematic processing, not that they exhibit it naturally in the way that classical systems do". Its verdict on the wider quarrel is that neither side took the field: connectionism "has continued to thrive" without coming "close to fully supplanting classical" techniques, leaving "much more of a peaceful coexistence between the two camps" ^[36].

The past tense wars

The sharpest empirical test case was English verb inflection. Rumelhart and McClelland included a model in the second PDP volume that learned to produce past tenses without any explicit rule for adding -ed, and it over-regularized partway through learning in the way children do ^[12][37]. If a single network could produce both the regular and the irregular cases, the standard psycholinguistic picture of a rule plus a list of exceptions looked less necessary.

Steven Pinker and Alan Prince replied in the same 1988 issue of Cognition that carried Fodor and Pylyshyn, in a 121-page analysis arguing that the model failed to generalize properly to novel regular verbs ^[22][37]. The argument was still running fourteen years later: in November 2002 Trends in Cognitive Sciences ran McClelland and Patterson's "Rules or connections in past-tense inflections: what does the evidence rule out?" alongside Pinker and Ullman's "The past and future of the past tense" ^[23][24]. Gary Marcus revisited the strongest version of the connectionist position, the one that would eliminate symbols altogether, in a 1998 Cognitive Psychology paper titled "Rethinking Eliminative Connectionism" ^[25].

Connectionism and modern deep learning

Nothing in the 1990s settled the argument; what changed was engineering. Hinton, Simon Osindero and Yee-Whye Teh's 2006 fast learning algorithm for deep belief nets showed that networks with many layers could be trained by greedy layer-by-layer pretraining ^[26]. Six years later the AlexNet entry by Alex Krizhevsky, Ilya Sutskever and Hinton won the ImageNet competition with a convolutional neural network trained on GPUs ^[27][37]. Yann LeCun, Yoshua Bengio and Hinton surveyed the resulting methods in Nature in 2015 ^[28], and deep learning became the default technique across vision, speech and language.

The recognition followed. ACM named Yoshua Bengio, Geoffrey Hinton and Yann LeCun recipients of the 2018 Turing Award "for conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing", noting that by the early 2000s the three "were among a small group who remained committed to this approach" and that their attempts to revive interest in neural networks "were initially met with skepticism" ^[29]. In 2024 the Royal Swedish Academy of Sciences awarded John Hopfield and Hinton the Nobel Prize in Physics; its press release notes that the laureates "have conducted important work with artificial neural networks from the 1980s onward" ^[30]. The recipe has not changed in kind. A large language model is a network of weighted connections whose values are set by gradient descent on an error signal, which is what Rumelhart, Hinton and Williams described in 1986 ^[13], at a scale they did not have the hardware for.

That is not the same as saying the 1980s objections were wrong. The exchange still running in 2026 shows how live the question is. Brenden Lake and Marco Baroni's 2023 Nature paper claimed to "successfully address Fodor and Pylyshyn's challenge" with a meta-learning procedure that trains a standard architecture on a stream of compositional tasks and matches human behavior in a head-to-head experiment ^[31]. Michael Goodale and Salvador Mascarenhas replied in Transactions of the Association for Computational Linguistics in 2026, reporting that the model "struggles to learn rules that are even slightly out of distribution compared to their training data" and "behaves unsystematically even on many within-distribution problems", and concluding that "Fodor and Pylyshyn's challenge to neural networks remains unmet" ^[32]. Devon Jarvis, Richard Klein, Benjamin Rosman and Andrew Saxe took a different route in the Proceedings of the National Academy of Sciences in May 2026, deriving exact learning dynamics for deep linear networks under iterated learning and confirming that multiple generations are required before compositional structure emerges ^[33].

There is also a question about what "connectionism" now names. Olivia Guest and Andrea Martin's "A metatheory of classical and modern connectionism", published in Psychological Review in April 2026, uses formal analysis to separate two tendencies inside the tradition, which they call classical and modern, with divergent goals, mechanisms and scientific questions. Their argument is that the field often fails to follow its own lines of argument to the end, and that engaging with the underlying theory could break the cycle of AI winters and summers ^[34]. The related dispute over whether today's models understand anything was surveyed by Melanie Mitchell and David Krakauer in the Proceedings of the National Academy of Sciences in 2023 ^[35]. Read one way, the modern deep learning stack vindicates connectionism. Read another, it inherits the explanatory gap Fodor and Pylyshyn identified, on hardware large enough that the gap is harder rather than easier to inspect ^[32][34].

See also

• Neural network
• Perceptron
• Backpropagation
• Symbolic AI
• Deep learning
• History of artificial intelligence

References

1. ^McCulloch, W. S., and Pitts, W. "A logical calculus of the ideas immanent in nervous activity." The Bulletin of Mathematical Biophysics 5(4): 115-133, December 1943. doi.org/...BF02478259
2. ^Hebb, D. O. The Organization of Behavior. Wiley, 1949. openlibrary.org/...OL2399623W
3. ^Rosenblatt, F. "The perceptron: A probabilistic model for information storage and organization in the brain." Psychological Review 65(6): 386-408, 1958. doi.org/...h0042519 (archived scan: web.archive.org/...Rosenblatt1958.pdf )
4. ^Cornell Chronicle. "Professor's perceptron paved the way for AI: 60 years too soon." September 2019. news.cornell.edu/...paved-way-ai-60-years-too-soon
5. ^Minsky, M., and Papert, S. A. Perceptrons: An Introduction to Computational Geometry. MIT Press, 1969 (reissued 2017). doi.org/...11301.001.0001
6. ^Olazaran, M. "A Sociological Study of the Official History of the Perceptrons Controversy." Social Studies of Science 26(3): 611-659, August 1996. doi.org/...030631296026003005
7. ^Feldman, J. A., and Ballard, D. H. "Connectionist Models and Their Properties." Cognitive Science 6(3): 205-254, July 1982. doi.org/...s15516709cog0603_1
8. ^Hopfield, J. J. "Neural networks and physical systems with emergent collective computational abilities." Proceedings of the National Academy of Sciences 79(8): 2554-2558, April 1982. doi.org/...pnas.79.8.2554
9. ^Ackley, D. H., Hinton, G. E., and Sejnowski, T. J. "A Learning Algorithm for Boltzmann Machines." Cognitive Science 9(1): 147-169, January 1985. doi.org/...s15516709cog0901_7
10. ^McClelland, J. L., and Rumelhart, D. E. "An interactive activation model of context effects in letter perception: I. An account of basic findings." Psychological Review 88(5): 375-407, September 1981. doi.org/...0033-295X.88.5.375
11. ^Rumelhart, D. E., and McClelland, J. L. (eds). Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 1: Foundations. MIT Press, 1986. doi.org/...5236.001.0001
12. ^McClelland, J. L., and Rumelhart, D. E. (eds). Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 2: Psychological and Biological Models. MIT Press, 1986. doi.org/...5237.001.0001
13. ^Rumelhart, D. E., Hinton, G. E., and Williams, R. J. "Learning representations by back-propagating errors." Nature 323(6088): 533-536, 9 October 1986. cs.toronto.edu/...naturebp.pdf
14. ^Sejnowski, T. J., and Rosenberg, C. R. "Parallel Networks that Learn to Pronounce English Text." Complex Systems 1: 145-168, 1987. content.wolfram.com/...01-1-10.pdf
15. ^Elman, J. L. "Finding Structure in Time." Cognitive Science 14(2): 179-211, March 1990. doi.org/...s15516709cog1402_1
16. ^Fodor, J. A., and Pylyshyn, Z. W. "Connectionism and cognitive architecture: A critical analysis." Cognition 28(1-2): 3-71, March 1988. doi.org/...0010-0277(88)90031-5
17. ^Fodor, J. A., and Pylyshyn, Z. W. "Connectionism and Cognitive Architecture: A Critical Analysis." Author manuscript, Rutgers Center for Cognitive Science. ruccs.rutgers.edu/...ConnectionistArchitecture.pdf
18. ^Smolensky, P. "On the proper treatment of connectionism." Behavioral and Brain Sciences 11(1): 1-23, March 1988. doi.org/...S0140525X00052432
19. ^Smolensky, P. "Tensor product variable binding and the representation of symbolic structures in connectionist systems." Artificial Intelligence 46(1-2): 159-216, November 1990. doi.org/...0004-3702(90)90007-M
20. ^Pollack, J. B. "Recursive distributed representations." Artificial Intelligence 46(1-2): 77-105, November 1990. doi.org/...0004-3702(90)90005-K
21. ^Fodor, J., and McLaughlin, B. P. "Connectionism and the problem of systematicity: Why Smolensky's solution doesn't work." Cognition 35(2): 183-204, May 1990. doi.org/...0010-0277(90)90014-B
22. ^Pinker, S., and Prince, A. "On language and connectionism: Analysis of a parallel distributed processing model of language acquisition." Cognition 28(1-2): 73-193, March 1988. doi.org/...0010-0277(88)90032-7
23. ^McClelland, J. L., and Patterson, K. "Rules or connections in past-tense inflections: what does the evidence rule out?" Trends in Cognitive Sciences 6(11): 465-472, November 2002. doi.org/...S1364-6613(02)01993-9
24. ^Pinker, S., and Ullman, M. T. "The past and future of the past tense." Trends in Cognitive Sciences 6(11): 456-463, November 2002. doi.org/...S1364-6613(02)01990-3
25. ^Marcus, G. F. "Rethinking Eliminative Connectionism." Cognitive Psychology 37(3): 243-282, December 1998. doi.org/...cogp.1998.0694
26. ^Hinton, G. E., Osindero, S., and Teh, Y.-W. "A Fast Learning Algorithm for Deep Belief Nets." Neural Computation 18(7): 1527-1554, July 2006. doi.org/...neco.2006.18.7.1527
27. ^Krizhevsky, A., Sutskever, I., and Hinton, G. E. "ImageNet classification with deep convolutional neural networks." Communications of the ACM 60: 84-90, 2017. doi.org/...3065386
28. ^LeCun, Y., Bengio, Y., and Hinton, G. "Deep learning." Nature 521: 436-444, 27 May 2015. doi.org/...nature14539
29. ^ACM. "Fathers of the Deep Learning Revolution Receive ACM A.M. Turing Award." awards.acm.org, 2018 Turing Award page (archived 9 January 2026). web.archive.org/...2018-turing
30. ^The Royal Swedish Academy of Sciences. "The Nobel Prize in Physics 2024." Press release, 8 October 2024. nobelprize.org/...press-release
31. ^Lake, B. M., and Baroni, M. "Human-like systematic generalization through a meta-learning neural network." Nature 623(7985): 115-121, 25 October 2023. doi.org/...s41586-023-06668-3
32. ^Goodale, M., and Mascarenhas, S. "Fodor and Pylyshyn's Systematicity Challenge Still Stands." Transactions of the Association for Computational Linguistics 14: 1598-1611, 2026. doi.org/...tacl.a.743
33. ^Jarvis, D., Klein, R., Rosman, B., and Saxe, A. M. "Compositionality and systematicity emerge from iterated learning in deep linear networks." Proceedings of the National Academy of Sciences 123(19), 5 May 2026. doi.org/...pnas.2509739123
34. ^Guest, O., and Martin, A. E. "A metatheory of classical and modern connectionism." Psychological Review 133(3): 719-736, April 2026. Preprint: doi.org/...eaf2z_v2 . Journal version: doi.org/...rev0000591
35. ^Mitchell, M., and Krakauer, D. C. "The debate over understanding in AI's large language models." Proceedings of the National Academy of Sciences 120(13), 21 March 2023. doi.org/...pnas.2215907120
36. ^Waskan, J. "Connectionism." Internet Encyclopedia of Philosophy. iep.utm.edu/connectionism
37. ^Buckner, C., and Garson, J. "Connectionism." Stanford Encyclopedia of Philosophy, substantive revision 16 August 2019. plato.stanford.edu/...connectionism
38. ^Brock, R. "Connectionism: Edward Thorndike." In Science Education in Theory and Practice, Springer Texts in Education, pp. 83-97, 2025. doi.org/...978-3-031-81351-1_6