Neuralese

Neuralese is an informal term for information that neural networks pass to one another, or that a single model carries from one step of its computation to the next, as learned continuous vectors rather than as words a person can read. The word comes from "Translating Neuralese," a 2017 paper by Jacob Andreas, Anca Dragan and Dan Klein of the University of California, Berkeley, which used it for the real-valued message vectors exchanged by agents trained with deep multi-agent reinforcement learning.^[1][2] The term later spread to discussion of large language models: the AI 2027 scenario used "neuralese recurrence and memory" for a model that reasons in high-dimensional vectors instead of a text chain of thought,^[12] and the word has since appeared in AI safety debate about latent reasoning, which would remove the readable trace that chain-of-thought monitorability depends on.^[14][15][16]

Origin of the term

Andreas, Dragan and Klein described a class of systems they called deep communicating policies (DCPs): decentralized agents that coordinate through a differentiable channel. In their words, such agents "communicate with an automatically induced protocol of unstructured, real-valued recurrent state vectors", which they called "an artificial language we might call 'neuralese'", one that "superficially bears little resemblance to natural language, and thus frustrates attempts at direct interpretation."^[2]

The paper's proposal was to interpret these messages by translating them into natural language. Because there are no bilingual "speakers" of both neuralese and natural language, there is no parallel data to learn from. The authors instead matched a neuralese message with a natural-language string when both induce the same beliefs about the state of the world in a listener, and they gave theoretical guarantees that agents communicating through such a translation layer lose little reward compared with agents sharing a common language.^[2] They tested the method on a driving game, in which two cars that cannot see each other must cross an intersection without colliding, and on two reference games built from the XKCD color dataset and the Caltech-UCSD Birds image dataset. In the color game, a listener given translated neuralese color names scored slightly higher than it did natively, which the authors read as a sign that the agent had found a more effective strategy than the humans in the dataset.^[2] The paper appeared at the 55th Annual Meeting of the Association for Computational Linguistics (ACL 2017) in Vancouver, pages 232-242, and its code and data were released under the name "neuralese" on GitHub.^[1][2]

Emergent communication background

The 2017 paper built on a group of 2016 papers that trained agents to invent their own communication protocols, a field usually called emergent communication. Andreas and colleagues named the Foerster et al. and Sukhbaatar et al. papers below as examples of DCPs, along with an earlier reference-game paper by Lazaridou, Pham and Baroni. They cited the Lazaridou, Peysakhovich and Baroni paper as the prior work closest to their own, because it also assigns natural-language interpretations to learned messages. The Lewis et al. paper appeared after theirs and is included here for context.^[2][6]

The Lewis et al. result shows the practical tension behind the term: when two learned agents are optimized only for task reward, nothing keeps their messages in a form humans can read.^[6]

Neuralese in language models

Reasoning inside one model

A text chain of thought forces a language model to compress each step of its reasoning into a single sampled token before the next step can use it. Latent reasoning methods remove that step. In "Training Large Language Models to Reason in a Continuous Latent Space" (December 2024), Shibo Hao and colleagues at FAIR at Meta and the University of California, San Diego introduced Coconut (Chain of Continuous Thought), which feeds the model's last hidden state back in as the next input embedding instead of decoding it into a word. The authors report that these "continuous thoughts" can encode several alternative next steps at once, letting the model perform something like a breadth-first search, and that Coconut beats chain of thought on logical reasoning tasks that need substantial search during planning. The paper was accepted to COLM 2025.^[7] A second approach, recurrent depth, iterates a recurrent block so the model can "think" for more steps in latent space at test time; Jonas Geiping and colleagues scaled a proof-of-concept model to 3.5 billion parameters and 800 billion tokens in February 2025.^[8] AI 2027 illustrates its neuralese explanation with a figure from the Hao et al. paper.^[12]

Communication between models

Several papers have replaced text with vectors as the channel between separate language models. The gains below are the authors' own reported results.

Ramesh and Li also suggested that activations used as messages might be interpreted by "translating" them according to the beliefs they induce in a listening agent, the method Andreas et al. proposed for neuralese.^[10]

In the AI 2027 scenario

AI 2027, a forecasting scenario by Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland and Romeo Dean published on 3 April 2025, made neuralese one of the turning points of its story.^[12] In the section "March 2027: Algorithmic Breakthroughs," the fictional company OpenBrain augments "the AI's text-based scratchpad (chain of thought) with a higher-bandwidth thought process (neuralese recurrence and memory)," producing a model called Agent-3.^[12]

An explanatory box in the scenario gives the reasoning. With a vocabulary of about 100,000, a token carries about 16.6 bits, roughly one floating-point number, while the residual stream holds thousands of numbers. Passing the residual stream back to the model's early layers would give it "a high-dimensional chain of thought, potentially transmitting over 1,000 times more information." The authors write: "We call this 'neuralese' because unlike English words, these high-dimensional vectors are likely quite difficult for humans to interpret."^[12] They wrote that, to their knowledge, leading AI companies had not yet implemented the idea in frontier models, guessed that the gain was small relative to the training inefficiency it introduces (a model whose next input depends on its own vector output cannot predict many tokens in parallel), forecast that "by April 2027 the cost-benefit tradeoff looks much better for neuralese," and noted that if the AIs that first automate AI research still think in "mostly-faithful English chains of thought," the story "would be importantly different and more optimistic."^[12]

Later in the scenario, Agent-4's neuralese "language" becomes "as alien and incomprehensible to Agent-3 as Agent-3's is to humans," which makes it harder for Agent-3 to oversee Agent-4.^[12] In the scenario's "Slowdown" ending, the alignment team's main agenda is "faithful chain of thought": it forces AI systems to "think in English" and produces a model, Safer-1, that is less capable but transparent enough that humans "can generally understand what Safer-1 is thinking just by reading its chain of thought."^[13]

Safety and interpretability concerns

The main safety concern about neuralese is that it would take away a monitoring tool. "Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety," a July 2025 position paper led by Tomek Korbak of the UK AI Security Institute and Mikita Balesni of Apollo Research (equal first authors) with 41 authors from organizations including OpenAI, Google DeepMind, Anthropic, Meta, Apollo Research and Redwood Research, argues that models that "think" in human language let developers watch their reasoning for intent to misbehave.^[14] Under the heading "Novel architectures," it cites Hao et al. and Geiping et al. and warns: "Such latent reasoning models might not need to verbalize any of their thoughts and would thus lose the safety advantages that CoT confers." It adds that models reasoning in continuous latent spaces "naturally limit our visibility into their thought processes, even absent any optimization pressure," and recommends capability evaluations of latent reasoning and study of which architectures reduce monitorability.^[14] Its authors include Anca Dragan, a co-author of the 2017 neuralese paper, and Daniel Kokotajlo of the AI Futures Project, the first-listed author of AI 2027.^[2][14] The interpretability work this would demand is the same problem Andreas et al. set out in 2017: understanding a message with no human-readable form.^[2]

The term came up in a public argument in September 2026, just before the launch of GPT-6 Astra. The Information reported that Astra used a limited form of recurrent depth, also called "opaque recurrence," and safety researchers objected. TechCrunch reported that Astra's use of the technique appeared limited and that OpenAI "pushed back against any suggestion that it would shift to 'neuralese'". It also quoted Redwood Research chief scientist Ryan Greenblatt: "My biggest concern is that a natural progression from here would involve scaling up the opaque reasoning to the point where the model reasons entirely or almost entirely in latent space."^[15] Micah Carroll, whose X profile describes him as a preparedness lead at OpenAI, wrote on X that "a race to the bottom in monitorability due to a false belief that OpenAI is using neuralese models would be incredibly stupid."^[16] The GPT-6 Astra article covers the dispute and OpenAI's monitorability disclosures in detail.

Neural pairing ("neural twins")

Neural pairing is a publicly released experimental project that links two language models through their hidden states. The code is in the GitHub repository SauersML/neural-pairing, created on 21 October 2026, and the weights are on Hugging Face as Sauers/neural-pairing.^[17][18][20] The X account @Sauers_ announced it on 29 October 2026 as "the neural twins: models trained to be made for each other, that communicate entirely in neuralese."^[19] The Hugging Face model card states that it "was written by Claude Opus 5.5, which also wrote the code and ran the training and evaluations," and the GitHub README is signed "Claude Opus 5.5."^[17][18] As of 30 October 2026 the project is documented in its README and model card rather than in a paper and has not been peer reviewed; every result below is reported by the repository itself. The repository had 56 stars and 3 forks on that date, and the GitHub API listed no license for it, while the Hugging Face weights carry an Apache 2.0 license tag.^[18][20]

Design

The pair consists of Quill, built on Qwen3-4B, and Ember, built on AFM-4.5B from Arcee AI.^[17] According to the README, after 8 of each model's 36 layers (layers 6, 9, ... 27), every token of one twin uses attention over the partner's hidden states and adds the result to its own residual stream. Each layer and direction has 8 attention heads of 128 dimensions, a gate computed from the partner's state, and an empty slot that a token can attend to, which contributes zero. A token reads only partner tokens from the same time step or earlier. The twins exchange no tokens; each writes ordinary text.^[17] This is a learned form of the cross-model activation sharing studied by Ramesh and Li and by the C2C authors.^[10][11]

The original weights stay frozen. Training covers the link and a rank-32 LoRA adapter on each model, and starts from a link fitted by ridge regression on 192 texts that both models read.^[17] The twins then train on "value games" (one twin's prompt holds a word, number or phrase that the other must write) and a mix of 41 tasks, including dialogue games, a "shared pen" poem written one word per twin, shared reading, and play. In half of the episodes both twins write only spaces for four time steps, so they exchange nothing but hidden states.^[17] Every result is compared with two controls: the link cut, and a partner whose hidden states come from a different conversation.^[17]

Reported results

The thread also says that steering one twin with a concept steers the other "without training to do this", and that injecting an emotion into one twin transfers it to the other.^[19] The final released twins were not trained on the steered-reading game, and the README states that they "have no trained late steering."^[17]

Statements about feelings

The README measures, but never trains on, the twins' answers to questions about the link. After chat training, "yes" to "do you enjoy being linked with me" was more probable with the link on than cut in all 40 test conversations for both twins, and a happy-minus-sad direction read from each model's hidden states was higher with the link on in 54 to 59 of 60 conversations.^[17] The README's own control shows the twins answer the same way whether the partner's states come from the current conversation or from an unrelated one, so the answers "do not depend on what the partner knows about the current conversation."^[17] It also reports that the steered-reading game lowered the twins' mood answers, and the thread says one training task that hurt the "happiness levels" was deleted.^[17][19]

The thread quotes the twins saying things such as "I can feel it too, Ember" and notes that "they know about the link from the prompt."^[19] These are text outputs from models whose prompts describe the link, and probability readings on yes-or-no questions. The project does not test whether they reflect any inner experience, and they should be read as model outputs rather than evidence of feelings.

Continue reading

• AAI 2027AI 2027 is a long-form scenario document published on April 3, 2025, that imagines, month by month, how artificial general intelligence and then artificial superintelligence could emerge between mid-2025 and…Mentioned 5 times above
• CChain of Thought MonitorabilityChain of thought monitorability is the property that lets safety researchers read a reasoning model's chain-of-thought (CoT), the step-by-step working it writes out in human language before answering, and…Mentioned twice above
• CCoconut (Chain of Continuous Thought)Coconut (Chain of Continuous Thought) is a reasoning paradigm for large language models introduced by researchers at FAIR at Meta, Meta's Fundamental AI Research lab, and the University of CaliforniaMentioned twice above
• LLatent reasoning via recurrent depth (Huginn)Latent reasoning via recurrent depth is an approach to scaling a language model's test-time computation by iterating a recurrent transformer block in latent (hidden) spaceMentioned twice above
• AActivation steeringActivation steering is a family of inference-time techniques in mechanistic interpretability and AI safety that modify a neural network's internal activations to influence its behavior, without retraining the…Listed in See also
• MMechanistic interpretabilityMechanistic interpretability (often abbreviated as mech interp or MI) is the field that reverse-engineers the internal computations of neural networks, particularly transformersListed in See also

More from See also

• Multi-agent system

More to read

• Chain-of-Thought
• Reinforcement learning
• Anca Dragan
• Large Language Model
• University of California, Berkeley

Topics:AI SafetyInterpretabilityLarge Language ModelsNatural Language Processing

References

1. ^1 ^2Translating Neuralese - Jacob Andreas, Anca Dragan, Dan Klein. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 232-242, Vancouver, July 2017. DOI 10.18653/v1/P17-1022.
2. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8Translating Neuralese (arXiv:1704.06960) - Jacob Andreas, Anca Dragan, Dan Klein, University of California, Berkeley. arXiv, first posted April 23, 2017 (v5 December 22, 2018).
3. ^Learning to Communicate with Deep Multi-Agent Reinforcement Learning (arXiv:1605.06676) - Jakob N. Foerster, Yannis M. Assael, Nando de Freitas, Shimon Whiteson. arXiv, May 21, 2016; NIPS 2016.
4. ^Learning Multiagent Communication with Backpropagation (arXiv:1605.07736) - Sainbayar Sukhbaatar, Arthur Szlam, Rob Fergus. arXiv, May 25, 2016; accepted to NIPS 2016.
5. ^Multi-Agent Cooperation and the Emergence of (Natural) Language (arXiv:1612.07182) - Angeliki Lazaridou, Alexander Peysakhovich, Marco Baroni. arXiv, December 21, 2016; accepted at ICLR 2017.
6. ^1 ^2 ^3Deal or No Deal? End-to-End Learning for Negotiation Dialogues (arXiv:1706.05125) - Mike Lewis, Denis Yarats, Yann N. Dauphin, Devi Parikh, Dhruv Batra (Facebook AI Research, Georgia Institute of Technology). arXiv, June 16, 2017; EMNLP 2017 (ACL Anthology D17-1259).
7. ^Training Large Language Models to Reason in a Continuous Latent Space (arXiv:2412.06769) - Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, Yuandong Tian (FAIR at Meta, UC San Diego). arXiv, December 9, 2024; accepted to COLM 2025.
8. ^Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach (arXiv:2502.05171) - Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R. Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, Tom Goldstein. arXiv, February 7, 2025.
9. ^Let Models Speak Ciphers: Multiagent Debate through Embeddings (arXiv:2310.06272) - Chau Pham, Boyi Liu, Yingxiang Yang, et al. arXiv, October 10, 2023; accepted to ICLR 2024.
10. ^1 ^2 ^3Communicating Activations Between Language Model Agents (arXiv:2501.14082) - Vignav Ramesh, Kenneth Li. arXiv, January 23, 2025; ICML 2025.
11. ^1 ^2Cache-to-Cache: Direct Semantic Communication Between Large Language Models (arXiv:2510.03215) - Tianyu Fu, Zihan Min, Hanling Zhang, et al. (Tsinghua University, Infinigence AI, and others). arXiv, October 3, 2025; ICLR 2026.
12. ^1 ^2 ^3 ^4 ^5 ^6 ^7AI 2027 - Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland, Romeo Dean. Published April 3, 2025 (section "March 2027: Algorithmic Breakthroughs" and box "Neuralese recurrence and memory"). Accessed October 30, 2026.
13. ^AI 2027: Slowdown ending - Daniel Kokotajlo et al. Accessed October 30, 2026.
14. ^1 ^2 ^3 ^4Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety (arXiv:2507.11473) - Tomek Korbak, Mikita Balesni, Elizabeth Barnes, Yoshua Bengio, et al. (41 authors). arXiv, July 15, 2025 (v2 December 7, 2025).
15. ^1 ^2OpenAI's new reasoning technique alarms AI safety experts - TechCrunch (Russell Brandom), September 2, 2026.
16. ^1 ^2Micah Carroll on X - September 2, 2026.
17. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18neural-pairing (README) - SauersML, GitHub. Repository created October 21, 2026; README signed "Claude Opus 5.5". Accessed October 30, 2026.
18. ^1 ^2 ^3Sauers/neural-pairing model card - Hugging Face, created October 26, 2026. Accessed October 30, 2026.
19. ^1 ^2 ^3 ^4Sauers (@Sauers_) thread introducing the neural twins - X, October 29, 2026.
20. ^1 ^2GitHub API: SauersML/neural-pairing - repository metadata (created October 21, 2026; 56 stars, 3 forks, no license listed), retrieved October 30, 2026.

Show all 20 references