Recursive language model

Scroll sideways for more →

A recursive language model (RLM) is an inference strategy in which a large language model does not read its input prompt directly. Instead, the prompt is stored as a variable in an external environment, typically a Python REPL, and the model writes code to inspect, slice, and transform it, launching recursive calls to itself or to other models over pieces of the content when it needs semantic work done ^[1]. The approach was introduced by Alex L. Zhang, Tim Kraska, and Omar Khattab of MIT CSAIL, first as a blog post in October 2025 and then as the paper "Recursive Language Models" (arXiv:2512.24601) in December 2025 ^[1][2]. The paper reports that RLMs can process inputs up to two orders of magnitude beyond a model's context window and outperform common long-context scaffolds at comparable cost ^[1].

Despite the similar name, an RLM is not a model architecture: it is a scaffold wrapped around an ordinary trained model at inference time. It is unrelated to recurrent neural networks, to the tree-structured "recursive neural networks" of earlier NLP research, and to recursive self-improvement, which concerns AI systems improving their own design.

Mechanism

A standard model call feeds the entire prompt through the network in one pass, so quality and cost are both tied to prompt length. An RLM call keeps the same interface (text in, text out) but changes what happens inside. The user's prompt is loaded into a persistent REPL environment as a string variable. The root model receives only constant-size metadata about the prompt, such as its length, a short prefix, and instructions for accessing it ^[1]. The root model then iterates: it writes code that is executed in the REPL, sees truncated output, and continues until it commits to an answer, either emitted directly (FINAL(...)) or returned from a REPL variable it has built up (FINAL_VAR(...)) ^[2].

Two design choices define the paradigm ^[2]:

1. Context as a variable. Because the prompt is data in the environment rather than tokens in the window, the model can grep it, chunk it, count over it, or transform it programmatically, and its own context grows slowly no matter how large the input is.
2. Recursive sub-calls. The environment exposes a function that spawns a fresh model (or RLM) instance over any string, so the root model can delegate semantic questions about slices of the context and combine the answers in code.

The authors frame this as a new axis of test-time compute scaling: the trajectory by which a model decomposes its context is learnable, in the same way chain-of-thought reasoning is trained into reasoning models ^[2]. Scaling context this way does not require any single model call to handle a huge context. The original blog experiments limited recursion to depth 1 (the root can call plain models but not other RLMs); the revised paper evaluates depths up to 3 and finds that deeper recursion helps most on tasks requiring quadratic amounts of semantic work ^[1][2].

Motivation: context rot

The work targets the phenomenon of "context rot", the degradation in model quality as prompts grow longer even within the advertised context limit, documented empirically in a July 2025 report by Chroma Research ^[9]. Zhang's blog post argues that popular needle-in-a-haystack benchmarks mask the problem because retrieval of a single fact stays easy at long lengths, while tasks that depend densely on the whole input (long Claude Code sessions, aggregation queries) collapse ^[2]. The paper illustrates this with GPT-5 on tasks of increasing density: performance on simple needle retrieval holds up as inputs scale from 2^13 to 2^20 tokens, while performance on dense aggregation tasks degrades steeply; the RLM wrapper largely removes the degradation and keeps working past the point where inputs no longer fit in GPT-5's 272K-token window at all ^[1].

The dominant practical mitigation, compaction (periodically summarizing the context once it nears a threshold, as coding agents such as Claude Code and Codex do), is lossy: it presumes early details will not matter later. RLMs instead keep the full context intact in the environment and never force the model to consume it wholesale ^[1][10].

Results reported in the paper

The paper evaluates a frontier closed model (GPT-5) and a frontier open model (Qwen3-Coder-480B-A35B, from the Qwen3 family) across four long-context tasks ^[1]:

Scroll sideways for more →

Selected scores for GPT-5-based systems from the paper's main table (v3, May 2026) ^[1]:

Scroll sideways for more →

Summarizing across the four tasks, the abstract reports that RLMs on GPT-5 beat compaction by a median of 26 percent, CodeAct with sub-calls by 130 percent, and Claude Code by 13 percent, at comparable or lower cost per query ^[1]. The earlier blog post reported the result that drew the most attention: on OOLONG questions with contexts over 128K tokens, an RLM built on GPT-5-mini answered more than twice as many questions correctly as plain GPT-5 while being cheaper per query on average ^[2]. On the BrowseComp-Plus setup the RLM answers over corpora of 10M+ tokens loaded directly into the environment, with no retriever, and the blog notes performance did not degrade as the document count grew ^[1][2]. All of these figures are the authors' own evaluations.

The paper also post-trains what it calls the first natively recursive model. Using roughly 1,000 rejection-fine-tuning samples distilled from RLM trajectories of Qwen3-Coder-480B-A35B on an unrelated benchmark (LongBenchPro), the resulting RLM-Qwen3-8B improves on its Qwen3-8B base by 28.3 percent across the four evaluation tasks and approaches vanilla GPT-5 on three of them ^[1]. The weights were released on Hugging Face under an MIT license in January 2026 ^[5]. A separate experiment trains a Qwen3-4B model with reinforcement learning from verifiable rewards (RLVR) on a short split of the synthetic MRCRv2 task and finds the learned recursive behavior generalizes to longer splits ^[1].

Origin and people

Alex L. Zhang published "Recursive Language Models" as a blog post on October 15, 2025, with an accompanying thread on X, while a first-year PhD student at MIT CSAIL ^[2][3][16]. His advisors and co-authors are Tim Kraska and Omar Khattab, the latter known for the ColBERT retrieval model and the DSPy framework ^[1][16]. The full paper appeared on arXiv on December 31, 2025 and was revised in January and May 2026; the May 2026 version added the deeper-recursion ablations, additional harness baselines, and the reinforcement learning experiment ^[1]. Both the blog post and the paper reached the Hacker News front page (135 and 161 points respectively) ^[17][18].

Zhang released two codebases: rlm-minimal, a small reference implementation published alongside the blog post, and rlm, a plug-and-play inference library supporting multiple sandboxes that accompanies the paper and had over 5,000 GitHub stars as of August 2026 ^[4][19].

Adoption and follow-up work

Prime Intellect, where Zhang is a research fellow ^[16], adopted the paradigm early. A January 1, 2026 blog post called RLMs "the simplest, most flexible method for context folding", argued the approach is more aligned with the Bitter Lesson than hand-engineered context-management schemes, and announced RLMs as a major focus of the company's research ^[10]. On August 5, 2026 the company launched Prime Agent, an open-source coding TUI it describes as "a self-improving RLM harness for coding and long-running autonomous tasks" ^[11][12]. Prime Agent gives the model a persistent IPython kernel as its only tool; sub-agents are spawned as asynchronous rlm() function calls inside the kernel, and the agent's own prompts, skills, memory, and sub-agents are state the agent can edit at runtime (a design the company calls Continual Harness) ^[11]. Zhang, a co-author of the launch post, described it as "an RLM-native coding TUI" ^[13]. Among other results, Prime Intellect reports that Prime Agent running Anthropic's Opus 5 scored 95.5 percent Best@1 on ARC-AGI-3, above the benchmark's reported human expert baseline of 95.4 percent; these are the company's own evaluations ^[11].

Other follow-ups treat the RLM as a base to build on. Researchers at Apple proposed SRLM (March 2026), which augments the programmatic context interaction with uncertainty-guided self-reflection and reports up to 22 percent improvement over RLM under the same time budget ^[14]. alphaXiv published work on RL fine-tuning small 4B models to act as native RLMs, training parent and child calls under a shared policy ^[15]. Numerous community reimplementations exist in various agent harness and framework ecosystems, and the idea is frequently discussed as an alternative or complement to retrieval-augmented generation for long inputs.

Relationship to other approaches

RLMs sit in a crowded space of context engineering techniques, and the authors position them against the main alternatives ^[1][2]:

• Retrieval-augmented generation selects relevant chunks with a retriever before the model runs. An RLM needs no index or retriever; the model itself decides at test time what to look at, which matters for arbitrary one-off contexts where building an index is expensive. On BrowseComp-Plus the RLM outperformed ReAct-plus-BM25 retrieval baselines ^[2].
• Compaction and summarization irreversibly compress history. An RLM keeps everything and defers access decisions to the model.
• CodeAct-style code agents also act by writing code, but treat code as a way to call tools. The RLM view is context-centric: the context is an object to understand, and code plus recursive sub-calls are the means of understanding it ^[2].
• Fixed multi-agent pipelines (map-reduce over chunks, hard-coded summarizer hierarchies) wire the decomposition into the workflow. RLMs defer the decomposition strategy entirely to the model, so the same wrapper works across tasks.

Limitations and critiques

The paper's own limitations section notes that guardrails for RLMs are under-explored, and that the added layer of complexity can produce side effects such as exploding sub-call costs; latency is also a concern, since sequential REPL iterations and synchronous sub-calls make RLMs slower than a single model call, something the authors expect asynchronous sub-calls and sandboxed REPLs to improve ^[1]. An appendix documents negative results, including experiments with a long-chain-of-thought variant ^[1].

The sharpest external critique to date is the Apple SRLM paper, which reports that recursion itself is not the primary driver of RLM performance: most gains come from the programmatic context interaction, a simple self-reflective program search can match or surpass RLM without explicit recursion, and for contexts that fit within the model's window, RLMs with recursion often degrade performance relative to the base model ^[14]. The study also finds RLMs less effective on semantically intensive tasks where heuristic program search over the context is insufficient ^[14].

See also

• Context window
• Long context
• Retrieval-augmented generation
• Context engineering
• Test-time compute
• Prime Agent
• AI agent
• In-context learning

References

1. ^Alex L. Zhang, Tim Kraska, Omar Khattab. "Recursive Language Models". arXiv:2512.24601, submitted December 31, 2025, revised May 11, 2026. arxiv.org/...2512.24601
2. ^Alex L. Zhang. "Recursive Language Models". Personal blog, October 15, 2025. alexzhang13.github.io/...rlm
3. ^Alex Zhang (@a1zhang). Announcement thread on X, October 15, 2025. x.com/...1978469116542337259
4. ^alexzhang13/rlm: "General plug-and-play inference library for Recursive Language Models (RLMs), supporting various sandboxes". GitHub, first published December 2025. github.com/...rlm
5. ^mit-oasys/rlm-qwen3-8b-v0.1. Hugging Face model repository, January 15, 2026. huggingface.co/...rlm-qwen3-8b-v0.1
6. ^Amanda Bertsch, Adithya Pratapa, Teruko Mitamura, Graham Neubig, Matthew R. Gormley. "Oolong: Evaluating Long Context Reasoning and Aggregation Capabilities". arXiv:2511.02817, November 4, 2025. arxiv.org/...2511.02817
7. ^Zijian Chen, Xueguang Ma, Shengyao Zhuang, Ping Nie, Kai Zou et al. "BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent". arXiv:2508.06600, August 8, 2025. arxiv.org/...2508.06600
8. ^Yushi Bai, Shangqing Tu, Jiajie Zhang et al. "LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks". arXiv:2412.15204, December 19, 2024. arxiv.org/...2412.15204
9. ^Kelly Hong et al. "Context Rot: How Increasing Input Tokens Impacts LLM Performance". Chroma Research, July 2025. research.trychroma.com/context-rot
10. ^Prime Intellect. "Recursive Language Models: the paradigm of 2026". Company blog, January 1, 2026. primeintellect.ai/...rlm
11. ^Seth Karten, Alex L. Zhang, Kevin Thomas, Sebastian Müller, Prime Intellect Team. "Prime Agent: A self-improving RLM agent". Prime Intellect blog, August 5, 2026. primeintellect.ai/...prime-agent
12. ^Prime Intellect (@PrimeIntellect). Prime Agent launch post on X, August 5, 2026. x.com/...2085086999267144083
13. ^Alex Zhang (@a1zhang). Post on X about the Prime Agent launch, August 5, 2026. x.com/...2085089855541813350
14. ^Keivan Alizadeh, Parshin Shojaee, Minsik Cho, Mehrdad Farajtabar (Apple). "Recursive Language Models Meet Uncertainty: The Surprising Effectiveness of Self-Reflective Program Search for Long Context". arXiv:2603.15653, March 7, 2026. arxiv.org/...2603.15653
15. ^Daniel Kim, Rehaan Ahmad. "Reinforcing Recursive Language Models". alphaXiv blog, May 13, 2026. alphaxiv.org/...reinforcement-learning-for-rlms
16. ^Alex L. Zhang. Homepage (bio: PhD student at MIT CSAIL advised by Omar Khattab and Tim Kraska; research fellow at Prime Intellect). Accessed August 7, 2026. alexzhang13.github.io
17. ^"Recursive Language Models (RLMs)". Hacker News discussion, October 15, 2025. news.ycombinator.com/item
18. ^"Recursive Language Models". Hacker News discussion, January 3, 2026. news.ycombinator.com/item
19. ^alexzhang13/rlm-minimal: "Super basic implementation (gist-like) of RLMs with REPL environments". GitHub, October 16, 2025. github.com/...rlm-minimal