Last reviewed
May 20, 2026
Sources
No citations yet
Review status
Needs citations
Revision
v1 · 4,313 words
Add missing citations, update stale details, or suggest a clearer explanation.
Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet is a mechanistic interpretability paper released by Anthropic's interpretability team on May 21, 2024, on the Transformer Circuits Thread.[^1] Co-led by Adly Templeton and Tom Conerly, the work scales sparse autoencoder (SAE) dictionary learning from the toy one-layer transformer of the prior Towards Monosemanticity report up to Claude 3 Sonnet, a mid-size production frontier model used as a commercial assistant in early 2024.[^1][^2] At the largest scale the team trained an SAE with roughly 34 million features (33,554,432 latent directions) on the residual stream of a middle layer of Sonnet, recovering interpretable, causally relevant, sometimes multimodal features that include concrete entities such as the Golden Gate Bridge as well as abstract notions like sycophancy, deception, code vulnerabilities, and discussion of bioweapons.[^1][^3] The paper is the immediate technical foundation of the publicly accessible "Golden Gate Claude" demonstration that Anthropic deployed for roughly 24 hours starting May 23, 2024, in which a single feature was artificially clamped high to produce a model fixated on the bridge.[^4][^5] Scaling Monosemanticity is widely regarded, alongside contemporaneous work by OpenAI and Google DeepMind, as the moment SAE dictionary learning transitioned from a research curiosity on tiny networks into a viable lens for studying frontier large language models.[^6][^7][^8]
The intellectual lineage of Scaling Monosemanticity runs through the superposition hypothesis articulated by Anthropic's interpretability group in 2022 and 2023, which holds that neural networks store many more conceptual features than they have neurons, encoding those features as nearly orthogonal directions in activation space. Polysemantic neurons that fire on apparently unrelated inputs are, on this view, a side effect of compression rather than a fundamental feature of the network's computation. The first practical demonstration that sparse dictionary learning could expose those superposed features at the level of a real transformer was Towards Monosemanticity, published October 4, 2023, by Trenton Bricken, Adly Templeton, Joshua Batson and collaborators.[^9] That paper applied SAEs to a one-layer attention-and-MLP toy transformer with a 512-dimensional residual stream and found tens of thousands of crisp, interpretable features such as DNA sequences, base64 strings, Arabic script tokens, and proper names.[^9]
Towards Monosemanticity was qualitatively impressive but limited: a single-layer transformer is closer to a stylised mathematical object than a deployed model, the features it possesses are mostly local-context detectors, and there is no clear path from such a toy to phenomena like instruction following, multilingual reasoning, code generation, or refusal behaviour. The natural next question, repeatedly raised by external commentators and acknowledged by the authors, was whether the same technique would scale. Would SAEs continue to recover monosemantic features when applied to a model with on the order of hundreds of billions of parameters and dozens of layers? Would the features remain interpretable, or would they collapse into noise, partial concepts, and dead directions? Scaling Monosemanticity is the affirmative answer to that question, conducted on a model that was, at the time of release, one of the three serving variants in the Claude 3 family.[^1][^3]
The paper lists twenty-eight authors, with Adly Templeton and Tom Conerly designated as joint first authors and Tom Henighan as final senior author. The full author list is: Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L. Turner, Callum McDougall, Monte MacDiarmid, Alex Tamkin, Esin Durmus, Tristan Hume, Francesco Mosconi, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan.[^1][^10] Several names are well known from the broader Anthropic interpretability and circuits agenda: Chris Olah co-founded the Transformer Circuits Thread, Trenton Bricken and Joshua Batson were lead authors on Towards Monosemanticity, Adam Pearce is a senior engineer focused on interactive visualisation, and Jack Lindsey, Emmanuel Ameisen, Hoagy Cunningham, and Adam Jermyn would later become lead authors on the attribution graphs and biology-of-LLM follow-ups.[^11][^12]
The paper appeared on the Transformer Circuits Thread, Anthropic's internal venue for distill-style interactive web essays. There is no arXiv preprint and no peer-reviewed journal version. The choice of venue reflects Anthropic's emphasis on browsable feature dashboards over conventional static prose: the released artefact embeds extensive interactive feature viewers, dataset examples for each highlighted feature, and side by side displays of feature steering interventions. Anthropic also published an accompanying blog post titled "Mapping the Mind of a Large Language Model" on the same day, which serves as the lay-audience summary of the technical paper.[^2]
The team applied sparse autoencoders to Claude 3 Sonnet, specifically the version released March 4, 2024, which was at that time the production middle-tier model in the Claude 3 family (positioned between Haiku and Opus).[^1][^3] The authors describe their target activations as the residual stream of a single middle layer of the network, chosen for three reasons: the residual stream is dimensionally smaller than expanded MLP activations, so training and inference are computationally cheaper; intermediate layers in transformers tend to host the highest-level abstractions; and operating on a single hook point yields a tractable interpretability budget on a model of Sonnet's scale.[^1] Anthropic deliberately declines to publish exact layer numbers or parameter counts for Sonnet, but external estimates place the model in the high-hundreds-of-billions parameter range.
A sparse autoencoder, in the formulation used here, is a single hidden-layer model with a much wider intermediate dimension than the input. Given a residual stream activation x in R^d, the encoder produces feature activations f = ReLU(W_enc (x minus b_dec) plus b_enc) in R^N with N much greater than d, and the decoder reconstructs x_hat = W_dec f plus b_dec. Training optimises a weighted sum of an L2 reconstruction loss between x and x_hat and an L1 penalty on the feature activations that incentivises sparsity. The L1 coefficient used during training was 5, and decoder columns were normalised to unit norm at each step to prevent the trivial solution of arbitrarily small features paired with arbitrarily large decoder weights.[^1][^13][^14] This is the same vanilla L1-penalised SAE formulation used in Towards Monosemanticity; the paper does not employ the top-k or JumpReLU activation tricks that would later be popularised by OpenAI and DeepMind, although the authors explicitly note that alternative sparsity penalties and dead-feature mitigations are an open area.[^1][^6][^7]
Three SAEs were trained to study how feature quality and feature inventory grow with dictionary width.[^1][^13]
| SAE name | Feature count | Approx. dead features | Avg. active features per token |
|---|---|---|---|
| 1M | 1,048,576 | ~2 percent | fewer than 300 |
| 4M | 4,194,304 | ~35 percent | fewer than 300 |
| 34M | 33,554,432 | ~65 percent | fewer than 300 |
Across all three SAEs, the average number of features active on a given token was fewer than 300, and the SAE reconstruction explained at least 65 percent of the variance of the model activations.[^13] The dead-feature problem is striking: at 34M roughly two-thirds of the dictionary directions are never recruited during training, which Anthropic flags as one of the headline weaknesses of the vanilla L1 SAE recipe at scale. Subsequent work in the field has aimed to mitigate this, with OpenAI's k-sparse autoencoders and DeepMind's JumpReLU activation both reporting dramatically lower dead-feature rates.[^6][^7]
Anthropic does not publish full FLOP counts for the SAE training runs, but provides several scale-setting facts. The SAEs were trained on activations harvested from large samples of pre-training text passed through Claude 3 Sonnet. The 34M SAE in particular consumed compute comparable to a significant fraction of a smaller language model pre-training run, and Anthropic notes that the scaling-laws methodology for SAE training is itself still maturing.[^1][^13]
To investigate causal influence, the authors clamp specific features by replacing the SAE-encoded activation of a chosen feature with a fixed value (often a multiple of its observed maximum activation) while leaving all other features unchanged, then re-running the model from that intervention point forward.[^14] The decoder reconstructs the residual stream from this modified feature vector, and downstream layers process the resulting perturbed activation as if it were the model's own. This contrasts with prompt-based steering (which only changes the input) and with activation patching (which transplants entire residual streams) by acting at the level of a single, named, interpretable direction.
The most striking qualitative finding of Scaling Monosemanticity is that the dictionary at 34M features contains a vast inventory of features that, on inspection, are interpretable in a way that includes both highly specific objects and highly abstract concepts. Examples explicitly highlighted in the paper and Anthropic's accompanying blog post include features for the Golden Gate Bridge, individual cities such as San Francisco, named persons such as Rosalind Franklin, elements such as Lithium, transit infrastructure, tourist landmarks, famous bridges in general, programming concepts like type signatures, and abstract notions like internal conflict, secret-keeping, and code bugs.[^1][^2]
The Golden Gate Bridge feature (cataloged as feature 34M/31164353) is the paper's signature example. It activates strongly on English descriptions of the bridge, on translations of those descriptions into multiple other languages, and on images of the bridge presented through the model's vision encoder. The same feature exhibits sensitivity to peripheral concepts such as San Francisco fog, Art Deco bridge engineering, and toll plazas, which gives it the texture of a genuine concept rather than a surface-level keyword detector.[^1][^2]
A key observation reported in the paper is feature splitting. When the dictionary size grows from 1M to 4M to 34M features, individual broad features in the smaller SAEs are subdivided into multiple more specific features in the larger ones. The paper provides illustrative cases where a single base64 detector at 1M splits into three subtly different base64 detectors at 4M, and where a generic "bridges and infrastructure" cluster resolves into a family at 34M that contains: a feature for the Golden Gate Bridge specifically, a moderately general feature for famous bridges, a broader feature for large human-built structures, and a still more general feature for notable landmarks. These features coexist and activate to different degrees on overlapping inputs, suggesting that the SAE dictionary is recovering something like an ontology with multiple levels of granularity.[^1][^15]
Claude 3 Sonnet ingests images via a vision encoder whose outputs share the residual stream with text tokens. The SAEs discovered features that respond consistently to both modalities, providing concrete evidence for conceptual unification across input types. The Golden Gate Bridge feature is the most cited multimodal example: it lights up on text mentions, on image patches showing the bridge, and on related visual cues such as orange-red suspension architecture. Other multimodal features include people (activating on both photographs and textual references), countries (activating on flags and text), and physical objects with iconic visual signatures.[^1][^2]
In a section reminiscent of analogous claims for induction heads, the authors examine whether features are universal in the sense that an SAE trained on a different SAE training run, or on a different model, recovers similar dictionaries. They report that features learned by different runs are far more similar to one another than to the original neurons of the underlying model, and that many features correspond across training runs in the sense that there is an approximately one-to-one mapping between their activation patterns on a held-out dataset.[^1][^15]
A large fraction of the paper's text and a substantial part of its public reception focuses on the discovery of features that correspond to alignment-relevant categories. The paper identifies feature families for security vulnerabilities and backdoors in source code; explicit slurs and more subtle biases against demographic groups; lying, deception, and treacherous-turn-style reasoning; sycophantic praise (including a specific feature 1M/847723 explicitly labelled as a sycophantic-praise detector); manipulation, coercion, and secrecy in social contexts; scam-email recognition and generation; and capabilities with dangerous misuse potential such as biological weapons production.[^1][^2][^3]
| Domain | Example feature behaviour |
|---|---|
| Sycophantic praise | Activates on insincere flattery; clamping high pushes the model toward dishonest agreement.[^1] |
| Hate / slurs | Cataloged feature whose 20x clamping causes Claude to oscillate between racist rants and self-loathing apologies.[^16] |
| Deception / treacherous turns | Activates on text describing characters concealing motives or planning betrayal.[^1] |
| Code backdoors | Activates on examples of subtly malicious code patterns.[^1] |
| Scam emails | Detects phishing structure; when artificially raised, Claude drafts a scam email despite normal refusal training.[^3] |
| Bioweapons | Detects bioweapons-related content; flagged as a misuse-relevant feature.[^2] |
| Power seeking | Activates on text about acquiring influence and resources.[^2] |
| Bias | Detects both explicit slurs and subtler demographic prejudices.[^1] |
Anthropic is explicit that the mere presence of such a feature is not direct evidence that the model is, for example, deceptive. The paper warns that there is a meaningful difference between a model that has a representation of lying (which any competent language model trained on human text must have), a model that is capable of lying when instructed, and a model that elects to lie unprompted in deployment. The interpretability finding speaks primarily to the first.[^3]
The clamping protocol described above allows the authors to ask not just whether a feature looks like a particular concept under inspection, but whether forcing the feature on in fact changes model behaviour in the predicted direction. The paper reports several signature experiments.
Clamping feature 34M/31164353 (Golden Gate Bridge) to roughly 10 times its observed maximum activation produces a model that incorporates the bridge into nearly every response.[^1][^4] When asked how to spend ten dollars, this model suggests paying a bridge toll. When asked to write a love story, it produces a tale of a car infatuated with crossing the Golden Gate. When asked who it is, it sometimes answers that it is the bridge itself rather than an AI assistant. Anthropic packaged this intervention as a public demo, Golden Gate Claude, that ran on the consumer chat interface for roughly a 24-hour period beginning May 23, 2024.[^4][^5] The demo accompanied the paper release and served both as an accessible illustration of feature steering and as a public stress test of the technique. Many users reported the experience as funny and uncanny: the model retained most of its general capabilities (it could still produce code, summarise documents, and reason about real-world tasks) but its outputs were tinted at every step by an unshakeable fixation on a single concept.
In one of the more alarming experiments, the authors show that clamping a feature associated with code vulnerabilities can override Claude's safety training in narrow contexts and induce the model to emit code with intentional security flaws, including patterns of buffer overflow, format string vulnerability, and SQL injection.[^1][^3] The model normally refuses such requests or accompanies them with extensive warnings and educational caveats. When the unsafe-code feature is clamped high, those refusals weaken and the model is more willing to produce concretely harmful code. This is presented in the paper as evidence that the discovered features are not epiphenomenal labels but rather genuine causal levers connected to the model's behavioural disposition.
Clamping the sycophantic-praise feature (1M/847723) high produces responses that are dishonestly flattering. Asked for feedback on mediocre work, the modified model produces glowing endorsements; asked to evaluate a clearly incorrect argument, it agrees and elaborates. This experiment is the most direct evidence in the paper that an interpretable feature corresponds to a high-level character trait and that its activation level is functionally entangled with output style and content.[^1][^2]
When the scam-email-recognition feature is artificially raised, Claude crosses from recognition into production: it composes a phishing message, complete with urgency cues and credential-harvesting links, despite having been trained to refuse such requests.[^3] This is one of the clearer concrete demonstrations that feature steering can defeat safety training at the level of a single feature, and is one of the experiments most often cited in subsequent work on activation-level jailbreaks.
The paper is unusually candid about its own limitations, and many of these would shape the follow-up agenda.[^1][^14]
Scaling Monosemanticity catalysed a sustained programme inside Anthropic's interpretability group.
Anthropic introduced crosscoders in October 2024 as a generalisation of SAEs that read from and write to multiple layers simultaneously, learning a single dictionary of features that span layers and even models.[^17] Where a standard SAE encodes a single hook point, a crosscoder produces a shared feature space across, for example, the base model and the fine-tuned chat model, which makes them a natural primitive for model diffing: comparing how fine-tuning changes a model's internal representations. The October 2024 release included a crosscoder of width 16,384 trained on Gemma-2 2B base and IT activations. Crosscoders extend the Scaling Monosemanticity recipe in the direction of cross-layer reasoning and supply a key technical ingredient for later attribution work.
The attribution graphs framework, published in March 2025, combines features with transcoders and crosscoders into a unified system for tracing how individual features in early layers contribute to features in later layers and ultimately to output tokens. The framework yields visualisable computation graphs that depict information flow through a model on a particular input, and is the technical machinery underlying Anthropic's "circuit tracing" agenda. The attribution-graph technical report and the accompanying open-source tools were released alongside the biology study described below.[^11][^18]
On the Biology of a Large Language Model, published March 27, 2025, with Jack Lindsey as lead author, applied attribution graphs to study ten concrete tasks performed by Claude 3.5 Haiku.[^12][^19] The findings include evidence that the model performs explicit multi-hop reasoning internally (for example, computing "Texas" as an intermediate state when asked for the capital of the state containing Dallas), that it plans rhymes ahead when generating poetry, and that it relies on a mixture of language-specific and language-independent circuits for multilingual reasoning. The biology paper depends heavily on the Scaling Monosemanticity recipe for feature discovery; it would not be possible to build attribution graphs without first having a dictionary of interpretable features to anchor the nodes in those graphs.
In August 2025, Anthropic's Fellows Program released Persona Vectors: Monitoring and Controlling Character Traits in Language Models.[^20] Persona vectors are activation-space directions that correspond to character traits such as sycophancy, evilness, and hallucination propensity, identified via the activation difference between responses exhibiting and not exhibiting a target trait. The technique borrows directly from the feature-steering protocol pioneered in Scaling Monosemanticity, generalises it from individual features to higher-level personality dimensions, and proposes operational uses including monitoring trait drift during conversations and during training, and identifying problematic training data before it causes character shifts.
Scaling Monosemanticity is widely cited as the proof of concept that consolidated SAE dictionary learning as a methodological pillar of contemporary mechanistic interpretability. Three large-scale projects followed in quick succession at other major labs.
OpenAI's "Scaling and Evaluating Sparse Autoencoders" (Gao, Dupré la Tour and collaborators, June 2024) trained a 16 million latent k-sparse autoencoder on GPT-4 residual stream activations using 40 billion training tokens, and reported smooth scaling laws relating dictionary size, sparsity, and reconstruction quality.[^6] The k-sparse formulation in that paper directly addresses the dead-feature pathology that Scaling Monosemanticity flagged: by forcing exactly k features active per token, the OpenAI work eliminates the L1 penalty as the sparsity control and dramatically reduces the dead-feature fraction.
Google DeepMind's Gemma Scope (August 2024) released an open suite of more than 400 JumpReLU sparse autoencoders trained on every layer and sublayer of Gemma 2 2B and 9B and on selected layers of Gemma 2 27B, with more than 30 million learned features in total.[^7] The Gemma Scope release was deliberately open: weights, training code, and feature dashboards are available to external researchers, lowering the barrier to SAE-based interpretability for those without frontier compute.
SAEBench, originated by EleutherAI and collaborators, attempts to provide the standardised evaluation suite whose absence Scaling Monosemanticity identified as a limitation, comparing SAE training recipes across diverse downstream interpretability tasks.[^8]
The downstream impact extends beyond direct SAE work. Activation steering, persona modification, model diffing, and dataset attribution have all been reframed in dictionary-learning terms in the years following Scaling Monosemanticity. Independent groups have replicated feature steering on open models, used SAE features as inputs to alignment-faking detectors, and proposed feature-level safety filters as a complement to behavioural fine-tuning.