Linear Probes
A linear probe is a small linear classifier (or linear regressor) trained on the frozen internal activations of a neural network to test whether a particular concept, property, or label is linearly decodable…
Interpretability
Explore Interpretability through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Neural Networks
Articles that also belong to these categories. Counts cover all of Interpretability.
All articles (44)Machine Learning (12)Anthropic (8)Deep Learning (8)AI Safety (6)Neural Networks (6)AI Research (4)Large Language Models (4)Computer Vision (2)Developer Tools (2)Open Source AI (2)Transformer Models (2)AI Companies (1)AI Ethics (1)Algorithms (1)Generative AI (1)Mathematics (1)Model Evaluation (1)OpenAI (1)People (1)
Showing 1-6 of 6 articles
Monosemanticity
Monosemanticity is a property of an internal feature or neuron in a neural network when that unit responds to a single, human-interpretable concept rather than to a heterogeneous collection of unrelated inputs.
Polysemanticity
Polysemanticity is the phenomenon in artificial neural networks in which a single neuron (or directional unit such as an attention head) activates strongly for multiple, semantically unrelated inputs or…
Sparse Coding
Sparse coding is a representation learning principle in which a signal is encoded as a linear combination of a small number of elements drawn from a (usually overcomplete) dictionary of atoms.
Superposition (Mechanistic Interpretability)
Superposition is the phenomenon in which an artificial neural network represents more distinct features than it has dimensions in its activation space, by assigning those features to nearly-orthogonal (rather…
Transcoder
A transcoder is a sparse neural network used in mechanistic interpretability research to approximate the input-to-output function of a component inside a transformer (most commonly an MLP sublayer) using a…