Induction Heads
Induction heads are a circuit pattern in Transformer language models in which a small set of attention heads, typically spread across two layers, perform an in-context "match and copy" operation that completes…
Interpretability
Explore Interpretability through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Transformer Models
Articles that also belong to these categories. Counts cover all of Interpretability.
All articles (44)Machine Learning (12)Anthropic (8)Deep Learning (8)AI Safety (6)Neural Networks (6)AI Research (4)Large Language Models (4)Computer Vision (2)Developer Tools (2)Open Source AI (2)Transformer Models (2)AI Companies (1)AI Ethics (1)Algorithms (1)Generative AI (1)Mathematics (1)Model Evaluation (1)OpenAI (1)People (1)
Showing 1-2 of 2 articles
Logit lens
The logit lens is a foundational technique in mechanistic interpretability for inspecting the intermediate computations of transformer language models.