-
MIB: A Mechanistic Interpretability Benchmark
Authors:
Aaron Mueller,
Atticus Geiger,
Sarah Wiegreffe,
Dana Arad,
Iván Arcuschin,
Adam Belfki,
Yik Siu Chan,
Jaden Fiotto-Kaufman,
Tal Haklay,
Michael Hanna,
Jing Huang,
Rohan Gupta,
Yaniv Nikankin,
Hadas Orgad,
Nikhil Prakash,
Anja Reusch,
Aruna Sankaranarayanan,
Shun Shao,
Alessandro Stolfo,
Martin Tutek,
Amir Zur,
David Bau,
Yonatan Belinkov
Abstract:
How can we know whether new mechanistic interpretability methods achieve real improvements? In pursuit of lasting evaluation standards, we propose MIB, a Mechanistic Interpretability Benchmark, with two tracks spanning four tasks and five models. MIB favors methods that precisely and concisely recover relevant causal pathways or causal variables in neural language models. The circuit localization…
▽ More
How can we know whether new mechanistic interpretability methods achieve real improvements? In pursuit of lasting evaluation standards, we propose MIB, a Mechanistic Interpretability Benchmark, with two tracks spanning four tasks and five models. MIB favors methods that precisely and concisely recover relevant causal pathways or causal variables in neural language models. The circuit localization track compares methods that locate the model components - and connections between them - most important for performing a task (e.g., attribution patching or information flow routes). The causal variable localization track compares methods that featurize a hidden vector, e.g., sparse autoencoders (SAEs) or distributed alignment search (DAS), and align those features to a task-relevant causal variable. Using MIB, we find that attribution and mask optimization methods perform best on circuit localization. For causal variable localization, we find that the supervised DAS method performs best, while SAE features are not better than neurons, i.e., non-featurized hidden vectors. These findings illustrate that MIB enables meaningful comparisons, and increases our confidence that there has been real progress in the field.
△ Less
Submitted 9 June, 2025; v1 submitted 17 April, 2025;
originally announced April 2025.
-
Updating CLIP to Prefer Descriptions Over Captions
Authors:
Amir Zur,
Elisa Kreiss,
Karel D'Oosterlinck,
Christopher Potts,
Atticus Geiger
Abstract:
Although CLIPScore is a powerful generic metric that captures the similarity between a text and an image, it fails to distinguish between a caption that is meant to complement the information in an image and a description that is meant to replace an image entirely, e.g., for accessibility. We address this shortcoming by updating the CLIP model with the Concadia dataset to assign higher scores to d…
▽ More
Although CLIPScore is a powerful generic metric that captures the similarity between a text and an image, it fails to distinguish between a caption that is meant to complement the information in an image and a description that is meant to replace an image entirely, e.g., for accessibility. We address this shortcoming by updating the CLIP model with the Concadia dataset to assign higher scores to descriptions than captions using parameter efficient fine-tuning and a loss objective derived from work on causal interpretability. This model correlates with the judgements of blind and low-vision people while preserving transfer capabilities and has interpretable structure that sheds light on the caption--description distinction.
△ Less
Submitted 3 October, 2024; v1 submitted 12 June, 2024;
originally announced June 2024.
-
Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability
Authors:
Atticus Geiger,
Duligur Ibeling,
Amir Zur,
Maheep Chaudhary,
Sonakshi Chauhan,
Jing Huang,
Aryaman Arora,
Zhengxuan Wu,
Noah Goodman,
Christopher Potts,
Thomas Icard
Abstract:
Causal abstraction provides a theoretical foundation for mechanistic interpretability, the field concerned with providing intelligible algorithms that are faithful simplifications of the known, but opaque low-level details of black box AI models. Our contributions are (1) generalizing the theory of causal abstraction from mechanism replacement (i.e., hard and soft interventions) to arbitrary mecha…
▽ More
Causal abstraction provides a theoretical foundation for mechanistic interpretability, the field concerned with providing intelligible algorithms that are faithful simplifications of the known, but opaque low-level details of black box AI models. Our contributions are (1) generalizing the theory of causal abstraction from mechanism replacement (i.e., hard and soft interventions) to arbitrary mechanism transformation (i.e., functionals from old mechanisms to new mechanisms), (2) providing a flexible, yet precise formalization for the core concepts of polysemantic neurons, the linear representation hypothesis, modular features, and graded faithfulness, and (3) unifying a variety of mechanistic interpretability methods in the common language of causal abstraction, namely, activation and path patching, causal mediation analysis, causal scrubbing, causal tracing, circuit analysis, concept erasure, sparse autoencoders, differential binary masking, distributed alignment search, and steering.
△ Less
Submitted 8 May, 2025; v1 submitted 11 January, 2023;
originally announced January 2023.
-
Causal Proxy Models for Concept-Based Model Explanations
Authors:
Zhengxuan Wu,
Karel D'Oosterlinck,
Atticus Geiger,
Amir Zur,
Christopher Potts
Abstract:
Explainability methods for NLP systems encounter a version of the fundamental problem of causal inference: for a given ground-truth input text, we never truly observe the counterfactual texts necessary for isolating the causal effects of model representations on outputs. In response, many explainability methods make no use of counterfactual texts, assuming they will be unavailable. In this paper,…
▽ More
Explainability methods for NLP systems encounter a version of the fundamental problem of causal inference: for a given ground-truth input text, we never truly observe the counterfactual texts necessary for isolating the causal effects of model representations on outputs. In response, many explainability methods make no use of counterfactual texts, assuming they will be unavailable. In this paper, we show that robust causal explainability methods can be created using approximate counterfactuals, which can be written by humans to approximate a specific counterfactual or simply sampled using metadata-guided heuristics. The core of our proposal is the Causal Proxy Model (CPM). A CPM explains a black-box model $\mathcal{N}$ because it is trained to have the same actual input/output behavior as $\mathcal{N}$ while creating neural representations that can be intervened upon to simulate the counterfactual input/output behavior of $\mathcal{N}$. Furthermore, we show that the best CPM for $\mathcal{N}$ performs comparably to $\mathcal{N}$ in making factual predictions, which means that the CPM can simply replace $\mathcal{N}$, leading to more explainable deployed models. Our code is available at https://github.com/frankaging/Causal-Proxy-Model.
△ Less
Submitted 28 September, 2022;
originally announced September 2022.
