claudia shi

Hypothesis Testing the Circuit Hypothesis in LLMs Large language models (LLMs) demonstrate surprising capabilities, but we do not understand how they are implemented. One hypothesis suggests that these capabilities are primarily executed by small sub...

Details are in the paper: arxiv.org/abs/2410.13032
We also developed a cool package for circuit testing: github.com/blei-lab/cir...
Find us at the NeurIPS Thursday poster session or at the bestest dim sum restaurant in Vancouver!

Our tests reveal gaps between the idealized version of the circuit representation and what we find in practice. By formalizing desirable properties, we hope to refine the circuit hypothesis, addressing questions such as what is the "optimal" level of granularity

Findings: Synthetic circuits align with all the ideal criteria. Semi-synthetic circuits pass some of the idealized tests. Circuits in the wild pass none of the idealized tests

We apply our tests to six benchmark circuits from the literature: two synthetic circuits, two semi-synthetic circuits (circuits discovered on toy transformer models), and two circuits in the wild (circuits discovered on transformer models such as GPT-2).

We compare the candidate circuit against random circuits drawn from a reference distribution. We vary the reference distribution to change the hardness of the test.

The idealized tests are stringent, so we developed two flexible tests that quantify:

Sufficiency Test: How faithful is faithful enough?
Partial Necessity Test: How much knockdown effect is significant?

Independence Test: Removing the circuit renders the model output independent of that of the circuit

Minimality Test: All edges in the circuit are necessary for the task

We translate these properties into three idealized tests:

Equivalence Test: The circuit and the original model have the same chance of outperforming each other

We formalize three criteria of an idealized circuit and develop hypothesis tests for them:
1️⃣ Mechanism Preservation: The circuit should preserve the model's behavior
2️⃣ Localization: Removing the circuit disables the task
3️⃣ Minimality: The circuit contains no redundant parts

The circuit hypothesis proposes that LLM capabilities emerge from small subnetworks within the model. But how can we actually test this? 🤔

joint work with @velezbeltran.bsky.social @maggiemakar.bsky.social @anndvision.bsky.social @bleilab.bsky.social Adria @far.ai Achille and Caro

hiiiii

I'd love to be added to the starter pack! I work on causal inference.

Hi Rob, I'd love to added to the starter pack.

I'd love to be added! i am bayesian adjacent!

i'd love to be added!

Hi! I'd love to be added to the starter pack!

Hi! could you also add me to the mech interp list? I do mech interp research.

@datatherapist.bsky.social i'd love to be added to the new one! thank you