BlackboxNLP

Writing your technical report for the MIB shared task?
Take a look at the task page for guidelines and tips!

The report deadline was also extended to August 10th!
Note that this is a final extension. We look forward to reading your reports! ✍️

Just 5 days left to submit your method to the MIB Shared Task at #BlackboxNLP!

Have last-minute questions or need help finalizing your submission?
Join the Discord server: discord.gg/n5uwjQcxPR

Results + technical report deadline: August 8, 2025
Full task details: blackboxnlp.github.io/2025/task/

With the new extended deadline, there's still plenty of time to submit your method to the MIB Shared Task!

We welcome submissions of existing methods, experimental POCs, or any approach addressing circuit discovery or causal variable localization 💡

Results deadline extended by one week!
Following requests from participants, we’re extending the MIB Shared Task submission deadline by one week.

🗓️ New deadline: August 8, 2025
Submit your method via the MIB leaderboard!

📝 Technical report guidelines are out!

If you're submitting to the MIB Shared Task at #BlackboxNLP, feel free to take a look to help you prepare your report: blackboxnlp.github.io/2025/task/

Just 10 days to go until the results submission deadline for the MIB Shared Task at #BlackboxNLP!

If you're working on:
🧠 Circuit discovery
🔍 Feature attribution
🧪 Causal variable localization
now’s the time to polish and submit!

Join us on Discord: discord.gg/n5uwjQcxPR

Are you attending ICML? 👀

I'm sadly not, but if you are, you should check out the MIB 🕶️poster at 11AM: icml.cc/virtual/2025...

The benchmark is used as the shared task at this year's
@blackboxnlp.bsky.social (blackboxnlp.github.io/2025/task/) - there's still time to participate 🏆

⏳ Three weeks left! Submit your work to the MIB Shared Task at #BlackboxNLP, co-located with @emnlpmeeting.bsky.social

Whether you're working on circuit discovery or causal variable localization, this is your chance to benchmark your method in a rigorous setup!

Have you started working on your submission for the MIB shared task yet? Tell us what you’re exploring!

New featurization methods?
Circuit pruning?
Better feature attribution?

We'd love to hear about it 👇

🗓️ Deadline: August 1
📜 Full task details: blackboxnlp.github.io/2025/task/
💬 Join the discussion: discord.gg/n5uwjQcxPR

Working on feature attribution, circuit discovery, feature alignment, or sparse coding?
Consider submitting your work to the MIB Shared Task, part of this year’s #BlackboxNLP

We welcome submissions of both existing methods and new or experimental POCs!

New to mechanistic interpretability?
The MIB shared task is a great opportunity to experiment:
✅ Clean setup
✅ Open baseline code
✅ Standard evaluation

Join the discord server for ideas and discussions: discord.gg/n5uwjQcxPR

The wait is over! 🎉 Our speakers for #BlackboxNLP 2025 are finally out!

🚨 Excited to announce two invited speakers at #BlackboxNLP 2025!

Join us to hear from two leading voices in interpretability:
🎙️ Quanshi Zhang (Shanghai Jiao Tong University)
🎙️ Verna Dankers (McGill University)

‪@vernadankers.bsky.social‬

That’s a great idea, thank you for the suggestion! We’ll make sure to keep the channel going beyond the deadline

📜 Check out the full task description: blackboxnlp.github.io/2025/task/

📅 Submission deadline: August 1

📢 Join the discord server for ideas, brainstorming, and q&a: discord.gg/n5uwjQcxPR

A typical pipeline:
• Build contrastive input pairs differing only in the target variable.
• (If supervised) train the featurizer on these pairs.
• To evaluate: Transform activation, intervene in feature space, transform back out, and check if behavior shifts as expected.

One month to go! ⏰
Working on featurization methods - ways to transform LM activations to better isolate causal variables?
Submit your work to the Causal Variable Localization Track of the MIB Shared Task!

Working on the MIB shared task?
Join the discord server: discord.gg/n5uwjQcxPR

🔍 Check out submission ideas
🔍 Brainstorm possible directions
🔍 Ask questions and get help with setup issues

Full task description: blackboxnlp.github.io/2025/task/

Check out the full task description: blackboxnlp.github.io/2025/task/

Important dates:
🗓️ Deadline for results: August 1
📄 Technical report: August 8

Join the discord server: discord.gg/n5uwjQcxPR

The Circuit Localization Track benchmarks methods for discovering causal circuits, subgraphs of a model responsible for specific behavior.

These methods typically:
• Score model components or edges
• Ablate all but the top-ranked ones
• Evaluate the performance of the resulting subgraph

Working on circuit discovery in LMs?
Consider submitting your work to the MIB Shared Task, part of #BlackboxNLP at @emnlpmeeting.bsky.social 2025!

The goal: benchmark existing MI methods and identify promising directions to precisely and concisely recover causal pathways in LMs >>

If you're working on feature attribution, circuit discovery, feature alignment, sparse coding, or related methods - this is for you!

🗓️ Aug 1 - Deadline for results
🗓️ Aug 8 - Deadline for technical report

More details: blackboxnlp.github.io/2025/task/
Join the discord server: discord.gg/n5uwjQcxPR

The task builds on the new Mechanistic Interpretability Benchmark (MIB) by Mueller* & Geiger* et al. (2025), with two tracks:
* Circuit Localization – identify subgraphs that carry out specific computations
* Causal Variable Localization – align internal representations with known causal factors

Have you heard about this year's shared task? 📢

Mechanistic Interpretability (MI) is quickly advancing, but comparing methods remains a challenge. This year at #BlackboxNLP, we're introducing a shared task to rigorously evaluate MI methods in language models 🧵

Hi Marco, if you refer to the regular CfP, it is already available on our website, and OpenReview links will be published in the upcoming weeks. The call for submissions for the shared task is also out, and submissions should be performed through the MIB leaderboard system for later evaluation.

Interested in mechanistic interpretability and care about evaluation? Please consider submitting to our shared task at #blackboxNLP this year!

BlackboxNLP 2025 The Eight Workshop on Analyzing and Interpreting Neural Networks for NLP

Have a look at our website for more information! blackboxnlp.github.io/2025