NDIF Team

NDIF Team We're a research computing project cracking open the mysteries inside large-scale AI systems. The NSF National Deep Inference Fabric consists of a unique combination of hardware and software that provides a remotely-accessible computing resource for scientists and students to perform detailed and reproducible experiments on large pretrained AI models, such as open large language models. We aim to make AI interpretability research more accessible through this channel by publishing lectures and educational content covering real interpretability research.

Subscribe to our channel for future updates:
www.youtube.com/@NDIFTeam

We will use this channel to post lectures on AI interpretability research, educational information, NDIF and NNsight updates, and more. If you're interested in collaborating on a video or would like to suggest a topic, please reach out!

David Bau is an Assistant Professor of Computer Science at Northeastern University's Khoury College. His lab studies the structure and interpretation of deep... ROME: Locating and Editing Factual Associations in GPT with David Bau

Our YouTube channel is live! Our first video features @davidbau.bsky.social‬ presenting the ROME project:
www.youtube.com/watch?v=eKd...

Want to try it for yourself? Check out our new mini-paper tutorial in NNsight to see how intervening on concept induction heads can reveal language-invariant concepts and cause a model to paraphrase text!

🔗 nnsight.net/notebooks/m...

Using causal mediation analysis on words that span multiple tokens, @sfeucht.bsky.social et al. found concept induction heads that are separate from token induction heads.

🔗 dualroute.baulab.info/

Induction heads are attention heads that help complete patterns by copying tokens (transformer-circuits.pub/2021/framew...), but can they also copy over concepts?

Great to present what’s coming next for NDIF at the @actinterp.bsky.social workshop at #ICML2025!

If you missed us, let’s chat after the conference. Reach out here: forms.gle/LtTyYnkaxDyg...

NNsight Applied Example Suggestion We welcome suggestions for tutorials using the NNsight library to explore model internals! If you would like to share an idea for an NNsight mini paper tutorial, please fill out the form below. Note: if you wish to share an existing walkthrough notebook on the NNsight website, please go fill out this form.

We welcome suggestions and contributions!
📨 To highlight your existing notebook using NNsight on our website, submit it here: forms.gle/T8aLr1YLbBA....
💡 If you'd like us to create a tutorial for a paper or concept, suggest it here: forms.gle/KfwrtbW2j9u...

We’re collaborating with researchers in the field to provide detailed, educational, and replicable notebook tutorials of recent papers. Check out nnsight.net/applied_tut... for a current list of mini paper tutorials. We plan to release a new tutorial every week.

We’re excited to announce a new series of applied "mini paper" tutorials! The goal of this series is to help researchers get hands-on experience with findings, methods, and results from recent papers in interpretability using NNsight and NDIF.

Google Colab

Excited to share our first paper replication tutorial, walking you through the main figures from "Do Language Models Use Their Depth Efficiently?" by @robertcsordas.bsky.social

🔎 Demo on Colab: colab.research.google.com/github/ndif-...

📖 Read the full manuscript: arxiv.org/abs/2505.13898

NNsight 0.5 Prerelease: Feedback Requested NNsight 0.5 Prerelease Now Available We’re excited to announce the prerelease of NNsight 0.5, packed with new features that you’ve been requesting. What’s New Enhanced intermediate value access during forward passes Flexible code execution within NNsight’s tracing context Improved debugging capabilities for smoother development Get Started Ready to explore NNsight 0.5? Our comprehensive Colab notebook includes installation instructions and feature overviews: Try NNsight 0.5 → Share Your Feed...

We’d love to hear your opinions on NNsight 0.5:

💭 Join our feedback roundtable event July 10th, 1 PM ET:
discord.gg/SzTPN8MUWc?...

‼️ Share all prerelease feedback and issues here: discuss.ndif.us/t/nnsight-0...

NNsight 0.5 Changes Colab notebook

Give NNsight 0.5 a try with our Colab walkthrough: colab.research.google.com/drive/1wjQh...

We are prereleasing NNsight 0.5 today! We've added many highly-requested new features, including easier access to intermediate values within the forward pass, ability to run any code within NNsight’s tracing context (no more proxies!), and easier debugging.

👀 Want to try it out for yourself? Check out our tutorial implementing the technique in just a few lines of NNsight code: nnsight.net/notebooks/t...

Diffusion Lens: Interpreting Text Encoders in Text-to-Image Pipelines Michael Toker, Hadas Orgad, Mor Ventura, Dana Arad, Yonatan Belinkov. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Diffusion Lens can help you understand model failure cases, how complex prompts are encoded, and more! Read the paper here: aclanthology.org/2024.acl-lo...

About Diffusion Lens: Developed in 2024 by Michael Toker, Diffusion Lens helps researchers better understand the text embedding process by harnessing diffusion models.

Are you interested in researching diffusion models in NNsight? We've released our first non-LLM tutorial, which covers the Diffusion Lens technique!

NSF National Deep Inference Fabric NDIF is a research computing project that enables researchers and students to crack open the mysteries inside large-scale AI systems.

Can't attend? Read the paper here: arxiv.org/abs/2407.14561
Want to try it out for yourself? Get started today: ndif.us/start.html

The NNsight API extends PyTorch to enable access and manipulation of model internals, while NDIF provides the scalable inference service that makes it all work, allowing researchers to remotely experiment on pretrained models without GPUs.

At #ICLR2025 in Singapore? Come visit our poster tomorrow (Saturday, April 26th)! This work addresses a critical gap in ML research: the ability to study the internals of very large neural networks.

What will be the linchpin for AI dominance?

Read our NSF/OSTP recommendations written with Goodfire's Tom McGrath tommcgrath.github.io, Transluce's Sarah Schwettmann cogconfluence.com, MIT's Dylan Hadfield-Menell @dhadfieldmenell.bsky.social

TLDR; Dominance comes from **interpretability** 🧵 ↘️

Please help amplify ARBOR, a fantastic new research opportunity! If you’d like to start contributing, NDIF is now hosting DeepSeek R1 8B and 70B, open for all researchers to experiment on via our API.

Sign up for API access here: login.ndif.us

GitHub - ARBORproject/arborproject.github.io Contribute to ARBORproject/arborproject.github.io development by creating an account on GitHub.

Today we're launching a multi-lab open collaboration, the ARBOR project, to accelerate AI interpretability research for reasoning models. Please join us!

github.com/ARBORproject...

(ARBOR = Analysis of Reasoning Behavior through Open Research)

Announcing ARBOR, an open research community for collectively understanding how reasoning models like openai-o3 and deepseek-r1 work. We invite all researchers and enthusiasts to this initiative by @wattenberg.bsky.social's and @davidbau.bsky.social's lab.

arborproject.github.io

The application deadline for our summer engineering fellowship is tomorrow! Don't forget to apply here: ndif.us/fellowship.h...

Release v0.4.0 · ndif-team/nnsight Changelog 0.4.0 released: 2025-1-31 Refer to the colab for interactive walkthrough. Breaking Changes The InterventionGraph now follows a sequential execution order. Module envoys are expected to ...

- Release Notes: github.com/ndif-team/nn...

- 0.4 Walkthrough: colab.research.google.com/github/ndif-...

- Full NNsight Walkthrough: colab.research.google.com/github/ndif-...

Note: NDIF's remotely-hosted models are now running on NNsight 0.4. You will need to update your NNsight version before running experiments remotely! 🎉

🌊 Streaming: Locally access and manipulate values during remote execution using .local()

🔄 Multiple Token Generation: Apply interventions to all generated tokens with .all(), streamlining the multiple token generation process.