Our spotlight paper is happening today at the #NeurIPS poster session! Drop by if you want to chat about the nitty-gritty details of large-scale transformer training!
📄 Paper: arxiv.org/abs/2505.22491
Catch our Spotlight at #NeurIPS2025 Today!
📅 Wed Dec 3 🕟 4:30 - 7:30 PM 📍 Exhibit Hall C,D,E — Poster #3903
Huge thanks to my amazing collaborators: @mohaas.bsky.social @sbordt.bsky.social @ulrikeluxburg.bsky.social
Ever wondered about the rationale behind transformer training details like qk-norm, learning rate, and z-loss? Read this blog post to find out more!
Here is a formal impossibility result for XAI: Informative Post-Hoc Explanations Only Exist for Simple Functions. I'll give an online presentation about this work next tuesday in @timvanerven.nl 's Theory of Interpretable AI Seminar:
arxiv.org/abs/2508.11441
tverven.github.io/tiai-seminar/
🚨 Workshop on the Theory of Explainable Machine Learning
Call for ≤2 page extended abstract submissions by October 15 now open!
📍 Ellis UnConference in Copenhagen
📅 Dec. 2
🔗 More info: sites.google.com/view/theory-...
@gunnark.bsky.social @ulrikeluxburg.bsky.social @emmanuelesposito.bsky.social
I am hiring PhD students and/or Postdocs, to work on the theory of explainable machine learning. Please apply through Ellis or IMPRS, deadlines end october/mid november. In particular: Women, where are you? Our community needs you!!!
imprs.is.mpg.de/application
ellis.eu/news/ellis-p...
We need new rules for publishing AI-generated research. The teams developing automated AI scientists have customarily submitted their papers to standard refereed venues (journals and conferences) and to arXiv. Often, acceptance has been treated as the dependent variable. 1/
This new center strikes the right tone in approaching the AI alignment problem. alignmentalignment.ai
I dont know if it's a good point to start, but you might want to take a look at the works by Daron Acemoglu and Pascual Restrepo pascual.scripts.mit.edu/research/
A new recording of our FridayTalks@Tübingen series is online!
How much can we forget about Data Contamination?
by
@sbordt.bsky.social
Watch here: youtu.be/T9Y5-rngOLg
Thanks for your comments. I don't think that neural networks are just a form of memory (though they can store a large number of memories). For example, transformers with unbounded steps are Turing-complete direct.mit.edu/tacl/article...
I see the point of the original post, but I think it's also important to keep in mind this other aspect.
www.inference.vc/we-may-be-su...
The stochastic parrot is now an IMO gold medalist parrot
Wednesday: Position: Rethinking Explainable Machine Learning as Applied Statistics icml.cc/virtual/2025...
I'm at #ICML in Vancouver this week, hit me up if you want to chat about pre-training experiments or explainable machine learning.
You can find me at these posters:
Tuesday: How Much Can We Forget about Data Contamination? icml.cc/virtual/2025...
Great to hear that you like it, and thank you for the feedback! I agree that stakeholders are important, although you are not going to find much about it in this paper. We might argue, though, that similar aspects with stakeholders arise in data science with large datasets, hence the analogy :)
Our #ICML position paper: #XAI is similar to applied statistics: it uses summary statistics in an attempt to answer real world questions. But authors need to state how concretely (!) their XAI statistics contributes to answer which concrete (!) question!
arxiv.org/abs/2402.02870
There are many more interesting aspects to this, so take a look at our paper!
arxiv.org/abs/2402.02870
We would also be happy for questions and comments on why we got it completely wrong.😊
If you are at ICML, I will present this paper on Wed 16 Jul 4:30 in the East Exhibition Hall A-B #E-501.📍
We think the literature on explainable machine learning can learn a lot from looking at these papers!📚
As I learned from our helpful ICML reviewers, there is a lot of existing research at the intersection of machine learning and statistics that takes the matter of interpretation quite seriously.
In this framework, another way to formulate the initial problems is: For many popular explanation algorithms, it is not clear whether they have an interpretation.
Having an interpretation means that the explanation formalizes an intuitive human concept, which is a fancy philosophical way of saying that it is clear what aspect of the function the explanation describes.🧠
In addition, the way to develop explanations that are useful "in the world" is to develop explanations that have an interpretation.
This has several important implications. Most importantly, explainable machine learning has often been trying to reinvent the wheel when we already have a robust framework for discussing complex objects in the light of pressing real-world questions.
It took us a while to recognize it, but once you see it, you can't unsee it: Explainable machine learning is applied statistics for learned functions.✨
Concretely, researchers in applied statistics study complex datasets by mapping their most important properties into low-dimensional structures. Now think:
Machine learning model ~ Large dataset
Explanation algorithm ~ Summary statistics, visualization
Here comes our key realization: This question has occurred in other disciplines before, specifically in applied statistics research.
So, how can we seriously discuss whether an explanation algorithm can be used to answer relevant questions about our trained model or the world?🌍
I have actually encountered this point in my own research before, where we did a detailed mathematical analysis of SHAP, but all the math could not reveal the right way to use the explanations in practice (arxiv.org/abs/2209.040...
