Sebastian Bordt

Theory of XAI Workshop, Dec 2, 2025 Explainable AI (XAI) is now deployed across a wide range of settings, including high-stakes domains in which misleading explanations can cause real harm. For example, explanations are required by law ...

🚨 Workshop on the Theory of Explainable Machine Learning

Call for ≤2 page extended abstract submissions by October 15 now open!

📍 Ellis UnConference in Copenhagen
📅 Dec. 2
🔗 More info: sites.google.com/view/theory-...

@gunnark.bsky.social @ulrikeluxburg.bsky.social @emmanuelesposito.bsky.social

I am hiring PhD students and/or Postdocs, to work on the theory of explainable machine learning. Please apply through Ellis or IMPRS, deadlines end october/mid november. In particular: Women, where are you? Our community needs you!!!

imprs.is.mpg.de/application
ellis.eu/news/ellis-p...

We need new rules for publishing AI-generated research. The teams developing automated AI scientists have customarily submitted their papers to standard refereed venues (journals and conferences) and to arXiv. Often, acceptance has been treated as the dependent variable. 1/

Center for the Alignment of AI Alignment Centers We align the aligners

This new center strikes the right tone in approaching the AI alignment problem. alignmentalignment.ai

I dont know if it's a good point to start, but you might want to take a look at the works by Daron Acemoglu and Pascual Restrepo pascual.scripts.mit.edu/research/

YouTube video by Friday Talks Tübingen How much can we forget about Data Contamination? - [Sebastian Bordt]

A new recording of our FridayTalks@Tübingen series is online!

How much can we forget about Data Contamination?
by
@sbordt.bsky.social

Watch here: youtu.be/T9Y5-rngOLg

What Formal Languages Can Transformers Express? A Survey Abstract. As transformers have gained prominence in natural language processing, some researchers have investigated theoretically what problems they can and cannot solve, by treating problems as forma...

Thanks for your comments. I don't think that neural networks are just a form of memory (though they can store a large number of memories). For example, transformers with unbounded steps are Turing-complete direct.mit.edu/tacl/article...

I see the point of the original post, but I think it's also important to keep in mind this other aspect.

www.inference.vc/we-may-be-su...

The stochastic parrot is now an IMO gold medalist parrot

Wednesday: Position: Rethinking Explainable Machine Learning as Applied Statistics icml.cc/virtual/2025...

I'm at #ICML in Vancouver this week, hit me up if you want to chat about pre-training experiments or explainable machine learning.

You can find me at these posters:

Tuesday: How Much Can We Forget about Data Contamination? icml.cc/virtual/2025...

Great to hear that you like it, and thank you for the feedback! I agree that stakeholders are important, although you are not going to find much about it in this paper. We might argue, though, that similar aspects with stakeholders arise in data science with large datasets, hence the analogy :)

Our #ICML position paper: #XAI is similar to applied statistics: it uses summary statistics in an attempt to answer real world questions. But authors need to state how concretely (!) their XAI statistics contributes to answer which concrete (!) question!
arxiv.org/abs/2402.02870

There are many more interesting aspects to this, so take a look at our paper!

arxiv.org/abs/2402.02870

We would also be happy for questions and comments on why we got it completely wrong.😊

If you are at ICML, I will present this paper on Wed 16 Jul 4:30 in the East Exhibition Hall A-B #E-501.📍

We think the literature on explainable machine learning can learn a lot from looking at these papers!📚

As I learned from our helpful ICML reviewers, there is a lot of existing research at the intersection of machine learning and statistics that takes the matter of interpretation quite seriously.

In this framework, another way to formulate the initial problems is: For many popular explanation algorithms, it is not clear whether they have an interpretation.

Having an interpretation means that the explanation formalizes an intuitive human concept, which is a fancy philosophical way of saying that it is clear what aspect of the function the explanation describes.🧠

In addition, the way to develop explanations that are useful "in the world" is to develop explanations that have an interpretation.

This has several important implications. Most importantly, explainable machine learning has often been trying to reinvent the wheel when we already have a robust framework for discussing complex objects in the light of pressing real-world questions.

It took us a while to recognize it, but once you see it, you can't unsee it: Explainable machine learning is applied statistics for learned functions.✨

Concretely, researchers in applied statistics study complex datasets by mapping their most important properties into low-dimensional structures. Now think:

Machine learning model ~ Large dataset
Explanation algorithm ~ Summary statistics, visualization

Here comes our key realization: This question has occurred in other disciplines before, specifically in applied statistics research.

So, how can we seriously discuss whether an explanation algorithm can be used to answer relevant questions about our trained model or the world?🌍

I have actually encountered this point in my own research before, where we did a detailed mathematical analysis of SHAP, but all the math could not reveal the right way to use the explanations in practice (arxiv.org/abs/2209.040...

And then they never make clear how the technical properties of the algorithm tie back to the grander goals of explainability research (meaning: how do we actually use the algorithm in practice?)🎯

The problem: Papers about explanation algorithms often start with grand promises, but then quickly dive into technical details.

During the last couple of years, we have read a lot of papers on explainability and often felt that something was fundamentally missing🤔

This led us to write a position paper (accepted at #ICML2025) that attempts to identify the problem and to propose a solution.

arxiv.org/abs/2402.02870
👇🧵

joint work with Suraj Srinivas, Valentyn Boreiko, and @ulrikeluxburg.bsky.social

Link to paper: arxiv.org/pdf/2410.03249
Link to code: github.com/tml-tuebinge...

We would be happy for feedback or questions about this work!