If you want to read more about uncertainty-aware neural networks, I highly recommend our #paper on machine-learned #uncertainties for the calibration of calorimeter signals in the #ATLAS experiment:

inspirehep.net/literature/2...

Spending the holidays teaching my little nephews some machine-learning basics — it's not that difficult; the Bayesian neural network (BNN) loss function follows a clear statistics logic

Precision calibration of calorimeter signals in the ATLAS experiment using an uncertainty-aware neural network The ATLAS experiment at the Large Hadron Collider explores the use of modern neural networks for a multi-dimensional calibration of its calorimeter signal defined by clusters of topologically connecte...

Precision calibration of calorimeter signals in the ATLAS experiment using an uncertainty-aware neural network
arxiv.org/abs/2412.04370v1
ATLAS Collaboration

This has become one of my most favorite (pre-)Christmas traditions: the annual #Glühwein workshop — this year at the Karlsruher Institut für Technologie (KIT), organized by Markus Klute (thank you!)

Summary: the #BNN not only yields a continuous and smooth topo-cluster #calibration function that improves the performance relative to the standard LCW calibration — but also provides meaningful single-cluster #uncertainties on the predicted responses and the calibrated energies

Both pulls follow an approximate Gaussian shape in the center (as expected for stochastic or noisy data) — and both networks slightly overestimate the uncertainty, meaning that the per-cluster error is conservative

After checking that the BNN and RE uncertainties are highly comparable, they can be further evaluated with respect to the spread of the predicted response around the target — the "pull" allows us to test if the learned single-cluster uncertainty covers the experimental spread

We can also directly compare the individual uncertainty predictions from the Bayesian neural network (BNN) and the repulsive ensemble (RE) cluster-by-cluster — the two uncertainty predictions track each other well

The #systematic uncertainty (part of the likelihood) approaches the same plateau as for the BNN when increasing the training-dataset size (green and brown curves) — and the #statistical uncertainty (induced by the repulsive force) again vanishes (red and blue curves)

The idea is to determine uncertainties by forcing an ensemble of simultaneously trained networks to not predict the same best-fit parameters, but to force the ensemble to spread out and to explore the loss landscape around the actual minimum

Learned #uncertainties on neural-network outputs are not a standard method used in HEP... To increase confidence in the uncertainty predictions from our BNN setup, we compare our BNN results with an alternative way of learning uncertainties — so-called #repulsive #ensembles (REs)

We see that these topo-clustes are all located in the tile-gap scintillator region: the tile-gap scintillator is not a regular calorimeter — the feature quality in this region is insufficient, so it is expected that the calibration in this region yields a large uncertainty

An interesting question is what role the learned uncertainties can play in understanding the data... When looking at the uncertainty spectrum, we see a distinctive secondary maximum — what feature leads the BNN uncertainties to flag these topo-clusters?

...(ii) the "systematic uncertainty" (green curve) captures the intrinsic data stochasticity (pile-up), and accounts for limited network expressivity and bad hyper-parameters — for learning the stochastic nature, more data helps, but it does not got to zero but approaches a finite plateau

The total BNN uncertainty actually consists of two terms: (i) the "statistical uncertainty" (red curve) accounts for a lack of knowledge due to a limited amount of training data and vanishes in the limit of infinite training data, and...

The improvements of the relative local energy resolution when evaluated as a function of the in-time (left) and out-of-time (right) pile-up activity shows a significant level of cluster-by-cluster pile-up mitigation when applying the ML-derived calibration

Another performance measure is the relative energy resolution — again, the BNN is better over the whole energy range, and especially spectacular at low energies (it best learns the signal-source transition from inelastic hadronic interactions to ionisation-dominated signals)

To evaluate the performance, we compare the BNN predictions to the target values in terms of the signal linearity (should peak at zero) — the BNN calibration performs significantly better than any other of the considered scales (with a significant precision gain at low energies)

The corresponding loss function then follows a clear statistics logic: the first term can be seen as a weight regularization avoiding over-training, and the second term tries to maximize the likelihood

The network training can be described as constructing a #variational approximation, where we approximate the intractable posterior with a simplified and tractable distribution — to learn the variational posterior we minimize the Kullback-Leibler (KL) divergence

BNN weights are not trained as fixed values, the parameters are described by weight distributions — during inference, the learned weight distributions are sampled multiple times to generate an ensemble of networks, from which we construct the central value and the uncertainty

Since control and #uncertainties are key in #HEP, our BNN is trained to also learn an uncertainty associated with the predicted #calibration function — this uncertainty allows for a better understanding of possible signal-quality issues in the data or training-related limitations

#ML can be used to learn multi-dimensional continuous #calibration functions — we do this by training a #regression network (an uncertainty-aware BNN) to learn the "response" of single topo-clusters as a function over feature space using a properly defined minimization task

The principal signals of the #ATLAS calorimeters are so-called "topo-clusters". The signals are calibrated to correctly measure the energy deposited by EM showers — but therefore they provide no compensation for energy losses in the complex development of hadronic showers...

So when used correctly, #ML is a perfect tool to quantify and control different kinds of #uncertainties in #LHC physics — and the future of the LHC really is triggered, inspired and shaped by data science as a new common language of particle experiment and theory

TL;DR: Neural networks are extremely powerful numerical tools that offer a lot of control: (i) an appropriate and well-defined loss function tells us exactly what the network training is trying to achieve; (ii) uncertainty-aware networks (like Bayesian NNs) come with an error bar on their output

Welcome to the machine!

After more than one year, the time has finally come: the @uniheidelberg.bsky.social non-ATLAS HEP-ML group has published its first (preprint) paper in collaboration with the @atlasexperiment.bsky.social

arxiv.org/abs/2412.04370
inspirehep.net/literature/2...