Nicola Branchini

Flying today towards Chicago 🌆 for MCM 2025

fjhickernell.github.io/mcm2025/prog...

Will give a talk on our recent/ongoing works on self-normalized importance sampling, including learning a proposal with MCMC and ratio diagnostics.

www.branchini.fun/pubs

Really cool work : ) @alexxthiery.bsky.social

www.tandfonline.com/doi/full/10....

agree; you should check out @yfelekis.bsky.social 's work on this line 😄

Just don't see that the PPD_q of the original post leads somewhere useful.
Anyway, thanks for engaging @alexlew.bsky.social : )

I agree, except I think it can be ok to shift the criteria of "good q" to instead some well-defined measure of predictive performance (under no model misspecification, let's say). Ofc Bayesian LOO-CV is one. We could discuss to use other quantities, and how to estimate them, ofc.

Genuine question: what is the estimated value used for then ?

(computed with the inconsistent method)

Well, re: [choose q1 or q2 based on whether P_q1 > P_q2]
I seem to understand that many VI papers say: here's a new VI method, it produces q1; old VI method gives q2. q1 is better than q2 because test-log PPD is higher !

Not entirely obvious to me, but I see the intuition !

Am definitely at least *trying* to think carefully about the evaluation here 😅 😇

Right ! Definitely not sure if necessary, but I like to think there would be value / would be interesting if we wanted to somehow speak formally about generalizing over unseen test points

It still seems "dangerous" to use the numerical value of (an estimate of) ∫ p(y|θ) q(θ) to decide which approximate q is better.

(Of course, you may argue we maybe shouldn't use even any MC estimates of the original ∫ p(y|θ) p(θ|D) with q as proposal, but the above is even less justified)

I don't see that it needs to get that philosophical ?
It is totally possible to formally estimate the pdf itself, since we have some 'test' samples of y, and consider MISE type errors, even if in this case pointwise evaluations of the pdfs have the intractable integral.

that’s why I like to just say posterior predictive integral(s) instead

This is an aside btw, but nobody in practice actually in principle estimates the marginal ppd density, right ?
Just the pointwise density evaluations at some specific y’s (whose randomness is not taken into account)

(I mean it is obvious in the unnormalized weights case, less so when self-normalizing, which was what I refer to here)

(also, with with approach using q as proposal, it is theoretically possible to choose q that is better than iid sampling from the true posterior ;) )

(not sure where that link came from)

I could even be somewhat ok with an inconsistent estimator, but we can’t then make a decision comparing values with different proposals in any mathematically sound way ?

but then you can just tune the underlying P_q to make it high, by changing / optimizing q !
It doesn’t make any sense

That’s my point about 2.
If you don’t use it as proposal, you have an inconsistent estimator of P*. With the inconsistent approach, you’re changing the true underlying quantity !
(They argue it’s of interest by itself)
This is bad as I see, since people choose q1 or q2 based on whether P_q1 > P_q2

This new ICML paper (openreview.net/forum?id=Tzf...) reinforces and insists on the notion that one would want to replace the posterior predictive P* := ∫ p(y|θ) p(θ|D) ,
with P^{q} := ∫ p(y|θ) q(θ) , with q(θ) ≈ p(θ|D), then estimate _that_ with MC.
You know me. I don't get it.
What do I miss?

It will seem very silly, but every single time I have a poster, I have at least one person being very surprised that you can do better Monte Carlo than "perfect" i.i.d. sampling from p, for E_p[f(θ)] 😃

Here's how the gradient flow for minimizing KL(pi, target) looks under the Fisher-Rao metric. I thought some probability mass would be disappearing on the left and appearing on the right (i.e. teleportation), like a geodesic under the same metric, but I was very wrong... What's the right intuition?

When Does Monte Carlo Depend Polynomially on the Number of Variables? We study the classical Monte Carlo algorithm for weighted multivariate integration. It is well known that if Monte Carlo uses n randomized sample points for a function of d variables then it has error...

link.springer.com/chapter/10.1...

Wish I had found this when I got started : P (or, maybe not)

it is to mimick the name of their coffee place, which is “cafetoria”

As Edinburgh's appointed minister of Gelato by @nolovedeeplearning.bsky.social, I look forward to try 😄

I have cleaned up the notebooks for my course on Optimal Transport for Machine Learners and added links to the slides and lecture notes. github.com/gpeyre/ot4ml

Towards Adaptive Self-Normalized Importance Samplers The self-normalized importance sampling (SNIS) estimator is a Monte Carlo estimator widely used to approximate expectations in statistical signal processing and machine learning. The efficiency of S...

🚨 New paper: “Towards Adaptive Self-Normalized IS”, @ IEEE Statistical Signal Processing Workshop.

TLDR;
To estimate µ = E_p[f(θ)] with SNIS, instead of doing MCMC on p(θ) or learning a parametric q(θ), we try MCMC directly on p(θ)| f(θ)-µ | (variance-minimizing proposal).

arxiv.org/abs/2505.00372

If one of the two distributions is an isotropic Gaussian, then flow matching is equivalent to a diffusion model. This is known as Tweedie's formula. In particular, the vector field is a gradient vector, as in optimal transport. speakerdeck.com/gpeyre/compu...