John Ryan

New paper just dropped! How do you combine pre-trained diffusion models without having to train a new one 🤓?

Turns out you can use our all new Ito density estimator 🔥 to compute densities under a diffusion model efficiently 🚀!

Thank you!

Jet: A Modern Transformer-Based Normalizing Flow In the past, normalizing generative flows have emerged as a promising class of generative models for natural images. This type of model has many modeling advantages: the ability to efficiently compute...

Here it is: arxiv.org/abs/2412.15129

The normalizing flow architecture they cite is from a paper which isn't yet published. Excited to read it though because JetFormer seems to have the largest normalising flows I've even, and they claim to have minimal issues with training stability 👀

DeMo: Decoupled Momentum Optimization Training large neural networks typically requires sharing gradients between accelerators through specialized high-speed interconnects. Drawing from the signal processing principles of frequency decomp...

DeMo was created in March 2024 by Bowen Peng and Jeffrey Quesnelle and has been published on arXiv in collaboration with Diederik P. Kingma, co-founder of OpenAI and inventor of the Adam optimizer and VAEs.

The paper is available here: arxiv.org/abs/2411.19870

And code: github.com/bloc97/DeMo

Some recent discussions made me write up a short read on how I think about doing computer vision research when there's clear potential for abuse.

Alternative title: why I decided to stop working on tracking.

Curious about other's thoughts on this.

lb.eyer.be/s/cv-ethics....

If you are interested in working as a research engineer in ML / Drug Discovery come find me/the team at Isomorphic Labs at Neurips! We are hiring!

The moderation lists you can subscribe to are an interesting mechanism, to make a good one will probably require a decent amount of resources so I hope bluesky will provide some solid ones / fund people who will build & maintain community trusted lists.

✋

Anybody have a bioml starter pack?

ADOPT: Modified Adam Can Converge with Any $β_2$ with the Optimal Rate Adam is one of the most popular optimization algorithms in deep learning. However, it is known that Adam does not converge in theory unless choosing a hyperparameter, i.e., $β_2$, in a problem-depende...

Nice theoretically motivated Adam variant "ADOPT" (remains to be seen if it stands out from the rest of the Adam zoo)
arxiv.org/abs/2411.02853