Frank Fundel

CleanDIFT: Diffusion Features without Noise CleanDIFT enables extracting Noise-Free, Timestep-Independent Diffusion Features

🧹 CleanDiFT: Diffusion Features without Noise
@rmsnorm.bsky.social*, @stefanabaumann.bsky.social*, @koljabauer.bsky.social*, @frankfundel.bsky.social, Björn Ommer
Oral Session 1C (Davidson Ballroom): Friday 9:00
Poster Session 1 (ExHall D): Friday 10:30-12:30, # 218
compvis.github.io/cleandift/

Our paper is accepted at WACV 2025! 🤗
Check out DistillDIFT. Code & weights are now public:
👉 github.com/compvis/dist...

🔥 We achieve SOTA in unsupervised & weakly-supervised semantic correspondence at just a fraction of the computational cost.

✨ By training just a tiny LoRA adapter, we transfer the power of a large diffusion model (SDXL Turbo) into a small ViT (DINOv2).

🔄 All done unsupervised by retrieving pairs of similar images.

🚀 Meet DistillDIFT:
It distills the power of two vision foundation models into one streamlined model, achieving SOTA performance at a fraction of the computational cost.

No need for bulky generative combos—just pure efficiency. 💡

This work was co-lead by: @joh-schb.bsky.social @vtaohu.bsky.social

📷Project Page: compvis.github.io/distilldift
💻Code: github.com/compvis/dist...
📝 Paper: arxiv.org/abs/2412.03512

👇

Did you know you can distill the capabilities of a large diffusion model into a small ViT? ⚗️
We showed exactly that for a fundamental task:
semantic correspondence📍

A thread 🧵👇

Our paper is accepted at WACV 2025! 🤗
Check out DistillDIFT. Code & weights are now public:
👉 github.com/compvis/dist...

🔥 We achieve SOTA in unsupervised & weakly-supervised semantic correspondence at just a fraction of the computational cost.

✨ By training just a tiny LoRA adapter, we transfer the power of a large diffusion model (SDXL Turbo) into a small ViT (DINOv2).

🔄 All done unsupervised by retrieving pairs of similar images.

🚀 Meet DistillDIFT:
It distills the power of two vision foundation models into one streamlined model, achieving SOTA performance at a fraction of the computational cost.

No need for bulky generative combos—just pure efficiency. 💡

📷Project Page: compvis.github.io/distilldift
💻Code: github.com/compvis/dist...
📝 Paper: arxiv.org/abs/2412.03512

👇