We cover it in the paper to some extent. We found that things work out as long as y resembles an image. Moreover, the encoder is robust to a small amount of noise.
23.01.2025 09:34 — 👍 0 🔁 0 💬 1 📌 0We cover it in the paper to some extent. We found that things work out as long as y resembles an image. Moreover, the encoder is robust to a small amount of noise.
23.01.2025 09:34 — 👍 0 🔁 0 💬 1 📌 0
🙌 This work was led by Ron Raphaeli under the guidance of Prof. Miki Elad.
🧵[7/7]
✨ The result?
✅ Sharper images
✅ Significant speedups
✅ A simple framework for inverse problems with latent diffusion priors.
🧵[6/7]
🔥 Our solution: What if we could bypass the decoder entirely?
We designed a latent operator that mimics image-space degradations directly in the latent space, eliminating the use of the decoder and its Jacobian.
🧵[5/7]
⚠️ Worse, backpropagating through the decoder introduces artifacts into the restored images due to its Jacobian.
🧵[4/7]
💡 The challenge: Solving inverse problems with latent diffusion models is tricky because degradation operators (e.g., blur, noise) are defined in image space.
This forces costly decoding steps at every iteration, slowing everything down.
🧵[3/7]
🔗 Project page: ronraphaeli.github.io/SILO-website/
📄 Arxiv: arxiv.org/abs/2501.11746
🧵[2/7]
🚀 Excited to share our latest research: “SILO: Solving Inverse Problems with Latent Operators”!
A surprisingly simple approach to image restoration with latent diffusion models that achieves SOTA results while being 2.5x–10x faster than prior methods.
🧵[1/7]
If we are talking about image-to-image tasks, its seems you need only one:
arxiv.org/abs/2406.00828
