Yue Chen

GitHub - fanegg/Human3R: An unified model for 4D human-scene reconstruction An unified model for 4D human-scene reconstruction - fanegg/Human3R

Code, model and 4D interactive demo now available

🔗Page: fanegg.github.io/Human3R
📄Paper: arxiv.org/abs/2510.06219
💻Code: github.com/fanegg/Human3R

Big thanks to our awesome team!
@fanegg.bsky.social @xingyu-chen.bsky.social Yuxuan Xue @apchen.bsky.social @xiuyuliang.bsky.social Gerard Pons-Moll

GT comparison shows our feedforward method, without any iterative optimization, is not only fast but also accurate.

This is achieved by reading out humans from a 4D foundation model, #CUT3R, with our proposed 𝙝𝙪𝙢𝙖𝙣 𝙥𝙧𝙤𝙢𝙥𝙩 𝙩𝙪𝙣𝙞𝙣𝙜.

Bonus: #Human3R is also a compact human tokenizer!

Our human tokens capture ID+ shape + pose + position of human, unlocking 𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴-𝗳𝗿𝗲𝗲 4D tracking.

#Human3R: Everyone Everywhere All at Once

Just input a RGB video, we online reconstruct 4D humans and scene in 𝗢𝗻𝗲 model and 𝗢𝗻𝗲 stage.

Training this versatile model is easier than you think – it just takes 𝗢𝗻𝗲 day using 𝗢𝗻𝗲 GPU!

🔗Page: fanegg.github.io/Human3R/

Again, training-free is all you need.

Excited to introduce LoftUp!

A strong (than ever) and lightweight feature upsampler for vision encoders that can boost performance on dense prediction tasks by 20%–100%!

Easy to plug into models like DINOv2, CLIP, SigLIP — simple design, big gains. Try it out!

github.com/andrehuang/l...

I was really surprised when I saw this. Dust3R has learned very well to segment objects without supervision. This knowledge can be extracted post-hoc, enabling accurate 4D reconstruction instantly.

Just "dissect" the cross-attention mechanism of #DUSt3R, making 4D reconstruction easier.

#Easi3R is a simple training-free approach adapting DUSt3R for dynamic scenes.

YouTube video by Yue Chen [CVPR 2025] Feat2GS: Probing Visual Foundation Models with Gaussian Splatting

💻Code: github.com/fanegg/Feat2GS
🎥Video: youtu.be/4fT5lzcAJqo?...

Big thanks to the amazing team!
@fanegg.bsky.social, @xingyu-chen.bsky.social, Anpei Chen, Gerard Pons-Moll, Yuliang Xiu

#DUSt3R #MASt3R #MiDaS #DINOv2 #DINO #SAM #CLIP #RADIO #MAE #StableDiffusion #Zero123

Our findings in 3D probe lead to a simple-yet-effective solution, by just combining features from different visual foundation models and outperform prior works.

Apply #Feat2GS in sparse & causal captures:
🤗Online Demo: huggingface.co/spaces/endle...

With #Feat2GS we evaluated more than 10 visual foundation models (DUSt3R, DINO, MAE, SAM, CLIP, MiDas, etc) in terms of geometry and texture — see the paper for comparison.

📄Paper: arxiv.org/abs/2412.09606
🔍Try it NOW: fanegg.github.io/Feat2GS/#chart

How much 3D do visual foundation models (VFMs) know?

Previous work requires 3D data for probing → expensive to collect!

#Feat2GS @cvprconference.bsky.social 2025 - our idea is to read out 3D Gaussains from VFMs features, thus probe 3D with novel view synthesis.

🔗Page: fanegg.github.io/Feat2GS