Animesh Garg

We’re really excited about the speakers we have lined up, including Animesh Garg (Georgia Tech) @animesh-garg.bsky.social , Daniel Ho (1X Technologies), Hao Su (UCSD), Katerina Fragkiadki (CMU), Yilun Du (Harvard) @yilundu.bsky.social, and Russ Tedrake (TRI + MIT).

Learning to Simulate Robot Worlds Join the Learning to Simulate Robot Worlds workshop.

🚀 We’re pleased to announce our workshop at CoRL 2025: Learning to Simulate Robot Worlds! Workshop website: simulatingrobotworlds.github.io
The workshop aims to cover topics like physics-grounded simulation, photorealistic digital twins, AI-controlled simulators, to learned neural world models.

This was led by Albert Wilcox with support from Mohamed Ghanem, Masoud Moghani, Pierre Barroso, Benjamin Joffe and myself.
.
Learn more:
🌐 Website: pairlab.github.io/Adapt3R
📄 Paper: arxiv.org/abs/2503.04877
🖥️ Code: github.com/pairlab/Adap...

On our real-world multitask IL benchmark, Adapt3R achieves a strong in-distribution success rate and sees the smallest performance loss by far under a dramatically new viewpoint.

Here we rotate the scene camera by θ radians about the EE starting position, and see that Adapt3R consistently achieves the strongest performance. Notably, it maintains >80% success rate on LIBERO, and has the only nonzero MimicGen success rate when θ ≥ 0.6.

With an unseen embodiment, Adapt3R consistently outperforms its 2D counterparts and DP3, and outperforms 3D Diffuser Actor in two of our four settings. 2D features lifted into 3D are an effective representation for this scenario and Adapt3R makes good use of them!

When we evaluate policies trained with Adapt3R on the multitask LIBERO benchmark or the high-precision tasks from the MimicGen paper, we see that they are just as performant as their RGB counterparts.

Adapt3R unprojects 2D features into a point cloud, transforms them into the end effector’s coordinate frame, and uses attention pooling to condense them into a single conditioning vector for IL. Notice that Adapt3R attends to the same points before and after the camera change!

Learning 3D representations is hard without 3D data
💡 The key idea is to use a 2D foundation model to extract semantic features, and use 3D information to localize those features in a canonical 3D space without extracting any semantic information from the 3D data.

Albert Wilcox has been working on the using canonical 3D reps instead.
Yet, naive 3D alternatives don't work since most of the data is not easily featurized.

Adapt3R is a 3D backbone that works with your favorite robot learning method and generalizes to unseen embodiments & camera viewpoints!

Imitation learning frameworks are often with 2D inputs.
but 2D limits generalization even to camera poses.

This has been an ongoing challenge, especially for humanoids since the camera pose is not steady and need not match the training data.

We build Adapt3R to solve this problem!
Read on for more

@unitreerobotics.bsky.social great job.
Looking forward to more specs on the robot.

Unitree R1 is a new $6K humanoid with a onboard LLM/VLM!

The price & complexity is not a barrier for ML folks to enter robot learning.

Stable, low-cost developer platforms will accelerate humanoid development with new ideas coming from everywhere.

Unitree did it for quadrupeds & now bipeds!

This summer, I've picked up my long-form writing again on my blog at praxiscurrents.substack.com.

I'm starting with a deep dive into the importance of data-driven methods in the robotics -- The age of empiricism in Physical AI

Feel free to visit and subscribe for regular updates.

Huge thanks to the #ICLR2025 Organizing Committee (including many who couldn't make it to the conference) 👏👏👏

Come chat with us at ICLR 2025 today 3-5pm in Hall 3 + Hall 2B

#182 EgoSim: Egocentric Exploration in VirtualWorlds with
Multi-modal Conditioning
egosim.github.io/EgoSim/
Wei Yu

#401 PWM: Policy Learning with Multi-task World Models
imgeorgiev.com/pwm/
Varun Giridhar ncklashansen.bsky.social

To all the folks arguing studying AI safety is scaremongering

Were you tracking the markets yesterday.

Unintended use cases by powerful yet uninformed users can cause immense irreversible damage nearly immediately!

#AI_SafetyMatters!

Humanoid Robotics Startup Agility is Raising $400 Million Agility Robotics, a humanoid robot maker led by former Microsoft executive Peggy Johnson, is raising $400 million at a pre-investment valuation of $1.75 billion, according to a person who has seen the...

A lot of kids who grew up on robotics science fiction are working hard to make it real.
The next few years in robotics 🚀 🔥!

Agility raises 400m on 1.75B pre!

www.theinformation.com/articles/hum...

Congrats to Agilityrobotics team!
@cpaxton.bsky.social, Peggy Johnson, Pras V. , Jonathan H.

Are we there yet? No!
Humanoids will need all the human help they can get

I believe working on this problem might well be one of the coolest things you'll ever do!

I am leading our AI efforts for Humanoids.
I will elaborate in a blogpost in the upcoming weeks.

for now...to the stars and beyond!🚀

It's a multifaceted decadal challenge.
There is a palpable excitement with tremendous progress - we are moving faster than ever before.

Apptronik has the ambition & heft to take on one of most impactful problems of this decade!
Thrilled to join Jeff Cardenas & Nick Paine on this journey

Andrew Barto and Richard Sutton are the recipients of the 2024 ACM A.M. Turing Award for developing the conceptual and algorithmic foundations of reinforcement learning. Andrew Barto and Richard Sutton are the recipients of the 2024 ACM A.M. Turing Award for developing the conceptual and algorithmic foundations of reinforcement learning. In a series of papers beginnin...

This is a moment to celebrate for the long standing foundational contributions to RL by Sutton and Barto

Congrats to them and all their collaborators over the years who made this long line of advancements possible

www.acm.org/media-center...

Check out our website any-place.github.io & paper www.arxiv.org/abs/2502.04531 for more details!

📣 to all our authors & collaborators: @allanzhao.bsky.social, Miroslav Bogdanovic, Chengyuan Luo, Steven Tohme, Kourosh Darvish, @aspuru.bsky.social, Florian Shkurti, @animesh-garg.bsky.social [5/5]

🔍 Check out more at
any-place.github.io
www.arxiv.org/abs/2502.04531

Yuchi Zhao Miroslav Bogdanovic Chengyuan Luo Steven Tohm Kourosh Darvish @aspuru.bsky.social Florian Skhurti & Animesh Garg

@uoftcompsci.bsky.social
@vectorinstitute.ai @accelerationc.bsky.social @gtresearchnews.bsky.social

How well does AnyPlace perform?
🏆 Simulation results: Outperforms baselines in
✔ Success rate
✔ Coverage of placement modes
✔ Fine-placement precision
📌 Real-world results: Our method transfers directly from synthetic to real-world tasks, succeeding where others struggle!

To generalize across objects & placements, we generate a fully synthetic dataset with:
✅ Randomly generated objects in Blender
✅ Diverse placement configurations (stacking, insertion, hanging) in IsaacSim
This allows us to train our model without real-world data collection! 🚀

Our key insight is that by leveraging a Vision-Language Model (VLM) to identify rough placement locations, we focus only on the relevant regions for local placement, which enables us to train the low-level placement-pose-prediction model to capture diverse placements efficiently.

How can robots reliably place objects in diverse real-world tasks?

🤖🔍 Placement is hard! - objects vary in shape & placement modes (such as stacking, hanging, insertion)

AnyPlace predicts placement poses of unseen objects in real-world with ony using synthetic training data!

Read on👇

The Ultra-Scale Playbook - a Hugging Face Space by nanotron The ultimate guide to training LLM on large GPU Clusters

sure you can read the cool new LLM paper.

but, you know what is needed to make it work behind the scenes?
Thousands of GPUs humming in perfect harmony.

Thx @thomwolf.bsky.social & @huggingface.bsky.social for writing such a wonderful treatise

huggingface.co/spaces/nanot...

a cartoon of peter griffin standing in front of a sign that says the beatles ALT: a cartoon of peter griffin standing in front of a sign that says the beatles

We must remind ourselves that those who hold views contrary to one’s own are rarely evil or stupid, and may know or understand things that we do not.

news.stanford.edu/stories/2017...

Resist intellectual monoculture 🙏🏼💪

I completely agree, however the synthesis often both requires a science and leads to more interesting questions

I often find myself defending CS systems research because of the outsized contributions they have in moving the field forward

It should not be relegated to industry