Collaboration with Boston Dynamics on extending our DextrAH-RGB work on their robot. The head movement is deliberate and learned thru training. Each arm has its own independent controller. Which one to invoke when is determined by its proximity to the object. www.youtube.com/watch?v=dFOb...

Nice substack post that has a funny legend about Brunelleschi and his challenge to make an egg stand on its end. Whoever can make it stand gets to build the dome of santa maria del fiore. www.james-lucas.com/p/it-always-...

We have been using "direct" image to action mapping for a while now and the word "direct" took me back to tracking in SLAM. Back in the 90s many people were looking at recovering camera pose directly from the images rather than doing feature tracking first and then recovering homography afterwards.

Our new work has made a big leap moving away from depth based end-to-end to raw rgb pixels based end-to-end. We have two versions: mono and stereo, all trained entirely in simulation (IsaacLab).

Love the AI cartoons here: timoelliott.com/blog/cartoon...

My workshop talk at CoRL on our work on dexterity that we've been doing for the past 5 years is here docs.google.com/presentation... We have only started to scratch the surface of what we can do with simulations & I hope we can leverage ideas of self-play and alphazero going forward for robotics.