Abstract

Many everyday objects require the use of complex interactions that are challenging with robot manipulators, especially for contact-rich behaviors such as rotating a cap. We present a method where object object-centric motion trajectories are defined by a sequence of wrenches, enabling the agent these interactions efficiently in differentiable simulation environments. Our approach, DexMOTS, integrates task-specific information with a differentiable simulator to provide policy gradients which backpropagate through an environment dynamics model to efficiently learn a goal-conditioned controller. Concretely, the trajectory of object pose and wrench requirements gives the necessary information to handle varying friction or damping forces, such as in the case of a tightening screw cap, or when pushing against a spring-loaded lever. We develop a set of dexterous manipulation tasks controlling operable objects with joints of varying stiffnesses, shapes, and functional task requirements to highlight the benefits of our approach. % model-free and model-based methods. Then, we demonstrate how object-centric motion trajectories can be leveraged to learn dexterous motion policies for different hand morphologies. We empirically validate our method against a set of model-based and model-free RL baselines, and show that it achieves up to 40\% higher success rates on a suite of realistic contact-rich manipulation tasks.

Paper

@InProceedings{srinivasan2024dexmots,
    title = {DexMOTS: Learning Contact-Rich Dexterous Manipulation in an Object-Centric Task Space with Differentiable Simulation},
    author = {Srinivasan, Krishnan and Heiden, Eric and Ng, Ian and Bohg, Jeannette and Garg, Animesh},
    booktitle = {In submission},
    year = {2024},
}

Video

Demo Videos

Method Diagram

Acknowledgements

This template was originally made by Phillip Isola and Richard Zhang for a colorful ECCV project. It was adapted to be mobile responsive by Jason Zhang for PHOSA. The code can be found here.