The study introduces H-InDex, a learning framework that uses visual representations from human hands to enhance robotic hand dexterity. This method includes three stages: pre-training representations with 3D human hand pose estimation, offline adapting representations with self-supervised keypoint detection, and reinforcement learning with exponential moving average. The researchers found that H-InDex significantly outperformed baseline methods and recent visual foundation models for motor control in 12 challenging dexterous manipulation tasks.
Publication date: 2 Oct 2023
Project Page: yanjieze.com/H-InDex
Paper: https://arxiv.org/pdf/2310.01404
