ForceSight is a system that uses text-guided mobile manipulation to predict visual-force goals using a deep learning network. By combining a single RGBD image with a text prompt, ForceSight can determine a target end-effector pose in the camera frame and the associated forces. The system has been tested in various environments and tasks with a success rate of 81%. The study shows that by ignoring force goals, the success rate drops from 90% to 45%, demonstrating their importance in enhancing performance.
Publication date: 22 Sep 2023
Project Page: https://force-sight.github.io/
Paper: https://arxiv.org/pdf/2309.12312
