The article introduces RFTrans, an RGB-D-based method for surface normal estimation and manipulation of transparent objects by leveraging refractive flow. The system includes the RFNet, which predicts refractive flow, object mask, and boundaries, and the F2Net, which estimates surface normal from the refractive flow. A global optimization module refines the raw depth and constructs the point cloud with normal. An analytical grasp planning algorithm, ISF, generates the grasp poses. The system outperforms the baseline ClearGrasp in synthetic and real-world benchmarks and achieves an 83% success rate in a real-world robot grasping task.
Publication date: 22 Nov 2023
Project Page: https://sites.google.com/view/rftrans
Paper: https://arxiv.org/pdf/2311.12398
