This paper presents a novel approach for creating a simple, compact, and actionable 3D world representation using 3D primitives from a set of calibrated images of a scene. The method, called Differentiable Blocks World (DBW), focuses on parsing a scene into mid-level 3D representations made of a small set of textured primitives. Unlike existing primitive decomposition methods that rely on 3D input data, DBW operates directly on images through differentiable rendering. The resulting textured primitives accurately model the visible 3D points and provide amodal shape completions of unseen object regions. The approach is robust, interpretable, and suited for physics-based simulations.
Publication date: July 11, 2023
Project Page: https://www.tmonnier.com/DBW/
Paper: https://arxiv.org/pdf/2307.05473.pdf
