CineMPC is an innovative system that autonomously controls a drone to film multiple targets while achieving user-specified aesthetic objectives. Unlike existing solutions, CineMPC includes camera intrinsic parameters in the control loop, which allows for controlling cinematographic effects like focus, depth-of-field, and zoom. The system uses an RGB-D image to estimate the relative poses between the targets and the camera and optimizes a trajectory for the extrinsic and intrinsic camera parameters to meet the user’s artistic and technical requirements. The drone and camera are controlled in a Model Predicted Control loop, re-optimizing the trajectory at each time step in response to current scene conditions. The system has been tested in a photo-realistic simulation and with a real platform, demonstrating its capability to achieve a full array of cinematographic effects.
Publication date: 12 Jan 2024
Project Page: https://github.com/ppueyor/CineMPC
Paper: https://arxiv.org/pdf/2401.05272
