This article presents a new task ‘Story-to-Motion’ that aims to generate natural human motion from a story. This task is particularly relevant to the animation, gaming, and film industries. The proposed system reads the text and generates controllable, infinitely long motions and trajectories that align with the input text. The system uses large language models to extract a series of (text, position, duration) pairs from the long text. The system also includes a motion retrieval scheme and a progressive mask transformer to address common artifacts in the transition motion. The system has been evaluated across three distinct sub-tasks and it outperforms previous state-of-the-art motion synthesis methods.
Publication date: 13 Nov 2023
Project Page: https://doi.org/10.1145/3610543.3626176
Paper: https://arxiv.org/pdf/2311.07446
