This research presents an interface based on Variational Autoencoders trained with natural sounds for the innovative creation of Foley effects. The model can transfer new sound features to prerecorded audio or microphone-captured speech in real time. It allows interactive modification of latent variables, facilitating precise and customized artistic adjustments. This innovative approach has been the basis for the artistic creation of the first Spanish short film with sound effects assisted by artificial intelligence, illustrating the transformative potential of this technology in the film industry.
Publication date: 25 Oct 2023
Project Page: Not provided
Paper: https://arxiv.org/pdf/2310.15663
