This article discusses the use of semi-supervised learning and a teacher-student training approach to improve Music Information Retrieval (MIR) tasks. The authors scaled up unlabeled music data to 240k hours for training, a size much larger than any public MIR datasets. They created and refined pseudo-labels in the noisy teacher-student training process and explored knowledge expansion to scale up model sizes. The study found a performance correlation between data size and model size. By scaling up both, their models achieved state-of-the-art results on several MIR tasks.
Publication date: 4 Oct 2023
Project Page: Not Provided
Paper: https://arxiv.org/pdf/2310.01353
