The article provides a study on the issue of ’embedding collapse’ in large recommendation models. As recommendation models are scaled up to handle vast amounts of data, the authors observe that the performance of these models does not improve as expected. This is due to the ’embedding collapse’ phenomenon, where the embedding matrix tends to reside in a low-dimensional subspace, hindering scalability. The authors propose a multi-embedding design that incorporates specific interaction modules for each embedding set, aiming to reduce the collapse issue and improve scalability. The proposed design has demonstrated promising results in various recommendation models.
Publication date: 6 Oct 2023
Project Page: https://arxiv.org/abs/2310.04400
Paper: https://arxiv.org/pdf/2310.04400
