The study discusses the use of neural network pruning as an effective method for compressing a multilingual ASR model with minimal performance loss. The proposed adaptive masking approach results in sparse monolingual models or a sparse multilingual model known as Dynamic ASR Pathways. The approach adapts the subnetwork dynamically, avoiding premature decisions about a fixed subnetwork structure. The authors demonstrate that this approach outperforms existing pruning methods when targeting sparse monolingual models. The study also illustrates that Dynamic ASR Pathways discovers and trains better subnetworks of a single multilingual model, thereby reducing the need for language-specific pruning.
Publication date: 25 Sep 2023
Project Page: Not provided
Paper: https://arxiv.org/pdf/2309.13018
