This article introduces a new approach to enhance skeleton-based action recognition using multi-modality self-supervision. The authors propose an Implicit Knowledge Exchange Module (IKEM) that mitigates the spread of incorrect knowledge between low-performing modalities. They also suggest three new modalities to enrich the complementary information between modalities. To maintain efficiency when introducing these new modalities, they propose a novel teacher-student framework for knowledge distillation from secondary to mandatory modalities. This approach is found to be effective in efficiently using skeleton-based multi-modality data.
Publication date: 22 Sep 2023
Project Page: https://github.com/desehuileng0o0/IKEM
Paper: https://arxiv.org/pdf/2309.12009
