cross-modal re-parameterization

This article presents a novel methodology named Multimodal Pathway which aims to improve transformers using irrelevant data from other modalities. For instance, an ImageNet model can be improved with audio…