In this article, the authors present MINT (Make your model Interactive), a wrapper method designed to optimize the use of multimodal information in AI medical diagnosis. Doctors, during the diagnostic process, use multimodal information, including imaging and medical history. Similarly, medical AI development has increasingly become multimodal. MINT works by determining the most valuable pieces of information at each step and asking for only the most useful information. The study demonstrates MINT’s efficacy by wrapping it around a skin disease prediction model. The results showed that MINT reduces the number of metadata and image inputs needed by 82% and 36.2% respectively, while maintaining predictive performance. The authors also showed that MINT can mimic the step-by-step decision-making process of a clinical workflow.
Publication date: 25 Jan 2024
Project Page: Not provided
Paper: https://arxiv.org/pdf/2401.12032
