The article presents SA-MDKIF, a scalable and adaptable framework designed to inject medical domain knowledge into Large Language Models (LLMs). The goal is to improve their performance in medical tasks. The framework consists of two stages: skill training and skill adaptation. The authors define 12 basic medical skills and use AdaLoRA to train these skills. Then, they train the skill router using task-specific downstream data. This router is used to integrate the acquired skills with LLMs during inference. The experimental results show that SA-MDKIF improves performance by 10-20% compared to the original LLMs. The improvement is even more significant for unseen medical tasks, showing an improvement of up to 30%.
Publication date: 1 Feb 2024
Project Page: https://arxiv.org/abs/2402.00474v1
Paper: https://arxiv.org/pdf/2402.00474
