The paper introduces ‘Superfiltering’, a method for data filtering in instruction tuning of Large Language Models (LLMs). Instruction tuning improves LLMs but often suffers from low-quality and redundant data. The traditional filtering process incurs extra cost and computation. Superfiltering proposes using a smaller, weaker model to select data for fine-tuning a larger, stronger model. Despite the performance gap between weak and strong models, the smaller model effectively perceives instruction difficulty and data selection. This speeds up data filtering and improves the performance of the fine-tuned LLM. The approach has been validated through extensive experiments.
Publication date: 1 Feb 2024
Project Page: https://github.com/tianyi-lab/Superfiltering
Paper: https://arxiv.org/pdf/2402.00530
