The article discusses the use of Supervised Finetuning (SFT) on instruction datasets for improving the performance of large language models (LLMs). However, the high cost of annotation for quality responses is a challenge. Active learning has been effective in identifying useful subsets for annotation, but is computationally expensive. The authors propose an experimental design framework to mitigate these costs, by selecting the most informative samples to label. This approach is found to provide significant gains in label efficiency with less computational overhead, achieving similar performance with only half the annotation cost of random sampling.
Publication date: 15 Jan 2024
Project Page: Not Provided
Paper: https://arxiv.org/pdf/2401.06692
