The article discusses the efficiency of black-box prompt search in large language models (LLMs). The authors highlight the importance of search space design and optimization, which has often been overlooked despite extensive research. They reveal that only a small number of tokens have a significant influence on LLM predictions. Based on this, they propose the Clustering and Pruning for Efficient Black-box Prompt Search (CLAPS), a method that clusters and prunes the search space to focus on influential prompt tokens. CLAPS outperforms complex approaches and reduces search costs. The findings emphasize the importance of search space design and optimization in improving the usefulness and efficiency of black-box prompt-based learning.
Publication date: 19 Oct 2023
Project Page: https://github.com/cambridgeltl/ClaPS
Paper: https://arxiv.org/pdf/2310.12774
