The paper introduces a new method for text generation termed as non-exchangeable conformal nucleus sampling. This method is a novel extension of the conformal prediction framework, which is used for providing predictions with statistical guarantees. The method can be used post-hoc for any model without extra training and it provides token-level, calibrated prediction sets equipped with statistical guarantees. The experiments conducted for machine translation and language modeling have shown promising results. This method presents a more theoretically principled way to perform sampling with conformal guarantees.
Publication date: 1 Feb 2024
Project Page: https://arxiv.org/abs/2402.00707v1
Paper: https://arxiv.org/pdf/2402.00707
