The article presents the Uncertainty-Aware Language Agent (UALA), a new framework that leverages uncertainty quantification to improve the interaction between language agents and the external world. This approach overcomes the limitations of existing methods, which neglect the concept of uncertainty. UALA has demonstrated significant performance improvements across various tasks and Large Language Model (LLM) sizes, while also reducing reliance on external resources. The study also highlights the unreliability of verbalised confidence of LLMs as a measure of uncertainty.
Publication date: 26 Jan 2024
Project Page: https://uala-agent.github.io
Paper: https://arxiv.org/pdf/2401.14016
