The study delves into the reflection abilities of Large Language Models (LLMs) and the issues encountered due to the unstable nature of self-evaluated or external feedback. The researchers found that LLMs often display overconfidence or high randomness when self-evaluating, leading to stubborn or inconsistent feedback. This inconsistency results in poor reflection. To address this, they propose a method named ‘Self-Contrast’ that explores diverse solving perspectives, contrasts the differences, and summarizes these discrepancies into a checklist. This checklist can be used to re-examine and eliminate inconsistencies, thus enabling LLMs to alleviate stubborn biases and catalyze more accurate and stable reflection. The effectiveness of the Self-Contrast method was tested through a series of reasoning and translation tasks with different LLMs.
Publication date: 5 Jan 2024
Project Page: Not provided
Paper: https://arxiv.org/pdf/2401.02009
