Modern language models incorporate vast amounts of factual knowledge. However, some facts can be incorrectly represented or become obsolete over time, leading to factual inaccuracies. In this paper, the authors argue that updating a single fact can cause a “ripple effect,” introducing the need for the model to update additional related facts. They propose a novel evaluation criterion to consider the implications of an edit on related facts and create a diagnostic benchmark of 5,000 factual edits, RIPPLEEDITS. Their analysis of prominent editing methods using this benchmark indicates that while these methods can effectively modify a single fact, they often fail to address the ripple effects entailed by the update.
Publication date: July 24, 2023
Project Page: https://github.com/edenbiran/RippleEdits/
Paper: https://arxiv.org/pdf/2307.12976.pdf
