The study is focused on the creation and evaluation of STORY ANALOGY, a novel large-scale corpus for assessing the analogy-making abilities in narratives of large language models. The researchers designed tests on this corpus and found that identifying analogies is a challenging task for language models, including recent ones like ChatGPT and LLaMa. However, they found that using the data in STORY ANALOGY can improve the quality of analogy generation by these models. The study also extends the Structure-Mapping Theory to establish evaluation on longer texts.
Publication date: 20 Oct 2023
Project Page: https://github.com/loginaway/StoryAnalogy
Paper: https://arxiv.org/pdf/2310.12874
