The article presents GEMFormer, a two-stage method that improves the performance of multi-hop question answering (MHQA) tasks. MHQA involves processing a long document and reasoning over its multiple parts. Existing approaches either focus on extracting supporting evidence or rely on the attention mechanism of long input encoding models. However, these approaches lack explicit global contextual information. GEMFormer addresses this by first collecting relevant information from the entire document to memory, then combining it with local context to solve the task. The study found that fine-tuning a pre-trained model with memory-augmented input improves the model’s performance on three MHQA datasets.
Publication date: 1 Dec 2023
Project Page: https://github.com/Aloriosa/GEMFormer
Paper: https://arxiv.org/pdf/2311.18151
