The article by Misra, Ettinger, and Mahowald investigates the impact of experimental contexts on the robustness of language models (LMs) in performing semantic property inference. The authors found that LMs can show significant improvements when presented with in-context examples and instructions. However, this ability is inconsistent, with some LMs picking up on shallow, non-semantic heuristics from their inputs. The findings suggest that the computational principles of semantic property inference are yet to be mastered by LMs.
Publication date: 15 Jan 2024
Project Page: arXiv:2401.06640v1
Paper: https://arxiv.org/pdf/2401.06640
