The article focuses on the limitations of predictive models when adapted into agent-like systems. Two structural reasons for failure are discussed: auto-suggestive delusions and predictor-policy incoherence. Auto-suggestive delusions refer to models failing to imitate agents that generated the training data if the agents relied on hidden observations. Predictor-policy incoherence refers to the issue where models choose actions as if they expect future actions to be suboptimal, leading to overly conservative decisions. The authors propose a solution in the form of a feedback loop from the environment, where the models are re-trained on their own actions.
Publication date: 9 Feb 2024
Project Page: Not Provided
Paper: https://arxiv.org/pdf/2402.05829
