The GENSIM project proposes an approach to generate rich simulation environments and expert demonstrations using large language models (LLM). It aims to address the challenge of task-level diversity in robotic simulation, which has been a limitation of existing methods. GENSIM operates in two modes: goal-directed generation and exploratory generation. In the former, a target task is given to the LLM, which then proposes a task curriculum to solve it. In the latter, the LLM iteratively proposes novel tasks based on previous ones. The use of GPT4 has allowed for a significant expansion of the existing benchmark to over 100 tasks. The study found that LLM-generated simulation programs significantly enhance task-level generalization when used for multitask policy training. Moreover, the multitask policies pretrained on GPT4-generated simulation tasks showed strong transfer to unseen tasks in the real world.
Publication date: 2 Oct 2023
Project Page: https://liruiw.github.io/gensim
Paper: https://arxiv.org/pdf/2310.01361
