The study conducted by Microsoft and the University of Utah investigates the obstacles encountered by data scientists when using Large Language Models (LLMs) like ChatGPT. The research was carried out through observations, interviews, and surveys. Major issues identified include difficulty in retrieving contextual data, formulating prompts for complex tasks, adapting generated code to local environments, and refining prompts iteratively. The authors propose design recommendations such as data brushing for context selection and inquisitive feedback loops for better communication with AI-based assistants.
Publication date: 26 Oct 2023
Project Page: https://arxiv.org/abs/2310.16164v1
Paper: https://arxiv.org/pdf/2310.16164
