The paper proposes OmniPred, a framework for training language models as universal regressors over evaluation data from various real-world experiments. This approach, which uses data from Google Vizier, demonstrates that language models can perform very precise numerical regression using only textual representations of mathematical parameters and values. Furthermore, when given the chance to train over multiple tasks, these models can significantly outperform traditional regression models. This work is crucial for both the traditional field of experimental design and the rapidly evolving field of large language model research.
Publication date: 22 Feb 2024
Project Page: https://github.com/google-research/optformer/tree/main/optformer/omnipred
Paper: https://arxiv.org/pdf/2402.14547
