This paper provides a comprehensive survey on the methods used for training and serving foundation models, which have become vital in areas like natural language processing and visual recognition. The paper discusses the challenges faced due to the growth of these models, such as increased computing power, memory consumption, and bandwidth demands. It also categorizes and examines the state-of-the-art methods used in various aspects such as network, computing, and storage. The paper concludes by providing a perspective on the future development direction of foundation model systems.
Publication date: 8 Jan 2024
Project Page: 10.1109/OJIM.2022.1234567
Paper: https://arxiv.org/pdf/2401.02643
