The paper investigates the impact of variations in machine learning training regimes and paradigms on energy consumption. The authors aim to create awareness about the energy implications of general training parameters, including learning rate, batch size, and knowledge transfer. They evaluate several setups with different hyperparameter initializations on two distinct hardware configurations. The study reveals the significant role of energy consumption in machine learning and the need to consider it in developing sustainable machine learning models.
Publication date: 4 Jan 2024
Project Page: Not available
Paper: https://arxiv.org/pdf/2401.01851
