Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos
This research paper explores the sharpness dynamics in neural network training. It specifically focuses on the top eigenvalue of the Hessian of the loss, which is referred to as ‘sharpness’….
Continue reading