Progressive Sharpening

This research paper explores the sharpness dynamics in neural network training. It specifically focuses on the top eigenvalue of the Hessian of the loss, which is referred to as ‘sharpness’….