Understanding Adam Optimizer via Online Learning of Updates: Adam is FTRL in Disguise
This paper offers a different understanding of the Adam optimizer, a popular choice in deep neural network training. Despite its practical success, the theoretical understanding of Adam’s algorithmic components has…
Continue reading