On the Adam optimizer convergence
Investigation of the convergence of popular optimization algorithms like Adam, RMSProp and propose new variants of these methods which provably converge to optimal solution in convex settings.
Link: https://openreview.net/forum?id=ryQu7f-RZ
PDF: https://openreview.net/pdf?id=ryQu7f-RZ
#iclr2018 #neuralnetworks #optimizers
Investigation of the convergence of popular optimization algorithms like Adam, RMSProp and propose new variants of these methods which provably converge to optimal solution in convex settings.
Link: https://openreview.net/forum?id=ryQu7f-RZ
PDF: https://openreview.net/pdf?id=ryQu7f-RZ
#iclr2018 #neuralnetworks #optimizers
openreview.net
On the Convergence of Adam and Beyond
We investigate the convergence of popular optimization algorithms like Adam , RMSProp and propose new variants of these methods which provably converge to optimal solution in convex settings.