| Note on Learning Rate Schedules for Stochastic Optimization, (9999) | |||||||||
Abstract | |||||||||
| We present and compare learning rate schedules for stochastic gradient descent, a general algorithm which includes LMS, on-line back-propagation and k-means clustering as special cases. We introduce search-then-converge type schedules which outperform the classical constant and running average (l/t) schedules both in speed of convergence and quality of solution.. This article is from 'Computing Science and Statistics: Proceedings of the Symposium on the Interface Critical Applications of Scientific Computing: Biology, Engineering, Medicine, Speech Held in Seattle, Washington on 21-24 April 1991,' AD-A252 938, p313-317. | |||||||||
Publication details | |||||||||
| |||||||||