Description: A simple, lightweight website to share my thoughts and projects
facebook twitter rss github linkedin2 stackoverflow Robindar About Blog Contact David A. R. Robin PhD candidate at ENS Machine Learning Publications [NeurIPS 22] Convergence beyond the overparameterized regime with Rayleigh quotients Neural Information Processing Systems , New Orleans, 2022
Abstract: We present a new strategy to prove the convergence of Deep Learning architectures to a zero training (or even testing) loss by gradient flow. Our analysis is centered on the notion of Rayleigh quotients in order to prove Kurdyka-Lojasiewicz inequalities for a broader set of neural network architectures and loss functions. We show that Rayleigh quotients provide a unified view for several convergence analysis techniques in the literature. Our strategy produces a proof of convergence for various exa
Full text & code : [ OpenReview ] [ Github ]