Rigorous Numerics for Optimizers
Modern neural networks are typically trained with adaptive optimizers like RMSProp and Adam, which often operate in a chaotic regime called the Edge of Stability. My work uses computer-assisted proofs to rigorously validate quasiperiodic and chaotic structures, and their corresponding routes to chaos in these optimizers. Eventually, I hope to develop a comprehensive theory explaining which mechanisms in an adaptive optimizer drive chaotic behavior. If you're interested, check out these visualizers for RMSProp and Adam.
