AutoBound computes polynomial upper and lower bounds on a given function, valid in a user-defined interval. The application of AutoBound to deep learning optimizers can be used to derive the meta-optimizer SafeNet which automatically adjust the learning rate (at the cost of an additional forward pass): https://ai.googleblog.com/2023/04/beyond-automatic-differentiation.html

#autobound #safenet #autodiff #jax #ai