README.org

• About This is the supplementary code (in Python 3) for the paper Y. Malitsky and K. Mishchenko "Adaptive Gradient Descent without Descent" (two-column [[https://proceedings.icml.cc/static/paper_files/icml/2020/2854-Paper.pdf][ICML]] or one-column [[https://arxiv.org/pdf/1910.09529.pdf][arxiv]])

The implemented adaptive method is a reliable tool for minimizing differentiable functions. It is among the most general gradient-based algorithms and its fast performance is theoretically guaranteed. The method is merely 2 lines:

#+html:

• Usage There are 5 experiments in total. The first four are provided in the form of a Jupyter notebook and for the neural networks we include a PyTorch implementation of the proposed optimizer.

• [[logistic_regression.ipynb][Logistic regression]]
• [[matrix_factorization.ipynb][Matrix factorization]]
• [[cubic_regularization.ipynb][Cubic regularization]]
• [[linesearch_logistic_regression_w8a.ipynb][Linesearch for logisitic regresion]]
• [[pytorch/optimizer.py][Neural networks]]

• Reference If you find this code useful, please cite our paper: #+BEGIN_SRC @article{malitsky2019adaptive, title={Adaptive gradient descent without descent}, author={Malitsky, Yura and Mishchenko, Konstantin}, journal={arXiv preprint arXiv:1910.09529}, year={2019} } #+END_SRC