1: \begin{abstract}
2: There has been great interest in using tools from
3: dynamical systems and numerical analysis of differential equations
4: to understand and construct new optimization methods.
5: In particular, recently a new paradigm has emerged that applies ideas from mechanics and geometric integration to obtain accelerated optimization methods on Euclidean spaces. This has
6: important consequences given that
7: accelerated methods are the workhorses behind many machine
8: learning applications.
9: In this paper we
10: build upon these advances and propose a
11: framework for dissipative and constrained Hamiltonian systems that
12: is suitable for solving optimization problems on arbitrary smooth manifolds.
13: Importantly, this allows us to leverage
14: the well-established theory of symplectic integration
15: to derive ``rate-matching'' dissipative integrators. This
16: brings a new perspective to optimization on manifolds whereby
17: convergence guarantees follow by construction from classical arguments in symplectic geometry and backward error analysis.
18: Moreover, we construct two dissipative generalizations of leapfrog that are straightforward to implement:
19: one for Lie groups and homogeneous spaces, that relies on the tractable geodesic flow or a retraction thereof,
20: and the other for
21: constrained submanifolds that is
22: based on a dissipative generalization of the famous RATTLE integrator.
23: \end{abstract}
24: