1: \begin{abstract}

2: We consider nonlinear convergence acceleration methods for fixed-point iteration $x_{k+1}=q(x_k)$, including Anderson

3: acceleration (AA), nonlinear GMRES (NGMRES), and Nesterov-type acceleration (corresponding to AA with window size one).

4: We focus on fixed-point methods that converge asymptotically linearly with convergence factor $\rho<1$

5: and that solve an underlying fully smooth and non-convex optimization problem.

6: %

7: It is often observed that AA and NGMRES substantially improve the asymptotic convergence behavior of the

8: fixed-point iteration, but this improvement has not been quantified theoretically. We investigate this problem

9: under simplified conditions.

10: %

11: First, we consider stationary versions of AA and NGMRES, and determine coefficients that result

12: in optimal asymptotic convergence factors, given knowledge of the spectrum of $q'(x)$ at the fixed point $x^*$.

13: This allows us to understand and quantify the asymptotic convergence improvement that can be provided by

14: nonlinear convergence acceleration, viewing $x_{k+1}=q(x_k)$ as a nonlinear preconditioner for AA and NGMRES.

15: %

16: Second, for the case of infinite window size, we consider linear asymptotic convergence bounds for GMRES

17: applied to the fixed-point iteration linearized about $x^*$. Since AA and NGMRES are equivalent to GMRES

18: in the linear case, one may expect the GMRES convergence factors to be relevant for AA and NGMRES as

19: $x_k \rightarrow x^*$.

20: %

21: Our results are illustrated numerically for a class of test problems from canonical tensor decomposition, comparing

22: steepest descent and alternating least squares (ALS) as the fixed-point iterations that are accelerated by AA and NGMRES.

23: Our numerical tests show that both approaches allow us to estimate asymptotic convergence speed for nonstationary

24: AA and NGMRES with finite window size.

25: %

26: \end{abstract}

27: