1: \begin{proof}

2: (1)

3: Algorithm \ref{alg1APP} is obtained by taking a step in each iteration $t$ in the direction of the tangent vector $Z_t$, such that for the coordinates $(i(t),j(t))$ we have $(Z_t)_{ij} = -(\nabla f(U_t))_{ij}$, $(Z_t)_{ji} = -(\nabla f(U_t))_{ji}$ , and $(Z_t)_{kl} = 0$ for all other coordinates $(k,l)$. 

4: 

5:   The sequence of tangent vectors $Z_t \in T_{U_t} \Od$ is easily seen to be gradient related: $\limsup{k\rightarrow \infty} \langle \nabla f (U_t), Z_t \rangle < 0$ \footnote{To obtain a rigorous proof we slightly complicated the sampling procedure in line 1 of Algorithm \ref{alg1}, such that coordinates with 0 gradient are not resampled until a non-zero gradient is sampled.}. This follows from $Z_t$ being equal to exactly two coordinates of $\nabla f(U_t)$, with all other coordinates being 0.

6: 

7: 

8: %Noting that the manifold $\Od$ is compact, by theorem 4.3.1 and corrolary 4.3.2 of \cite{absil2009optimization}, setting the step size using the Armijo rule \citep{armijo1966minimization,bertsekas1999nonlinear} assures us %that $\lim_{t \to \infty} ||\nabla f (U_t) ||  = 0$.

9: Using the optimal step size as we do assures at least as large an increase $f(U_t) - f(U_{t+1})$ as using the Armijo step size rule \citep{armijo1966minimization,bertsekas1999nonlinear}. Using the fact that the manifold $\Od$ is compact, we obtain by theorem 4.3.1 and corrolary 4.3.2 of \citet{absil2009optimization} that $\lim_{t \to \infty} ||\nabla f (U_t) ||  = 0$

10: 

11: (2) Since Algorithm \ref{alg1APP} produces a monotonically decreasing sequence $f(U_t)$, and since the manifold $\Od$ is compact, we are in the conditions of Theorems 4.4.1 and 4.4.2 of \citet{absil2009optimization}. These imply that the only critical points which are local minima are asymptotically stable.

12: 

13: \end{proof}

14: