1: \begin{abstract}

2:   We present an efficient stochastic algorithm (RSG+) for canonical correlation analysis (CCA) using a reparametrization of the projection matrices. We show how this reparametrization (into structured matrices), simple in hindsight,  

3:   directly presents an opportunity to repurpose/adjust 

4:   mature techniques  

5:   for numerical optimization on Riemannian manifolds. 

6:   Our developments nicely 

7:   complement existing methods for this problem which either require

8:   $O(d^3)$ time complexity per iteration with 

9:   $O(\frac{1}{\sqrt{t}})$ convergence rate (where $d$ is the dimensionality)

10:   or only extract the top $1$ component with $O(\frac{1}{t})$ convergence rate.

11:   In contrast, our algorithm offers a strict improvement for this classical problem: it achieves $O(d^2k)$ runtime complexity per iteration for

12:   extracting the top $k$ canonical components with $O(\frac{1}{t})$ convergence rate.

13:   While the paper primarily focuses 

14:   on the formulation and technical analysis of its 

15:   properties, our 

16:   experiments show that 

17:   the empirical behavior on 

18:   common datasets is quite promising. We also explore a potential application in training fair models where the label of protected attribute is 

19:   missing or otherwise unavailable. 

20: \end{abstract}

21: