1: \begin{abstract}

2: We theoretically analyze the Feedback Alignment (FA) algorithm, an efficient alternative to backpropagation for training neural networks. We provide convergence guarantees with rates for deep linear networks for both continuous and discrete dynamics. %optimization flow.

3: Additionally, we study incremental learning phenomena for shallow linear networks. Interestingly, certain specific initializations imply that negligible components are learned {before} the principal ones, thus potentially negatively affecting the effectiveness of such a learning algorithm; a phenomenon we classify as implicit anti-regularization. We also provide initialization schemes where the components of the problem are approximately learned by decreasing order of importance, thus providing a form of implicit regularization. 

4: % The occurrence of drastically opposite regularization by only modifying an algorithm's initialization showcases the critical importance of the latter for machine learning applications.

5: 

6: \end{abstract}

7: