1: \begin{abstract}

2: Hardware implementation of neural network are an essential step to implement next generation efficient and powerful artificial intelligence solutions.

3:  Besides the realization of a parallel, efficient and scalable hardware architecture, the optimization of the system's extremely large parameter space with sampling-efficient approaches is essential.

4:  Here, we analytically derive the scaling laws for highly efficient Coordinate Descent applied to optimizing the readout layer of a random recurrently connection neural network, a reservoir.

5:  We demonstrate that the convergence is exponential and scales linear with the network's number of neurons.

6:  Our results perfectly reproduce the convergence and scaling of a large-scale photonic reservoir implemented in a proof-of-concept experiment.

7:  Our work therefore provides a solid foundation for such optimization in hardware networks, and identifies future directions that are promising for optimizing convergence speed during learning leveraging measures of a neural network's amplitude statistics and the weight update rule.

8: \keywords{Reservoir Computing \and Hardware implementation \and Photonics \and Stochastic Coordinate Descent \and Convergence Analysis.}

9: \end{abstract}

10: