1: \begin{abstract}

2: Attempts from different disciplines to provide a fundamental understanding of deep learning have advanced rapidly in recent years,

3: yet

4: a unified framework remains relatively limited.

5: In this article, we provide one possible way to align existing branches of deep learning theory

6: through the lens of dynamical system and optimal control.

7: By viewing deep neural networks as discrete-time nonlinear dynamical systems,

8: we can analyze how information propagates through layers using mean field theory.

9: When optimization algorithms are further recast as controllers,

10: the ultimate goal of training processes can be formulated as an optimal control problem.

11: In addition, we can reveal convergence and generalization properties by studying the stochastic

12: dynamics of optimization algorithms.

13: This viewpoint features a wide range of theoretical study

14: from information bottleneck to statistical physics.

15: It also provides a principled way for hyper-parameter tuning

16: when optimal control theory is introduced.

17: Our framework fits nicely with supervised learning and can be extended to

18: other learning problems, such as Bayesian learning, adversarial training, and specific forms of meta learning,

19: without efforts.

20: The review aims to shed lights on the importance of dynamics and optimal control

21: when developing deep learning theory.

22: \end{abstract}