1: \begin{abstract}

2: % Deep learing empirically achieves high performance in many applications, but its training dynamics has not been fully understood thoretically. In this paper, we explore theoretical analysis on training two-layer ReLU neural networks in a teacher student regression model, 

3: % in which a student network learns an unkwown teacher network through its outputs. We show that with a specific regularization and sufficient over-parameterizarion, the student network can identify the parameters of the teacher network with high probability

4: % via gradient descent with a norm-dependent stepsize even though the objective function is highly non-convex. The key theoretical tool is the measure representation of the neural networks, and a novel application of a dual certificate argument for sparse estimation on a measure space. We analyze the global minima and global convergence property in the measure space. 

5: Deep learning empirically achieves high performance in many applications, but its training dynamics has not been fully understood theoretically. In this paper, we explore theoretical analysis on training two-layer ReLU neural networks in a teacher-student regression model, in which a student network learns an unknown teacher network through its outputs. We show that with a specific regularization and sufficient over-parameterization, the student network can identify the parameters of the teacher network with high probability via gradient descent with a norm dependent stepsize even though the objective function is highly non-convex. The key theoretical tool is the measure representation of the neural networks and a novel application of a dual certificate argument for sparse estimation on a measure space. We analyze the global minima and global convergence property in the measure space. 

6: \end{abstract}

7: