1: \begin{abstract}

2:   We propose a principled framework that combines adversarial training and provable robustness verification for training certifiably robust neural networks.

3:   We formulate the training problem as a joint optimization problem with both empirical and provable robustness objectives and develop a novel gradient-descent technique that can eliminate bias in stochastic multi-gradients. We perform both theoretical analysis on the convergence of the proposed technique and experimental comparison with state-of-the-arts. Results on MNIST and CIFAR-10 show that our method can consistently match or outperform prior approaches for provable $\evl_\infty$ robustness. Notably, we achieve 6.60\% verified test error on MNIST at $\epsilon=0.3$, and 66.57\% on CIFAR-10 with $\epsilon=8/255$.

4: \end{abstract}

5: