1: \begin{abstract} 

2: 

3:   Kernel machines often yield superior predictive performance on various tasks; however, they suffer from severe computational challenges. In this paper, we show how to overcome the important challenge of speeding up kernel machines. In particular, we develop a parallel block minimization framework for

4: solving kernel machines, including kernel SVM and kernel logistic regression. 

5: Our framework proceeds by dividing the problem into smaller subproblems

6:   by forming a block-diagonal approximation of the Hessian matrix. 

7:   The subproblems are then solved approximately in parallel. 

8:   After that, a communication efficient line search procedure is developed to

9:   ensure sufficient reduction of the objective function value at each iteration. 

10:   We prove global linear convergence rate of the proposed method 

11:   with a wide class of subproblem solvers, and our analysis covers

12:   strongly convex and some non-strongly convex functions. 

13:   We apply our algorithm to solve large-scale kernel SVM problems on distributed 

14:   systems, and show a significant improvement over existing parallel solvers. 

15:   As an example, on the covtype dataset with half-a-million samples, our algorithm 

16:   can obtain an approximate solution with 96\% accuracy

17:   in 20 seconds using 32 machines, while 

18: all the other parallel kernel SVM solvers require more than 2000 seconds to achieve a solution with 95\% accuracy. 

19: Moreover, our algorithm can scale to very large data sets, such as the kdd algebra dataset with 8 million samples and 20 million features. 

20: \end{abstract}