1: \begin{abstract}

2: 

3: 

4: 

5: The message passing-based graph neural networks (GNNs) have achieved great success in many real-world applications.

6: However, training GNNs on large-scale graphs suffers from the well-known {\it neighbor explosion} problem, i.e., the exponentially increasing dependencies of nodes with the number of message passing layers.

7: Subgraph-wise sampling methods---a promising class of mini-batch training techniques---discard messages outside the mini-batches in backward passes to avoid the neighbor explosion problem at the expense of gradient estimation accuracy.

8: This poses significant challenges to their convergence analysis and convergence speeds, which seriously limits their reliable real-world applications.

9: To address this challenge, we propose a novel subgraph-wise sampling method with a convergence guarantee, namely \textbf{L}ocal \textbf{M}essage \textbf{C}ompensation (LMC).

10: To the best of our knowledge, LMC is the {\it first} subgraph-wise sampling method with provable convergence.

11: The key idea of LMC is to retrieve the discarded messages in backward passes based on a message passing formulation of backward passes.

12: By efficient and effective compensations for the discarded messages in both forward and backward passes, LMC computes accurate mini-batch gradients and thus accelerates convergence.

13: We further show that LMC converges to first-order stationary points of GNNs.

14: Experiments on large-scale benchmark tasks demonstrate that LMC significantly outperforms state-of-the-art subgraph-wise sampling methods in terms of efficiency.

15: 

16: 

17: 

18: \end{abstract}

19: