36dcef9b6f599fb5.tex
1: \begin{abstract}
2: Recently,
3: an idling mechanism has been introduced
4: in the context of distributed \emph{first order}
5: methods for minimization of
6: a sum of nodes' local convex costs
7: over a generic, connected network.
8: With the idling mechanism, each node~$i$,
9: at each iteration~$k$,
10: is active -- updates its
11:  solution estimate and exchanges
12:   messages with its network neighborhood --
13:    with probability~$p_k$,
14:    and it stays idle with probability~$1-p_k$, while
15:    the activations are independent
16:    both across nodes and across iterations.
17: %    The idling mechanism involves an increasing
18: %    number of nodes in the algorithm (on average)
19: %    as the iteration counter~$k$ grows,
20: %    thus avoiding unnecessarily expensive exact updates
21: %    at the initial iterations while performing
22: %     beneficial close-to-exact updates near the solution.
23:       In this paper, we
24:       demonstrate that the
25:       idling mechanism can be successfully
26:      incorporated in \emph{distributed second order methods} also.
27:       Specifically, we apply
28:       the idling mechanism to the recently
29:       proposed Distributed Quasi Newton method~(DQN). We first show theoretically that,
30:       when $p_k$ grows to one across iterations in a controlled manner,
31:       DQN with idling exhibits very similar
32:       theoretical convergence and convergence rates
33:  properties as the standard DQN method, thus
34:   achieving the same order of
35:  convergence rate~(R-linear) as the standard DQN, but  with
36:  significantly cheaper updates.
37:  Simulation examples confirm the benefits of
38:  incorporating the idling mechanism, demonstrate
39:  the method's flexibility with respect to
40:  the choice of the~$p_k$'s, and compare the proposed idling method with
41:  related algorithms from the literature.
42: \end{abstract}
43: