1: %%
2: %% S is unused
3: %%
4:
5: \newcommand\sss{\boldsymbol{\mathit{s}}}
6: \def\approximation#1{\tilde{#1}}
7:
8: \section{One-Shot Algorithms}\label{sec:}
9:
10: Our first algorithm constructs
11: a preconditioner $B$ for the matrix $A$,
12: performs a partial Cholesky factorization
13: of $B$ by eliminating the vertices in trim order
14: to obtain $B = L [I, 0; 0, A_{1}] L^{T}$,
15: performs a further Cholesky factorization of
16: $A_{1}$ into $L_{1} L_{1}^{T}$,
17: and applies the preconditioned Chebyshev algorithm.
18: In each iteration of the preconditioned Chebyshev
19: algorithm, we solve the linear systems in $B$
20: by back-substitution through the Cholesky factorizations.
21:
22: %Our one-shot algorithms all proceed by constructing
23: % a preconditioner $B$ for the matrix $A$,
24: % performing a partial Cholesky factorization
25: % of $B$ by eliminating the vertices in trim order
26: % to obtain $B = L [I, 0; 0, A_{1}] L^{T}$,
27: % performing a further Cholesky factorization of
28: % $A_{1}$ into $L_{1} L_{1}^{T}$,
29: % and applying the preconditioned Chebyshev algorithm.
30: %In each iteration of the preconditioned Chebyshev
31: % algorithm, we solve the linear systems in $B$
32: % by back-substitution through the Cholesky factorizations.
33:
34: \begin{theorem}[One-Shot]\label{thm:oneShot}
35: Let $A$ be an $n$-by-$n$ PSDDD matrix with
36: $m$ non-zero entries.
37: Using a single application of our preconditioner,
38: one can solve the system $A \xx = \bb$
39: to relative accuracy $\epsilon $
40: in time
41: $\bigO{m^{18/13 + o (1)} \log (\kappa (A)/ \epsilon )}$.
42: Moreover, if
43: if the sparsity graph of $A$ does not contain
44: a minor isomorphic to the complete graph on $m^{\theta }$
45: vertices,
46: or if it has genus at most $m^{2\theta }$, for
47: $\theta < 1/3$,
48: then the exponent of $m$ can be reduced to
49: $1.125 (1 + \theta ) + o (1)$.
50: \end{theorem}
51: \begin{proof}
52: The time taken by the algorithm is the sum of the time required
53: to compute the preconditioner, perform the partial
54: Cholesky factorization of $B$,
55: pre-process $A_{1}$ (either performing Cholesky factorization
56: or inverting it),
57: and the product of the number of iterations and the time
58: required per iteration.
59: In each case, we will set $t = m^{\gamma }$ for some constant
60: $\gamma $, and note that the number of iterations will be
61: $m^{(1 - \gamma )/2 + o (1)}$,
62: and that the matrix $A_{1}$ will depend on $m^{\gamma }$.
63:
64: If we do not assume that $A$ has special topological structure,
65: then $A_{1}$ is a matrix on $m^{2 \gamma +o (1)}$ vertices.
66: If we solve systems in $A_{1}$ by Cholesky factorization, then
67: it will take time $\bigO{m + m^{6 \gamma +o (1)}}$
68: to perform the factorization
69: and time $\bigO{m + m^{4 \gamma +o (1)}}$ to solve each system.
70: So, the total time will be
71: $m^{(1 - \gamma )/2 + 4 \gamma + o (1)} + m^{6 \gamma +o (1)}$.
72: Setting $\gamma = 3/13$, we obtain the first result.
73:
74: %If we use an $\bigO{n^{\omega }}$ time
75: % matrix inversion algorithm to invert $A_{1}$,
76: % and use this inverse to solve systems in $A_{1}$,
77: % it will take time $\bigO{m + m^{2 \omega } + o (1)}$
78: % to pre-process, and the same time as before to
79: % solve the system.
80: %If $\omega \leq 2.75$, then we obtain result $(2)$
81: % by setting $\gamma = 1/4$.
82:
83: If the graph has genus $\theta^{2}$ or does not have
84: a $K_{m^{\theta}}$ minor, or is the Gremban cover
85: of such a graph, then
86: can apply part $(2')$ of Theorem~\ref{thm:main}.
87: Thus,
88: $A_{1}$ is a matrix on $m^{\gamma + \theta +o (1)}$
89: vertices.
90: In the Gremban cover case, the preconditioner is a Gremban
91: cover, and so the partial Cholesky factorization
92: can ensure that $A_{1}$ is a Gremban cover as well.
93: As the Gremban cover of a graph has a similar family
94: of separators to the graph it covers, in either
95: case we can apply the algorithm of Lipton, Rose
96: and Tarjan to obtain the Choleksy factorization of
97: $A_{1}$.
98: By Theorem~\ref{thm:nestesdissection}, with
99: $\alpha = 1/2 + \theta $, the
100: time required to perform the factorization
101: will be $\bigO{m + m^{\gamma (3 \theta + 3/2) + o (1)}}$,
102: and the time required to solve the system will
103: be
104: \[
105: \bigO{m^{(1-\gamma )/2} (m + m^{\gamma (2 \theta + 1) + o (1)} }
106: =
107: \bigO{m^{(1-\gamma )/2 + 1 + o (1) }},
108: \]
109: provided $\gamma (2 \theta + 1) \leq 1$.
110: We will obtain the desired result by setting
111: $\gamma = (3 - 9 \theta ) / 4$.
112: \end{proof}
113:
114:
115:
116:
117: \section{Recursive Algorithms}\label{sec:recursive}
118:
119: We now show how to apply our algorithm recursively to improve
120: upon the running time of the algorithm presented in
121: Theorem~\ref{thm:oneShot}.
122:
123: For numerical reasons, we will use partial $LDL^{T}$-factorization in this section
124: instead of partial Cholesky factorizations.
125: We remind the reader that the $LDL^{T}$-factorization of a matrix
126: $B$ is comprised of a lower-triangular matrix $L$ with ones
127: on the diagonal, and a diagonal matrix $D$.
128: The partial $LDL^{T}$ factorization of a matrix $B_{1}$
129: has the form
130: \[
131: B_{1} =
132: L
133: \left(\begin{array}{ll}
134: D & 0 \\
135: 0 & A_{1}
136: \end{array}\right)
137: L^{T},
138: \]
139: where $D$ is diagonal $L$ has the form
140: \[
141: L = \left(\begin{array}{ll} L_{1,1} & 0 \\
142: L_{2,1} & I\end{array}\right)
143: \]
144: and $L_{1,1}$ has 1s on the diagonal.
145:
146: The recursive algorithm is quite straightforward:
147: it first constructs the top-level
148: preconditioner $B_{1}$ for matrix $A_{0} = A$.
149: It then eliminates to vertices of $B_{1}$ in
150: the trim order to obtain the partial $LDL^{T}$-factorization
151: $B_{1} = L_{1} C_{1}L_{1}^{T}$, where
152: $C_{1} = [D_{1},0;0,A_{1}]$.
153: When an iteration of the preconditioned Chebyshev algorithm
154: needs to solve a linear system in $B_{1}$,
155: we use forward- and backward-substitution to solve
156: the systems in
157: $L_{1}$ and $L_{1}^{T}$, but recursively apply
158: our algorithm to solve the linear system in $A_{1}$.
159:
160: We will use a recursion of depth $r$,
161: a constant to be determined later.
162: We let $A_{0} = A$ denote the initial matrix.
163: We let $B_{i+1}$ denote the preconditioner
164: for $A_{i}$,
165: $L_{i} C_{i} L_{i}^{T}$ be the partial $LDL^{T}$
166: factorization of $B_{i}$ in trim order,
167: and $C_{i} = [D_{i}, 0; 0, A_{i}]$.
168: To analyze the algorithm, we must determine
169: the relative error $\epsilon_{i}$ to which
170: we will solve the systems in $A_{i}$.
171: The bound we apply is derived from the following lemma,
172: which we derive from a result of
173: Golub and Overton~\cite{GolubOverton}.
174:
175: \begin{lemma}[Preconditioned Inexact Chebyshev Method]
176: \label{lem:GolubOverton}
177: Let $A$ and $B$ be Laplacian matrices satisfying
178: $\sigma (B, A) \geq 1$.
179: Let $\xx$ be the solution to $A\xx = \bb$.
180: If, in each iteration of the preconditioned Chebyshev Method,
181: a vector $\zz_{k}$ is returned satisfying
182: \[
183: B \zz_{k} = \rr_{k} + \qq_{k}, \mbox{ where $\norm{\qq_{k}}
184: \leq \delta \norm{\rr_{k}}$},
185: \]
186: where
187: $\delta
188: \leq
189: \left(128 \sqrt{\kappa_{f} (B)} \sigma (A,B) \right)^{-1}$,
190: then the $k$-th iterate, $\xx_{k}$, output by the algorithm will satisfy
191: \[
192: \norm{\xx-\xx_{k}}
193: \leq
194: 6 \cdot 2^{- k / \sqrt{\kappa_{f} (A,B)}}
195: \kappa_{f} (A) \sqrt{\kappa_{f} (B)} \norm{\xx}
196: .
197: \]
198: \end{lemma}
199:
200:
201:
202: %Using partial Gaussian elimination of Lemma \ref{lem:Vaidya},
203: % we factor $B_{1}$ into $L_{1}D_{1}L_{1}^{T}$, where
204: % by iteratively eliminating all rows and columns that correspond to
205: % degree 1 or 2 vertices in $G (B)$.
206: %Upon termination, $L_{1}$ is a triangular matrix
207: % with $\sizeof{L_{1}}=O (n)$ and $D_{1}$ is a matrix that can be written
208: % as
209: %\begin{eqnarray}\label{eqn:PartialGaussian}
210: %D_{1} = \left(\begin{array}{ll} I & 0 \\
211: % 0 & A_{1}\end{array}\right),
212: %\end{eqnarray}
213: %where %$I$ is the identity matrix of size equal to the number
214: % of eliminations performed, $0$ is an all zero matrix,
215: % and
216: % $A_{1}$ is a symmetric diagnoally dominant matrix
217: % obtained after partial Gaussian elimination and
218: % and has $O (\sizeof{E_{B}})$ rows.
219:
220: Our main theorem is:
221:
222: \begin{theorem}[Recursive]\label{thm:recursive}
223: Let $A$ be an $n$-by-$n$ PSDDD matrix with $m$
224: non-zero entries.
225: Using the recursive algorithm, one can solve the system
226: $A\xx = \bb $ to relative accuracy $\epsilon $ in
227: time
228: \[
229: \bigO{m^{1.31+o (1)} (\log (\epsilon^{-1} )
230: \log (n \kappa (A)))^{O (1)}}.
231: \]
232: Moreover,
233: if the graph of $A$ does not contain a
234: minor isomorphic to the complete graph on $m^{\theta }$
235: vertices, or has genus at most $m^{2\theta }$,
236: or is the Gremban cover of such a graph,
237: then the exponent of $m$ can be reduced to $1 + 5 \theta + o (1)$.
238: \end{theorem}
239:
240: We note that if $G (A)$ is planar, then the algorithm
241: take time nearly linear in $m$.
242:
243: The following two lemmas allow us to bound the accuracy
244: of the solutions to systems in $B_{i}$ in terms
245: of the accuracy of the solutions to the corresponding systems
246: in $A_{i}$.
247:
248: \begin{lemma}\label{lem:conditionL}
249: Let $L C L^{T}$ be a partial $LDL^{T}$-decomposition
250: of a symmetric diagonally dominant matrix.
251: Then,
252: \[
253: \kappa (L) \leq 2 n^{3/2}.
254: \]
255: \end{lemma}
256:
257: \begin{proof}
258: As $L$ is column diagonally-dominant
259: and has 1s on its diagonal,
260: $\norm{L}_{1} \leq 2$;
261: so, $\norm{L} \leq 2 \sqrt{n}$.
262: By a result of Malyshev~\cite[Lemma~1]{Malyshev},
263: $\norm{L^{-1}} \leq n$
264: (also see Pe{\~n}a~\cite{Pena}).
265: \end{proof}
266:
267:
268:
269: \begin{lemma}\label{lem:partialLDL}
270: Let $B$ be a Laplacian matrix, let
271: $L C L^{T}$ be the partial $LDL^{T}$-factorization obtained
272: by eliminating vertices of $B$ in the trim order.
273: Then, $\kappa (C) \leq \kappa (B)$.
274: \end{lemma}
275: \begin{proof}
276: We recall that $C$ has form
277: \[
278: \left(\begin{array}{ll}
279: D & 0 \\
280: 0 & A_{1}
281: \end{array}\right).
282: \]
283: The factor $A_{1}$ is identical to that obtained from partial
284: Cholesky factorization, so $\kappa (A_{1}) \leq \kappa (B)$
285: follows from Lemma~\ref{lem:partialCholesky}.
286: To now bound $\kappa (C)$, we need merely show that each entry
287: of $D$ lies between the smallest and largest non-zero eigenvalues of
288: $B$.
289: This follows from the facts that the $i$th diagonal of $D$ equals
290: the value of the diagonal of the corresponding vertex
291: in the lower factor right before it is eliminated,
292: this value lies between the smallest and largest non-zero elements
293: of the corresponding factor, and by Lemma~\ref{lem:partialCholesky},
294: these lie between the largest and smallest non-zero eigenvalues of $B$.
295: \end{proof}
296:
297:
298: \begin{lemma}\label{lem:errors}
299: Let $B$ be a Laplacian matrix and let
300: $L C L^{T}$ be the partial $LDL^{T}$-factorization obtained
301: by eliminating vertices of $B$ in the trim order.
302: For any $\cc \in \Span{B}$,
303: let $\sss$ be the solution to $C \sss = L^{-1}\cc $
304: and let $\sst$ satisfy
305: $\norm{\sss -\sst} \leq \epsilon \norm{\sss}$.
306: Let $\approximation{\yy}$
307: be the solution to $L^{T}\approximation{\yy}
308: = \sst$.
309: Then
310: \[
311: \norm{\cc - B \yyt } \leq \epsilon\kappa (L)\kappa (C) \norm{\cc }.
312: \qedhere
313: \]
314: \end{lemma}
315:
316: \begin{proof}[Proof of Lemma~\ref{lem:errors}]
317: First, note that $\cc - B \yyt = L C (\sss- \sst)$
318: and $\cc = L C \sss $.
319: Moreover, $L^{-1}\cc $ must lie in $\Span{C}$.
320: Thus, $\norm{C \sss} \geq \lambda_{2} (C) \norm{\sss }$, and
321: so
322: \[
323: \norm{C (\sss - \sst)} \leq \epsilon \kappa_{f} (C) \norm{C \sss }.
324: \]
325: As $L$ is non-degenerate, we may conclude
326: \[
327: \norm{L C (\sss - \sst)}
328: \leq \epsilon \kappa (L) \kappa_{f} (C) \norm{L C \sss }.
329: \]
330: \end{proof}
331:
332:
333: %\begin{lemma}\label{lem:errors}
334: %Let $L D L^{T}$ be a partial Cholesky
335: % decomposition of $B$ where
336: %\[
337: %L = \left(\begin{array}{ll} L_{1,1} & 0 \\
338: % L_{2,1} & I\end{array}\right) \quad \mbox{and} \quad
339: %D = \left(\begin{array}{ll} I & 0 \\
340: % 0 & A\end{array}\right).
341: %\]
342: %For any $\cc $, let $(\sss_{1},\sst_{2}})^{T}$ be
343: % an approximate solution to the linear system
344: %\begin{eqnarray*}
345: %\left(\begin{array}{ll} I & 0 \\
346: % 0 & A\end{array}\right) \left(
347: % \begin{array}{c}\sss_{1}\\\sss_{2}\end{array}\right) = L^{-1}\cc
348: %\end{eqnarray*}
349: %such that $\norm{\sss_{2} -\sst_{2} }} \leq \epsilon \norm{\sss_{2}}$.
350: %Let $\approximation{\yy}$
351: %be the solution to $L^{T}\approximation{\yy}
352: % = (\sss_{1},\sst_{2}})^{T}$.
353: %Then
354: %\[
355: %\norm{\cc - B\approximation{\yy}} \leq ???\epsilon \norm{\cc }.
356: %\]
357: %\end{lemma}
358:
359:
360: %To prove this lemma, we will use the following proposition about norms
361: % of matrices and vectors.
362:
363: %\begin{proposition}\label{prop:normPartialCholesky}
364: %For any symmetric positive definite matrix $B$,
365: % let $B = LDL$ be a partial Cholesky factorization of $B$
366: % by repeatedly eliminating rows in $B$ of degree 1 or 2.
367: %Then
368: %\[
369: %\kappa (L) \geq \max B / ???n.
370: %\]
371: %\end{proposition}
372:
373: %\begin{lemma}[Error Bounds]\label{lem:errors}
374: %In solving $B_{i}$, suppose we apply
375: % partial symmetric Gaussian elimination to
376: % factor $B_{i}$ into
377: %\begin{eqnarray}\label{eqn:PartialGaussian}
378: %B_{i} = L_{i}\left(\begin{array}{ll} I & 0 \\
379: % 0 & A_{i}\end{array}\right)L_{i}^{T}.
380: %\end{eqnarray}
381: %For any linear system
382: % let
383:
384: %\begin{eqnarray}\label{eqn:PartialGaussian}
385: %\left(\begin{array}{ll} I & 0 \\
386: % 0 & A_{i}\end{array}\right) \left(
387: % \begin{array}{c}\sss'\\\sss\end{array}\right) = L_{i}^{-1}\cc
388: %\end{eqnarray}
389: %Suppose we have an inexact method to find a solution
390: % $\sst}$ of $A_{i}\sss =\cc'$
391: % to $\epsilon_{i}$-precision.
392: %Assume $\sss$ is the exact solution to $A_{i}\sss =\cc'$.
393: %Then we can bound the residue error of the induced solution
394: %$\approximation{\yy} =
395: %L_{i}\left(\begin{array}{c}\sss'\\\sss\end{array}\right)$ by
396: %\[
397: %\norm{L_{i}\left (\begin{array}{c} 0\\
398: % \sss-\sst}\end{array}\right)}
399: %\leq \norm{L_{i}}\norm{\sss-\sst}}
400: %\]
401: %\end{lemma}
402:
403: %Thus, $\norm{\cc-B\approximation{\yy}} \leq
404: %\norm{A}\norm{\sss_{2}-\sst_{2}}}
405: % \leq \epsilon \norm{A}\norm{\sss_{2}}$.
406:
407:
408:
409:
410:
411:
412: \begin{proof}[Proof of Theorem~\ref{thm:recursive}]
413: For $A_{0}, \ldots , A_{r}$,
414: $B_{1}, \ldots , B_{r}$,
415: $C_{1}, \ldots , C_{r}$,
416: and $L_{1}, \ldots , L_{r}$ as defined above,
417: we can apply
418: Lemma~\ref{lem:partialLDL} and Theorem~\ref{thm:main}
419: to show:
420: \begin{itemize}
421: \item $\kappa _{f} (A_{i}) \leq \kappa _{f} (B_{i})
422: \leq m^{i (1+o (1))} \kappa _{f} (A)$,
423: \item $\kappa _{f} (B_{i}) \leq m^{1+o (1)} \kappa _{f} (A_{i-1})
424: \leq m^{i (1+o (1))} \kappa _{f} (A)$
425: \end{itemize}
426:
427: In the recursive algorithm we will solve systems
428: in $A_{i}$, for $i \geq 1$, to accuracy
429: \[
430: \epsilon_{i} =
431: \left(128 m^{i (1+o (1))} (2 n^{3/2} \kappa (A)) \right)^{-1}.
432: \]
433: By Lemma~\ref{lem:errors} and the above bounds,
434: we then obtain solutions to
435: the systems in $B_{i}$ to sufficient accuracy
436: to apply Lemma~\ref{lem:GolubOverton}.
437:
438: Let $m_{i}$ be the number of edges of $A_{i}$.
439: When constructing the preconditioner, we
440: set $t_{i} = (m_{i})^{\gamma }$, for a
441: $\gamma$ to be chosen later.
442: Thus, by Theorem~\ref{thm:main}
443: and Proposition~\ref{pro:trim},
444: $m_{i} \leq m^{(2\gamma)^{i}}$,
445: and
446: $\kappa_{f} (A_{i}, B_{i+1}) = m^{(2\gamma)^{i} (1-\gamma) + o (1)}$.
447:
448: We now prove by induction that the running time
449: of the algorithm obtained from a depth $r$
450: recursion is
451: \[
452: \bigO{m^{\beta_{r} + o (1)} \left(r \log (n \kappa (A) \right)^{r}},
453: \mbox{ where }
454: \]
455: \[
456: \beta_{r} \defeq
457: (\frac{1-\gamma}{2})
458: \sum_{i=1}^{r} (2 \gamma)^{i-1}
459: + 2 (2\gamma)^{r},
460: \]
461: and $\gamma \defeq (3 - \sqrt{5})/2$.
462: In the limit,
463: $\beta_{r}$ approaches
464: $\beta_{\infty} \defeq (3 + \sqrt {5})/4$ from above.
465: The base case, $r = 1$,
466: follows from Theorem~\ref{thm:oneShot}.
467:
468: The preprocessing time is negligible as
469: the partial Cholesky factorizations used to produce
470: the $C_{i}$ take linear time,
471: and the full Cholesky factorization is only
472: performed on $A^{r}$.
473:
474: Thus, the running time is bounded by the
475: iterations.
476: The induction follows by
477: observing that the iteration time
478: is
479: $m^{\frac{1-\gamma}{2} + o (1)}
480: \left(m + m_{1}^{\beta_{r-1}} \right)
481: \log \left(\kappa (A_{r}) \kappa (B_{r}) / \epsilon_{r} \right)
482: $, which proves the inductive hypothesis because
483: $ m_{1}^{\beta_{r-1}} > m$.
484: As $1.31 > \beta_{\infty}$, there exists an
485: $r$ for which $\beta_{r} < 1.31$.
486:
487: When the graph of $A$ does not contain a
488: $K_{m^{\theta}}$ minor or has genus at
489: most $m^{2\theta }$,
490: we apply a similar analysis.
491: In this case, we have
492: $m_{i} \leq m_{i-1} m^{\theta + o (1)}$.
493: Otherwise, our proof is similar, except that
494: we set
495: $\gamma = (3 - \theta - \sqrt{1 + 6 \theta + \theta^{2}})/2$,
496: and obtain
497: $\beta_{\infty} = (3 + \theta + \sqrt{1 + 6 \theta + \theta^{2}})/4$,
498: and note that
499: $\beta_{\infty} \leq 1 + 5 \theta$.
500: \end{proof}
501:
502:
503: % Local Variables: ***
504: % TeX-master:"final.tex" ***
505: % End: ***
506:
507:
508: