1: \section{Smoothed Analysis of a Two-Phase Simplex Algorithm}
2: \label{sec:phaseI}
3:
4: In this section, we will analyze the smoothed
5: complexity of the {two-phase shadow-vertex simplex method}
6: introduced in Section \ref{sec:introSVM2phase}.
7: The analysis of the algorithm will use as a black-box
8: the bound on the expected
9: sizes of shadows proved in the previous section.
10: However, the analysis is not immediate from this bound.
11:
12: \setcounter{theorem}{0}
13:
14:
15: The most obvious difficulty in applying the shadow bound
16: to the analysis of an algorithm is that, in the statement of
17: the shadow bound, the plane onto which the polytope was projected
18: to form the shadow was fixed, and unrelated to the
19: data defining the polytope.
20: However, in the analysis of the shadow-vertex algorithm,
21: the plane onto which the polytope is projected will
22: necessarily depend upon data defining the linear program.
23: This is the dominant complication in the analysis of
24: the number of steps taken to solve $LP'$.
25:
26: Another obstacle will stem from the fact that, in
27: the analysis of $LP^{+}$,
28: we need to consider the expected sizes of shadows
29: of the convex hulls of points
30: of the form $\aap _{i}/y_{i}^{+}$,
31: which do not have a Gaussian distribution.
32: In our analysis of $LP^{+}$, we essentially handle
33: this complication by demonstrating that in almost every
34: small region the distribution can be approximated by
35: some Gaussian distribution.
36:
37: The last issue we need to address is that if
38: $\smin{\AA _{I}}$ is too small, then the resulting values
39: for $y'_{i}$ and $y^{+}_{i}$ can be too large.
40: In Section~\ref{sec:phaseIManyGood}
41: we resolve this problem by proving that one of $3nd\ln n$
42: randomly chosen $I$ will have
43: reasonable $\smin{\AA_{I}}$
44: with very high probability.
45: Having a reasonable $\smin{\AA_{I}}$ is also essential
46: for the analysis of $LP'$.
47:
48: As our two-phase shadow-vertex simplex algorithm is randomized,
49: we will measure its expected complexity on each input.
50: For an input linear program specified by $\AA$, $\yy $ and $\zz$,
51: we let
52: \[
53: \calC (\AA , \yy ,\zz)
54: \]\index{C@$\calC$}%
55: denote the expected number of simplex steps taken by
56: the algorithm on input
57: $(\AA ,\yy ,\zz)$.
58: As this expectation is taken over the choices for
59: $\calI $ and $\aalpha$,
60: and can be divided into the number of steps taken
61: to solve $LP^{+}$ and $LP'$,
62: we introduce the functions
63: \[
64: \calS'_{\zz} (\AA , \yy ,\calI ,\aalpha),
65: \]\index{Sprime@$\calS'$}%
66: to denote the number of simplex steps taken by the algorithm
67: in step~(5) to solve $LP'$ for a given $\AA$, $\yy$, $\calI $
68: and $\aalpha$,
69: and
70: \[
71: \calSp_{\zz} (\AA , \yy ,\calI) + 2
72: \]\index{Splus@$\calSp$}%
73: to denote the number of simplex steps\footnote{
74: The seemingly odd appearance of $+2$ in this definition
75: is explained by~\ref{pro:lp++}.
76: }
77: taken by the
78: algorithm
79: in step~(7) to solve $LP^{+}$ for a given $\AA$, $\yy$ and $\calI $.
80: We note that the complexity of the second phase does not depend
81: upon $\aalpha$, however it does depend upon $\calI $ as $\calI$ affects
82: the choice of $\kappa $ and $M$.
83: We have
84: \[
85: \calC (\AA , \yy ,\zz)
86: \leq
87: \expec{\calI , \aalpha}{\calS'_{\zz} (\AA , \yy ,\calI ,\aalpha)}
88: +
89: \expec{\calI , \aalpha}{\calSp_{\zz} (\AA , \yy ,\calI ,\aalpha)}
90: + 2.
91: \]
92:
93:
94: \begin{theorem}[Main]\label{thm:twoPhaseMain}
95: There exists a polynomial $\calP$ and a constant $\sigma_{0}$
96: such that for every
97: $n > d \geq 3$,
98: $\orig{\AA} = [\vs{\orig{\aa}}{1}{n}]\in \Reals{n\times d}$,
99: $\orig{\yy} \in \Reals{n}$ and $\zz \in\Reals{d}$,
100: and $\sigma > 0$,
101: \[
102: \expec{\AA ,\yy }{\calC (\AA ,\yy , \zz)}
103: \leq
104: \min \left(\calP (d, n, 1/\min (\sigma ,\sigma_{0})),
105: \binom{n}{d} + \binom{n}{d+1} + 2 \right),
106: \]
107: where
108: $\AA$ is a Gaussian random matrix centered at $\orig{\AA}$
109: of standard deviation $\sigma \max _{i}\norm{(\orig{y}_{i}, \orig{\aa}_{i})}$,
110: and $\yy$ is a Gaussian random vector centered at
111: $\orig{\yy}$ of standard deviation
112: $\sigma \max _{i}\norm{(\orig{y}_{i}, \orig{\aa}_{i})}$.
113: \end{theorem}
114: \begin{proof}
115: We first observe that the behavior of the algorithm is
116: unchanged if one multiplies $\AA$ and $\yy$ by a power of two.
117: That is,
118: \[
119: \calC (\AA ,\yy ,\zz) = \calC (2^{k}\AA ,2^{k}\yy , \zz),
120: \]
121: for any integer $k$.
122: When $\AA$ and $\yy$ are Gaussian random variables
123: centered at $\AAo$ and $\orig{\yy}$ of standard
124: deviation $\sigma \max _{i}\norm{(\orig{y}_{i}, \orig{\aa}_{i})}$,
125: $2^{k} \AA$ and $2^{k} \yy$ are Gaussian random variables
126: centered at $2^{k} \AAo$ and $2^{k} \orig{\yy}$
127: of standard deviation
128: $\sigma \max _{i}\norm{(2^{k}\orig{y}_{i}, 2^{k}\orig{\aa}_{i})}$.
129: Accordingly, we may assume without loss of generality
130: in our analysis that
131: $\max _{i}\norm{(\orig{y}_{i}, \orig{\aa}_{i})} \in (1/2,1]$.
132:
133: The Theorem now follows from
134: Proposition~\ref{pro:trivial} and
135: Lemmas~\ref{lem:LP'} and~\ref{lem:LP+}.
136:
137: %Under this assumption, we set
138: % $\sigma ' = \sigma \max _{i}\norm{(\orig{y}_{i}, \orig{\aa}_{i})}$,
139: % and apply Lemma~\ref{lem:LP'} to
140: % show
141: %\[
142: % \expec{\calI , \aalpha}{\calS'_{\zz} (\AA , \yy ,\calI ,\aalpha)}
143: % \leq \cdots (\sigma ')
144: %\]
145: %We then apply Lemma~\ref{lem:LP+} to show
146: %\[
147: % \expec{\calI , \aalpha}{\calSp_{\zz} (\AA , \yy ,\calI ,\aalpha)}
148: % \leq \cdots (\sigma ').
149: %\]
150: %Combining these two bounds, we obtain
151: %\begin{align*}
152: % \expec{\AA ,\yy }{\calC (\AA ,\yy , \zz)}
153: %& \leq
154: % \expec{\calI , \aalpha}{\calS'_{\zz} (\AA , \yy ,\calI ,\aalpha)}
155: % +
156: % \expec{\calI , \aalpha}{\calSp_{\zz} (\AA , \yy ,\calI ,\aalpha)}
157: % + 2\\
158: %& \leq \cdots
159: %\end{align*}
160: \end{proof}
161:
162: Before proceeding with the proof of Theorem~\ref{thm:twoPhaseMain},
163: we state a trivial upper bound on $\calS'$ and $\calSp$:
164:
165: \begin{proposition}[trivial shadow bounds]\label{pro:trivial}
166: For all $\AA$, $\yy$, $\zz$, $\calI$ and $\aalpha$:
167: \[
168: \calS'_{\zz} (\AA , \yy ,\calI ,\aalpha) \leq
169: \binom{n}{d}
170: \qquad \text{ and } \qquad
171: \calSp_{\zz} (\AA , \yy ,\calI ,\aalpha) \leq
172: \binom{n}{d+1}.
173: \]
174: \end{proposition}
175: \begin{proof}
176: The bound on $\calS'$ follows from the fact that
177: there are $\binom{n}{d}$ $d$-subsets of $[n]$.
178: The bound on $\calSp$ follows from the observation in
179: Lemma~\ref{pro:lp++} that the number of steps taken
180: by the second phase is at most $2$ plus
181: the number of $(d+1)$-subsets of $[n]$.
182: \end{proof}
183:
184:
185: % Local Variables: ***
186: % TeX-master:"shadow.tex" ***
187: % End: ***
188: