1: %%%%%%%%%%%%% October 2005 %%%%%%%%%%%
2: %
3: %% Version 1, October 2005.
4: %
5:
6: \documentclass[epsbox,12pt,reqno]{amsart}
7: % reqno: equations numbers on the right for amsart.
8:
9: \usepackage{amsmath,amssymb,latexsym,amscd,amsthm} % ams auxiliaries.
10: \usepackage{graphicx}
11:
12: % \usepackage{times} % Causes problems with accents & spacing in PDF.
13:
14: %%% Custom page size for draft versions.
15: \setlength{\textwidth}{6.5in}
16: \setlength{\textheight}{9.45in}
17: \setlength{\oddsidemargin}{0in}
18: \setlength{\evensidemargin}{0in}
19: \setlength{\voffset}{-0.3in}
20: \setlength{\topmargin}{-0.1in} % Needed this for the ArXiv version
21: \setlength{\headsep}{0.2in}
22:
23: %%% Theorem environments.
24: \newtheorem{theorem}{Theorem}[section] % If subsection, get 3.1.2, etc.
25: \newtheorem{corollary}[theorem]{Corollary}
26: \newtheorem{proposition}[theorem]{Proposition}
27: \newtheorem{lemma}[theorem]{Lemma}
28: \theoremstyle{definition}
29: \newtheorem{definition}[theorem]{Definition}
30: \newtheorem{example}[theorem]{Example}
31: \newtheorem{question}[theorem]{Question}
32: \newtheorem{conjecture}[theorem]{Conjecture}
33: \theoremstyle{remark}
34: \newtheorem{remark}[theorem]{Remark}
35: \numberwithin{equation}{section}
36:
37: %%% HEADERS
38: \title{The Colourful Feasibility Problem}
39:
40: \author{Antoine Deza}
41: \author{Sui Huang}
42: \address{Advanced Optimization Laboratory,
43: Department of Computing and Software,
44: % Faculty of Engineering,
45: % 1280 Main St.~West,
46: McMaster University,
47: Hamilton, Ontario,
48: Canada
49: L8S 4K1.
50: }
51: \email{\{deza,huangs3,terlaky\}@mcmaster.ca}
52: \author{Tamon Stephen}
53: \address{
54: Department of Mathematics,
55: Simon Fraser University,
56: 8888 University Drive,
57: Burnaby, British Columbia, Canada V5A 1S6.
58: }
59: \email{tamon\_stephen@sfu.ca}
60: \author{Tam{\' a}s Terlaky}
61:
62:
63: \dedicatory{Dedicated to Leonid Khachiyan.}
64:
65: \subjclass[2000]{52C45, 68W40, 90C60, 68Q25}
66:
67: %
68: %%% BEGIN HERE %%%
69: %
70: \begin{document}
71:
72: %%%%% Insert abstract here.
73:
74: \begin{abstract}
75: We study a colourful generalization of the linear programming
76: feasibility problem, comparing the algorithms introduced by
77: B{\' a}r{\' a}ny and Onn with new methods.
78: We perform benchmarking on generic and ill-conditioned problems,
79: as well as as recently introduced highly structured problems.
80: We show that some algorithms can lead to cycling or slow convergence,
81: but we provide extensive numerical experiments which show that
82: others perform much better than
83: predicted by complexity arguments. We conclude that the most
84: efficient method for all but the most ill-conditioned problems
85: is a proposed multi-update algorithm.
86: \end{abstract}
87:
88: \newif\ifapps % Include appendices 3 and 4?
89: \appstrue
90:
91: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Insert body here.
92:
93: %%%%%%%%%%%%% November 2005 %%%%%%%%%%%
94: %
95: %% Include file for main template.
96: %
97:
98: %%% Special commands used.
99: \def\sd{\operatorname{depth}} % Simplicial depth.
100: \def\csd{\mathbf{depth}} % Colourful simplicial depth.
101: \def\S{\mathbf{S}} % Configuration.
102: \def\R{\mathbb{R}}
103: \def\core{\operatorname{core}}
104: \def\cone{\operatorname{cone}}
105: \def\conv{\operatorname{conv}}
106: \def\new{\operatorname{new}}
107: \def\aff{\operatorname{Aff}}
108: \def\interior{\operatorname{int}}
109: \def\bara{B\'ar\'any}
110: \def\baraonn{B\'ar\'any and Onn}
111: \def\cara{Carath\'eodory}
112: \def\Sph{\mathbb{S}} % Sphere.
113: \def\zero{{\bf 0}}
114: \def\Deltab{\mathbf{\Delta}}
115: \def\B{{\bf A1}} % Algorithms
116: \def\BO{{\bf A2}}
117: \def\Bp{{\bf A3}}
118: \def\BOp{{\bf A4}}
119: \def\H{{\bf A5}}
120: \def\MV{{\bf A6}}
121: \def\Rand{{\bf A7}}
122: \def\A{{\bf Ax}}
123: \def\rg{{\bf G1}} % Generators
124: \def\tz{{\bf G2}}
125: \def\to{{\bf G3}}
126: \def\Sp{{\bf G4}}
127: \def\Sd{{\bf G5}}
128: \def\Sm{{\bf G6}}
129: \def\Sdelta{$\mathbf{S^{\Delta}}$}
130: \def\Splus{$\mathbf{S^{+}}$}
131: \def\Sminus{$\mathbf{S^{-}}$}
132:
133: \maketitle
134:
135: %
136: % INTRODUCTION
137: %
138: \section{Introduction}
139: Given colourful sets $S_1, \ldots, S_{d+1}$ of points in $\R^d$
140: and a point $p$ in $\R^{d}$, the {\it colourful feasibility problem}
141: is to express $p$ as a convex combination of
142: points $x_1, \ldots, x_{d+1}$ with $x_i \in S_i$ for each $i$.
143: This problem was presented by \bara\ in 1982 \cite{Bar82}.
144: The monochrome version of this problem, expressing $p$ as a
145: linear combination of points in a set $S$, is a traditional
146: linear programming feasibility problem.
147:
148: In this paper, we study algorithms for the colourful feasibility problem
149: with a core condition from an experimental point of view.
150: We learn several things.
151: First this problem is easy in a practical sense --
152: we expend more effort to generate difficult examples than to solve
153: them.
154: Second, while the classical algorithms for this problem already perform
155: quite well, we introduce modifications that achieve a substantial improvement
156: in practical performance.
157: Third, we construct examples where ill-conditioning leads
158: to slow convergence for the some otherwise very effective
159: algorithms.
160: And finally, we remark that a simple greedy heuristic
161: provides competitive results in practice but we find a case where
162: it fails to solve the problem at all.
163: Additionally we provide benchmarking that we hope will encourage
164: research on this attractive problem.
165:
166: \section{Definitions and Background}
167: We concentrate on the important subcase of colourful feasibility problem
168: where we have $d+1$ points of
169: each colour, and $p \in \conv(S_i)$ for $i=1,\ldots,d+1$.
170: We call $\bigcap_{i=1}^{d+1} \conv(S_i)$ the {\it core} of the configuration.
171: We will call such a problem a {\it colourful feasibility} problem,
172: in this paper colourful feasibility problems are
173: assumed to have a non-empty core.
174: In this core case, by \bara's colourful \cara\ theorem \cite{Bar82},
175: a solution is guaranteed to exist, and the problem is to exhibit
176: a solution.
177: Recently \bara's result has been strengthened to show that quadratically
178: many solutions must exist, see \cite{BM06} and \cite{ST06}.
179: The problem of finding a solution to a colourful feasibility problem
180: is described
181: in \cite{BO97} as ``an outstanding problem on the border line
182: between tractable and intractable problems''.
183:
184: Several close relatives of the colourful feasibility problem
185: are known to be difficult.
186: For example, the case where we have $d$ colours in $\R^d$ and no restriction
187: on the size of the sets has been shown to be strongly NP-complete
188: through a reduction of 3-SAT. We refer to \cite{BO97} for more details.
189:
190: In \cite{Bar82}, \bara\ proposed a finite algorithm \B\ to solve
191: colourful feasibility,
192: and in \cite{BO97} \baraonn\ analyzed the complexity of \B\
193: and a second algorithm \BO.
194: Both these algorithms are essentially geometric, and
195: the complexity guarantees depend crucially on having the
196: point $p$ in the {\it interior} of the core.
197: In effect, the distance between $p$ and the boundary
198: of the core can be considered as a measure of the conditioning
199: of the problem. Thus for a configuration
200: $\S$ we define $\rho$ to be the radius of the largest ball around
201: $p$ that is contained in the core. % $\bigcap_{i=1}^{d+1} \conv(S_i)$.
202: The results for \B\ and \BO\ are effectively that they are
203: polynomial in $d$ and $1/\rho$.
204: We remark, though, that for configurations of $d+1$
205: points in $d+1$ colours on the unit sphere $\Sph^d \subseteq \R^d$,
206: $\rho$ will be small even if
207: the problem has a favourable special structure, and quite small
208: otherwise.
209:
210: Without loss of generality, we can take the point $p$ to
211: be the vector $\zero$ in $\R^d$.
212: Some additional preprocessing will be helpful.
213: If $\zero$ is a point
214: in one of the $S_i$'s, then the solution to the colourful
215: feasibility problem is trivial. Otherwise, we can % without loss of generality
216: scale the points of the $S_i$'s
217: so that they lie on the unit sphere $\Sph^d$. The coordinates in
218: any resulting convex combination can then be unscaled as a
219: post-processing step.
220:
221: We call a system of $d+1$ sets of $d+1$ points a {\it configuration},
222: and often denote it as $\S=\{S_1,\ldots,S_{d+1}\}$.
223: We use a the bold font to signal a colourful object, except with
224: $\zero$ where bold is used to distinguish the vector from a scalar.
225: We remark that restricting the sets to have size $d+1$
226: is not a burden since, given a larger set, solving a monochrome linear
227: feasibility problem allows us to efficiently find a basis of size $d+1$
228: with $\zero$ in its convex hull.
229:
230:
231: %
232: % ALGORITHMS
233: %
234: \section{Seven Algorithms}\label{se:algs}
235: In this paper we consider the theoretical and practical performance
236: of seven algorithms for finding a colourful basis.
237: The algorithms considered are the algorithms of \bara\
238: \B\ and \baraonn\ \BO, modifications of these algorithms which
239: update multiple colours at each stage, which we will call \Bp\ and \BOp\,
240: and a hybrid \H\ of these designed to take advantage of the strengths of
241: both algorithms. For purposes of comparison, we also consider
242: two simple approaches that perform well under certain
243: circumstances:
244: a greedy heuristic where we choose the adjacent simplex of
245: maximum volume \MV\
246: and a random sampling approach \Rand.
247: All our implementations are initialized with using the first points from
248: each colour.
249: % Sui Huang has written MATLAB code to implement each of these algorithms.
250: % We have made this freely available on-line \cite{Hua05}.
251: Following are descriptions of the algorithms,
252: see \cite{Hua05} for MATLAB implementations of each.
253: Besides \Rand, they are implemented as
254: pivoting algorithms with the respective pivot selection rule.
255:
256: \subsection{\bara's Algorithm \B}\label{se:b}
257: % We now offer brief descriptions of the algorithms.
258: We begin with the algorithm proposed by \bara\ \cite{Bar82},
259: which is a pivoting algorithm. It begins with say a random colourful
260: simplex $\Deltab$. The point $x$ nearest to $\zero$ in $\Deltab$ is
261: computed. If $x \ne \zero$, then $x$ must lie on some facet of $\Deltab$.
262: Consider the colour $i$ of the vertex of $\Deltab$ that is not
263: on this facet. Look for
264: the point $t$ of colour $i$ minimizing the inner product
265: $\langle t,x \rangle$. Then we replace the point of colour $i$
266: from $\Deltab$ with the point $t$ to get a new simplex.
267: The algorithm then repeats beginning with the new simplex.
268:
269: The convergence of this algorithm relies on the fact that $\zero$
270: is in the core of the
271: configuration. For this reason the affine hyperplane perpendicular
272: to the vector $x$ cannot
273: separate $\zero$ from the points of colour $i$. Thus the next simplex
274: will have a point closer to $\zero$ than $\Deltab$ did, and the algorithm
275: will converge in finitely many steps. If, additionally, the core
276: has radius at least $\rho$ around $\zero$, then there is a guarantee
277: on the amount of progress in a given step, which depends on $\rho$.
278: Effectively the guarantee is that the number of iterations of \B\
279: is $O(1/\rho^2)$. Since an iteration can be done in polynomial
280: time, this proves that \B\ runs in time polynomial in the input data
281: and $1/\rho$.
282: Consult \cite{BO97} for details and a proof.
283:
284: We note that the complexity of a single iteration is dominated
285: by the cost of the nearest point subroutine.
286: This is can be solved as a continuous optimization problem,
287: but complicates our life with numerical issues: It can be
288: solved to less or greater precision, either risking numerical
289: error or increasing the running time. For the purposes of
290: our benchmarking, we used the {\tt MATLAB} built-in {\tt quadprog()}
291: which gave fairly good results,
292: see Section~\ref{se:tpi}.
293:
294: \subsection{\baraonn's Algorithm \BO}\label{se:bo}
295: The reliance of \B\ on nearest point calculations is certainly a
296: disadvantage.
297: Partly motivated by this, \baraonn\ proposed an alternate
298: algorithm for the colourful feasibility problem whose calculations involve
299: only linear algebra.
300: This algorithm, \BO, is described in \cite{BO97}.
301:
302: Essentially, the closest point $x$ to $\zero$ on
303: the simplex $\Deltab$ is replaced in this algorithm by
304: a point $y$ on the boundary of $\Deltab$ that can be computed
305: algebraically. % as a convex combination of the vertices of $\Deltab$.
306: The initial choice of $y$ could be one of the vertices of the initial
307: simplex. In subsequent iterations,
308: a colour $j$ corresponding to a zero coefficient in $y$ is chosen.
309: An improving vertex $v$ of colour $j$ is found, and $y_{\new}$
310: is updated by projecting $\zero$ onto the line segment between
311: $y$ and $v$ and finding where the resulting vector enters the
312: new simplex. As with \B, this algorithm takes $O(1/\rho^2)$
313: iterations, and hence is polynomial in the input data and $1/\rho$,
314: see \cite{BO97}.
315:
316: The implementation of \BO\ proposed in \cite{BO97} takes time
317: $\Theta(d^4)$ for a single iteration. The bottleneck is
318: computing $y_{\new}$, which is the intersection of the line segment
319: from $\zero$ to
320: a point $p$ and the new simplex. In fact we observe that this
321: can be done in time $O(d^3)$.
322: First, compute the defining equations for the simplex $Ay_{\new} \ge b$
323: by inverting the homogenized matrix of the vertices.
324: We know the intersection point will be of the form $y_{\new}= \alpha p$.
325: We can substitute this into the above inequalities to get
326: $\alpha (Ap) \ge b$ and simply take $\alpha$ to be the maximum
327: value of $b_i/A_i p$ for $i=1,2, \ldots, d+1$.
328: This is implemented in \cite{Hua05}.
329:
330: \subsection{Multi-update \bara\ \Bp}\label{se:bp}
331: We are interested in getting practically effective algorithms
332: for the colourful feasibility problem.
333: To that end, we propose the following modification of \B.
334: If it happens that the nearest point $x$ to $\zero$ of the current
335: simplex $\Deltab$ lies on a lower-dimensional face of $\Deltab$
336: - i.e.,~on more than one facet -
337: then we update {\it every} colour that is not a vertex of that face
338: before recomputing $x$. Since all the new points will be on the
339: $\zero$ side of hyperplanes separating $\zero$ and $\Deltab$
340: through $x$, the convergence proofs of \B\ and \BO\ still apply
341: to this algorithm. The advantage of this new algorithm, which
342: we call \Bp, is that when possible it updates several colours
343: without recomputing a nearest point.
344:
345: Since this algorithm makes at least as much progress as \B\ at
346: each iteration, we get convergence in at most the same
347: number of iterations. A given iteration may take longer,
348: since it has to update multiple points. However, aside
349: from the nearest point calculation, all steps in an iteration
350: of \B\ can be performed in $O(d^2)$ arithmetic operations.
351: Hence the additional
352: work per iteration of \Bp\ is $O(d^3)$, and the bottleneck
353: remains the single nearest point calculation.
354:
355: \subsection{Multi-update \baraonn\ \BOp}\label{se:bop}
356: Similarly, we can adjust algorithm \BO\ to update $y$ only after
357: pivoting multiple colours in the case where $y$ lies on a
358: low-dimensional face. This is particularly useful at the
359: start if we use the setup proposed in \cite{BO97} where the
360: initial point $y$ is a vertex of $\Deltab$.
361: We call this algorithm \BOp.
362:
363: As with \Bp, we expect this algorithm to take no more iterations
364: than the algorithm on which it is based, namely \BO.
365: Again we note that all steps in an iteration of \BO\ except
366: for computing the intersection of a line segment and a point
367: take $O(d^2)$ arithmetic operations,
368: so the additional work per iteration of \BOp\
369: as compared to \BO\ is at most $O(d^3)$.
370: Thus an iteration of \BOp\ will be asymptotically at most a
371: constant factor slower than an iteration of \BO.
372:
373: \subsection{Multi-update Hybrid \H}\label{se:h}
374: In Section~\ref{se:results} we describe a situation where
375: \BO\ and \BOp\ make extremely slow progress because they
376: repeatedly return to the same simplex, see the example in
377: Section~\ref{se:mve}.
378: A practical solution to this is to run \BOp, but
379: use a computationally heavy step from \Bp\ if we detect that \BOp\
380: is returning to the same simplex. We implemented such a hybrid
381: algorithm \H.
382:
383: \subsection{Maximum Volume \MV}\label{se:mv}
384: For purposes of comparison, we also consider the performance of
385: a greedy heuristic,
386: where we move from $\Deltab$ to an adjacent simplex of maximum
387: volume given that the pivoting hyperplane separates $\Deltab$
388: from $\zero$. This heuristic, which we call \MV, uses simpler
389: linear algebra than \BO, and by taking large simplices often
390: gets to $\zero$ in a small number of steps.
391:
392: For a given candidate pivoting facet it is possible to choose
393: the point that generates the maximum volume simplex with that
394: facet by looking at the distances of the points of the candidate
395: colour to the hyperplane containing the facet.
396: A single volume computation via a determinant can be done in
397: time $O(d^3)$ per candidate colour, thus an iteration of \MV\
398: takes $O(d^4)$ time. Since the list of candidate colours may
399: not be all that large in typical situations, we can hope that
400: the cost of an iteration will often be less than that.
401:
402: \subsection{Random Sampling \Rand}\label{se:rand}
403: Finally, we consider a very simple
404: {\it guess and check} algorithm where we sample simplices at
405: random and check to see if they contain $\zero$. Intuitively
406: we would not expect such an algorithm to work well. However, as
407: discussed in \cite{DHST06}, solutions to a given colourful
408: feasibility problem may not be
409: all that rare, and in some cases can be quite frequent.
410: Since guessing and checking are relatively fast operations, it
411: worth considering the possibility that this naive algorithm
412: is faster than more sophisticated algorithms at least in low
413: dimension. We call this algorithm \Rand.
414:
415: One attractive feature of \Rand\ is that the cost of an iteration
416: is low -- we only have to generate a random simplex and then
417: test if it contains $\zero$. The test can be done in $O(d^3)$ time
418: by linear programming.
419:
420: %
421: % EXAMPLES
422: %
423: \section{Random, Ill-conditioned and Extremal Problems}\label{se:ex}
424: To better understand how various algorithms perform in practice,
425: we produced a test suite of challenging colourful
426: feasibility problems, which includes generic, ill-conditioned
427: and highly structured problems.
428: In this section we describe three types of colourful feasibility
429: problems that we consider when evaluating the practical performance of an
430: algorithm.
431: See \cite{Hua05} for a MATLAB implementation of each of these
432: problem generators.
433:
434: \subsection{Unstructured Random Problems}\label{se:rg}
435: The first class of problems we consider are unstructured random problems.
436: We take $d+1$ points in each of $d+1$ colours on $\Sph^d$. The only
437: restriction we require is that $\zero$ is in the core
438: We achieve this by taking the last point to be a random
439: convex combination of the antipodes on $\Sph^d$ of the first $d$
440: points. We call this generator \rg.
441:
442: \subsection{Ill-conditioned Random Problems}\label{se:tg}
443: Next, we consider ill-conditioned problems.
444: We place $d$ points of a given colour on the spherical cap
445: around the point $(0,0,\ldots,0,1)$ and the final point of that
446: colour in the opposite spherical cap, again as a convex combination
447: of the antipodes. In our implementation of this, the maximum
448: angle between a chosen vector and the final coordinate axis is
449: a parameter, and points are concentrated towards the centre
450: rather than uniformly distributed on the cap.
451: Since the points all lie in a tube around the final coordinate
452: axis, we call these {\it tube} generators. We implemented two
453: tube generators: \tz\ randomly places either 1 or $d$ points
454: of colour $i$ on the positive side of the axis, while \to\
455: always places $d$ points of colour $i$ on the positive side
456: of the axis.
457:
458: \subsection{Problems with a Restricted Number of Solutions}\label{se:sg}
459: Finally, we consider problems where we control the number of
460: colourful simplices containing $\zero$. The paper \cite{DHST06}
461: provides new bounds for the number of possible solutions to a colourful
462: linear program with
463: $\zero$ in the interior of the core. It turns out that the
464: number of simplices containing $\zero$ in dimension $d$ can be
465: as low as quadratic in $d$,
466: but not lower, see \cite{BM06} and \cite{ST06},
467: or as high as $d^{d+1}+1$ (with $\rho>0$), which is more than
468: one third of the total number of simplices.
469: Constructions are given for colourful feasibility problems
470: attaining both these values.
471:
472: The probability that a simplex generated by $d+1$ points
473: chosen randomly on $\Sph^d$ contains $\zero$ is $1/2^d$,
474: see for example \cite{WW01}. Thus in a uniformly generated
475: random problem of the type generated by \rg, we would expect
476: about $1/2^d$ of the $(d+1)^{d+1}$ colourful simplices
477: to contain $\zero$. This is not a large fraction, but in the
478: context of an effective pivoting algorithm such as \B\
479: which may pivot several neighbours to a given solution, and
480: pivot several neighbours of the first neighbour onto it, etc.,
481: we can entertain the idea that for a random
482: configuration most simplices are close to a solution.
483: See Section~\ref{se:effective} for further discussion.
484:
485: In any case, we would not be surprised if the difficulty
486: of a colourful feasibility problem increases as the number of solutions,
487: i.e.~simplices containing $\zero$, decreases.
488: To that end, we have written three problem generators
489: based on the constructions in \cite{DHST06}.
490: The first, \Sp\, generates
491: perturbed versions of the configuration from \cite{DHST06}
492: with many solutions.
493: These problems have $d^{d+1}+1$ of the $(d+1)^{d+1}$ simplices
494: containing $\zero$, many more than random configurations,
495: and we would expect them to be quite easy.
496: The second, \Sd, generates configurations where one point of
497: each colour is close to each vertex of a regular simplex on $\Sph^d$.
498: There are $d!$ solutions corresponding to picking a
499: different colour from each vertex, note that this is
500: still much less than the $(d+1)^{d+1}/2^d$
501: expected in a random configuration.
502: Finally, we have \Sm, which generates perturbed versions of the
503: configuration from \cite{DHST06} which has only
504: $d^2+1$ solutions.% in dimension $d$.
505: The generators \Sp, \Sd\ and \Sm\ randomly permute the
506: order the points appear within each colour.
507:
508: All these problems are ill-conditioned in the sense that
509: points are clustered closely together. Also $\rho$ will
510: be quite small for \Sp\ and \Sm, although the construction
511: \Sd\ effectively maximizes $\rho$ for configurations on $\Sph^d$
512: at $1/d$.
513:
514: %
515: % RESULTS
516: %
517: \section{Benchmarking and Results}\label{se:results}
518: In this section, we describe the results of computational
519: experiments in which we run our colourful feasibility
520: algorithms against our problem generators.
521: We focus on the number of iterations that an algorithm
522: takes to find a solution, but in Section~\ref{se:tpi}
523: we also include information about the cost of iterations.
524: The two particularly difficult, but fragile, examples of
525: Sections~\ref{se:mve} and~\ref{se:boe}
526: are not included in these results.
527:
528: \subsection{Iteration Counts}\label{se:ic}
529: For each type of problem we ran tests of the algorithms in
530: dimensions $3 \times 2^n$ for $n=0,1,2,3,4,5,6,7$.
531: Dimension 3 is our starting point since the seven algorithms
532: degenerate to three simple and effective algorithms in
533: dimension 2.
534: We use the factor 2 increase to sample higher dimensions
535: with less frequency as we get higher. We believe this yields
536: a reasonable sample of low, intermediate and high dimensional
537: problems.
538:
539: Note that a colourful feasibility problem instance in dimension $d$ consists of
540: $(d+1)^2$ points in dimension $d$. Thus the size of
541: the input is cubic in $d$. At present it is logistically difficult
542: to generate and store a colourful feasibility problem in dimension $d=1,000$.
543: After dimension 100, it also becomes increasingly difficult
544: to cope with numerical errors, especially for the algorithms
545: that include nearest point calculations, namely \B, \Bp\ and \H.
546: For this reason we do not include results for these algorithms
547: beyond $d=96$ for except for the relatively well-conditioned
548: \rg\ problems where we stopped at $d=192$.
549:
550: As one would expect, the guess-and-check algorithm \Rand\
551: performs badly as $d$ increases, except on problems from the
552: \Sp\ generator which have an abundance of solutions.
553: We only include results from the \Rand\ algorithm when they
554: can be completed in a reasonable amount of time.
555:
556: \ifapps
557: The results of our computational experiments are presented
558: in the graphs below and the tables in Appendix~\ref{ap:tables}.
559: \else
560: The results of our computational experiments are presented
561: in the graphs below. We have made the tables containing the
562: raw data for these graphs available at \cite{Advol}.
563: \fi
564: Each graph presents results for
565: a single random generator on a log-log scale with the average
566: iteration count of each algorithm plotted against the dimension.
567: Additionally, the tables contain the values of the largest
568: iteration count observed in each type of trial; these show
569: the similar trends to the averages, although we notice that
570: \BO\ and \BOp\ sometimes perform substantially worse than the
571: average, especially in the presence of ill-conditioning.
572: The reasons for this are discussed in Section~\ref{se:boe}.
573:
574: For each generator at $d=3$ we sampled 100,000 problems,
575: at $d=6$ and $d=12$ we sampled 10,000 problems, at $d=24$ and
576: $d=48$ we sampled 1,000 problems and finally for $d \ge 96$
577: we sampled 100 problems.
578: \ifapps
579: Because of the varying sample sizes,
580: it may not be entirely fair to compare the maxima listed in
581: Appendix~\ref{ap:tables} between dimensions.
582: \fi
583: % All plots are generated using Sui Huang's MATLAB code \cite{Hua05}.
584: The results are plotted on as log-log graphs in
585: Figures~\ref{fig:random}--\ref{fig:sm}.
586: We remark that polynomials appear asymptotically linear in
587: log-log plots, with the slope of the asymptote being the exponent
588: of the leading term of the polynomial and the $y$-intercept of
589: the asymptote representing the lead coefficient.
590:
591:
592: \begin{figure}[h!bt]
593: \begin{center}
594: \includegraphics[width=16cm,height=10.2cm]{gen1}
595: \caption{Results for \rg.} \label{fig:random}
596: \end{center}
597: \end{figure}
598:
599: In Figure~\ref{fig:random} we see that \B\ and \BO\ appear to
600: be taking a polynomial number of iterations to solution, while \MV\ and
601: \Rand\ do not appear to be polynomial. Since each algorithm
602: takes a polynomial time per iteration, the graphs of time versus
603: dimension show similar trends.
604:
605: For the tube experiments, we used an angle parameter of
606: $\pi/6$, which is to say that all the vectors used made
607: an angle of at most $\pi/6$ with the $x$-axis.
608: Smaller angles produce worse results for \BO, \BOp\
609: and \MV. The example of \MV\ cycling, see Section~\ref{se:mve}
610: and Appendix~\ref{ap:mve},
611: was found using a smaller angle with \tz.
612:
613: \begin{figure}[h!bt]
614: \begin{center}
615: \includegraphics[width=16cm,height=10.2cm]{gen2}
616: \caption{Results for \tz.} \label{fig:tz}
617: \end{center}
618: \end{figure}
619:
620: \begin{figure}[h!bt]
621: \begin{center}
622: \includegraphics[width=16cm,height=10.2cm]{gen3}
623: \caption{Results for \to.} \label{fig:to}
624: \end{center}
625: \end{figure}
626:
627: The tube experiments summarized in Figures~\ref{fig:tz}
628: and~\ref{fig:to} show the impact of ill-conditioning on
629: all the algorithms. For \B, \Bp, \H\ and \MV, convergence is
630: slightly slower and numerical errors become more common.
631: With these algorithms, our experiments began to
632: crash at dimension 192. By contrast for the better conditioned
633: problems from \rg, the three algorithms with minimum distance
634: calculations crashed only at dimension 384 and \MV\
635: would in any case take too long on problems of this size.
636: Nevertheless, these algorithms remain effective at $d=96$.
637:
638: The algorithms \BO\ and \BOp\ are more robust in the sense
639: that they are not as prone to crashes due to numerical errors.
640: This is the advantage of relying entirely on straightforward
641: linear algebra computations rather than considering nearest points
642: or volumes.
643: At the same time, they converge much more slowly due to problems
644: of the type described in Section~\ref{se:boe} and Appendix~\ref{se:boe}.
645:
646: If we decrease the angle parameter which controls
647: the width of the tube and hence the conditioning, the results
648: become more pronounced. That is to say, \B, \Bp, \H\ and \MV\
649: become less stable numerically and experience a
650: further mild degradation in performance when not affected
651: by numerical errors, while \BO\ and \BOp\ become substantially
652: slower.
653:
654: We comment that the \Rand\ algorithm performs about the same on \tz\
655: problems as it did on \rg\ problems. This simply means that \tz\
656: problems typically have a similar number of solutions to \rg\
657: problems. As one would expect, solutions to the one-sided tube problems
658: generated by \to\ are rarer than solutions to \rg\ and \tz\ problems
659: since the most of the points are clustered on one side.
660: Hence \Rand\ performs much worse on this type of
661: problem.
662:
663: \begin{figure}[h!bt]
664: \begin{center}
665: \includegraphics[width=16cm,height=10.2cm]{gen4}
666: \caption{Results for \Sp.} \label{fig:sp}
667: \end{center}
668: \end{figure}
669:
670: The problems with many solutions produced by \Sp\
671: are solved very quickly by all the algorithms, as
672: illustrated in Figure~\ref{fig:sp}. In this case
673: the random sampling algorithm \Rand\ offers excellent
674: performance. With the abundance of solutions, most of
675: the algorithms solve such problems in an expected
676: constant number of iterations. The exception is \BO\
677: which needs $\Theta(d)$ iterations at the
678: start to unwind the nearest point substitute $y$ from
679: a vertex to an interior point on a facet.
680: Since all the algorithms begin by checking the feasibility
681: of the initial simplex, the \Sp\ problems are often
682: solved in 0 iterations.
683:
684: \begin{figure}[h!bt]
685: \begin{center}
686: \includegraphics[width=16cm,height=10.2cm]{gen5}
687: \caption{Results for \Sd.} \label{fig:sd}
688: \end{center}
689: \end{figure}
690:
691: For the simplex structured problems of \Sd, we see all
692: the algorithms except \Rand\ perform very well, despite
693: the relative scarcity of solutions.
694: We see that the other algorithms have
695: exactly the proper response to this structure -- they
696: systematically take points near vertices that are not
697: part of the current set.
698: In the case of \B, a new vertex of the simplex will be
699: added at each step to give convergence in at most $d$
700: iterations, for \BO\ it takes one pass through the
701: $d+1$ colours, and for the multi-update algorithms
702: \Bp, \BOp\ and \H\ one or two passes through the colours.
703: Algorithm \MV\ also solves these problems in a reasonable
704: number of iterations.
705:
706: \begin{figure}[h!bt]
707: \begin{center}
708: \includegraphics[width=16cm,height=10.2cm]{gen6}
709: \caption{Results for \Sm.} \label{fig:sm}
710: \end{center}
711: \end{figure}
712:
713: Finally, we see that the problems from \Sm\
714: where solutions are scarce are indeed more difficult
715: than random problems, but that, except for the \Rand\
716: algorithm, the impact on algorithmic performance is mild.
717: See Figure~\ref{fig:sm}.
718: Curiously, the \Sm\ problems are the most difficult
719: problems for the \B\ algorithm. % At the same time,
720: The multi-update algorithms \Bp, \BOp\ and \H\
721: perform extremely well.
722:
723: \subsection{Cost per Iteration}\label{se:tpi}
724: In Figure~\ref{fig:tpi} we present
725: the average iteration times observed for all seven algorithms
726: on problems from the \rg\ generator.
727: \ifapps
728: The raw data for this graph is in Appendix~\ref{ap:tpi}.
729: \fi
730: We comment that the average time to complete an iteration does
731: not change significantly with the problems type, so we have not
732: included the similar graphs for other generators.
733: The data shows that in our implementation of these algorithms,
734: the average time for an iteration is never very large.
735: For the slowest algorithms in the highest dimensions
736: the average iteration took less than 2 seconds.
737:
738: \begin{figure}[h!bt]
739: \begin{center}
740: \includegraphics[width=16cm,height=10.2cm]{tpi}
741: \caption{Average iteration time of the algorithms.} \label{fig:tpi}
742: \end{center}
743: \end{figure}
744:
745: We see some interesting trends in the graphs.
746: First, in low dimensions all the iteration times are
747: very fast and are presumably dominated by fixed startup
748: costs. As the dimension increases, we begin to see
749: the asymptotic behaviour. The algebraic algorithms \BO\ and \BOp\
750: show the expected $\Theta(d^3)$ behaviour, which appears linear
751: in the log-log plot.
752: Asymptotically, the average time for an iteration of \BOp\
753: is about 10 times longer for an iteration of \BO.
754:
755: The algorithms \B\ and \Bp, which depend on a minimum distance
756: calculation, take longer on average to complete an iteration than \BOp.
757: The extra cost for the multiple updates in \Bp\
758: is relatively small. However, the asymptotic slope of
759: these lines appear higher than for \BO, which means that
760: the nearest point calculations are causing the iterations
761: to take time $\Omega(d^3)$.
762: The algorithm \MV\ has iteration times not much worse than \BO\
763: in low dimension, but its asymptotics look
764: close to $O(d^4)$ as suggested in Section~\ref{se:mv}.
765: Algorithm \Rand\ exhibits $\Theta(d^3)$ iteration time and
766: is asymptotically about twice as fast on average per iteration
767: than \BO.
768:
769: Unlike the other algorithms, the average iteration time for \H\
770: will be substantially affected by the conditioning of the problem.
771: Using the well-conditioned \rg\ problems, \H\ usually degenerates
772: to \BOp\ and has a very similar average iteration time.
773: As the problems become more ill-conditioned, \H\ will begin
774: to use \Bp\ steps as well, and the average iteration time will
775: increase towards the average iteration time for \Bp.
776:
777: %
778: % CONCLUSIONS
779: %
780: \section{Conclusions}\label{se:conclusions}
781: Our experiments reveal several features of colourful
782: feasibility algorithms. After considerable searching,
783: we found a problem instance which caused \MV\ to cycle.
784: We also found that \BO\ and \BOp\ can converge extremely
785: slowly in the face of ill-conditioning although \B\ and \Bp\
786: continue to perform reasonably well on the same
787: examples. We conclude that computationally the best algorithms are
788: \Bp\ and \BOp\ and remark that these tightened algorithms
789: do yield substantial gains over the originals.
790:
791: \subsection{A Cycling Example for \MV\ in Dimension 4}\label{se:mve}
792: % \ifapps % Now include this appendix in all versions.
793: In Appendix~\ref{ap:mve} we exhibit an example in dimension 4
794: for which the maximum volume heuristic cycles.
795: % \else
796: % In \cite{Advol} we exhibit an example in dimension 4
797: % for which the maximum volume heuristic cycles.
798: % \fi
799: This example was found using our tube generator \tz\
800: to produce configurations where for each colour, four points are
801: tightly bunched around (-1,0,0,0,0) and the fifth point
802: is close to (1,0,0,0,0) or vice-versa.
803: The example is fairly ill-conditioned,
804: but not excessively so: we rounded the values we found
805: for text formatting purposes,
806: and observed that $\zero$ remained in the core and
807: that the behaviour of the algorithm was unaffected.
808:
809: Close examination of the iterations of this example turns
810: up nothing out of the ordinary. Since this example shows
811: that \MV\ can cycle, it is remarkable that it happens
812: so rarely. It did not occur in the entire test suite of
813: Section~\ref{se:results}. We tested extensively in dimensions
814: 3 and 4, and were unable to find any examples of cycling in
815: dimension 3 or any examples of cycling in dimension 4 with
816: cycle length shorter than 6. Higher dimensions and longer
817: cycle lengths do occur.
818:
819: One explanation for the results is that as one might expect, \MV\
820: is an effective heuristic in a typical situation.
821: The distinguishing feature of the few bad examples is that
822: the points are placed in such a way that the simplices cluster
823: into a few groups of similar shape and volume.
824: The heuristic of taking the maximum volume is then not very
825: helpful in choosing promising simplices.
826: We note that this example is solved easily by the other algorithms.
827:
828: \subsection{Flip-flopping During Convergence for \BO: 40,847 Iterations
829: in Dimension 3}\label{se:boe}
830: We constructed an example of a colourful feasibility problem
831: in dimension 3 that takes 40,847 iterations to solution
832: using a basic implementation of \BO.
833: % \ifapps % Now this appendix is included in all versions.
834: The exact points we used are contained in Appendix~\ref{ap:boe}.
835: % \else
836: % The exact points we used are contained in \cite{Advol}.
837: % \fi
838: The algorithm is initialized with the simplex that uses
839: the first point of each colour. At the
840: fifth iteration, the algorithm reaches a situation where the
841: current point $y$ lies on a facet $F$ of colours 2, 3 and 4 very close
842: to $\zero$. Using this point the algorithm will pick the
843: point of colour 1 that has minimum dot product with $y$.
844: The second and third points of colour 1 lie almost in the directions of $y$
845: and $-y$, however neither of these forms a simplex with $F$
846: containing $\zero$. In fact the fourth point of colour 1 does
847: form a simplex containing $\zero$ with $F$, but it is nearly
848: orthogonal to $y$. As a result,
849: after two iterations, \BO\ returns to the same simplex.
850: The point $y$ will be recomputed at each step, and is slightly
851: closer to $\zero$ when the algorithm returns to the previous
852: simplex. However, the improvement is quite small.
853: Of course $\rho$ is also very small, so this is consistent with
854: the performance guarantee described in Section~\ref{se:bo}.
855: The algorithm then proceeds to return to the same simplex
856: more than 20,000 times, with an incremental improvement to
857: $y$ at each iteration before finally taking the fourth
858: point of colour 1 and terminating.
859:
860: As one would expect with a very ill-conditioned problem, this
861: example is numerically fragile -- the current version of our code
862: normalizes the coordinates before starting and does not suffer
863: the same fate. However bad behaviour is fairly typical.
864: The tube generator for ill-conditioned problems in \cite{Hua05}
865: produces problems whose ill-conditioning depends on a parameter
866: defining the width of the tube. As the width decreases, we
867: get an increasing number of cases where \BO\ and \BOp\ take
868: enormous numbers of iterations.
869:
870: We remark that, in contrast, \B\ never returns to the same
871: simplex, so it cannot suffer from this
872: type of flip-flopping. Indeed in dimension 3 it could do
873: no worse than visiting all $4^4=256$ simplices.
874: At least 10 of these must contain $\zero$,
875: see \cite{BM06}, so the algorithm must terminate in at
876: most 246 iterations. It is quite hard to see how this
877: limit could be approached. The authors wonder if
878: a Klee-Minty-like example, see \cite{KM72},
879: of worst-case behaviour for \bara's pivoting
880: algorithm could be constructed.
881:
882: \subsection{Overall Effectiveness of Algorithms}\label{se:effective}
883: Despite the examples of Sections~\ref{se:mve} and~\ref{se:boe},
884: the results presented in Section~\ref{se:results} show that,
885: except for \Rand\ and to a lesser degree \MV, all the algorithms
886: did a good job of solving all the problems.
887: We did find that the methods which include nearest point
888: calculations were more vulnerable to numerical errors than
889: \BO\ and \BOp, since our implementations began to crash
890: once we got much past $d=100$, especially on ill-conditioned
891: problems. For the most part, reduced iteration counts of
892: the nearest point algorithms do not offset the extra time
893: spent per iteration compared to \BO\ and \BOp, since neither
894: iteration count is very high.
895: In some cases of extreme ill-conditioning, such as in
896: Section~\ref{se:boe}, \BO\ and \BOp\ will take many
897: additional iterations and be much slower compared to the
898: nearest point algorithms.
899: In this situation either a hybrid algorithm such as \H,
900: or the basic \B\ or \Bp\ would work better.
901:
902: We had hoped that the hybrid algorithm \H\ would offer
903: the benefits of \BOp, namely speed and robustness in
904: high dimensions, while stopping long periods of
905: flip-flopping from occurring. This did happen to a degree,
906: but in our benchmarking experiments the net time savings
907: were negligible, while \H\ retained \Bp's tendency to
908: crash due to numerical errors in high dimension.
909:
910: \subsection{Advantages of Multiple Updates and Initialization}\label{se:multi}
911: The multi-update algorithms \Bp\ and \BOp\ do provide
912: substantial gains over their single update counterparts,
913: \B\ and \BO. In the case of \Bp, we get a large reduction
914: in iteration count at very little cost in terms of iteration
915: time. In our benchmarking experiments, this produced times
916: that were competitive with \BO\ and much better than \B.
917: The gains for \BOp\ relative to \BO\ are less impressive.
918: In our benchmarking experiments, \BOp\ consistently
919: averaged a 10\% to 40\% savings in total time to solution.
920:
921: In fact, \BO\ is not as well suited as \B\ to take
922: advantage of multiple updates. The point $y$ close to
923: $\zero$ computed by \BO\ will almost always lie in the
924: interior of a facet of $\Deltab$, meaning that \BO\
925: will only have a single candidate colour to pivot.
926: In contrast, in high dimension, the closest point $x$ to
927: $\zero$ will often lie on
928: a relatively low dimensional face of $\Deltab$, allowing
929: multiple updates throughout the algorithm.
930:
931: One difficulty for \BO\ is that it
932: begins with $y$ at a vertex. In a normal situation,
933: the first $d$ steps of \BO\ will each increase the
934: dimension of the smallest face containing $y$ by one
935: until $y$ lies in the interior of a facet, without
936: necessarily yielding a much better current simplex.
937: The multi-update \BOp\ does this all in the first iteration
938: in less time than it takes \BO\ to do $d$ steps.
939:
940: We have not discussed the effects of the initial simplex
941: in this paper, but we can employ various heuristics to
942: choose a good initial simplex. A few of these are implemented
943: in \cite{Hua05}. We found that the most useful initialization
944: heuristic was to run the first iteration of \BOp. This runs
945: in $O(d^3)$ time and improves the subsequent iteration counts of
946: the algorithms, with the obvious exception of \Rand.
947: Even \BOp\ experiences a reduced iteration count, since the
948: point $y$ found by the initialization is not passed to the
949: algorithm.
950:
951: \subsection{Theoretical Complexity of the Algorithms}\label{se:theory}
952: In Section~\ref{se:algs}, we remarked that \baraonn\
953: proved a worst-case bound for \B\ and \BO\ of $O(1/\rho^2)$
954: iterations up to numerical considerations and we improved
955: their iteration time for \BO\ from $O(d^4)$ to $O(d^3)$.
956: We also mentioned that
957: we do not expect the multi-update and hybrid algorithms
958: to improve the theoretical bounds.
959: From the example of Section~\ref{se:mve}, we see that \MV\
960: is not guaranteed to converge. The expected running time
961: of \Rand\ is 1 over the probability that random simplex
962: contains $\zero$, i.e.~around $2^d$ for random problems,
963: and as bad as $(d+1)^{d+1}/(d^2+1)$ for the type of
964: problems generated by \Sm.
965:
966: The poor performance of \BO\
967: on ill-conditioned problems and examples like that
968: of Section~\ref{se:boe} confirm the worst-case
969: predictions of \baraonn's analysis.
970: On the other hand, we did not see this type of
971: behaviour for \B, and it is hard to see how
972: it could occur.
973:
974: The model proposed in Section~\ref{se:sg} is that a
975: pure pivoting algorithm such as \B, defines a set of
976: rooted trees on the $(d+1)^{d+1}$ simplices.
977: Each simplex which contains $\zero$ is the root of a tree, and
978: we draw an edge between the vertices representing simplices
979: $\Deltab_1$ and $\Deltab_2$ if when \B\ encounters $\Deltab_1$
980: it pivots to $\Deltab_2$. Then the worst performance of the
981: algorithm in terms of the number of iterations would be the height
982: of the highest tree. A smart algorithm will produce short
983: trees by pivoting several simplices to a given simplex at
984: a lower level.
985:
986: Consider a situation where trees have a constant expansion
987: factor $k$ near the base, that is, low level vertices are
988: connected to roughly $k$ vertices in the level above.
989: The number of trees is $p(d+1)^{d+1}$ where $p$ is the
990: probability that a simplex contains $\zero$.
991: If the trees expand up to height $h$, each
992: tree will contain on the order of $k^h$ vertices.
993: Then we must have $k^h p(d+1)^{d+1} \le (d+1)^{d+1}$,
994: the total number of vertices. Rearranging, we get
995: $h \le -\log_k(p)$. This expression predicts the average
996: iteration count for \B\ to grow linearly for \rg\
997: problems, to be constant for \Sp\ problems and to
998: grow at $\Theta(d \log d)$ for \Sm\ problems.
999: All of these match very well with our observed results.
1000: The \Sd\ problems are predicted to be more difficult than
1001: they are observed to be, but that is not surprising given
1002: their simple structure.
1003:
1004: \subsection{Future Considerations}\label{se:future}
1005: We finish by returning to the motivating question of \baraonn:
1006: Is there a polynomial time algorithm for colourful feasibility?
1007: By improving the implementation of \BO, we have improved the
1008: worst case for this algorithm from $O(d^4/\rho^2)$ to
1009: $O(d^3/\rho^2)$, however the dependence on $\rho$ has
1010: not improved. Indeed our experiments give strong evidence
1011: that the analysis for \BO\ is tight.
1012:
1013: The situation for \B\ is less clear. We do not see the
1014: same bad behaviour with ill-conditioned problems that
1015: we found for \BO, so it is possible that a better
1016: guarantee exists for this algorithm.
1017: In light of the model suggested in Section~\ref{se:theory}
1018: it is quite difficult to see how to construct a
1019: Klee-Minty-like bad case for \B\ as discussed in
1020: Section~\ref{se:boe}. We view this as an appealing challenge.
1021:
1022:
1023: %
1024: % ACKNOWLEDGMENTS
1025: %
1026: \section{Acknowledgments}
1027: This research was supported
1028: by NSERC Discovery grants for the four authors,
1029: by the Canada Research Chair program for the first and last authors
1030: and by a MITACS grant for the second and third authors.
1031: The third author worked on this project as part of the
1032: Discrete Optimization project of the IMO at the University of Magdeburg.
1033:
1034:
1035:
1036: %
1037: % APPENDICES
1038: %
1039: \appendix
1040:
1041:
1042: %
1043: % APPENDIX A: Max_vol example.
1044: %
1045: \section{Example in dimension 4 where \MV\ cycles}\label{ap:mve}
1046: This example consists of 5 normalized points in each of the
1047: 5 colours in $\R^4$.
1048: The points are presented in Table~\ref{ta:mvc}.
1049: They are grouped by colour, with the rows
1050: representing $x$, $y$, $z$ and $w$ coordinates, respectively.
1051:
1052: \begin{table}[h!bt]
1053: %\begin{tabular}{rc}
1054: Red points
1055:
1056: \begin{tabular}{|c|c|c|c|c|} \hline
1057: -0.98126587 & 0.99234170 & -0.99375618 & -0.98428021 & -0.99649986 \\ \hline
1058: 0.13481464 & 0.01125213 & -0.01676635 & -0.03542019 & 0.03152825 \\ \hline
1059: 0.00569666 & -0.12300509 & 0.10203928 & 0.17121850 & 0.07625092 \\ \hline
1060: 0.13751313 & 0.00104048 & -0.04189897 & 0.02494182 & -0.01340880 \\ \hline
1061: \end{tabular} \vspace{0.5mm} \\
1062:
1063: Green points
1064:
1065: \begin{tabular}{|c|c|c|c|c|} \hline
1066: 0.99924734 & -0.99225276 & 0.95301586 & 0.99770745 & 0.98808067 \\ \hline
1067: 0.03530047 & -0.07048563 & 0.17760263 & 0.03405179 & -0.00874509 \\ \hline
1068: -0.01500068 & 0.10036231 & -0.24516979 & -0.01526145 & -0.12973853 \\ \hline
1069: 0.00579663 & -0.01984027 & 0.01048096 & 0.05645716 & 0.08238952 \\ \hline
1070: \end{tabular} \vspace{0.5mm} \\
1071:
1072: Blue points
1073:
1074: \begin{tabular}{|c|c|c|c|c|} \hline
1075: -0.98758195 & -0.99742900 & -0.97286388 & -0.97433105 & 0.99536963 \\ \hline
1076: -0.03897365 & 0.02836725 & 0.13575382 & 0.14413058 & -0.06519965 \\ \hline
1077: -0.14957699 & -0.06348104 & -0.17638005 & 0.17286629 & 0.06380946 \\ \hline
1078: -0.02810110 & -0.01734511 & -0.06322067 & -0.00475659 & 0.03027639 \\ \hline
1079: \end{tabular} \vspace{0.5mm} \\
1080:
1081: Tan points
1082:
1083: \begin{tabular}{|c|c|c|c|c|} \hline
1084: 0.99782436 & 0.99917562 & 0.95584087 & -0.98768930 & 0.96962649 \\ \hline
1085: 0.01692290 & 0.03972232 & 0.17806542 & -0.10337937 & 0.14481818 \\ \hline
1086: 0.03437294 & -0.00816965 & -0.21878711 & 0.09313650 & -0.12491250 \\ \hline
1087: 0.05365310 & 0.00186470 & 0.08242045 & -0.07147128 & 0.15247636 \\ \hline
1088: \end{tabular} \vspace{0.5mm} \\
1089:
1090: White points
1091:
1092: \begin{tabular}{|c|c|c|c|c|} \hline
1093: -0.99979855 & -0.97268376 & -0.97231627 & -0.95622769 & 0.99791825 \\ \hline
1094: 0.00600345 & 0.06950105 & 0.21172943 & -0.29221243 & -0.02997771 \\ \hline
1095: 0.00415788 & -0.00409898 & -0.03733932 & -0.01550644 & 0.01616939 \\ \hline
1096: 0.01869548 & 0.22144776 & 0.09152860 & 0.00022801 & -0.05476362 \\ \hline
1097: \end{tabular}
1098: %\end{tabular}
1099: \caption{Coordinates of points of an example where \MV\ cycles in dimension 4.}
1100: \label{ta:mvc}
1101: \end{table}
1102:
1103: The initial simplex is taken to be (1,1,1,1,1), i.e.,~the first point
1104: of each colour. The algorithm proceeds to visit simplices
1105: (1,1,4,1,1), (3,1,4,1,1), (3,1,4,3,1), (3,1,1,3,1) and (1,1,1,3,1)
1106: before returning to the original simplex and repeating.
1107:
1108:
1109: %
1110: % APPENDIX B: Flip-flopping example.
1111: %
1112: \section{Example in dimension 3 where \BO\ takes 40,847 iterations}
1113: \label{ap:boe}
1114: This example consists of 4 unnormalized points in each of the
1115: 4 colours in $\R^3$.
1116: The points are presented in Table~\ref{ta:bol}.
1117: They are grouped by colour, with the rows
1118: representing $x$, $y$ and $z$ coordinates, respectively.
1119: \begin{table}[h!bt]
1120: Red points
1121:
1122: \begin{tabular}{|c|c|c|c|} \hline
1123: 1.00000320775369 & -0.01000436049274 & -0.01000129525998 & 1.00000089660284 \\
1124: \hline
1125: 0.00000340785030 & 0.99999739350954 & -1.00000497855619 & 0.00000051797159 \\
1126: \hline
1127: 0.00999859615603 & 0.00000371775824 & 0.00000030149139 & -0.01999639732055 \\
1128: \hline
1129: \end{tabular}
1130:
1131: Green points
1132:
1133: \begin{tabular}{|c|c|c|c|} \hline
1134: 1.00000363763560 & -0.00999644886160 & -0.00999943004295 & 1.00000335962280 \\
1135: \hline
1136: -0.00000325123594 & 1.00000064545156 & -1.00000169806216 & -0.00000080450760 \\
1137: \hline
1138: 0.01000493174811 & -0.00000024088601 & 0.00000009099437 & -0.01999811804365 \\
1139: \hline
1140: \end{tabular}
1141:
1142: Blue points
1143:
1144: \begin{tabular}{|c|c|c|c|} \hline
1145: 0.99999949817337 & -0.00999587145461 & -0.00999627213896 & 0.99999551963712 \\
1146: \hline
1147: -0.00000260397964 & 1.00000485455718 & -1.00000419710665 & -0.00000024626161 \\
1148: \hline
1149: 0.00999854691703 & 0.00000123671997 & -0.00000381812529 & -0.01999801526314 \\
1150: \hline
1151: \end{tabular}
1152:
1153: Tan points
1154:
1155: \begin{tabular}{|c|c|c|c|} \hline
1156: 0.99999980645233 & 0.10000000280522 & -0.60000327600988 & 0.99999642880542 \\
1157: \hline
1158: 0.00000024487465 & -0.98999719313413 & 0.79999695643245 & -0.00000429109491 \\
1159: \hline
1160: 0.01000455311709 & -0.00000405877812 & 0.00000372117690 & -0.01000272055280 \\
1161: \hline
1162: \end{tabular}
1163: \caption{Coordinates of points of an example taking 40,847 iterations of \BO\
1164: in dimension 3.}
1165: \label{ta:bol}
1166: \end{table}
1167:
1168: The initial simplex is taken to be (1,1,1,1), i.e.,~the first point
1169: of each colour. It then updates to (1,3,1,1), (1,3,2,1), (1,3,2,3),
1170: (1,3,2,2) and reaches (3,3,2,2) on the fifth iteration.
1171: At this point, it begins to flip
1172: between (3,3,2,2) and (2,3,2,2) with $y$ initially alternating
1173: between values close to (0.2,$\pm$0.00200,0.00285).
1174: The values of all these coordinates decrease very slowly as the
1175: algorithm continues. At iteration 40,847 it chooses fourth point
1176: of colour 1 instead of the third. This makes the current simplex
1177: (4,3,2,2) which contains $\zero$.
1178:
1179:
1180: %
1181: % APPENDIX C: Raw data.
1182: %
1183: \section{Iteration counts from our experiments}\label{ap:tables}
1184: In this Appendix we present the raw data from our computational
1185: experiments. % graphed in Section~\ref{se:ic}.
1186: Each table presents results for
1187: a single random generator. The entries give the average number
1188: of iterations to solution for each algorithm at the given dimension.
1189: For each generator at $d=3$ we sampled 100,000 problems,
1190: at $d=6$ and $d=12$ we sampled 10,000 problems, at $d=24$ and
1191: $d=48$ we sampled 1,000 problems and finally for $d \ge 96$
1192: we sampled 100 problems.
1193:
1194: \begin{table}[h!bt]\label{ta:rga}
1195: \begin{tabular}{|l||c|c|c|c|c|c|c|} \hline
1196: ~ & \B & \BO & \Bp & \BOp & \H & \MV & \Rand \\ \hline \hline
1197: $d=3$ & 1.31 & 2.96 & 1.15 & 1.15 & 1.15 & 1.31 & 7.15 \\ \hline
1198: $d=6$ & 2.56 & 6.87 & 1.77 & 1.67 & 1.67 & 2.90 & 63.48 \\ \hline
1199: $d=12$ & 4.84 & 13.93 & 2.42 & 2.16 & 2.16 & 7.01 & 4133.15 \\ \hline
1200: $d=24$ & 8.84 & 27.70 & 3.07 & 2.87 & 2.87 & 19.07 & Large \\ \hline
1201: $d=48$ & 16.14 & 54.88 & 3.77 & 4.14 & 4.14 & 56.12 & Large \\ \hline
1202: $d=96$ & 28.80 & 108.71 & 4.26 & 6.39 & 6.39 & 185.57 & Large \\ \hline
1203: $d=192$ & 51.96 & 217.59 & 4.99 & 11.68 & 11.68 & 808.78 & Large \\ \hline
1204: $d=384$ & Unstable & 425.26 & Unstable & 21.63 & Unstable & Large & Large
1205: \\ \hline
1206: \end{tabular}
1207: \caption{Average iteration counts in \rg\ generator tests.}
1208: \end{table}
1209:
1210: \begin{table}[h!bt]\label{ta:rgm}
1211: \begin{tabular}{|l||c|c|c|c|c|c|c|} \hline
1212: ~ & \B & \BO & \Bp & \BOp & \H & \MV & \Rand \\ \hline \hline
1213: $d=3$ & 5 & 136 & 4 & 4 & 4 & 5 & 102 \\ \hline
1214: $d=6$ & 7 & 21 & 5 & 5 & 5 & 12 & 579 \\ \hline
1215: $d=12$ & 10 & 30 & 6 & 6 & 6 & 20 & 47362 \\ \hline
1216: $d=24$ & 15 & 37 & 6 & 8 & 8 & 43 & Large \\ \hline
1217: $d=48$ & 22 & 67 & 6 & 9 & 9 & 105 & Large \\ \hline
1218: $d=96$ & 39 & 120 & 6 & 10 & 10 & 269 & Large \\ \hline
1219: $d=192$ & 63 & 241 & 7 & 19 & 19 & 1574 & Large \\ \hline
1220: $d=384$ & Unstable & 472 & Unstable & 30 & Unstable & Large & Large
1221: \\ \hline
1222: \end{tabular}
1223: \caption{Maximum iteration counts found in \rg\ generator tests.}
1224: \end{table}
1225:
1226: \begin{table}[h!bt]\label{ta:tza}
1227: \begin{tabular}{|l||c|c|c|c|c|c|c|} \hline
1228: ~ & \B & \BO & \Bp & \BOp & \H & \MV & \Rand \\ \hline \hline
1229: $d=3$ & 1.39 & 5.62 & 1.25 & 1.43 & 1.43 & 1.38 & 7.30 \\ \hline
1230: $d=6$ & 2.92 & 17.00 & 2.17 & 3.14 & 2.89 & 3.54 & 66.02 \\ \hline
1231: $d=12$ & 5.83 & 33.48 & 3.23 & 6.65 & 5.64 & 10.26 & 4296.66 \\ \hline
1232: $d=24$ & 11.18 & 64.30 & 4.29 & 13.86 & 10.86 & 31.75 & Large \\ \hline
1233: $d=48$ & 20.24 & 123.02 & 5.51 & 27.91 & 21.11 & 106.11 & Large \\ \hline
1234: $d=96$ & 37.12 & 240.49 & 6.54 & 56.70 & 40.91 & 406.10 & Large \\ \hline
1235: $d=192$ & Unstable & 468.52 & Unstable & 111.84 & Unstable & 3367.60 & Large
1236: \\ \hline
1237: $d=384$ & Unstable & 909.82 & Unstable & 220.50 & Unstable & Large & Large
1238: \\ \hline
1239: \end{tabular}
1240: \caption{Average iteration counts in \tz\ generator tests.}
1241: \end{table}
1242:
1243: \begin{table}[h!bt]\label{ta:tzm}
1244: \begin{tabular}{|l||c|c|c|c|c|c|c|} \hline
1245: ~ & \B & \BO & \Bp & \BOp & \H & \MV & \Rand \\ \hline \hline
1246: $d=3$ & 5 & 4783 & 4 & 5 & 5 & 6 & 109 \\ \hline
1247: $d=6$ & 8 & 2880 & 6 & 44 & 10 & 14 & 1079 \\ \hline
1248: $d=12$ & 13 & 842 & 8 & 60 & 14 & 33 & 78418 \\ \hline
1249: $d=24$ & 21 & 217 & 9 & 36 & 23 & 78 & Large \\ \hline
1250: $d=48$ & 31 & 249 & 9 & 55 & 41 & 258 & Large \\ \hline
1251: $d=96$ & 47 & 323 & 9 & 77 & 76 & 840 & Large \\ \hline
1252: $d=192$ & Unstable & 561 & Unstable & 140 & Unstable & 11784 & Large
1253: \\ \hline
1254: $d=384$ & Unstable & 1013 & Unstable & 260 & Unstable & Large & Large
1255: \\ \hline
1256: \end{tabular}
1257: \caption{Maximum iteration counts found in \tz\ generator tests.}
1258: \end{table}
1259:
1260: \begin{table}[h!bt]\label{ta:toa}
1261: \begin{tabular}{|l||c|c|c|c|c|c|c|} \hline
1262: ~ & \B & \BO & \Bp & \BOp & \H & \MV & \Rand \\ \hline \hline
1263: $d=3$ & 1.51 & 5.93 & 1.31 & 1.51 & 1.51 & 1.48 & 9.16 \\ \hline
1264: $d=6$ & 3.48 & 17.26 & 2.35 & 3.31 & 3.01 & 4.10 & 150.31 \\ \hline
1265: $d=12$ & 7.64 & 37.22 & 3.62 & 8.06 & 6.43 & 13.61 & Large \\ \hline
1266: $d=24$ & 16.59 & 75.73 & 5.11 & 19.11 & 13.92 & 48.51 & Large \\ \hline
1267: $d=48$ & 33.51 & 155.48 & 6.57 & 42.81 & 28.70 & 159.29 & Large \\ \hline
1268: $d=96$ & 61.97 & 306.64 & 8.32 & 90.98 & 58.44 & 602.07 & Large \\ \hline
1269: $d=192$ & Unstable & 619.55 & Unstable & 186.86 & Unstable & Large & Large
1270: \\ \hline
1271: $d=384$ & Unstable & 1221.43 & Unstable & 382.10 & Unstable & Large & Large
1272: \\ \hline
1273: \end{tabular}
1274: \caption{Average iteration counts in \to\ generator tests.}
1275: \end{table}
1276:
1277: \begin{table}[h!bt]\label{ta:tom}
1278: \begin{tabular}{|l||c|c|c|c|c|c|c|} \hline
1279: ~ & \B & \BO & \Bp & \BOp & \H & \MV & \Rand \\ \hline \hline
1280: $d=3$ & 6 & 2756 & 5 & 6 & 6 & 6 & 127 \\ \hline
1281: $d=6$ & 9 & 3704 & 7 & 38 & 9 & 14 & 1709 \\ \hline
1282: $d=12$ & 16 & 689 & 8 & 55 & 16 & 46 & Large \\ \hline
1283: $d=24$ & 28 & 195 & 9 & 52 & 27 & 124 & Large \\ \hline
1284: $d=48$ & 50 & 257 & 10 & 83 & 47 & 505 & Large \\ \hline
1285: $d=96$ & 78 & 374 & 11 & 133 & 83 & 2023 & Large \\ \hline
1286: $d=192$ & Unstable & 736 & Unstable & 226 & Unstable & Large & Large
1287: \\ \hline
1288: $d=384$ & Unstable & 1399 & Unstable & 454 & Unstable & Large & Large
1289: \\ \hline
1290: \end{tabular}
1291: \caption{Maximum iteration counts found in \to\ generator tests.}
1292: \end{table}
1293:
1294: \begin{table}[h!bt]\label{ta:spa}
1295: \begin{tabular}{|l||c|c|c|c|c|c|c|} \hline
1296: ~ & \B & \BO & \Bp & \BOp & \H & \MV & \Rand \\ \hline \hline
1297: $d=3$ & 0.89 & 2.07 & 0.82 & 0.71 & 0.71 & 0.89 & 2.12 \\ \hline
1298: $d=6$ & 0.99 & 3.96 & 0.68 & 0.66 & 0.66 & 0.99 & 1.94 \\ \hline
1299: $d=12$ & 0.97 & 7.61 & 0.63 & 0.63 & 0.63 & 0.97 & 1.78 \\ \hline
1300: $d=24$ & 0.99 & 15.46 & 0.64 & 0.64 & 0.64 & 0.99 & 1.83 \\ \hline
1301: $d=48$ & 1.01 & 31.15 & 0.65 & 0.65 & 0.65 & 1.01 & 1.87 \\ \hline
1302: $d=96$ & 1.06 & 61.44 & 0.64 & 0.64 & 0.64 & 1.06 & 1.81 \\ \hline
1303: $d=192$ & 0.90 & 122.88 & 0.64 & 0.64 & 0.64 & 0.90 & 1.77 \\ \hline
1304: $d=384$ & 0.77 & 211.20 & 0.55 & 0.55 & 0.55 & 0.77 & 1.50
1305: \\ \hline
1306: \end{tabular}
1307: \caption{Average iteration counts in \Sp\ generator tests.}
1308: \end{table}
1309:
1310: \begin{table}[h!bt]\label{ta:spm}
1311: \begin{tabular}{|l||c|c|c|c|c|c|c|} \hline
1312: ~ & \B & \BO & \Bp & \BOp & \H & \MV & \Rand \\ \hline \hline
1313: $d=3$ & 2 & 5 & 2 & 3 & 3 & 2 & 38 \\ \hline
1314: $d=6$ & 3 & 7 & 2 & 2 & 2 & 3 & 17 \\ \hline
1315: $d=12$ & 6 & 12 & 1 & 1 & 1 & 6 & 30 \\ \hline
1316: $d=24$ & 6 & 24 & 1 & 1 & 1 & 6 & 19 \\ \hline
1317: $d=48$ & 5 & 48 & 1 & 1 & 1 & 5 & 16 \\ \hline
1318: $d=96$ & 5 & 96 & 1 & 1 & 1 & 5 & 14 \\ \hline
1319: $d=192$ & 3 & 192 & 1 & 1 & 1 & 4 & 15 \\ \hline
1320: $d=384$ & 4 & 384 & 1 & 1 & 1 & 4 & 9
1321: \\ \hline
1322: \end{tabular}
1323: \caption{Maximum iteration counts found in \Sp\ generator tests.}
1324: \end{table}
1325:
1326: \begin{table}[h!bt]\label{ta:sda}
1327: \begin{tabular}{|l||c|c|c|c|c|c|c|} \hline
1328: ~ & \B & \BO & \Bp & \BOp & \H & \MV & \Rand \\ \hline \hline
1329: $d=3$ & 1.26 & 2.72 & 0.99 & 0.91 & 0.91 & 1.26 & 9.67 \\ \hline
1330: $d=6$ & 2.39 & 5.97 & 1.09 & 0.99 & 0.99 & 2.39 & 161.93 \\ \hline
1331: $d=12$ & 4.61 & 12.00 & 1.12 & 1.00 & 1.00 & 4.61 & Large \\ \hline
1332: $d=24$ & 8.94 & 24.00 & 1.13 & 1.00 & 1.00 & 8.94 & Large \\ \hline
1333: $d=48$ & 17.82 & 48.00 & 1.15 & 1.00 & 1.00 & 17.82 & Large \\ \hline
1334: $d=96$ & 35.58 & 96.00 & 1.19 & 1.00 & 1.00 & 35.58 & Large \\ \hline
1335: $d=192$ & 71.15 & 192.00 & 1.47 & 1.00 & 1.00 & 71.15 & Large
1336: \\ \hline
1337: \end{tabular}
1338: \caption{Average iteration counts in \Sd\ generator tests.}
1339: \end{table}
1340:
1341: \begin{table}[h!bt]\label{ta:sdm}
1342: \begin{tabular}{|l||c|c|c|c|c|c|c|} \hline
1343: ~ & \B & \BO & \Bp & \BOp & \H & \MV & \Rand \\ \hline \hline
1344: $d=3$ & 3 & 5 & 3 & 2 & 2 & 3 & 128 \\ \hline
1345: $d=6$ & 5 & 6 & 3 & 1 & 1 & 5 & 1371 \\ \hline
1346: $d=12$ & 9 & 12 & 3 & 1 & 1 & 9 & Large \\ \hline
1347: $d=24$ & 14 & 24 & 2 & 1 & 1 & 14 & Large \\ \hline
1348: $d=48$ & 24 & 48 & 2 & 1 & 1 & 24 & Large \\ \hline
1349: $d=96$ & 41 & 96 & 2 & 1 & 1 & 41 & Large \\ \hline
1350: $d=192$ & 81 & 192 & 3 & 1 & 1 & 81 & Large
1351: \\ \hline
1352: \end{tabular}
1353: \caption{Maximum iteration counts found in \Sd\ generator tests.}
1354: \end{table}
1355:
1356:
1357: \begin{table}[h!bt]\label{ta:sma}
1358: \begin{tabular}{|l||c|c|c|c|c|c|c|} \hline
1359: ~ & \B & \BO & \Bp & \BOp & \H & \MV & \Rand \\ \hline \hline
1360: $d=3$ & 2.19 & 3.54 & 1.96 & 1.59 & 1.59 & 2.26 & 24.39 \\ \hline
1361: $d=6$ & 6.27 & 7.67 & 3.24 & 2.23 & 2.23 & 6.65 & 21041.05 \\ \hline
1362: $d=12$ & 14.64 & 15.23 & 3.63 & 2.92 & 2.92 & 16.03 & Large \\ \hline
1363: $d=24$ & 30.55 & 30.42 & 3.40 & 3.71 & 3.71 & 34.25 & Large \\ \hline
1364: $d=48$ & 61.96 & 60.95 & 3.27 & 4.89 & 4.89 & 69.65 & Large \\ \hline
1365: $d=96$ & 125.31 & 121.73 & 3.45 & 6.26 & 6.26 & 140.79 & Large \\ \hline
1366: $d=192$ & Unstable & 242.06 & Unstable & 9.31 & Unstable & Unstable & Large
1367: \\ \hline
1368: \end{tabular}
1369: \caption{Average iteration counts in \Sm\ generator tests.}
1370: \end{table}
1371:
1372: \begin{table}[h!bt]\label{ta:smm}
1373: \begin{tabular}{|l||c|c|c|c|c|c|c|} \hline
1374: ~ & \B & \BO & \Bp & \BOp & \H & \MV & \Rand \\ \hline \hline
1375: $d=3$ & 5 & 7 & 5 & 4 & 4 & 6 & 242 \\ \hline
1376: $d=6$ & 12 & 15 & 7 & 6 & 6 & 12 & 173941 \\ \hline
1377: $d=12$ & 25 & 25 & 8 & 9 & 9 & 25 & Large \\ \hline
1378: $d=24$ & 47 & 49 & 9 & 13 & 13 & 51 & Large \\ \hline
1379: $d=48$ & 101 & 94 & 13 & 22 & 22 & 95 & Large \\ \hline
1380: $d=96$ & 154 & 174 & 6 & 35 & 35 & 183 & Large \\ \hline
1381: $d=192$ & Unstable & 331 & Unstable & 69 & Unstable & Unstable & Large
1382: \\ \hline
1383: \end{tabular}
1384: \caption{Maximum iteration counts found in \Sm\ generator tests.}
1385: \end{table}
1386:
1387: \clearpage
1388:
1389:
1390: %
1391: % APPENDIX D: Times.
1392: %
1393: \section{Average time per iteration}\label{ap:tpi}
1394: In Table~\ref{ta:tpi} we give the average CPU time per iteration
1395: for our \rg\ experiments. This was computed using the MATLAB
1396: {\tt cputime} function.
1397:
1398: \begin{table}[h!bt]
1399: \begin{tabular}{|l||c|c|c|c|c|c|c|} \hline
1400: ~ & \B & \BO & \Bp & \BOp & \H & \MV & \Rand \\ \hline \hline
1401: $d=3$ & 0.0075 & 0.0009 & 0.0078 & 0.0019 & 0.0021 & 0.0012 & 0.0002 \\ \hline
1402: $d=6$ & 0.0087 & 0.0010 & 0.0095 & 0.0033 & 0.0035 & 0.0012 & 0.0002 \\ \hline
1403: $d=12$ & 0.0124 & 0.0013 & 0.0141 & 0.0073 & 0.0074 & 0.0016 & 0.0004 \\ \hline
1404: $d=24$ & 0.0229 & 0.0022 & 0.0267 & 0.0182 & 0.0184 & 0.0030 & 0.0007 \\ \hline
1405: $d=48$ & 0.0625 & 0.0043 & 0.0702 & 0.0474 & 0.0477 & 0.0085 & 0.0014 \\ \hline
1406: $d=96$ & 0.2510 & 0.0099 & 0.2608 & 0.1318 & 0.1324 & 0.0495 & 0.0035 \\ \hline
1407: $d=192$ & 1.5592 & 0.0277 & 1.2623 & 0.3275 & 0.3268 & 0.7843 & 0.0121 \\ \hline
1408: $d=384$ & Unstable & 0.1144 & Unstable & 1.1381 & Unstable & Unstable & 0.0619
1409: \\ \hline
1410: \end{tabular}
1411: \caption{Average iteration times on \rg\ generator tests.}\label{ta:tpi}
1412: \end{table}
1413:
1414: The time per iteration is fairly constant across problem types
1415: so we do not include data from the other generators. One
1416: difference that will occur is that \H\ will have a higher average
1417: iteration time as that \BOp\ for ill-conditioned problems.
1418: In random problems, we rarely see slow convergence of \BOp\
1419: so it is unnecessary to use the slower steps from \Bp.
1420: With ill-conditioned problems the \Bp\ steps become more frequent
1421: and increase the average time per iteration.
1422:
1423: %
1424: % BIBLIOGRAPHY
1425: %
1426: % Setup for bibtex
1427: \bibliographystyle{hacked}
1428: % End setup for bibtex.
1429:
1430: \bibliography{refs}
1431:
1432: %
1433: \end{document}
1434:
1435: