1: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
2: % name: eval.tex
3: % author: Leo Liberti
4: % purpose: performance comparison of different evaluation methods
5: % history: 010806 work started
6: % 020606 submitting to Journal of the ACM
7: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
8: \documentclass[10pt,a4paper]{article}
9: \usepackage{theorem} %additional LaTeX2e package (see pctex\latex2e)
10: \usepackage{graphicx} %graphic and .eps files
11:
12: \setlength{\parindent}{0.5cm}
13: \setlength{\parskip}{0.3cm}
14: \setlength{\oddsidemargin}{0.5cm}
15: \setlength{\textwidth}{15cm}
16: \setlength{\textheight}{22cm}
17: \setlength{\footskip}{1cm}
18: \setlength{\topmargin}{0cm}
19:
20: \renewcommand{\baselinestretch}{1.0}
21: \newtheorem{result}{\ }[section]
22: \theoremstyle{changebreak} % (see LATEX2E\THEOREM.DTX)
23: \newtheorem{thm}[result]{Theorem}
24: \newtheorem{defn}[result]{Definition}
25: \newtheorem{lem}[result]{Lemma}
26: \newtheorem{cor}[result]{Corollary}
27: \newtheorem{prop}[result]{Proposition}
28: \newtheorem{eg}[result]{Example}
29: \renewcommand{\vec}[1]{\underline{#1}}
30: \newcommand{\R}{\mathbb{R}}
31: \newcommand{\N}{\mathbb{N}}
32:
33: \title{Performance Comparison of Function Evaluation Methods}
34:
35: \author{Leo Liberti\thanks{Centre for Process Systems Engineering,
36: Imperial College of Science, Technology and Medicine, London,
37: UK. E-mail: {\tt l.liberti@ic.ac.uk}.}}
38:
39: \begin{document}
40:
41: \maketitle
42:
43: \begin{abstract}
44: We perform a comparison of the performance and efficiency of four
45: different function evaluation methods: black-box functions, binary
46: trees, $n$-ary trees and string parsing. The test consists in
47: evaluating 8 different functions of two variables $x,y$ over 5000
48: floating point values of the pair $(x,y)$. The outcome of the test
49: indicates that the $n$-ary tree representation of algebraic
50: expressions is the fastest method, closely followed by black-box
51: function method, then by binary trees and lastly by string
52: parsing.
53: \end{abstract}
54:
55: {\bf Keywords:}
56: function evaluation, tree, algebraic expression, parser
57:
58: \pagestyle{myheadings}
59: \thispagestyle{plain}
60: \markboth{L. LIBERTI}{FUNCTION EVALUATION METHODS}
61:
62: {\bf Important warning}. There is a mistake in the test code that
63: invalidates the most important result of this paper, i.e. that $n$-ary
64: tree based function evaluation is faster than black-box function
65: evaluation. It is not true: it is slower by about an order of
66: magnitude. However it is true that $n$-ary tree based evaluation is
67: faster than the other methods discussed in this paper.
68:
69: \section{Introduction}
70: In this article we describe a test designed to measure the comparative
71: efficiency of four different function evaluation methods (see section
72: \ref{desc} for details):
73: \begin{itemize}
74: \item black-box functions;
75: \item binary tree representation;
76: \item $n$-ary tree representation;
77: \item string parsing.
78: \end{itemize}
79: Because of the huge number of parameters involved in such a test
80: (efficiency of compiler, quality of test source code, type of
81: hardware, type of test functions, number of variables, object code
82: optimization level, and so on) it is evident that this test can be neither
83: definitive nor undebatable. However, the results indicate a clear
84: winner in the $n$-ary tree representation, closely followed by
85: black-box functions and binary tree representation. Last (expectedly)
86: comes string parsing. It is somewhat of a surprise to discover that
87: $n$-ary tree representation gathers better results than the
88: precompiled black-box functions method. This finding is discussed below
89: (section \ref{results}).
90:
91: Existing literature in this topic is scarce or non-existent. Some of
92: the early work focused on how to handle floating point computation
93: efficiently, rather than on the actual method used for the evaluation
94: \cite{ashenhurst}; in other instances, the evaluation of certain
95: classes of functions (e.g. polynomials, see \cite{fike}) was
96: investigated.
97:
98: Most people tend to use the black-box functions method by exploiting
99: the compiler's capabilities in this sense. Further efforts for better
100: evaluation methods are usually sought after only in connection to very
101: specific functions and problems, e.g. in astronomy \cite{schlitt},
102: in number theory \cite{odlyzko}, when using discrete/boolean
103: functions \cite{mcgeer}.
104:
105: A novel evaluation method, based on threaded binary trees, was
106: proposed in \cite{keeping}; this method partially eliminates the cost
107: of recursion by ``threading'' the binary expression tree before the
108: evaluation. Because the operation of threading the binary tree is
109: recursive in nature, the CPU time savings are only possible if the
110: same tree is evaluated many times (as is the case in most engineering
111: applications). However, the benefits of this approach should decrease
112: with the use of $n$-ary trees, as threading a list of like operators
113: in an $n$-ary tree has no effect.
114:
115: The test consists in evaluating eight different functions of one and
116: two variables $x,y$ over 5000 randomly generated pairs of values for
117: $(x,y)$, all in the interval $[0,1]$. The functions are:
118: \begin{enumerate}
119: \item $x$;
120: \item $x+y$;
121: \item $x^y$;
122: \item $(x+y)x^y$;
123: \item $\sin(x)$;
124: \item $\sin((x+y)x^y)$;
125: \item $x+y+1$;
126: \item $2xy(x+y+1)$.
127: \end{enumerate}
128: \label{functions}
129: The above functions have been chosen as a representative set for unary
130: and binary operators, in the sense that both binary operators (sum,
131: power) and unary operators (the sine function) are present. Little
132: does it matter for the outcome of the test that not all operator types
133: have been employed, for the time taken to carry out floating point
134: computation would have been exactly the same in all cases.
135:
136: The number of variables has been limited to two for
137: simplicity. However, for reasons which will become apparent below (see
138: section \ref{narytree} about the description of the $n$-ary tree
139: representation), adding more variables to the expressions would only
140: have served to overemphasize the outcome of the tests, especially in
141: the case where long sequences of like operands are employed
142: (e.g. linear equations, or products of the kind $x_1x_2x_3\cdots
143: x_n$).
144:
145: The test code has been written in pure C in order to minimize the
146: effect of compiler overhead, and compiled with the GNU C Compiler
147: version 2.95.2. All optimization options have been tested
148: ({\tt -O}, {\tt -O2}, {\tt -O3}) as well as ``debug'' mode and
149: ``normal'' (no flags) mode. In all cases the test results have been
150: consistent with the order: $n$-ary trees, black-box functions, binary
151: trees, string parsing.
152:
153: The test has been run on a Pentium-III 450MHz with 192MB RAM with the
154: Linux operating system. All results have been obtained by running the
155: executables in single-user mode and by flushing RAM caches after each
156: run. Enabling the caches and running in multi-user modes gathers
157: similar results but occasionally the black-box functions wins out (by
158: very little indeed) over the $n$-ary tree method. However, situations
159: where the black-box functions method wins over the other methods
160: cannot be replicated; they depend very much on the behaviour of the
161: caching code and on the general load of the machine at any given
162: moment. In any case, this kind of outcome only occurs when repeatedly
163: runnning the same executable over and over again as different
164: processes, {\it not} when calling the same function many times within
165: the same process.
166:
167: Section \ref{desc} describes the four types of evaluation methods
168: tested. Section \ref{code} describes the implementation of the
169: methods. Section \ref{results} discusses the results of the test.
170: The code used to run the test can be downloaded, inspected and reused
171: under the GNU public license from
172: \begin{quote}
173: {\tt http://liberti.dhs.org/liberti/evaltest}.
174: \end{quote}
175:
176: \section{Evaluation Methods}
177: \label{desc}
178: In this section we shall carry out a theoretical analysis of the
179: evaluation methods tested in this article.
180:
181: \subsection{Black-box Functions}
182: This method for function evaluation is by far the easiest to implement
183: and the most commonly used within the scientific community, especially
184: where test code has to be rigged up or once-only jobs need to be
185: run. It basically lets the compiler do the work of parsing the
186: expression into a binary tree which is hardwired in the object code at
187: compile time. Evaluations are supposed to be very fast (mainly because
188: most of the work is carried out once only at compile time); its main
189: drawback is that changes to the function formulation entail
190: recompilation of the source code, which for most pieces of software is
191: not an acceptable solution.
192:
193: The programming paradigm for black-box functions follows the lines of
194: the pseudocode below:
195: {\small
196: \begin{verbatim}
197: main:
198: float x, y, f;
199: f = blackbox(x, y);
200: end
201:
202: function blackbox(float x, float y):
203: float z;
204: z = sin((x + y)*x^y);
205: return z;
206: end function
207: \end{verbatim}
208: }
209:
210: The reason why this method is called ``black-box functions'' is that
211: from the main procedure point of view, the function is really a black
212: box, in the sense that apart from knowing what argument it requires,
213: there is no run-time control over it.
214:
215: \subsection{Binary Trees}
216: This representation is based on the idea that operators, variables and
217: constants are nodes of a digraph; binary operators have two outcoming
218: edges and unary operators have only one; leaf nodes have no outcoming
219: edges (for graph-related terminology and definitions, see \cite{harary},
220: \cite{korte}). For example, the function $x+y+1$ would be represented
221: as in fig. \ref{f:binary}.
222: \begin{figure}[h]
223: \begin{center}
224: \setlength{\unitlength}{4144sp}%
225: %
226: \begingroup\makeatletter\ifx\SetFigFont\undefined%
227: \gdef\SetFigFont#1#2#3#4#5{%
228: \reset@font\fontsize{#1}{#2pt}%
229: \fontfamily{#3}\fontseries{#4}\fontshape{#5}%
230: \selectfont}%
231: \fi\endgroup%
232: \begin{picture}(2311,2266)(308,-1509)
233: \thinlines
234: \put(1216,524){\circle{450}}
235: \put(541,-421){\circle{450}}
236: \put(1801,-421){\circle{450}}
237: \put(1216,-1276){\circle{450}}
238: \put(2386,-1276){\circle{450}}
239: \special{ps: gsave 0 0 0 setrgbcolor}\put(1216,299){\line(-4,-3){669.600}}
240: \special{ps: grestore}\special{ps: gsave 0 0 0 setrgbcolor}\put(1216,299){\line( 6,-5){588.688}}
241: \special{ps: grestore}\special{ps: gsave 0 0 0 setrgbcolor}\put(1801,-646){\line(-3,-2){591.923}}
242: \special{ps: grestore}\special{ps: gsave 0 0 0 setrgbcolor}\put(1801,-646){\line( 3,-2){591.923}}
243: \special{ps: grestore}\put(1171,479){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{\rmdefault}{\mddefault}{\updefault}\special{ps: gsave 0 0 0 setrgbcolor}+\special{ps: grestore}}}}
244: \put(496,-466){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{\rmdefault}{\mddefault}{\updefault}\special{ps: gsave 0 0 0 setrgbcolor}$x$\special{ps: grestore}}}}
245: \put(1756,-466){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{\rmdefault}{\mddefault}{\updefault}\special{ps: gsave 0 0 0 setrgbcolor}$+$\special{ps: grestore}}}}
246: \put(1126,-1321){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{\rmdefault}{\mddefault}{\updefault}\special{ps: gsave 0 0 0 setrgbcolor}$y$\special{ps: grestore}}}}
247: \put(2341,-1321){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{\rmdefault}{\mddefault}{\updefault}\special{ps: gsave 0 0 0 setrgbcolor}1\special{ps: grestore}}}}
248: \end{picture}
249: \end{center}
250: \caption{Binary tree representation for $x+y+1$.}
251: \label{f:binary}
252: \end{figure}
253: Where unary operators are employed, a dummy second operand is often used.
254:
255: This type of function representation is the most commonly used where
256: there is a need for some degree of run-time control over the
257: definition of the mathematical function being represented. However, it
258: should be noted that performing algebraic operations on this
259: representation is not overly simple. Most software that does not do
260: symbolic manipulation of algebraic expressions employs this kind of
261: representation. Furthermore, most general-purpose compilers (including
262: the GNU C compiler) use this representation too.
263:
264: \subsection{$n$-ary Trees}
265: \label{narytree}
266: This technique is a combination of binary trees and lists. Operators
267: can have any number of operands. This allows for much more efficient
268: handling of sequences of like operands, e.g. in linear expressions or
269: long products ($x_1x_2\cdots x_n$). For example, the function $x+y+1$
270: would be represented as in fig. \ref{f:nary}.
271: \begin{figure}[h]
272: \begin{center}
273: \setlength{\unitlength}{4144sp}%
274: %
275: \begingroup\makeatletter\ifx\SetFigFont\undefined%
276: \gdef\SetFigFont#1#2#3#4#5{%
277: \reset@font\fontsize{#1}{#2pt}%
278: \fontfamily{#3}\fontseries{#4}\fontshape{#5}%
279: \selectfont}%
280: \fi\endgroup%
281: \begin{picture}(2491,1411)(308,-654)
282: \thinlines
283: \put(541,-421){\circle{450}}
284: \put(496,-466){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{\rmdefault}{\mddefault}{\updefault}\special{ps: gsave 0 0 0 setrgbcolor}$x$\special{ps: grestore}}}}
285: \put(2566,-421){\circle{450}}
286: \put(2521,-466){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{\rmdefault}{\mddefault}{\updefault}\special{ps: gsave 0 0 0 setrgbcolor}1\special{ps: grestore}}}}
287: \put(1576,524){\circle{450}}
288: \put(1531,479){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{\rmdefault}{\mddefault}{\updefault}\special{ps: gsave 0 0 0 setrgbcolor}$+$\special{ps: grestore}}}}
289: \put(1576,-421){\circle{450}}
290: \put(1486,-466){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{\rmdefault}{\mddefault}{\updefault}\special{ps: gsave 0 0 0 setrgbcolor}$y$\special{ps: grestore}}}}
291: \special{ps: gsave 0 0 0 setrgbcolor}\put(1576,299){\line(-2,-1){1026}}
292: \special{ps: grestore}\special{ps: gsave 0 0 0 setrgbcolor}\put(1576,299){\line( 0,-1){495}}
293: \special{ps: grestore}\special{ps: gsave 0 0 0 setrgbcolor}\put(1576,299){\line( 2,-1){990}}
294: \special{ps: grestore}\end{picture}
295: \end{center}
296: \caption{$n$-ary tree representation for $x+y+1$.}
297: \label{f:nary}
298: \end{figure}
299:
300: This type of function representation is often employed when symbolic
301: manipulation is required. The data structures used by languages like
302: Prolog and especially LISP are very similar in concept to $n$-ary
303: trees.
304:
305: \subsection{String Parsing}
306: String parsing is a process by which a string containing an algebraic
307: expression is evaluated directly without middle steps like tree
308: representation. String parsers usually consist of a lexical analyser,
309: which returns tokens (operators), symbols and constants, and a
310: grammatical interpreter which drives the lexical analyser on the given
311: string. It then performs the mathematical operations signalled by the
312: tokens on the operands. When a symbol is returned, it is looked up on
313: a symbol table to discover its numeric value. Because of the design
314: complexity, it is to be expected that string parsing is slower than
315: other methods. It is mainly employed where the parsing is to be
316: carried out once only (possibly as a pre-processing step to some main
317: algorithm).
318:
319: By changing the grammatical interpreter, a string parser can be used
320: to build binary or $n$-ary trees for algebraic expressions input as
321: strings. This is how compilers transform source code
322: (i.e. strings) into object code.
323:
324: \section{Implementation in the C Language}
325: \label{code}
326:
327: The implementation of the techniques described in section \ref{desc}
328: above has been carried out in C in order to minimize the amount of
329: compiler-generated overhead code, as the C language has very low
330: requirements in this respect. Furthermore, no external library has
331: been used as it would have invalidated the timing tests. Instead,
332: all the code necessary to the test has been written from scratch.
333:
334: The part of the test concerning black-box functions was the easiest to
335: code. No particular coding technique was employed. Functions returning
336: results of the test expressions were compiled into the executable
337: and called from the main routine.
338:
339: Tree handling, in both the binary and $n$-ary forms, required more
340: work. A tree, for the purposes of this test, is defined as follows.
341: {\small
342: \begin{verbatim}
343: struct tree {
344: int optype; // operator type
345: long varindex; // variable index if node is a variable
346: double value; // value if node is a constant
347: struct tree** nodes; // subnodes if node is not a leaf node
348: long nodesize; // number of subnodes
349: };
350: \end{verbatim}
351: }
352: The above definition is generic enough to be able to accommodate both
353: binary and $n$-ary trees. For binary trees, {\tt nodesize} is always
354: set to 2. No ``string to tree'' parser has been included as the test
355: code did not need that kind of generality; all the function trees
356: (both binary and $n$-ary) have been manually coded in.
357:
358: The string parser has been derived from the ideas given in
359: \cite{stroustrup}. The parser code given therein has been modified to
360: support exponentiation and unary functions in the form $f(x)$.
361:
362: \subsection{Code Validation}
363: The validity of the code has been verified along with the test
364: proper. All results from evaluations with the four methods described
365: above coincide (up to at least three significant digits) for each of
366: the test expressions.
367:
368: \section{Results}
369: \label{results}
370:
371: As has been mentioned in the introduction, the test consists in
372: evaluating the eight expressions above (see page \pageref{functions}) over
373: 5000 randomly generated pair values for $(x,y)$ (all in the interval
374: $[0,1]$) with each of the four described evaluation methods, trying
375: all the possible compiler code optimization flags. The test is carried
376: out in a single-tasking environment where memory cache has little or
377: no effect. The results of the test are reported in table \ref{t:results}.
378:
379: \begin{table}
380: \begin{center}
381: \begin{tabular}{|l|r|r|r|r|r|} \hline Test Results
382: &\multicolumn{5}{|c|}{\bf Compiler Code Optimization Level} \\ \hline
383: {\bf Evaluation Method} &{\sc Normal} & {\sc Debug} ({\tt -g}) &
384: {\tt -O} & {\tt -O2} & {\tt -O3} \\
385: \hline
386: {\it Black-box functions} &0.76s & 0.76s & 0.75s & 0.74s & 0.73s \\
387: {\it Binary trees} & 0.84s & 0.84s & 0.80s & 0.82s & 0.82s \\
388: {\it $n$-ary trees} & 0.73s & 0.74s & 0.70s & 0.70s & 0.70s \\
389: {\it String parsing} & 2.88s & 2.97s & 2.63s & 2.23s & 2.56s \\ \hline
390: \end{tabular}
391: \end{center}
392: \caption{Test results. Values are expressed in seconds of ``user'' CPU
393: time (time spent on system calls was 0.00s in all cases).}
394: \label{t:results}
395: \end{table}
396:
397: These results are surprising because normally we would expect
398: black-box functions to be the most efficient evaluation
399: method, whereas the actual ``test winner'' is the $n$-ary tree
400: representation (although, as has been noted in the introduction, this
401: test is far from definitive --- the parameters that can affect
402: performance are too many to be controlled all at once). This
403: result is strengthened by the consideration that black-box functions
404: are so easy to program that the test cannot be invalidated because of
405: ``programming errors'' or inefficient coding. On the other hand, it
406: may be true that with more careful coding, the results referring to
407: tree evaluation could be made even better.
408:
409: \subsection{$n$-ary Trees: the Best Evaluation Method}
410: In order to explain this result, one has to consider the similarities
411: between black-box functions and binary trees. Although from the
412: programmer's point of view the two methods are far from similar, the
413: resulting machine code need not be all that different. As has been
414: explained earlier, most compilers work in such a way as to hard-code
415: binary trees representing the expressions within the object code. This
416: binary tree structure may be hidden behind a simpler logic flow than
417: that generated by the binary tree method, but the operations are
418: carried out much in the same order in both methods. Thus, it comes to
419: no surprise that the binary tree method gathers results which are
420: worse, but not by much, than those of black box functions. The two
421: methods are very similar, but in the black-box function case the code
422: is created directly by the compiler and can be better optimized.
423:
424: The $n$-ary tree method performs in the same way as the binary tree
425: method, except where sequences of like operands with length greater
426: than 2 appear in the expressions (e.g. in linear expressions with at
427: least 3 nonzero coefficients), where it performs much faster. The
428: evaluation algorithm is as follows:
429: {\small
430: \begin{verbatim}
431: double eval(struct tree* expression, double varvalues) {
432: int i;
433: double ret;
434: switch(expression->optype) {
435: case CONSTANT:
436: return expression->value; // leaf node
437: case VARIABLE:
438: return varvalues[expression->varindex]; // leaf node
439: case SUM:
440: ret = 0;
441: for (i = 0; i < expression->nodesize; i++) {
442: ret += eval(expression->nodes[i], varvalues); // recursive call
443: }
444: return ret;
445: case PRODUCT:
446: ret = 1;
447: for (i = 0; i < expression->nodesize; i++) {
448: ret *= eval(expression->nodes[i], varvalues); // recursive call
449: }
450: return ret;
451: // ... all other cases
452: }
453: }
454: \end{verbatim}
455: }
456: Consider the cases depicted in fig. \ref{f:binary} and \ref{f:nary}. In
457: the first case, where the {\tt nodesize} is always 2, a
458: recursive function call has to be performed for each of the two '+'
459: operators; in the second case, however, only one recursive function
460: call needs to be carried out (for the only '+' operator).
461:
462: This explains the advantage of using $n$-ary trees in evaluation of
463: algebraic expressions. The computational cost of generating a
464: recursive function call is high for most compilers. It becomes evident
465: that the longer ``like operand sequences'' are, the better this
466: evaluation method becomes.
467:
468: \subsection{Results on String Parsing}
469: String parsing, although definitely the worst method, was also
470: somewhat surprising in that it was {\it not} as bad as a superficial
471: analysis would suggest. After all the code complexity of a lexical
472: analyser and a grammatical interpreter is far greater than the other
473: evaluation algorithms presented above. However, especially when memory
474: caching was allowed, the timings of the string parsing method got
475: better and better. The best result we obtained was close to 1.00s; so
476: even though it still worse than the other methods, it was the
477: technique that benefited most from caching (in different processes,
478: however, not in the same process). However, the same test carried
479: out using the popular {\sc Unix} utility {\tt bc} (in most
480: implementations based around the lex and yacc compiler tools)
481: gathered an appalling result of over 26s of user CPU time,
482: notwithstanding the fact that Stroustrup's adapted parser is highly
483: recursive in nature whereas lex and yacc usually generate
484: non-recursive (and hence theoretically faster) code.
485:
486: \section{Conclusion}
487: In this paper we have analysed the performance of four common
488: evaluation methods: black-box functions, binary trees, $n$-ary trees
489: and string parsing. The result of the test indicates that the $n$-ary
490: tree expression representation is the best function evaluation method.
491:
492: % end paper
493:
494: \bibliographystyle{alpha}
495:
496: \newcommand{\etalchar}[1]{$^{#1}$}
497: \begin{thebibliography}{MMS{\etalchar{+}}95}
498:
499: \bibitem[Ash64]{ashenhurst}
500: R.L. Ashenhurst.
501: \newblock Function evaluation in unnormalized arithmetic.
502: \newblock {\em Journal of the ACM}, 11(2):168--187, April 1964.
503:
504: \bibitem[Fik67]{fike}
505: C.T. Fike.
506: \newblock Methods of evaluating polynomial approximations in function
507: evaluation routines.
508: \newblock {\em Communications of the ACM}, 10(3):175--178, March 1967.
509:
510: \bibitem[Har71]{harary}
511: Frank Harary.
512: \newblock {\em Graph Theory}.
513: \newblock Addison-Wesley, Reading, MA, 2nd edition, 1971.
514:
515: \bibitem[KP97]{keeping}
516: B.R. Keeping and Constantinos~C. Pantelides.
517: \newblock Novel methods for the efficient evaluation of stored mathematical
518: expressions on scalar and vector computers.
519: \newblock {\em AIChE Annual Meeting}, Paper \#204b, nov 1997.
520:
521: \bibitem[KV00]{korte}
522: Bernhard Korte and Jens Vygen.
523: \newblock {\em Combinatorial Optimization, Theory and Algorithms}.
524: \newblock Springer Verlag, Berlin, 2000.
525:
526: \bibitem[MMS{\etalchar{+}}95]{mcgeer}
527: Patrick~C. McGeer, Kenneth~L. McMillan, Alexander Saldanha, Alberto~L.
528: Sangiovanni-Vincentelli, and Patrick Scaglia.
529: \newblock Fast discrete function evaluation using decision diagrams.
530: \newblock In {\em Proceedings of the Conference on Computer Aided Design},
531: pages 402--407, San Jose, CA, 1995. IEEE/ACM.
532:
533: \bibitem[Odl90]{odlyzko}
534: A.M. Odlyzko.
535: \newblock Primes, quantum chaos, and computers.
536: \newblock In {\em Number Theory}, pages 35--46. National Research Council,
537: 1990.
538:
539: \bibitem[Sch00]{schlitt}
540: Wayne Schlitt.
541: \newblock The xstar $n$-body solver: Theory of operation.
542: \newblock {\em {\tt http://www.midwestcs.com/xstar/n-body}}, February 2000.
543:
544: \bibitem[Str99]{stroustrup}
545: Bjarne Stroustrup.
546: \newblock {\em The C++ Programming Language}.
547: \newblock Addison-Wesley, Reading, MA, third edition, 1999.
548:
549: \end{thebibliography}
550:
551: \end{document}
552: