1: % ****** Start of file apssamp.tex ******
2: %
3: % This file is part of the APS files in the REVTeX 4 distribution.
4: % Version 4.0 of REVTeX, August 2001
5: %
6: % Copyright (c) 2001 The American Physical Society.
7: %
8: % See the REVTeX 4 README file for restrictions and more information.
9: %
10: % TeX'ing this file requires that you have AMS-LaTeX 2.0 installed
11: % as well as the rest of the prerequisites for REVTeX 4.0
12: %
13: % See the REVTeX 4 README file
14: % It also requires running BibTeX. The commands are as follows:
15: %
16: % 1) latex apssamp.tex
17: % 2) bibtex apssamp
18: % 3) latex apssamp.tex
19: % 4) latex apssamp.tex
20: %
21: \documentclass[aps,twocolumn,showpacs,preprintnumbers,amsmath,amssymb]{revtex4}
22: %\documentclass[preprint,showpacs,preprintnumbers,amsmath,amssymb]{revtex4}
23:
24: % Some other (several out of many) possibilities
25: %\documentclass[preprint,aps]{revtex4}
26: %\documentclass[preprint,aps,draft]{revtex4}
27: %\documentclass[prb]{revtex4}% Physical Review B
28:
29: \usepackage{graphicx}% Include figure files
30: \usepackage{dcolumn}% Align table columns on decimal point
31: \usepackage{bm}% bold math
32:
33: %\nofiles
34:
35: \begin{document}
36: %\renewcommand{\vec}[1]{{\bf #1}}
37: %\newcommand{\vecg}[1]{\mbox{\boldmath $#1$ }}
38: %\newcommand{\Hbar}{{\bf \overline{H}}}
39: %\newcommand{\Hright}{\stackrel{\rightarrow}{\bf H}}
40: %\newcommand{\Hleft}{\stackrel{\leftarrow}{\bf H}}
41: %\newcommand{\Hright}{\Hbar}
42: %\newcommand{\Hleft}{\Hbar}
43: \newlength{\figwidth}
44: %\newlength{\figheight}
45: \setlength{\figwidth}{0.5 \textwidth}
46: %\setlength{\figwidth}{\textwidth}
47: \addtolength{\figwidth}{-0.5 \columnsep}
48: \addtolength{\figwidth}{-1cm}
49: %\setlength{\figheight}{ 0.7 \figwidth}
50:
51: \preprint{}
52:
53: \title{Random phase vector
54: for calculating the trace of a large matrix}% Force line breaks with \\
55:
56: \author{Toshiaki Iitaka}
57: \email{tiitaka@riken.jp}
58: \homepage{http://www.iitaka.org/}
59: \author{Toshikazu Ebisuzaki}%
60: \affiliation{
61: Computational Astrophysics Laboratory, \\
62: RIKEN (The Institute of Physical and Chemical Research) \\
63: 2-1 Hirosawa, Wako, Saitama 351-0198, Japan}
64: % \altaffiliation{Computational Science Division, RIKEN (Institute of Physical and Chemical Research)}
65:
66: \date{\today}% It is always \today, today,
67: % but any date may be explicitly specified
68:
69: \begin{abstract}
70: We derive an estimate of statistical error in calculating the trace of a large matrix by
71: using random vector, and show that {\em random phase vector} gives the results with the smallest
72: statistical error for a given basis set.
73: This result supports use of random phase vectors in the calculation of density of states and
74: linear response functions of large quantum systems.
75: \end{abstract}
76:
77: \pacs{02.60.Dc,05.10.-a,05.30.-d,}% PACS, the Physics and Astronomy
78: % Classification Scheme.
79: \keywords{linear scaling, random vector, trace}%Use showkeys class option if keyword
80: %display desired
81: \maketitle
82:
83: %\section{}
84: In fast algorithm called {\em Order-N methods} for calculating DOS and linear response functions \cite{Skilling1989,Drabold1993,Silver1994,Wang1994a,Wang1994b,Sankey1994,Alben1975,Iitaka1995b,Iitaka1997,Hams2000,Gelman2003,Nomura1997,Nomura1999,Iitaka1999,Iitaka2003,DeVries1993}, Monte Carlo calculation of the trace of large matrices by using {\em random vector} often plays an important role. The central limit theorem guarantees the convergence of the result as the sample number $K$ increases. Further experience shows another useful feature called {\em self-averaging effect}: the fluctuation in some physical quantities such as energy density and linear response functions decreases as the dimension $N$ of the Hilbert space increases\cite{Iitaka1997,Hams2000,Gelman2003,Nomura1997,Nomura1999,Iitaka1999,Iitaka2003,DeVries1993}. These two types of convergence make the random vector a very efficient numerical tool.
85:
86: A special class of random vector called {\em random phase vector} \cite{Alben1975,Iitaka1995b,Iitaka1997} has been used in later papers \cite{Nomura1997,Nomura1999,Iitaka1999,Iitaka2003,Gelman2003} without examining its efficiency rigorously.
87: In this article, we prove , by following the scheme of \cite{Hams2000}, that random phase vector is really the most efficient random vector in a wide class of random vectors. Then we illustrate the mechanism of self-averaging with a simple model Hamiltonian.
88:
89: A {\it (complex) random vector} is defined by
90: \begin{equation}
91: \label{random_eq:randomvec}
92: |\Phi \rangle \equiv \sum_{n=1}^N |n \rangle \xi_n
93: \end{equation}
94: where $\{ |n \rangle \}$ is the basis set used in the computation and
95: $\xi_n$ are a set of complex random variables with identical probability distribution satisfying the statistical relations
96: \begin{eqnarray}
97: \label{eq:statistical1}
98: \left\langle \left\langle \ \xi_{n} \ \right\rangle \right\rangle &=& 0 \\
99: \label{eq:statistical2}
100: \left\langle \left\langle \ \xi_{n_1} \xi_{n_2} \ \right\rangle \right\rangle &=& 0 \\
101: \label{eq:statistical3}
102: \left\langle \left\langle \ \xi_{n_1}^* \xi_{n_2} \ \right\rangle \right\rangle &=& \delta_{n_1n_2}
103: \end{eqnarray}
104: where $\left\langle \left\langle \ \cdot \ \right\rangle \right\rangle \ $ stands for the statistical average.
105: This class of random vectors defined in one orthonormal basis set have the coefficients $\zeta_l$ in another orthonormal basis set $|l\rangle$ (for example, energy eigenstates $|E_l\rangle$ of a Hamiltonian $H$),
106: \begin{equation}
107: %\label{random_eq:randomvec.inH}
108: |\Phi \rangle = \sum_{l=1}^N |l \rangle \zeta_l
109: \end{equation}
110: that satisfy the same statistical relations as (\ref{eq:statistical1})-(\ref{eq:statistical3}):
111: Since the coefficients in the two basis sets are related by unitary transformation
112: \begin{eqnarray}
113: \label{random_eq:unitary}
114: \zeta_l &=& \sum_{n=1}^N \langle l | n \rangle \xi_n \\
115: \zeta_l^{*} &=& \sum_{n=1}^N \xi_n^{*} \langle n | l \rangle
116: ,
117: \end{eqnarray}
118: the statistical relations of $\zeta_n$ are derived as
119: \begin{eqnarray}
120: \left\langle \left\langle \ \zeta_{l} \ \right\rangle \right\rangle
121: &=& \sum_{n=1}^N
122: \langle l |n \rangle
123: \left\langle \left\langle \ \xi_{n} \ \right\rangle \right\rangle
124: = 0 \\
125: %\label{random_eq:stat.zeta}
126: \left\langle \left\langle \ \zeta_{l_1} \zeta_{l_2} \ \right\rangle \right\rangle
127: &=& \sum_{l_1=1}^N \sum_{l_2=1}^N
128: \langle l_1 |n_1 \rangle \langle l_2 | n_2 \rangle
129: \left\langle \left\langle \ \xi_{n_1} \xi_{n_2} \ \right\rangle \right\rangle = 0 \\
130: \label{random_eq:stat.zeta}
131: \left\langle \left\langle \ \zeta_{l_1}^* \zeta_{l_2} \ \right\rangle \right\rangle
132: &=& \sum_{l_1=1}^N \sum_{l_2=1}^N
133: \langle n_1 |l_1 \rangle \langle l_2 | n_2 \rangle
134: \left\langle \left\langle \ \xi_{n_1}^* \xi_{n_2} \ \right\rangle \right\rangle \nonumber \\
135: &=& \sum_{n=1}^N \langle l_2 | n \rangle \langle n | l_1 \rangle
136: %\nonumber \\
137: = \langle l_2 | l_1 \rangle
138: = \delta_{l_1l_2}.
139: \end{eqnarray}
140: In view of energy eigenstates, uniform random vector contains all eigenstates with equal probability and represents the system at a very high temperature.
141: The {\it orthonormality} and {\it completeness} of random vectors
142: \begin{eqnarray}
143: \label{random_eq:randomvector.normal}
144: \langle \langle \ \ \ \langle \Phi | \Phi \rangle \ \ \ \rangle \rangle &=& N \\
145: \label{random_eq:randomvector.ortho}
146: \langle \langle \ \ \ | \Phi \rangle \langle \Phi | \ \ \ \rangle \rangle &=& {\bf I}
147: \end{eqnarray}
148: where $\bf I$ is the identity operator, are shown by using (\ref{eq:statistical1})-(\ref{eq:statistical3}).
149:
150: The most important feature of random vectors is that statistical average of $\langle \Phi | X | \Phi \rangle$ gives the trace of $X$ as follows:
151: \begin{eqnarray}
152: \label{random_eq:trace.monte.basis}
153: && \left\langle \left\langle \ \langle \Phi | X | \Phi \rangle \ \right\rangle \right\rangle \nonumber \\
154: &=& \sum_{n}
155: X_{nn}
156: + \sum_{ n_1, n_2} \left\langle \left\langle \ \xi_{n_1}^* \xi_{n_2} -\delta_{n_1n_2} \ \right\rangle \right\rangle \ X_{n_1,n_2} \\
157: &=& \sum_{ n } X_{nn} = {\rm tr} \left[ X \right] \nonumber
158: .
159: \end{eqnarray}
160: For numerical evaluation of (\ref{random_eq:trace.monte.basis}) with $K$ samples of random vectors, the second term of (\ref{random_eq:trace.monte.basis}) gives the statistical fluctuation,
161: \begin{equation}
162: \label{eq:fluctuation0}
163: \delta X = \frac{1}{K} \sum_{k}\sum_{ n_1, n_2} \left( \xi_{n_1}^{*(k)} \xi_{n_2}^{(k)} -\delta_{n_1n_2} \right) X_{n_1n_2}.
164: \end{equation}
165: In the special case of Hermitian matrix $X_{n_1n_2}=X_{n_2n_1}^*$, it is shown that $\delta X$ becomes real number by adding the expression (\ref{eq:fluctuation0}) with the subscripts $n_1$ and $n_2$ exchanged.
166: The order of the fluctuation for a general matrix $X$ is estimated as
167: \begin{eqnarray}
168: \label{eq:fluctuation2}
169: \left| \delta X \right|^2 &=&
170: \frac{1}{K^2} \sum_{k,k'} \sum_{n_1n_2n_3n_4}
171: \left( \xi_{n_1}^{*(k)} \xi_{n_2}^{(k)} -\delta_{n_1n_2} \right) X_{n_1n_2} \nonumber \\
172: && \times \left( \xi_{n_3}^{(k')} \xi_{n_4}^{*(k')} -\delta_{n_3n_4} \right) X_{n_3n_4}^* \\
173: &=&
174: \frac{1}{K^2} \sum_{k,k'} \sum_{n_1n_2n_3n_4}
175: \xi_{n_1}^{*(k)} \xi_{n_2}^{(k)} \xi_{n_3}^{(k')} \xi_{n_4}^{*(k')} X_{n_1n_2}X_{n_3n_4}^* \nonumber \\
176: &+&
177: \frac{1}{K} \sum_{k} \sum_{n_1n_2n_3n_4}
178: \xi_{n_1}^{*(k)} \xi_{n_2}^{(k)} \left( -\delta_{n_3n_4} \right) X_{n_1n_2}X_{n_3n_4}^* \nonumber \\
179: &+&
180: \frac{1}{K} \sum_{k} \sum_{n_1n_2n_3n_4}
181: \left( -\delta_{n_1n_2} \right) \xi_{n_3}^{(k)} \xi_{n_4}^{*(k)} X_{n_1n_2}X_{n_3n_4}^* \nonumber \\
182: &+&
183: \sum_{n_1n_2n_3n_4} \delta_{n_1n_2}\delta_{n_3n_4} X_{n_1n_2}X_{n_3n_4}^* \nonumber
184: .
185: \end{eqnarray}
186: By taking statistical average of (\ref{eq:fluctuation2}) and carefully evaluating $\langle\langle \xi_{n_1}^{*(k)} \xi_{n_2}^{(k)} \xi_{n_3}^{(k')} \xi_{n_4}^{*(k')} \rangle\rangle$ by using (\ref{eq:statistical1})-(\ref{eq:statistical3}), we obtain
187: \begin{equation}
188: \label{eq:fluctuation}
189: \left| \delta X \right|^2 =
190: \frac{1}{K} \left\{
191: \left( \langle\langle |\xi_n|^4 \rangle\rangle -1\right) \sum_n |X_{nn}|^2
192: + \sum_{n_1 \ne n_2} |X_{n_1n_2}|^2
193: \right\}
194: \end{equation}
195: where the factor $\left( \langle\langle |\xi_n|^4 \rangle\rangle -1\right)$ is factored out because we assumed identical probability distribution for all $\xi_n$.
196: The factor $1/K$ ensures the behavior $|\delta X| \sim 1/\sqrt{K}$ expected from the central limit theorem.
197: According to the inequality
198: \begin{equation}
199: \left\langle \left\langle \ |\xi_n|^4 \ \right\rangle \right\rangle
200: \ge
201: \left( \left\langle \left\langle \ |\xi_n|^2 \ \right\rangle \right\rangle \right)^2
202: = 1
203: ,
204: \end{equation}
205: the fluctuation becomes the smallest for a given basis set if and only if $ |\xi_n|=1$ for each $n$ and for each sample of random variables.
206: One of such random vectors is called {\em random phase vector} \cite{Alben1975,Iitaka1995b,Iitaka1997} and defined by
207: \begin{equation}
208: \label{random_eq:randomphasevec}
209: |\Phi_{random \ phase} \rangle \equiv \sum_{n=1}^N |n \rangle e^{i\theta_n}
210: \end{equation}
211: where $\theta_n$ are a set of independent uniform random variables defined in $[-\pi,\pi]$.
212: Obviously, the random variables $\xi_n=e^{i\theta_n}$ satisfies the statistical relations
213: (\ref{eq:statistical1})-(\ref{eq:statistical3}).
214: Further, each sample of random phase vectors is automatically normalized without statistical fluctuation,
215: \begin{eqnarray}
216: \label{random_eq:normalize_randomphase}
217: &&\langle \Phi_{random \ phase}| \Phi_{random \ phase} \rangle
218: \nonumber \\
219: &=& \sum_{n',n} e^{i(\theta_n-\theta_n')} \langle n'| n \rangle
220: = \sum_{n=1}^N 1 =N
221: .
222: \end{eqnarray}
223: This result is consistent with the observation in \cite{Hams2000} that normalized random vectors give less statistical error than unnormalized random vectors.
224: It is important to notice that the definition of random phase vector, $|\xi_n|=1$, depends on the choice of basis set. For example, let us examine a unitary transformation of a random phase vector $\xi_i=e^{i\theta_i}, \ (i=1,2)$
225: \begin{eqnarray}
226: \zeta_1 &=& \frac{1}{\sqrt{2}} \xi_1 + \frac{1}{\sqrt{2}} \xi_2 \\
227: \zeta_2 &=& \frac{1}{\sqrt{2}} \xi_1 - \frac{1}{\sqrt{2}} \xi_2
228: .
229: \end{eqnarray}
230: The transformed random vector $\zeta_n$ of course satisfies the relation (\ref{eq:statistical1})-(\ref{eq:statistical3}), but $|\zeta_n|=1 \pm \cos(\theta_1-\theta_2)\ne 1$. Therefore the random phase vector in the original basis set is not a random phase vector in the new basis set.
231: For random phase vectors in a given basis set, the fluctuation (\ref{eq:fluctuation}) reduces to a simple form \cite{Iitaka1995b},
232: \begin{equation}
233: \label{eq:fluctuation_randomphase}
234: \left| \delta X \right|^2 =
235: \frac{1}{K} \sum_{n_1 \ne n_2} |X_{n_1n_2}|^2
236: \end{equation}
237: which becomes zero for diagonal matrices as expected.
238: Note that the fluctuation (\ref{eq:fluctuation_randomphase}) depends on the choice of basis set and that it is very important for reducing fluctuation to choose a basis set that makes off-diagonal matrix elements smallest.
239: For a Hermitian matrix $X$, in theory, we can choose a basis set that diagonalizes $X$ and remove the fluctuation completely.
240:
241: As a complex random vector (\ref{random_eq:randomvec}), a real random vector can be defined by using real random variables with identical probability distribution satisfying the statistical relations
242: \begin{eqnarray}
243: \label{eq:statistical1real}
244: \left\langle \left\langle \ \xi_{n} \ \right\rangle \right\rangle &=& 0 \\
245: \label{eq:statistical2real}
246: \left\langle \left\langle \ \xi_{n_1} \xi_{n_2} \ \right\rangle \right\rangle &=& \delta_{n_1n_2}.
247: \end{eqnarray}
248: Unlike a complex random vector, a real random vector in one basis set is not necessarily mapped by unitary formation (\ref{random_eq:unitary}) to a real random vector in another basis set. However, if we stick to the original basis set, the equations (\ref{random_eq:randomvector.normal})-(\ref{eq:fluctuation2}) are also valid for real random vectors. Before evaluating $|\delta X|^2$ for real random vectors, let us assume that $X$ is {\em symmetric} matrix for simplicity. This does not limit the generality of our argument because any matrix $X$ can be decomposed into the sum of symmetric and antisymmetric parts,
249: \begin{equation}
250: X=\frac{1}{2}(X+X^t) + \frac{1}{2}(X-X^t)=S+A
251: \end{equation}
252: where $X^t$ represents the transpose of $X$,
253: and ${\rm tr} [X]= {\rm tr} [S]$ since ${\rm tr} [A]=0$. Therefore if $X$ is not symmetric then we may calculate the trace of the symmetric part $S$ in place of $X$.
254: By taking statistical average of (\ref{eq:fluctuation2}) and carefully evaluating $\langle\langle \xi_{n_1}^{(k)} \xi_{n_2}^{(k)} \xi_{n_3}^{(k')} \xi_{n_4}^{(k')} \rangle\rangle$ by using (\ref{eq:statistical1real})-(\ref{eq:statistical2real}), we obtain
255: \begin{equation}
256: \label{eq:fluctuation_real}
257: \left| \delta X \right|^2 =
258: \frac{1}{K} \left\{
259: \left( \langle\langle |\xi_n|^4 \rangle\rangle -1\right) \sum_n |X_{nn}|^2
260: + 2 \sum_{n_1 \ne n_2} |X_{n_1n_2}|^2
261: \right\}
262: \end{equation}
263: As in the case of complex random vectors, the real random vectors with minimum fluctuation are random vectors with $\xi_n=\pm 1$, ({\em random sign vector}). Random sign vector may be regarded as a random phase vector with binary phase $\theta_n=0, \ \pi$ and satisfies the normalization (\ref{random_eq:normalize_randomphase}) without fluctuation.
264: For random sign vectors in a given basis set, the fluctuation (\ref{eq:fluctuation_real}) reduces to a simple form
265: \begin{equation}
266: \label{eq:fluctuation_randomsign}
267: \left| \delta X \right|^2 =
268: \frac{2}{K} \sum_{n_1 \ne n_2} |X_{n_1n_2}|^2
269: \end{equation}
270: which is twice of the fluctuation of random phase vectors (\ref{eq:fluctuation_randomphase}).
271: Therefore use of random phase vector rather than random sign vector is recommended except in special cases where the evaluation of matrix elements is substantially accelerated by using real numbers instead of complex numbers.
272:
273:
274: Let us illustrate efficiency of various types of random vectors in the case of the Hamiltonian operator for a particle moving in one-dimensional space under the influence of local potential $V(x)$,
275: \begin{equation}
276: H=\frac{p_x^2}{2m}+V(x)
277: .
278: \end{equation}
279: Discretizing the one-dimensional space of size $L=N\Delta x$ into $N$ meshes $x_i= i \Delta x, \ (i=1,\cdots,N)$ by finite difference method gives the matrix of the Hamiltonian, which is tridiagonal,
280: \begin{equation}
281: H_{ij}=\frac{\hbar}{2m(\Delta x)^2} \left( \delta_{i,j+1} -2\delta_{i,j}+\delta_{i,j-1}\right)
282: +\delta_{i,j} V(x_i)
283: .
284: \end{equation}
285: The fluctuation $|\delta H|^2$ for random phase vector, complex random Gaussian vector, random sign vector, and real Gaussian vector is respectively estimated as 2, 6, 4, and 12 in the unit of $\displaystyle \left( \frac{\hbar}{2m(\Delta x)^2} \right)^2$ by using the fact that $\langle\langle |\xi_n|^4 \rangle\rangle =1, 2, 1, \ {\rm and } \ 3$ respectively and the equations (\ref{eq:fluctuation}) and (\ref{eq:fluctuation_real}).
286: The fluctuation of random phase vector is 6 times less than that of real Gaussian random vector, which means that 6 times less sampling is necessary for a given accuracy.
287:
288: In contrast to the random vector dependence of fluctuation, it is rather difficult to discuss the self-averaging effect in a general way because the effect depends on the choice of basis set and also on the physical nature of the matrix $X$ such as whether the quantity is intensive or extensive.
289: Therefore let us first examine self-averaging effect of energy density in the above example.
290: The fluctuation for random phase vector becomes
291: \begin{equation}
292: \delta H/L \sim \left( \frac{\hbar}{2m(\Delta x)^3} \right) \frac{\sqrt{2}}{\sqrt{KN}}
293: \end{equation}
294: as $N \rightarrow \infty$, indicating self-averaging behavior $1/\sqrt{N}$.
295: It is interesting that the local potential $V(x)$ does not contribute to the fluctuation at all in the calculation with the real space basis set.
296: When the spectral density such as DOS and linear response function, the dimension $N$ of matrix in the above estimation should be replaced by $N_{eff}$, the number of resonances with in the spectral resolution $\eta$, e.g. $N_{eff}=\rho(\omega)\eta$ in case of DOS. Therefore to reach the same accuracy we need more random vectors for higher energy resolution or low temperatures. See Ref.~\cite{Hams2000} for more sophisticated analysis.
297: To understand general tendency of self-averaging, let us assume typical matrix elements of $X$ have value of O(1). Then the fluctuation $|\delta X|^2$ in (\ref{eq:fluctuation}) and (\ref{eq:fluctuation_real}) becomes $O(N)$ for sparse or banded matrices and $O(N^2)$ for dense matrices. Since the average value, ${\rm tr} [X]$, becomes $O(N)$, the relative fluctuation $\delta X/{\rm tr} [X]$ becomes $O(1/\sqrt{N})$ and $O(1)$, respectively. Therefore self-averaging is effective for sparse or banded matrices but not for dense matrices.
298:
299:
300:
301: In summary, we have proved that random phase vector is the most efficient choice among random vectors with identical probability distributions satisfying (\ref{eq:statistical1})-(\ref{eq:statistical3}). The fluctuation for random phase vector is expressed as the sum of the square norm of off-diagonal elements. We show also that the {\em smallness} and {\em sparseness} of the off-diagonal elements is important for the self-averaging effect. Therefore we conclude that it is important for efficient calculation to use random phase vector and to choose a basis set that makes the off-diagonal elements sparse (or banded) and small.
302:
303: \begin{acknowledgments}
304: One of the authors (T.I.) thanks S.~Nomura, T.~Tanaka, and M.~Machida for useful discussions.
305: \end{acknowledgments}
306:
307: %\bibliographystyle{unsort}
308: \bibliography{all}% Produces the bibliography via BibTeX.
309:
310: \end{document}
311:
312: