1: %\documentclass[10pt,preprint]{aastex}
2: %\documentclass[12pt,preprint]{aastex}
3: %\documentclass[preprint2]{aastex}
4: %\documentclass[twocolumn]{emulateapj}
5: %\documentclass[onecolumn]{emulateapj}
6: \documentclass[apj]{emulateapj}
7: \usepackage{apjfonts}
8: %\usepackage{amsmath, amsthm, amssymb}
9:
10: \begin{document}
11:
12: \title{Probabilistic Cross-Identification of Astronomical Sources}
13: \journalinfo{To appear in the Astrophysical Journal}
14:
15: \author{Tam\'as Budav\'ari and Alexander S. Szalay}
16: \affil{Dept.\ of Physics and Astronomy, The Johns Hopkins University, 3400 North Charles Street, Baltimore, MD 21218}
17: \affil{Max-Planck-Institute f\"ur Astrophysik, Karl-Schwarzschild-Strasse 1, 85748 Garching, Germany}
18:
19: \shortauthors{Budav\'ari and Szalay}
20: \shorttitle{Probabilistic Cross-Identification of Astronomical Sources}
21:
22: \begin{abstract}
23: We present a general probabilistic formalism for cross-identifying
24: astronomical point sources in multiple observations. Our Bayesian approach,
25: symmetric in all observations, is
26: the foundation of a unified framework for object matching, where not only
27: spatial information, but physical properties, such as
28: colors, redshift and luminosity,
29: can also be considered in a natural way.
30: We provide a practical recipe to implement an efficient recursive algorithm to
31: evaluate the Bayes factor over a set of catalogs with known circular errors
32: in positions.
33: This new methodology is crucial for
34: studies leveraging the synergy of today's multi-wavelength observations and
35: to enter the time-domain science of the upcoming survey telescopes.
36: \end{abstract}
37:
38: \keywords{astrometry --- catalogs --- galaxies: statistics --- methods: statistical}
39:
40:
41: \section{Motivation} \label{sec:intro}
42:
43: Observational astronomy has changed drammatically over the last decade.
44: With the introduction of large-format, high-resolution detectors at
45: all wavelengths of the electromagnetic spectrum,
46: astronomers now face an avalanche of data pouring from the instruments of
47: dedicated telescopes. While most imaging surveys today obtain multicolor
48: information, no one telescope can cover the entire spectrum because the
49: physics of the detectors is very different at different frequencies.
50: %
51: To fully utilize the available observations, e.g.,
52: to boost the chances of discovering new kinds of sources, and understanding
53: the underlying physical relations of object properties in a statistical way,
54: one needs to merge the datasets of various telescopes by federating the archives.
55: %
56: The Virtual Observatory
57: initiative spearheaded by the International Virtual Observatory Alliance%
58: \footnote{http://www.ivoa.net} (IVOA)
59: is pursuing automated data exchange protocols with catalog cross-identification,
60: and the US National Virtual Observatory%
61: \footnote{http://us-vo.org} (NVO) is building tools, e.g., Open SkyQuery
62: \citep{budavari04} to facilitate a standard unified framework.
63: %
64: The key step in
65: the process is the cross-identification of the sources in multiple catalogs
66: to link observations of one telescope to other's.
67: %
68: Previous attempts to alleviate the problem utilized likelihood analysis
69: \citep{ss92} and machine learning (ML) techniques \citep{rohde06} that
70: addressed specific issues of the matching problem of two catalogs.
71: %
72: \citet{mann97} successfully applied the former likelihood ratio method
73: to associate sources in the
74: Infrared Space Observatory and the Hubble Deep Field catalogs,
75: and the ML techniques were used to study the SuperCOSMOS observations
76: and HI Parkes All Sky Survey.
77:
78: Today astronomers typically join two catalogs by setting some threshold on the angular
79: separation of sources that is motivated by the astrometric accuracies of the
80: datasets involved. When more than two catalogs are to be crossmatched,
81: astronomers often hatch a chaining rule based on the implicit
82: prior knowledge about the sources. For example, one might decide to match
83: all lower-accuracy datasets to the best one, or to go from wavelength to
84: wavelength, hoping that the sources do not change significantly over a shorter
85: wavelength range.
86: %
87: The problem with these traditional ways is not that they are based on implicit
88: assumptions and intuitions but that they are not symmetric. While the pairwise
89: matches might be acceptable, there is no guarantee, or any measure of quality,
90: that the elected final matches are plausible or if the list is complete.
91: After all picking a different order of pairwise matching would yield a different
92: catalog.
93:
94: We need algorithms that are symmetric in the catalogs and provide a reliable
95: measure of quality that one can use to exclude or downweight unlikely combinations of
96: sources. We need a unified framework, where on top of the spatial
97: information, other measurements can also be incorporated along with explicit
98: models and physical priors.
99: %
100: In Section~\ref{sec:bf} we discuss the Bayesian approach to address these issues,
101: and in Section~\ref{sec:norm} the spherical normal distribution is studied.
102: %
103: In Sections~\ref{sec:physics} we demonstrate how to
104: include other observational evidence such as from multicolor photometric measurements.
105: %
106: Section~\ref{sec:thres} focuses on the effects of a limited field of view on
107: the observational evidence, the prior and posterior probabilities.
108: %
109: In Section~\ref{sec:prac} an efficient implementation of the framework is
110: described in the detail, and Section~\ref{sec:sum} concludes our study.
111:
112: Throughout the paper we follow the usual convention of using the lower-case $p$ symbol
113: for representing probability density functions and the capital $P$ symbol for probabilities.
114:
115:
116: \section{Observational Evidence} \label{sec:bf}
117:
118: Often Bayesian analysis is refered to as the calculus of belief, however, it
119: should rather be thought of as the calculus of observational evidence.
120: %
121: When presented with a series of observed positions, one would like to know
122: whether they are truely from the same source. If the coordinates are scattered
123: all over the celestial sphere, it seems very unlikely that they are
124: measurements of the same astronomical object, but when the coordinates are only
125: a tiny fraction of an arcsecond apart, we ``know'' that we found a good match.
126: How good is that match? Or what is the evidence that it is a match?
127:
128:
129: \subsection{Modelling the Astrometry}
130:
131: First let us examine what astrometric precision means. In the process of
132: calibrating the positions in a catalog of extracted sources, one can
133: characterize the properties of the observations by comparing the positions to
134: astrometric standards, and even correct for systematic offsets. Yet, there
135: remains a random scatter around the true positions. This uncertainty is often
136: modelled as a normal distribution, and catalogs would quote a
137: single $\sigma$-value for their accuracy, e.g., $\sigma=0.1$ arcseconds.
138: %
139: In general, our understanding of the astrometry is described by a
140: probability density function (PDF) that may even vary on the sky.
141: We parameterize our model $M$ that the object is on the celestial sphere using a
142: three-dimensional normal vector $\vec{m}$, and write
143: $p(\vec{x}|\vec{m},M)$ for the probability density that an object at its
144: true location $\vec{m}$ is observed at a position $\vec{x}$.
145: As any PDF, this function is normalized,
146: \begin{equation}
147: \int\,p(\vec{x}|\vec{m},M)\,d^3\!x = 1
148: \end{equation}
149: %
150: Now we take a single source observed at $\vec{x}_1$ and apply Bayes'
151: theorem to find the posterior density of the true location $\vec{m}$ given
152: the obtained data,
153: \begin{equation}
154: p(\vec{m}|\vec{x}_1,M)=\frac{p(\vec{x}_1|\vec{m},M)p(\vec{m}|M)}{p(\vec{x}_1|M)}
155: \end{equation}
156: where the trivial prior $p(\vec{m}|M)$ of $\vec{m}$ being on the
157: celestial sphere is expressed with Dirac's $\delta$-symbol,
158: \begin{equation}
159: p(\vec{m}|M) = \frac{1}{4\pi} \delta(|\vec{m}|-1)
160: \end{equation}
161: and the normalizing constant guarantees the law of total probability,
162: \begin{equation}
163: p(\vec{x}_1|M) = \int p(\vec{m}|M)\,p(\vec{x}_1|\vec{m},M)\,d^3\!m
164: \end{equation}
165: %
166: Another interesting direct application is the calculation of the chance that
167: we find a visible object at position $\vec{m}$ in
168: a given footprint. If the angular window function is $\Omega$, this probability
169: is simply
170: \begin{equation}
171: P(\Omega|\vec{m},M) = \int_\Omega p(\vec{x}|\vec{m},M)\,d^3\!x
172: \end{equation}
173: which one can use to infer the PDF on the true position by applying
174: Bayes' rule
175: \begin{equation} \label{eq:win}
176: p(\vec{m}|M_{\Omega}) \equiv{}\, p(\vec{m}|\Omega,M) = \frac{p(\vec{m}|M)P(\Omega|\vec{m},M)}
177: {\displaystyle\int p(\vec{m}|M)P(\Omega|\vec{m},M)\,d^3\!m}
178: \end{equation}
179: %
180: This is our best understanding of where an object might be on the sky
181: (prior to measuring its actual position)
182: that is
183: seen in the specified $\Omega$ footprint assuming astronometric precision
184: $p(\vec{x}|\vec{m},M)$ derived from the calibration.
185:
186: \subsection{The Bayes Factor}
187:
188: With multiple observations through various instruments of possibly different
189: astrometric accuracies, we now turn to compute the evidence that all
190: observations are from the same source.
191: %
192: We introduce the Bayes factor to test this hypothesis $H$ against the case
193: when separate sources are possible, $K$.
194: After the observation are obtained,
195: \mbox{$D=\{\vec{x}_1,\vec{x}_2, \dots, \vec{x}_n\}$} locations on the sky,
196: we compute the ratio of the posterior and prior probabilities of each hypothesis.
197: The Bayes factor is defined as the ratio of these odds,
198: \begin{eqnarray} \label{eq:bf1}
199: B(H,K|D)&=&\left(\frac{P(H|D)}{P(H)}\right)\Big/\left(\frac{P(K|D)}{P(K)}\right)
200: \end{eqnarray}
201: which, after applying Bayes' theorem, becomes
202: \begin{eqnarray} \label{eq:bf2}
203: B(H,K|D) & = & \frac{p(D|H)}{p(D|K)}
204: \end{eqnarray}
205: for continuous observables.
206: %
207: The actual calculation is done by parameterizing the two models $H$ and $K$, and
208: integrating the likelihood functions for the entire configuration space.
209: Our hypothesis $H$ says that the positions are from a single source, thus
210: can be parameterized by a single common location $\vec{m}$.
211: Due to the independence of the measurements in $D$, the joint PDF is just
212: the product of the astrometric precisions $p_1, p_2, \dots, p_n$,
213: and the integral simplifies to
214: \begin{equation}
215: p(D|H) = \int p(\vec{m}|H) \prod_{i=1}^{n} p_i(\vec{x}_i|\vec{m},H)\,d^3\!m
216: \end{equation}
217: %
218: On the other hand, the alternative hypothesis $K$ is parameterized by
219: separate $\{\vec{m}_i\}$
220: positions, and the integral factorizes into the product of the independent
221: components
222: \begin{equation}
223: p(D|K) = \prod_{i=1}^{n} \left\{\int p(\vec{m}_i|K)\,p_i(\vec{x}_i|\vec{m}_i,K)\,d^3\!m_i \right\}
224: \end{equation}
225: %
226: When the Bayes factor is large, the observations support the hypothesis that the
227: association is a match, if it is in the order of unity, the evidence is not convincing,
228: and finally if the ratio is less than one, the data prefers the alternative
229: hypothesis.
230:
231:
232: \section{The Normal Distribution} \label{sec:norm}
233:
234: Normal distributions emerge often in nature, where a number of effects play
235: roles in shaping up the probability density, cf.~the Central Limit theorem.
236: Although many of the usual arguments do not hold over closed topological
237: manifolds,
238: %
239: e.g., the Central Limit theorem leads to isotropic distribution on the circle \citep{levy39},
240: %
241: it is possible to introduce an analogue to the normal distribution
242: function on the sphere \citep{fisher53,breitenberger63}.
243: The spherical normal distribution is often elected to
244: characterize the precision of astronomy observations, hence it is of great
245: importance to understand its properties, and to apply the Bayesian framework
246: described in the previous section.
247:
248: The spherical normal distribution in its normalized form using the previous
249: 3-D vector notation is written as
250: \begin{equation}
251: N(\vec{x}|\vec{m},w) = \frac{w\,\delta(|\vec{x}|\!-\!1)}{4\pi \sinh w}\
252: \exp \left( {w\,\vec{m}\vec{x}} \right)
253: \end{equation}
254: where the weight $w$ is typically very large. When this is the case, the weight
255: is related to
256: the more intuitive precision parameter $\sigma$ by the equation
257: \begin{equation}
258: w = 1/\sigma^2
259: \end{equation}
260: For example, when $\sigma$ is in the order of an arcsecond, the weight takes values of
261: $\sim 10^{10}$.
262: %
263: Having observed a set of positions independently with corresponding weights,
264: we can compute
265: the Bayes factor for the two hypotheses $H$ and $K$ introduced earlier.
266: %
267: Because the function $N(\vec{x}|\vec{m}, w) p(\vec{m}|M)$ is symmetric in
268: $\vec{x}$ and $\vec{m}$ for the trivial prior, and the PDFs are normalized,
269: the Bayes factor is computed analytically, and becomes
270: \begin{equation} \label{eq:sinh}
271: B(H,K|D)=\frac{\sinh{}w}{w}\,\prod_{i=1}^{n} \frac{w_i}{\sinh{}w_i}
272: \end{equation}
273: with
274: \begin{equation}
275: w = \left| \sum_{i=1}^{n} w_i \vec{x}_i \right|
276: \end{equation}
277: where we exploit the fact that the product of normal distributions has the same
278: functional form. A detailed derivation is given in Appendix~\ref{app:1}.
279:
280: In case of only two observations, this weight depends on the astrometric
281: precisions and the angle $\psi$ between the positions
282: \begin{equation}
283: w = \sqrt{w_1^2 + w_2^2 + 2 w_1 w_2 \cos \psi}
284: \end{equation}
285: For the typical large weights and small angular separations between the
286: measurements, we get
287: \begin{equation}
288: B = \frac{2}{\sigma^2_1 + \sigma^2_2}
289: \exp \left\{-\frac{\psi^2}{2(\sigma^2_1 + \sigma^2_2)} \right\}
290: \end{equation}
291: In Figure~\ref{fig:ex} the 10-based logarithm of the Bayes factor, also known as
292: the weight of evidence, is shown as a function of angular separation
293: for the three cases of matching two catalogs of $\sigma_1=0.1"$ and $\sigma_2=0.5"$
294: to each other and to themselves. This is the problem of matching the Sloan
295: Digital Sky Survey \citep[SDSS;][]{york,pier} and the Galaxy Evolution Explorer
296: \citep[GALEX;][]{martin,morrissey} science archives.
297:
298: \begin{figure}[bh]
299: \epsscale{1}
300: \plotone{f1.eps}
301: \caption{The weight of evidence as a function of angular separation
302: for the three cases of matching two catalogs of $\sigma_1=0.1"$ and $\sigma_2=0.5"$
303: to each other and to themselves. For example, matching SDSS and GALEX sources:
304: SDSS--SDSS (solid), SDSS--GALEX (dotted) and GALEX--GALEX (dashed).}
305: \label{fig:ex}
306: \end{figure}
307:
308: \begin{deluxetable}{cccr|cccr}
309: \tablecolumns{8}
310: \small
311: \tablewidth{0pt}
312: \tablecaption{Weight of evidence for three surveys
313: as a function of the angular separations in $\sigma=\sigma_1=\sigma_2=\sigma_3=0.1"$ units}
314: \tablehead{
315: \colhead{$\psi_{12}$} &
316: \colhead{$\psi_{23}$} &
317: \colhead{$\psi_{31}$} &
318: \colhead{$W$} &
319: \colhead{$\psi_{12}$} &
320: \colhead{$\psi_{23}$} &
321: \colhead{$\psi_{31}$} &
322: \colhead{$W$}
323: }
324: \startdata
325: 0 & 0 & 0 & 25.38 & 0 & 0 & 0 & 25.38 \\
326: 1 & 1 & 1 & 25.17 & 0 & 1 & 1 & 25.24 \\
327: 2 & 2 & 2 & 24.51 & 0 & 2 & 2 & 24.80 \\
328: 3 & 3 & 3 & 23.43 & 0 & 3 & 3 & 24.08 \\
329: 4 & 4 & 4 & 21.91 & 0 & 4 & 4 & 23.07 \\
330: 5 & 5 & 5 & 19.95 & 0 & 5 & 5 & 21.76 \\
331: 6 & 6 & 6 & 17.57 & 0 & 6 & 6 & 20.17 \\
332: 7 & 7 & 7 & 14.74 & 0 & 7 & 7 & 18.29 \\
333: 8 & 8 & 8 & 11.49 & 0 & 8 & 8 & 16.12 \\
334: 9 & 9 & 9 & 7.79 & 0 & 9 & 9 & 13.66 \\
335: 10 & 10 & 10 & 3.67 & 0 & 10 & 10 & 10.91 \\
336: 11 & 11 & 11 & -0.89 & 0 & 11 & 11 & 7.87 \\
337: 12 & 12 & 12 & -5.89 & 0 & 12 & 12 & 4.54 \\
338: 13 & 13 & 13 & -11.32 & 0 & 13 & 13 & 0.92 \\
339: 14 & 14 & 14 & -17.18 & 0 & 14 & 14 & -2.99 \\
340: \enddata
341: \label{tbl:b3}
342: \end{deluxetable}
343:
344:
345: Matching three catalogs also makes an interesting case study for the various
346: potential configurations of three positions. The Bayes factor for this case,
347: in the same limit as previously, takes the form of
348: %
349: \begin{equation}
350: B = \frac {4 \exp \left\{ -\frac
351: { \sigma^2_3 \psi_{12}^2 + \sigma^2_1 \psi_{23}^2 + \sigma^2_2 \psi_{31}^2 }
352: { 2(\sigma^2_1 \sigma^2_2 + \sigma^2_2 \sigma^2_3 + \sigma^2_3 \sigma^2_1)} \right\}}
353: {\sigma^2_1 \sigma^2_2 + \sigma^2_2 \sigma^2_3 + \sigma^2_3 \sigma^2_1}
354: \end{equation}
355: %
356: In Table~\ref{tbl:b3} the weight of evidence is shown for various configurations
357: from the matching of three similar catalogs with equal astrometric accuracies,
358: $\sigma_1 = \sigma_2 = \sigma_3 = 0.1"$ (separations listed in $\sigma$ units.)
359: The astrometric precision was chosen to match the nominal SDSS limitations.
360:
361: In general, the Bayes factor for the typical large weights and small
362: angular separations takes the form of
363: \begin{equation}
364: \label{eq:gensml}
365: B = 2^{n-1} \frac{\prod w_i}{\sum w_i} \exp \left\{ -
366: \frac{\sum_{i<j} w_i w_j \psi_{ij}^2}{2\sum{w_i}} \right\}
367: \end{equation}
368: where all summations and products run on the members of the tuple from
369: the $n$ number of catalogs; see Appendix~\ref{app:2} for the details of the
370: calculation.
371:
372: In scenarios where individual errors are different or even anisotropic,
373: one can generalize our expression in a fairly straightforward manner
374: in the above approximation. Instead of the scalar weight, one can use
375: the inverse of the covariance matrix, however, the elegant simplicity
376: of the expressions is sacrificed.
377:
378:
379: \section{Folding in the Physics} \label{sec:physics}
380:
381: Naturally the formalism introduced in Section~\ref{sec:bf} is not specific
382: to astrometric observations.
383: In fact, it is rather straightforward to fold other measured quantities
384: into the calculations.
385: %
386: This is especially important when dealing with multiple matches.
387: Picking the ``correct'' combination of sources from various spatially similar
388: configurations is a degenerate problem
389: that requires extra information to resolve. The use of photometric
390: information is a natural choice for its wide availability,
391: however, its application requires further assumptions on the spectral energy
392: distributions (SEDs). Often models exist to help out with the solution,
393: but extra caution is needed to avoid any undesirable effect. For example,
394: when the goal is to discover new types of objects with unknown SEDs, one
395: should not apply known SEDs as priors but rather look for combinations that
396: are likely matches based on spatial detections but excluded by SED modelling.
397:
398: As a demonstration of these ideas, let us apply the introduced Bayesian framework to photometric
399: measurements in various passbands.
400: The ingredients include the following further explicit models:
401: %
402: \begin{itemize}
403: \item[1] Model $S$ for the spectrum energy distributions,
404: e.g., by \citet{bc03}, described by a set of parameters, $\vec{\eta}$:
405: $s(\lambda|\vec{\eta},S)$ along with the corresponding $p(\vec{\eta}|S)$ priors;
406: %
407: \item[2] Model $R$ for the transmission of the passbands to calculate
408: simulated fluxes $\vec{\gamma}(\vec{\eta}|S,R)$
409: by integrating the SEDs $s(\lambda|\vec{\eta},S)$ with the
410: appropriate response functions; and
411: %
412: \item[3] Model $C$ for the uncertainty of the catalog
413: from the photometric calibration, $p(\vec{g}|\vec{\gamma}, C)$, where $\vec{g}$
414: is the observed flux set and $\vec{\gamma}$ is the true.
415: \end{itemize}
416: %
417: These separate models can be folded into a single model $M$, for simplicity,
418: so one can write $p(\vec{g}|\vec{\eta},M)$ for the probability density of measuring
419: $\vec{g}$ fluxes for an object with $\vec{\eta}$ physical properties of $S$
420: seen through the filters in $R$ with the $C$ photometric accuracy.
421: %(Note that this description does not account for the uncertainties in the SED modelling and the transmission curves.)
422: %
423: The Bayes factor for the photometry in the face of the observed fluxes
424: $D'=\{\vec{g}_1,\vec{g}_2,\dots{},\vec{g}_n\}$, similarly to the astrometric
425: formulas, is given by the ratio
426: %
427: \begin{equation}
428: B(H,K|D') = \frac{\displaystyle%
429: \int p(\vec{\eta}|H) \prod_{i=1}^{n} p_i(\vec{g}_i|\vec{\eta},H)\,d^r\!\eta
430: }{\displaystyle%
431: \prod_{i=1}^{n} \left\{\int p(\vec{\eta}_i|K)\,p_i(\vec{g_i}|\vec{\eta}_i,K)\,d^r\!\eta_i\right\}
432: }
433: \end{equation}
434:
435: In the simplest case, $S$ is parameterized by a discrete spectral type $T$,
436: the redshift $z$ and an overall scaling factor for the brightness, $\alpha$:
437: \begin{equation}
438: \vec{\gamma} = \alpha \vec{f}(T,z)
439: \end{equation}
440: where $\vec{f}$ is a vector of the simulated photometry in the various
441: passbands.
442: %
443: Photometric uncertainties are often assumed to be Gaussian with a diagonal
444: covariance matrix of elements $\sigma_l^2$, where $l$ runs on the
445: $L$ number of passbands. After substitution, we arrive at the familiar formula of
446: \begin{equation}
447: p(\vec{g}|\vec{\eta},M) = \frac{1}{\cal{}N}
448: \exp \left\{-\sum_{l=1}^{L} \frac{[g_l-\alpha f_l(T,z)]^2}{2\sigma_l^2} \right\}
449: \end{equation}
450: where constant $\cal{}N$ is the usual normalization factor of the multivariate
451: normal distribution, which in our special case is just
452: ${\cal{}N} = (2\pi)^{L/2} \sigma_1 \sigma_2 \cdots \sigma_L$.
453: %
454: Integrating these models to get the Bayes factor is a very similar problem
455: to template fitting photometric redshift estimation. In fact, the two
456: procedures can be done in a self-consistent way within the same application.
457: %
458: Naturally, spectroscopic redshift measurements can be directly incorporated
459: in this analysis, when available, but other data can also enter in
460: a straightforward manner.
461:
462: The Bayesian analysis is inherently recursive. As soon as we obtain new
463: measurements, and compute the posterior probability, that becomes the
464: prior for subsequent studies. This is an extremely powerful property, and
465: simplifies the computations enormously.
466: %
467: A consequence of this is that the combined Bayes factor of the astrometric and
468: photometric measurements is simply the product of the two,
469: \begin{equation} \label{eq:bb}
470: B = B_{\rm{}pos} \cdot B_{\rm{}phot}
471: \end{equation}
472: as also seen from the Bayes factor's definition.
473: %
474: This means that one can just do the spatial join first, and consider
475: additional measurements and physical priors in subsequent steps, if needed.
476:
477:
478: \section{From Priors to Posteriors} \label{sec:thres}
479:
480: The Bayes factor naturally relates the prior and posterior probabilities. When
481: $K$ is the complementary hypothesis of $H$,
482: %$P(H)+P(K)=1$ and $P(H|D)+P(K|D)=1$, hence
483: the posterior probability is
484: \begin{equation} \label{eq:post}
485: %P(H|D) = \left[ 1 + \frac{1-P(H)}{B(H,K|D)\,P(H)} \right]^{-1}
486: P(H|D) = \left[ 1 + \frac{1-P(H)}{B\,P(H)} \right]^{-1}
487: \end{equation}
488: which, in the limit of vanishing priors, becomes
489: \begin{equation} \label{eq:post2}
490: P(H|D) = \frac{B\,P(H)}{1+B\,P(H)}
491: \end{equation}
492: %
493: To make a definitive decision on whether a set of detections
494: should be considered a match, one would like to set a limit on the posterior
495: probability and derive the Bayes factor threshold from that, however,
496: this can only be done with an initial estimate of the prior.
497:
498:
499: \subsection{The Prior and the Selection Function}
500:
501: The prior probability depends on the angular and radial selection functions
502: of the observations.
503: %
504: If the visible universe contains $N$ objects,
505: and we select two of them at random, the probability of picking the
506: same object is $1/N$. When selecting $n$ objects, the probability
507: is $1/N^{n-1}$. A limited field of view shrinks the observable volume,
508: hence decreases the number of objects, and increases the prior probability.
509: When the angular selection functions of the catalogs overlap only partially
510: then one can just consider the intersection of the sky coverage
511: and the smaller number of sources within.
512:
513: The various radial
514: selection functions also have a significant role, and make the situation
515: more complicated. In order to consider their effect, one has to estimate
516: the overlap of the selections in the input catalogs. Every
517: catalog has observational constraints, other than the field of view,
518: like flux limits,
519: that set the radial selection function. The superset
520: of these contraints defines the restrictions on the {\em{}overlap catalog}.
521: %
522: Let $N_\star$
523: denote the number of objects in that catalog.
524: In this general case, the prior probability takes the form of
525: \begin{equation} \label{eq:uniform}
526: P(H) = N_\star \Big/ \prod_i^n N_i
527: \end{equation}
528: %
529: When the limitations are identical, all catalogs have equal number of
530: objects, $N_\star = N_1 = \dots{} = N_n$, and we get back the same formula
531: of $P(H)=1/N^{n-1}$ as before, but when, for example, one catalog consists of only
532: low-redshift galaxies (e.g., $z<0.2$), and the other has high-redshift quasars
533: (e.g., $z>3$), there is no overlap between the two radial selection functions,
534: hence $N_\star=0$, which means $P(H)=0$. One can get vanishing priors even if
535: the redshift histograms overlap significantly, e.g., two catalogs of
536: red (e.g., $u-g<2$) and blue galaxies ($u-g>2$).
537: %
538: In general, all these complex selection effects
539: are captured in a single scalar quantity, $N_\star$, which is estimated based on
540: prior physical knowledge, e.g., by using template SEDs
541: and the known characteristics of the input catalogs (e.g., the luminosity
542: functions), or
543: alternatively, when no prior information is available, one can invoke
544: self-consistency arguments to derive it; see later.
545: %
546: We now rewrite the prior with the surface densities, $\nu = N/\Omega$,
547: or the scaled number of objects for the entire sky, $\rho=4\pi\nu$, as
548: \begin{equation} \label{eq:omega}
549: P(H) = \frac{\nu_\star}{\prod \nu_i}\,\Omega^{1-n}
550: = \frac{\rho_\star}{\prod \rho_i} \left(\frac{\Omega}{4\pi}\right)^{1-n}
551: \end{equation}
552: %
553: This formula also provides a straightforward way to include a model
554: for varying surface density on the sky, e.g., for stars,
555: where $\nu = \nu(\vec{x})$. In this case, a constant limiting posterior
556: probability yields a varying threshold on the Bayes factor as a function of
557: the position on the celestial sphere.
558:
559:
560: \subsection{The Bayes Factor and the Window Function}
561:
562: The field of view not only changes the prior probabilities but also
563: modifies the Bayes factor.
564: %
565: When the window function is known, one can
566: refine the prior probability density that enters the integral of the
567: numerator and denominator of the Bayes factor. This is done by adopting
568: eq.~\ref{eq:win} as the prior.
569: %
570: In first order, for the typical catalogs with large weights (high accuracy)
571: and large contiguous observation areas, this new prior is uniform over the window
572: function, neglecting the fuzzy boundary, except scaled by the area coverage
573: \begin{equation}
574: p(\vec{m}|M_\Omega) = \frac{\Omega(\vec{m})}{\Omega}\,\delta(|\vec{m}|-1)
575: \end{equation}
576: where again $\Omega(\vec{m})$ is the window function that takes the value 1
577: when $\vec{m}$ is inside and 0 otherwise, and $\Omega$ is its area.
578: %
579: The Bayes factor inside the footprint will be essentially same as before in
580: eq.~\ref{eq:sinh} but also scaled with these fractional coverage:
581: %
582: \begin{equation} \label{eq:wsinh}
583: B = \left(\frac{\Omega}{4\pi}\right)^{n-1}
584: \frac{\sinh{}w}{w}\,\prod_{i=1}^{n} \frac{w_i}{\sinh{}w_i}
585: \end{equation}
586: %
587: The edge effect modifies this only for a tiny fraction of the objects at
588: the boundary of the observations. The proper integral is, of course, much
589: more expensive than this analytical formula, but can be evaluated or
590: re-evaluated, e.g., by an MCMC algorithm.
591:
592: For the typical small priors, the posterior depends only on the product of the
593: Bayes factor and the prior; see eq.~\ref{eq:post2}. This means that the footprint
594: effect cancels out in the posterior
595: probability; cf.\ eqs.~\ref{eq:omega} and \ref{eq:wsinh}.
596: %
597: Hence it is still sensible to just simply use the all-sky formula
598: in eq.~\ref{eq:sinh} and \ref{eq:gensml} as
599: long as the prior is written accordingly, i.e., $\rho_\star/\rho_1\rho_2\cdots\rho_n$.
600: %
601: From the data providers' point of view, who often do not know the field
602: of view, e.g., the legacy catalogs in VizieR \citep{vizier}
603: %\footnote{http://vizier.u-strasbg.fr/}
604: or the NASA/IPAC Extragalactic Database \citep[NED;][]{ned},
605: %\footnote{NASA/Ipac Extragalactic Database; http://nedwww.ipac.caltech.edu/}
606: the best quantity to
607: publish along with the matched tuples is also the analytic all-sky Bayes factor,
608: so researchers can incorporate their own prior knowledge, and set the thresholds
609: on the posterior accordingly that are often specific to the science application.
610:
611:
612: \subsection{Self-Consistent Estimation}
613:
614: In principle, the cross-identification process is now complete,
615: one just has to formulate the prior, possibly varying on the sky,
616: and set a threshold on the posterior probabilities to select the matches.
617: %
618: For the ignorant without a priori knowledge, these are not completely independent
619: choices, and, at least in the limit when all observables are being considered
620: in the Bayes factors, could be derived from requirements of a self-consistent
621: field theory.
622: %
623: When prior knowledge is available and dictates a preference, one could
624: and probably should still check for the consistency outlined here to understand
625: the discrepancies, if any.
626:
627: The formula for the prior in eq.~\ref{eq:uniform}
628: is in fact equivalent to stating that $P(H)$ is constant and
629: \begin{equation}
630: \sum P(H) = N_\star
631: \end{equation}
632: where the summation runs over the direct product of all
633: sets of sources in the $n$ catalogs, i.e., all possible combinations
634: of detections with $N_1 N_2\cdots{}N_n$ contributions.
635: %
636: The self-consistency argument requires that
637: \begin{equation} \label{eq:self}
638: \sum P(H|D) = N_\star
639: \end{equation}
640: which is an equation for $N_\star$ that can be solved by, e.g.,
641: some iterative approximation method starting from
642: an initial value of $N_\star = \min \left\{ N_i \right\}$.
643: Initial experiments support our expectations that these procedures indeed converge
644: very rapidly, only in a few iterations,
645: and are insensitive to the matching limit once the Bayes factor is less than unity.
646: %
647: For varying unknown priors one can use some sky tesselation schemes,
648: such as HEALPix \citep{healpix}, Igloo \citep{igloo} or HTM \citep{szalay05},
649: and estimate a piecewise constant prior (uniform in the cells) using
650: the same methodology. Naturally other more sophisticated models can also be
651: used in the same spirit, e.g., specific functional forms or smoothing to limit
652: the gradient, as well as tapered windows when required.
653:
654: The threshold on the posterior, $P_T$, can also be established in a consistent way.
655: Here the requirement is that
656: \begin{equation}
657: \sum_{P(H|D)>P_T}\!\!\!\!\!\!\!1 = N_\star
658: \end{equation}
659: This is equivalent to applying a Bayes classifier.
660: By changing the right hand side of the above equation, it is possible to make
661: the selection more restrictive or less depending on the scientific goal.
662: In the case, where the prior changes on the sky and eq.~\ref{eq:self} is solved in cells of
663: some pixelization, one can still just use a single $P_T$ limit obtained from
664: the entire catalog by ensuring that the total number of objects are consistent.
665: The counts in individual cells may not be perfectly recovered but, if the prior
666: is right, there should be no significant trends.
667:
668:
669: \section{Practical Considerations} \label{sec:prac}
670:
671: The question remains how to evaluate the Bayes factor efficiently for multiple
672: catalogs without considering all possible combinations of sources.
673: Fast algorithms exist to match two sets of point sources
674: using an angular separation limit \citep{budavari03,malik03,gray04,gray06,szalay05,nieto07}.
675: Ideally one would like to leverage the power of these two-way crossmatch
676: engines in a recursive manner, and get rid of unlikely combinations with small
677: Bayes factors as early as possible.
678:
679: Matching two catalogs is straightforward; any Bayes factor limit corresponds
680: to a single distance cut, and hence our existing tools are adequate.
681: To go from $n$ number of catalogs to $n+1$, we need to make this
682: process iterative, and prune the match list step-by-step.
683: We do this by computing the overall Bayes factor in every step assuming that
684: all other subsequent catalogs will contribute sources at the best possible
685: position.
686: This optimization problem may be expensive to solve in general, but can
687: be analytically calculated in special cases, and for the spherical normal
688: distribution the solution is evident: the center position of the mode is
689: the correct choice.
690:
691: In fact, for the normal distribution one can do even better. In every
692: %the $(k+1)^{\rm{}th}$
693: step, a new catalog is added to the current sub-matches.
694: %$k$-tuple of current matches.
695: Since the product of normal distributions is still of the same functional form,
696: one can compute the Bayes factor as a
697: function of angular separation from that position, derive the limiting radius,
698: and utilize a two-way crossmatch engine for joining the current $k$-tuples with
699: the new $(k+1)^{\rm{}th}$ catalog using that threshold.
700: %
701: For this we rewrite the logarithm of the Bayes factor in eq.~\ref{eq:gensml},
702: in the more convenient form of
703: \begin{eqnarray} \label{eq:bfinc}
704: \ln B = \ln N - \frac{1}{2}\sum_{i=2}^n \frac{a_{i-1}}{a_i} w_i \Delta_i^2
705: \end{eqnarray}
706: with the newly introduced variables
707: \begin{eqnarray}
708: %\ln N & = & (1-n)\ln{}2\pi+\sum_{i=1}^n\ln{}w_i-\ln\sum_{i=1}^n{}w_i \\
709: %N & = & (2\pi)^{1-n} \frac{\prod w_i}{\sum w_i} \\
710: N & = & 2^{n-1} \frac{\prod w_i}{\sum w_i} \\
711: a_k & = & \sum_{i=1}^k w_i \\
712: \vec{\Delta}_i & = & \vec{x}_i - \vec{c}_{i-1}
713: \end{eqnarray}
714: where $\vec{c}_{k}$ is the unit vector of the best position for the current
715: $k$-tuple of sub-match,
716: \begin{equation}
717: \vec{c}_{k} = {\displaystyle\sum_{i=1}^k w_i \vec{x}_i}
718: \Big/ {\left| \displaystyle\sum_{i=1}^k w_i \vec{x}_i \right|}
719: \end{equation}
720: %
721: With these we compute the weight of evidence in a recursive manner.
722: The iteration starts by substituting $\vec{c}_1 = \vec{x}_1$. In the $k^{\rm{}th}$ step,
723: the maximum search radius $\rho_{k+1}$ is computed from eq.~\ref{eq:bfinc} to yield
724: the Bayes factor threshold $B_{0}$ by assuming optimal matches from the subsequent
725: catalogs with vanishing $\Delta_i^2$ contributions,
726: \begin{equation}
727: b_{k+1} \rho_{k+1}^2 = 2\ln\frac{N}{B_{0}} - \sum_{i=2}^k b_i \Delta_i^2\ \ \
728: {\rm{}with}\ \ \
729: b_{k}=\frac{w_k a_{k-1}}{a_k}
730: \end{equation}
731: We assign every source within that radius to each $k$-tuple sub-match, and go
732: to the next catalog.
733: %
734: In general, the search radius will be different for every tuple for
735: their different spatial configurations. When the two-way matching algorithm
736: requires a fixed radius, one can take the maximum value in linear time, use
737: that more generous search radius in the matching,
738: and filter the result set later, just before going to the next catalog.
739: %
740: From catalog to catalog we propagate only the quantities that are
741: necessary to calculate the weight of evidence. The recursion formulas are
742: given by the following expressions:
743: \begin{eqnarray}
744: a_{k} &=& a_{k-1} + w_{k} \\
745: q_{k} &=& q_{k-1} + \frac{a_{k-1}}{a_{k}} w_{k} \Delta_{k}^2 \\
746: \vec{c}_{k} &=& \left({\vec{c}_{k-1} + \frac{w_{k}}{a_{k}} \vec{\Delta}_{k}}\right)
747: \Big/ {\left|\vec{c}_{k-1} + \frac{w_{k}}{a_{k}} \vec{\Delta}_{k} \right|}
748: \end{eqnarray}
749: %
750: This stepwise method for evaluating the weight of evidence not only
751: provides an accurate match list that meets all our requirements enumerated
752: in Section~\ref{sec:intro}, e.g., symmetry in the catalogs,
753: but also exhibits the performance of the
754: current state-of-the-art two-way crossmatching tools.
755:
756:
757: \section{Summary} \label{sec:sum}
758:
759: We presented a general probabilistic formalism for cross-identifying astronomical
760: point sources. The framework is based on Bayesian hypothesis testing to decide
761: whether a series of observations truly belong to a single astronomical object.
762: %
763: The expression we derived is completely general, symmetric in all observations, and
764: accommodates any model of the astrometric precision.
765: %
766: We introduced the spherical normal distribution, and calculated the Bayes factor
767: for the generic $n$-way matching problem both in the general case and in the
768: typical limit of high precision and small angular separations.
769: The cases of 2- and 3-way matching were studied in detail.
770: %
771: We discussed an efficient evaluation strategy of the Bayes factor that leverages
772: the power of existing high-performance two-way matching tools in a recursive manner,
773: yet, it provides accurate measurements of the observational evidence that are independent
774: of the order of the catalogs considered.
775: %
776: While the normal distribution is the simplest to work with for its unique
777: properties, other specific PDFs can be handled in the same spirit.
778: %
779: Our technique provides a natural mechanism to include other observed properties.
780: We demonstrated how multicolor survey data, even at different wavelengths,
781: can be utilized in the matching process by invoking SED models.
782: %
783: Morphological classification or redshift measurements, when available,
784: will also increase the accuracy of the results.
785:
786: The beauty of our approach to the cross-identification problem is
787: that it completely separates the dependence on each parameter,
788: while providing the opportunity to incorporate
789: them in a fairly straightforward way.
790: %
791: Including expert knowledge about the physics of the objects in the analysis is
792: easily achievable by adopting the right priors,
793: and when such information is not available, self-consistency
794: arguments can guide the process to a stable solution in a few iterations.
795: %
796: With the pre-computed Bayes factors in the matched catalogs,
797: astronomers can define custom thresholds to derive specialized crossmatch catalogs
798: based on their own explicit assumptions. For example, using a database of
799: the same set of associations,
800: researchers can optimize for completeness of the galaxy population,
801: or even search for unusually red objects.
802:
803: \appendix
804:
805: \section{The Bayes factor and the Spherical Normal Distribution}
806: \label{app:1}
807:
808: In this appendix we discuss the mathematical calculation of the Bayes factor
809: in the common case, when a spherical normal distribution is assumed for modelling
810: the astrometric accuracy. In addition we also adopt an all-sky prior in this
811: derivation.
812:
813: The Bayes factor is the ratio of the likelihoods, $p(D|H)$ and $p(D|K)$,
814: where again $D$ represents the observed positions, $\left\{\vec{x}_i\right\}$.
815: \begin{eqnarray}
816: B = \frac{p(D|H)}{p(D|K)}
817: \end{eqnarray}
818: We recall that hypothesis $H$ is parameterized by a single position, $\vec{m}$ unit vector,
819: and $K$ is parameterized by a set of $n$ position vectors, $\left\{\vec{m}_i\right\}$.
820: The basic equations to start from are
821: \begin{eqnarray}
822: p(D|H) & = & \int\!d^3\!m\ p(\vec{m}|H)\ p(D|\vec{m},H) \\
823: p(D|K) & = & \int\!d^3\!m_1\!\int\!d^3\!m_2\dots\!\int\!d^3\!m_n
824: \ p(\vec{m}_1|K)p(\vec{m}_2|K)\dots{}p(\vec{m}_n|K)\ p(D|\left\{\vec{m}_i\right\},K)
825: \end{eqnarray}
826: %
827: where
828: \begin{eqnarray}
829: p(\vec{m}|M) &=& \frac{\delta(|\vec{m}|\!-\!1)}{4\pi} \\
830: p(\left\{\vec{x}_i\right\}|\vec{m},H) &=& \prod_i^n N(\vec{x}_i|\vec{m},w_i) =
831: \prod_i^n \frac{w_i\delta(|\vec{x}_i|\!-\!1)}{4\pi\sinh{w_i}}
832: \exp\left({w_i\vec{x}_i\vec{m}}\right) \\
833: p(\left\{\vec{x}_i\right\}|\left\{\vec{m}_i\right\},K) &=& \prod_i^n N(\vec{x}_i|\vec{m}_i,w_i) =
834: \prod_i^n \frac{w_i\delta(|\vec{x}_i|\!-\!1)}{4\pi\sinh{w_i}}
835: \exp\left({w_i\vec{x}_i\vec{m}_i}\right)
836: \end{eqnarray}
837: First we focus on hypothesis $H$
838: \begin{eqnarray}
839: p(D|H) & = & \int d^3\!m \frac{\delta(|\vec{m}|\!-\!1)}{4\pi}
840: \prod_i^n \frac{w_i\delta(|\vec{x}_i|\!-\!1)}{4\pi\sinh{w_i}} \exp\left({w_i\vec{x}_i\vec{m}}\right) \\
841: & = & \left( \prod_i^n \frac{w_i\delta(|\vec{x}_i|-1)}{4\pi\sinh{w_i}} \right)
842: \int d^3\!m \frac{\delta(|\vec{m}|\!-\!1)}{4\pi} \exp\left({\sum_i^n w_i\vec{x}_i\vec{m}}\right)
843: \end{eqnarray}
844: introduce
845: \begin{equation}
846: w\vec{x} = \sum_i^n w_i\vec{x}_i
847: \end{equation}
848: where $\vec{x}$ is a unit vector, and write
849: \begin{eqnarray}
850: p(D|H) & = & \left( \prod_i^n \frac{w_i\delta(|\vec{x}_i|\!-\!1)}{4\pi\sinh{w_i}} \right)
851: \int d^3\!m \frac{\delta(|\vec{m}|\!-\!1)}{4\pi} \exp\left({w\vec{x}\vec{m}}\right) \\
852: & = & \left( \frac{\sinh{w}}{w} \prod_i^n \frac{w_i\delta(|\vec{x}_i|\!-\!1)}{4\pi\sinh{w_i}} \right)
853: \int d^3\!m \frac{w\delta(|\vec{m}|\!-\!1)}{4\pi\sinh{w}} \exp\left({w\vec{x}\vec{m}}\right)
854: \\
855: %\end{eqnarray}
856: %Since the last integral is unity, we arrive at
857: %\begin{eqnarray}
858: %p(D|H)
859: & = & \frac{\sinh{w}}{w} \prod_i^n \frac{w_i}{\sinh{w_i}}\, \frac{\delta(|\vec{x}_i|\!-\!1)}{4\pi}
860: \end{eqnarray}
861: %
862: The likelihood of the alternative hypothesis $K$ is calculated similarly
863: \begin{eqnarray}
864: p(D|K) & = & \prod_i^n \int d^3\!m_i \frac{\delta(|\vec{m}_i|\!-\!1)}{4\pi}
865: \frac{w_i\delta(|\vec{x}_i|\!-\!1)}{4\pi\sinh{w_i}} \exp\left({w_i\vec{x}_i\vec{m_i}}\right) \\
866: & = & \prod_i^n \frac{\delta(|\vec{x}_i|\!-\!1)}{4\pi}
867: \end{eqnarray}
868: %
869: Hence the Bayes factor is
870: \begin{equation}
871: B = \frac{\sinh{w}}{w} \prod_i^n \frac{w_i}{\sinh{w_i}}
872: \end{equation}
873: as also shown in eq.~\ref{eq:sinh}.
874:
875:
876: \section{High Astrometric Accuracy and Small Separations}
877: \label{app:2}
878:
879: The astrometric precision of the actual observations is almost always extremely
880: high in the absolute sense,
881: so it is worth examining the approximation of the Bayes factor in this limit.
882: %
883: We also assume small angular separations. In the chain of equations below we only
884: use the ``$\approx$'' sign to signal new approximations.
885: %
886: We start from the previous result
887: \begin{eqnarray}
888: B & = & \frac{\sinh{w}}{w} \prod_i^n \frac{w_i}{\sinh{w_i}} \\
889: & \approx & 2^{n-1} \frac{e^w}{w} \prod_i^n \frac{w_i}{e^{w_i}} \\
890: & = & 2^{n-1} \frac{\prod w_i}{w}\,e^{w-\sum w_i} \\
891: & = & 2^{n-1} \frac{\prod w_i}{w}\,e^{\sum w_i \left(\frac{w}{\sum w_i}-1\right)}
892: \label{eq:bfsmlapp}
893: \end{eqnarray}
894: where we exploit the fact that all $w$ weights are large, hence the $\sinh{w}$ is
895: approximately $\frac{1}{2}\exp w$.
896: %
897: We proceed by calculating
898: \begin{eqnarray}
899: \left(\frac{w}{\sum_i w_i}\right)^2
900: & = & \frac{w^2}{\left(\sum_i w_i\right)^2} \\
901: & = & \frac{\sum_i w_i^2 + 2\sum_{i<j} w_i w_j \vec{x}_i\vec{x}_j}{\sum_i w_i^2 + 2\sum_{i<j} w_i w_j} \\
902: & = & \frac{\sum_i w_i^2 + 2\sum_{i<j} w_i w_j \cos\psi_{ij}}{\sum_i w_i^2 + 2\sum_{i<j} w_i w_j} \\
903: & \approx & \frac{\sum_i w_i^2 + 2\sum_{i<j} w_i w_j \left(1-\psi_{ij}^2/2\right)}{\sum_i w_i^2 + 2\sum_{i<j} w_i w_j} \\
904: & = & \frac{\sum_i w_i^2 + 2\sum_{i<j} w_i w_j -\sum_{i<j} w_i w_j \psi_{ij}^2}{\sum_i w_i^2 + 2\sum_{i<j} w_i w_j} \\
905: & = & 1-\frac{\sum_{i<j} w_i w_j \psi_{ij}^2}{\left(\sum_i w_i\right)^2}
906: \end{eqnarray}
907: After taking the square root of the above equation, we get
908: \begin{eqnarray}
909: %\left(
910: \frac{w}{\sum_i w_i}
911: %\right)
912: & \approx & 1 - \frac{\sum_{i<j} w_i w_j \psi_{ij}^2}{2\left(\sum_i w_i\right)^2}
913: \\
914: %\end{eqnarray}
915: & \rm{and} & \nonumber \\
916: %\begin{eqnarray}
917: \sum_i w_i \left(\frac{w}{\sum_i w_i}-1\right)
918: & = & -\frac{\sum_{i<j} w_i w_j \psi_{ij}^2}{2\sum_i w_i}
919: \end{eqnarray}
920: %
921: From the above equations we also see that
922: \begin{equation}
923: \frac{1}{w} \approx \frac{1}{\sum_i w_i} \left(1+\frac{\sum_{i<j} w_i w_j \psi_{ij}^2}{2\left(\sum_i w_i\right)^2}\right)
924: \approx \frac{1}{\sum_i w_i}
925: \end{equation}
926: in this context to only keep the leading term.
927: By substituting the above two equations to eq.~\ref{eq:bfsmlapp}, we arrive at our generic small angle result shown in
928: eq.~\ref{eq:gensml}.
929: %
930: The 2- and 3-way matching cases are straightforward specializations of the generic equation, where one substitutes
931: $w_i=1/\sigma_i^2$ to work out the simplified formulae.
932:
933:
934: \iffalse
935: \section{Small angle approximation formulas}
936: These equations are provided for reference and not meant to be part of
937: the official publications.
938: \begin{eqnarray}
939: q_n & = & \sum_{i<j}^n w_i w_j \psi_{ij}^2 \Big/ \sum_i^n w_i \\
940: & = & \frac{1}{a_n} \sum_{i<j}^n w_i w_j \psi_{ij}^2 \\
941: & = & \frac{1}{a_n} \sum_{i<j}^n w_i w_j \vec{r}_{ij}^2 \\
942: & = & \frac{1}{a_n} \left( \sum_{i<j}^{n-1} w_i w_j \vec{r}_{ij}^2 + \sum_i^{n-1} w_i w_n \vec{r}_{in}^2 \right) \\
943: & = & \frac{a_{n-1}}{a_n} q_{n-1} + \frac{1}{a_n} \sum_{i}^{n-1} w_i w_n (\vec{x}_i-\vec{x}_n)^2 \\
944: & = & \frac{a_{n-1}}{a_n} q_{n-1} + \frac{1}{a_n} \sum_{i}^{n-1} w_i w_n (2-2\vec{x}_i\vec{x}_n) \\
945: & = & \frac{a_{n-1}}{a_n} q_{n-1} + 2\frac{a_{n-1}w_n}{a_n} - 2 \frac{u_{n-1}w_n}{a_n} \vec{x}_n\vec{c}_{n-1} \\
946: & = & \frac{a_{n-1}}{a_n} q_{n-1} + 2\frac{a_{n-1}w_n}{a_n} - 2 \frac{a_{n-1}w_n}{a_n} \vec{x}_n\vec{c}_{n-1}
947: + 2 \frac{(a_{n-1}-u_{n-1})w_n}{a_n} \vec{x}_n\vec{c}_{n-1}\\
948: & = & \frac{a_{n-1}}{a_n} q_{n-1} + \frac{a_{n-1}w_n}{a_n} (\vec{x}_n - \vec{c}_{n-1})^2
949: + 2 \frac{(a_{n-1}-u_{n-1})w_n}{a_n} \vec{x}_n\vec{c}_{n-1}\\
950: & = & \frac{a_{n-1}}{a_n} q_{n-1} + \frac{a_{n-1}w_n}{a_n} (\vec{x}_n - \vec{c}_{n-1})^2
951: + \frac{q_{n-1}w_n}{a_n} \vec{x}_n\vec{c}_{n-1}\\
952: & = & \frac{a_{n-1}}{a_n} q_{n-1} + \frac{a_{n-1}w_n}{a_n} (\vec{x}_n - \vec{c}_{n-1})^2
953: + \frac{q_{n-1}w_n}{a_n} \\
954: & = & \frac{a_{n-1}+w_n}{a_n} q_{n-1} + \frac{a_{n-1}w_n}{a_n} (\vec{x}_n - \vec{c}_{n-1})^2 \\
955: & = & q_{n-1} + \frac{a_{n-1}}{a_n} w_n \Delta_n^2
956: \end{eqnarray}
957: \fi
958:
959:
960: \acknowledgements %
961: %
962: The authors are grateful for invaluable discussions
963: with Mar\'{\i}a Nieto-Santisteban, Istv\'an Csabai and Zolt\'an R\'acz
964: on various aspects of the topic,
965: and gladly acknowledge the generous support from the following
966: organizations:
967: %
968: Gordon and Betty Moore Foundation GBMF 554,
969: W. M. Keck Foundation KECK D322197,
970: NSF NVO AST-0122449,
971: NASA AISRP NNG05GB01G,
972: NASA GALEX 44G1071483,
973: Hungarian National Scientific Foundation OTKA-T047244,
974: European Research Training Network MRTN-CT-2004-503929.
975: %
976: Part of this research was done while A.~S. was a recipient of
977: an Alexander von Humboldt Fellowship at the MPA.
978:
979: \begin{thebibliography}{}
980:
981: \bibitem[Breitenberger(1963)]{breitenberger63} Breitenberger, E.\ 1963, Biometrika, 50, 81
982:
983: \bibitem[Bruzual \& Charlot(2003)]{bc03} Bruzual, G., \& Charlot, S.\ 2003, \mnras, 344, 1000
984:
985: \bibitem[Budav{\'a}ri et al.(2003)]{budavari03} Budav{\'a}ri, T.,
986: Malik, T., Szalay, A.~S., Thakar, A.~R., \& Gray, J.\
987: 2003, in ASP Conf.\ Ser., Vol.\ 295, Astronomical Data Analysis Software and Systems (ADASS) XII,
988: eds.\ H.~E. Payne, R.~I. Jedrzejewski, and R.~N. Hook
989: (San Francisco: ASP), p.31
990:
991: \bibitem[Budav{\'a}ri et al.(2004)]{budavari04} Budav\'ari, T., et~al.\ 2004,
992: in ASP Conf.\ Ser., Vol.\ 314, Astronomical Data Analysis Software and Systems (ADASS) XIII,
993: eds.\ F. Ochsenbein, M. Allen, and D. Egret (San Francisco: ASP), p.177
994:
995: \bibitem[Crittenden(2000)]{igloo} Crittenden, R.~G.\ 2000,
996: Astrophysical Letters Communications, 37, 377
997:
998: \bibitem[Fisher(1953)]{fisher53} Fisher, R., 1953, Proceedings of the Royal Society of London,
999: Series A, Mathematical and Physical Sciences, Vol.~217, No.~1130., pp.295--305
1000:
1001: \bibitem[G{\'o}rski et~al.(2005)]{healpix} G{\'o}rski, K.~M.,
1002: Hivon, E., Banday, A.~J., Wandelt, B.~D., Hansen, F.~K., Reinecke, M., \&
1003: Bartelmann, M.\ 2005, \apj, 622, 759
1004:
1005: % There Goes the Neighborhood: Relational Algebra for Spatial Data Search
1006: \bibitem[Gray et~al.(2004)]{gray04} Gray, J., %et~al.\
1007: Szalay, A.~S. Fekete, G., Nieto-Santisteban, M.~A., O’Mullane, W.,
1008: Thakar, A.~R., Heber, G., \& Rots, A.~H.\
1009: 2004, Microsoft Research Technical Report, MSR-TR-2004-32
1010:
1011: % zone algorithm
1012: \bibitem[Gray et~al.(2006)]{gray06} Gray, J., %et~al.\
1013: Nieto-Santisteban, M.~A., \& Szalay, A.~S.\ 2006,
1014: Microsoft Research Technical Report, MSR-TR-2006-52
1015:
1016: \bibitem[L\'evy(1939)]{levy39} L\'evy, P.\ 1939, Bull.\ Soc.\ Math.\ Fr., 67, 1%--41
1017:
1018: \bibitem[Madore et~al.(1992)]{ned} Madore, B.~F., Helou,
1019: G., Corwin, H.~G., Jr., Schmitz, M., Wu, X., \& Bennett, J.\ 1992,
1020: Astronomical Data Analysis Software and Systems I, 25, 47
1021:
1022: \bibitem[Malik et~al.(2003)]{malik03} Malik, T., %et~al.\
1023: Szalay, A.~S., Budav{\'a}ri, T., \& Thakar, A.~R.\
1024: 2003, Proceedings of Conference on Innovative Data Systems Research (CIDR), 17
1025:
1026: \bibitem[Mann et~al.(1997)]{mann97} Mann, R.~G., et al.\ 1997, \mnras, 289, 482
1027:
1028: \bibitem[Martin et~al.(2005)]{martin} Martin, D.~C., et al.\ 2005, \apjl, 619, L1
1029:
1030: \bibitem[Morrissey et~al.(2007)]{morrissey} Morrissey, P., et~al.\ 2007,
1031: \apjs{} in press, ArXiv e-prints, 706 (arXiv:0706.0755)
1032:
1033: \bibitem[Nieto-Santisteban(2007)]{nieto07}
1034: Nieto-Santisteban, M.~A.\ 2007, PhD thesis, in preparation
1035:
1036: \bibitem[Ochsenbein et~al.(2000)]{vizier} Ochsenbein, F.,
1037: Bauer, P., \& Marcout, J.\ 2000, \aaps, 143, 23
1038:
1039: \bibitem[Pier et~al.(2003)]{pier} Pier, J.~R., et~al.\ 2003, \aj, 125, 1559
1040:
1041: \bibitem[Rohde et~al.(2006)]{rohde06} Rohde, D.~J., Gallagher, M.~R.,
1042: Drinkwater, M.~J., \& Pimbblet, K.~A.\ 2006, \mnras, 369, 2
1043:
1044: \bibitem[Sutherland \& Saunders(1992)]{ss92} Sutherland, W.,
1045: \& Saunders, W.\ 1992, \mnras, 259, 413
1046:
1047: \bibitem[Szalay et~al.(2005)]{szalay05} Szalay, A.~S.,
1048: Gray, J., Fekete, G., Kunszt, P., Kukol, P., \& Thakar, A.~R.\
1049: 2005, Microsoft Research Technical Report, MSR-TR-2005-123
1050:
1051: \bibitem[York et~al.(2000)]{york} York, D.G., et~al.\ 2000, \aj, 120, 1579
1052:
1053:
1054: \end{thebibliography}
1055:
1056:
1057: \end{document}
1058:
1059:
1060:
1061: