1: \documentclass[twoside]{article}
2: \usepackage{fleqn,espcrc2}
3: \usepackage{graphicx}
4: \usepackage{amssymb}
5: %\bibliographystyle{physrev3}
6: %\usepackage[figuresright]{rotating}
7:
8: % put your own definitions here:
9: \newcommand{\ttbs}{\char'134}
10: \newcommand{\AmS}{{\protect\the\textfont2
11: A\kern-.1667em\lower.5ex\hbox{M}\kern-.125emS}}
12:
13: % add words to TeX's hyphenation exception list
14: \hyphenation{author another created financial paper re-commend-ed Post-Script}
15:
16: % declarations for front matter
17: \title{Statistical Analysis of Solar Neutrino Data\thanks{
18: Talk presented by C. Giunti at
19: NOW 2000, Conca Specchiulla (Otranto, Italy), 9-16 Sep. 2000;
20: DFTT 46/00, hep-ph/0012247.}
21: }
22:
23: \author{M.V. Garzelli and C. Giunti\\[0.2cm]
24: INFN, Sez. di Torino, and Dip. di Fisica Teorica,
25: Univ. di Torino, I--10125 Torino, Italy}
26:
27: \begin{document}
28:
29: \begin{abstract}
30: We calculate with Monte Carlo the goodness of fit
31: and the confidence level of the standard allowed regions
32: for the neutrino oscillation parameters
33: obtained from the fit of the total rates
34: measured in solar neutrino experiments.
35: We show that they are significantly overestimated
36: in the standard method.
37: We also calculate exact allowed regions
38: with correct frequentist coverage.
39: We show that
40: the exact VO, LMA and LOW regions are much larger than the standard ones
41: and merge together
42: giving an allowed band at large mixing angles
43: for all $\Delta{m}^2 \gtrsim 10^{-10} \, \mathrm{eV}^2$.
44: \end{abstract}
45:
46: \maketitle
47:
48: \section{Introduction}
49: \label{Introduction}
50:
51: The data of solar neutrino experiments
52: provide strong indications in favor of neutrino oscillations
53: and their statistical analysis
54: give indications on the values of the neutrino mixing parameters.
55: For two neutrino generations,
56: these parameters are
57: the mass-squared difference
58: $\Delta{m}^2 \equiv m_2^2 - m_1^2$,
59: where $m_1$ and $m_2$ are the two neutrino masses,
60: and
61: $\tan^2\theta$,
62: where
63: $\theta$
64: is the mixing angle
65: (see, for example, Ref.\cite{BGG-review-98-brief}).
66:
67: In this paper we present statistical methods
68: based on Monte Carlo numerical calculations
69: that allow
70: to improve the standard statistical analysis
71: of solar neutrino data,
72: which is approximate
73: \cite{Garzelli-Giunti-sf-00}.
74: We consider the data relative to the total rates
75: measured in the Homestake
76: and Super-Kamiokande
77: experiments,
78: and the weighted average of the total rates measured
79: in the two Gallium experiments
80: GALLEX and SAGE,
81: as given in Table I of Ref.\cite{Concha-sun-99-brief}.
82:
83: As in the standard method,
84: we use as estimator of
85: $\Delta{m}^2$
86: and
87: $\tan^2\theta$
88: the global minimum
89: $X^2_{\mathrm{min}}$
90: of the least-squares function
91: \begin{eqnarray}
92: X^2
93: &=&
94: \sum_{j_1,j_2=1}^{N_{\mathrm{exp}}}
95: \left( R^{\mathrm{(thr)}}_{j_1} - R^{\mathrm{(exp)}}_{j_1} \right)
96: (V^{-1})_{j_1j_2}
97: \nonumber
98: \\
99: &&
100: \hspace{1cm}
101: \times
102: \left( R^{\mathrm{(thr)}}_{j_2} - R^{\mathrm{(exp)}}_{j_2} \right)
103: \,,
104: \label{X2}
105: \end{eqnarray}
106: where
107: $N_{\mathrm{exp}}=3$
108: is the number of experimental data points,
109: $V$ is the covariance matrix of
110: experimental and theoretical uncertainties,
111: $R^{\mathrm{(exp)}}_{j}$
112: is the event rate measured in the $j^{\mathrm{th}}$ experiment
113: and
114: $R^{\mathrm{(thr)}}_{j}$
115: is the corresponding theoretical event rate,
116: that depends on
117: $\Delta{m}^2$ and $\tan^2\theta$.
118: We calculate the covariance matrix $V$
119: following the standard method
120: (see, for example, Ref.\cite{Concha-sun-99-brief}),
121: %described in
122: %Refs.\cite{Fogli-Lisi-correlations-95,%
123: %Fogli-Lisi-Montanino-Palazzo-sun-3nu-00-brief},
124: with the correction proposed in
125: Ref.\cite{Garzelli-Giunti-cs-00}.
126:
127: The standard procedure to calculate
128: the allowed regions in the neutrino oscillation parameter space
129: (see, for example, Ref.\cite{Concha-sun-99-brief})
130: is based on the assumption that
131: $X^2$
132: has a $\chi^2$ distribution with $N_{\mathrm{exp}}$ degrees of freedom.
133: This would be correct if
134: the theoretical rates
135: $R^{\mathrm{(thr)}}_{j}$
136: depended \emph{linearly} on the parameters
137: to be determined in the fit
138: and the errors of
139: $R^{\mathrm{(thr)}}_{j}-R^{\mathrm{(exp)}}_{j}$
140: were \emph{multinormally} distributed
141: with \emph{constant} covariance matrix $V$.
142: In this case there should be only one minimum of $X^2$
143: and the allowed regions at
144: $100\beta\%$ confidence level (CL),
145: given by
146: $X^2 \leq X^2_{\mathrm{min}} + \Delta{X^2}(\beta)$,
147: should be elliptical.
148:
149: It is well-known
150: that in reality
151: there are several local minima of $X^2$,
152: each one determining an allowed region,
153: and
154: these allowed regions do not have elliptic form
155: (see, for example, Ref.\cite{Concha-sun-99-brief}).
156: This is due to the fact that the requirements above
157: are not satisfied.
158: In particular,
159: the stronger effect is due to
160: the non-linear dependence of the theoretical rates
161: $R^{\mathrm{(thr)}}_{j}$
162: from the parameters,
163: which generates
164: several local minima of $X^2$.
165:
166: In the following two sections we present estimations
167: of the goodness of fit
168: (Section~\ref{GOF})
169: and
170: the confidence level of the standard allowed regions
171: (Section~\ref{CL})
172: obtained with a Monte Carlo
173: calculation of the distribution of $X^2$.
174: In Section~\ref{Exact}
175: we present the results of a Monte Carlo calculation
176: of allowed regions with exact coverage.
177:
178: \begin{figure*}[tb]
179: \begin{center}
180: \begin{tabular*}{\textwidth}{@{\extracolsep{\fill}}lr}
181: \rotatebox{90}{\includegraphics[bb=120 80 555 741,width=6cm]{cfi-all-msw-1.eps}}
182: &
183: \rotatebox{90}{\includegraphics[bb=120 309 555 741,width=6cm]{cfi-all-vo-1.eps}}
184: \end{tabular*}
185: \end{center}
186: \caption{ \label{cfi}
187: Allowed 90\%, 95\%, 99\%, 99.73\% CL regions.
188: The gray areas are the allowed regions with exact frequentist coverage.
189: The areas enclosed by the solid lines
190: are the standard allowed regions.
191: }
192: \end{figure*}
193:
194: \section{Goodness of Fit}
195: \label{GOF}
196:
197: Goodness of fit (GOF)
198: is the probability to find a value of the global minimum
199: $X^2_{\mathrm{min}}$
200: of $X^2$
201: larger than the one obtained from the fit.
202: Since there are more than one
203: local minima of $X^2$ with relatively close values of $X^2$,
204: there are more possibilities to obtain
205: good fits of the data
206: with respect to the case of one minimum,
207: and the true goodness of fit
208: is smaller than the one obtained with the standard method.
209:
210: We calculate the distribution of
211: $X^2_{\mathrm{min}}$
212: assuming that the best-fit values
213: $\widehat{\Delta{m}^2}$, $\widehat{\tan^2\theta}$
214: of
215: $\Delta{m}^2$, $\tan^2\theta$
216: are reasonable surrogates of the true unknown values
217: $\Delta{m}^2_{\mathrm{true}}$, $\tan^2\theta_{\mathrm{true}}$
218: and
219: the probability distribution of the differences
220: $\widehat{\Delta{m}^2}_{(k)} - \widehat{\Delta{m}^2}$,
221: $\widehat{\tan^2\theta}_{(k)} - \widehat{\tan^2\theta}$
222: is not too different from the true distribution of the differences
223: $\widehat{\Delta{m}^2}_{(k)} - \Delta{m}^2_{\mathrm{true}}$,
224: $\widehat{\tan^2\theta}_{(k)} - \tan^2\theta_{\mathrm{true}}$
225: in a large set of best-fit parameters
226: $\widehat{\Delta{m}^2}_{(k)}$, $\widehat{\tan^2\theta}_{(k)}$
227: ($k=1,2,\ldots$)
228: obtained with hypothetical experiments.
229:
230: Using $\widehat{\Delta{m}^2}$, $\widehat{\tan^2\theta}$
231: as surrogates of the true values,
232: we generate $N_s$ synthetic random data sets
233: with the usual gaussian distribution for the
234: experimental and theoretical uncertainties.
235: We apply the least-squares method
236: to each synthetic data set,
237: leading to an ensemble of simulated best-fit parameters
238: $\widehat{\Delta{m}^2}_{(s)}$, $\widehat{\tan^2\theta}_{(s)}$
239: with $s=1,\ldots,N_s$,
240: each one with his associated
241: $(X^2_{\mathrm{min}})_{s}$.
242: Then we calculate the goodness of the fit
243: as the fraction of simulated
244: $(X^2_{\mathrm{min}})_{s}$
245: in the ensemble that are larger than the one actually observed,
246: $X^2_{\mathrm{min}}$.
247:
248: The global minimum of the least-squares function (\ref{X2}),
249: $X^2_{\mathrm{min}} = 0.42$,
250: occurs in the SMA region\footnote{
251: We use the standard terminology for the allowed regions
252: (see, for example, Ref.\cite{Concha-sun-99-brief}):
253: SMA for
254: $\Delta{m}^2 \sim 5 \times 10^{-6} \, \mathrm{eV}^2$,
255: $\tan^2 \theta \sim 10^{-3}$,
256: LMA for
257: $\Delta{m}^2 \sim 3 \times 10^{-5} \, \mathrm{eV}^2$,
258: $\tan^2 \theta \sim 0.3$,
259: LOW for
260: $\Delta{m}^2 \sim 10^{-7} \, \mathrm{eV}^2$,
261: $\tan^2 \theta \sim 0.5$,
262: VO for
263: $\Delta{m}^2 \lesssim 10^{-8} \, \mathrm{eV}^2$.
264: }
265: for
266: $\Delta{m}^2 = 5.1 \times 10^{-6} \, \mathrm{eV}^2$
267: and
268: $\tan^2 \theta = 1.6 \times 10^{-3}$.
269: The Monte Carlo method yields a GOF of 40\%,
270: that must be compared with the 52\% GOF obtained with the standard method.
271: Therefore,
272: the standard method significantly overestimates the GOF.
273:
274: \section{Confidence Level of Allowed Regions}
275: \label{CL}
276:
277: The allowed regions with
278: $100\beta\%$ CL
279: are defined by the property that they belong to a
280: set of allowed regions,
281: obtained with hypothetical experiments,
282: which cover (\textit{i.e.} include) the true value of the parameters
283: with probability $\beta$.
284: This property is called \emph{coverage}.
285:
286: When there are several local minima of $X^2$
287: with relatively close values of $X^2$,
288: in repeated experiments
289: the global minimum has significant chances
290: to occur far from the true (unknown)
291: value of the parameters,
292: leading to a smaller probability that the allowed regions
293: cover the true value
294: with respect to the case in which there is only one minimum,
295: which is assumed for the validity of
296: the standard method for the calculation of allowed regions.
297: Hence,
298: the true confidence level of a standard
299: $100\beta\%$ CL allowed region is smaller than $\beta$.
300:
301: We estimate the true confidence level
302: of the standard $100\beta\%$ CL allowed regions
303: using the best fit values of the parameters,
304: $\widehat{\Delta{m}^2}$, $\widehat{\tan^2\theta}$,
305: as surrogates of the true values,
306: $\Delta{m}^2_{\mathrm{true}}$, $\tan^2\theta_{\mathrm{true}}$,
307: for the generation of a large number of synthetic data sets.
308: We apply the standard procedure to each synthetic data set
309: and obtain the corresponding standard $100\beta\%$ CL allowed regions
310: in the space of the neutrino oscillation parameters.
311: Then we count the number of synthetic standard
312: $100\beta\%$ CL allowed regions
313: that cover the assumed surrogate
314: $\widehat{\Delta{m}^2}$, $\widehat{\tan^2\theta}$
315: of the true values.
316: The ratio of this number and the total number of synthetically
317: generated data set gives a Monte Carlo estimation
318: $\beta_{\mathrm{MC}}$
319: of the true confidence level
320: of the standard $100\beta\%$ CL allowed regions.
321:
322: We obtained that the confidence level of the standard
323: 90\% CL allowed regions is 86\%,
324: which is significantly smaller.
325: Other results are presented in Ref.\cite{Garzelli-Giunti-sf-00}.
326:
327: \section{Exact Allowed Regions}
328: \label{Exact}
329:
330: The calculation
331: of the confidence level of the standard allowed regions
332: presented in the previous section
333: is approximate,
334: because it is based on the assumption of a surrogate
335: for the unknown true values of the neutrino oscillation parameters.
336: It would be useful to be able to calculate exact allowed
337: with the desired confidence level,
338: \textit{i.e.} with correct coverage.
339: The procedure that allows to
340: perform this task has been invented by Neyman in 1937
341: (see references in Ref.\cite{Garzelli-Giunti-sf-00}).
342: Let us emphasize that this procedure
343: gives allowed regions with proper coverage
344: for any unknown true values of the parameters.
345:
346: Our implementation of Neyman's procedure
347: for the calculation of the allowed regions
348: in the
349: $\tan^2\theta$--$\Delta{m}^2$
350: plane
351: is described in details in Ref.\cite{Garzelli-Giunti-sf-00}.
352: The results are presented in Fig.\ref{cfi},
353: where the gray areas are the exact allowed regions,
354: and the standard allowed regions are enclosed by solid lines.
355: One can see that the exact
356: LMA, LOW and VO regions
357: are much larger than the standard ones
358: and
359: there is no separation between them.
360: Hence,
361: large mixing angles with
362: $0.2 \lesssim \tan^2\theta \lesssim 1$
363: are allowed for $\Delta m^2 \gtrsim 10^{-10} \, \mathrm{eV}^2$.
364: On the other hand,
365: the exact SMA region approximately coincides with the standard one.
366: This is due to the fact that in the SMA region
367: the assumption of a linear dependence of
368: the theoretical rates
369: $R^{\mathrm{(thr)}}_{j}$
370: in Eq.(\ref{X2})
371: from the parameters
372: $\tan^2\theta$, $\Delta m^2$
373: is approximately correct,
374: whereas it is violated quite badly
375: in the LMA, LOW and VO regions.
376:
377: \section{Conclusions}
378: \label{Conclusions}
379:
380: In conclusion,
381: we have shown that the standard method
382: used in the analysis of solar neutrino data
383: in terms of neutrino oscillations
384: significantly overestimates the goodness of fit
385: and the confidence level of the allowed regions.
386: We have also calculated allowed regions
387: with correct coverage.
388: The SMA region approximately coincides with the standard one,
389: but
390: the VO, LMA and LOW regions are much larger than the standard ones
391: and merge together
392: giving an allowed band at large mixing angles
393: for all $\Delta{m}^2 \gtrsim 10^{-10} \, \mathrm{eV}^2$.
394:
395: %\bibliography{sun}
396:
397: \begin{thebibliography}{1}
398:
399: \bibitem{BGG-review-98-brief}
400: S.~M. Bilenky, C.~Giunti, and W.~Grimus,
401: \newblock Prog. Part. Nucl. Phys. {\bf 43}, 1 (1999).
402: %%CITATION = PPNPD,43,1;%%
403:
404: \bibitem{Garzelli-Giunti-sf-00}
405: M. V. Garzelli and C. Giunti, hep-ph/0007155.
406:
407: \bibitem{Concha-sun-99-brief}
408: M.~C. Gonzalez-Garcia {\em et~al.},
409: \newblock Nucl. Phys. {\bf B573}, 3 (2000), hep-ph/9906469.
410: %%CITATION = NUPHA,B573,3;%%
411:
412: \bibitem{Garzelli-Giunti-cs-00}
413: M.~V. Garzelli and C.~Giunti,
414: \newblock Phys. Lett. {\bf B488}, 339 (2000), hep-ph/0006026.
415: %%CITATION = PHLTA,B488,339;%%
416:
417: \end{thebibliography}
418:
419: \end{document}
420: