1: 

2: \section{Analysis Procedure}

3: 

4: Signal yields are obtained using an extended unbinned

5: maximum likelihood (2-D ML) fit to the $M_{\rm{bc}}$ and $\Delta E$

6: distributions in the $M_{\rm{bc}}-\Delta E$

7: box region after the ${\mathcal R}$

8: cut is applied.

9: 

10: For $N$ input candidates, the likelihood is defined as

11: %

12: \begin{eqnarray}

13: L(N_S,N_C) = \frac{e^{-(N_S+N_C+N_{b\bar b}+N_{ra})}}{N!} \prod_{i=1}^{N}

14: [N_{S} P_{S_i} + N_{C} P_{C_i} + N_{b\bar b} P_{{b\bar b}_i}

15: +N_{ra} P_{{ra}_i}],

16: \end{eqnarray}

17: %

18: where $P_{S_i}$, $P_{C_i}$, $P_{b{\bar b}_i}$ 

19: and $P_{{ra}_i}$ are the two-dimensional probability densities for

20: event $i$ to be the signal, continuum, charm containing ($b \to c$) $B$ decay backgrounds and

21: charmless $B$ decay backgrounds in the variables $M_{\rm{bc}}$

22: and $\Delta E$, respectively. Poisson fluctuations

23: for $N_S$ and $N_C$

24: are considered 

25: in this type of likelihood. 

26: For backgrounds other than continuum, $N_{b\bar b}$ and $N_{ra}$ 

27: are obtained from

28: MC samples. Uncertainties in the MC PDFs are included in the 

29: systematics study.

30: The continuum, $b \to c$ and charmless $B$ decay background PDFs are  

31: all obtained from the respective MC samples.

32: 

33: The $M_{\rm{bc}}$ and $\Delta E$ shapes from continuum MC events are modeled by an ARGUS function~\cite{argus} with a fixed end point at 5.29 GeV/c$^2$ and by a 2nd order Chebyshev polynomial, respectively. The shapes of signal, $B\overline{B}$, and other rare charmless $B$ decays are modeled by 2D smooth functions.

34: The $\Delta E$ distribution is found to be asymmetric, with a tail on the lower side due to $\gamma$ interactions with material in the front of the calorimeter and shower leakage out of the back side of the crystals. As a result, the $\Delta E$ resolution and the tail distribution strongly depend on the $\eta$ energy. In the inclusive $B^0 \to \eta K^+ \pi^-$ and $B^0 \to \eta \pi^+ \pi^-$ studies, the $\eta $ energy distribution for the signal events is not known apriori so we divide the data into three samples: $P_{\eta}<$ 1 GeV/$c$, 1 GeV/$c$ $<$ $P_{\eta}$ $<$ 2 GeV/$c$ and $P_{\eta} >$ 2 GeV/$c$. 

35: 

36: 

37: For decays with more than one sub-decay process,

38: the final average results are obtained

39: by fitting the sub-decay modes simultaneously  

40: with the expected 

41: efficiencies included in the fit and with the branching fraction

42: as the common output.

43: This is equivalent to minimizing the sum of 

44: $\chi^2 = -2\ln(L)$

45: as a function of the branching fraction over all considered 

46: sub-decay channels. The statistical significance ($\Sigma$) of the signal

47: is

48: defined as $\sqrt{-2\ln(L_0/L_{\rm max})}$,

49: where $L_0$ and $L_{\rm max}$ denote

50: the likelihood values for zero signal events,

51: and the fit number of signal events, respectively.

52: The 90\% C.L. upper limit is calculated by finding $x_{\rm max}$ such that

53: \begin{eqnarray}

54: \frac{\int_0^{x_{\rm max}} L(x) \,dx}{\int_0^\infty L(x) \,dx}

55:  = 90\% \,.

56: \end{eqnarray}

57: %

58: %

59: 

60: