1: \subsection{Function discriminant analysis (FDA)}\index{Function Discriminant Analysis}\index{FDA}
2: \label{sec:fda}
3:
4: The common goal of all TMVA discriminators is to determine an optimal separating
5: function in the multivariate space of all input variables. The Fisher
6: discriminant solves this analytically for the linear case, while artificial neural
7: networks, support vector machines or boosted decision trees provide nonlinear
8: approximations with -- in principle -- arbitrary precision if enough training
9: statistics is available and the chosen architecture is flexible enough.
10:
11: The function discriminant analysis (FDA) provides an intermediate solution to the
12: problem with the aim to solve relatively simple or partially nonlinear problems.
13: The user provides the desired function with adjustable parameters via the configuration
14: option string, and FDA fits the parameters to it, requiring the function value to be as
15: close as possible to the real value (to 1 for signal and 0 for background in classification).
16: Its advantage over the more involved and automatic nonlinear discriminators is the simplicity
17: and transparency of the discrimination expression. A shortcoming is that FDA will underperform
18: for involved problems with complicated, phase space dependent nonlinear correlations.
19:
20: \subsubsection{Booking options}
21:
22: FDA is booked via the command:
23: \begin{codeexample}
24: \begin{tmvacode}
25: factory->BookMethod( Types::kFDA, "FDA", "<options>" );
26: \end{tmvacode}
27: \caption[.]{\codeexampleCaptionSize Booking of the FDA classifier: the first argument is
28: a predefined enumerator, the second argument is a user-defined
29: string identifier, and the third argument is the configuration options string.
30: Individual options are separated by a ':'.
31: See Sec.~\ref{sec:usingtmva:booking} for more information on the booking.}
32: \end{codeexample}
33:
34: The configuration options for the FDA classifier are listed in Option Table~\ref{opt:mva::fda}
35: (see also Sec.~\ref{sec:fitting}).
36:
37: % ======= input option table ==========================================
38: \begin{option}[t]
39: \input optiontables/MVA__FDA.tex
40: \caption[.]{\optionCaptionSize
41: Configuration options reference for MVA method: {\em FDA}.
42: Values given are defaults. If predefined categories exist, the default category
43: is marked by a '$\star$'. The options in Option Table~\ref{opt:mva::methodbase} on
44: page~\pageref{opt:mva::methodbase} can also be configured.
45: The input variables In the discriminator expression are denoted
46: \code{x0}, \code{x1}, \dots (until $\Nvar-1$), where the
47: number follows the order in which the variables have been
48: registered with the Factory; coefficients to be determined
49: by the fit must be denoted \code{(0)}, \code{(1)}, \dots (the
50: number of coefficients is free) in the formula; allowed is
51: any functional expression that can be interpreted by a ROOT
52: \href{http://root.cern.ch/root/html/TFormula.html}{TFormula}.
53: See Code Example~\ref{ce:fdaexample} for an example expression.
54: The limits for the fit parameters (coefficients) defined in the
55: formula expression are given with the syntax: "\code{(-1,3);(2,10);...}",
56: where the first interval corresponds to parameter \code{(0)}.
57: The converger allows to use (presently only) Minuit
58: fitting in addition to Monte Carlo sampling or a Genetic Algorithm. More
59: details on this combination are given in Sec.~\ref{sec:converger}. The
60: various fitters are configured using the options given in Tables~\ref{opt:fitter_mc},
61: \ref{opt:fitter_minuit}, \ref{opt:fitter_ga} and \ref{opt:fitter_sa}, for MC, Minuit, GA and SA,
62: respectively.
63: }
64: \label{opt:mva::fda}
65: \end{option}
66: % =====================================================================
67:
68: A typical option string could look as follows:
69: \begin{codeexample}
70: \begin{tmvacode}
71: "Formula=(0)+(1)*x0+(2)*x1+(3)*x2+(4)*x3:\
72: ParRanges=(-1,1);(-10,10);(-10,10);(-10,10);(-10,10):\
73: FitMethod=MINUIT:\
74: ErrorLevel=1:PrintLevel=-1:FitStrategy=2:UseImprove:UseMinos:SetBatch"
75: \end{tmvacode}
76: \caption[.]{\codeexampleCaptionSize FDA booking option example simulating a linear Fisher
77: discriminant (\cf\ Sec.~\ref{sec:fisher}). The
78: top line gives the discriminator expression, where the $xi$ denote the
79: input variables and the $(j)$ denote the coefficients to be determined by the
80: fit. Allowed are all standard functions and expressions, including the functions
81: belonging to the ROOT \href{http://root.cern.ch/root/html/TMath.html}{TMath} library.
82: The second line determines the limits for the fit parameters, where the
83: numbers of intervals given must correspond to the number of fit parameters defined.
84: The third line defines the fitter to be used (here Minuit), and the last line
85: is the fitter configuration. }
86: \label{ce:fdaexample}
87: \end{codeexample}
88:
89: \subsubsection{Description and implementation}
90:
91: The parsing of the discriminator function employs ROOT's
92: \href{http://root.cern.ch/root/html/TFormula.html}{TFormula} class, which requires that the
93: expression complies with its rules (which are the same as those that apply for the
94: \code{TTree::Draw} command). For simple formula with a single global fit solution, Minuit will
95: be the most efficient fitter. However, if the problem is complicated, highly nonlinear, and/or
96: has a non-unique solution space, more involved fitting algorithms may be required. In that
97: case the Genetic Algorithm combined or not with a Minuit converger should lead to the
98: best results. After fit convergence, FDA prints the fit results (parameters and estimator
99: value) as well as the discriminator expression used on standard output. The smaller the
100: estimator value, the better the solution found. The normalised estimator is given by
101: \beq
102: \begin{array}{lrcl}
103: \mbox{\em For classification:} &
104: {\cal E} &=& \frac{1}{\WS}\sum_{i=1}^{\NS} \left(F({\bf x}_i)-1\right)^2w_i +
105: \frac{1}{\WB}\sum_{i=1}^{\NB} F^2({\bf x}_i) w_i\;, \\[0.3cm]
106: \mbox{\em For regression:} &
107: {\cal E} &=& \frac{1}{W}\sum_{i=1}^{N} \left(F({\bf x}_i)-{\bf t}_i\right)^2w_i\;,
108:
109: \end{array}
110: \eeq
111: where for classification the first (second) sum is over the signal (background)
112: training events, and for regression it is over all training events,
113: $F({\bf x}_i)$ is the discriminator function, ${\bf x}_i$ is the tuple of the
114: $\Nvar$ input variables for event $i$, $w_i$ is the event weight, ${\bf t}_i$ the
115: tuple of training regression targets, $\WSB$ is the sum of all signal (background)
116: weights in the training sample, and $W$ the sum over all training weights.
117:
118: \subsubsection{Variable ranking}
119:
120: The present implementation of FDA does not provide a ranking
121: of the input variables.
122:
123: \subsubsection{Performance}
124:
125: The FDA performance depends on the complexity and fidelity of the user-defined
126: discriminator function. As a general rule, it should be able to reproduce the
127: discrimination power of any linear discriminant analysis. To reach into the nonlinear
128: domain, it is useful to inspect the correlation profiles of the input variables, and
129: add quadratic and higher polynomial terms between variables as necessary. Comparison
130: with more involved nonlinear classifiers can be used as a guide.
131:
132: %%% Local Variables:
133: %%% mode: latex
134: %%% TeX-master: "TMVAUsersGuide"
135: %%% End:
136: