39e7c65ce2cb4ec0.tex
1: \begin{abstract}
2: The Expectation-Maximization (EM) algorithm is an iterative method
3: to maximize the log-likelihood function for parameter estimation.
4: Previous works on the convergence analysis of the EM algorithm have
5: established results on the asymptotic (population level) convergence
6: rate of the algorithm. In this paper, we give a data-adaptive analysis
7: of the sample level local convergence rate of the EM algorithm. In
8: particular, we show that \emph{the local convergence rate of the EM
9: algorithm is a random variable} $\overline{K}_{n}$ derived from the
10: data generating distribution, which adaptively yields the convergence
11: rate of the EM algorithm on each finite sample data set from the same
12: population distribution. We then give a non-asymptotic concentration
13: bound of $\overline{K}_{n}$ on the population level optimal convergence
14: rate $\overline{\kappa}$ of the EM algorithm, which implies that
15: $\overline{K}_{n}\to\overline{\kappa}$ in probability as the sample
16: size $n\to\infty$. Our theory identifies the effect of sample size
17: on the convergence behavior of sample EM sequence, and explains a
18: surprising phenomenon in applications of the EM algorithm, i.e. the
19: finite sample version of the algorithm sometimes converges faster
20: even than the population version. We apply our theory to the EM algorithm
21: on three canonical models and obtain specific forms of the adaptive
22: convergence theorem for each model.
23: \end{abstract}
24: