1: \begin{abstract}

2: We study behavior of the restricted maximum likelihood (REML)

3: estimator under a misspecified linear mixed model (LMM) that has

4: received much attention in recent gnome-wide association studies.

5: The asymptotic analysis establishes consistency of the REML

6: estimator of the variance of the errors in the LMM, and convergence

7: in probability of the REML estimator of the variance of the random

8: effects in the LMM to a certain limit, which is equal to the true

9: variance of the random effects multiplied by the limiting proportion of

10: the nonzero random effects present in the LMM. The aymptotic results

11: also establish convergence rate (in probability) of the REML estimators

12: as well as a result regarding convergence of the asymptotic conditional

13: variance of the REML estimator. The asymptotic results are fully

14: supported by the results of empirical studies, which include extensive

15: simulation studies that compare the performance of the REML estimator

16: (under the misspecified LMM) with other existing methods.

17: \end{abstract}

18: