1: \begin{abstract}

2: We tackle the problem of penalty selection of regularization on the basis of the minimum description length~(MDL) principle.

3: In particular, we consider that the design space of the penalty function is high-dimensional.

4: In this situation, the luckiness-normalized-maximum-likelihood~(LNML)-minimization approach is favorable, because LNML quantifies the goodness of regularized models with any forms of penalty functions in view of the minimum description length principle, and guides us to a good penalty function through the high-dimensional space.

5: However, the minimization of LNML entails two major challenges:

6: 1) the computation of the normalizing factor of LNML and

7: 2) its minimization in high-dimensional spaces.

8: In this paper, we present a novel regularization selection method~(MDL-RS),

9: in which a tight upper bound of LNML~(uLNML) is minimized with local convergence guarantee.

10: Our main contribution is the derivation of uLNML, which is a uniform-gap upper bound of LNML in an analytic expression.

11: This solves the above challenges in an approximate manner because it allows us to accurately approximate LNML and then efficiently minimize it.

12: The experimental results show that MDL-RS improves the generalization performance of regularized estimates specifically when the model has redundant parameters.

13: \keywords{minimum description length principle \and luckiness normalized maximum likelihood \and regularized empirical risk minimization \and penalty selection \and concave-convex procedure}

14: % \PACS{PACS code1 \and PACS code2 \and more}

15: % \subclass{MSC code1 \and MSC code2 \and more}

16: \end{abstract}

17: