1: \begin{abstract}

2: %We study continuity properties of discrete-time stochastic control problems with respect to transition kernels and robustness of optimal control policies applied to systems with incomplete or incorrect probabilistic models. We study both fully observed and partially observed setups under an infinite horizon discounted expected cost criterion. We show that continuity and robustness cannot be established under weak and setwise convergences of transition kernels in general, but that the expected induced cost is robust under total variation in that it is continuous in the mismatch of transition kernels under the convergence in total variation. By imposing further assumptions on the measurement models and on the kernel itself, we show that the optimal cost can be made continuous under weak convergence of transition kernels. Using these continuity results we find convergence results and error bounds due to the mismatch that occurs by the application of a control policy which is designed for an incorrectly estimated system model to a true model, thus establishing positive and negative results on robustness. We show in particular that in general continuity does not imply robustness for fully observed systems. Compared to the existing literature, we obtain refined robustness results that are applicable even under the incorrect models that can be investigated under weak convergence and setwise convergence criteria, in addition to the total variation criteria. These lead to practically important results on empirical learning in (data-driven) stochastic control since often, in many applications, system models are learned through training data.

3: %\end{abstract}

4: