1: \begin{abstract}

2:     We propose a novel algorithmic framework for distributional reinforcement learning, based on learning finite-dimensional mean embeddings of return distributions. 

3:     %

4:     The framework reveals a wide variety of new algorithms for dynamic programming and temporal-difference algorithms that rely on the \emph{sketch Bellman operator}, which updates mean embeddings with simple linear-algebraic computations.

5:     %

6:     We provide asymptotic convergence theory, and examine the empirical performance of the algorithms on a suite of tabular tasks.

7:     Further, we show that this approach can be straightforwardly combined with deep reinforcement learning.

8: \end{abstract}

9: