1: \begin{abstract}

2:     The combination of Formal Methods with Reinforcement Learning (RL) has recently

3:     attracted interest as a way for single-agent RL to learn multiple-task specifications.

4:     In this paper we extend this convergence to multi-agent settings and formally define

5:     Extended Markov Games as a general mathematical model that allows multiple RL agents

6:     to concurrently learn various non-Markovian specifications. To introduce this new

7:     model we provide formal definitions and proofs as well as empirical tests of RL

8:     algorithms running on this framework. Specifically, we use our model to train two

9:     different logic-based multi-agent RL algorithms to solve diverse settings of

10:     non-Markovian {\em co-safe} $LTL$ specifications.

11: 

12: \end{abstract}

13: