1: \begin{definition}

2: \label{eq:CE-switching}

3: Let $\mixedstrat$ denote a probability distribution on the space of action profiles $\actset$. The set of correlated equilibria of the regime-switching game \index{game!regime-switching game} is the convex polytope

4: %

5: 

6: \vspace{-0.1cm}

7: {\small

8: \begin{equation}

9: \label{eq:CE_defn_switching}

10: \CEe(\markovgame_\dtimee) = \lbr \mixedstrat:

11: \sum_{\actprof^{-\plyrind}} \mixedstratind^{\plyrind}\big(i, \actprof^{-\plyrind}\big)\left[U^{\plyrind}\big(j, \actprof^{-\plyrind},\markovgame_\dtimee\big) - U^\plyrind\big(i, \actprof^{-\plyrind},\markovgame_\dtimee\big)\right] \leq 0, \forall i,j,n \rbr\nonumber

12: %\in\actset^\plyrind, \plyrind\in\plyrset \rbr\nonumber

13: \end{equation}

14: }

15: %

16: \begin{figure}

17: \centerline{\includegraphics[width=0.6\textwidth]{game_switching.eps}}

18: \caption{Tracking convex polytope of correlated equilibria $\CE(\markovgame_\dtimee)$ randomly evolving with time according to a Markov process $\lbr\markovgame_\dtimee\rbr$. Theorem~\ref{theorem:CE-convg-switch} asserts that such tracking is attainable by each agent individually following the regret-matching algorithms presented in \S\ref{sec:local-behavior-game}.}

19: \label{fig:switching-game}

20: \end{figure}

21: 

22: \vspace{-0.1cm}

23: \noindent

24: where $\mixedstratind^\plyrind\big(i, \actprof^{-\plyrind}\big)$ denotes the probability of agent $\plyrind$ playing action $i$ and the rest playing $\actprof^{-\plyrind}$.

25: \end{definition}

26: