1: \begin{abstract}

2: 	Main objects of the paper are stationary and weak KAM Hamilton-Jacobi equations on the finite-dimensional torus. 

3: 	The key idea of the paper is to replace the underlying calculus of variations problems with continuous time Markov decision problems. This directly leads to an approximation of the stationary Hamilton-Jacobi equation by the Bellman equation for a discounting Markov decision problem.

4: 	 Developing elements of the weak KAM theory for the Markov decision problem, we obtain an approximation of the effective Hamiltonian. Additionally,  convergences of the functional parts of the discrete weak KAM equations and  Mather measures are shown. It turns out that the approximating equations are systems of algebraic equations. Thus, the paper's result can be seen as  numerical schemes for stationary and weak KAM Hamilton-Jacobi equations. 

5: 	\msccode{49L25, 37J51, 60J28, 93E20, 70H09}

6: 	\keywords{Hamilton-Jacobi equation, discrete weak KAM theory, Markov chain approximation, discrete  Mather measure}	

7: \end{abstract}

8: