1: \begin{abstract} \small\baselineskip=9pt 

2: % Network embedding has been a

3: 

4: In this paper, we study the fundamental problem of random walk for network embedding.

5: We propose to use non-Markovian random walk, variants of vertex-reinforced random walk (VRRW), to fully use the history of a random walk path. 

6: To solve the getting stuck problem of VRRW, we introduce an exploitation-exploration mechanism to help the random walk jump out of the stuck set.

7: The new random walk algorithms share the same convergence property of VRRW and thus can be used to learn stable network embeddings.

8: Experimental results on two link prediction benchmark datasets and three node classification benchmark datasets show that our proposed approach \textit{reinforce2vec} can outperform state-of-the-art random walk based embedding methods by a large margin.

9: % Random walks have been a widely used method for graph-based tasks, such as PageRank computation, or network embedding. Existing random walk on graphs can be generalized as a Markovian process, which is memoryless, i.e., the next state is only dependent on the current state. In this paper, we propose a novel type of random walk on graphs, which is a non-Markovian process, i.e., the next state is dependent on the whole history states. Motivated by the established vertex-reinforced random walk (VRRW), we design a distribution-reinforced random walk (reinforce2vec-d), which can not only converge efficiently to a stationary distribution but also alleviate the inherent ``stuck'' problem of VRRW. By applying reinforce2vec-d to the graph embedding tasks, reinforce2vec-d outperforms existing Markov based random walk methods. Moreover, reinforce2vec-d enjoys the advantages of automatically and efficiently stopping given a graph comparing to the fine-tuned longer walk length of Markov based random walk methods, leading to the huge practical potential for graph-based tasks.

10: \end{abstract}

11: