1: \begin{abstract}

2: 

3: Recent papers have demonstrated the possibility of energy-based text generation by adapting gradient-based sampling algorithms, a paradigm of MCMC algorithms that promises fast convergence.

4: However, as we show in this paper, previous attempts on this approach to text generation all fail to sample correctly from the target language model distributions.

5: To address this limitation, we consider the problem of designing text samplers that are faithful, meaning that they have the target text distribution as its limiting distribution.

6: We propose several faithful gradient-based sampling algorithms to sample from the target energy-based text distribution correctly, and study their theoretical properties.

7: Through experiments on various forms of text generation, we demonstrate that faithful samplers are able to generate more fluent text while adhering to the control objectives better.

8: 

9: 

10: \end{abstract}

11: