1: \begin{abstract}

2: The main objective of this paper is to explore the precise relationship 

3: between the Bethe free energy (or entropy) and the Shannon conditional 

4: entropy of graphical error

5: correcting codes. 

6: The main result shows that the Bethe free energy

7: associated with a low-density parity-check code used over a binary

8: symmetric channel in a large noise regime is, with high probability,

9: asymptotically exact as the block length grows. To arrive at this result we 

10: develop new techniques for rather general graphical models based on the loop sum as a starting point and 

11:  the polymer

12: expansion from statistical mechanics.

13: The true free energy is computed as a series expansion containing

14: the Bethe free energy as its zero-th order term plus

15: a series of corrections. It is easily seen that convergence criteria

16: for such expansions are satisfied for general high-temperature models.

17: % In particular, when the graphical model has large girth the Bethe

18: % free energy is asymptotically exact. 

19: We apply these general results

20: to ensembles of low-density generator-matrix and parity-check codes.

21: While the application to generator-matrix codes follows standard ``high temperature''

22: methods, the case of parity-check codes requires non-trivial new ideas

23: because the hard constraints correspond to a zero-temperature regime.

24: Nevertheless one can combine the polymer expansion with expander and

25: counting arguments to show that the difference between the true and

26: Bethe free energies vanishes with high probability in the large block

27: length limit. \end{abstract}

28: