1: \documentclass[aps,pre,amsmath,amsfonts,amssymb,11pt,nofootinbib,showpacs]{revtex4}

2: \usepackage{color,epsfig,graphics,psfrag,rotating,theorem}

3: %\draft

4: 

5: \newtheorem{lemma}{Lemma}

6: \newtheorem{remark}{Remark}

7: \newtheorem{propo}{Proposition}

8: \newtheorem{conj}{Conjecture}

9: 

10: \newcommand{\eps}{\varepsilon}

11: \newcommand{\s}{\sigma}

12: \newcommand{\<}{\langle}

13: \renewcommand{\>}{\rangle}

14: \newcommand{\sign}{\text{sign}}

15: 

16: \def\Tree{{\mathbb T}}

17: \def\T{{\sf T}}

18: 

19: \def\E{\mathbb E}

20: \def\prob{{\mathbb P}}

21: \def\var{\text{Var}}

22: \def\ed{\stackrel{\rm d}{=}}

23: \def\ind{{\mathbb I}}

24: \def\P{{\sf P}}

25: \def\M{{\mathfrak M}_q}

26: \def\Q{\widehat{Q}}

27: \def\QQ{\widetilde{Q}}

28: \def\W{\widehat{W}}

29: \def\F{{\sf F}}

30: \def\cA{{\cal A}}

31: \def\pih{\widehat{\pi}}

32: \def\phih{\widehat{\varphi}}

33: \def\eo{\overline{\eta}}

34: \def\bX{{\bf X}}

35: 

36: \def\ux{\underline{x}}

37: \def\uX{\underline{X}}

38: \def\uY{\underline{Y}}

39: \def\uy{\underline{y}}

40: \def\de{{\rm d}}

41: \def\cG{{\cal G}}

42: \def\cN{{\cal N}}

43: 

44: \def\eh{\hat{\eta}}

45: 

46: \def\prooft{\hspace{0.5cm}{\bf Proof:}\hspace{0.1cm}}

47: \def\endproof{\hfill$\Box$\vspace{0.4cm}}

48: 

49: 

50: \begin{document}

51: 

52: \title{Reconstruction on trees and spin glass transition}

53: 

54: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

55: \author{Marc M\'ezard}

56: 

57: \affiliation{Laboratoire de Physique Th\'eorique et Mod\`eles Statistiques,

58: Universit\'e de Paris-Sud, b\^atiment 100, 91405, Orsay Cedex, France}

59: 

60: \author{Andrea Montanari}

61: 

62: \affiliation{Laboratoire de Physique Th\'eorique de l'Ecole Normale

63: Sup\'erieure,\\

64:  24 rue Lhomond 75231 Paris Cedex 05, France}

65: 

66: 

67: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

68: 

69: \begin{abstract}

70: Consider an information source generating a symbol at the 

71: root of a tree network whose links correspond to noisy

72: communication channels, and broadcasting it through the network.

73: We study the problem of reconstructing the transmitted

74: symbol from the information received at the leaves.

75: In the large system limit, reconstruction is possible when the 

76: channel noise is smaller than a threshold. 

77: 

78: We show that this threshold coincides with the dynamical 

79: (replica symmetry breaking) glass transition for an associated 

80: statistical physics problem. Motivated by this correspondence,

81: we derive a variational principle which implies new rigorous bounds 

82: on the reconstruction threshold. Finally, we apply a standard

83: numerical procedure used in statistical physics, 

84: to predict the reconstruction thresholds in various channels. 

85: In particular, we prove a bound on the

86: reconstruction problem for  the antiferromagnetic ``Potts'' channels, which

87: implies, in the noiseless limit, new results on random  proper colorings of 

88: infinite regular trees.

89: 

90: This  relation to the reconstruction problem also offers  interesting perspective for

91: putting  on a clean mathematical basis the theory of  glasses on random graphs.

92: \end{abstract}

93: 

94: \pacs{02.50.-r

95: (Probability theory, stochastic processes, and statistics),

96: 64.70.Pf (Glass transitions), 89.75.Hc

97: (Networks and genealogical trees)}

98: 

99: \maketitle

100: 

101: \section{Introduction}

102: \label{sec:intro}

103: 

104: Consider the following broadcast problem~\cite{Mossel03}.

105: An information source at the root of a tree network

106: produces a letter taken from a $q$-ary alphabet

107: $x\in \{1,\dots,q\}$ (we shall sometimes refer to a letter 

108: from this alphabet as to a `color'). 

109: The symbol is propagated along the edges of the tree. 

110: For simplicity we start with a regular $k$-ary tree $\Tree_k$, 

111: cf. Fig.~\ref{fig:broadcast},

112: in which every vertex has exactly $k$ descendants 

113: (every vertex has degree $k+1$ except the root which

114: has degree $k$), a more general setting is described in~\cite{EvaKenPerSch} and in Sect.~\ref{se:gene}. 

115: Each edge of the tree is an instance of the

116: same noisy communication channel: If the letter $x$ is transmitted 

117: through the channel, $y\in\{1,\dots,q\}$  is received with

118: probability $\pi(y|x)$ (with $\pi(y|x)\ge 0$ and $\sum_y\pi(y|x)=1$). 

119: The problem of reconstruction is the following:  consider all the

120: symbols received at the vertices of the $\ell^{\rm th}$ generation. Does this

121: configuration contain a non-vanishing information on the letter transmitted 

122: by the root, in the large $\ell$ limit?

123: 

124: Beyond its fundamental interest in

125: probability, this problem is relevant to genetics (propagation of genes

126: from an ancestor) \cite{MosselSteel05}, 

127: to statistical physics (models on  Bethe lattices),

128: and information theory (the problem being equivalent to computing

129: the information capacity of the tree network)

130: \cite{CoverThomas}.

131: 

132: An important general bound was obtained by Kesten and Stigum (KS)

133: \cite{KestenStigum1,KestenStigum2}. 

134: Consider the matrix $\pi$ with entries $\pi(y|x)$,

135: $x,y\in\{1,\dots,q\}$ and let $\lambda_2(\pi)$ be its

136: eigenvalue with the second largest absolute value. Then, if $k 

137: |\lambda_2(\pi) |^2 >1$, the reconstruction problem is solvable: the leaves

138: asymptotically contain some information on the letter sent by the root. 

139: In fact in this case the census of the variables in the $\ell^{\rm th}$ 

140: generation (the number of leaves which have received each letter) 

141: contains some information on the root. 

142: Conversely, if $k | \lambda_2(\pi)|^2 <1$,

143: the census contains asymptotically no information on the root

144:  \cite{MosselPeres01}. 

145: Therefore, the

146: KS condition $k \vert \lambda_2(\pi)\vert ^2=1$ defines a

147: threshold for the maximum amount of noise allowing census

148: reconstruction. For larger noise ( $k \vert \lambda_2(\pi)\vert ^2 <1$)

149: one may wonder whether reconstruction is possible exploiting the 

150: whole set of symbols received at the

151: $\ell^{\rm th}$ generation, through a clever use of the correlations 

152: between the symbols received on the leaves. The answer depends

153: on the channel.

154: 

155: %

156: \begin{figure}

157: \includegraphics[width=0.4\linewidth,angle=-0]{broadcast.eps}\hspace{1.5 cm}

158: \includegraphics[width=0.38\linewidth,angle=-0]{reconstr.eps}

159: \caption{Left: Broadcast on a tree. The signal is sent from the root. Each

160: edge is a noisy communication channel broadcasting upwards. Right: The reconstruction problem asks to find what 

161: signal was sent from the root, given the signals received on the leaves }

162: \label{fig:broadcast}

163: \end{figure}

164: %

165: 

166: In most of this paper we shall focus onto

167: transition kernels $\pi(\,\cdot\,|\,\cdot\,)$ satisfying

168: the detailed balance condition (reversible) 

169: with respect to the uniform distribution $\eo(x) = 1/q$. 

170: In other words $\pi(y|x) = \pi(x|y)$. With a slight

171: abuse of notation we shall write  $\pi(y|x) = \pi(y,x)$.

172: For the problem to be non-trivial, we also assume 

173: $\pi(\,\cdot\,|\,\cdot\,)$ to be irreducible and aperiodic.

174: A particularly important example in this family is provided by

175: $q$-ary symmetric channels (or, borrowing from the statistical mechanics

176: terminology, `Potts' channels)

177: %

178: \begin{equation}

179: %

180: \pi(y|x) = \left\{\begin{array}{ll} 1-\eps & \;\; \mbox{if $y=x$.} \\

181: 		\eps/(q-1)  & \;\; \mbox{otherwise}\end{array}

182: \right. \ .\label{eq:PottsChannelDef}

183: %

184: \end{equation}

185: %

186: If $\eps<1-1/q$, $y=x$ is the most likely channel output when the input

187: is $x$: we shall refer to this case as the `ferromagnetic' Potts channel.

188: If $\eps>1-1/q$, the opposite happens and we shall speak of

189: `antiferromagnetic' Potts channel. The particular case 

190: $\eps=1$ is of special interest, since the broadcast 

191: process provides a uniformly random proper coloring of the 

192: $\ell$-generations $k$-ary tree $\Tree_k(\ell)$.

193: 

194: It is intuitively clear that the channel (\ref{eq:PottsChannelDef}) 

195: `gets worse' as $\eps$ increases from $0$ to $1-1/q$ (ferromagnetic channel) and 

196: `improves' as $\eps$ goes from $1-1/q$ to $1$ (antiferromagnetic

197: channel).

198: A result by Mossel~\cite{Mossel01} implies that there exist 

199: a ferromagnetic and an antiferromagnetic threshold,

200: respectively $\eps^+_{\rm r}(k,q)\in[0,1-1/q]$ and 

201: $\eps^-_{\rm r}(k,q)\in[1-1/q,1]$,

202: such that the reconstruction problem is solvable when

203: $\eps \in [0,\eps^+_{\rm r}[\; \cup\;  ]\eps^-_{\rm r},1]$ and insolvable

204: if $\eps \in \; ]\eps^+_{\rm r},\eps^-_{\rm r}[$. 

205: Hereafter we shall drop the $\pm$ superscripts whenever they are clear from the context.

206: 

207: The KS condition $k|\lambda_2(\pi)|^2>1$ is satisfied 

208: (and the problem is census-solvable) for  the channel 

209: (\ref{eq:PottsChannelDef})  if and only if 

210: $\eps\in [0,\eps_{\rm KS}^+(k,q)[\;  \cup\;  ]\eps_{\rm KS}^-(k,q),1]$, 

211: where: 

212: %

213: \begin{eqnarray}

214: %

215: \eps_{\rm KS}^{\pm}(k,q) &=& \frac{q-1}{q} \; \left(1\mp

216: \frac{1}{\sqrt{k}}\right)\, .

217: %

218: \end{eqnarray}

219: %

220: Notice that the above formula yields $\eps_{\rm KS}^{-}(k,q)>1$ for some pairs

221: of $(k,q)$. In fact, for the antiferromagnetic channel, the 

222: census-reconstruction problem (as well as the general reconstruction problem)

223: is not necessarily solvable for $\eps=1$.

224: 

225: It is known \cite{BleherRuizZagrebnov} that, for $q=2$ (the ``binary

226: symmetric'' channel, also known as the ``symmetric Ising'' case), the

227: reconstruction threshold is equal to the KS one: 

228: $\eps_{\rm r}(k,2)=\eps_{\rm KS}(k,2)$ (for $q=2$ the ferromagnetic

229: and antiferromagnetic cases are equivalent via the mapping

230: $\eps\mapsto 1-\eps$). In general, the KS bound implies 

231: $\eps^{+}_{\rm KS}(k,q)\ge \eps_{\rm r}^{+}(k,q)$, and

232: $\eps^{-}_{\rm r}(k,q)\ge \eps_{\rm KS}^{-}(k,q)$. 

233: Furthermore, in \cite{Mossel01} it was shown that, for all $k$, when

234: $q$ is large enough, $\eps^{+}_{\rm KS}(k,q)> \eps_{\rm r}^{+}(k,q)$

235: strictly: reconstruction is possible at noise levels where

236: census reconstruction does not work. 

237: However, several fundamental 

238: questions remain open even for simple Potts channels:

239: Is there any pair $(k,q)$, with $q>2$, such that 

240: $\eps_{\rm r}(k,q)=\eps_{\rm KS}(k,q)$? How to distinguish systematically 

241: between $\eps_{\rm r}(k,q)$ and $\eps_{\rm KS}(k,q)$? How to determine

242: $\eps_{\rm r}(k,q)$ accurately when it does not coincide with 

243: $\eps_{\rm KS}(k,q)$? We shall address these issues in the following. 

244: 

245: The reconstruction problem is intimately related to statistical physics. 

246: Consider a model of Potts spins $y_i \in \{1,\dots,q\}$, 

247: on a finite rooted tree with $\ell$ generations, to be denoted by 

248: $\Tree_k(\ell)$.  

249: Suppose that the energy of a configuration $\uy^{\ell}\equiv \{y_i :

250: i\in \Tree_k(\ell)\}$ is given by:

251: %

252: \begin{equation}

253: %

254: E(\uy^{\ell})= - J \!\!\sum_{(i,j)\in \Tree_k(\ell)}\delta_{y_i, y_j} \ ,

255: \label{eq:PottsEnergy}

256: %

257: \end{equation}

258: %

259: where $(i,j)$ denotes pairs of spins connected by an edge of the

260: tree. Let $\uY^{\ell}$ be the random configuration produced by the broadcast

261: process with channel (\ref{eq:PottsChannelDef}) {\it  up to} generation 

262: $\ell$, 

263: when the transmitted symbol is uniformly random in $\{1,\dots,q\}$.

264: Then

265: %

266: \begin{eqnarray}

267: %

268: \prob\left\{\uY^{\ell}=\uy^{\ell}\right\}

269: =\frac{1}{Z}\, \exp\{-\beta E(\uy^{\ell})\}

270: %

271: \end{eqnarray}

272: %

273: provided we make the identification

274: %

275: \begin{equation}

276: %

277: e^{-\beta J}=\frac{\eps}{(q-1) (1-\eps)}\ .

278: %

279: \end{equation}

280: %

281: In other words, the broadcast process allows to construct one 

282: particular Gibbs measure (state) associated to the energy 

283: function (\ref{eq:PottsEnergy}): in the

284: statistical physics terminology this is the free-boundary measure.

285: In general this is not the unique Gibbs measure for 

286: this energy function. For instance, if $\eps< \frac{q-1}{q} \;

287: \left(1-\frac{1}{k}\right)$, one can construct $q$ `ferromagnetic' states

288: as well. Even if more than one Gibbs state exists, the free-boundary 

289: state can be extremal (or `pure'). It turns out that

290:  the reconstruction problem is 

291: solvable if and only if the Gibbs state with free boundary conditions

292: is not extremal.

293: 

294: Given the strong connection between extremality of Gibbs states and 

295: spatial decay of correlations \cite{Georgii}, the last remark is not surprising.

296: What is more surprising (and constitutes the main theme of this paper)

297: is the relation of the reconstructibility with the existence of 

298: a dynamical glass phase. In recent years, an ongoing effort has been 

299: devoted to the study of glassy models on sparse random graphs.

300: These are graphs which contain cycles but locally `look like' a tree

301: (e.g. uniformly random graphs with given degree).

302: One of the most widespread features of these models, is the

303: occurrence of glass phases in which the Boltzmann measure gets split

304: into an exponential number of `lumps' (also referred to as clusters or 

305: pure states). This phenomenon is usually studied by solving

306: some `one-step replica symmetry breaking' (1RSB) distributional equations.

307: In the following we show that these equations, as well as the criterion 

308: used to detect glass phases, do indeed coincide 

309: with the solvability of an appropriate reconstruction problem. 

310: 

311: In spin glass theory, one can encounter

312: two types of transitions to a glass phase. In the first 

313: case the transition is continuous in a properly defined order parameter.

314: In spin glass jargon this leads to a 

315: phase with `full replica symmetry breaking' (FRSB).

316: In the second it is discontinuous, leading to 1RSB.

317: Both situations occur in the reconstruction problem, depending on the alphabet

318: and the channel. In the continuous case, the phase transition

319: location is given by a local instability which coincides with the 

320: KS threshold, and one has $\eps_{\rm r}(k,q)=\eps_{\rm KS}(k,q)$.

321: This happens, for instance, when $q=2$.

322: In the opposite case, the `dynamical' glass transition is discontinuous and its

323: location (which still coincides with the reconstruction threshold)

324: is distinct from the KS one. In the ferromagnetic Potts model one has, 

325: for instance,  $\eps_r(k,q)>\eps_{KS}(k,q)$ at large enough $q$.

326: 

327: The coincidence of the reconstruction threshold with the dynamical 

328: glass transition, apart from being interesting in itself, allows us

329: to adapt several techniques developed within the theory of spin glasses

330: in order to study the reconstruction problem.

331: On the one hand, importing a numerical procedure currently used in this field, we 

332: determine the threshold for several pairs $k,q$. These results 

333: lead us to conjecture that $\eps_{\rm r}(k,q)=\eps_{\rm KS}(k,q)$,

334: for $k$ not too large and $q\le 4$ (in the ferromagnetic case) or $q\le 3$ 

335: (in the antiferromagnetic case).

336: 

337: Furthermore, we derive a variational principle for the reconstruction problem. 

338: In the antiferromagnetic case, this implies

339: a rigorous bound on the reconstruction threshold, which allows to confirm 

340: the strict inequality $\eps^{-}_{\rm r}(k,q)<\eps^{-}_{\rm KS}(k,q)$ 

341: in most of the cases in which this was found to be the case numerically.

342: Although we conjecture such a bound to hold in much greater generality,

343: we weren't able to prove it, and we leave it as a conjecture.

344: 

345: The paper is organized as follows. In Section~\ref{sec:Recursion}

346: we define the main objects studied in the paper and prove the coincidence 

347: between reconstruction and dynamical glass transition.

348: In Section \ref{se:varprinc} we state our variational principle

349: and prove that it provides a rigorous bound for a class of 

350: kernels $\pi(\,\cdot\,|\,\cdot\,)$ including the antiferromagnetic model.

351: In Secs. \ref{sec:Ferro} and \ref{sec:AntiFerro} we apply this principle 

352: as well as a numerical procedure to the determination of thresholds for

353: the Potts channel, respectively in the ferromagnetic and antiferromagnetic case. 

354: Section \ref{se:spinglass} discusses the physical meaning of

355: the relation between reconstruction and glass transitions. 

356: Section \ref{se:gene} explains how our methods (and the glass - reconstruction correpondance)

357:  can be generalized to a broad

358: category of broadcast and reconstruction problems on trees, going much beyond the Potts channel.

359: We conclude in Section \ref{sec:Conclusion} by summarizing a few 

360: conjectures and pointing out some interesting open problems.

361: %

362: %***************************************************************************

363: %

364: \section{Distributional recursion}

365: \label{sec:Recursion}

366: 

367: \subsection{Definitions}

368: 

369: We denote by $V$ and $E$ the vertex and edge sets of the infinite 

370: $k$-ary tree $\Tree_k$, by  $0$ its root and by $V_{\ell}$ the set of 

371: generation-$\ell$ vertices ($|V_{\ell}|= k^{\ell}$). The broadcast process 

372: generates a random color

373: configuration $\uX\equiv\{ X_i\; :\;i\in V\}$ with $X_i\in \{1,\dots, q\}$.

374: The root color $X_0 \in \{1,\dots, q\}$, which we also call the transmitted color,  is uniformly random. 

375: Then, given the values

376: of $X$ up to the $\ell$-th generation, the values at the $(\ell+1)$-th

377: generation are conditionally independent. If a vertex in the $\ell$-th

378: generation has color $y$, the probability that a vertex connected to it in the

379: $(\ell+1)$-th generation has color $z$ is $\pi(z|y)$.

380: 

381: We shall denote by $\uX_{\ell}$ the configurations of

382: colors at the $\ell^{\rm th}$ generation, and by 

383: $\uY^{\ell}$ the configuration {\em up to} the $\ell^{\rm th}$ generation

384: (i.e. $\uY^{\ell} = \{\uX_0,\uX_1,\dots,\uX_{\ell}\}$). 

385: The probability distribution of $\uX_{\ell}$, conditioned to the choice

386: $X_0=x$ of the root color will be denoted by 

387: $B^{(\ell)}_x(\ux_\ell)\equiv \prob\{\uX_{\ell} = \ux_{\ell}|X_0=x\}$.

388: 

389: Suppose now that the configuration of colors at the $\ell$-th generation,

390: $\ux_{\ell}$, is given. We denote by $\eta_{\ell}(y)$

391: the probability that the root had sent the color $y$, given $\uX_{\ell}$:

392: %

393: \begin{eqnarray}

394: %

395: \eta_{\ell}(y) = \prob[X_0 = y\, |\uX_\ell=\ux_{\ell}]\, .\label{eq:DefEta}

396: %

397: \end{eqnarray}

398: %

399: $\eta_{\ell}(\cdot)$ is a probability distribution over $\{1,\dots, q\}$ (i.e.

400: $\eta_{\ell}(y)\ge 0$ and $\sum_y\eta_{\ell}(y) = 1$). We shall denote the

401: space of such distributions as $\M$. 

402: In order to emphasize the dependency of $\eta_{\ell}$ upon the configuration

403: received in shell $\ell$, $\ux_{\ell}$,

404: we shall sometimes write  $\eta_{\ell}(y) =\eta_{\ux_\ell}(y)  $.

405: It is easy to realize that, given the colors received

406: at the $\ell^{\rm th}$ generation, $\eta_{\ell}(\cdot)$ constitutes 

407: a sufficient statistics for the root color $x$. In other terms,

408: given $\uX_{\ell}=\ux_\ell$, there is no loss of information in computing

409: $\eta_{\ux_\ell}(\cdot)$ and then guessing $X_0$ from $\eta_{\ux_\ell}(\cdot)$.

410: 

411: Since $\uX_\ell$ is chosen randomly 

412: according to the broadcast process, $\eta_{\ell}(\cdot)$ is a

413: random probability distribution, i.e. a random point in $\M$. We

414: denote by $Q^{(\ell)}_{x}(\eta)$ its 

415: distribution\footnote{Notice that we adopt here the standard physicists 

416: convention: we carelessly denote probability distributions by their densities 

417: even if such densities do not exist. We shall also denote by

418: $\int \!f(\eta)\, \de Q^{(\ell)}_x(\eta)$ the expectation with respect to 

419: such a distribution. The fussy reader can easily translate

420: all the formulae below in the standard probability language.} conditional to

421: the broadcast being started from $X_0=x$, and call it the `distribution at the root'.

422: Hereafter a distribution $Q$ over $\M$ will be said {\em trivial} if it is a

423: singleton on the uniform measure $\eo$ (defined by $\forall x:\ \eo(x) = 1/q$). 

424: Clearly the reconstruction problem is solvable if and only if the large 

425: $\ell$ limit of  $Q_{x}^{(\ell)}$ is non trivial. 

426: 

427: There are several ways of characterizing quantitatively the large

428: $\ell$ behavior of $Q_{x}^{(\ell)}$. We shall consider below two parameters

429: $I_{\ell} \equiv I(X_0;\uX_\ell)$ and 

430: $\Psi_{\ell}$, which are defined by:

431: %

432: \begin{eqnarray}

433: %

434: I_{\ell}  =  \frac{1}{q}\sum_x\int\log_2\frac{\eta(x)}{\eo(x)}\; 

435: \de Q_x^{(\ell)}(\eta)\, ,\;\;\;\;

436: \Psi_{\ell} = \frac{1}{q}\sum_x\int [\eta(x)-\eo(x)]\;\de 

437: Q_x^{(\ell)}(\eta)\, .\

438: %

439: \end{eqnarray}

440: %

441: 

442:  $I_{\ell}$ gives the number of information bits that can be transmitted reliably 

443: per network use. $\Psi_{\ell}$ is the probability that the reconstruction

444: is successful when the receiver guesses color $y$ with probability

445: $\eta_{\ell}(y)$, minus the the same probability when the receiver 

446: guesses uniformly.

447: These are 

448: non-negative quantities and can be shown to be non-increasing 

449: functions of $\ell$. We furthermore let $I_{\infty}\equiv\lim_{\ell\to\infty} I_{\ell}$,

450: and $\Psi_{\infty}\equiv\lim_{\ell\to\infty} \Psi_{\ell}$.

451: 

452: The tree reconstruction problem can be rephrased by saying that 

453: the problem is solvable if and only if $I_{\infty}>0$ (or, equivalently, 

454: $\Psi_{\infty}>0$). For instance, for the ferromagnetic

455: Potts channel, the threshold 

456: $\eps^+_{\rm r}(k,q)$  is the

457: supremum of the values of $\eps$ such that $I_{\infty}>0$.

458: %

459: %*************************************************************************

460: %

461: \subsection{Merging rooted trees}

462: \label{se:merging}

463: 

464: How does one compute the distribution $\eta(y)$ on the root, given a boundary

465: $\uX_{\ell}=\ux_{\ell}$? 

466: Using the tree-structure, this can be done iteratively by a

467: dynamical programming procedure starting from the leaves. Suppose that at some

468: point in this iteration we have determined the probability distributions

469: $\eta_1(\,\cdot\,),\dots,\eta_k(\,\cdot\,)$ of the $k$ vertices in the 

470: tree which lie above a given vertex (see Fig.~\ref{fig:iter}, left). 

471: Then the probability $\eta(y)$ that this vertex had color $y$ 

472: during the broadcast is given by:

473: %

474: \begin{eqnarray}

475: %

476: \eta(y) = \frac{1}{z(\{\eta_i\})}\, 

477: \prod_{i=1}^k\left(\sum_{y_i=1}^q\pi(y_i|y)\, \eta_i(y_i)\right)\, ,\;\;

478: \;\;\;\;\;\; z(\{\eta_i\}) \equiv \sum_{y=1}^q \prod_{i=1}^k\left(\sum_{y_i}\pi(y_i|y)\, \eta_i(y_i)\right)\, .

479: \label{eq:Site}

480: %

481: \end{eqnarray}

482: %

483: This equation defines a mapping between distributions in $\M$: given $k$

484: distributions $\eta_1,\dots,\eta_k $, one generates a new one $\eta =

485: \F(\eta_1,\dots,\eta_k)$. Iterating this mapping

486: downwards from the leaves down to the root, one can derive the the conditional

487: distribution of the transmitted symbol.

488: %

489: %

490: \begin{figure}

491: \includegraphics[width=0.3\linewidth,angle=-0]{iter.eps}\hspace{1.8 cm}

492: \includegraphics[width=0.5\linewidth,angle=-0]{funct_rec.eps}

493: \caption{Left: a  pictorial representation of the mapping $\eta =

494: \F(\eta_1,\dots,\eta_k)$ defined in (\ref{eq:Site}). Here $k=2$,

495: a straight  line corresponds to the channel $\pi$, a wiggly line arriving on

496: a vertex $y_j$ corresponds to a weight $\eta_j(y_j)$. Right: a  pictorial representation of the

497:  recursion (\ref{eq:Iteration}). A triangle of depth $r$ rooted on variable $x_j$ denotes $Q_{x_j}^{(r)}(\eta_r)$}

498: \label{fig:iter}

499: \end{figure}

500: %

501: 

502: Equation (\ref{eq:Site}) naturally induces a recursion equation for the 

503: distribution $Q_{x}^{(\ell)}$.

504: Consider the reconstruction of the root in a rooted tree with $\ell+1$

505: generations (see Fig.~\ref{fig:iter}, right). This graph is formed by $k$ subtrees rooted in the vertices

506: $1,\dots,k$, which are all joined to the root $0$.

507: Each of these subtrees gives an instance of the reconstruction with $\ell$

508: generations. 

509: Therefore:

510: %

511: \begin{eqnarray}

512: %

513: Q_{x}^{(\ell+1)} (\eta)= \sum_{x_1\dots x_k}\prod_{i=1}^k\pi(x_i|x)

514: \int \; \delta\left[\eta- \F(\eta_1,\dots,\eta_k)\right]\;

515: \prod_{i=1}^k \de Q^{(\ell)}_{x_i}(\eta_i)

516: \, ,\label{eq:Iteration}

517: %

518: \end{eqnarray}

519: %

520: where $\delta[\cdots]$ represents a Dirac delta function on $\M$. 

521: In words, in order to generate $\eta(\,\cdot\, )$ with distribution 

522: $Q_{x}^{(\ell+1)}$, one can proceed as follows.

523: First draw $k$ independent colors $x_1,\dots,x_k$

524: from the distribution $\pi(\,\cdot\,|x)$.

525: Then, draw $\eta_1,\dots,\eta_k$ independently with distribution, respectively,

526:  $Q_{x_1}^{(\ell)},\dots,Q_{x_k}^{(\ell)}$. Finally,

527: let $\eta = \F(\eta_1,\dots,\eta_k)$.

528: 

529: The initial condition is 

530: %

531: \begin{eqnarray}

532: %

533: Q^{(0)}_x(\eta) = \delta\left[\eta-\delta_x\right]\, ,\label{eq:Initial}

534: %

535: \end{eqnarray}

536: %

537: where $\delta_x$ is the distribution in $\M$ which has weight unity on color

538: $x$ (it is given by $\delta_x(y)=1$ if $y=x$, and $\delta_x(y)=0$ otherwise).

539: The equations (\ref{eq:Iteration}) and (\ref{eq:Initial}) fully characterize 

540: the distributions $Q^{(\ell)}_x$. The whole reconstruction problem amounts to

541: understanding the large $\ell$ properties of these recursions.

542: 

543: %

544: %*************************************************************************

545: %

546: \subsection{Unconditional distribution and symmetry properties}

547: 

548: While $Q^{(\ell)}_x$ gives the distribution of $\eta_{\ell}(\, \cdot\, )$

549: (defined in Eq.~(\ref{eq:DefEta})) {\em conditional} on  the 

550: transmitted color being equal to $x$, it is equally interesting to consider the

551: {\em unconditional} distribution. We will denote it by  $\Q^{(\ell)}$.

552: Bayes theorem implies the following relation between 

553: $Q^{(\ell)}_x$ and  $\Q^{(\ell)}$

554: %

555: \begin{eqnarray}

556: %

557: Q^{(\ell)}_{x}(\eta) = q\, \eta(x)\, \Q^{(\ell)}(\eta)\, .

558: \label{eq:SymmetricRepresentation}

559: %

560: \end{eqnarray}

561: %

562: This is in fact a rephrasing of the identity

563: %

564: \begin{eqnarray}

565: %

566: \prob\{ \eta_{\uX_\ell}=\eta |X_0=x\} = 

567: \frac{\prob\{X_0=x|\eta_{\uX_\ell}=\eta\}

568: \prob\{\eta_{\uX_\ell}=\eta\}}{\prob\{X_0=x\}}\, .

569: %

570: \end{eqnarray}

571: %

572: 

573: An alternative (analytic) proof can be obtained writing $Q^{(\ell)}_x$

574: in terms of $B^{(\ell)}_x(\ux_\ell)$, the probability that the output of the 

575: broadcast process at generation $\ell$ is $\ux_{\ell}$, given that the

576: transmitted color is $x$:

577: %

578: \begin{eqnarray}

579: %

580: Q^{(\ell)}_x(\eta) = \sum_{\ux_{\ell}} B^{(\ell)}_x(\ux_\ell)\;\; \delta\!\left[

581: \eta(\cdot)-

582: \frac{B^{(\ell)}_\cdot(\ux_\ell)}{\sum_z B^{(\ell)}_z(\ux_\ell)}\right]

583: \, .\label{RepresentationSymm}

584: %

585: \end{eqnarray}

586: %

587: It is then easy to show that, if $\lambda_0+\lambda_1+\dots +\lambda_{q-1}=1$, 

588: then the expectation value

589: %

590: \begin{eqnarray}

591: %

592: \int  \;  \frac{\eta(0)^{\lambda_0}\cdots \eta(q-1)^{\lambda_{q-1}}}

593: {\eta(x)} \; \de Q^{(\ell)}_x(\eta) \, 

594: %

595: \end{eqnarray}

596: %

597: does not depend upon $x$. This in turns imply that $Q_x^{(\ell)}$

598: can be written in the form (\ref{eq:SymmetricRepresentation})

599: where $\Q^{(\ell)}$ is a distribution which does not depend on $x$ 

600: (the normalization can be found by summing over $x$).

601: 

602: If the channel  is symmetric with respect to permutations

603: of the colors (as is the case for Potts channels), the distributions

604: $Q_x^{(\ell)}$ and $Q^{(\ell)}$ inherit the same symmetry.

605: More precisely, given a permutation acting on the colors 

606: $\sigma\in S_q$, and a distribution $\eta\in\M$, let 

607: $\eta^{\sigma}$ be the permuted distribution defined by

608: $\eta^{\sigma}(x) \equiv \eta(\sigma(x))$. Then, for any permutation $\sigma$,

609: $Q^{(\ell)}_x(\eta)=Q^{(\ell)}_{\sigma(x)}(\eta^{\sigma})$, and 

610: %

611: \begin{eqnarray}

612: %

613: \Q^{(\ell)}(\eta^{\sigma}) =  \Q^{(\ell)}(\eta)\, . \label{eq:Symmetry}

614: %

615: \end{eqnarray}

616: %

617:  A distribution satisfying

618:  condition  (\ref{eq:Symmetry}) will be called  `symmetric'.

619: 

620: Let us finally notice that the parameters introduced in 

621: Sec.~\ref{sec:Recursion} to measure the amount of information on the transmitted color 

622: available at the $\ell^{\rm th}$ generation  can be 

623: expressed in terms of the distribution $\Q^{(\ell)}$

624: %

625: \begin{eqnarray}

626: %

627: I_{\ell}  =  \int D(\eta||\eo)\; \de \Q^{(\ell)}(\eta)\, ,\;\;\;\;

628: \Psi_{\ell} = \sum_x\int [\eta(x)-\eo(x)]^2\;\de 

629: \Q^{(\ell)}(\eta)\, .

630: %

631: \end{eqnarray}

632: %

633: Here we use the standard notation for the Kullback-Leibler distance

634: \cite{CoverThomas} $D(\eta||\eo) \equiv \sum_x \eta(x)\log_2[\eta(x)/\eo(x)]$.

635: In deriving the second of these expressions, we used the fact that

636: $\int \eta(x)\, \de\Q^{(\ell)}(\eta) = \eo(x)=\frac{1}{q}$ which follows

637: from (\ref{eq:SymmetricRepresentation}).

638: %

639: %******************************************************************

640: %

641: \subsection{Recursion for the unconditional distribution and 

642: spin glass correspondence}

643: 

644: The recursion relation (\ref{eq:Iteration}) on $Q^{(\ell)}_x $ 

645:  implies the following recursion for the unconditional 

646: distribution:

647: %

648: \begin{eqnarray}

649: %

650: \Q^{(\ell+1)} (\eta)= q^{k-1}\;

651: \int \;

652: z(\{\eta_i\})\;\; \delta\left[\eta- \F(\eta_1,\dots,\eta_k)\right]\;

653: \prod_{i=1}^k \de \Q^{(\ell)} (\eta_i)

654: \, ,\label{eq:1RSB}

655: %

656: \end{eqnarray}

657: %

658: where $z(\{\eta_i\} )$ is defined as in Eq.~(\ref{eq:Site}).

659:  The initial  condition (\ref{eq:Initial}) converts into 

660: $\Q^{(0)}(\eta)=\frac{1}{q} 

661: \sum_{y=1}^q \delta\left[\eta(\,\cdot\,)-\delta_y(\,\cdot\,)\right]$. 

662: It is also interesting to study the fixed points of this recursion, 

663: i.e. the distributions $\Q^{*}$ satisfying:

664: %

665: \begin{eqnarray}

666: %

667: \Q^{*} (\eta)= q^{k-1}\;

668: \int \;

669: z(\{\eta_i\})\;\; \delta\left[\eta- \F(\eta_1,\dots,\eta_k)\right]\;

670: \prod_{i=1}^k \de \Q^{*} (\eta_i)

671: \, ,\label{eq:1RSB_fixed}

672: %

673: \end{eqnarray}

674: %

675: Notice that any solution of this equation has necessarily

676: expectation $\int\,\eta(x)\,\de\Q^*(\eta) = \eo(x)$ (this is proved by

677: taking expectation on both sides). Any probability distribution

678: over $\M$ satisfying this condition will be hereafter said to be

679: `consistent'.

680: 

681: The distributional equation (\ref{eq:1RSB_fixed}) is well known in

682: spin glass theory and usually referred to as `1RSB equation with Parisi parameter $m=1$'

683: (in the general 1RSB scheme  the factor $z(\{\eta_i\})$ is raised to a  power $m \in [0,1]$).

684: It is used to determine whether an associated statistical mechanics model

685: is in a glass phase. We shall return to the definition of the 

686: associated model in Sec.~\ref{se:spinglass}. For the time being,

687: we shall adopt the usual physicists criterion as a {\em definition}:

688: We will say that the statistical mechanics model associated

689: to the reconstruction problem (characterized by a degree/kernel pair  $k,\pi$) {\em admits a glass phase}

690: if and only if Eq.~(\ref{eq:1RSB_fixed}) has a non-trivial solution.

691: 

692: When considering a continuous family of kernels $\pi(\,\cdot\,|\, \cdot\,)$,

693: parametrized by a noise level $\eps$,

694: the value of $\eps$  where a non-trivial solution appears is 

695: called a dynamical glass transition.

696: The result below implies that this coincides indeed with the 

697: reconstruction threshold (i.e. with the extremality threshold for the free

698: boundary Gibbs measure on the infinite tree). 

699: %

700: \begin{propo}\label{propo:Reco}

701: The statistical mechanics model associated

702: with the degree/kernel pair  $k,\pi$ {\em admits a glass phase}, if 

703: and only if the corresponding reconstruction problem is solvable.

704: \end{propo}

705: %

706: \prooft

707: As  noticed for instance in \cite{Winkler03}, the sequence of 

708: random variables $\eta_{\ell}(\, \cdot\, )$ (not conditioned on the root 

709: color), converges almost surely to a limit $\eta_{\infty}(\, \cdot\, )$.

710: As a consequence, the sequence of distributions $\Q^{(\ell)}$ converges

711: weakly to the distribution  $\Q^{(\infty)}$ of $\eta_{\infty}(\,\cdot\,)$.

712: By taking the limit of Eq.~(\ref{eq:1RSB}) (and noticing that 

713: $\F(\eta_1,\cdots,\eta_k)$ and $z(\{\eta_i\})$ are continuous 

714: and bounded) we find that $\Q^{(\infty)}$ must satisfy the fixed

715: point condition (\ref{eq:1RSB_fixed}).

716: If the reconstruction problem is solvable, then $\Q^{(\infty)}$ is

717: non-trivial and therefore, according to our definition,

718: the pair  $k,\pi$ {\em admits a glass phase}.

719: 

720: Conversely\footnote{The idea of the converse is due to James Martin

721: who kindly agreed to let us publish it here.}, let $\Q^*$ 

722: be a non-trivial solution of (\ref{eq:1RSB_fixed}). Following

723: (\ref{eq:SymmetricRepresentation}), define the distribution 

724: $Q^*_x(\eta) = q\, \eta(x) \Q^*(\eta)$. Because of the above calculations,

725: the $q$ distributions  $Q^*_x$, $x\in\{1,\dots,q\}$

726:  are a fixed point of the recursion (\ref{eq:Iteration}). 

727: We will now show that they 

728: can be used to reconstruct the transmitted color from the 

729: output at generation $\ell$, with probability of success independent 

730: of $\ell$ and strictly larger than $1/q$.

731: 

732: The reconstruction procedure goes as follows.

733: Suppose that the broadcast has generated the values $X_i=x_i$ for $i\in

734: V_\ell$. For each vertex $i$, generate $\eta_i$ from the

735: distribution $Q^*_{x_i}$. Consider now a vertex $a \in

736: V_{\ell-1}$,connected to 

737: $a_1,\dots,a_k$ in $V_\ell$. 

738: Compute $\eta_a= \F(\eta_{a_1},\dots,\eta_{a_k})$, where $\F(\cdots)$ is

739: defined as in Eq.~(\ref{eq:Site}). Proceeding downwards from the leaves

740: to the root, this allows to construct $\eta_0$. At this point

741: the transmitted symbol can be guessed, for instance, by choosing 

742: $X_0=y$ with probability proportional to $\eta_0(y)$.

743: 

744: We claim that for each vertex $j\in \Tree_k(\ell)$, and conditional to the broadcast having produced

745: $X_j=x_j$, the  $\eta_j(\,\cdot\,)$ provided by the above procedure is 

746: distributed according to $Q^*_{x_j}$. This in particular implies

747: that the probability of guessing correctly the root color is

748: %

749: \begin{eqnarray}

750: %

751: \frac{1}{q}\sum_x\int \eta(x)\;  \de Q_x^*(\eta) =

752: \sum_x\int \eta(x)^2\;  \de \Q^*(\eta) >\frac{1}{q}\, .

753: %

754: \end{eqnarray}

755: %

756: The claim is proved by induction starting from the leaves 

757: and proceeding downwards to the root. It is true by construction for

758: the vertices of the last generation. Assume it to be true

759: up to generation $r$ and consider a site $a$ in generation

760: $r-1$ connected to  $a_1,\dots,a_k$ in $V_r$, under the condition

761: $X_a=x_a$. It is clear that the distribution  of $\eta_a$ is obtained  

762: through the recursion (\ref{eq:Iteration}) (with $Q^{(\ell)}_{x_i}$

763: replaced by $Q^{*}_{x_{a_i}}$), and since $Q^{*}_{x_{a_i}}$ is a fixed

764: point of this recursion, this proves the claim.

765: \endproof

766: %

767: %***********************************************************

768: %

769: \section{Variational principle} 

770: \label{se:varprinc}

771: %

772: %*****************************

773: %

774: \subsection{The general principle}

775: 

776: Here we establish a variational principle from which the fixed point equation

777: (\ref{eq:1RSB_fixed}) for the distribution at the root

778: can be deduced. We shall not try

779: to explain here  its physical origin, which is related

780: to spin glass theory \cite{MP_Bethe}, but just discuss the relation

781: with the reconstruction problem. Throughout this

782: section we use the notation $\pi(x|y) = \pi(y|x) \equiv

783: \pi(x,y)$. Given a distribution $\Q$ over $\M$, we define its {\em complexity}

784: as 

785: %

786: \begin{eqnarray}

787: %

788: \Sigma(\Q) = -\frac{k+1}{2}\int \W_{\rm e}(\eta_1,\eta_2)\;\; 

789: \de\Q(\eta_1)\,\de\Q(\eta_2)+ \int   \W_{\rm v}(\eta_1,\dots,\eta_{k+1})\;\;

790: \prod_{i=1}^{k+1}\de\Q(\eta_i)\, ,

791: %

792: \end{eqnarray}

793: %

794: where

795: %

796: \begin{eqnarray}

797: %

798: \W_{\rm e} &\equiv & - \left[\frac{\sum_{x_1,x_2}\eta_1(x_1)\eta_2(x_2)\pi(x_1,x_2) }

799: {\sum_{x_1,x_2} \eo(x_1)\eo(x_2)\pi(x_1,x_2)}\right] 

800: \log\left[\frac{\sum_{x_1,x_2}\eta(x_1)\eta(x_2)\pi(x_1,x_2)}

801: {\sum_{x_1,x_2}\eo(x_1)\eo(x_2)\pi(x_1,x_2)}\right] \, ,\\

802: \W_{\rm v} &\equiv & - \left[\frac{\sum_x\prod_i

803: \sum_{x_i}\eta_i(x_i)\pi(x,x_i)}

804: {\sum_x\prod_i\sum_{x_i}\eo(x_i)\pi(x,x_i)}\right] 

805: \log\left[\frac{\sum_x\prod_i\sum_{x_i}\eta_i(x_i)\pi(x,x_i)}

806: {\sum_x\prod_i\sum_{x_i}\eo(x_i)\pi(x,x_i)}\right] \, .

807: %

808: \end{eqnarray}

809: %

810: The complexity is interesting for the reconstruction problem because of the

811:  following 

812: remark:

813: %

814: \begin{propo}\label{propo:Stat}

815: %

816: Let $\Q^*$ be a distribution over $\M$ which satisfies the fixed point equation

817: (\ref{eq:1RSB_fixed}). Then $\Q^*$ is a stationary point of the complexity 

818: $\Sigma(\, \cdot\,)$. More precisely, given any consistent distribution 

819:  $\Q$ over $\M$,

820: define $\Sigma^*(t)\equiv\Sigma((1-t)\Q^*+t\Q)$. Then 

821: %

822: \begin{eqnarray}

823: %

824: \left.\frac{\de\Sigma^*}{\de t}\right|_{t=0} = 0\, .

825: %

826: \end{eqnarray}

827: %

828: \end{propo}

829: %

830: \prooft

831: This proposition is a direct consequence of Lemma \ref{remark1} in Appendix \ref{app:Variational}   

832: (the proof consists in explicitly computing the derivative 

833: of $\Sigma^*(t)$ and checking that it vanishes under the fixed point 

834: conditions). 

835: \endproof

836: 

837: The complexity $\Sigma$ can also be written in terms of the 

838: conditional distributions $Q_x(\eta) = q\eta(x)\Q(\eta)$.

839: Define $p(x_1,x_2)$ to be the marginal distribution of two 

840: neighboring variables on the tree: $p(x_1,x_2) = \pi(x_1,x_2)/q$.

841: Similarly, let $p(x_1\dots x_{k+1})= [\sum_x \pi(x,x_1)\dots\pi(x,x_{k+1})]/q$,

842: the distribution of $k+1$ variables with one common neighbor.

843: The complexity is then given, in terms of $Q_x(\eta)$, by:

844: %

845: \begin{eqnarray}

846: %

847: \Sigma(Q) &=& -\frac{k+1}{2}\sum_{x_1,x_2}p(x_1,x_2)

848: \int  W_{\rm e}(\eta_1,\eta_2)\; \de Q_{x_1}(\eta_1)

849: \,\de Q_{x_2}(\eta_2) +\nonumber\\

850: && +\sum_{\{x_i\}}p(x_1\dots x_{k+1})\int  

851: W_{\rm v}(\eta_1,\dots,\eta_{k+1})\,\prod_{i=1}^{k+1} \de Q_{x_i}(\eta_i)\, ,

852: %

853: \end{eqnarray}

854: %

855: where

856: %

857: \begin{eqnarray}

858: %

859: W_{\rm e} &\equiv & - 

860: \log\left[\frac{\sum_{x_1,x_2}\eta_1(x_1)\eta_2(x_2)\pi(x_1,x_2)}

861: {\sum_{x_1,x_2}\eo(x_1)\eo(x_2)\pi(x_1,x_2)}\right] \, ,\\

862: W_{\rm v} &\equiv & -

863: \log\left[\frac{\sum_x\prod_i\sum_{x_i}\eta_i(x_i)\pi(x,x_i)}

864: {\sum_x\prod_i\sum_{x_i}\eo(x_i)\pi(x,x_i)}\right] \, .

865: %

866: \end{eqnarray}

867: %

868: %*****************************

869: %

870: \subsection{Implications on reconstructibility}

871: 

872: 

873: Experience from spin glass theory, and the  physical interpretation of the complexity,

874: suggests the following conjecture:

875: %

876: \begin{conj}\label{conj:Sigma}

877: Consider the reconstruction problem for the $k$-ary tree and a reversible

878: channel $\pi(y|x) = \pi(x|y)$.

879: If there exists a consistent distribution $\Q^{\rm tr}$ over $\M$, such that $\Sigma(\Q^{\rm tr})<0$,

880: then the reconstruction problem is solvable.

881: \end{conj}

882: %

883: Let us give here a few comments in favor of the plausibility of this

884: conjecture. Notice first that, if $\Q$ is trivial, then $\Sigma(\Q)=0$. Let

885: $\P(\M)$ denote the space of consistent probability distributions over $\M$.

886: Suppose that there exists $\Q^{\rm tr}$ with $\Sigma(\Q^{\rm tr})<0$.

887: Consider now the distribution $\Q^*\in \P(\M)$ such that the complexity is

888: minimal. Of course $\Sigma(\Q^*)\le \Sigma(\Q^{\rm tr})<0$ and therefore

889: $\Q^*$ is non-trivial. If $(i)$ $\Q^*$ is a stationary point of the complexity,

890: and $(ii)$ the stationary points of $\Sigma(\Q)$ in $\P(\M)$ coincide with the

891: solutions of the fixed point equation (\ref{eq:1RSB_fixed}), then the

892: existence of $\Q^{\rm tr}$ implies that the

893: reconstruction problem is solvable. Point $(i)$ amounts to banishing the

894: possibility that $\Q^*$ is on the `border' of $\P(\M)$. Point $(ii)$ is a

895: stronger version of Proposition \ref{propo:Stat}. 

896: 

897: Notice that a priori one

898: could formulate a similar conjecture with a $\Q^{\rm tr}$ having 

899: $\Sigma(\Q^{\rm tr})>0$, replacing `minimum' with `maximum' and 

900: `negative' with `positive' in the above. It is easy to find 

901: counterexamples showing that this `reverse' conjecture is false.  

902: The reason is  probably that the distribution $\Q^*$ maximizing 

903: $\Sigma(\Q)$ is on the border of $\P(\M)$, and therefore $(i)$ does not hold.

904: 

905: Assuming Conjecture \ref{conj:Sigma} to hold, it implies a simple variational technique

906: for proving that reconstruction is possible. Just consider an explicit finite-dimensional family of

907: distributions $\Q_{\mu}$ depending on some parameters $\mu\in{\mathbb R}^d$, and 

908: minimize $\Sigma(\Q_{\mu})$ over $\mu$. If the minimum is negative,

909: then reconstruction is possible.

910: We will apply the {\em variational principle} in this form in the next 

911: Sections. In the rest of this Section (and in Appendix

912: \ref{app:Variational}) we shall prove the

913: principle  for a special family of kernels $\pi(\,\cdot\,|\,\cdot\,)$

914: including the antiferromagnetic Potts channel.

915: 

916: We define a kernel $\pi(y|x) = \pi(x,y)$, $x,y\in\{1,\dots,q\}$

917: to be `frustrated' if it can be decomposed as $\pi(x,y) = \pi_*-\pih(x,y)$

918: where $\pi_*\in {\mathbb R}$ is a constant and $\pih(x,y)$,

919: $x,y\in\{1,\dots,q\}$ is a positive-definite matrix.

920: The antiferromagnetic Potts kernel is a particular instance of this family, with

921:  $\pi_* = \eps/(q-1)$, and $\pih(x,y)= |\lambda_2|\, 

922: \delta_{x,y}$ where $\lambda_2 = 1-q\eps/(q-1)$.

923: 

924: Our basic result is the following.

925: %

926: \begin{lemma}\label{lemma:Variational}

927: Let $\pi(\, \cdot\, ,\,\cdot\,)$ be a frustrated kernel and $\Q^*$

928: a consistent distribution over $\M$ which is not a solution of the 

929: associated fixed point equation (\ref{eq:1RSB_fixed}).

930: Then

931: there exists a consistent distribution $\Q$ over $\M$ such that:

932: %

933: \begin{eqnarray}

934: %

935: \left.\frac{\de\phantom{t}}{\de t}\Sigma((1-t)\Q^*+t\Q)\right|_0<0\, .

936: %

937: \end{eqnarray}

938: \end{lemma}

939: %

940: The proof of this statement is postponed to Appendix \ref{app:Variational}.

941: Here we limit ourselves to proving that it implies the desired 

942: principle.

943: %

944: \begin{propo}\label{propo:VariationalAnti}

945: %

946: Conjecture \ref{conj:Sigma} holds true in the case of frustrated kernels:

947: Let $\pi(\, \cdot\, ,\,\cdot\,)$ be a frustrated kernel, and

948: $\Sigma(\,\cdot\,)$ the associated complexity function. If

949:  there exists a consistent distribution $\Q$ over $\M$ such that 

950: $\Sigma(\Q^{\rm tr})<0$, 

951: then the reconstruction problem is solvable.

952: %

953: \end{propo}

954: %

955: \prooft

956: Let $\Sigma_{\min}\equiv\inf \Sigma(\Q)$, the $\inf$ being taken in $\P(\M)$. Since this 

957: space is subsequentially compact with respect to the weak 

958: topology~\cite{Shiryaev},

959: and $\Sigma$ is continuous with respect to this topology, there exists

960: a consistent distribution $\Q^*$, such that $\Sigma(\Q^*) =\Sigma_{\rm min}$.

961: Because of Lemma \ref{lemma:Variational}, $\Q_*$ is a solution

962: of Eq.~(\ref{eq:1RSB_fixed}).

963: Furthermore $\Sigma(\Q_*)\le \Sigma(\Q^{\rm tr})<0$ and therefore 

964: $\Q_*$ is non-trivial. The result is a consequence of Proposition \ref{propo:Reco}.

965: \endproof

966: %

967: %*****************************

968: %

969: \subsection{An application to Potts channels}

970: \label{eq:VariationalBound}

971:  Here we describe 

972: a simple family of distributions which can be used variationally

973: when studying the Potts channels. We will show in the next sections that, 

974: in spite of its simplicity, it leads to rather accurate results.

975: 

976: The family is indexed by a single real parameter 

977: $\mu\in[0,1]$. We shall denote by $\Q_{\mu}$ the corresponding distribution

978: and will write, with some abuse of notation, 

979: $\Sigma(\mu)\equiv\Sigma(\Q_{\mu})$.

980: The distribution $\Q_{\mu}$ attributes equal weight $1/q$ to the 

981: $q$ points in $\M$ denoted by $\gamma^{(x)}$, $x\in\{1,\dots,q\}$, defined as follows

982: %

983: \begin{eqnarray}

984: %

985: \gamma^{(x)}(y) = \left\{ \begin{array}{ll}

986: 1-\mu & \;\;\;\mbox{if $y=x$},\\

987: \mu/(q-1) & \;\;\;\mbox{otherwise}. 

988: \end{array}

989: \right.

990: %

991: \label{eq:Qmudef}

992: \end{eqnarray}

993: %

994: Some calculus shows that $\Sigma(\mu) = -\frac{k+1}{2}\, w_{\rm e}(\mu)+

995: w_{\rm v}(\mu)$, where 

996: %

997: \begin{eqnarray}

998: %

999: w_{\rm e}(\mu) & = & -\frac{1}{q}\, A\log A-

1000: \frac{q-1}{q}\, B\log B\, ,\\

1001: A & = & q\left\{\frac{\eps}{q-1}+\left(1-\frac{q\eps}{q-1}\right)

1002: \left[(1-\mu)^2+\frac{\mu^2}{q-1}\right]\right\}\, ,\\ 

1003: B & = & q\left\{\frac{\eps}{q-1}+\left(1-\frac{q\eps}{q-1}\right)

1004: \left[\frac{2\mu(1-\mu)}{q-1}+\frac{(q-2)\mu^2}{(q-1)^2}\right]

1005: \right\}\, ,

1006: %

1007: \end{eqnarray}

1008: %

1009: and 

1010: %

1011: \begin{eqnarray}

1012: %

1013: w_{\rm v}(\mu) & = & -\frac{1}{q^{k+1}}\sum_{n_1,\dots, n_q}

1014: \binom{k+1}{n_1,\dots,n_q}\, z[n]\log z[n]\, ,\\

1015: z[n] & = & q^k\left[\frac{\eps+\mu}{q-1}-\frac{q\eps\mu}{(q-1)^2}\right]^{k+1}

1016: \sum_{x=1}^q\left[\frac{\eps + (q-1-q\eps)(1-\mu)}{\eps + (q-1-q\eps)\mu/(q-1)

1017: }\right]^{n_x}\, ,

1018: %

1019: \end{eqnarray} 

1020: %

1021: the first sum  being restricted to $n_1,\dots,n_q\ge 0 $ and 

1022: $n_1+\cdot+n_q = k+1$.

1023: 

1024: Let us briefly discuss how these formulae are used in the following.

1025: To be definite, we refer here to the ferromagnetic case, the antiferromagnetic

1026: one being completely analogous.

1027: Given $k$, $q$ and $\epsilon$, we compute $\Sigma(\mu)$,

1028: and minimize it numerically for $\mu\in[0,1]$.

1029: The largest value (more precisely, the supremum) 

1030: of $\eps$ such that the minimum value is negative, is denoted by 

1031: $\eps_{\rm var}(k,q)$. According to conjecture \ref{conj:Sigma}, we expect $\eps_{\rm r}(k,q)\ge\eps_{\rm var}(k,q)$.

1032: Although we have proved it only for frustrated kernels 

1033: (which do not include the ferromagnetic Potts channel), we shall loosely

1034: use the term `variational bound' also in the other cases..

1035: 

1036: One can show that the variational bound is always at least as good 

1037: as the KS one:

1038: $\eps_{\rm var}(k,q)\ge \eps_{\rm KS}(k,q)$ by looking

1039: at the behavior of  $\Sigma(\mu)$ near to $\mu = (1-1/q)$.

1040:   By Taylor expanding $\Sigma(\mu)$

1041: for $\mu = (1-1/q)+\delta\mu$, we obtain 

1042: $\Sigma(\mu) = c_{k,q}(\eps) \delta\mu^4+O(\delta\mu^5)$.

1043: Furthermore $c_{k,q}(\eps)<0$ for $\eps<\eps_{\rm KS}(k,q)$ and

1044:  $c_{k,q}(\eps)>0$ for $\eps>\eps_{\rm KS}(k,q)$. 

1045: In Fig.~\ref{fig:q7} we plot $\Sigma(\mu)$ for the ferromagnetic Potts channel with $k=2,q=7$,

1046: showing that the variational  bound $\eps_{\rm var}(k,q)$ is strictly larger than the KS one.

1047: We shall discuss in the next section for which values of $k,q$ this happens.

1048: If the variational principle were proved for the ferromagnetic

1049: channel, this

1050: would prove $\eps_{\rm r}(k,q)>\eps_{\rm KS}(k,q)$ 

1051: in these cases.

1052: \begin{figure}

1053: \begin{tabular}{cc}

1054: \includegraphics[width=0.4\linewidth,angle=-0]{q7_var1.eps}\hspace{1 cm}

1055: \includegraphics[width=0.4\linewidth,angle=-0]{q7_var2.eps}

1056: \put(-90,-10){$\mu$}

1057: \put(-310,-10){$\mu$}

1058: \put(-200,70){$\Sigma(\mu)$}

1059: \put(-420,70){$\Sigma(\mu)$}

1060: \end{tabular}

1061: \caption{The complexity for the ferromagnetic Potts channel with 

1062: $k=2$ and $q=7$ within the variational ansatz $\Q_{\mu}$

1063: described in Sec.~\ref{eq:VariationalBound}. A negative complexity

1064: implies that the reconstruction problem is solvable. The three curves

1065: correspond (from bottom to top) to $\eps=0.250$, $0.253$, $0.256$. 

1066: The right plot is a zoom near  $\mu=6/7$. 

1067: The KS threshold for $k=2,q=7$ is $\eps_{\rm KS}\approx 0.2510$. For

1068: $\eps=0.250<\eps_{\rm KS}$,  $\Sigma(\mu)$ is negative in

1069: the neighborhood of $\mu=6/7$. For $\eps=0.253>\eps_{\rm KS}$, 

1070: as $\mu$ decreases from its maximum value $6/7$, $\Sigma(\mu)$ is 

1071: first positive, but then becomes negative with a minimum for 

1072: $\mu\approx 0.6$, implying $\eps_{\rm r}>0.253$. 

1073: This behavior is typical of a first order phase

1074: transition. For $\eps=0.256$, $\Sigma(\mu)$ is always positive, 

1075: and one cannot draw any conclusion. }

1076: \label{fig:q7}

1077: \end{figure}

1078: 

1079: Let us notice that we do not expect the 

1080: variational lower bound to be tight. More precisely, even minimizing it over

1081: the space of distributions over $\M$, $\min \Sigma(\Q)$ becomes

1082: negative only below a threshold $\eps_{\rm c}(k,q)$

1083: with $\eps_{\rm KS}(k,q)<\eps_{\rm c}(k,q)<\eps_{\rm r}(k,q)$

1084: (in the case where $\eps_{\rm KS}(k,q)<\eps_{\rm r}(k,q)$). Our numerical 

1085: simulations confirm this expectation which is motivated by the physical 

1086: interpretation of the complexity.

1087: 

1088: %

1089: %********************************************************************

1090: %

1091: \section{Thresholds for the ferromagnetic Potts channel}

1092: \label{sec:Ferro}

1093: 

1094: %

1095: \begin{figure}

1096: \includegraphics[width=0.45\linewidth,angle=0]{onerun.eps}

1097: \put(-100,-10){$\ell$}

1098: \put(-210,75){$I_\ell$}

1099: \caption{The information (in bits) that can be transmitted reliably through

1100: a $k$-ary tree network of $q$-ary symmetric channels (ferromagnetic

1101: Potts channels), as determined with the population dynamics algorithm.

1102: Here $k=2$, $q=15$ and the noise parameter is (from top to bottom)

1103: $\eps = 0.333082$, $0.308057$, $0.282444$, $0.256422$. 

1104: We used populations of size $M=10^5$, and  averaged over $10$ runs.}

1105: \label{Onerun}

1106: \end{figure}

1107: %

1108: In order to determine reconstruction thresholds numerically,

1109: we simulate the recursion (\ref{eq:Iteration}), 

1110: by representing the distributions $Q^{(\ell)}_x$ through 

1111: a large enough sample. We will estimate reconstruction to be 

1112: possible if the sample does not concentrate, for $\ell$ large 

1113: around the point $\eo$. 

1114: 

1115: This procedure is very similar to the  `population dynamics' method  

1116: used to solve similar equations in spin glass theory~

1117: \cite{Abou-Chacra,MP_Bethe}.

1118: We work with $q$ samples (`populations')

1119: $P^{(\ell)}_1,\dots P^{(\ell)}_q$, each containing $M$ points 

1120: $\eta_i\in \M$, $i\in\{1,\dots,M\}$ 

1121: (i.e. $M$ vectors $\eta_i(x)$, $x \in\{1,\dots,q\}$ with $\eta_i(x)\ge 0$ 

1122: and $\sum_x\eta_i(x) = 1$). 

1123: The population $P^{(\ell)}_x$ represents an i.i.d. sample 

1124: from the distribution $Q^{(\ell)}_x$. The population

1125: $P^{(\ell+1)}_x$, $x\in\{1,\dots,q\}$ is computed, for each 

1126: $\ell\ge 0$ as follows.

1127: %

1128: \begin{itemize}

1129: %

1130: \item Choose $k$ iid colors $x_1,\dots,x_k$ with distribution 

1131: $\pi(\, \cdot\,\vert x)$.

1132: \item Choose $k$ vectors $\eta_1,\dots\eta_k$, with $\eta_i$ uniformly 

1133: random in $P_{x_i}$.

1134: \item Compute $\eta= \F(\eta_1,\dots,\eta_k)$ according to (\ref{eq:Site}).

1135: \item Store this new $\eta$ in the population $P^{(\ell+1)}_{x}$, 

1136: and repeat until the  population contains $M$ elements.

1137: %

1138: \end{itemize}

1139: %

1140: This whole cycle is repeated until the populations $P_x^{(\ell)}$ become 

1141: stationary (by this we mean that their moments no longer depend on $\ell$)

1142: within some prescribed accuracy.

1143: 

1144: Reconstructibility can be monitored by computing the parameters 

1145: $I_{\ell}$ and $\Psi_{\ell}$ on in the populations 

1146: $P_x^{(\ell)}$. If $I_\ell,\Psi_\ell\to 0$ as $\ell\to\infty$,

1147: then we estimate that reconstruction is not possible. If they 

1148: instead converge to a finite value, we take this value

1149: as an estimate of $I_{\infty}$, $\Psi_{\infty}$. Figure 

1150: \ref{Onerun} shows an example of such a calculation. 

1151: Reconstructibility thresholds are determined by repeating the same 

1152: experiment for several values of the channel noise $\eps$.

1153: 

1154: %

1155: \begin{figure}

1156: \includegraphics[width=0.5\linewidth,angle=0]{sumk2zoom.eps}

1157: \put(-205,90){\includegraphics[width=0.2\linewidth,angle=0]{sumk2.eps}}

1158: \put(-100,-5){$q$}

1159: \put(-260,80){$\eps_{\rm r}/\eps_{\rm KS}$}

1160: %\put(-200,85){$0$}

1161: %\put(-135,85){$100$}

1162: %\put(-112,95){$1$}

1163: %\put(-112,145){$1.2$}

1164: \caption{Ratio between the reconstructibility and the KS thresholds 

1165: $\eps_{\rm r}(k,q)/\eps_{\rm KS}(k,q)$,

1166: for the ferromagnetic Potts channel and $k=2$. 

1167: Squares correspond to the numerical determination of $\eps_{\rm r}(k,q)$

1168: and crosses to the variational lower bound $\eps_{\rm var}(k,q)$.

1169: The inset refer to larger number of colors (up to $100$).}

1170: \label{ExampleFig}

1171: \end{figure}

1172: 

1173: \begin{table}

1174: \begin{tabular}{|c|c|c|c|c|c|c|c|c|}

1175: \hline

1176: $q$ & $k$ & $\eps_{\rm r}$ & $\eps_{\rm KS}$ & $\eps_{\rm var}$ 

1177: & $\eps_{\rm alg}$ &  $\eps_{\rm MP}$ &

1178: $I_{*}$  & $\Psi_{*}$ \\

1179: \hline

1180: \hline

1181: $5$ & $2$ & $0.2348(1)$ & $0.2343146$ & $0.23491$ & $---$ & 

1182: $0.30264$ &  $0.052(5)$ & $0.0152(16)$ \\

1183: $5$ & $3$ & $0.33881(5)$ & $0.3381198$ & $0.33887$ & $0.19047$ & 

1184: $0.41712$ & $0.06(2)$ & $0.016(4)$  \\

1185: $5$ & $4$ & $0.4008(1)$ & $0.4$ & $0.40081$ & $0.29046$ &  

1186: $0.48$ & $0.06(1)$ & $0.020(4)$  \\

1187: $5$ & $7$ & $0.4986(1)$ & $0.4976284$ & $0.49847$ & $0.41114$ &  

1188: $0.57143$ & $0.07(1)$ & $0.020(4)$ \\

1189: $5$ & $15$ & $0.5955(1)$ & $0.5934409$ & $0.59422$ & $0.53965$ &  

1190: $0.65238$ & $0.14(1)$ & $0.040(8)$  \\

1191: \hline

1192: $7$ & $2$ & $0.25432(5)$ & $0.2510513$ & $0.25369$ & $---$ &  

1193: $0.34577$ & $0.14(1)$ & $0.028(4)$  \\

1194: $7$ & $4$ & $0.43325(5)$ & $0.4285714$ & $0.43250$ & $0.30769$ &  

1195: $0.53909$ & $0.195(5)$ & $0.045(2)$\\

1196: \hline

1197: $10$ & $2$ & $0.2716(2)$ & $0.2636039$ & $0.26977$ & $---$ &  

1198: $0.38325$ & $0.23(2)$ & $0.040(5)$   \\

1199: \hline

1200: $15$ & $2$ & $0.2881(1)$ & $0.2733670$ & $0.28472$ & $---$ &  

1201: $0.41652$ & $0.37(3)$ & $0.053(4)$  \\

1202: \hline

1203: \end{tabular}

1204: \caption{Thresholds (numerical results and bounds) for the ferromagnetic

1205: Potts channel. 

1206: The reconstruction threshold $\eps_{\rm r} $, 

1207: whose numerical estimate is shown in the first column,

1208: satisfies the rigorous bounds

1209: $\eps_{\rm r}\ge \eps_{\rm KS}$, $\eps_{\rm r}\ge\eps_{\rm alg} $, and 

1210: $\eps_{\rm r}\le \eps_{\rm MP}^-$.

1211: The `algorithmic bound' $\epsilon_{\rm alg}$ is computed by analyzing

1212: reconstruction through recursive majority along the lines of  

1213: Ref.~\cite{MosselRec}.

1214: The variational principle (that is not proven for this ferromagnetic channel

1215: would imply $\eps_{\rm r}\ge \eps_{\rm var}$.

1216: The symbol $--$ means that the corresponding bound does not provide any 

1217: information.

1218: }

1219: \label{TableFerro}

1220: \end{table}

1221: 

1222: Numerical simulations clearly show that the reconstructibility

1223: and Kesten Stigum threshold coincide for $q=3$ and $q=4$. 

1224: We checked this to be the case for $q=3$ and $k=2$--$7,10,15,20,30,50$,

1225: and $q=4$ and $k=2,3,5,10,15,30$ and expect it to be the case generically, at

1226: least for $k$ not too large. When this is the case, the order

1227: parameters $I_{\infty}$, $\Psi_{\infty}$ decrease continuously and vanish at 

1228: $\eps_{\rm r}(k,q)=\eps_{\rm KS}(k,q)$.

1229: 

1230: For $q\ge 5$ we always find $\eps_{\rm r}(k,q)>\eps_{\rm KS}(k,q)$.

1231: In these cases $I_{\infty}(\eps)\downarrow I_*>0$,  

1232: $\Psi_{\infty}(\eps)\downarrow \Psi_*>0$ as $\eps\uparrow \eps_{\rm r}(k,q)$.

1233: In spin glass language, the transition is discontinuous:

1234: we refer to next Section for some illustrations. 

1235: We report our numerical results in Table 

1236: \ref{TableFerro}. This table also contains

1237:  the  variational lower bound $\eps_{\rm var}(k,q)\le\eps_{\rm r}(k,q)$, 

1238: as well as the upper bound derived in

1239:  \cite{MosselPeres01}:

1240: $\eps_{\rm r}(k,q)\le \eps_{\rm MP}^{+}(k,q)$,

1241: where

1242: %

1243: \begin{eqnarray}

1244: %

1245: \eps_{\rm MP}^{\pm}(k,q) = (q-1)\frac{(2-q+2kq)\mp\sqrt{(2-q+2kq)^2

1246: -4k(k-1)q^2}}{2kq^2}\, .

1247: %

1248: \end{eqnarray}

1249: %

1250: 

1251: In Fig.~\ref{ExampleFig} we plot the thresholds as a function of

1252: $q$ for $k=2$.

1253: %

1254: %***********************************************************************

1255: %

1256: \section{Thresholds for the antiferromagnetic Potts channel} 

1257: \label{sec:AntiFerro} 

1258: 

1259: \begin{table}

1260: \begin{tabular}{|c|c|c|c|c|c|c|c|c|c|}

1261: \hline

1262: $q$ & $k$ & $\eps_{\rm r}$ & $\eps_{\rm KS}$ & $\eps_{\rm var}$ 

1263: & $\eps_{\rm alg}$ & $\eps_{\rm MP}^-$ 

1264: & $I_{*}$  & $\Psi_{*}$ & $\Sigma_{*}$ \\

1265: \hline

1266: \hline

1267: $4$ & $8$ & $0.99953(4)$ & $--$ & $--$ & $--$ & $0.91552$ & 

1268: $1.56(4)$ & $0.56(1)$ & $0.026(3)$\\

1269: $4$ & $9$ & $0.9908(4)$ & $1$ & $0.99298$ & $--$ & $0.90717$ & 

1270: $1.31(2)$ & $0.47(2)$ & $0.009(1)$\\ 

1271: $4$ & $10$ & $0.9820(8)$ & $0.9871708$ & $0.98304$ & $--$ & $0.9$ &

1272: $1.2(2)$ & $0.42(4)$ & $0.005(4)$\\

1273: $4$ & $11$ & $0.9725(3)$ & $0.9761335$ & $0.97363$ & $0.99736$ & $0.89376$ & 

1274: $1.07(5)$ & $0.39(1)$ & $\lesssim 0.005$\\

1275: $4$ & $12$ & $0.9643(3)$ & $0.9665063$ & $0.96498$ & $0.98946$ & $0.88826$ &

1276: $0.26(3)$ & $4.2(5)$ & $\lesssim 0.005$ \\ 

1277: $4$ & $15$ & $0.9431(3)$ & $0.9436492$ & $0.94338$ & $0.96903$ & $0.875$ &

1278: $0.5(1)$ & $0.16(3)$ & $\lesssim 0.001$ \\

1279: $4$ & $18$ & $0.9267(2)$ & $0.9267766$ & $0.92686$ & $0.95264$ & $0.86502$ &

1280: $0.3(1)$ & $0.11(4)$ & $\lesssim 0.001$ \\

1281: \hline

1282: $5$ & $13$ & $0.99741(5)$ & $--$ & $0.99982$ & $--$ & $0.92308$ & 

1283: $1.76(4)$ & $0.59(1)$ & $0.042(5)$\\

1284: $5$ & $14$ & $0.9932(1)$ & $--$ & $0.99555$ & $--$ & $0.91916$ &

1285: $1.7(1)$ & $0.54(2)$ & $0.03(1)$\\

1286: $5$ & $15$ & $0.9888(1)$ & $--$ & $0.99092$ & $--$ & $0.91561$ &

1287: $1.48(5)$ & $0.48(2)$ & $0.03(1)$\\

1288: $5$ & $20$ & $0.9685(3)$ & $0.9788854$ & $0.96991$ & $0.98581$ & $0.90177$ &

1289: $1.1(5)$ & $0.36(2)$ & $0.01(1)$\\

1290: \hline

1291: $6$ & $17$ & $0.999924(5)$ & $--$ & $--$ & $--$ & $0.93482$ & 

1292: $2.20(4)$ & $0.667(15)$ & $0.095(5)$  \\ 

1293: $6$ & $20$ & $0.9932(3)$ & $--$ & $0.99546$ & $--$ & $0.92792$ &

1294: $1.87(6)$ & $0.569(15)$ & $0.04(2)$\\ 

1295: \hline

1296: \end{tabular}

1297: \caption{Thresholds (numerical results and bounds) for the antiferromagnetic

1298: Potts channel.

1299: The reconstruction threshold $\eps_{\rm r} $, 

1300: whose numerical estimate is shown in the first column,

1301:  satisfies the rigorous bounds

1302:  $\eps_{\rm r}\le \eps_{\rm KS}$ (from \cite{KestenStigum2}), 

1303: $\eps_{\rm r}\le\eps_{\rm alg}$ (cf. \cite{MosselRec}), 

1304: $\eps_{\rm r}\le\eps_{\rm var} $ (from Proposition \ref{propo:VariationalAnti}), 

1305: and 

1306:  $\eps_{\rm r}\ge \eps_{\rm MP}^-$ (from \cite{MosselPeres01}).

1307:  The symbol $--$ means that the corresponding bound does not 

1308: provide any information.}

1309: \label{TableAnti}

1310: \end{table}

1311: %

1312: %

1313: \begin{figure}

1314: \includegraphics[width=0.45\linewidth,angle=0]{q3k6.eps}

1315: \hspace{1cm}

1316: \includegraphics[width=0.45\linewidth,angle=0]{q4k9.eps}

1317: \put(-465,95){\includegraphics[width=0.475\linewidth,angle=0]{q3k6cplx.eps}}

1318: \put(-221,95){\includegraphics[width=0.475\linewidth,angle=0]{q4k9cplx.eps}}

1319: \put(-100,-10){$\eps$}

1320: \put(-350,-10){$\eps$}

1321: \put(-234,50){$\Psi_{\infty},I_{\infty}$}

1322: \put(-480,50){$\Psi_{\infty},I_{\infty}$}

1323: \put(-225,138){$\Sigma_{\infty}$}

1324: \put(-474,138){$\Sigma_{\infty}$}

1325: \caption{Asymptotic complexity $\Sigma_{\infty}$, information capacity 

1326: $I_{\infty}$ and conditional variance $\Psi_{\infty}$ as a function

1327: of the noise parameter for the antiferromagnetic Potts channel.

1328: On the left: typical continuous reconstructibility transition,

1329: $q=3$, $k=6$. On the right: typical discontinuous transition,

1330: $q=4$, $k=9$. }

1331: \label{DisContinuousFig}

1332: \end{figure}

1333: %

1334: 

1335: In Table \ref{TableAnti} we present our numerical results for the 

1336: reconstruction thresholds of the antiferromagnetic 

1337: Potts channel in the cases in which it differs from $\eps_{\rm KS}$, together with the

1338: bounds. 

1339: 

1340: One distinctive feature of this channel is that, even

1341: in the limit $\eps\to 1$ reconstruction may be impossible. For any 

1342: given $q\ge 3$ reconstruction becomes possible only for 

1343: $k\ge k_*(q)$. Numerically we found $k_*(3) = 5$, $k_*(4)=8$, $k_*(5)=13$,

1344: $k_*(6)=17$.  In fact the case $\eps=1$ has a special 

1345: interest. In this case the configuration produced by the broadcast process 

1346: is  distributed according to the free boundary

1347: Gibbs measure for proper colorings of the (infinite) tree $\Tree_k$.

1348: Our numerical results imply that this measure is extremal only for 

1349: $k<k_*(q)$, with $k_*(q)$ as above. Using the variational principle

1350: (which in this case is proved, cf. Proposition \ref{propo:VariationalAnti}), 

1351: we can show that $k_*(3)\le 5$,  $k_*(4)\le 9$, $k_*(5)\le 13$,

1352: $k_*(6)\le 17$\dots

1353: 

1354: For $q=3$ we found the reconstructibility threshold to coincide

1355: always with the KS threshold. This was checked for

1356: $k=4$--$7$, $10$, $20$. The parameters $I_{\infty}$ and $\Psi_{\infty}$ are 

1357: continuous functions of $\eps$ vanishing at $\eps_{\rm KS}$.

1358: An example is provided in Fig.~\ref{DisContinuousFig}, left frame.

1359: For $q\ge 4$ and $k\ge k_*(q)$ the transition at the reconstructibility

1360: threshold is discontinuous, cf. Fig.~\ref{DisContinuousFig}, right frame.

1361: Table   \ref{TableAnti} gives the values of $I_{*}$, $\Psi_{*}$

1362: and $\Sigma_{*}\equiv \lim_{\eps\to \eps_r}\Sigma(\Q^{(\infty)})$

1363: (in the ferromagnetic channel, this number is so small that it cannot be measured reliably in the numerics).

1364: Most of the remarks made for the ferromagnetic channel apply to this case.

1365: 

1366: %

1367: %************************************************************************

1368: %

1369: \section{Relation to spin glass theory} 

1370: \label{se:spinglass}

1371: 

1372: In this section we explore the link between reconstruction and spin glass

1373: theory. 

1374: For simplicity we keep to the Potts channel, but the discussion

1375: can be easily generalized. 

1376: Consider a configuration $\uY^{L}$ generated by the

1377: broadcast along a finite rooted tree $\Tree_k(L)$ with $L$ 

1378: generations, starting from a  uniformly random symbol in 

1379: $\{1,\dots,q\}$ at the root.

1380: As we saw in the introduction, $\uY^L$ is an equilibrium 

1381: configuration of the Potts model with free boundary conditions

1382: on $\Tree_k(L)$, i.e. is distributed according to the

1383: Boltzmann law for the energy function (\ref{eq:PottsEnergy}).

1384: The coupling $J$ of this model is given by 

1385: $e^{-\beta J}=\frac{\eps}{(q-1) (1-\eps)}$, 

1386: and is ferromagnetic (resp. antiferromagnetic) if $J>0$ (resp $J<0$).

1387: 

1388: Once the  broadcast process has fixed the variables on

1389: the boundary at distance $L$ from the root, the reconstruction

1390: problem can be phrased in terms of the conditional distribution

1391: $\prob\{X_0=x|\uX_{L}=\ux_{L}\}$. 

1392: The distribution of the first $L-1$ generations given the received 

1393: symbols, $\prob\{\uY^{L-1}|  \uX_{L}=\ux_{L}\}$,

1394: is also given by Boltzmann law for the energy function (\ref{eq:PottsEnergy}).

1395: However the boundary condition is now given by the received symbols.

1396: One fundamental reason why reconstruction is related to a spin glass 

1397: problem is that this boundary condition tends

1398: to frustrate the system, in the sense of creating conflicting constraints.

1399: 

1400: It is well known that on trees, frustration comes only through the choice 

1401: of boundary conditions. Here we discuss the spin glass phase induced in the 

1402: Potts model on the tree by various possible choices. We shall first show how a

1403: `naive' choice of boundary conditions leads to a simple replica symmetric

1404: recursion relation. Then we show how well-chosen self-consistent boundary

1405: conditions lead to the correct 1RSB fixed point equation of 

1406: reconstruction.

1407: Finally we discuss the explicit realization of the corresponding spin 

1408: glass

1409: model as a model of Potts spins on a random graph (not a tree).

1410: %

1411: %************************************************************************

1412: %

1413: \subsection{Boundary conditions with independent spins}

1414: %

1415: As before, we call $\uX_{L}$ the set of all spins in the $L^{\rm th}$

1416: generation. A boundary condition (BC) is a probability distribution  on 

1417: these spins. One first possibility is when spins on the boundary are

1418: independent random variables: a given spin $X_i$, $i\in V_{L}$ takes 

1419: value $x_i$ with probability $\eta_i(x_i)$. 

1420: The overall distribution is therefore $\prod_{i \in V_{L}}\eta_i(x_i)$. 

1421: Once a set of $\eta_i(\, \cdot\, )$'s is given, the partition function of 

1422: the tree is obtained as:

1423: %

1424: \begin{equation}

1425: %

1426: Z_L(\{\eta_i\}_{i\in V_L})=\sum_{\uy^L} \prod_{(i,j)\in \Tree_k(L)} 

1427: \pi(x_i,x_j) \prod_{i\in V_L} \eta_i(x_i)\, ,

1428: \label{Zdef}

1429: %

1430: \end{equation}

1431: %

1432: and the Boltzmann distribution is 

1433: %

1434: \begin{equation}

1435: %

1436: \prob^L_{\{\eta_i\} }(\uy^{L}) =\frac{1}{Z_L(\{\eta_i\}}\; 

1437: \prod_{(ij)\in \Tree_k(L)} 

1438: \pi(x_i,x_j) \prod_{i\in V_L} \eta_i(x_i)\, .\label{eq:BoltzmannBC}

1439: %

1440: \end{equation}

1441: %

1442: 

1443: We have still the freedom of chosing the $\eta_i(\, .\,)$.

1444: One  simple possibility would be to take them identical:

1445: $\eta_i(\, \cdot\, )= \overline{\eta}(\, \cdot\, )$ 

1446: \cite{Peruggi83,Peruggi84a,Peruggi84b}, but in order to have a

1447:  disordered and frustrated problem one can choose to sample 

1448: the $\eta_i$'s independently  from a {\it symmetric} distribution 

1449: $P^{(0)}(\eta)$. This definition of a spin glass problem on a tree 

1450: was adopted for instance in \cite{Chayes86} in the Ising case ($q=2$),

1451: where each of the boundary spins was fixed to 

1452: $\pm 1$ independently. In our formulation, this corresponds to the choice

1453: $P^{(0)}(\eta)=\frac{1}{2}\left(\delta\left[\eta(\, \cdot\,),\delta_{+1}\right]

1454: +\delta\left[\eta(\,\cdot\,),\delta_{-1}\right]\right)$.

1455:  

1456: The recursive procedure for merging rooted trees applies in the same way 

1457: as in Sec.~\ref{se:merging}. Consider the marginal distribution on the

1458: first $L-\ell\le L$ generations, $\prob(\uy^{L-\ell})$. It is clear that

1459: this has the same form as in Eq.~(\ref{eq:BoltzmannBC}),

1460: with some new $\eta'_i(\,\cdot\, )$, $i\in V_{L-\ell}$.

1461: When one generate BCs randomly as described above, the $\eta'_i$ are 

1462: iid random variables with common distribution 

1463: $P^{(\ell)}(\eta)$. A little thought  shows

1464: that $P^{(\ell)}(\eta)$ is related to the one in the shell just 

1465: above by:

1466: %

1467: \begin{equation}

1468: P^{(\ell+1)} (\eta)=

1469: \int 

1470: \delta\left[\eta- \F(\eta_1,\dots,\eta_k)\right]\;\; \prod_{i=1}^k 

1471: \de P^{(\ell)}(\eta_i)

1472: \, .

1473: \label{eq:iterRS}

1474: \end{equation}

1475: %

1476: Notice that in each shell, $P^{(\ell)}$ is symmetric.

1477: 

1478: The marginal distribution at the root of the tree $\Tree_k(L)$, 

1479: under the Boltzmann law (\ref{eq:BoltzmannBC}),

1480: is a random variable with distribution $P^{(L)}(\eta)$.

1481: In this model, the existence of a spin glass phase is characterized by

1482: a non trivial limit of $P^{(L)}(\eta)\to P^{(\infty)}(\eta)$ as $L\to\infty$.

1483: Such a limit solves  a  fixed point equation corresponding to

1484: (\ref{eq:iterRS}). 

1485: 

1486: The reader will notice that Eq.~(\ref{eq:iterRS}) is similar to 

1487: the reconstruction equation (\ref{eq:1RSB}), with one crucial difference:  

1488: the `reweighting' factor $z(\{\eta_i\})$ in

1489: (\ref{eq:1RSB}) is absent here. In the spin-glass jargon, 

1490: Eq.~(\ref{eq:iterRS}) is the  `replica symmetric' (RS) equation,

1491: while Eq.~(\ref{eq:1RSB}) is the 1RSB

1492: equation with Parisi parameter $m=1$.

1493: 

1494: 

1495: We want to argue that  the model defined by Eq.~(\ref{eq:BoltzmannBC}), with iid $\eta_i$'s

1496: in not a `good' model of spin glass on a tree.

1497: Technically this is seen from the fact that its glass

1498: phase is  a RS one, while the spin glass models on graphs with loops typically show RSB.

1499: Fundamentally, the drawback of this model is precisely

1500: that it neglects correlations between spins on the boundary. 

1501: As we will discuss in Sec.~\ref{Bethe_lattice}, such correlations are necessary in order

1502: to study the existence of many pure states, a distinguished mark of spin glasses on graphs

1503: with loops. 

1504: 

1505: 

1506: A minimalistic way of introducing such correlations is

1507: to keep uniquely those correlations induced by the tree itself;

1508: this is precisely what is done in reconstruction,

1509: as we now discuss.

1510: %

1511: %**************************************************************

1512: %

1513: \subsection{Self-consistent boundary conditions}

1514: %

1515: It is clear that the broadcast/reconstruction process generates correlated BCs.

1516: By this we mean that the conditional distribution

1517: $\prob(\uY^{L-1}=\uy^{L-1}|\uX_L)$ has still the form 

1518: (\ref{eq:BoltzmannBC})

1519: but the $\eta_i$ are no longer independent. 

1520: More explicitely, the $\eta_i$'s, $i\in  V_L$ 

1521: have distribution:

1522: %

1523: \begin{equation}

1524: \prob(\{\eta_i\}_{i\in V_L})= \frac{1}{\Xi_L}\, 

1525: Z_L(\{\eta_i\}_{i\in V_L}) \,

1526: \prod_{i\in V_L} \QQ^{(0)}(\eta_i) \ ,

1527: \label{scBC}

1528: \end{equation} 

1529: %

1530: where $Z(\{\eta_i\})$ is the partition function 

1531: (\ref{Zdef}) of the tree  $\Tree_k(L)$ with  BC $\{\eta_i\}$,

1532: and $\QQ^{(0)}(\eta)$ is the uniform distribution on the $q$ 

1533: `corners' of the simplex 

1534: $\eta(x)=\delta_{x,r}$, $r\in\{1,\dots,q\}$.

1535: 

1536: We can analyze the system with BC 

1537: (\ref{scBC}) as we did in the previous section for uncorrelated BCs.

1538: Consider, as before, the marginal distribution of the first $L-\ell\le L$

1539: generations of the tree. It also has the form 

1540: (\ref{eq:BoltzmannBC}) and

1541: the new $\eta_i$'s at distance $\ell$ retain the same correlation structure. 

1542: Their joint distribution is

1543: %

1544: \begin{equation}

1545: %

1546: \prob(\{\eta_i\}_{i\in V_\ell})= \frac{1}{\Xi_{\ell}}\, 

1547: Z_\ell(\{\eta_i\}_{i\in V_\ell}) \,

1548: \prod_{i\in V_{\ell}} \QQ^{(\ell)}(\eta_i) \, .

1549: \label{scBC2}

1550: %

1551: \end{equation}

1552: %

1553: Finally, the distribution

1554: $\QQ^{(\ell+1)}$ is related to $\QQ^{(\ell)}$ through the recursion 

1555: (\ref{eq:1RSB}) that

1556: we found when discussing reconstruction, with the correct reweighting 

1557: factor. Since we took $\QQ^{(0)}(\eta) = \Q^{(0)}(\eta)$,

1558: this implies $\QQ^{(\ell)}(\eta) = \Q^{(\ell)}(\eta)$ for any

1559: $\ell\ge 0$.

1560: 

1561: It is interesting to define the spin glass problem on the tree associated 

1562: with the non-trivial fixed point of Eq.~(\ref{eq:1RSB_fixed}). 

1563: This just amounts to  generating

1564: the BCs from (\ref{scBC}) using $\QQ^{(0)}=\Q^{*}$. This problem has the

1565: virtue of being statistically translation 

1566: invariant~\footnote{Provided one replaces the rooted tree

1567: (with a root of degree $k$) with a regular Cayley tree 

1568: (with all the vertices  of degree $k+1$).}

1569: (although, for any given realization, the resulting Gibbs measure is 

1570: not translation invariant).

1571: In particular the properties of a spin don't depend on its distance to the 

1572: root.

1573: %

1574: %***********************************************************************

1575: %

1576: \subsection{Spin glass on the Bethe lattice}

1577: %

1578: \label{Bethe_lattice}

1579: While the previous definition of a spin glass on a tree is perfectly

1580: correct, it is clear that a lot of the physics has been put into the choice 

1581: of the BC distribution. This is necessary because of the crucial role of BCs 

1582: on trees. An alternative definition of the Bethe lattice spin glass, proposed 

1583: in \cite{MP_Bethe}, is to use, instead of trees, random graphs which have a 

1584: tree structure on finite length scales. Let us consider for instance the 

1585: problem of $N$ Potts spins on the vertices of a random regular graph 

1586: $\cG_N$ with  degree

1587: $k+1$, with pairwise interactions given by the kernel $\pi(x,y)$

1588: The partition function of such a model is

1589: %

1590: \begin{equation}

1591: Z=\sum_{x_1,\dots,x_N} \prod_{(i,j)\in \cG_N} \pi(x_i,x_j)

1592: \label{eq:randGr} 

1593: \end{equation}

1594: %

1595: In any finite neighborhood of a randomly chosen node $i$, the local 

1596: structure~\footnote{By this we mean the subgraph within any fixed distance 

1597: from $i$. The property described here can also be phrased in  

1598: terms of {\em local weak convergence}~\cite{AldousSteele}} of $\cG_N$ 

1599: is (with high probability) the one of a regular tree with degree $k+1$. 

1600: In fact, the shortest loop through $i$

1601: is typically of size $\log N$ which diverges when $N\to\infty$. 

1602: This setting is interesting for two reasons:

1603: $(i)$ Loops, although large, can create some frustration; 

1604: $(ii)$ The system is approximately homogeneous (unlike on a regular tree,

1605: where vertices on the boundary have a neighborhood very 

1606: different from the others).

1607: 

1608: Spin glasses on random lattices with a local tree-like structure have been 

1609: the object of many studies in recent times. The cavity method of

1610: \cite{MP_Bethe,MP_Bethe_T0} is an iterative procedure which exploits the

1611: tree-like structure. Here we shall just mention some of its main results 

1612: without justification, the aim being to clarify the correspondence

1613: between the spin glass model on a random graph and the reconstruction 

1614: problem.

1615: 

1616: %

1617: \begin{figure}

1618: \includegraphics[width=0.25\linewidth,angle=0]{cavrec.eps}

1619: \put(-50,-5){$j$}

1620: \put(-50,40){$i$}

1621: \put(-50,85){$l$}

1622: \put(-90,25){$\eta_{i\to j}$}

1623: \caption{The basic cavity recursion.}

1624: \label{fig:CavityRecursion}

1625: \end{figure}

1626: %

1627: In the cavity method one first considers the graph, rooted in a site $i$,

1628: obtained by cutting the edge $(i,j)$ to one of its neighbors, cf. 

1629: Fig.~\ref{fig:CavityRecursion}. The 

1630: marginal distribution of the root, $\eta_{i \to j}(x_i)$, 

1631: with respect to the model on the `amputated' graph, is then 

1632: written in terms of the distributions $\eta_{l\to i}(x_l)$ 

1633: where $l$ are the  neighbors of $i$ different from $j$. It is possible to write 

1634: such a recursion only if the variables $x_l$, 

1635: in the absence of the edges $(l,j)$, become  uncorrelated in the large $N$ 

1636: limit. Such a property is made possible by the local 

1637: tree-like structure,  but  also requires 

1638: a fast decay of correlations in the graph.

1639: This is expected to happen either when the system admits 

1640: a single Gibbs state, or when the Boltzmann measure is restricted 

1641: to a pure (extremal) Gibbs state~\footnote{The definition of 

1642: extremal Gibbs state on a {\em finite} graph goes beyond the scope of this

1643: paper.}.  In 

1644: the first case, the problem is described by a unique distribution $\eta(x)$, 

1645: and $\eta_{i \to j}=\eta$ for all directed edges $i\to j$. 

1646: This distribution is a fixed

1647: point of (\ref{eq:Site}), satisfying thus $\eta=\F(\eta,\dots,\eta)$. It 

1648: is called the 'paramagnetic' or 'liquid' phase. 

1649: In the case where there exist

1650: several pure states, the recursion holds when the measure is restricted to 

1651: one pure state $\alpha$: on a given (large) graph one could thus generate a 

1652: set of `messages' $\eta_{i\to j}^\alpha(x_i)$ for each state $\alpha$. Notice 

1653: that, for a given $\alpha$, the messages now depend explicitely on the 

1654: edge: the measure is no longer uniform, but it is modulated. The 1RSB 

1655: cavity method assumes that there exist exponentially 

1656: many such pure states, the number $\cN(f)$ of states with free energy density 

1657: $F^\alpha/N=f$ is written in terms of the complexity function $\Sigma(f)$ as 

1658: $\cN(f)=\exp(N \Sigma(f))$. In such a case one can perform a statistics in the space of 

1659: pure

1660: states, by introducing, for each edge $(i,j)$, the probability $R_{i\to

1661: j}(\eta)$ that the message $\eta_{i\to j}^\alpha=\eta$, when $\alpha$ is 

1662: chosen

1663: randomly with a weight proportional to the total Boltzmann weight of state

1664: $\alpha$. After performing this average over states the various edges 

1665: become

1666: again equivalent, and one finds that the distribution $R_{i\to 

1667: j}(\eta)=\Q^*$

1668: satisfies exactly the 1RSB fixed point equation (\ref{eq:1RSB_fixed}). So

1669: there exists a 1RSB glass phase if and only if this equation has a non 

1670: trivial

1671: symmetric solution. Notice that this equation can also have other

1672: non-symmetric solutions. For instance in the case of the ferromagnetic 

1673: Potts

1674: channel, at low enough temperature there is a solution where $Q^*$ is 

1675: peaked

1676: on a $\eta$ with a ferromagnetic bias, but it does not satisfy the 

1677: symmetry

1678: property that we impose for the study of the glass state. In such a system 

1679: the

1680: glass solution exists, but it is not realized on a random graph: the 

1681: system

1682: will transit to a ferromagnetic phase. On the contrary in some other cases 

1683: the

1684: glass phase will be realized. For instance we expect this to be the case 

1685: for

1686: the antiferromagnetic Potts model on the random graph \cite{Mulet02,Brauenstein03}.

1687: 

1688: 

1689: The tree  reconstruction problem on the one hand, and the spin glass on a

1690: random graph on the other, thus naturally lead to the same equations.

1691: Some aspects of this correspondence call for a better understanding. 

1692: Consider the model on the random graph defined in 

1693: Eq.~(\ref{eq:randGr}). Let us suppose that it has several pure states, and that

1694: the 1RSB cavity solution of the problem is correct. Now isolate around an

1695: arbitrary point the set of all its neighbors up to distance $\ell$.

1696: Generically it is a tree $\Tree_k(\ell)$. The vertices outside this tree create

1697: some boundary condition on the leaves of this tree, depending 

1698: on the pure state $\alpha$ that we are considering. 

1699: We have found  that

1700: the statistics of these BC on the pure states corresponds to the statistics

1701: of the boundaries in the broadcast/reconstruction, and both are described

1702: by the distribution $\Q^*$.

1703:  In spin glass theory (within 1RSB) one can count

1704: the pure states through the computation of the complexity function

1705: $\Sigma(f)$. It would be very interesting to have an

1706: interpretation of this function in terms of the reconstruction problem.

1707: %

1708: %********************************************************************

1709: %

1710: \section{Generalizations}

1711: \label{se:gene}

1712: 

1713: So far we have focused on $k$-ary trees whose links corresponds 

1714: to identical copies of the same $q$-ary channel satisfying the symmetry 

1715: condition $\pi(y|x) = \pi(x|y)$. However, none of these hypotheses is  

1716: crucial to our approach. In this section we define a considerably more

1717: general context, and sketch how to adapt the above formalism to this case.

1718: Some cases of broadcast through non regular trees, or with  asymmetric channels have been considered

1719: for instance in  \cite{EvaKenPerSch,Mossel01,MosselPeres01,Martin03}. The present formalism encompasses

1720: all these cases and generalizes them to broadcast through hypergraphs.

1721: 

1722: We consider a finite set of kernels 

1723: $\{\pi^{(\alpha)}(\, \cdot\, |\,\cdot\,)\, ;\, \alpha=1,\dots,n\}$,

1724: each kernel describing a one-to-many communication channel.

1725: For each $x\in\{1,\dots,q\}$, and each $k_{\alpha}$-uple

1726: $y_1,\dots, y_{k_\alpha}$,

1727: $\pi^{(\alpha)}(y_1,\dots,y_{k_{\alpha}} |x)$ gives is the probability

1728: that users $1,\dots,k_\alpha$ receive outputs $y_1,\dots,y_{k_{\alpha}}$

1729: if the channel input was $x$. These kernels must

1730: satisfy the conditions

1731: %

1732: \begin{eqnarray}

1733: %

1734: \pi^{(\alpha)}(y_1,\dots,y_{k_{\alpha}} |x)\ge 0\, ,

1735: \;\;\;\;\;\;\; \sum_{y_1,\dots,y_{k_{\alpha}}}

1736: \pi^{(\alpha)}(y_1,\dots,y_{k_{\alpha}} |x) = 1\, ,

1737: %

1738: \end{eqnarray}

1739: %

1740: and $x$ is called the parent of $y_1,\dots,y_{k_{\alpha}}$. Such `one-to-$k$' communication 

1741: channels can be represented graphically

1742: using factor nodes of degree $k+1$, cf. Fig.~\ref{fig:fnode}. 

1743: 

1744: %

1745: \begin{figure}

1746: \includegraphics[width=0.3\linewidth,angle=-0]{fnode.eps}\hspace{1 cm}

1747: \includegraphics[width=0.6\linewidth,angle=-0]{fnode2.eps}

1748: \caption{Left: a function node representing the  `one-to-$k$' channel $\pi^{(\alpha)}(y_1,\dots,y_{k} |x)$.

1749: Right: An example of a small random tree network, with $l_0=3$ }

1750: \label{fig:fnode}

1751: \end{figure}

1752: %

1753: 

1754: 

1755: Next, we define a {\em random} tree network ensemble depending

1756: on two probability distributions $q_{\alpha}$, $\alpha\in\{1,\dots,n\}$ 

1757: ($q_{\alpha}\ge 0$, $\sum q_{\alpha}=1$) and

1758: $p_l$, $l\ge 0$ ($p_{l}\ge 0$, $\sum p_{l}=1$).

1759: One (infinite) random network $\Tree$ 

1760: from this ensemble is generated as follows

1761: starting from the root $0$.

1762: %

1763: \begin{itemize}

1764: %

1765: \item Draw an integer $l_0$ with distribution $p_l$. This is the degree

1766: of the vertex $0$

1767: %

1768: \item For each $a\in\{1,\dots, l_0\}$, draw $\alpha_a$ independently with 

1769: distribution $q_{\alpha}$, and attach a channel of type $\alpha$

1770: to the vertex $0$. The root will transmit through such channels.

1771: %

1772: \item For each  $a\in\{1,\dots, l_0\}$, and each $i_a\in\{1,\dots,

1773: k_{\alpha_a}\}$, associate a vertex to the $i_a$-th output of 

1774: channel $a$. Repeat the above construction for each of these vertices.  

1775: %

1776: \end{itemize}

1777: %

1778: In Fig.~\ref{fig:fnode} we show a small example of such a network.

1779: We denote by $\Tree(i)$ the random (sub)network rooted at $i$, by

1780: $\Tree(i,\ell)$ its first $\ell$ generations (starting from $i$),

1781: and by $\uX_{i,\ell}$ the received colors, $\ell$ generations above

1782: $i$ ( $\uX_{0,\ell}\equiv\uX_{\ell}$).

1783: 

1784: The network is used to communicate. A color $x_0\in\{1,\dots,q\}$

1785: is chosen at the root with probability $\varphi_0(x_0)$ and

1786: broadcast through  the $l_0$ channels connected to the root itself.

1787: Each of the first generation vertices receives a corrupted 

1788: version $x_i\in\{1,\dots,q\}$ of this color, with joint distribution

1789: %

1790: \begin{eqnarray}

1791: %

1792: \prod_{a=1}^{l_0}\pi^{(\alpha)}(x_{a,1},\dots,x_{a,i_a}|x_0)\, ,

1793: %

1794: \end{eqnarray}

1795: %

1796: where $(a,r)$ denotes the $r$-th output vertex of the $a$-th channel.

1797: In other words distinct channels act independently. 

1798: The same transmission process is repeated at the first generation and so 

1799: forth, through the entire network. 

1800: The problem is to reconstruct the transmitted color from the output 

1801: at generation $\ell$, denoted as $\ux_{\ell}$.

1802:  

1803: Analogously to the case investigated in the previous sections,

1804: we say that the reconstruction problem is solvable if the 

1805: conditional mutual information

1806: $I(X_0;\uX_{\ell}|\Tree)$ does not vanish as $\ell\to\infty$.

1807: Equivalently, the problem is solvable if there is a reconstruction

1808: procedure which succeeds with probability strictly larger than 

1809: $\max_{x_0}\varphi_0(x_0)$ in the $\ell\to\infty$ limit. In these definitions

1810: we assume the network structure to be known at the 

1811: receiver (this is why we consider a  mutual information which is {\em conditional} to this structure). 

1812: 

1813: Unlike for $q$-ary reversible channels, the distribution of the 

1814: color $X_i$ received at vertex $i$, is not uniform and depends upon the 

1815: vertex. We shall denote it by $\varphi_i(x_i)\equiv \prob[X_i=x_i|\Tree]$.

1816: In fact $\varphi_i(\, \cdot\,)$ can be determined recursively:

1817: if $j$ is the parent of $i$, and $i_1$, $i_{k-1}$ the other vertices

1818: that share this parent, then

1819: %

1820: \begin{eqnarray}

1821: %

1822: \varphi_i(x_i) = \sum_{x_j}\sum_{x_{i_1},\dots,x_{i_{k-1}}}

1823: \pi^{(\alpha)}(x_i,x_{i_1},\dots,x_{i_{k-1}}|x_j)\,\varphi_j(x_j)\, .

1824: %

1825: \end{eqnarray}

1826: %

1827: 

1828: Let us consider the reconstruction problem.

1829: The conditional distribution of the transmitted color given the observation

1830: at generation $\ell$, $\prob[X_0=x|\uX_{\ell}=\ux_{\ell}]$ can be computed

1831: recursively proceeding from the leaves downwards to the root

1832: as in Sec.~\ref{se:merging}.

1833: In order to simplify the analysis, it is convenient to `factor out'

1834: the {\em a priori} information $\varphi_i(x_i)$, and define

1835: %

1836: \begin{eqnarray}

1837: %

1838: \eta_{i,\ell}(x) = \prob_{i}[X_i=x|\uX_{i,\ell}]\, .

1839: %

1840: \end{eqnarray}

1841: %

1842: where $\prob_{i}$ denotes probability with respect to a modified 

1843: process in which the boundary $ \uX_{i,\ell}$ is obtained from a broadcast

1844: starting from  $X_i$  chosen uniformly at random in 

1845: $\{1,\dots,q\}$. 

1846: Of course we have

1847: %

1848: \begin{eqnarray}

1849: %

1850:  \prob[X_i=x|\uX_{i,\ell}] = \frac{1}{z_i}\, \varphi_i(x)\, 

1851: \eta_{i,\ell}(x)\, ,

1852: %

1853: \end{eqnarray}

1854: %

1855: where $z_i \equiv \sum_x \varphi_i(x)\eta_{i,\ell}(x)$ ensures the 

1856: correct normalization.

1857: Notice that $\eta_{i,\ell}(x)$ depends uniquely on the portion 

1858: of the tree above $i$, more precisely on $\Tree(i,\ell)$ and $\uX_{i,\ell}$.

1859: 

1860: With a slight abuse of notation, let us denote by $\pi^{(a)}=\pi^{\alpha(a)}$, 

1861: $a \in\{ 1,\dots, l_i\}$ be the channels whose input is $x_i$,

1862: and by $j_1,\dots,j_{k(a)}$ the corresponding output vertices.

1863: It is easy to derive the following recursion

1864: which generalizes Eq.~(\ref{eq:Site})

1865: %

1866: \begin{eqnarray}

1867: %

1868: \eta_{i,\ell+1}(x) = \frac{1}{z(\{\eta_{j,\ell}\},\{\pi^{(a)}\})}

1869: \,\prod_{a=1}^{l_i}\sum_{x_{1},\dots x_{k(a)}}

1870: \pi^{(a)}(x_{1},\dots x_{k(a)}|x)\;

1871: \eta_{j_1,\ell}(x_1)\cdots\eta_{j_{k(a)},\ell}(x_{k(a)})\, ,

1872: %

1873: \end{eqnarray}

1874: %

1875: where $\ell$ is the distance from the leaves.

1876: The constant $z(\{\eta_{j,\ell}\},\{\pi^{(a)}\})$ is defined

1877: by the constraint $\sum_{x}\eta_{i,\ell+1}(x) =1$. 

1878: We shall denote the above mapping synthetically by writing

1879: $\eta_{i,\ell+1} = \F(\{\eta_{j,\ell}\},\{\pi^{(a)}\})$.

1880: 

1881: One can define two types of probability distributions 

1882: associated to $\eta_{i,\ell}$. We first assume that  the

1883: tree $\Tree$ is given, and  

1884: consider the distribution of $\eta_{i,\ell}$ conditional to $X_i=x$

1885: and $\Tree$. This will be denoted by $Q^{(i,\ell)}_x(\eta)$.

1886: Arguing as in the case of regular trees, one derives the recursion

1887: %

1888: \begin{eqnarray}

1889: %

1890: Q^{(i,\ell+1)}_x(\eta) =\sum_{\{x_j\}} \prod_{a=1}^{l_i}

1891: \pi^{(a)}(x_{j_1},\dots x_{j_{k(a)}}|x)\;

1892: \int\;\delta [\eta- \F(\{\eta_{j}\},\{\pi^{(a)}\})]\;

1893: \prod_j \de Q^{(j,\ell)}_{x_j}(\eta_j)\, .\label{eq:GeneralIteration}

1894: %

1895: \end{eqnarray}

1896: %

1897: 

1898: Next, we consider the distribution of $\eta_{i,\ell}$

1899: {\em unconditional} of $\Tree$, which we denote by $Q^{(\ell)}_x(\eta)$.

1900: This can also be regarded as the expectation of the previous

1901: distribution: $Q^{(\ell)}_x(\eta) = \E_{\Tree} Q^{(i,\ell)}_x(\eta)$.

1902: Notice that  $Q^{(i,\ell)}_{x}(\eta)$ depends on 

1903: $\Tree$ only through the first $\ell$ generations of the subtree rooted at $i$

1904: $\Tree(i,\ell)$. Since the structures of distinct subtrees are

1905: independent, if we average Eq.~(\ref{eq:GeneralIteration})

1906: with respect to $\Tree$, the averages factorize, yielding

1907: a recursion equation for $Q^{(\ell)}_x(\eta)$:

1908: %

1909: \begin{eqnarray}

1910: %

1911: Q^{(\ell+1)}_x(\eta) = \E\sum_{\{x_j\}} \prod_{a=1}^{l_i}

1912: \pi^{(a)}(x_{j_1},\dots x_{j_{k(a)}}|x)\;

1913: \int\;\delta [\eta- \F(\{\eta_{j}\},\{\pi^{(a)}\})]\;

1914: \prod_j \de Q^{(\ell)}_{x_j}(\eta_j)\, .\label{eq:GeneralIterationSimpl}

1915: %

1916: \end{eqnarray}

1917: %

1918: Here $\E$ denotes expectation with respect to the degree $l_i$ and the

1919: channel types. The last expression is particularly convenient for 

1920: numerical simulations and it is not more complex than the iteration 

1921: (\ref{eq:Iteration}) studied in the previous Sections.

1922: 

1923: We can also consider the distributions unconditional to

1924: the transmitted color: $\Q^{(i,\ell)}(\eta)$ and $\Q^{(\ell)}(\eta)$.

1925: The same relation as for regular trees

1926: hold in this case $Q^{(i,\ell)}_x(\eta) = q\eta(x) \Q^{(i,\ell)}(\eta)$

1927: and $Q^{(\ell)}_x(\eta) = q\eta(x) \Q^{(\ell)}(\eta)$. It is easy to

1928: derive the corresponding recursions. We just

1929: write down the equation for the last (non-random) distribution

1930: %

1931: \begin{eqnarray}

1932: %

1933: \Q^{(\ell+1)}(\eta) = \E\;

1934: \int\;\frac{z(\{\eta_j\}\{\pi^{(a)}\})}{z(\{\eo\}\{\pi^{(a)}\})}\;\;

1935: \delta [\eta- \F(\{\eta_{j}\},\{\pi^{(a)}\})]\;

1936: \prod_j \de \Q^{(\ell)}(\eta_j)\, .\label{eq:SymmetricGeneral}

1937: %

1938: \end{eqnarray}

1939: %

1940: 

1941: In order to discuss the correspondence with the dynamical glass transition 

1942: in this more general setting, it is necessary to distinguish two cases. 

1943: In the simplest one, the RS cavity equations for the associated statistical

1944: mechanics model admit the solution $\eta_{i\to j}(x) =\eo(x)$

1945: for any directed link $i\to j$ in the graph. Under this hypothesis

1946: it is not hard to show  that Eq.~(\ref{eq:SymmetricGeneral}) is equivalent 

1947: (in the same sense discussed in Sec.~\ref{sec:Recursion}) to the

1948: $m=1$ 1RSB equation~\footnote{The expert will perhaps be surprised

1949: by this remark since it is usually said that the order parameter for

1950: such systems is a `measure over the space of distributions'. 

1951: However

1952: it turns out that, for $m=1$, the expectation of this measure

1953: satisfies an equation which is Eq.~(\ref{eq:SymmetricGeneral}).}

1954: 

1955: In the general case (i.e. if  $\eta_{i\to j}(x) =\eo(x)$ does not solve the 

1956: RS equations), the dynamical glass transition still corresponds to 

1957: the extremality of the free boundary measure on an infinite tree. 

1958: The last problem, however, cannot be formulated in the same framework as 

1959: described here. One can still write an equation of the form 

1960: (\ref{eq:GeneralIteration}), conditioned to the graph structure, as is usually

1961: done in statistical physics. But the average over the graph structure can only be 

1962: performed conditioning upon the value of $\eta_{i\to j}(\,\cdot\,)$

1963: in the RS solution~\cite{ReconstrNext}, and so the relation between the spin glass problem and the

1964: reconstruction problem unconditioned to the structure of the tree, is not as simple as before. We

1965: shall not enter these details here.  

1966: %

1967: %********************************************************************

1968: %

1969: \section{Conclusion and open problems}

1970: \label{sec:Conclusion}

1971: 

1972: The coincidence between the reconstruction threshold and the dynamical glass

1973: transition explored in this paper is interesting from several points

1974: of view.

1975: 

1976: First of all, it provides some  perspective for putting on a firmer

1977: basis the theory of glassy systems on locally tree-like graphs 

1978: developed in the last few years. 

1979: As a concrete example, notice that the population dynamics algorithm

1980: defined in Section \ref{sec:Ferro} presents two important 

1981: advantages with respect to the procedure adopted by statistical physicists. 

1982: First, in spin glass theory the usual

1983: prescription is to look for a solution of the fixed point

1984: equation (\ref{eq:1RSB_fixed}). This poses the problem of the initial

1985: condition: even if a given initial condition yields a trivial fixed point,

1986: this may not be the case for all the initial conditions. In the present 

1987: formulation the iteration is initialized with a very specific initial 

1988: condition, and one is guaranteed that, if it converges to a trivial fixed 

1989: point, no non-trivial fixed point exists.  

1990: Second, simulating Eq.~(\ref{eq:1RSB_fixed}) 

1991: requires a `reweighting' of the sample which is usually the trickier

1992: part of the calculation. No reweighting is needed in the new approach:

1993: Eq.~(\ref{eq:Iteration}) can be handled easily.

1994: 

1995: Also, it provides some indication on the correctness of simple 

1996: analytic and algorithmic approaches to these systems, 

1997: such as the replica symmetric cavity method, or the belief 

1998: propagation algorithm. While it has long been known that their 

1999: correctness should be related to a fast correlation decay in a

2000: model on a tree, a precise criterion has never been 

2001: formulated

2002: \footnote{A frequently used sufficient  criterion is the uniqueness of

2003: the Gibbs state on the infinite tree (see \cite{BanGam} and references 

2004: therein). As it emerges from our discussion, this criterion is often much stronger 

2005:  than needed. It is also interesting to recall that 

2006: Tatikonda and Jordan \cite{Tatikonda} first connected the convergence 

2007: properties of belief propagation to the extremality of the free boundary Gibbs 

2008: on a properly defined tree.}. 

2009: Ous work suggests that the extremality of  the associated Gibbs measure

2010: on a tree provides such a criterion.

2011: 

2012: More broadly, it illustrate some subtleties of the physics of

2013: the glass transition. It is well known that, in a 1RSB

2014: glass transition, point to point correlations (static scattering factors)

2015: do not present any diverging correlation length. This paper

2016: shows that such a length can be derived quite generally from

2017: point-to-set correlations. Indeed the definition considered here is 

2018: essentially equivalent to the one of Ref.~\cite{BoBi04}.

2019: It was shown in Ref.~\cite{MoSe05} that this length scale divergence 

2020: implies a lower bound on the time scale divergence.

2021: 

2022: Finally, statistical physics ideas can inspire new results

2023: on the original reconstruction problem. The most interesting such

2024: idea is, in our view, the complexity functional introduced in 

2025: Section \ref{se:varprinc}. Apart from being conceptually innovative

2026: with respect to classical techniques, it seems to provide by

2027: far the best rigorous quantitative estimates of the reconstruction 

2028: threshold, cf. Proposition \ref{propo:VariationalAnti}. 

2029: This is well illustrated by the results in Table

2030:  \ref{TableAnti}. It will certainly be interesting 

2031: to find a concrete interpretation of this object in terms of

2032: the original reconstruction problem. Also, the values of the channel parameter

2033: at which the asymptotic complexity $\Sigma(\Q^{(\infty)})$ vanish have

2034: a particularly important role in statistical mechanics, but did not

2035: find any role here.

2036: 

2037: There are several results that we have not been able to  prove rigorously.

2038: We already formulated one such results as Conjecture \ref{conj:Sigma},

2039: and proved it for a particular class of channel models as Proposition

2040: \ref{propo:VariationalAnti}. Another interesting fact which has emerged from our

2041: numerical simulations is the coincidence of the KS and reconstruction 

2042: thresholds when the number of colors is small.

2043: %

2044: \begin{conj}\label{conj:KS}

2045: Consider the reconstruction problem for the $k$-ary tree and the 

2046: ferromagnetic Potts channel ($q$-ary symmetric channel) with $q\le 4$,

2047: or the antiferromagnetic Potts channel with $q\le 3$

2048: Then, there exists a $k_{\rm max}\ge 30$ such that, if $k<k_{\rm max}$

2049: the reconstruction threshold coincides with the Kesten-Stigum threshold.

2050: \end{conj}

2051: %

2052: A stronger version of this conjecture would be to require the thesis 

2053: to be valid for {\em all} values of $k$ (i.e. to state 

2054: that $k_{\rm max} = \infty$). Although we didn't find any $k$ 

2055: contradicting this stronger version, this might of course be due to

2056: the limitation on the values of $k$ that we can  treat numerically.

2057: 

2058: Finally, let us single out the case of completely antiferromagnetic

2059: ($\varepsilon=1$) Potts channels:

2060: \begin{conj}\label{conj:Col}

2061: Let $k_*(q)$ be the maximum value of $k$ such that the free boundary 

2062: Gibbs measure for uniformly random proper colorings on the infinite $k$-ary

2063: tree is extremal. Then  $k_*(3) = 5$, $k_*(4)=8$, $k_*(5)=13$,

2064: $k_*(6)=17$.

2065: \end{conj}

2066: %

2067: %***********************************************************

2068: %

2069: \section*{Acknowledgments} 

2070: This work has been carried out mostly during our stay

2071: at the MSRI on the occasion of the program on ``Probability, Algorithms, and

2072: Statistical Physics''. We have benefited from numerous discussions,

2073: suggestions and comments from James Martin, Elchanan Mossel and Yuval Peres; 

2074: it is a great pleasure to thank them for these stimulating exchanges. 

2075: Useful discussions with Olivier Rivoire and Guilhem Semerjian

2076: are also gratefully acknowledged.

2077: 

2078: %

2079: %***********************************************************

2080: %

2081: \appendix 

2082: 

2083: \section{Variational principle for frustrated kernels}

2084: \label{app:Variational} 

2085: 

2086: 

2087: This appendix is devoted to the proof of Lemma \ref{lemma:Variational}.

2088: It is convenient to introduce some notations. If

2089: $A(x,y)$, $x,y\in\{1,\dots,q\}$ is a symmetric matrix and 

2090: $\eta_1(x)$, $\eta_2(x)$,  $x\in\{1,\dots,q\}$ are two vectors,

2091: we shall write $A\eta_1(x)\equiv\sum_y A(x,y)\eta_1(y)$ and

2092: $\eta_1 A\eta_2\equiv\sum_y A(x,y)\eta_1(x)\eta_2(y)$ .

2093: Furthermore, if $\Q$ is a distribution over $\M$ we will

2094: denote by $\T\Q$ the distribution obtained 

2095: by using  Eq.~(\ref{eq:1RSB_fixed}): $\T\Q$ is the left hand side

2096: of Eq.~(\ref{eq:1RSB_fixed}) when in the right hand side  $\Q^*$

2097: has been substituted by $\Q$.

2098:  Finally, given 

2099: $\eta_1,\eta_2\in\M$, we let 

2100: 

2101: %

2102: \begin{eqnarray}

2103: %

2104: \Delta(\eta_1,\eta_2) \equiv \left(\frac{\eta_1\pi\eta_2}{\eo\pi\eo}\right)

2105: \log\left(\frac{\eta_1\pi\eta_2}{\eo\pi\eo}\right)\, .

2106: %

2107: \end{eqnarray}

2108: %

2109: 

2110: We also write $\eta\ed \Q$ when $\eta$ has distribution $\Q$.

2111: We first derive two simple lemmas.

2112: %

2113: \begin{lemma}\label{remark1}

2114: %

2115: Let $\Q^*$ and $\Q$ be two consistent distributions over $\M$ and 

2116: $\Sigma^*(t) \equiv\Sigma((1-t)\Q^*+t\Q)$. Then

2117: %

2118: \begin{eqnarray}

2119: %

2120: -\frac{1}{(k+1)}\left.\frac{\de\Sigma^*}{\de t}\right|_0 = 

2121: \E\left\{\Delta(\nu,\eta_2')-\Delta(\nu,\eta_2)-\Delta(\eta_1,\eta_2')

2122: +\Delta(\eta_1,\eta_2)\right\}\, ,

2123: %

2124: \label{eq:der_result}

2125: \end{eqnarray}

2126: %

2127: where the expectation is taken with respect to the independent random variables

2128: $\eta_1,\eta_2 \ed \Q^*$, $\eta_2'\ed \T\Q^*$ and $\nu\ed \Q$. 

2129: %

2130: \end{lemma}

2131: %

2132: \prooft

2133: Elementary calculus yields

2134: %

2135: \begin{eqnarray}

2136: %

2137: -\frac{1}{(k+1)}\left.\frac{\de\Sigma^*}{\de t}\right|_0 =\psi(1)-\psi(0)\, ,

2138: %

2139: \label{eq:derivee}

2140: \end{eqnarray}

2141: %

2142: where

2143: %

2144: \begin{eqnarray}

2145: %

2146: \psi(t) & \equiv &-\E\left\{\left(\frac{\nu^t\pi\eta}{\eo\pi\eo}\right)

2147: \log\!\left(\frac{\nu^t\pi\eta}{\eo\pi\eo}\right)\right\}+\\

2148: && +\E\left\{ \left(\frac{\sum_x\pi\nu^t(x)\prod_{i=1}^k

2149: \pi\eta_i(x)}{\sum_x\prod_{i=0}^k \pi\eo(x)}\right)\log\!

2150: \left(\frac{\sum_x\pi\nu^t(x)\prod_{i=1}^k

2151: \pi\eta_i(x)}{\sum_x\prod_{i=0}^k \pi\eo(x)}\right)\right\}\, .\nonumber

2152: %

2153: \end{eqnarray}

2154: %

2155: Here $\eta,\eta_1,\dots,\eta_k\ed \Q^*$ and $\nu^t\ed (1-t)\Q^*+t\Q$ 

2156: are independent random variables. The first term, when integrated on $t$,

2157: gives the contribution $\E\left\{\Delta(\eta_1,\eta_2)-\Delta(\nu,\eta_2)\right\}$

2158: to (\ref{eq:der_result}).

2159: 

2160: As for the second term, 

2161: observing that 

2162: $\prod_{i=1}^k \pi\eta_i(x) = z(\{\eta_i\})\, \F(\eta_1,\dots,\eta_k)$,

2163: it  can be rewritten as

2164: %

2165: \begin{eqnarray}

2166: %

2167: q^{k-1}\E\left\{z(\{\eta_i\}) \left[ \frac{\nu^t\pi\F(\eta_1\dots\eta_k)}

2168: {\eo\pi\eo}\right]\log\left[ \frac{\nu^t\pi\F(\eta_1\dots\eta_k)}

2169: {\eo\pi\eo}\right] \right\}+\nonumber\\

2170: q^{k-1}\E\left\{z(\{\eta_i\}) \left[ \frac{\nu^t\pi\F(\eta_1\dots\eta_k)}

2171: {\eo\pi\eo}\right]\log\left[ q^{k-1}z(\{\eta_i\})\right] \right\}\, .

2172: %

2173: \end{eqnarray}

2174: %

2175: Since $\E\nu^t(x) =\eo(x)$, the second term is $t$-independent 

2176:  and does not contribute to (\ref{eq:derivee}). The first term is  equal

2177: to $\E\Delta(\nu^t,\eta'_2)$ where $\eta'_2\ed\T\Q^*$.

2178: %

2179: \endproof

2180: 

2181: %

2182: \begin{lemma}\label{remark2}

2183: %

2184: Let $\eta_1,\eta_2\in\M$ and define $\delta\eta_i(x) = \eta_i(x)-\eo(x)$.

2185: If $\pi(x,y)=\pi_*-\pih(x,y)$ is a frustrated kernel, then 

2186: %

2187: \begin{eqnarray}

2188: %

2189: |\delta\eta_1\pih\delta\eta_2|\le \eo\pi\eo\, .\label{eq:remark2}

2190: %

2191: \end{eqnarray}

2192: %

2193: \end{lemma}

2194: %

2195: \prooft

2196: %

2197: Since $\pih$ is positive definite, $\phi\pih\psi\equiv\sum_{x,y}

2198: \pih(x,y)\phi(x)\psi(y)$ is a well defined scalar product.

2199: Cauchy-Schwarz inequality implies

2200: %

2201: \begin{eqnarray}

2202: %

2203: |\delta\eta_1\pih\delta\eta_2|\le\sqrt{(\delta\eta_1\pih\delta\eta_1)

2204: (\delta\eta_2\pih\delta\eta_2)}\le \max\left\{(\delta\eta_1\pih\delta\eta_1),

2205: (\delta\eta_2\pih\delta\eta_2)\right\}\,. 

2206: %

2207: \end{eqnarray}

2208: %

2209: Therefore it is sufficient to prove Eq.~(\ref{eq:remark2}) 

2210: for $\delta\eta_1=\delta\eta_2=\delta\eta$. Let $\eta(x) = \eo(x)+

2211: \delta\eta(x)$. Since $\pi(x,y),\eta(x)\ge 0$, 

2212: and $\sum_x\delta\eta(x) = 0$, we have

2213: %

2214: \begin{eqnarray}

2215: %

2216: 0\le\eta\pi\eta = \eo\pi\eo+\delta\eta\pi\delta\eta=\eo\pi\eo-

2217: \delta\eta\pih\delta\eta\, .

2218: %

2219: \end{eqnarray}

2220: %

2221: \endproof

2222: %

2223: 

2224: We can now turn to the proof of Lemma \ref{lemma:Variational}. In the 

2225: following, given $\eta\in\M$, we define $\delta\eta(x)\equiv

2226: \eta(x)-\eo(x)$. Obviously we have

2227: %

2228: \begin{eqnarray}

2229: %

2230: \Delta(\eta_1,\eta_2) = \left(1-\frac{\delta\eta_1\pih\delta\eta_2}{\eo\pi\eo}\right)

2231: \log\!\left(1-\frac{\delta\eta_1\pih\delta\eta_2}{\eo\pi\eo}\right)\, .

2232: %

2233: \end{eqnarray}

2234: %

2235: Because of Lemma \ref{remark2}  we can expand this expression in an 

2236: absolutely convergent series

2237: %

2238: \begin{eqnarray}

2239: %

2240: \Delta(\eta_1,\eta_2) = -\frac{\delta\eta_1\pih\delta\eta_2}{\eo\pi\eo}

2241: +\sum_{n=2}^{\infty} C_n\, 

2242: \left(\frac{\delta\eta_1\pih\delta\eta_2}{\eo\pi\eo}\right)^n\, ,

2243: %

2244: \end{eqnarray}

2245: %

2246: where $C_n\equiv 1/n(n-1)>0$. If $\eta_1\ed \Q_1$ and $\eta_2\ed\Q_2$ are 

2247: independent random variables with consistent distributions, we get

2248: %

2249: \begin{eqnarray}

2250: %

2251: \E\Delta(\eta_1,\eta_2) = \sum_{n=2}^{\infty} C_n\,q^n

2252: \phi_1^{(n)}\pih^{\otimes n}\phi_2^{(n)}\, ,

2253: \label{eq:Expansion}

2254: %

2255: \end{eqnarray}

2256: %

2257: where $\pih^{\otimes n}(x_1\dots x_n;y_1\dots y_n) \equiv \pih(x_1,y_1)

2258: \cdots \pih(x_n,y_n)$ is the $n$-fold

2259: tensor product of $\pih$, $\phi_a^{(n)}(x_1\dots x_n)

2260: \equiv \E\{\delta\eta_a(x_1)\cdots \delta\eta_a(x_n)\}$ 

2261: are the moments of the distribution $\Q_a$, and

2262: %

2263: \begin{eqnarray}

2264: %

2265: \phi_1^{(n)}\pih^{\otimes n}\phi_2^{(n)} \equiv \sum_{x_1\dots x_n}

2266: \sum_{x_1\dots x_n} \pih^{\otimes n}(x_1\dots x_n;y_1\dots y_n)

2267: \phi_1^{(n)}(x_1\dots x_n)\phi_2^{(n)}(x_1\dots x_n)\, .

2268: %

2269: \end{eqnarray}

2270: %

2271: 

2272: Now consider Remark \ref{remark1} and take $\Q = \T\Q^*$. We get 

2273: %

2274: \begin{eqnarray}

2275: %

2276: -\frac{1}{(k+1)}\left.\frac{\de\Sigma^*}{\de t}\right|_0 = 

2277: \E\left\{\Delta(\eta_1',\eta_2')-\Delta(\eta_1',\eta_2)-\Delta(\eta_1,\eta_2')

2278: +\Delta(\eta_1,\eta_2)\right\}\, ,

2279: %

2280: \end{eqnarray}

2281: %

2282: where $\eta_1,\eta_2\ed\Q^*$ and $\eta_1',\eta_2'\ed\T\Q^*$. Applying

2283: Eq.~(\ref{eq:Expansion}) we get 

2284: %

2285: \begin{eqnarray}

2286: %

2287: -\frac{1}{(k+1)}\left.\frac{\de\Sigma^*}{\de t}\right|_0 = 

2288: \sum_{n=2}^{\infty} C_n\,q^n

2289: (\phi_{\T}^{(n)}-\phi^{(n)})\pih^{\otimes n}

2290: (\phi_{\T}^{(n)}-\phi^{(n)})\, ,\label{eq:LastProofVariational}

2291: %

2292: \end{eqnarray}

2293: %

2294: where $\phi^{(n)}$ and $\phi^{(n)}_{\T}$ denote the moments (respectively)

2295: of $\Q^*$ and $\T\Q^*$. Since $\pih$ is positive definite,

2296:  $\pih^{\otimes n}$ is positive definite as well and therefore the

2297: right hand side is a sum of non-negative terms. In order for this right hand side

2298: to vanish, each of the terms must vanish, which implies

2299: $\phi_{\T}^{(n)}=\phi^{(n)}$ for each $n$. But, since $\Q^*$ and 

2300: $\T\Q^*$ have bounded support, this implies $\Q^* = \T\Q^*$, which

2301: is false by hypothesis. Therefore the right hand side of 

2302: Eq.~(\ref{eq:LastProofVariational}) is strictly positive and 

2303: $\left.\frac{\de\Sigma^*}{\de t}\right|_0 <0$ as desired. \endproof

2304: %

2305: %***********************************************************

2306: %

2307: \bibliographystyle{alpha}

2308: %\bibliography{bookref}

2309: 

2310: \newcommand{\etalchar}[1]{$^{#1}$}

2311: \begin{thebibliography}{MPWZ02}

2312: 

2313: \bibitem[AS04]{AldousSteele}

2314: David Aldous and J.~Michael Steele.

2315: \newblock The {O}bjective {M}ethod: {P}robabilistic {C}ombinatorial

2316:   {O}ptimization and {L}ocal {W}eak {C}onvergence.

2317: \newblock In H.~Kesten, editor, {\em Probability on discrete structures}.

2318:   Springer, New York, 2004.

2319: 

2320: \bibitem[BG05]{BanGam}

2321: Antar Bandyopadhyay and David Gamarnik.

2322: \newblock Counting without sampling. New algorithms for enumeration problems using statistical physics.

2323: \newblock {\tt arXiv:math.PR/0510471}

2324: 

2325: \bibitem[BB04]{BoBi04}

2326: Jean-Philippe Bouchaud and Giulio Biroli.

2327: \newblock On the {Adam-Gibbs-Kirkpatrick-Thirumalai-Wolynes} scenario for the

2328:   viscosity increase in glasses.

2329: \newblock {\em J.~Chem.~Phys.}, 121:7347--7354, 2004.

2330: 

2331: \bibitem[BMP{\etalchar{+}}03]{Brauenstein03}

2332: Alfredo Braunstein, Roberto Mulet, Andrea Pagnani, Martin Weigt, and Riccardo

2333:   Zecchina.

2334: \newblock Polynomial iterative algorithms for coloring and analyzing random

2335:   graphs.

2336: \newblock {\em Phys.~Rev.~E}, 68:036702, 2003.

2337: 

2338: \bibitem[BRZ95]{BleherRuizZagrebnov}

2339: Pavel~M. Bleher, Jean Ruiz, and Valentin~A. Zagrebnov.

2340: \newblock On the purity of limiting {G}ibbs state for the {I}sing model on the

2341:   {B}ethe lattice.

2342: \newblock {\em J. Stat. Phys.}, 79:473--482, 1995.

2343: 

2344: \bibitem[BW03]{Winkler03}

2345: Grahm~R. Brightwell and Peter Winkler.

2346: \newblock Gibbs extremality for the hard-core model on a {B}ethe lattice.

2347: \newblock {\em submitted}, 1, 2003.

2348: 

2349: \bibitem[CCST86]{Chayes86}

2350: Jennifer~T. Chayes, Lincoln Chayes, James.~P. Sethna, and David.~J. Thouless.

2351: \newblock A {M}ean {F}ield {S}pin {G}lass with {S}hort-{R}ange {I}nteractions.

2352: \newblock {\em Commun.~Math.~Phys.}, 106:41--89, 1986.

2353: 

2354: \bibitem[CT91]{CoverThomas}

2355: Thomas~M. Cover and Joy~A. Thomas.

2356: \newblock {\em Elements of Information Theory}.

2357: \newblock Wiley Interscience (?), New York, 1991.

2358: 

2359: \bibitem[EKPS00]{EvaKenPerSch}

2360: William Evans, Claire Kenyon, Yuval Peres, and Leonard~J. Schulman.

2361: \newblock Broadcasting on trees and the {I}sing model.

2362: \newblock {\em Ann. Appl. Probab.}, 10:410--433, 2000.

2363: 

2364: \bibitem[FP84]{Peruggi83}

2365: Gabriella~Monroy Fulvio~Peruggi, Francesco~di~Liberto.

2366: \newblock The potts model on bethe lattices. i. general results.

2367: \newblock {\em J.~Math.~Phys.}, 16:811--817, 1984.

2368: 

2369: \bibitem[Geo88]{Georgii}

2370: Hans-Otto Georgii.

2371: \newblock {\em {G}ibbs {M}easures and {P}hase {T}ransition}.

2372: \newblock de Gruyter, Berlin, 1988.

2373: 

2374: \bibitem[KS66a]{KestenStigum2}

2375: Harry Kesten and Bernt~P. Stigum.

2376: \newblock Additional limit theorems for decomposable multi-dimesional

2377:   {G}alton-{W}atson processes.

2378: \newblock {\em Ann. Math. Statist.}, 37:1463--1481, 1966.

2379: 

2380: \bibitem[KS66b]{KestenStigum1}

2381: Harry Kesten and Bernt~P. Stigum.

2382: \newblock Limit theorems for decomposable multi-dimesional {G}alton-{W}atson

2383:   processes.

2384: \newblock {\em J. Math. Anal. Appl.}, 17:309--338, 1966.

2385: 

2386: \bibitem[Mar03]{Martin03}

2387: James B. Martin

2388: \newblock Reconstruction Thresholds on Regualr Trees

2389: \newblock {\em Discrete Mathematics and Theoretical Computer Science }{\bf AC}, 2003, 191-204

2390: 

2391: \bibitem[MM05]{ReconstrNext}

2392: Marc M\'ezard and Andrea Montanari.

2393: \newblock Unpublished, 2005.

2394: 

2395: \bibitem[Mos98]{MosselRec}

2396: Elchanan Mossel.

2397: \newblock Recursive reconstruction on periodic trees.

2398: \newblock {\em Rand.~Struct.~Alg.}, 13:81--97, 1998.

2399: 

2400: \bibitem[Mos01]{Mossel01}

2401: Elchanan Mossel.

2402: \newblock Reconstruction on trees: beating the second eigenvalue.

2403: \newblock {\em Ann. Appl. Probab.}, 11:285--300, 2001.

2404: 

2405: \bibitem[Mos04]{Mossel03}

2406: Elchanan Mossel.

2407: \newblock Survey: {I}nformation flows on trees.

2408: \newblock In J.~Nestril and P.~Winkler, editors, {\em Graphs, Morphisms and

2409:   Statistical Physics. DIMACS Series in Discrete Mathematics and Theoretical

2410:   Computer Science}. American Mathematical Society, 2004.

2411: 

2412: \bibitem[MP01]{MP_Bethe}

2413: Marc M\'ezard and Giorgio Parisi.

2414: \newblock The {B}ethe lattice {S}pin glass revisited.

2415: \newblock {\em Eur. Phys. J. B}, 20:217--233, 2001.

2416: 

2417: \bibitem[MP03a]{MP_Bethe_T0}

2418: Marc M{\'e}zard and Giorgio Parisi.

2419: \newblock The cavity method at zero temperature.

2420: \newblock {\em J.~Stat.~Phys.}, 111:1--34, 2003.

2421: 

2422: \bibitem[MP03b]{MosselPeres01}

2423: Elchanan Mossel and Yuval Peres.

2424: \newblock Information flow on trees.

2425: \newblock {\em Ann. Appl. Probab.}, 13:817--844, 2003.

2426: 

2427: \bibitem[MPWZ02]{Mulet02}

2428: Roberto Mulet, Andrea Pagnani, Martin Weigt, and Riccardo Zecchina.

2429: \newblock Coloring random graphs.

2430: \newblock {\em Phys.~Rev.~Lett.}, 89:268701, 2002.

2431: 

2432: \bibitem[MS05a]{MoSe05}

2433: Andrea Montanari and Guilhem Semerjian.

2434: \newblock On the dynamics of the glass transition on {B}ethe lattices.

2435: \newblock {\em {\tt cond-mat/0509366}}, 2005.

2436: 

2437: \bibitem[MS05b]{MosselSteel05}

2438: Elchanan Mossel and Mike Steel.

2439: \newblock How much can evolved characters tell us about the tree that generated

2440:   them?

2441: \newblock In O.~Gascuel, editor, {\em Mathematics of Evolution and Phylogeny}.

2442:   Oxford University Press, 2005.

2443: 

2444: \bibitem[Per83]{Peruggi84b}

2445: Fulvio Peruggi.

2446: \newblock Probability measures and {H}amiltonian models on {B}ethe lattices:

2447:   {II}. {T}he solution of thermal and configurational problems.

2448: \newblock {\em J.~Phys.~A:~Math.~Gen.}, 25:3316--3323, 1983.

2449: 

2450: \bibitem[Per84]{Peruggi84a}

2451: Fulvio Peruggi.

2452: \newblock Probability measures and {H}amiltonian models on {B}ethe lattices:

2453:   {I}. {P}roperties and construction of {MRT} probability measures.

2454: \newblock {\em J.~Math.~Phys.}, 25:3303--3315, 1984.

2455: 

2456: \bibitem[TACA73]{Abou-Chacra}

2457: David~J.~Thouless R.~Abou-Chacra and Philip~W. Anderson.

2458: \newblock A selfconsistent theory of localization.

2459: \newblock {\em J.~Phys.~C}, 6:1734--1752, 1973.

2460: 

2461: \bibitem[Shi96]{Shiryaev}

2462: Albert~N. Shiryaev.

2463: \newblock {\em {P}robability}.

2464: \newblock Springer, New York, 1996.

2465: 

2466: \bibitem[TJ02]{Tatikonda}

2467: Sekhar Tatikonda and Michael I. Jordan 

2468: \newblock {\em Loopy Belief Propagation and Gibbs Measures}.

2469: \newblock 18th Conference on Uncertainty in Artificial Intelligence, August 

2470: 2002 (Edmonton, Canada) 

2471: 

2472: \end{thebibliography}

2473: 

2474:   

2475: %\end{thebibliography}

2476: 

2477: 

2478: \end{document}

2479: