1: \documentclass[pre,epsfig,aps]{revtex4}

2: \usepackage{epsfig}

3: \usepackage{fancyheadings}

4: \usepackage{amstext,amsmath,amssymb}

5: 

6: \begin{document}

7: \title{Introduction to Phase Transitions in Random Optimization Problems}

8: \author{R. Monasson}

9: \address{Laboratoire de Physique Th\'eorique de l'ENS\\ 24 rue

10:   Lhomond, 75005 Paris, France}

11: \begin{abstract}

12: Notes of the lectures delivered in Les Houches during

13: the Summer School on Complex Systems (July 2006). 

14: \end{abstract}

15: \maketitle

16: 

17: \section{Introduction}

18: 

19: \subsection{Preamble}

20: 

21: The connection between the statistical physics of disordered systems

22:  and optimization problems

23: in computer science dates back from twenty years at least

24: \cite{Me87}. After all zero temperature statistical physics is

25: simply the search for the state with minimal energy, while

26: the main problem in combinatorial optimization is to look for the

27: configurations of parameters minimizing some cost function (the length

28: of a tour in the traveling salesman problem (TSP), the number of violated

29: constraints in constrained satisfaction problems, ...) \cite{Pa83}.

30: Yet, despite the beautiful studies of the average

31:  properties of the TSP, Graph partitioning, Matching, ..., based on 

32: the recently developed  mean-field spin glass theory \cite{Me87},  

33: a methodological gap between the fields could not be bridged\cite{Fu85}. 

34: In statistical physics statements are usually made on the 

35: properties of samples given some quenched disorder

36: distribution  such as the typical number of solutions, minimal

37: energy ...

38: In optimization, however, one is interested in solving one (or

39: several) particular instances of a problem, and needs efficient

40: ways to do so, that is, requiring a computational effort growing

41: not too quickly with the number of data defining the instance.

42: Knowing precisely the typical properties for a given, 

43: academic distribution of instances does not help much to

44: solve practical cases. 

45:  

46: 

47: At the beginning of the nineties practitionners in artificial

48: intelligence realized that classes of random constrained satisfaction problems

49: used as artificial benchmarks for search algorithms exhibited abrupt

50: changes of behaviour when some control parameter were finely tuned

51: \cite{Mi92}. The most celebrated example was random K-Satisfiability, 

52: where one looks for a solution to a set of random logical constraints

53: over a set of Boolean variables. It appeared that, for large sets of

54: variables,  there was a critical

55: value of the number of constraints per variable below which there

56: almost surely existed solutions, and above which solutions were

57: absent. An important feature was that search algorithms performances

58: drastically worsened in the vicinity of this critical ratio.

59: 

60: This phenomenon, strongly reminiscent of phase transitions in

61: condensed matter physics, led to a revival of the

62: interface between statistical physics and computer science, which has

63: not vanished yet. The purpose of the present lecture is to introduce

64: the non specialist reader to the concepts and techniques required to

65: understand the literature in the field. For the sake of simplicity 

66: the presentation will be limited to one 

67: computational problem, namely, linear systems of Boolean equations. 

68: A good reason to do so is that this problem concentrates most of the features 

69: encountered in other optimization problems, while being 

70: technically simpler to study. In addition it is closely related to

71: error-correcting codes in communication theory, see lectures by

72: A. Montanari and R. Urbanke in the present book.

73: Extension to other problems will be mentioned in the conclusions.

74: 

75: The lecture is divided into three parts. Sections 1 and 2 are

76: devoted to the presentation of the model and of elementary concepts 

77: related to phase transitions e.g. finite-size scaling, large

78: deviations, critical exponents, symmetry breaking, ... Sections 3

79: and 4 expose the specific statistical mechanics techniques and concepts

80: developed in disordered systems to deal with highly interacting and

81: random systems, namely the replica and cavity approaches. Finally

82: Section 5 focuses on dynamics and the study of search

83: algorithms.

84: 

85: 

86: \subsection{Linear systems of Boolean equations}

87: 

88: Linear systems of Boolean equations look very much like their

89: well known counterparts for integer-valued variables, except that

90: equalities are defined modulo two. 

91: Consider a set of $N$ Boolean variables $x_i$ with indices $i=1,\ldots

92: ,N$. Any variable shall be False (F) or True (T). The sum of two variables,

93: denoted by $+$, corresponds to the logical exclusive OR between these

94: variables defined through,

95: \begin{eqnarray} \label{sumrule}

96: F + T &=& T + F = T \quad , \nonumber \\ 

97: F + F &=& T + T = F  \quad .

98: \end{eqnarray}

99: In the following we shall use an alternative representation of the above 

100: sum rule. Variables will be equal to 0 or 1, instead of $F$ or $T$ 

101: respectively. Then the $+$ operation 

102: coincides with the addition between integer numbers modulo two. 

103: 

104: The following is a linear equation involving three variables,

105: \begin{equation} \label{xoreq01}

106: x_1 + x_2 + x_3 = 1 \quad .

107: \end{equation}  

108: Four among the $2^3=8$ assignments of $(x_1,x_2,x_3)$ satisfy 

109: the equation: $(1,0,0)$, $(0,1,0)$, $(0,0,1)$ and $(1,1,1)$.

110: A Boolean system of equations is a set of Boolean equations

111: that have to be satisfied together. For instance,

112: the following Boolean system involving four variables 

113: \begin{equation} \label{xoreq02}

114: \left\{ \begin{array} {l}

115: x_1 + x_2 + x_3 = 1 \\

116: x_2 + x_4  = 0 \\

117: x_1 + x_4 = 1 

118: \end{array} \right.

119: \end{equation}  

120: has two solutions: $(x_1,x_2,x_3,x_4)=(1,0,0,0)$ and $(0,1,0,1)$. 

121: A system with one or more solutions is called satisfiable. A trivial

122: example of an unsatisfiable Boolean system is

123: \begin{equation} \label{xoreq03}

124: \left\{ \begin{array} {l}

125: x_1 + x_2 + x_3 = 1 \\x_1 + x_2 + x_3 = 0 

126: \end{array} \right. \qquad .

127: \end{equation}  

128: Determining whether a Boolean system admits 

129: an assignment of the Boolean variables satisfying all the

130: equations constitutes the XORSAT 

131: (exclusive OR Satisfaction) problem. In the following,

132: we shall restrict for some reasons to be clarified in Section~\ref{secbasic}

133: to K-XORSAT, a variant of XORSAT where each Boolean equation include $K$

134: variables precisely.

135: 

136: K-XORSAT belongs to the class P of polynomial problems \cite{Pa83}. 

137: Determining whether a system is satisfiable or not 

138: can be achieved by the standard Gaussian 

139: elimination algorithm in a time (number of elementary

140: operations) bounded from above by some constant times 

141: the cube of the number of bits necessary to store the

142: system\footnote{The storage space is $K$ times the number of equations

143:   times the number of bits necessary to label a variable, that is, the

144: logarithm of the number of variables appearing in the system.}\cite{Pa83}. 

145: 

146: If the decision version of K-XORSAT is easy its optimization version

147: is not. Assume you are given a system $F$, run the Gauss procedure and

148: find that it is not satisfiable. Determining the maximal number

149: $M_S(F)$ of satisfiable equations is a very hard problem. Even 

150: approximating this number is very hard. It is known that there is no

151: approximation algorithm (unless P=NP) for XORSAT with ratio $r>\frac

152: 12$, that is, guaranteed to satisfy at least $r\times M_S(F)$ 

153: equations for any $F$. But $r=\frac 12$ is achieved, on average, by

154: making a random guess\footnote{Any equation is satisfied by half of

155:   the configurations of a variables, so a randomly chosen

156:   configuration satisfies on average $\frac M2 \ge \frac {M_S(F)}2$

157: equations.}! 

158: 

159: \subsection{Models for random systems}\label{secmodel}

160: 

161: There are many different ways of generating random Boolean systems.

162: Perhaps the simplest one is the following, called

163: {\em fixed-size ensemble}. To build an equation we pick up 

164: uniformly at random $K$ distinct indices among the $N$ ones, say,

165: ${i_1},{i_2}$ and  ${i_k}$. Then we consider the equation

166: \begin{equation}

167: x_{i_1} + x_{i_2} + \ldots + x_{i_k} = v \ .

168: \end{equation}

169: The second member, $v$, is obtained by tossing a coin: 

170: $v=0$ or $v=1$ with equal probabilities (one half) and independently of the 

171: indices of the variables in the first member. 

172: The process is repeated $M$ times, without correlation between 

173: equations to obtain a system with $M$ equations.

174: 

175: Another statistical ensemble is the {\em fixed-probability ensemble}.

176: One scans the set of all $H=2 {N\choose K}$  equations one after the other.

177: Each equation is added to the system with probability $p$, discarded with

178: probability $1-p$. Then a system with, on average, $p\,H$ equations

179: (without repetition) is obtained. In practice one chooses $p=\frac MH$ to

180: have the same (average) number of equations as in the fixed-size 

181: ensemble.

182: 

183: The above distributions are not the only possible

184: ones. However they are easy to implement on a computer, are 

185: amenable to mathematical studies, and last but not least, 

186: lead to a surprisingly rich phenomenology. One of the key 

187: quantities which exhibits an interesting behaviour is

188: \begin{eqnarray}

189: P_{SAT} (N,\alpha) &=& \mbox{Probability that a system of random

190:   K-XORSAT with} \nonumber \\

191: && \mbox{ $N$ variables and $M=\alpha\, N$ equations is satisfiable}

192:  \nonumber \ ,

193: \end{eqnarray}

194: which obviously depends on $K$ and the statistical ensemble.

195: Given $N$ $P_{SAT}$ is a decreasing function of $\alpha$.  

196: We will see that, in the infinite size limit (and for $K\ge 2$), 

197: the decrease is abrupt

198: at some well defined ratio, defining a phase transition between Satisfiable

199: and Unsatisfiable phase \cite{Cr99}. 

200: The scope of the lecture is to give some tools

201: to understand this transition and some related phenomena.

202: 

203: % basics

204: 

205: \section{Basic concepts: overview of static phase transitions in K-XORSAT} 

206: \label{secbasic}

207: 

208: In this Section we introduce the basic concepts necessary to the

209: study of random K-XORSAT. It turns out that even the $K=1$ case,

210: trivial from a computer science point of view (each equation

211: contains a single variable!), can be used as an illustration to

212: important concepts such as scaling and self-averageness. Ideas 

213: related to the percolation phase transition and random graphs are

214: illustrated on the $K=2$ case. Finally the

215: solution space of 3-XORSAT model exemplifies the notion of clusters

216: and glassy states.

217: 

218: \subsection{Finite-size scaling (I): scaling function}\label{secsf1}

219: 

220: Figure \ref{proba1-fig}(left) shows the probability $P_{SAT}$ that a randomly

221: extracted 1-XORSAT formula is satisfiable as a function of the ratio

222: $\alpha$, and for sizes $N$ ranging from 100 to 1000. We see that

223: $P_{SAT}$ is a decreasing function of $\alpha$ and $N$. 

224: 

225: Consider the subformula made of the $n_i$ equations with first member equal to 

226: $x_i$. This formula is always satisfiable if $n_i=0$ or $n_i=1$. If

227: $n_i\ge 2$ the formula is

228: satisfiable if and only if all second members are equal (to 0, or to

229: 1), an event with probability $(\frac 12)^{n_i-1}$ 

230: decreasing exponentially with the number

231: of equations. Hence we have to consider the following variant of the celebrated

232: Birthday problem\footnote{The Birthday problem is a classical

233: elementary  probability problem: given a class with $M$ students, what is the

234:   probability that at least two of them have the same birthday date?

235:   The answer for $M=25$ is $p\simeq 57\%$, while a much lower value is

236:   expected on intuitive grounds when $M$ is much smaller than the number

237: $N=365$   of days in a year.}. Consider a year with a number $N$

238: of days, how should scale the number $M$ of students in a 

239: class to be sure that no two students have the same birthday date?

240: \begin{equation}

241: \bar p =\prod_{i=0}^{M-1}\left(1-\frac iN\right) = \exp

242: \left(-\frac{M(M-1)}{2N} + O(M^3/N^2) \right) \ .

243: \end{equation}

244: Hence we expect a cross-over from large to small $\bar p$ when $M$

245: crosses the scaling regime $\sqrt N$. Going back to the 1-XORSAT model

246: we expect $P_{SAT}$ to have a non zero limit value when the number of

247: equations and variables are both sent to infinity at a fixed ratio

248: $y=M/\sqrt N$. In other words, random 1-XORSAT formulas with $N$

249: variables, $M$ equations or with, say, $100\times N$ variables,

250: $10\times M$ equations should have roughly the same probabilities of 

251: being satisifiable. To check this hypothesis we replot the data in 

252: Figure~\ref{proba1-fig} after multiplication of the abscissa of each

253: point by $\sqrt N$ (to keep $y$ fixed instead of $\alpha$). The

254: outcome is shown in the right panel of Figure~\ref{proba1-fig}. Data

255: obtained for various sizes nicely collapse on a single limit curve

256: function of $y$.

257: 

258: The calculation of this limit function, usually called scaling

259: function, is done hereafter in the fixed-probability

260: 1-XORSAT model where the number of equations

261: is a Poisson variable of mean value $\bar M=y\sqrt N$.  We will

262: discuss  the equivalence between the fixed-probability

263: and the fixed-size ensembles later.

264: In the fixed-probability ensemble the numbers $n_i$ of occurence of each

265: variable $x_i$ are independent Poisson variables with average value

266: $\bar M/N=y/\sqrt N$. Therefore the probability of satisfaction is 

267: \begin{eqnarray} \label{psatp}

268: P_{SAT} ^{p} (N,\alpha=\frac y{\sqrt N}) 

269: &=& \left[  e^{- y/\sqrt N} \left( 1+ \sum _{n\ge 1}

270: \frac {(y/\sqrt N)^n}{n!} \left(\frac 12\right)^{n-1}\right) \right] ^N  

271: \nonumber \\ &=& \left[ 2 e^{-y/(2\sqrt N)}-e^{-y/\sqrt N}\right]^N  \ , 

272: \end{eqnarray}

273: where the $p$ subscript denotes the use of the fixed-probability ensemble.

274: We obtain the desired  scaling function

275: \begin{equation} \label{sca1}

276: \Phi _1(y)\equiv \lim _{N\to\infty} \ln P^p_{SAT} (N,\alpha=\frac y{\sqrt

277:   N}) = -\frac{y^2}4 \ ,

278: \end{equation}

279: in excellent agreement with the rescaled data of

280: Figure~\ref{proba1-fig} (right) \cite{Cr03a}. 

281:     

282: \begin{figure}[t]

283: \includegraphics*[width=5.truecm,angle=0]{fig211.eps}

284: \includegraphics*[width=5.truecm,angle=0]{fig212.eps}

285: \caption{Left: Probability that a random 1-XORSAT formula is satisfiable as

286:   a function of the ratio $\alpha$ of equations per variable, and for

287:   various sizes $N$. Right: same data as in the left panel after the

288: horizontal rescaling $\alpha \to \alpha \times \sqrt N = y$; note the use of

289:   a log scale for the vertical axis. The dashed line

290:   shows the scaling function $\Phi_1(y)$ (\ref{sca1}).}

291: \label{proba1-fig}

292: \end{figure}

293: 

294: \subsection{Self-averageness of energy and entropy}\label{secselfav}

295: 

296: Let us now consider random 1-XORSAT formulas at a finite ratio

297: $\alpha$, and ask for the distribution of the minimal fraction of

298: unsatisfied equations, hereafter called ground state (GS) energy $e_{GS}$. 

299: For simplicity we work in the fixed-probability ensemble again. The

300: numbers $n_i^0, n_i^1$  of, respectively, $x_i=0, x_i=1$ are

301: independent Poisson variables with mean $\frac \alpha 2$. The

302: minimal number of unsatisfied equations is clearly $\min (n_i^0,

303: n_i^1)$. The GS energy is the sum (divided by $M$) of $N$ such 

304: i.i.d. variables; from the law of large number it almost surely 

305: converges towards the average value

306: \begin{equation}

307: e_{GS} (\alpha) = \frac 12 \left( 1 - e^{-\alpha} I_0(\alpha) -

308: e^{-\alpha} I_1(\alpha) \right) \ ,

309: \end{equation}

310: where $I_\ell$ denotes the $\ell^{th}$ modified Bessel function.

311: In other words almost all formulas have the same GS energy in the

312: infinite $N$ limit, a property called self-averageness in physics, and

313: concentration in probability. 

314: 

315: How many configurations of variables realize have minimal energy?

316: Obviously a variable is free (to take 0 or 1 value) if $n_i^0=n_i^1$,

317: and is frozen otherwise. Hence the number of GS configurations is

318: ${\cal N} = 2^{N_f}$ where $N_f$ is the number of free variables. 

319: Call 

320: \begin{equation}

321: \rho = \sum _{n\ge 0} e^{-\alpha} \left(

322: \frac{\alpha} 2\right)^n \frac 1{(n!)^2} = e^{-\alpha}\; I_0(\alpha)

323: \end{equation}

324: the probability that a variable is free. Then $N_f$ is a binomial

325: variable with parameter $\rho$ among $N$; it is sharply concentrated

326: around $\overline{N_f}=\rho\, N$ with typical fluctuations of the

327: order of $N^{1/2}$. As a consequence, the GS entropy per variable, 

328: $s_{GS} = (\log {\cal N})/N$, is self-averaging and almost surely

329: equal to its average value $s_{GS} = \rho \log 2$. 

330: 

331: 

332: Self-averageness is the very useful property. It allows us to study

333: the average value of a random variable, instead of its full

334: distribution. We shall use it in Section \ref{secreplicas} 

335: and also in the analysis of algorithms of Section \ref{secuc}. 

336: This property is not restricted to XORSAT but was proven to 

337: hold for the GS energy \cite{Br93} and entropy \cite{Mo07} of other

338: optimization problems.  

339: 

340: Not all variables are self-averaging of course. A straightforward

341: example is the number ${\cal N}$ of GS configurations itself. 

342: Its $q^{th}$ moment reads

343: $\overline{{\cal N}^q} = \left(1-\rho + \rho \; 2^q\right)^N$ 

344: where the overbar denotes the average over the formulas. We see that

345: $\overline{{\cal N}^q} \gg (\overline{{\cal N}})^q$: ${\cal N}$

346: exhibits large fluctuations and is not concentrated around its

347: average. Very rare formulas with atypically large number

348: $N_f$ of free variables contribute more to the $q^{th}$ moment than

349: the vast majority of formulas, and spoil the output. This is the very

350: reason we will need the introduction of the replica approach in

351: Section \ref{secreplicas}. 

352:  

353: 

354: \subsection{Large deviations for $P_{SAT}$ (I): 1-XORSAT}\label{seclargedev}

355: 

356: As we have seen in the previous sections 1-XORSAT formulas with a

357: finite ratio $\alpha$ are unsatifiable with high probability {\em

358:   i.e.} equal to unity in the infinite $N$ limit. For finite but large

359: $N$ there is a tiny probability that a randomly extracted formula is 

360: actually satisifiable. A natural question is to characterize the

361: `rate' at which $P_{SAT}$ tends to zero as $N$ increases (at fixed

362: $\alpha$). Answering to such questions is the very scope of large

363: deviation theory (see \ref{applargedev} for an elementary

364: introduction). Looking for events with very small probabilities 

365: is not only interesting from an academic point of view, but can also

366: be crucial in practical applications. We will see in Section 

367: \ref{secescape} that

368: the behaviour of some algorithms is indeed dominated by rare events.

369: 

370: Figure~\ref{proba1b-fig} shows minus the logarithm of $P_{SAT}$, divided by

371: $N$, as a function of the ratio $\alpha$ and for various sizes

372: $N$. Once again the data corresponding to different sizes collapse on

373: a single curve, meaning that

374: \begin{equation}

375: P_{SAT} (N,\alpha) = e^{-N\; \omega _1(\alpha) + o(N)} \ .

376: \end{equation}

377: Decay exponent $\omega_1$ is called rate function in probability

378: theory. We can derive its value in the fixed-probability ensemble from 

379: (\ref{psatp}) with $y=\alpha\times \sqrt N$, with the immediate result 

380: \begin{equation} \label{sca2p}

381: \omega_1 ^p (\alpha) = \alpha - \ln \big( 2\; e^{\alpha/2} -1 \big) \ .

382: \end{equation}

383: The agreement with numerics is very good for small ratios,

384: but deteriorates as $\alpha$ increases. The reason is simple. In the

385: fixed-probability ensemble the number $M$ of equations is not fixed but may

386: fluctuate around the average value $\bar M =\alpha N$. The 

387: ratio $\tilde \alpha =M/N$, is with high probability equal to

388: $\alpha$, but large deviations ($\tilde \alpha \ne \alpha)$ are

389: possible and described  by the rate

390: function\footnote{$M$ obeys a Poisson law with parameter $\bar

391:   M$. Using Stirling formula, $$e^{-\bar M} \frac{\bar M ^M}{M!}

392:   \simeq e^{-\alpha N } (\tilde \alpha N)^{\alpha N}{\sqrt {2\pi N}} 

393: \left( \frac e{\alpha N}\right)^{\alpha N} = e^{-N \, \Omega

394:   (\tilde\alpha | \alpha) + o(N)}\ ,$$ where $\Omega$ is defined in

395: (\ref{defOmega}). },

396: \begin{equation}\label{defOmega}

397: \Omega (\tilde \alpha | \alpha ) = \tilde \alpha -\alpha - \alpha\;

398: \ln (\alpha/\tilde \alpha ) \ .

399: \end{equation}

400: However the probability that a random 1-XORSAT formula with $M$

401: equations is satisfiable is also exponentially small in $N$, with a

402: rate function $\omega_1(\alpha)$ increasing with $\alpha$. Thus, in

403: the fixed-probability ensemble, a trade-off is found between ratios $\tilde

404: \alpha$ close to $\alpha$ (formulas likely to be generated) and close

405: to 0 (formulas likely to be satisfiable). As a result 

406: the fixed-probability rate function is 

407: \begin{equation} \label{sca2}

408: \omega_1 ^p (\alpha) = \min _{\tilde \alpha } \big[ \omega _1 (\tilde

409:   \alpha) + \Omega (\tilde \alpha | \alpha )\big] \ ,

410: \end{equation}

411: and is smaller than $\omega_1(\alpha)$. 

412: It is an easy check that the optimal ratio

413: $\tilde \alpha ^*=\alpha /(2-e^{-\alpha/2}) < \alpha$ as

414: expected. Inverting (\ref{sca2}) we deduce the rate function

415: $\omega_1$ in the

416: fixed-size ensemble, in excellent agreement with numerics

417: (Figure~\ref{proba1b-fig}). This example underlines that 

418: thermodynamically equivalent ensembles

419: have to be considered with care as far as rare events are concerned.

420: 

421: Remark that, when $\alpha \to 0$, $\tilde \alpha =\alpha +

422: O(\alpha ^2)$, and $\omega _1 ^p(\alpha)=\omega_1 (\alpha)

423: +O(\alpha^3)$.  This common value coincides with the scaling

424: function $- \Phi_1(\alpha)$ (\ref{sca1}). This identity is expected on

425: general basis (Section \ref{secld2}) and justifies the agreement between the

426: fixed-probability scaling function and the numerics based on the fixed-size

427: ensemble (Figure~\ref{proba1-fig}, right).

428: 

429: \begin{figure}[t]

430: \includegraphics*[width=5.truecm,angle=0]{fig221.eps}

431: \caption{Same data as Figure\ref{proba1-fig} (left) with: logarithmic

432:   scale on the vertical axis, and rescaling by $-1/N$. The scaling

433:   functions $\omega_1$ (\ref{sca2})

434: and $\omega_1 ^p$ (\ref{sca2p}) for, respectively, the fixed-size

435:   and fixed-probability ensembles are shown.}

436: \label{proba1b-fig}

437: \end{figure}

438: 

439: \subsection{Percolation in random graphs}\label{secrg}

440: 

441: Though 1-XORSAT allowed us to understand some general features of random

442: optimization problems it is very limited due to the absence of interactions 

443: between variables. A more interesting problem is 2-XORSAT where 

444: every equation define a joint constraint on two variables. 

445: Formulas of 2-XORSAT can be represented by a graph with $N$ vertices

446: (one for each variable), and $\alpha N$ edges. 

447: To each equation of the type $x_i+x_j=e$

448: corresponds an edge linking vertices $i$ and $j$, and carrying

449: 0 or 1 label (the value $e$ of the second member).

450: Depending on the input model chosen (Section \ref{secmodel}) multiple edges 

451: are present or not. 

452: 

453: As the formula is random so is graph. 

454: Figure~\ref{graphealeatoire} shows examples of graphs obtained 

455:  for various values of $\alpha$. Notice the

456: qualitative change of structure of graphs when 

457: the ratio $\alpha$ varies from low values (graphs are mostly made

458: of small isolated trees) to higher ones (a large part of vertices

459: are now connected together). This change is known as the percolation

460: transition in physics, or the appearance of a giant component in

461: mathematics literature.

462: 

463: \begin{figure}

464: \begin{center}

465: \includegraphics[width=70pt,angle=0]{fig241.eps} 

466: \hskip 1cm

467: \includegraphics[width=70pt,angle=0]{fig242.eps}

468: \hskip 1cm

469: \includegraphics[width=70pt,angle=0]{fig243.eps}

470: \caption{Examples of random graphs generated at fixed number $M=\alpha N$ of

471: edges (fixed-size model without repetition). All graph include $N=20$ 

472: vertices (grey dots). The average degree of valency, $2\alpha$,

473: is equal to $0.5$ ({\bf A}),  $1$ ({\bf B}), and

474: $2$ ({\bf C}). The labels of the vertices have been permuted to

475: obtain planar graphs, {\em i.e.} avoid crossing of edges.}

476: \label{graphealeatoire}

477: \end{center}

478: \end{figure}

479: 

480: Before reviewing some of the aspects of the percolation transition

481: let us mention an important fact on the valency of vertices.

482: As a result of the randomness of the graph generation process,

483: each node share edges with a variable number of neighboring vertices. 

484: In the large $N$ limit the degree $v$ of a vertex, {\em i.e.} 

485: the number of its neighbors, 

486: is a Poisson variable with mean $2\alpha$,

487: \begin{equation} \label{poissonconnec}

488: \mbox{Proba}[v] = e^{-2\alpha}\; \frac{(2\alpha)^v}{v!} \ .

489: \end{equation}

490: For instance the fraction of isolated vertices is $e^{-2\alpha}$.

491: The average degree of a vertex, $c=2\alpha$, is called

492: connectivity. 

493: 

494: It is natural to decompose the graphs into its connected subgraphs,

495: called components. Erd\"os and R\'enyi were able in 1960 to characterize the 

496: distribution of sizes of the largest component \cite{Bo89},

497: \begin{itemize}

498: \item When $c<1$, the largest component includes

499: $\sim \ln N/(c-1-\ln c)$  vertices  with high probability. 

500: Most components include only

501: a finite number of vertices, and are trees {\em i.e.} contain no circuit.

502: \item For $c=1$ the largest component contain $O(N^{2/3})$ vertices.

503: \item When $c>1$ there is one giant component containing $\sim \gamma

504: (c) N$ vertices; the others components are small {\em i.e.} look

505: like the components in the $c<1$ regime.

506: The fraction of vertices in the giant component is the unique

507: positive solution of 

508: \begin{equation} \label{sizegiant}

509: 1-\gamma = e^{-c \, \gamma} \ .

510: \end{equation}

511: It is a non analytic function of $c$, equal to 0 for $c\le 1$, and 

512: positive above, tending to unity when $c$ increases.

513: \end{itemize}

514: 

515: The phenomenon taking place at $c=1$ is an example of (mean-field) 

516: percolation transition. We now give a hand-waving derivation of

517: (\ref{sizegiant}).   

518: Consider a random graph $G$ over $N$ vertices, with connectivity

519: $c$. Add a new vertex $A$ to the graph to obtain $G'$. If we want $G'$

520: to be drawn from the same distribution as $G$, a number $v$ of edges

521: must be attached to $A$, where $v$ an integer--valued random number

522: following the Poisson distribution (\ref{poissonconnec}). 

523: After addition of $A$, some connected components of $G$ will merge

524: in $G'$. In particular, with some probability $p_v$, $A$ will be part

525: of the giant component of $G'$. To estimate $p_v$, we note that this

526: event will not happen if and only if none of the $v$ neighbors of $A$ in

527: $G'$  belongs to the giant component of $G$. Thus,

528: \begin{equation} \label{processusgraphe}

529: 1-p_v = (1-\gamma ) ^v \qquad ,

530: \end{equation}

531: where $\gamma$ is the size (fraction of vertices) of the giant component.

532: Summing both sides of (\ref{processusgraphe})

533: over the distribution (\ref{poissonconnec}) for $v$, and

534: asserting that the change in size of the giant component between

535: $G$ and $G'$ is $o(1)$ for large $N$, we obtain  (\ref{sizegiant}).

536: 

537: 

538: The above derivation illustrates an ubiquitous idea in 

539: probability and

540: statistical physics, which could be phrased as follows: `if a system

541: is very large, its statistical properties should be, in some sense, 

542: unaffected by a small increase in size'. This idea will be useful, in

543: a more sophisticated context, in Section \ref{seccavity}.

544: 

545: \subsection{Sat/Unsat transition in 2-XORSAT} \label{secsatunsat2}

546: 

547: \begin{figure}[t]

548: \includegraphics*[width=5.truecm,angle=0]{fig251.eps}

549: \caption{Probability that a random 2-XORSAT formula is satisfiable as

550:   a function of the ratio $\alpha$ of equations per variable, and for

551:   various sizes $N$. The full line is the asymptotic analytical

552: formula (\ref{psaty2}).}

553: \label{proba2-fig}

554: \end{figure}

555: 

556: Figure \ref{proba2-fig} shows the probability $P_{SAT}$ 

557: that a randomly extracted 

558: 2-XORSAT formula is satisfiable as function of $\alpha$, and for

559: various sizes $N$. It appears that $P_{SAT}$  drops quickly 

560: to zero for large $N$ when $\alpha$ reaches 

561: the percolation threshold $\alpha_c = \frac 12$. For ratios smaller than

562: $\alpha _c$ the probability of satisfaction is positive, but smaller 

563: than unity. 

564: 

565: Take $\alpha < \frac 12$. Then the random graph $G$ associated to a random

566: 2-XORSAT formula is non percolating, and made of many small components. 

567: Identical components (differing only by a relabelling of

568: the variables) may appear several times, depending on their topology.

569: For instance consider a connected graph $G'$ made of $E$ edges and $V$ 

570: vertices.

571: The average number of times $G'$ appears in $G$ is a function of $E$ and 

572: $V$ only, 

573: \begin{equation}

574: N_{E,V} = {N \choose V} \left( \frac{2\alpha}N \right)^E 

575:  \left( 1 - \frac{2\alpha}N \right)^{\frac {V(V-1)}2 + V(N-V)} 

576: \end{equation}

577: since any vertex in $G'$ can establish edges with other vertices in

578: $G'$, but is not allowed to be connected to any of the $N-V$ outside vertices.

579: When $N$ is very large compared to $E,V$ we have

580: \begin{equation} \label{avtree}

581: N_{E,V} \simeq N^{V-E} \; \frac{ (2\alpha)^E}{V!}\;  

582: e^{-2\alpha\, V} \ .

583: \end{equation}

584: Three cases should distinguished, depending on the value of $V-E$:

585: \begin{itemize}

586: \item{$V-E=1$}: this is the largest value compatible with connectedness, 

587: and corresponds to the case of trees. From (\ref{avtree}) every 

588: finite tree has of the order of $N$ copies in $G$.

589: \item {$V-E=0$}: this correspond to trees with one additional edge, that is, to graphs

590: having one cycle (closed loop). The average number of unicyclic graphs is,

591: from (\ref{avtree}), finite when $N\to\infty$. 

592: \item{$V-E\le -1$}: the average number of components with more than one cycle 

593: vanishes in the large $N$ limit; those graphs are unlikely to be

594: found and can be ignored\footnote{The probability that such a graph exists is

595: bounded from above by the average number, see \ref{appmoment}.}. 

596: \end{itemize}

597: Obviously a 2-XORSAT formula with tree

598: structure is always satisfiable\footnote{Start from one leaf, assign the 

599: attached variable to 0, propagate to the next variable according to

600: the edge value, and so on, up to the completion of the tree.}. Hence

601: dangerous subformulas, as far as satisfiability is concerned, are

602: associated to unicyclic graphs. A simple thought shows that a unicyclic

603: formula is

604: satisfiable if and only if the number of edges carrying label 1

605: along the cycle is

606: even. Since the values attached to the edges (second members in the 

607: formula) are uncorrelated 

608: with the topology of the subgraph (first members)

609: each cycle is satisfiable with probability one half.

610: We end up with the simple formula

611: \begin{equation} \label{psatx2}

612: P_{SAT} (N,\alpha) = \langle 2 ^{- C (G)} \rangle 

613: \end{equation}

614: where $C(G)$ denotes the number of cycles in $G$, and 

615: $\langle .\rangle$ the average over $G$.  For a reason which will become

616: clear below let us classify cycles according to their length $L$. 

617: How many cycles of length $L$ can we construct? We have to choose first

618: $L$ vertices among $N$, and join them one after the order according

619: to some order. As 

620: neither the starting vertex nor the direction along the cycle matter, 

621: the average number of $L$-cycles is

622: \begin{equation} \label{deflaml}

623: N_L =\frac{ N(N-1)\ldots (N-L+1)}{2L} \times \left( \frac{2\alpha}N \right)^L 

624: \to \Lambda _L = \frac{(2\alpha)^L}{2L}\ .

625: \end{equation}

626: when $N\to\infty$. As the emergence of a cycle between $L$ vertices is a 

627: local event (independent of the environment) we expect the number of

628: $L$-cycles to be Poisson distributed in the large $N$ limit

629: with parameter $\Lambda_L$. This statement can actually be proven, and

630: extended to any finite collection of cycles of various lengths\cite{Bo89}:

631: in the infinite size limit, the joint distribution of the numbers of cycles

632: of lengths $1,2, \ldots, L$ is the product of Poisson laws with

633: parameters $\Lambda_1, \Lambda _2, \ldots,\Lambda _L$

634: calculated in (\ref{deflaml}). The probability of satisfaction 

635: (\ref{psatx2}) therefore converges to 

636: \begin{equation}

637: \lim _{N\to\infty} P_{SAT} (N,\alpha) 

638: = \prod_{L\ge L_0}  \left\{ \sum _{C\ge 0} 

639: e^{-\Lambda _L} \; \frac{{(\Lambda _L/2)}^C}{C!} 

640: \right\} = \prod _{L\ge L_0} e^{-\Lambda_L/2}

641: \end{equation}

642: where $L_0$ is the minimal cycle length. In normal random graphs $L_0=3$ 

643: since triangles are the shortest cycles. However in our 2-XORSAT model

644: any equation, or more precisely, any first member  can appear twice or 

645: more, hence $L_0=2$. We conclude that \cite{Cr03a}

646: \begin{equation} \label{psaty2}

647: \lim _{N\to\infty} P_{SAT} (N,\alpha) 

648: = e^{\alpha/2}\; (1-2\alpha) ^{\frac 14} \qquad 

649: \mbox{when} \qquad \alpha < \alpha_c =\frac 12 \ .

650: \end{equation}

651: The agreement of this result  

652: with the large size trend coming out from numerical 

653: simulations is visible in Figure \ref{proba2-fig}. As $P_{SAT}$ is a 

654: decreasing function of $\alpha$ it remains null for all ratios larger than 

655: $\alpha_c$. The non analyticity of $P_{SAT}$ at $\alpha_c$

656: locates the Sat/Unsat phase transition of 2-XORSAT. 

657: 

658: It is an implicit assumption of statistical physics that asymptotic 

659: results of the kind of (\ref{psaty2}), rigorously valid in the $N\to

660: \infty$ limit, should reflect with good accuracy the finite but large

661: $N$ situation. An inspection of Figure \ref{proba2-fig} shows this is 

662: indeed the case. For instance, for ratio $\alpha =.3$, (\ref{psaty2}) 

663: cannot be told from the probability of satisfaction 

664: measured for formulas with $N=100$ variables. This statement

665: does not hold for $\alpha =.4$, where the agreement between infinite size

666:  theory and numerics sets in when $N = 1000$ at least. It appears

667: that such finite-size effects become bigger and bigger as $\alpha$

668: gets closer and closer to the Sat/Unsat threshold. This issue, of broad 

669: importance in the

670: context of phase transitions and the pratical application of asymptotic

671: results, is studied in Section \ref{secfs2}.

672: 

673: \subsection{Large deviations for $P_{SAT}$ (II): bounds in

674: the Unsat phase of 2-XORSAT.} \label{secld2}

675: 

676: Consider ratios $\alpha > \alpha_c$. 

677: The giant components of the corresponding formulas  contain an 

678: extensively large number of independent cycles, so we expect 

679: from  (\ref{psatx2}) that the probability of satisfaction is 

680: exponentially small in $N$, $P_{SAT} =\exp (-N \omega_2(\alpha)+o(N))$. 

681: Lower and upper bounds to the rate function $\omega _2$ can be obtained 

682: from, respectively, the first and second moment inequalities 

683: described in \ref{appmoment}. Denoting by ${\cal N}$ the number of solutions

684: of a formula $P_{SAT}$ is the probability that ${\cal N}\ge 1$, and

685: is bracketed according to (\ref{ineq}). 

686: 

687: To calculate the first moment of ${\cal N}$ remark that an equation

688: is satisfied by one half of the configurations. This result remains true 

689: for a restricted set of configurations when we average over the possible

690: choices of (the second member of) the equation. The average number

691: of solutions is thus $2^N/2^M$, from which we get

692: \begin{equation} \label{lowerom2}

693: \omega _2 (\alpha) \ge (\alpha -1) \ln 2 \ .

694: \end{equation}

695: This lower bound is useless for $\alpha <1$ since $\omega_2$ is positive

696: by definition. As for the upper bound we need to calculate the 

697: second moment $\langle {\cal N}^2\rangle$ of ${\cal N}$. As equations are 

698: independently drawn 

699: \begin{equation} \label{n2}

700: \langle {\cal N} ^2\rangle = \sum _{X,Y} q (X,Y) ^M \, 

701: \end{equation} 

702: where the sum is carried out over the pairs $X,Y$ of configurations

703: of the $N$ variables, and $q(X,Y)$ is the

704: probability that both $X$ and $Y$ satisfies the same randomly

705: drawn equation. $q$ can be easily expressed in terms of 

706: the Hamming distance $d$ between

707: $X$ and $Y$, defined as the fraction of variables having opposite

708: values in $X$ and $Y$. The general

709: expression for K-XORSAT is\footnote{The

710: equation is satisfied if the number of its variables taking opposite

711: values in $Y$ as in $X$ is even.

712: definition of $d$ the probability (over its index $i$) that a variable

713: takes different value in $X$ and $Y$ is $d$. Hence expression

714: (\ref{qk})

715: for $q(d)$. Beware of  the  $O(\frac 1N)$ corrections to this expression

716: e.g. if variable $x_1\ne y_1$ (which happens with probability $d$) then

717: the probability that   $x_2\ne y_2$ is $(dN-1)/(N-1) = d + (1-d)/-N-1)$. 

718: Those corrections are relevant for the calculation of Gaussian

719: fluctuations around the saddle-point (\ref{appfluctu}).}  

720: \begin{equation} \label{qk}

721: q  (d)=\frac 12( 1 - (1-2d)^K) \,

722: \end{equation}

723: and we specialize in this section to $K=2$.

724: Going back to (\ref{n2}) we can sum over $Y$ at fixed $X$, that is,

725: over the distances $d$ taking multiple values of $\frac 1N$ with the 

726: appropriate binomial multiplicity, and then sum over $X$ with the result

727: \begin{equation} \label{n2p}

728: \langle {\cal N} ^2\rangle = 2^{N} \sum _{d} {N \choose N\,d} \; q(d)^M

729: = \exp( N\, \max_{d \in [0;1]}A(d,\alpha) +o(N))

730: \end{equation} 

731: in the large $N$ limit, where

732: \begin{equation} \label{upperom2}

733: A(d,\alpha) =  (2\alpha-1) \; \ln 2 

734:   -d \ln d -(1-d) \ln (1-d) +\alpha \ln  q(d)  \ .

735: \end{equation}

736: For $\alpha < \frac 12$ the maximum of $A$ is located in $d^*=\frac 12$, 

737: and equal to $A^*=0$.  When $\alpha > \frac 12$,  $A$ has two global 

738: maxima located in $d^*(\alpha) <\frac 12$ and $1-d^*(\alpha)$, with 

739: equal value $A^*(\alpha) >0$.  

740: 

741: We plot in  Figure \ref{proba2b-fig} the lower (\ref{lowerom2}) and

742: upper bounds to the rate function, 

743: \begin{equation} \label{om2}

744: \omega _2 (\alpha) \le 2(1-\alpha)\ln 2 - \max_{d \in [0;1]}A(d,\alpha)

745: \end{equation}

746: from  (\ref{ineq}). 

747: At large ratio both bounds asymptotically match, proving that

748: $\omega_2 (\alpha) = (\alpha-1) \ln 2 + O(e^{-2\alpha})$. 

749: As the ratio departs from its threshold value by 

750: $\epsilon = \alpha - \alpha _c$ the upper bound grows quadratically,  

751:  $A^*(\alpha_c + \epsilon )\simeq \frac 34 

752: \epsilon^2 + O(\epsilon ^3)$. Numerics suggest that  the increase of

753: the rate function is slower, 

754: \begin{equation} \label{om2clth}

755: \omega _2 (\alpha_c + \epsilon) \simeq \Omega\;   \epsilon ^3 +

756: O(\epsilon ^4)\ ,

757: \end{equation} 

758: for some constant $\Omega\simeq 1$ (Figure \ref{proba2b-fig}). 

759: We will see in Section \ref{secreplicas} that

760: a sophisticated statistical physics technique, called the replica

761: method, actually predict this scaling with $\Omega = \frac {32}{27}$.

762: Actually the rate function can be estimated with the replica approach

763: for any ratio $\alpha$ with the result shown in Figure \ref{proba2b-fig}.

764: 

765: \begin{figure}[t]

766: \includegraphics*[width=5.truecm,angle=0]{fig261.eps}

767: \caption{Rate function $\omega_2 (\alpha)$ associated to the probability

768: of satisfaction of 2-XORSAT formulas with ratio $\alpha$. The dotted line

769: is the upper bound (\ref{upperom2}) and the dot-dashed line the lower bound

770: (\ref{lowerom2}). The full line is the output of the replica calculation of

771: Section \ref{secresu}, squares and circles represent numerical results for

772: $N=200$, 100 respectively from $10^6$ formulas. 

773: Inset: $P_{SAT}$ as a function of the size $N$ at the

774: Sat/Unsat ratio. The slope $-\frac 1{12}$ (\ref{predic112}) is shown for 

775: comparison.} 

776: \label{proba2b-fig}

777: \end{figure}

778: 

779: \subsection{Order parameter and symmetry breaking}\label{secop}

780: 

781: What is the meaning of the Hamming distance $d^*(\alpha)$ appearing

782: in the calculation of the second moment of the number of solutions?

783: An easy guess would be the average distance between pairs

784: of solutions

785: \begin{equation} \label{dav}

786:  d_{av}(\alpha) = \lim _{N\to\infty} \langle \frac{\sum _{X,Y\ \mbox{solutions

787: of}\ F} d(X,Y)}{{\cal N}(F)^2}\rangle_F

788: \end{equation}

789: where the average is taken over the satisfiable 

790: formulas $F$ with ratio $\alpha$, and

791: $d(X,Y)$ denotes the (intensive) Hamming distance between two 

792: solutions $X,Y$. However an inspection of the calculation of Section 

793: \ref{secld2} shows that

794: \begin{equation} \label{davstar}

795:  d^*(\alpha) = \lim _{N\to\infty} \frac{\langle \sum _{X,Y\ \mbox{solutions

796: of}\ F} d(X,Y)\rangle _F}{\langle {\cal N}(F)^2 \rangle_F} \ne 

797: d_{av}(\alpha)\ .

798: \end{equation}

799: Actually, though $d^*(\alpha)$ is not the average distance

800: between solutions with the unbiased distribution over  formulas, it 

801: is the average distance for a biased distribution  where 

802: each formula is weighted with

803: \begin{equation}\label{biased}

804: w(F) = \frac{  {\cal N}(F)^2}{\sum_{F'}  {\cal N}(F')^2}

805: \end{equation}

806: as can be readily checked upon insertion of $w(F)$ in the numerator

807: of (\ref{dav}). We will see in Section (\ref{secreplicas}) how 

808: to calculate average properties with the unbiased measure. 

809: 

810: Even so definition (\ref{davstar}) (and (\ref{dav}) too) is sloppy. 

811: If $X$ is a solution so is $-X$, the configuration where variables

812: values are flipped. Thus the average distance, whatever the weights

813: over formulas, is equal $\frac 12$ for any $N$! The difficulty comes from

814: the ambiguity in how the thermodynamic limit is taken, and

815: is the signature of spontaneous symmetry breaking. In the

816: low temperature phase of the Ising model 

817: the magnetization is either $m^*>0$ or $-m^*<0$ 

818: if an external field $h$ with, respectively, positive or negative vanishing  

819: amplitude is added prior to taking the infinite size limit.

820: In the present case what plays the role of the field  is a coupling

821: between solutions as is well-known in splin-glass theory 

822: \cite{Pa86}. Inserting $\exp[- N \, h\, d(X,Y)]$ in the numerator

823: of (\ref{davstar}) we obtain, when $N\to\infty$, $d^*$ if $h \to 0^+$

824: and $1-d^*$ if $h\to 0^-$.  The density $\mu$ of probability

825: of distances $d$ between solutions, with the biased measure (\ref{biased}), 

826: is concentrated below the Sat/Unsat threshold,

827: \begin{equation}

828: \mu (d) = \delta\big( d - \frac 12 \big) \qquad \mbox{for} \qquad

829: \alpha <\alpha _c \ , 

830: \end{equation}

831: and split into two symmetric peaks above the critical ratio,

832: \begin{equation}

833: \mu (d) = \frac 12 \delta\big( d - d^* \big) +

834: \frac 12 \delta\big( d - (1-d^*) \big)\qquad \mbox{for} \qquad

835: \alpha >\alpha _c \ . 

836: \end{equation}

837: The concept of spontaneous symmetry breaking will play a key role

838: in our study of 3-XORSAT (Section \ref{secpin}).

839: 

840: \subsection{Finite-size scaling (II): critical exponents}

841: \label{secfs2}

842: 

843: Let us summarize what we have found about the probability of satisfying 

844: random 2-XORSAT formulas in Section \ref{secsatunsat2} 

845: and \ref{secld2}. Close to the transition we have from (\ref{psaty2}) and 

846: (\ref{om2clth}),

847: \begin{equation} \label{bornesp2}

848: \ln P_{SAT} (N,\alpha_c +\epsilon) \simeq \left\{ \begin{array} {c c c}

849: \frac 14 \ln (-\epsilon) & \mbox{when} & \epsilon<0, N\to \infty\\

850: - \Omega\; N \; \epsilon ^3  & \mbox{when} & \epsilon>0, N\gg 1 \nonumber

851: \end{array}\right . \quad .

852: \end{equation}

853: The lesson of Section \ref{secsf1} is that $\ln P_{SAT}$ may 

854: have a non trivial limit  when $N\to \infty$,  $\epsilon \to 0$ provided we 

855: keep $y=\epsilon\, N^{\psi}$ constant. For 1-XORSAT the exponent 

856: $\psi$ was found to be equal to $\frac 12$, and $\ln P_{SAT}$ to 

857: converge to the scaling function $\Phi _1(y)$ (\ref{sca1}). 

858: The situation is similar but slightly more

859: involved for 2-XORSAT. A natural assumption is to look for 

860: the existence of a scaling function  such that

861: \begin{equation}\label{sca2a}

862: \ln P_{SAT} ( N, \epsilon) \simeq  N^\rho \; 

863: \Phi _2 ( \epsilon \, N^\psi)  \ .

864: \end{equation}

865: Let us see if (\ref{sca2a}) is compatible with the limiting

866: behaviours (\ref{bornesp2}). Fixing $\epsilon<0$ and sending $N\to\infty$ 

867: we obtain, for $y=\epsilon N^\psi \to -\infty$,  

868: $\frac 14 \ln |y| - \frac \psi 4 \ln N$ for the l.h.s, and

869: $N^\rho \times \Phi_2(y)$ for the r.h.s. Hence $\rho=0$ as in the 

870: 1-XORSAT case, but an additive correction is necessary, and we modify 

871: scaling Ansatz (\ref{sca2a}) into

872: \begin{equation}\label{sca2b}

873: \ln P_{SAT} ( N, \epsilon) \simeq  

874: \Phi _2 ( y= \epsilon \, N^\psi) - \frac{\psi}{4}\; \ln N .

875: \end{equation}

876: The above equation is now compatible with (\ref{bornesp2}) if

877: $\Phi_2(y) \sim \frac 14 \ln |y|$ when $y\to -\infty$.

878: Fixing now $\epsilon >0$ and sending $N$ to infinity we see that

879: (\ref{bornesp2}) is fulfilled if  $\Phi_2(y) \sim 

880: - \Omega\,  y^3$ when $y\to +\infty$ and 

881: \begin{equation} \label{valpsi}

882: \psi = \frac 13 \ .

883: \end{equation}

884: The above value for $\psi$ is expected from the study of random graphs

885: \cite{Bo89} and is related to the size 

886: $N^{1-\psi}=N^{\frac 23}$ of the largest components at the percolation

887: threshold (Section \ref{secrg}). 

888: $\psi$ is called 

889: critical exponent and characterize the width of the critical region 

890: of 2-XORSAT. Loosely speaking it means that a formula of with $N$

891: variables and $\frac N2+\Delta$ equations is 'critical'  when $\Delta \sim 

892: N^{\frac 23}$. This information will be useful for the analysis of

893: search algorithms in Section \ref{secuni}.

894: 

895: A consequence of (\ref{sca2b},\ref{valpsi}) is that, right at the

896: threshold, the probability of satisfaction decays as\footnote{This 

897: scaling is correct provided there is no diverging 

898: e.g. $O(\ln \ln N)$ corrections to (\ref{sca2b}).}

899: \begin{equation} \label{predic112}

900: P_{SAT}\left(N, \alpha_c \right) \sim N^{-\frac 1{12}} \ .

901: \end{equation}

902: This scaling agrees with numerical experiments, though the

903: small value of the decay exponent makes an accurate check delicate

904: (Inset of Figure \ref{proba2b-fig}).

905:   

906: \subsection{First and second moments inequalities for 

907: the 3-XORSAT threshold}

908: 

909: \begin{figure}[t]

910: \includegraphics*[width=5.truecm,angle=0]{fig291.eps}

911: \caption{Probability that a random 3-XORSAT formula is satisfiable as

912: a function of the ratio $\alpha$ of equations per variable, and for

913: various sizes $N$. The dotted line locates the threshold $\alpha_c

914: \simeq 0.918$.} 

915: \label{proba3-fig}

916: \end{figure}

917: 

918: Figure \ref{proba3-fig} shows the probability that a random 3-XORSAT

919: formula is satisfiable as a function of $\alpha$ for increasing sizes

920: $N$. It appears that formulas with ratio $\alpha < \alpha _c \simeq

921: 0.92$ are very likely to be satisfiable in the large $N$ limit,

922: while formulas with ratios beyond this critical value are almost

923: surely unsatisfiable. This behaviour is different from the 2-XORSAT

924: case (Figure \ref{proba2-fig}) 

925: in that $P_{SAT}$ seems to tend to unity below threshold.

926: 

927: It is important to realize that, contrary to the 2-XORSAT case, the

928: Sat/Unsat transition is not related to connectivity percolation.

929: Consider indeed a variable, say, $x_1$. This variable appear, on

930: average, in $3\alpha$ equations. Each of those equations contain

931: other 2 variables. Hence the `connectivity' of $x_1$ is $c=6\alpha$,

932: which is larger than unity for $\alpha _p=\frac 16$. In the range

933: $[\alpha_p,\alpha_c]$ the formula is percolating but still satisfiable

934: with high probability. The reason is that cycles do not

935: hinder satisfiability as much as in the 2-XORSAT case.

936: 

937: Use of the first and second moment inequalities (\ref{appmoment}) for

938: the number ${\cal N}$ 

939: of solutions provides us with upper and lower bounds to the

940: Sat/Unsat ratio $\alpha_c$. The calculation follows the same line as 

941: the one of the 2-XORSAT case (Section \ref{secld2}). The

942: first moment $\langle {\cal N} \rangle =2 ^{N(1-\alpha)}$ vanishes for

943: ratios larger than unity, showing that

944: \begin{equation}

945: \alpha _c \le \alpha _1 =1 \ .

946: \end{equation}

947: This upper bound is definitely larger than the true threshold from the 

948: numerical findings of Figure~\ref{proba3-fig}. We have already

949: encountered

950: this situation in 2-XORSAT: in the $\frac 12 < \alpha < 1$ range 

951: formulas are unsatisfiable with probability one (when $N\to\infty$),

952: yet the average number of solutions is exponentially large! The reason

953: is, once more, that the average result is spoiled by rare, satisfiable

954: formulas with many solutions.

955: 

956: As for the second moment expression (\ref{n2p},\ref{upperom2}) still holds 

957: with $q(d)$ given by (\ref{qk}) with $K=3$. The absolute maximum of the

958: corresponding function $A(d,\alpha)$ is located in

959: $d^* = \frac 12$ when $\alpha < \alpha _2 \simeq 0.889$, and

960: $d^* < \frac 12$ when $\alpha > \alpha_2$. In the latter case 

961: $\langle {\cal N}^2\rangle$ is exponentially larger than $\langle

962: {\cal N}\rangle ^2$, and the second moment inequality

963: (\ref{ineq}) does not give any information about $P_{SAT}$. In

964: the former case  $\langle {\cal N}^2\rangle$ and

965: $\langle {\cal N}\rangle ^2$ are equivalent to exponential-in-$N$

966: order. It is shown in \ref{appfluctu} that their  ratio actually tends to 

967: one as $N\to\infty$. We conclude that formulas with

968: ratios of equations per variable 

969: less than $\alpha_2$ are satisfiable with high probability in

970: the infinite size limit, or, equivalently \cite{Cr03b},

971: \begin{equation}

972: \alpha _c \ge \alpha _2 \simeq 0.889 \ .

973: \end{equation}

974: Unfortunately the lower and upper bounds do not match and the precise

975: value of the threshold remains unknown at this stage. We explain in

976: the next section how a simple preprocessing of the formula, before the

977: application of the first and second moment inequalities, can close the

978: gap, and shed light on the structure of the space of solutions.

979: 

980: \subsection{Space  of solutions and clustering}\label{seccluster}

981: 

982: We start from a simple observation. Assume we have a formula $F$ 

983: of 3-XORSAT where a variable, say, $x$, appears only once, that is, in 

984: one equation, say, $E:x+y+z=0$. Let us call $F'$ the subformula obtained

985: from $F$ after removal of equation $E$. Then the following statement

986: is true: {\em $F$ is satisfiable if and only if $F'$ is satisfiable.}

987: The proof is obvious: whatever the values of $y,z$ required to satisfy

988: $F'$ equation $E$ can be satisfied by an adequate choice of $x$, and

989: so can be the whole formula $F$.

990: 

991: In a random 3-XORSAT formula $F$ with ratio $\alpha$ there are about

992: $N\times 3\alpha \, e^{-3\alpha}$ variables appearing only once in the

993: formula. Removal of those variables (and their equations) produces

994: a shorter formula with $O(N)$ less equations. Furthermore it may

995: happen that variables with multiple occurrences in the original

996: formula have disappeared from the output formula, or appear only

997: once. Hence the procedure can be iterated until no single-occurrence

998: variables are present. We are left with $F_2$, the largest

999: subformula (of the original formula) where every variable

1000: appears at least twice.

1001: 

1002: Many questions can be asked: how many equations are left in

1003: $F_2$? how many variables does it involve? how many solutions does it

1004: have? Giving the answers requires a thorough 

1005: analysis of the removal procedure, with the techniques exposed in Section

1006: \ref{secuc} \cite{Co03,Me03,Du02}. 

1007: The outcome depends on the value of the ratio 

1008: compared to 

1009: \begin{equation}

1010: \alpha _d = \min _b -\frac{\log (1-b)}{3\, b^2} \simeq 0.8184\ldots

1011: \end{equation}

1012: hereafter called clustering threshold. With 

1013: high probability when $N\to\infty$ $F_2$ is empty if $\alpha <

1014: \alpha_d$, and contains an extensive number 

1015: of equations, variables when $\alpha > \alpha_d$. In the latter case

1016: calculation of the first and second moments 

1017: of the number of solutions of $F_2$ shows that this number does

1018: not fluctuate around the value $e^{N\,

1019:   s_{cluster}(\alpha)+o(N)}$ where

1020: \begin{equation}\label{scluster}

1021: s_{cluster} (\alpha) = (b - 3 \alpha \,b^2 + 2 \alpha \, b^3)\ln 2

1022: \end{equation}

1023: and $b$ is the strictly positive solution of the self-consistent equation 

1024: \begin{equation} \label{eqb}

1025: 1-b=e^{-3\, \alpha\, b^2} \ .

1026: \end{equation}

1027: Hence $F_2$ is satisfiable if and only if $\alpha < \alpha _c$ defined

1028: through $s_{cluster} (\alpha_c)=0$, that is,

1029: \begin{equation}

1030: \alpha_c \simeq 0.9179 \ldots \ .

1031: \end{equation}

1032: This value is, by virtue of the equivalence between $F$ and $F_2$ 

1033: the Sat/Unsat threshold for 3-XORSAT, in excellent agreement with 

1034: Figure~\ref{proba3-fig}.

1035: 

1036: How can we reconstruct the solutions of $F$ from the ones of $F_2$?

1037: The procedure is simple. Start from one solution of $F_2$ (empty

1038: string if $\alpha < \alpha_d$). Then introduce back the last  

1039: equation which was removed since it contained $n\ge 1$ single-occurrence

1040: variable. If $n=1$ we fix the value of this variable in a unique

1041: way. If $n=2$ (respectively $n=3$) there are 2 (respectively, 4) ways

1042: of assigning the reintroduced variables, defining as many solutions

1043: from our initial, partial solution. Reintroduction of equations one after

1044: the other according to the Last In -- First Out order gives us more and

1045: more solutions from the initial one, until we get a bunch of solutions

1046: of the original formula $F$. It turns out that the number of solutions

1047: created this way is $e^{N\, s_{in}(\alpha)+o(N)}$ where

1048: \begin{equation}\label{sin}

1049: s_{in} (\alpha) = (1-\alpha)\,\ln 2 - s_{cluster}(\alpha) \ .

1050: \end{equation}

1051: The above formula is true for $\alpha> \alpha_d$, and should be

1052: intended as $s_{in}(\alpha) =(1 - \alpha)\,\ln 2$ for  $\alpha< \alpha_d$.

1053: These two entropies are shown in Figure \ref{entroxor}.

1054: The total entropy, $s^*(\alpha) = s_{in}(\alpha)+s_{cluster}(\alpha)$,

1055: is simply 

1056: $(1-\alpha)\,\ln 2$ for all ratios smaller than the

1057: Sat/Unsat threshold. It shows no singularity at the clustering threshold.

1058: However a drastic change in the structure of the space of solutions

1059: takes place, symbolized in the phase diagram of Figure \ref{cluster-fig}: 

1060: \begin{itemize}

1061: \item For ratios $\alpha<\alpha_d$ the intensive

1062: Hamming distance between two solutions

1063: is, with high probability, equal to $d=1/2$. Solutions thus differ 

1064: on $N/2 +o(N)$ variables, as if they were statistically unrelated 

1065: assignments of the $N$ Boolean variables. 

1066: In addition the space of solutions enjoys some connectedness 

1067: property. Any two solutions are connected by

1068: a path (in the space of solutions) along which successive solutions

1069: differ by a bounded number of variables. Losely speaking 

1070: one is not forced to cross a big region prived of solutions

1071: when going from one solution to another.

1072: \item For ratios $\alpha > \alpha_d$ 

1073: the space of solutions is not connected any longer.  It is made

1074: of an exponentially large (in $N$) number ${\cal N}_{clu}

1075: =e^{N\, s_{cluster}}$ of connected

1076: components, called clusters, each containing an exponentially large

1077: number ${\cal N}_{in}=e^{N\, s_{in}}$ of solutions.  Two solutions belonging to

1078: different clusters lie apart at a Hamming distance $d_{clu}=1/2$ while,

1079: inside a cluster, the distance is $d_{in} < d_{clu}$. $b$ given by

1080: (\ref{eqb}) is the fraction of variables having the same value in all

1081: the solutions of a cluster (defined as the backbone).

1082: \end{itemize}

1083: 

1084: We present in Sections \ref{secreplicas} and \ref{seccavity} 

1085: statistical physics

1086: tools developed to deal with the scenario of Figure \ref{cluster-fig}.

1087: 

1088: \begin{figure}

1089: \begin{center}

1090: \includegraphics[width=120pt,angle=0]{fig2101.eps}

1091: \end{center}

1092: \caption{Entropies (base 2 logarithms divided by

1093: size $N$) of the numbers of solutions and clusters as a function of the 

1094: ratio $\alpha$. The entropy of solutions  

1095: equals $1-\alpha$ for $\alpha<\alpha_c\simeq 0.918$. 

1096: For $\alpha<\alpha_d\simeq 0.818$, solutions are uniformly

1097: scattered on the $N$-dimensional hypercube. 

1098: At $\alpha_d$ the  solution space discontinuously

1099: breaks into disjoint clusters. The entropies of clusters, $s_{cluster}$,

1100: and of solutions in each cluster, $s_{in}$, are such that 

1101: $s_{cluster}+s_{in}=s$. 

1102: At $\alpha_c$ the number of clusters stops being exponentially large

1103: ($s_{cluster}=0$). Above $\alpha_c$ there is almost surely no solution.}

1104: \label{entroxor}

1105: \end{figure}

1106: 

1107: 

1108: \begin{figure}

1109: \begin{center}

1110: \includegraphics[width=150pt]{fig2102.eps}

1111: \end{center}

1112: \caption{Phase diagram of 3-XORSAT. A `geometrical' phase 

1113: transition takes place in the satisfiable phase at  

1114: $\alpha_d\simeq 0.818$. At small ratios $\alpha<\alpha_d$ 

1115: solutions are uniformely

1116: scattered on the $N$-dimensional hypercube, with a typical normalized 

1117: Hamming distance $d=\frac 12$. At $\alpha_d$ the solution space discontinuously

1118: breaks into disjoint clusters: the Hamming distance $d_{in}\simeq 0.14$ 

1119: between solutions inside a cluster is much smaller than

1120: the typical distance $d_{clu}=\frac 12$ between two clusters.}

1121: \label{cluster-fig}

1122: \end{figure}

1123: 

1124: % replicas

1125: 

1126: \section{Advanced methods (I): replicas} \label{secreplicas}

1127: 

1128: \subsection{From moments to large deviations for the entropy}

1129: \label{secmoment}

1130: 

1131: The analysis of Section \ref{secld2} has shown that the first, and second

1132: moments of the number ${\cal N}$ of solutions

1133: are dominated by rare formulas with a lot of solutions. 

1134: Let us define the intensive entropy $s$ through ${\cal N} = e^{N\,

1135: s}$. As ${\cal N}$ is random (at fixed $\alpha,N$) so is $s$. We

1136: assume that the distribution of $s$ can be described, in the large

1137: size limit, by a rate function $\omega(s)$ (which depends on $\alpha$).  

1138: Hence,

1139: \begin{equation}

1140: \langle {\cal N}^q \rangle = \int ds\; e^{-N \, \omega(s)} \times

1141: \left(e^{N\, s}\right)^q \sim \exp\big[N  \max _{s} \big(q\, s - 

1142: \omega (s) \big)\big] 

1143: \end{equation}

1144: using the Laplace method. If we are

1145: able to estimate the leading behaviour of the $q^{th}$ moment of

1146: the number of solutions when $N$ gets large at fixed $\alpha$,

1147: \begin{equation}

1148: \langle {\cal N}^q \rangle \sim e^{N\, g(q)} \ ,

1149: \end{equation}

1150:  then $\omega$ can be easily

1151: calculated by taking the Legendre transform of $g$. In particular

1152: the typical entropy is obtained by $s^* = \frac{dg}{dq}(q\to 0)$.

1153: This is the road we will follow below. We will show how $g(q)$ can be 

1154: calculated when $q$ takes integer values, and then perform an analytic

1155: continuation to non integer $q$. The continuation leads to substantial 

1156: mathematical difficulties, but is not uncommon in statistical physics

1157: e.g. the $q\to 1$ limit of the $q$-state Potts model to recover

1158: percolation, or

1159: the $n\to 0$ limit of the $O(n)$ model to describe self-avoiding walks.

1160: 

1161: To calculate the $q^{th}$ moment we will have to average over the

1162: random components of formulas $F$, that is, the $K$-uplets of index

1163: variables in the first members and the $v=0,1$ second

1164: members. Consider now  homogeneous formulas $F_h$ whose first

1165: members are randomly drawn in the same way as for $F$, but with all

1166: second members $v=0$. The number ${\cal N}_h$ of solutions of a 

1167: homogeneous formula is always larger or equal to one. It is a simple

1168: exercise to show that

1169: \begin{equation} \label{momentxorequality}

1170: \langle {\cal N} ^{q+1} \rangle =

1171: 2^{N(1-\alpha)}\times \langle \big({\cal N}_h\big)^q \rangle \quad ,

1172: \end{equation}

1173: valid for any positive integer $q$\footnote{Actually the identity

1174:   holds for $q=0$ too, and is known under the name of harmonic

1175: mean formula \cite{Al89}.}. Therefore it is sufficient to calculate

1176: the moments of ${\cal N}_h=e^{N\, g_h(q)}$  since 

1177: (\ref{momentxorequality}) gives a simple identity between

1178: $g(q+1)$ and $g_h(q)$. This technical simplification has a

1179: deep physical meaning  we will  comment in Section \ref{secpin}. 

1180: 

1181: \subsection{Free energy  for replicated variables}

1182: 

1183: The $q^{th}$ power of the number of solutions to a homogeneous system

1184: reads

1185: \begin{equation}\label{qxor1}

1186: \big({\cal N}_h\big)^q = \left[ \sum _X \prod _{\ell=1}^M 

1187: e_\ell ( X) \right] ^q = \sum _{X^1,X^2, \ldots , X^q} 

1188: \prod _{\ell=1}^M \prod_{a=1}^q e_\ell ( X^a) \ ,

1189: \end{equation} 

1190: where $e_\ell(X)$ is 1 if equation $\ell$ is satisfied by assignment

1191: $X$. The last sum runs over $q$ assignments $X^a$, with $a=1,2, \ldots ,

1192: q$ of the Boolean variables, called replicas of the original

1193: assignment $X$.

1194: It will turn useful to denote by $\vec x_i = (x_i^1,x_i^2, \ldots ,

1195: x_i^q)$ the $q$-dimensional vector whose components are the values of

1196: variable $x_i$ in the $q$ replicas. To simplify notations we consider

1197: the case $K=3$ only here, but extension to other values of $K$ is

1198: straightforward. 

1199: Averaging over the instance, that is, the triplets of integers labelling

1200: the variables involved in each equation $\ell$, leads to the following

1201: expression for the $q^{th}$ moment, 

1202: \begin{eqnarray} \label{qxor2}

1203: \langle\big({\cal N}_h\big)^q \rangle&=& \sum _{X^1,X^2, \ldots , X^q} 

1204: \langle \prod_{a=1}^q e ( X^a) \rangle ^M \nonumber \\

1205: &=& \sum _{X^1,X^2, \ldots , X^q} \left[ \frac 1{N^3}

1206: \sum _{1\le i,j,k\le N} \delta _{\vec x_i + \vec x_j + \vec x_k }

1207: + O\bigg(\frac 1N\bigg)\right] ^M

1208: \end{eqnarray} 

1209: where $\delta _{\vec x} =1$ if the compoments of $\vec x$ are all

1210: null mod. 2, and 0 otherwise.

1211: We now procede to some formal manipulations of the above equation

1212: (\ref{qxor2}). 

1213: 

1214: {\bf First step.}

1215: Be ${\cal X} = \{X^1,X^2,\ldots , X^q\}$ one of the $2^{\,qN}$ replica 

1216: assignment. Focus on variable $i$, and its attached assignment

1217: vector, $\vec x_i$. The latter may be any of the $2^q$ possible vectors 

1218:  e.g. $\vec x_i = (1,0,1,0,0, \ldots , 0)$ if variable $x_i$ is equal

1219: to $0$ in all but the first and third replicas. The histogram of 

1220: the assignments vectors given replica assignment ${\cal X}$,  

1221: \begin{equation} \label{qxor2b}

1222: \rho \big( \vec x |{\cal X}\big) = 

1223: \frac 1N \sum _{i=1}^N \delta _{\vec x- \vec x_i} 

1224: \quad ,

1225: \end{equation}

1226: counts the fraction of assignments vectors $\vec x _i$ having value

1227: $\vec x$ when $i$ scans the whole set of variables from 1 to $N$.

1228: Of course, this histogram is normalised to unity,

1229: \begin{equation}\label{qxor2c}

1230: \sum _{\vec x} \rho \big( \vec x\big) = 1

1231: \quad ,

1232: \end{equation}

1233: where the sum runs over all $2^q$ assignment vectors.

1234: An simple but essential observation is that the r.h.s. of

1235: (\ref{qxor2}) may be rewritten in terms of the above histogram,

1236: \begin{equation} \label{qxor3}

1237: \frac 1{N^3}

1238: \sum _{1\le i,j,k\le N} \delta _{ \vec x_i + \vec x_j + \vec x_k }

1239: = \sum _{\vec x , \vec x'} 

1240: \rho \big( \vec x \big) \;\rho \big( \vec x'\big) \;\rho \big( \vec 

1241: x + \vec x' \big) 

1242: \ .

1243: \end{equation} 

1244: Keep in mind that $\rho$ in (\ref{qxor2b},\ref{qxor3}) depends on the

1245: replica assignement ${\cal X}$ under consideration.\\

1246: 

1247: {\bf Second step.} According to (\ref{qxor3}), two

1248: replica assignments ${\cal X}_1$ and ${\cal X}_2$ defining the same

1249: histogram $\rho$ will give equal contributions to 

1250: $\langle\big({\cal N}_h\big)^q\rangle$.

1251: The sum over replica assignments ${\cal X}$ can therefore be replaced

1252: over the sum over possible histograms provided the multiplicity ${\cal

1253: M}$ of the latter is taken properly into account.

1254: This multiplicity is also equal to the number of combinations

1255: of $N$ elements (the $\vec x_i$ vectors) into $2^q$ sets labelled

1256: by $\vec x$ and of cardinalities $N\, \rho(\vec x)$. We obtain

1257: \begin{equation}\label{qxor5}

1258: \langle \big({\cal N}_h\big)^q \rangle= \sum _{\{ \rho\}} ^{(norm)}

1259: e^{\displaystyle{\;  N \; {\cal G}_h \big( \{\rho\},\alpha \big) + o(N) }}

1260: \quad ,

1261: \end{equation} 

1262: where the $(norm)$ subscript indicates that the sum runs

1263: over histograms $\rho$ normalized according to (\ref{qxor2c}), and

1264: \begin{equation}\label{qxor6}

1265: {\cal G}_h \big( \{\rho\},\alpha \big) = - \sum _x \rho(x) \, \ln \rho(x) +

1266: \alpha\; \ln \bigg[ \sum _{\vec x , \vec x'} 

1267: \rho \big( \vec x \big) \;\rho \big( \vec x'\big) \;

1268: \rho \big( \vec x +\vec x' \big) 

1269: \bigg]

1270: \ .

1271: \end{equation} 

1272: In the large $N$ limit, the sum in (\ref{qxor5}) is 

1273: dominated by the histogram $\rho ^*$ maximizing the functional ${\cal G}_h$.

1274: 

1275: {\bf Third step.}

1276:  Maximisation of function ${\cal G}_h$

1277: over normalized histograms can be done within the Lagrange multiplier 

1278: formalism. The procedure consists in considering the modified function  

1279: \begin{equation}

1280: {\cal G}^{LM}_h \big( \{\rho\} , \lambda , \alpha \big) =

1281: {\cal G}_h \big( \{\rho\} , \alpha \big) + \lambda \; \bigg( 1 -

1282: \sum _{\vec x} \rho \big( \vec x\big)  \bigg) \quad ,

1283: \end{equation}

1284: and first maximise ${\cal G}^{LM}_h$ with respect to

1285: histograms $\rho$ without caring about the normalisation constraint,

1286: and then optimise the result with respect to $\lambda$.

1287: We follow this procedure with ${\cal G}_h$ given by (\ref{qxor6}). 

1288: Requiring that ${\cal G}^{LM}_h$ be maximal provides us with a set

1289: of $2^q$ coupled equations for $\rho^*$, 

1290: \begin{equation} \label{spxor}

1291: \ln \rho ^*(\vec x ) + 1 +\lambda -3\;\alpha\ \frac

1292: {\displaystyle{ \sum _{\vec x' } 

1293: \rho ^* \big( \vec x'\big) \;

1294: \rho ^*\big( \vec x+\vec x' \big) }}

1295: {\displaystyle{ \sum _{\vec x' , \vec x''} 

1296: \rho ^*\big( \vec x' \big) \;\rho ^*\big( \vec x''\big) \;

1297: \rho ^*\big( \vec x'+\vec x '' \big) }}

1298: = 0 \ ,

1299: \end{equation}

1300: one for each assignment vector $\vec x$. The optimisation equation 

1301: over $\lambda$ implies that $\lambda$ in  (\ref{spxor}) is such that 

1302: $\rho^*$ is normalised.

1303: At this point of the above and rather abstract calculation it may help

1304: to understand the interpretation of 

1305: the optimal histogram $\rho^*$. 

1306: 

1307: \subsection{The order parameter}

1308: 

1309: We have already addressed a similar question at the end 

1310: of the second moment calculation in Section \ref{secop}. 

1311: The parameter $d^*$ coming out from

1312: the calculation was the (weighted) average Hamming distance 

1313: (\ref{davstar}) between two solutions of the same random instance. 

1314: The significance of $\rho^*$ is identical. Consider $q'$ solutions

1315: labelled by $a=1,2,\ldots, q'$ of the same random and homogeneous 

1316: instance and a variable, say, $x_i$. What is the probability,

1317: over instances and solutions, that this variable takes, for instance,

1318: value 0 in the first and fourth solutions, and 1 in all other solutions?

1319: In other words, what is the probability that the assignment vector 

1320: $\vec x _i = (x_i^1, x_i^2, \ldots , x_i^{q'})$ 

1321: is equal to $\vec x' = (0,1,1,0,1,1,\ldots,1)$? The answer is

1322: \begin{equation}\label{defpxor}

1323: p(\vec x') = \left\langle \frac 1{({\cal N}_h)^{q'}} 

1324: \sum _{X^1,X^2, \ldots, X^{q'}}\;\delta _{\vec x_i - \vec x} \;

1325: \prod _{l=1}^M \prod _{a=1}^q e_\ell ( X^a) \right\rangle

1326: \end{equation}

1327: where the dependence on $i$ is wiped out by the average over the instance. 

1328: The above probability is an interesting quantity; it provides us information

1329: about the `microscopic' nature of solutions. Setting $q'=1$ gives

1330: us the probabilities $p(0),p(1)$ that a variable is false or true

1331: respectively, that is, takes the same value as in the null assignment or not.

1332: For generic $q'$ we may think of two extreme situations:

1333: \begin{itemize}

1334: \item a flat $p$ over assignment vectors, $p(\vec x ')=1/2^{q'}$, corresponds

1335: to essentially orthogonal solutions;

1336: \item on the opposite, a concentrated probability e.g. $p(\vec x ')=

1337: \delta_{\vec x'}$ implies that variables

1338: are extremely constrained, and that the (almost) unique solution

1339: is the null assignment.

1340: \end{itemize}

1341: 

1342: The careful reader will have already guessed that our calculation

1343: of the $q^{th}$ moment gives access to a weighted counterpart of $p$.

1344: The order parameter 

1345: \begin{equation} \label{defrhoetoilexor}

1346: \rho ^*(\vec x) = \frac 1{\langle ({\cal N}_h)^q\rangle}

1347: \sum _{X^1,X^2, \ldots, X^q}\; \delta _{\vec x_i - \vec x} \;

1348: \left\langle \prod _{l=1}^M \prod_{a=1}^q  e_\ell ( X^a) \right\rangle \quad ,

1349: \end{equation}

1350: is not equal to $p$ 

1351: even when $q=q'$. However, at the price

1352: of mathematical rigor, the exact probability $p$ over 

1353: vector assignments of integer length $q'$ can be reconstructed from

1354: the optimal histogram $\rho ^*$ associated to moments of

1355: order $q$ when $q$ is real-valued and sent to $0$. 

1356: The underlying idea is the following. Consider

1357: (\ref{defrhoetoilexor}) and an integer $q'<q$. From any assignment

1358: vector $\vec x$ of length $q$, we define two assignment vectors 

1359: $\vec x', \vec x''$ of respective lengths $q', q-q'$ corresponding to

1360: the first $q'$ and the last $q-q'$ components of $\vec x$ respectively. 

1361: Summing (\ref{defrhoetoilexor}) over the $2^{q-q'}$ assignment

1362: vectors $\vec x''$ gives,

1363: \begin{equation} \label{defrhoetoilexor2}

1364: \sum _{\vec x''} \rho ^*(\vec x', \vec x'') 

1365: = \frac 1{\langle ({\cal N}_h)^q \rangle}

1366: \sum _{\{X^a\}}\; \delta _{\vec x'_i - \vec x'} \;

1367: \left\langle \big({\cal N}_h\big) ^{q-q'}

1368: \prod _{l,a} e_\ell ( X^a) \right\rangle \ .

1369: \end{equation}

1370: As $q$ now appears in the powers of ${\cal N}_h$ in the numerator and

1371: denominator only, it can be formally send to zero at fixed $q'$, 

1372: yielding

1373: \begin{equation} \label{replicaess}

1374: \lim _{q\to 0}\ \sum _{\vec x''} \rho ^*(\vec x', \vec x'') = p(\vec x ')

1375: \end{equation}

1376: from (\ref{defpxor}).

1377: This identity justifies the denomination order parameter given to

1378: $\rho^*$. 

1379: 

1380: Having understood the significance of $\rho^*$ helps us to find

1381: appropriate solutions to (\ref{spxor}). Intuitively and from

1382: the discussion of the first moment case $q=1$,

1383: $p$ is expected to reflect both the special role of the null assignment

1384: (which is a solution to all homogeneous systems) and the ability of

1385: other solutions of a random system to be essentially orthogonal to 

1386: this special assignment. A possible guess is thus

1387: \begin{equation} \label{rhoetoilexor2}

1388: p(\vec x ') = \frac {1-b}{2^{q'}}+ b \; \delta 

1389: _{\vec x'} \quad ,

1390: \end{equation}

1391: where $b$ expresses some degree of `correlation' of solutions with

1392: the null one. Hypothesis (\ref{rhoetoilexor2}) interpolates between the 

1393: fully concentrated ($b=1$) and flat ($b=0$) probabilities.

1394: $b$ measures the fraction of variables (among the $N$ ones) 

1395: that take the 0 values in all $q'$ solution, and coincides with the

1396: notion of backbone introduced in Section \ref{seccluster}. Hypothesis 

1397: (\ref{rhoetoilexor2}) is equivalent, from the connection (\ref{replicaess})  

1398: between $p$ and the annealed histogram $\rho ^*$ to the following

1399: guess for the solution of the maximisation condition (\ref{spxor}),

1400: \begin{equation} \label{rhoetoilexor}

1401: \rho ^*(\vec x ) = \frac {1-b}{2^q}+ b \; \delta

1402: _{\vec x} \ .

1403: \end{equation}

1404: Insertion of Ansatz (\ref{rhoetoilexor}) in (\ref{spxor}) shows

1405: that it is indeed a solution provided $b$ is shrewdly chosen

1406: as a function of $q$ and $\alpha$, $b=b^*(q,\alpha)$. 

1407: Its value can be either found from direct resolution of (\ref{spxor}),

1408:  or from insertion of histogram (\ref{rhoetoilexor}) 

1409: in ${\cal G}_h$ (\ref{qxor6}) and maximisation over $b$, with the

1410: result, 

1411: \begin{equation} \label{xormaxgh}

1412: g_h(q,\alpha) =  \max _{0\le b\le 1} A_h (b,q,\alpha)

1413: \end{equation}

1414: where

1415: \begin{eqnarray} \label{xormaxgh2}

1416: A _h (b,q,\alpha) &=& -\left( 1 -\frac 1{2^q} \right)

1417: \; (1-b) \; \ln \left(\frac{1-b}{2^q}\right) \\

1418: &-& \left( b + \frac{1-b}{2^q} \right) \; \ln 

1419: \left( b + \frac{1-b}{2^q} \right) + \alpha\; 

1420: \ln \left( b^3 + \frac{1-b^3}{2^q} \right) \ ,  

1421: \nonumber 

1422: \end{eqnarray}

1423: where the maximum is precisely reached in $b^*$. Notice that, since

1424: $\rho^*$ in (\ref{rhoetoilexor}) is entirely known from the value

1425: of $b^*$, we shall indifferently call order parameter $\rho^*$, or

1426: $b^*$ itself.

1427: 

1428: \subsection{Results}\label{secresu}

1429: 

1430: Numerical investigation of  $A _h$ (\ref{xormaxgh2}) shows

1431: that: for $\alpha<\alpha_M(q)$ the only local maximum of $A _h$ 

1432: is located in $b^*=0$, and $A_h(q,\alpha)=q(1-\alpha)\ln 2$; 

1433: when $\alpha_M(q) < \alpha<\alpha^*(q)$, there exists another local maximum 

1434: in $b>0$ but the global maximum is still reached in $b^*=0$;

1435: when $\alpha>\alpha^*(q)$, the global maximum is located in $b^*>0$.

1436: This scenario extends to generic $q$ the findings of the second moment

1437: calculation carried out in Section

1438: \ref{secld2}. The $\alpha_M$ and $\alpha^*$ lines divide the

1439: $q,\alpha$  plane as

1440: shown in Figure~\ref{qetoilexor}. Notice that, while the black dots in

1441: Figure~\ref{qetoilexor} correspond to integer-valued $q$, the continuous

1442: lines are the output of the implicit analytic continuation to real $q$

1443: done by the replica calculation.  

1444: 

1445: \begin{figure}

1446: \begin{center}

1447: \includegraphics[width=110pt,angle=-90]{fig341.eps}

1448: \end{center}

1449: \caption{The $q,\alpha$ 

1450: plane and the critical lines $\alpha_M(q)$ (dashed),

1451: $\alpha^*(q)$ (full tick), and $\alpha_s(q)$ (full thin) 

1452: appearing in the calculation

1453: of the $q^{th}$ moment for homogeneous 3-XORSAT systems. 

1454: Full dots correspond to integer $q$ values, while

1455: continuous curves result from the analytic continuation to

1456: real $q$. The fraction of variables in the backbone, $b^*$, vanishes below the 

1457: line $\alpha_M(q)$; the global maximum of $A_h$ in 

1458: (\ref{xormaxgh2}) is located in $b^*>0$ for ratios $\alpha>\alpha^*(q)$.

1459: Ansatz (\ref{rhoetoilexor}) is locally unstable in the hardly visible 

1460: domain $q<0,\alpha_M(q) <\alpha <\alpha _s(q)$.}

1461: \label{qetoilexor}

1462: \end{figure}

1463: 

1464: Taking the derivative of (\ref{xormaxgh}) with respect to $q$ and

1465: sending $q\to 0$ we obtain the typical entropy of a homogeneous

1466: 3-XORSAT system at ratio $\alpha$,

1467: \begin{equation}\label{commentxor1}

1468: s_h ^*(\alpha)=  \ln 2 \times \max _{0\le b\le 1} 

1469: \big[ (1-b) \big(1 - \ln(1-b)\big) - \alpha \, (1-b^3) \big] \ . 

1470: \end{equation} 

1471: The optimal value for $b$ coincides with the solution of (\ref{eqb}). 

1472: The typical entropy is plotted in Figure~\ref{xorcurve}, and is equal to:

1473: \begin{itemize}

1474: \item $(1-\alpha) \ln 2$ when $\alpha<\alpha_c\simeq 0.918$ 

1475: (Figure~\ref{qetoilexor}); in this range of ratios, homogeneous and

1476:   full (with random second members)

1477: systems have essentially the same properties, with the same cluster

1478:  organisation of solutions, and identical entropies of solutions. 

1479: \item a positive but rapidly decreasing function given by

1480: (\ref{commentxor1}) when $\alpha>\alpha_c$; above the

1481: critical ratio, a full system has no solution any more, while a homogeneous

1482: instance still enjoys a positive entropy. The expression for

1483: $s^*_h(\alpha)$ coincides with the continuation to $\alpha>\alpha_c$ of 

1484: the entropy $s_{in}(\alpha)$ (\ref{sin}) of solutions in

1485: a single cluster for a full system.

1486: In other words, a single cluster of solutions, the one with the 

1487: null solution, survive for ratios $\alpha>\alpha_S$ in homogeneous systems.  

1488: \end{itemize}

1489: 

1490: Atypical instances can be studied and the large deviation rate 

1491: function for the entropy can be derived from (\ref{xormaxgh}) for

1492: homogeneous systems, and using equivalence (\ref{momentxorequality}), 

1493: for full systems. Minimizing over the entropy we obtain the rate

1494: function $\omega_3(\alpha)$ associated to the probability that a

1495: random 3-XORSAT system is satisfiable,

1496: with the result shown in Figure~\ref{xorcurve}. As expected we find

1497: $\omega_3 =0$ for $\alpha < \alpha_c$ and $\omega_3 >0 $ for

1498: $\alpha>\alpha_c$, allowing us to locate the Sat/Unsat threshold.

1499: 

1500: Notice that the emergence of clustering can be guessed from Figure 

1501: \ref{qetoilexor}. It coincides with the appearance of a local

1502: maximum of $A_h$ (\ref{xormaxgh2}) with a non vanishing 

1503: backbone $b$. While in the intermediate phase $\alpha_d<\alpha<

1504: \alpha_c$, the height of the 

1505: global maximum equals the total entropy $s^*$, the height of the local 

1506: maximum coincides with the entropy of clusters $s _{cluster}$ 

1507: (\ref{scluster}).

1508: 

1509: \subsection{Stability of the replica Ansatz}

1510: 

1511: The above results rely on Ansatz (\ref{rhoetoilexor}). A necessary criterion

1512: for its validity is that $\rho^*$ locates a true local maximum

1513: of ${\cal G}_h$ , and not merely a saddle-point. Hence we have to calculate

1514: the Hessian  matrix of ${\cal G}_h$ in $\rho^*$, and check that the eigenvalues

1515: are all negative \cite{De79}. 

1516: Differentiating (\ref{qxor6}) with respect to $\rho(\vec x)$

1517: and $\rho (\vec x')$ we obtain the Hessian matrix

1518: \begin{equation} \label{hvalue}

1519: H(\vec x,\vec x') = -\frac{\delta _{\vec x+\vec x'}}{\rho^* (\vec x)}

1520: +6\alpha\; \frac{\rho^* (\vec x+\vec x')}D - 9\alpha \;\frac{N(\vec x)}D

1521:   \, \frac{N(\vec x')}D\ ,

1522: \end{equation}

1523: where $D=\frac {1-b^3}{2^q} + b^3$, $N(\vec x) = \frac {1-b^2}{2^q} 

1524: + b^2\,\delta _{\vec x}$. We use $b$ instead of $b^*$ to 

1525: ligthen the notations, but it is intended that $b$ is the backbone

1526: value which maximizes $A_h$ (\ref{xormaxgh2}) at fixed $q,\alpha$.

1527: To take into account the global constraint over the histogram 

1528: (\ref{qxor2c}) one can express one fraction, say, $\rho (\vec 0)$, as

1529: a function of the other fractions $\rho(\vec x)$, $\vec x\ne \vec 0$.

1530: ${\cal G}_H$ is now a fonction of $2^q-1$ independent variables, 

1531: with a Hessian matrix $\tilde H$ simply related to $H$,

1532: \begin{equation} \label{htilde}

1533: \tilde H(\vec x,\vec x') = H(\vec x,\vec x') - H(\vec x,\vec 0) -

1534: H(\vec 0,\vec x')  + H(\vec 0,\vec 0) \ .

1535: \end{equation}

1536: Plugging expression (\ref{hvalue}) into (\ref{htilde}) we obtain  

1537: \begin{eqnarray}

1538: \tilde H(\vec x,\vec x') &=& \lambda _R\; \delta_{\vec x+\vec x'}

1539: + \frac 1{2^q-1}\big( \lambda _L - \lambda _R) \qquad \mbox{where} 

1540: \nonumber \\

1541: \lambda_R &=& 6\alpha \; \frac bD - \frac{2^q}{1-b} \\

1542: \lambda _L &=& 2^q \left( 6\alpha \; \frac bD - \frac{2^q}

1543: {(1-b)(1-b+2^qb)} - 9\alpha (1-2^{-q})\frac{b^4}{D^2}\right) \ .

1544: \nonumber 

1545: \end{eqnarray}

1546: Diagonalization of $\tilde H$ is immediate, and we find two

1547: eigenvalues:

1548: \begin{itemize}

1549: \item $\lambda _L$ (non degenerate). The eigenmode corresponds

1550: to a uniform infinitesimal

1551: variation of $\rho (\vec x$) for all $\vec x\ne \vec 0$, that is, a

1552: change of $b$ in (\ref{rhoetoilexor}). It is an easy check that 

1553: \begin{equation}

1554: \lambda _L = \frac{2^q}{1-2^{-q}}\; \frac{\partial ^2 A_h}{\partial

1555: b^2} (b,q,\alpha) \ ,

1556: \end{equation}

1557: where $A_h$ is defined in (\ref{xormaxgh2}).

1558: As we have chosen $b$ to maximize $A_h$ this mode, called longitudinal 

1559: in replica literature \cite{De79}, is stable\footnote{Actually $b^*$

1560: is chosen to {\em minimize} $A_h$ when $q<0$, thus $\lambda_L$ has

1561: always the right negative sign.}. 

1562:  

1563: \item $\lambda _R$ ($2^q-2$-fold degenerate): the eigenmodes correspond

1564: to fluctuations of the order parameter $\rho$ transverse to

1565: the replica subspace described by (\ref{rhoetoilexor}), and are

1566: called replicon in spin-glass theory\cite{De79}. Inspection of 

1567: $\lambda_R$ as a function

1568: of $\alpha,q$ shows that it is always negative when $q>0$. For $q<0$ 

1569: the replicon  mode is stable if 

1570: \begin{equation}

1571: \alpha > \alpha_s (q) = \frac{1- b^3 + 2^q\, b^3}{6 \, b(1-b)} \ .

1572: \end{equation}

1573: which is a function of $q$ only once we have chosen $b=b^*(q,\alpha_s)$.

1574: \end{itemize}

1575: The unstable region $q<0, \alpha_M (q) < \alpha <\alpha _s(q)$ is 

1576: shown in Figure \ref{qetoilexor} and is hardly visible when $q>-3$. 

1577: In this region a continuous symmetry breaking is expected \cite{Me87}.

1578: In particular $\alpha_s$ stay below the $\alpha^*$ line for

1579: small (in absolute value) and negative $q$. We conclude 

1580: that our Ansatz (\ref{rhoetoilexor}) defines a maximum

1581: of ${\cal G}_h$. 

1582: 

1583: Is it the global maximum of ${\cal G}_h$? There is no simple way to 

1584: answer this question. Local stability does not rule out the possibility

1585: for a discontinuous transition to another maximum in the replica order

1586: parameter space not described by (\ref{rhoetoilexor}). 

1587: A final remark is that a similar calculation can be done for any value

1588: of $K$. The outcome for $K=2$ is the rate function $\omega_2$ plotted

1589: in Figure~\ref{proba2b-fig}, in good agreement with numerics

1590: close to the threshold.

1591: 

1592: \begin{figure}

1593: \begin{center}

1594: \includegraphics[width=120pt,angle=0]{fig342.eps}

1595: \end{center}

1596: \caption{Rate function $\omega _3$ for the probability of satisfaction

1597: of full (bottom curve) and entropy $s_h^*$ of solutions

1598: for homogeneous (top curve) 3-XORSAT systems vs. $\alpha$. 

1599: The vertical dotted lines indicate the critical

1600: Sat/Unsat threshold, $\alpha_c\simeq 0.918$. For $\alpha <\alpha_c$

1601: $\omega _3=0$, and  $s_h^*=(1-\alpha)\,\ln 2$ is the same as for full systems.

1602: Above the threshold  $\omega ^* <0$. Homogeneous

1603: systems are, of course, always satisfiable: the entropy $s_h^*$ is a positive

1604: but quickly decreasing function of $\alpha$.}

1605: \label{xorcurve}

1606: \end{figure}

1607: 

1608: % cavity

1609: 

1610: \section{Advanced methods (II): cavity}\label{seccavity}

1611: 

1612: The cavity method, in the context of disordered systems, was 

1613: historically developed as an alternative to the the replica method

1614: \cite{Me87}. Its application to spin systems on random graphs is

1615: extensively explained in \cite{Me01y}, and we limit ourselves here 

1616: to briefly show how it gives back the 3-XORSAT scenario of Section

1617: \ref{seccluster} \cite{Me03}. 

1618: 

1619: Let us consider a system $F$ involving variables $x_i$, $i=1,\ldots,

1620: N$. In the following we will indifferently 

1621: use the variable $x_i=0,1$ or its spin  representation

1622: $S_i=(-1)^{x_i}=\pm 1$ when convenient. 

1623: Let us define the GS energy $E^F _i (S_i)$ 

1624: of the system when the $i^{th}$ spin is kept fixed, that is, the

1625: minimal number of violated equations in $F$, taken over the $2^{N-1}$ 

1626: configurations. We may always write 

1627: \begin{equation} \label{defec}

1628: E^F_i (S_i) = -w_i-h_i\, S_i\ ,

1629: \end{equation}

1630: where $h_i$ is called `field' acting on spin $S_i$.

1631: For a homogeneous system $E^F_i(+1)=0$, and $E^F_i(-1)=n_i$ for some integer

1632: $n_i$. Hence $h_i=\frac {n_i}2$ takes half-integer values.

1633: 

1634: The above definition can be  extended to the case of

1635: $\ell > 1$ fixed spins. Let $I\subset \{1,2,\ldots, N\}$ be a subset

1636: of the indices of cardinal $|I|\ge 2$, and $S_I$ denote one of the $2^{|I|}$

1637: configurations of the spins $S_i, i\in I$. The GS energy of $F$ for given 

1638: $S_I$  can in general be written as   

1639: \begin{equation} \label{defec2}

1640: E^F_{I} (S_I) = -w_{I} - \sum _{i \in I}  h_{i} \, S_{i}

1641: -\sum  _{I'\subset I : |I'|\ge 2} J_{I'}\,\prod

1642: _{i\in I' }  S_i

1643: \end{equation}

1644: where the $h_i$s are the fields and the $J_{I'}$s are

1645: effective couplings between subsets of spins. 

1646: 

1647: The basic cavity assumption is that effective couplings are

1648: vanishingly small: $J_{I'}=0$ for every subset $I'$. This apparently bold

1649: hypothesis critically relies on a general property of random graphs

1650: (from which our system is built on). Define the distance between two

1651: vertices as the minimal number of edges on pathes linking these two

1652: points. Then vertices in a finite subset are, with high probability

1653: when $N\to\infty$, typically at infinite distance from each

1654: other\footnote{An alternative formulation is, for finite size $N$,

1655:   that the shortest loops (in extensive number) have lengths of the order of

1656:   $\log N$ \cite{Bo89}.}. When correlations 

1657: between variables in GS extinguish with the distance {\em i.e.} when

1658: the correlation length is finite the cavity assumption is

1659: correct in the large $N$ limit  \cite{Me87,Mo05}. The assumption will break 

1660: down when correlations subsist on infinite distance, which 

1661: happens to be the case in the clustered phase. 

1662: 

1663: \subsection{Self-consistent equation for the fields} \label{seccav1}

1664: 

1665: Under the assumption that couplings between randomly picked up 

1666: spins are null we are left with the fields only. The goal of this

1667: section is to show how to calculate those fields, or more precisely,

1668: their probability distribution. The derivation is based on the 

1669: addition procedure 

1670: already used in the calculation of the size of the giant component in

1671: random graphs (Section \ref{secrg}). 

1672: 

1673: Consider a system $F$ over $N$ variables to which we want

1674: to add one 

1675: equation involving one new variable $S$, and two variables $S_1,S_2$

1676: appearing in $F$. The energy function 

1677: associated to this equation is

1678: \begin{equation}

1679: e(S,S_1,S_2) = \frac 12 \big( 1 - \sigma\, S\, S_1\, S_2 \big)

1680: \end{equation}

1681: where $\sigma=+1$, respectively $-1$, when the second member of the 

1682: equation is 0, resp. 1. Let us calculate the GS

1683: energy of the new system $F'=F$ +  added equation when the new

1684: variable $S$ is kept fixed, 

1685: \begin{equation}

1686: E^{F'} (S) = \min _{S_1,S_2}\big[ e(S,S_1,S_2) + E^F_{1,2}

1687:   (S_1,S_2) \big] = - w  - u \, S \ .

1688: \end{equation}

1689: With the cavity hypothesis the couplings between spins $S_1,S_2$ is

1690: null and the minimization is straightforward. We deduce the 

1691: following explicit

1692: expression for the field acting on $S$ (called bias in the cavity

1693: literature \cite{Me01y}), 

1694: \begin{equation}\label{expressu}

1695: u=\frac \sigma2\; \mbox{sign} \big(h_1\; h_2\big) \ .

1696: \end{equation}

1697: Suppose we now add $\ell \ge 1$ (and not only one) equations. The above

1698: calculation can be easily repeated. The absence of couplings make the

1699: total field acting on $S$ a linear combination of the fields coming

1700: from each new equation, 

1701: \begin{equation}\label{uj}

1702: h = \sum _{j=1}^\ell u ^{j} \ ,

1703: \end{equation}

1704: where $u^{j}$ is calculated from (\ref{expressu}) and each pair of fields

1705: $(h_1^{j}, h_2^{j})$ acting on the spins in the $j^{th}$ equation,

1706: $j=1,\ldots, \ell$. 

1707: 

1708: How many equations should we add for our new system over $N+1$

1709: variables to have the same statistical features as old one over $N$ 

1710: variables? First $\ell$ should be Poisson distributed 

1711: with parameter $3\alpha$. Then, given $\ell$, we randomly chose $\ell$

1712: pairs of variables; for each pair the corresponding bias

1713: $u$ can be calculated from (\ref{expressu}). Assume the output is a

1714: set of $\ell$ independent biases, taking values

1715: \begin{equation}\label{uk}

1716: u = \left\{ \begin{array}{c c c}

1717: + \frac 12 & \mbox{with probability} & a_+ \\

1718: 0 & \mbox{with probability} & a_0 \\

1719: - \frac 12 & \mbox{with probability} & a_- 

1720: \end{array} \right. \ 

1721: \end{equation}

1722: Obviously $a_++a_0+a_-=1$.

1723: Summing over the equations as in (\ref{uj}) we obtain the distribution of the

1724: field $h$ acting on the new spin at fixed $\ell$,

1725: \begin{equation}

1726: p(h | \ell )=\sum _{\ell_+,\ell_0,\ell_-} {\ell \choose

1727:   \ell+,\ell_0,\ell_-}

1728: a_+^{\ell_+}\, a_0^{\ell_0}\, a_-^{\ell_-}\, \delta

1729:   _{h-\frac 12(\ell_+-\ell_-)} \ .

1730: \end{equation} 

1731: Finally we sum over the Poisson distribution for $\ell$ to obtain the

1732: distribution of fields $h$,

1733: \begin{equation} \label{defph}

1734: p(h)=e^{-3\alpha (1-a_0)} \sum _{\ell_+,\ell_-} \frac{

1735:   (3\alpha)^{\ell_++\ell _-}}{\ell_+!\ell_-!} a_+^{\ell_+}\, 

1736:  a_-^{\ell_-}\, \delta

1737:   _{h-\frac 12(\ell_+-\ell_-)} \ .

1738: \end{equation}

1739: In turn we calculate the distribution of the biases from the one of

1740: the fields through (\ref{expressu}). The outcome are the values of the

1741: probabilities (\ref{uk}) in terms of $p$, 

1742: \begin{eqnarray}

1743: a_+ &=& \sum _{h_1,h_2: h_1 h_2 > 0} p(h_1)\, p(h_2) \ , 

1744: \quad 

1745: a_- = \sum _{h_1,h_2: h_1 h_2 < 0} p(h_1)\, p(h_2) \ ,\nonumber \\

1746: a_0 &=& \sum _{h_1,h_2: h_1 h_2 = 0} p(h_1)\, p(h_2) = 2\, p(0) -p(0)^2 \ .

1747: \label{conda0} 

1748: \end{eqnarray}

1749: The above equations together with (\ref{defph}) define three

1750: self-consistent conditions for $a_0,a_+,a_-$. Notice that the free

1751: energy can be calculated along the same lines \cite{Me01y}.

1752: 

1753: \subsection{Application to homogeneous and full systems}\label{seccavapp}

1754: 

1755: In the case of homogeneous systems ($\sigma = +1$) 

1756: we expect all the fields to be

1757: positive, and look for a solution of (\ref{conda0}) with $a_-=0$. Then

1758: $p(h)$ (\ref{defph})

1759: is a Poisson distribution for the integer-valued variable $2h$,

1760: with parameter $3\,\alpha\, a_+$. 

1761: The self-consistent equation (\ref{conda0}) reads

1762: \begin{equation}

1763: a_0=1-a_+= 2 \; e^{-3 \alpha a_+} - e^{-6\, \alpha a_+} = 1 -

1764: \big( 1 -  e^{-3\, \alpha a_+} \big)^2

1765: \end{equation}

1766: which coincides with (\ref{eqb}) with the definition $b=\sqrt{a_+}$.

1767: As expected

1768: \begin{equation}

1769: b= \sum _{h\ge \frac 12} p(h)  

1770: \end{equation}

1771: is the fraction of frozen variables (which cannot be flipped from 0 to

1772: 1 in GS assignments), in agreement with the notion of backbone of Section

1773: \ref{seccluster}.

1774: 

1775: The energy is zero at all ratio $\alpha$ by construction. As for the

1776: entropy consider adding a new equation to the system $F$ (but with no

1777: new variable). With probability

1778: $1-b^3$ at least one of the three variables in the new equation $e$ was

1779: not frozen prior to addition, and the number of solutions of the new 

1780: system $F+e$ is half the one of $F$. With probability $b^3$ all three

1781: variables are frozen in $F$ (to the zero value) and the number of

1782: solutions of $F+e$ is the same as the one of $F$. Hence the average

1783: decrease in entropy is

1784: \begin{equation} \label{evolsh}

1785: N\, s_h^* (\alpha + \frac 1N) - N \, s_h^*(\alpha) \simeq 

1786: \frac{ds ^*_h}{d\alpha} = -(1-b^3) \; \ln 2 \ .

1787: \end{equation} 

1788: The same differential equation can be obtained by differentiating 

1789: (\ref{commentxor1}). With the limit condition

1790: $s_h^*(\alpha\to\infty)=0$ we obtain back the correct expression for 

1791: the average entropy of homogeneous systems. The entropy is equal to

1792: $(1-\alpha)\ln 2$ at $\alpha=\alpha_c$, and becomes smaller when the

1793: ratio decreases. This shows that the solution $b=0$ must be

1794: preferred in this regime to the metastable $b>0$ solution.

1795: We conclude that the cavity assumption leads to sensible results 

1796: for homogeneous systems at all ratios

1797: $\alpha$.

1798: 

1799: 

1800: 

1801: In full systems the sign $\sigma$ entering (\ref{expressu}) takes $\pm 1$

1802: values with equal probabilities. We thus expect $p(h)$ to be an even 

1803: distribution, and $a_+=a_-=\frac 12 (1-a_0)$. 

1804: Remark that a solution with  $a_0<1$ cannot exist in the satifiable

1805: phase. It would allow two added equations to impose opposite non zero biases

1806: to the new variable {\em i.e.} to constraint this variable to take

1807: opposite values at the same time.

1808: Given $a_0$ we calculate from (\ref{defph}) the probability that the

1809: field vanishes,

1810: \begin{equation}\label{e1a}

1811: p(0)= e^{-3\,\alpha (1-a_0)} \sum _{\ell =0} ^\infty \left[ 

1812: \frac {3\alpha}2 (1-a_0)\right]^{2\ell} \frac 1{\ell !^2}

1813: \end{equation}

1814: and, in turn, derive from (\ref{conda0}) a self-consistent equation 

1815: for $a_0$.

1816: Numerical investigations

1817: show that $a_0=1$ is the unique solution for $\alpha < \alpha_T=

1818: 1.167$. When $\alpha > \alpha_T$ there appears another solution with

1819: $a_0<1$. 

1820: The clustering and Sat/Unsat transitions are totally absent. 

1821: This result, incompatible with the exact picture of random 3-XORSAT

1822: exposed in Section \ref{seccluster}, shows that the simple cavity 

1823: hypothesis does not hold for full systems. 

1824: 

1825: \subsection{Spontaneous symmetry breaking between clusters}

1826: \label{secpin}

1827: 

1828: In the clustered phase variables are known to be strongly correlated

1829: and the cavity assumption has to be modified. Actually from what we

1830: have done above in the homogeneous case we guess that the independence

1831: condition still holds if we can in some way restrict the whole space

1832: of solutions to one cluster. To do so we explicitely

1833:  break the symmetry between clusters as follows\cite{Pa86,Mon95}.

1834: 

1835: Let $S_i^*,i=1,\ldots, N$ be a reference solution of a full

1836: satisfiable system $F$, and $F_h$ the corresponding homogeneous

1837: system. We define the local gauge transform $S_i \to \hat S_i= S_i \times

1838: S_i^*$.  $\{S\}$ is a solution of $F$ if and only if $\{\hat S\}$ is a

1839: solution of $F_h$. As the cavity assumption is correct for the

1840: homogeneous system we obtain the distribution of fields $\hat

1841: h_i \ge  0$ from (\ref{defph}). Gauging back to the original spin 

1842: configuration gives us the fields

1843: \begin{equation}\label{gauge}

1844: h_i = S_i^*\times \hat h_i \ .

1845: \end{equation}

1846: It turns out that the above fields depend only on the cluster to which

1847: belong the reference solution.  Indeed 

1848: for the fraction $1-b$ of the non frozen spins, $\hat h_i = 

1849: h_i=0$. For the remaining fraction $b$ of spins in the backbone 

1850: $\hat h_i >  \frac 12$ and $S_i^*$ has a unique value for all

1851: solutions in the cluster (Section \ref{seccluster}). Hence

1852: the fields $h_i$ are a function of cluster $(c)$ containing $\{S^*\}$,

1853: and will be denoted by $h_i ^{(c)}$.

1854: 

1855: What modification has to be brought to the cavity assumption of

1856: Section \ref{seccav1} is now clear. Given a subset $I$ of the spins

1857: with configuration $S_I$ we define $E_F^{(c)}(S_I)$ as the GS

1858: energy over configurations in the cluster $(c)$. Then the cavity

1859: assumption is correct (spins in $I$ are uncorrelated) and $E_F^{(c)}$

1860: define the fields $h_i^{(c)}$. How do we perform this restriction in

1861: practice? A natural procedure is to break the symmetry between

1862: clusters in an explicit manner by adding a small coupling to the

1863: reference solution \cite{Pa86,Mo95}. 

1864: Remark that symmetry was broken (naturally but explicitly!) in the case of

1865: homogeneous systems when we looked for a distribution $p(h)$ with

1866: support on positive fields only. It is a remarkable feature of XORSAT 

1867: (rather unique among disordered systems) that symmetry between 

1868: disordered clusters can be broken in a constructive and simple way.

1869: 

1870: The main outcome of the above discussion is that the field attached to

1871: variable $i$ is not unique, but depends on the cluster $(c)$. We define

1872: the distribution $p_i(h)$ of the fields attached to variable $i$ over

1873: the clusters (with uniform weights since all clusters contain the same

1874: number of solutions) \cite{Mon98}. The naive cavity assumption

1875: corresponds to 

1876: \begin{equation}

1877: p_i(h) = \delta _{h-h_i} \ .

1878: \end{equation}

1879: In presence of many

1880: clusters $p_i(h)$ is not highly concentred. From (\ref{gauge}) and the fact

1881: that $S_i^*=\pm 1$ depending on the cluster from which we pick up the 

1882: reference solution we find that  

1883: \begin{equation}\label{ef}

1884: p_i(h) = \frac 12 \left[ \delta _{h-\hat h_i} + \delta _{h+\hat h_i}\right] \ .

1885: \end{equation}

1886: As $\hat h_i$ is itself randomly distributed we are led to introduce

1887: the distribution ${\cal P}$

1888: of the field distributions $p_i(h)$. This mathematical object, ${\cal

1889: P}(p(h))$, is the order parameter of the cavity theory in the

1890: clustered phase \cite{Mon98,Me01y}.

1891: 

1892: \subsection{Distribution of field distributions} \label{seccavrsb}

1893: 

1894: Let us see how ${\cal P}$ can be obtained within the one-more variable

1895: approach of Section \ref{seccav1}. A new equation  

1896: contains two variables $S_i,S_j$ from $F$, with fields

1897: $h_i^{(c)},h_j^{(c)}$ in each cluster $(c)$. The bias $u$ 

1898: is a deterministic 

1899: function of those two fields for each cluster (\ref{expressu}). We

1900: define its distribution over clusters $\rho$. As $u$ can take three

1901: values only and $\rho$ is an even distribution due to the randomness of 

1902: the second member of the new equation  we may write 

1903: \begin{equation} \label{db}

1904: \rho (u ) = (1-\varphi_{ij}) \; \delta_u + \frac{\varphi_{ij} }2\;

1905: \big( \delta _{u-\frac 12} +  \delta _{u+\frac 12} \big) \ .

1906: \end{equation}

1907: The weight $\varphi$ is a random variable which varies from pair

1908: $(ij)$ to pair. 

1909: 

1910: What is the probability distribution {\sc p}$(\varphi)$ of $\varphi$?

1911: Either the two variables in the pair belong to the backbone and they

1912: are frozen in all clusters; then $u$ will be non zero and

1913: $\varphi=1$. Or one (at least) of the two variables is not frozen and $u=0$

1914: in all clusters, giving $\varphi=0$. We may write

1915: \begin{equation}

1916: \mbox{\sc p}(\varphi ) = (1-w)\; \delta _{\varphi} + w\;

1917: \delta_{\varphi-1} \ . 

1918: \end{equation}

1919: From the above argument we expect $w=b^2$. Let us derive this result.

1920: 

1921: Assume we add $\ell\ge 1$ equations to our system.  For each one of

1922: those equations a bias $u^{j}$ is drawn randomly according to distribution 

1923: (\ref{db}). Denote by $m(\le

1924: \ell)$ the number of those equations with parameter $\varphi=1$; $m$

1925: is binomially distributed with probability $w$ among $\ell$.

1926: Then $\ell-m$ biases are null, and $m$ biases are not equal to zero.

1927: For the formula to remain satisfiable the non-zero biases must be all

1928: positive or negative \cite{Me01y}, see  Section 

1929: \ref{seccavapp}. Hence the distribution of the field on the new variable 

1930: is

1931: \begin{equation}\label{dispm}

1932: p^m(h) = \frac 12 \left[ \delta_{h-\frac m2} + \delta _{h+\frac m2}\right] \ ,

1933: \end{equation}

1934: in agreement with the expected form (\ref{ef}).

1935: The upperscript $m$ underlines that field distributions with non zero

1936: probability are can be labelled by an integer $m$; they define

1937: a countable set and the distribution ${\cal P}$ can be defined

1938: as a discrete probability ${\cal P}_m$ over the set of positive

1939: integers $m$.

1940: The probability ${\cal P}_m$ of distribution (\ref{dispm}) 

1941: is the convolution of 

1942: binomial distribution for $m$ at fixed $\ell$ with the Poisson

1943: distribution over $\ell$,

1944: \begin{equation}\label{dispmp}

1945: {\cal P} ^m = e^{-3\,\alpha\, w} \frac{(3\,\alpha\, w)^m

1946: }{m!} \ .

1947: \end{equation}

1948: Identities (\ref{dispm},\ref{dispmp}) fully determine the distribution of field

1949: distributions in term of a single parameter, $w$.

1950: 

1951: To close the self-consistency argument consider the two variables in

1952: $F$ in, say, the first added equation. Call $h,h'$ their fields,

1953: distributed according to $p_m(h), p_{m'}(h')$ for

1954: some $m,m'$. The bias  created onto the new variable will be non

1955: zero if $h$ and $h'$ may both take non zeros value in some clusters,

1956: that  is, if $m$ and $m'$ are not equal to zero. 

1957: This translates into the mathematical identity 

1958: \begin{equation}

1959: w = \sum _{m\ge \frac 12,m'\ge \frac 12} 

1960: {\cal P}_m\; {\cal P}_{m'} = \big(1-{\cal

1961:   P}_0 \big)^2 = \big(1 - e^{-3\,\alpha\, w}\big)^2

1962: \end{equation}

1963: from (\ref{dispmp}).

1964: The above equation coincides with (\ref{eqb}) for $w=b^2$. Notice that

1965: $w$ is equal to the probability $a_+$ 

1966: that the bias is non zero in the homogeneous

1967: case (\ref{uk}), in agreement with the discussion of Section \ref{secpin}.

1968: 

1969: It is easy to find back the expressions for the entropies of clusters,

1970: $s_{cluster}$, and solutions in a cluster, $s_{in}$, given in Section

1971: \ref{seccluster}.  As for the latter entropy the argument leading to

1972: (\ref{evolsh}) can be repeated, with the modification that the second

1973: member of the added equation is not necessarily zero but the value it

1974: should have for the equation to be satisfied when all three variables

1975: are frozen. Hence (\ref{evolsh}) holds with $s_{h}^*$ replaced with

1976: $s_{in}$. As for the entropy of clusters the same argument again tells us

1977: that, on average, half of the clusters will disappear when the 

1978: three variables are frozen and the second member of the equation is

1979: randomly chosen. Therefore

1980: \begin{equation}\label{sc2}

1981: \frac{d s_{cluster}}{d\alpha} = -b^3\, \ln 2 \ ,

1982: \end{equation}

1983: in agreement with equations (\ref{scluster},\ref{eqb}).

1984: Summing differential equations (\ref{sc2}) and (\ref{evolsh}) 

1985: for $s_{cluster}$ and $s_{in}$ respectively shows that the total

1986: entropy of solutions is $(1-\alpha)\, \ln 2$ (Section \ref{seccluster}).

1987: 

1988: 

1989: 

1990: % algorithms

1991: 

1992: \section{Dynamical phase transitions and search algorithms}

1993: 

1994: The deep understanding of the statistical properties of

1995: 3-XORSAT makes this problem a valuable benchmark for assessing the 

1996: performances of

1997: various combinatorial search algorithms. At first sight, the idea

1998: seems rather odd since 3-XORSAT is a polynomial problem. 

1999: Interestingly most of the search procedures devised to deal with 

2000: NP-complete problems e.g. SAT have poor performances {\em i.e.}

2001: take exponentially long average running times

2002: on XORSAT above some algorithmic-dependent critical ratio ... 

2003: The purpose of this Section is to present two algorithms exhibiting

2004: such a dynamical phase transition, and the techniques required for 

2005: their analysis.

2006: 

2007: \subsection{Random WalkSAT (RWSAT): definition, worst-case bound}

2008: 

2009: The first algorithm we consider is the Random WalkSAT (RWSAT) 

2010: algorithm introduced by Papadimitriou \cite{Pa92}.

2011: RWSAT is based on the observation that a violated equation can be satisfied 

2012: through negation of one of its variables:

2013: 

2014: {\sc

2015: \begin{itemize}

2016: \item   Start from a randomly chosen configuration of the 

2017: variables. Call energy the number $E$ of unsatisfied equations.

2018: \item While $E\ge 1$;   

2019: \begin{itemize}

2020: \item pick up uniformly at random one of the $E$ unsatisfied equations;

2021: \item  pick up  uniformly at random one of its 3 variables;

2022: \item  negate the value of this variable, update $E$;

2023: \end{itemize}

2024: \item Print 'Satisfiable', and Halt.

2025: \end{itemize}

2026: }

2027: 

2028: \vskip .3cm

2029: Notice that, as a result of the negation, some equations 

2030: that were satisfied may become violated. Therefore the energy is

2031: not guaranteed to decrease with the number of steps of the algorithm.

2032: RWSAT is able to escape from local minima

2033: of the energy landscape, and is {\em a priori} capable of better 

2034: performances. From the other hand, RWSAT may run forever...

2035: A major question is how long should the algorithm be running 

2036: before we stop thinking that the studied system has solutions

2037: hard to find and get some confidence that there is really no 

2038: solution. 

2039: 

2040: This question was addressed by Sch\"oning \cite{Sc00}, who

2041: showed that RWSAT could easily be used as a one-sided

2042: randomized algorithm \cite{Mo95}\footnote{Sch\"oning's 

2043: original work was devoted to

2044: the analysis of RWSAT on K-SAT, but his result holds for K-XORSAT

2045: too.}. Consider one instance of 3-XORSAT and run RWSAT for $3N$ steps

2046: from a randomly chosen configuration of variables. Choose again a

2047: random initial configuration and run RWSAT another $3N$ steps, and so on

2048: ... The probability that no solution has been found after $T$ 

2049: repetitions of this procedure though the formula is satisfiable is

2050: \begin{equation} \label{ressch}

2051: p_{SAT} \le \exp \left( - T \times \left(\frac 34\right)^{N +o(N)} \right) \ .

2052: \end{equation}

2053: Hence we obtain a probabilistic proof that the instance is not

2054: satisfiable if the algorithm has run unsuccessfully for more than

2055: $(\frac 43)^N$ sets of $3N$ steps. It must be clear that this result

2056: holds for any instance, no assumption being made on the distribution

2057: of formulas. The probability appearing in (\ref{ressch}) is on the

2058: random choices done by RWSAT and the choices of the restart

2059: configurations for a fixed formula.

2060: 

2061: The proof of (\ref{ressch}) can be sketched as

2062: follows. Assume that the formula is satisfiable, and called $X^*$ one

2063: of its solutions. Consider now the (extensive)  Hamming

2064: distance between the solution and the configuration $X$ of variables 

2065: produced by RWSAT at some instant. After each step only one variable is

2066: changed so $D$ changes into $D+1$ (bad move) or $D-1$ (good move).

2067: Call $x,y,z$ the variables in the equation which was not satisfied by

2068: $X$. One or three of those variables have opposite values in $X^*$.

2069: In the latter case the flip is always a good move;

2070: in the former case the good move happens with probability  $\frac 13$ and

2071: a bad move with probability $\frac 23$. On the overall the probability of

2072: a good move is $\frac 13$ at least. 

2073: 

2074: Think of $D$ has the position of a random walker on the $[0;N]$ 

2075: segment. Initially the position of the walker is a binomial variable, 

2076: centered in $\frac N2$. At each step the walker moves to the left 

2077: with probability $\frac 13$, and to the right with probability $\frac 23$. 

2078: We look for the probability $\rho$ that the walker is absorbed by the 

2079: boundary $D=0$ after  $S$ steps. A standard calculation shows that

2080: $\rho$ is maximal for $S=3N$, with the value $\rho\simeq

2081: (\frac 34)^N$. After $T$ repetitions the probability of not having been

2082: absorbed is $(1-\rho)^T<\exp(-\rho\,T)$, hence (\ref{ressch}).

2083: The proof can be easily extended to $K$-XORSAT with higher values of $K$.

2084: The number of repetitions necessary to prove unsatisifiability

2085: scales as $(\frac{2(K-1)}K)^N$; it is essentially equal to $2^N$ for large 

2086: $K$, showing that RWSAT does not beat exhaustive search in this limit. 

2087: 

2088: \subsection{Dynamical transition of RWSAT on random XORSAT instances}

2089: \label{secrwsat}

2090: 

2091: \begin{figure}

2092: \begin{center}

2093: \includegraphics[width=110pt,angle=0]{fig521.eps}

2094: \hskip.5cm

2095: \includegraphics[width=120pt,angle=0]{fig522.eps}

2096: \end{center}

2097: \caption{Fraction $e$ of unsatisfied equations as a function of time $t$

2098: (number of steps divided by $N$) during the operation of RWSAT on a

2099: random 3-XORSAT formula at ratio $\alpha =0.2$ (left)

2100: and $\alpha =0.4$ (right) with $N=10^3$ (dotted), $10^4$ (dashed), $10^5$

2101: (full curve) variables.   Note the difference

2102: of horizontal scales between the two figures. Inset:  blow

2103: up of the $t\in[9.5;10.5]$ region; the amplitude of fluctuations around 

2104: the plateau decreases with increasing size $N$. }

2105: \label{wsat_phen}

2106: \end{figure}

2107: 

2108: Result (\ref{ressch}) is true for any instance; what is the typical

2109: situation for random systems? Numerical experiments 

2110: indicate that there is critical value of the ratio of

2111: equations per variables, $\alpha_E\simeq 0.33$, hereafter referred to 

2112: as dynamical  threshold, separating two regimes:

2113: \begin{itemize}

2114: \item for $\alpha < \alpha _E$, RWSAT generally finds a solution very

2115: quickly, namely with a number of flips growing linearly with the

2116: number of variables $N$\footnote{A proof of this statement was

2117:   obtained by \cite{Al02} for the random SAT model.}.  

2118: Figure~\ref{wsat_phen} shows the plot of

2119: the fraction $e$ of unsatisfied clauses as a function of the time

2120: (number of steps) $T$ for one randomly drawn system with ratio

2121: $\alpha=0.2$ and $N=500$ variables. The curve shows a fast decrease from

2122: the initial value ($e(T=0)=\frac 12$ 

2123: independently of $\alpha$ for large values of $N$, but deviations can be

2124: found at small sizes, see Figure~\ref{wsat_phen}) down to zero on a time scale

2125: of the order of $N$\footnote{This decrease characterises the overall operation

2126: of RWSAT. A precise look at the $e(T)$ curve reveals that the energy $e$

2127: may occasionally increase.}. The resolution time $T_{res}$ depends

2128: both on the system of equations under consideration and the choices of

2129: the algorithm; its average value scales as 

2130: \begin{equation}

2131: \langle T_{res} \rangle  =  N\, t _{res} + o(N) \ .

2132: \end{equation}

2133: where $t_{res}$ is an increasing function of 

2134: $\alpha$\footnote{On intuitive grounds, as a step of the algorithm can satisfy 

2135: $\theta(1)$ equations at a time, we expect the average value of 

2136: $T_{res}$ to be of the order of the number $M$ of equations at least. 

2137: Thus $t_{res}$ should grow at least linearly with $\alpha$. Experiments

2138: shows that the growth is in fact more than linear.}.

2139: 

2140: \item for systems with ratios of equations per variable in the 

2141: $\alpha_E<\alpha <\alpha_c$ range, the initial relaxation regime taking place 

2142: on the $O(N)$ time scale does not allow RWSAT

2143: to reach a solution (Figure~\ref{wsat_phen}B). The fraction

2144: $e$ of unsat equations then fluctuates around some plateau value

2145: $e_{plateau}$ 

2146: for a very long time. Fluctuations are smaller and smaller (and the height of

2147: the plateau better and better defined) as the size $N$ increases.

2148: As a result of fluctuations, the fraction $e$ of unsatisfied equations

2149: may temporarily either increase or decrease. When a fluctuation happens to 

2150: drive RWSAT to $e=0$, a solution is found and the algorithm stops.

2151: The corresponding resolution time, $T_{res}$, is stochastic;

2152: numerical experiments for different sizes $N$ indicate that its

2153: expectation value scale as

2154: \begin{equation} \label{deftaures}

2155: \langle T_{res} \rangle= \exp( N\, \tau _{res}+o(N)) \ .

2156: \end{equation}

2157: where the coefficient $\tau_{res}$ is an increasing function of $\alpha$.

2158: The plateau energy $e_{plateau}$  and the logarithm $\tau _{res}$ of

2159: the resolution time are shown in Figure~\ref{wsat_plateau}.  

2160: \end{itemize}

2161: 

2162: \begin{figure}

2163: \begin{center}

2164: A\includegraphics[width=110pt,angle=-90]{fig523.eps}

2165: B\includegraphics[width=110pt,angle=-90]{fig524.eps}

2166: \end{center}

2167: \caption{Fraction $e _{plateau}$ of unsatisfied equations on 

2168: the plateau  ({\bf A}) and logarithm

2169: $\tau _{res}$ of the average resolution time divided by $N$ ({\bf B})  as a 

2170: function of the ratio $\alpha$ of equations per variable. 

2171: Diamonds are the output of numerical experiments, and have been obtained 

2172: through average of data from simulations over 1,000 systems and runs of RWSAT

2173: for various sizes $N$, and  extrapolation to $N\to\infty$ \cite{Se03}.

2174: Full lines are theoretical approximations (\ref{eplateau}),(\ref{trestheo}).} 

2175: \label{wsat_plateau}

2176: \end{figure}

2177: 

2178: Notice that the dynamical threshold $\alpha_E$ 

2179: above which the plateau energy is positive is strictly smaller than the

2180: critical threshold $\alpha_c\simeq 0.918$, where systems go from 

2181: satisfiable with

2182: high probability to unsatisfiable with high probability.  In the

2183: intermediate range $\alpha_E<\alpha<\alpha_c$, systems are almost

2184: surely satisfiable but RWSAT needs an exponentially large time to

2185: prove so. The reason is that  RWSAT remains trapped at a high energy level 

2186: (plateau of Figure~\ref{wsat_plateau}) for an exponentially large

2187: time. The emergence of metastability can be qualitatively studied with simple 

2188: tools we now expose. 

2189: 

2190: \subsection{Approximate theory for the metastable plateau and the escape time}

2191: \label{secescape}

2192: 

2193: Assume that after $T$ steps of the algorithm the energy (number of unsatisfied

2194: equations) is $E_T\ge 1$. Then pick up an unsatisfied equation, say, 

2195: $C$, and a variable in $C$, say, $x$, and flip it. The energy after the flip is

2196: \begin{equation} \label{evole}

2197: E_{T+1} = E_T -1 -U+S \ ,

2198: \end{equation}

2199: where $S$ (respectively $U$) is the number of equations including $x$ 

2200: which were satisfied (resp. unsatisfied after exclusion of equation $C$) 

2201: prior to the flip. $S$ and $U$ are random variables with binomial 

2202: distributions,

2203: \begin{eqnarray}

2204: \mbox{Proba}[U] &=& {E_T-1 \choose U} \left( \frac 3N\right)^U

2205: \left( 1-\frac 3N\right)^{E_T-1-U} \ , \nonumber \\

2206: \mbox{Proba}[S] &=& {M-E_T \choose S} \left( \frac 3N\right)^S

2207: \left( 1-\frac 3N\right)^{M-E_T-S} \ . 

2208: \end{eqnarray}

2209: where the probabilities are intended over the formula content.

2210: Taking the average evolution equation (\ref{evole}) we obtain

2211: \begin{equation} \label{evole2}

2212: \langle E_{T+1}\rangle =\langle E_T\rangle -1 -\frac 3N \big(

2213:  \langle E_T\rangle -1\big) +\frac 3N \big( M - \langle E_T\rangle\big) \ .

2214: \end{equation}

2215: The above equation is exact. It is now tempting to iterate it with time, from

2216: the initial condition $\langle E_{T=0}\rangle = \frac M2$. This is what we do

2217: hereafter but one should realize that this procedure is not

2218: correct from a mathematical standpoint. The catch is that one is allowed to

2219: average over the formula only once, and certainly not at each time step 

2220: of the algorithm. Evolution

2221: equation (\ref{evole2}) amounts to redraw randomly the instance

2222: at each time step, conditioned to the energy. This approximation

2223: nevertheless allows

2224: us to write down a simple equation for $\langle E_T\rangle$, which

2225: captures much of the true behaviour of RWSAT.

2226: 

2227: The next step in our analysis is the large size, large time limit. As

2228: the energy can typically change by a quantity of the order of unity 

2229: in one time step we expect the fraction of unsatisfied equations to

2230: vary of a time scale of the order of $N$,

2231: \begin{equation}\label{evola}

2232: \langle E_T\rangle = M\; e\left( \frac TM =t\right) \ ,

2233: \end{equation}

2234: for some smooth function $e(t)$ of the reduced time $t$.

2235: Finite difference equation (\ref{evole2}) turns into a differential equation 

2236: after insertion of (\ref{evola}),

2237: \begin{equation}\label{evole3}

2238: \frac{de}{dt} = - 1 + 3\alpha \, ( 1 -2\, e) \ ,

2239: \end{equation}

2240: with the initial condition $e(0)=\frac 12$. Clearly (\ref{evole3}) makes

2241: sense as long as $e>0$; if $e$ vanishes the algorithm stops.

2242: Resolution of (\ref{evole3}) shows the following scenario. 

2243: If $\alpha$ is smaller than

2244: \begin{equation}

2245: \alpha _E = \frac 13 \ ,

2246: \end{equation}

2247: the fraction $e$ of unsatisfied equations quickly decreases, and

2248: vanishes at some time $t_{res}(\alpha)$. This regime corresponds to

2249: a successfull action of RWSAT in a $O(N)$ number of steps.

2250: $t_{res}$ is an increasing function of $\alpha$ which diverges as

2251: $\alpha \to\alpha_E$. Above this critical ratio $e$ shows a different

2252: behaviour: after a decreasing transient regime $e$ saturates to a

2253: positive plateau value 

2254: \begin{equation}\label{eplateau}

2255: e_{plateau} (\alpha) = \frac 12 \left( 1 - \frac {\alpha_E}{\alpha}\right) 

2256: \ .

2257: \end{equation}

2258: The value of the plateau energy is compared to numerics in Figure

2259: \ref{wsat_plateau}A. The agreement on the location of the dynamical 

2260: threshold $\alpha_E$ as well as the plateau energy are satisfactory.

2261: 

2262: The remaining point is to understand how RWSAT finally finds a solution 

2263: when $\alpha >\alpha_E$. The above theory, based on taking the 

2264: $N\to\infty$ limit first, washes out the fluctuations of the energy around

2265: its metastable value, of crucial importance for resolution

2266: \cite{Se03}. To take into account these fluctuations let us define

2267: the probability $Q_{plateau}(E)$ that the energy takes value $E$ 

2268: in the plateau regime of Figure \ref{wsat_phen}B. A stationary 

2269: distribution is well defined if we discard the initial transient

2270: regime (choose large $t$) and collect values for $E$ on exponentially 

2271: large--in--$N$ time scales. The procedure is standard in the

2272: study of long-time metastable states.

2273: 

2274: Within our draw-instance-at-each--step approximation we may write

2275: a self-consistent equation for the stationary distribution

2276: of energies,  

2277: \begin{equation} \label{statq}

2278: Q_{plateau}(E) = \sum _{U,S} \mbox{Proba}[U]\, \mbox{Proba}[S]\,

2279: \, Q_{plateau}(E+1+U-S) 

2280: \end{equation}

2281: where the meaning of $U,S$ was explained right

2282: after (\ref{evole}). From Section \ref{secrwsat} we expect fluctuations

2283: to decreases sharply with the system size. A reasonable guess for the

2284: scaling of the distribution with $M$ is 

2285: \begin{equation}

2286: Q_{plateau}(E) = 

2287: \exp\left[ -M\; \omega \left(\frac EM =e\right) +o(M)\right]

2288: \end{equation}

2289: where $\omega$ is the rate function associated to the fraction

2290: of unsatisfied equations. Plugging the above Ansatz into (\ref{statq})

2291: and taking the large $M$ limit we find that $\omega$ fulfills the

2292: following differential equation 

2293: \begin{equation}\label{pde1}

2294: F\left( \frac{\partial \omega}{\partial e},e\right) =0\ ,

2295: \end{equation}

2296: where $F(x,y)=3\alpha y(e^{-x}-1) +3\alpha (1-y) (e^x-1) -x$.

2297: This equation has to be solved with the condition 

2298: $\omega(e_{plateau}) =0$. 

2299: 

2300: An analytical solution can be found for (\ref{pde1}) when we restrict

2301: to the vicinity of the dynamical transition {\em i.e.} to small

2302: values of $\omega$. 

2303: Expanding $F$ to the second order in its first argument and solving

2304: (\ref{pde1}) we obtain

2305: \begin{equation} \label{ome}

2306: \omega(e) \simeq 2 \, (e - e_{plateau})^2\ , 

2307: \end{equation}

2308: where $e_{plateau}$ is defined in (\ref{eplateau}).

2309: 

2310: What happens when time increases is now clear. Assume we have run RWSAT

2311: up to time $t\sim e^{M\tau}$. Then configurations with energy $e$ such that

2312: $\omega (e) <\tau$ have been visited many times and are 'equilibrated'

2313: with probability (\ref{statq}), (\ref{ome}). Configurations with

2314: energies outside the band $e_{plateau} \pm \sqrt{\tau /2}$ are not

2315: accessible. When the time scales reaches

2316: \begin{equation} \label{trestheo}

2317: \tau _{res} = \omega(0) \simeq \frac 12 \left( 1 - \frac 1{3\alpha}\right)^2  

2318: \ ,

2319: \end{equation}

2320: zero energy configurations are encountered, and RWSAT comes to a stop.

2321: The agreement between the theoretical estimate (\ref{trestheo}) and

2322: the numerical findings (\ref{deftaures}) visible in Figure \ref{wsat_plateau}B

2323: is acceptable in regard to the crudeness of the approximation done.

2324: 

2325: \subsection{Davis-Putnam-Loveland-Logemann (DPLL) algorithm}\label{secdpll}

2326: 

2327: The second procedure is the Davis-Putnam-Loveland-Logemann

2328: (DPLL) algorithm \cite{dpll}. 

2329: Contrary to RWSAT DPLL can provide exact proofs for

2330: unsatisfiability. The procedure, widely used in practice, is based on 

2331: the trial-and-error principle. Variables are assigned

2332: according to some heuristic rule (split step), and

2333: equations involving those variables simplified. If  an

2334: equation involving a single variable (unit-equation)  appears

2335: its variable is chosen accordingly prior to any other 

2336: heuristic assignment

2337: (unit-propagation).  If a contradiction is found (two opposite

2338: unit-equations) DPLL backtracks to the last heuristically assigned 

2339: variable, flips it, and resumes the search process. 

2340: The procedure halts either when all equations have been

2341: satisfied (a solution is then found), 

2342: or when all possible values for the variables have been tried in vane and

2343: found to be contradictory (a proof of unsatisfiability

2344: is then obtained). 

2345: 

2346: DPLL can be described as a recursive 

2347: function of the variable assignment $A$. 

2348: Given a system $S$  DPLL is first called

2349: with the empty assignment $A=\emptyset$:

2350: \vskip .3cm

2351: {\sc Procedure DPLL[$A$]

2352: \begin{itemize}

2353: \item Let $S_A$ be what is left from $S$ given variable assignment $A$;

2354: \item if $S_A$ is empty,  Print `Satisfiable'; Halt;

2355: \item If $S_A$ contains a violated equation, Print `Contradiction', Return; 

2356: {\em (backtracking)}  

2357: \item Otherwise, let $U$ be the set of unit-equations in $S_A$;     

2358: \begin{itemize}

2359: \item If $U \ne \emptyset$, pick-up one of the equations in

2360: $U$, say, $e$, and call DPLL[A$\cup \{e\}$];

2361: {\em (unit-propagation)}

2362: \item if $U=\emptyset$, choose a not-yet-assigned

2363: variable, say, $x$, and its value $v$ according to some heuristic rule, and

2364:  call DPLL[A$\cup \{x=v\}$], then

2365: DPLL[A$\cup \{x=\bar v\}$]; {\em (variable splitting)}

2366: \end{itemize}

2367: \end{itemize}

2368: }

2369: 

2370: \vskip .3cm

2371: Rules for assigning variables in the absence of unit-equations are heuristic

2372: in that they aim at doing good assumptions {\em i.e.} diminishing as much 

2373: as possible the search process to come from limited information

2374: about the current system of equations. Of course, perfect heuristic 

2375: do exist:  trying all possible values for not-yet-assigned variables

2376: would ensure that no wrong guess is ever done!  But the time required 

2377: would be exponentially long. In practice, heuristics have to make 

2378: their decision in polynomial time. Two simple splitting heuristics 

2379: are:

2380: \begin{itemize}

2381: \item[$\diamond$] \underline{UC:} 

2382: choose at random and uniformly any unset variable,

2383: and assign it to 0 or 1 with equal probabilities ($\frac 12$). 

2384: \item[$\diamond$]  \underline{GUC:}

2385: choose at random and uniformly any equation with minimal length {\em i.e.}

2386: involving 2 variables if any, or 3 variables otherwise. Pick up 

2387: at random and uniformly  one its variable, and assign it to 0 or 1 

2388: with equal probabilities ($\frac 12$). 

2389: \end{itemize}  

2390: UC, which stands for unit-clause \cite{Ch90}, 

2391: amounts to make a random guess and is the simplest possible heuristic.

2392: GUC (Generalized UC) 

2393: is more clever: each time a split is done from an equation with 2 

2394: variables, this equation is turned into a unit-equation, and

2395: eliminated through unit-propagation. 

2396: In the following, we call DPLL-UC and DPLL-GUC the variants of

2397: DPLL based on the UC and GUC heuristics respectively.

2398: 

2399: \begin{figure}

2400: \begin{center}

2401: \includegraphics[width=110pt,angle=-90]{fig541.eps}

2402: \hskip .3cm 

2403: \includegraphics[width=110pt,angle=-90]{fig542.eps}

2404: \end{center}

2405: \caption{Median number of splits required by DPLL 

2406: with the UC (left) and GUC (right) heuristics as a function of the ratio 

2407: $\alpha$, and for $N=20,40,60$ variables (from bottom to

2408: top). Data have been extracted from the resolution

2409: of 10,000 randomly drawn systems; continuous lines are guidelines for the 

2410: eye. Note the difference of (logarithmic) scale between UC and GUC

2411: curves showing that DPLL-GUC is much more efficient than DPLL-UC. 

2412: The polynomial/exponential transition is located at ratios

2413: $\alpha_E=\frac 23$ and $\alpha_E=0.7507...$ for UC and GUC respectively.}

2414: \label{dpllxorsat}

2415: \end{figure}

2416: 

2417: A  measure of the computational effort required

2418: by DPLL is the number $T_{split}$ 

2419: of variable splittings. This number varies from

2420: system to system (at fixed number $N$ of variables and ratio $\alpha$), and

2421: from run to run of DPLL due to the stochasticity introduced by

2422: the heuristic rule. The outcome of numerical experiments for

2423: the  median number of splits\footnote{The median is more

2424: representative of the typical value of the number of splits than the

2425: expectation value, since the latter may be dominated by huge and

2426: unlikely samples, see discussion of Section \ref{secselfav}.}.

2427: For a given size $N$ $T_{split}$ shows a maximum located

2428: around $\alpha\simeq \alpha_c$. If one fixes $\alpha$ 

2429: $T_{split}$ is an increasing function of the size $N$; 

2430: numerical data 

2431: support the existence of a dynamical threshold, $\alpha_E$, separating

2432: linear and exponential scalings in $N$,

2433: \begin{equation} \label{mesure}

2434: T_{split}\sim \left\{\begin{array} {c c c}

2435:  N\, t _{split} + o(N) &\hbox{\rm if} & \alpha < \alpha_E \\ 

2436: \exp(  N\, \tau _{split}+o(N) ) &\hbox{\rm if} & \alpha > \alpha_E 

2437: \end{array} \right. \ ,

2438: \end{equation}

2439: where $t_{split}$ and $\tau _{split}$ are functions of the ratio $\alpha$.

2440: The value of the dynamical threshold can be derived from theoretical

2441: calculations shown in Section \ref{secuc} and is equal to $\alpha_E=\frac 23$ 

2442: and $\alpha_E\simeq 0.7507...$ for UC and GUC heuristics respectively. 

2443: Three dynamical regimes are therefored identified \cite{Co01,Ac02}:

2444: \begin{itemize}

2445: \item \underline{Linear \& satisfiable phase ($\alpha<\alpha_E$):} systems with

2446: small ratios are solved with essentially no backtracking. A solution

2447: is found after $O(N)$ splits. 

2448: \item \underline{Exponential\& satisfiable phase ($\alpha_E<\alpha

2449: <\alpha_c$):}

2450: systems with ratios slightly below threshold have solutions, but 

2451: DPLL generally requires an exponential number of splits to find

2452: one of them. An explanation for this drastic breakdown of performances 

2453: will be given in Section \ref{secuc}.

2454: \item \underline{Exponential \& unsatisfiable phase ($\alpha>

2455: \alpha_c$):} finally,

2456: finding a proof of unsatisfiability typically requires an exponentially

2457: large number of splits \cite{Ch88}. Note that, as $\alpha$ gets higher 

2458: and higher, each variable assignment affects more and more equations 

2459: (of the order of $\alpha$), and contradictions

2460: are detected earlier and earlier. Rigorous calculations show that

2461: $\tau_{split} \sim \frac 1\alpha$\cite{beame}, 

2462: and the computational effort decreases

2463: with increasing $\alpha$ (Figure~\ref{dpllxorsat}). The median number 

2464: of splits 

2465: is considerably smaller for DPLL-GUC than for DPLL-UC, a result expected 

2466: from the advantages of GUC against UC discussed above.

2467: \end{itemize}

2468:  

2469: \subsection{Linear phase: resolution trajectories in the $2+p$-XORSAT

2470: phase diagram}\label{secuc}

2471: 

2472: Action of DPLL on an instance of 3-XORSAT causes changes to the

2473: numbers of variables and equationses, and thus to the ratio

2474: $\alpha$.  Furthermore DPLL turns equations with 3 variables into

2475: equation with 2 variables. A mixed $2+p$-XORSAT distribution, where $p$ 

2476: is the fraction of 3-equations and $\alpha$ the ratio of the

2477: total number of 2- and 3- equations over the number of variables

2478: can be used to model what remains of the input 

2479: system\footnote{Equations with a single variable

2480: are created too, but are eliminated through unit-propagation. When 

2481: a heuristic assignment has to be made the system is a mixture of

2482: equations with 2 and 3 variables only.}. Repeating the calculations of

2483: Section \ref{secreplicas} for the $2+p$-XORSAT models we derive the

2484: phase diagram of Figure \ref{diag2+p}. The Sat/Unsat critical

2485: line $\alpha_c(p)$ separates the satisfiable from the unsatisfiable 

2486: phases. For $p \le p_0 = \frac 14$ {\em i.e.} to the left of

2487: point T, the threshold line coincides with the percolation 

2488: transition as in the 2-XORSAT model, and is given by 

2489: $\alpha _c(p)=\frac 1{2(1-p)}$. 

2490: For $p>p_0$ an intermediate clustered phase is found as in the

2491: 3-XORSAT model, and the threshold coincides with the vanishing of the

2492: cluster entropy $s_{cluster}$ (Section \ref{seccluster}).

2493: 

2494: 

2495: The phase diagram of 2+p-XORSAT is the natural space in which DPLL

2496: dynamic takes place. An input 3-XORSAT instance with ratio $\alpha$ shows

2497: up on the right vertical boundary of Figure~\ref{diag2+p} as a point of

2498: coordinates $(p=1,\alpha )$. Under the action of DPLL the

2499: representative point moves aside from the 3-XORSAT axis and follows a

2500: trajectory, very much alike real-space renormalization, which 

2501: depends on the splitting heuristic. Trajectories enjoy two essential

2502: features \cite{Ac01}. First the representative point

2503: of the system treated by DPLL does not `leave' the 2+p-XORSAT phase

2504: diagram. In other words, the instance is, at any stage of the search 

2505: process, uniformly distributed from the 2+p-XORSAT 

2506: distribution conditioned to its equation per variable ratio $\alpha$ and 

2507: fraction $p$ of 3-equations. This assumption is not true for

2508: all heuristics of split, but holds for UC and 

2509: GUC\cite{Ch90}\footnote{Analysis of more sophisticated heuristics 

2510: e.g. based on the number of occurences of variables require to handle 

2511: more complex instance distributions \cite{Ka02}.}.

2512: Secondly, the trajectory followed by an instance in the course of

2513: resolution is a stochastic object, due to the randomness of 

2514: the instance and of the assignments done by DPLL. 

2515: In the large size limit ($N\to\infty$) the trajectory becomes 

2516: self-averageing {\em i.e.}

2517: concentrated around its average locus in the 2+p-XORSAT phase 

2518: diagram \cite{Wo95}. We will come back below on this concentration 

2519: phenomenon. 

2520: 

2521: 

2522: \begin{figure}

2523: \begin{center}

2524: \includegraphics[width=160pt,angle=0]{fig551.eps}

2525: \caption{Phase diagram of $2+p$-XORSAT and dynamical trajectories of DPLL.

2526: The threshold line $\alpha_c (p)$ (bold full line) separates sat

2527: from unsat  phases. Departure points for DPLL trajectories are located on the

2528: 3-XORSAT vertical axis with ratios $.4,\frac 23,.8,1.$ from

2529: bottom to top. The arrow indicates the direction of motion along trajectories

2530: parametrized by the fraction $t$ of variables set by DPLL.  For small

2531: ratios $\alpha < \alpha _E$ ($=\frac 23$ for the UC heuristic)

2532: trajectories remain

2533: confined in the sat phase, end in S of coordinates $(0,0)$, where a

2534: solution is found. At $\alpha_E$ the trajectory hits

2535: tangentially the threshold line in T of coordinates $(\frac 14,\frac 23)$. 

2536: When $\alpha >\alpha _E$ the

2537: trajectories intersect the threshold line at some point G (which depends

2538: on $\alpha$), and stops before hitting the

2539: $\alpha _D (p)$ dotted line (\ref{defcon}). 

2540: After massive backtracking DPLL will find a solution;  G corresponds

2541: to the highest node in the search tree.}

2542: \label{diag2+p}

2543: \end{center}

2544: \end{figure}

2545: 

2546: Let $\alpha_0$ denote the equation per variable 

2547: ratio of the 3-XORSAT instance to be solved. 

2548: We call $E_j (T)$ the number of $j$--equations (including $j$ variables)

2549: after $T$ variables have been assigned by the solving procedure. 

2550: $T$ will be called hereafter `time', not to be confused with the 

2551: computational effort. At time $T=0$ we

2552: have $E_3 (0)= \alpha_0 N$, $E_2 (0)=E_1 (0)=0$. 

2553: Assume that the variable $x$ assigned at time $T$ is chosen through

2554: unit-propagation, that is, independently of the $j$-equation content.

2555: Call $n_j(T)$ the number of occurrences of $x$ in 

2556: $j$-equations ($j=2,3$). The evolution equations for 

2557: the populations of 2-,3-equations read

2558: \begin{equation}

2559: E_3 (T+1) = E_3 (T) - n_3(T)\ , \quad E_2(T+1)=E_2(T) - n_2(T)+ n_3(T)   \ .

2560: \label{evolsto}

2561: \end{equation}

2562: Flows $n_2,n_3$ are of course random variables that depend

2563: on the instance under consideration at time $T$, and on the choice of

2564: variable  done by DPLL. What are their distributions? At time $T$

2565: there remain $N-T$ untouched variables; $x$ appears in any of

2566: the $E_j(T)$ $j$-equation with probability $p_j = \frac j{N-T}$, 

2567: independently of the other equations. In the

2568: large $N$ limit and at fixed fraction of assigned variables, 

2569: $t=\frac TN$, the binomial distribution

2570: converges to a Poisson law with mean

2571: \begin{equation} \label{avpois}

2572: \langle n_j \rangle _T=\frac{j\, e_j}{1-t} \qquad \mbox{where} \qquad

2573: e_j= \frac{E_j(T)}N

2574: \end{equation}

2575: is the density of $j$-equations at time $T$. The key remark is that,

2576: when $N\to\infty$, $e_j$ is a slowly varying and non stochastic 

2577: quantity and is a function of the fraction 

2578: $t=\frac TN$ rather than $T$ itself. Let us iterate (\ref{evolsto})

2579: between times $T_0=t\,N$ and $T_0+ \Delta T$ where $1\ll \Delta T

2580: \ll N$ e.g. $\Delta T =O(\sqrt N)$. Then the change $\Delta E_3$ in the number 

2581: of $3$-equations is (minus) the sum of the stochastic variables

2582: $n_j(T)$ for $T=T_0,T_0+1, \ldots,T_0+\Delta T$. As these variables

2583: are uncorrelated Poisson variables with $O(1)$ mean (\ref{avpois}) 

2584: $\Delta E_3$ will be of the order of $\Delta T$, and the change in

2585: the density $e_3$ will be of order of $\Delta T/N \to 0$. Applying

2586: central limit theorem $\Delta E_3/\Delta T$ will be almost surely

2587: equal to $-\langle n_3\rangle_t$ 

2588: given by (\ref{avpois}) and with the

2589: equation density measured at reduced time $t$. The argument can be

2590: extended to 2-equations, and we conclude that

2591: $e_2,e_3$ are deterministic (self-averaging) quantities obeying

2592: the two coupled differential equations\cite{Ch90}

2593: \begin{equation}

2594: \frac{de_3}{dt}(t) = -\frac{3\, e_3}{1-t} \quad

2595: , \qquad \frac{de_3}{dt}(t) = \frac{3\,

2596:   e_3}{1-t}  - \frac{2\, e_2}{1-t} \ .

2597: \end{equation}

2598: Those equations, together with the initial condition $e_3(0)=\alpha

2599: _0$, $e_2(0)=0$ can be easily solved,

2600: \begin{equation}\label{sole23}

2601: e_3(t) = \alpha_0 (1-t)^3 \quad , \qquad e_2(t)= 3\,\alpha_0

2602: \,t\,(1-t)^2 \ .

2603: \end{equation}

2604: To sum up, the dynamical evolution of the equation populations may be 

2605: seen as a slow  and deterministic evolution of the 

2606: equation densities to which are superimposed fast, small fluctuations. 

2607: The distribution of the fluctuations adiabatically follows 

2608: the slow trajectory. This scenario is pictured in Figure~\ref{deterstoch}.

2609: 

2610: \begin{figure}

2611: \begin{center}

2612: \includegraphics[width=160pt,angle=0]{fig552.eps}

2613: \caption{Deterministic versus stochastic dynamics of the equation

2614: population $E$ as a function of the number of steps $T$ of the algorithm. 

2615: On the slow time scale (fraction $t=T/N$) the density

2616: $e=E/N$ of (2- or 3-) equations varies smoothly according to a deterministic 

2617: law. Blowing up of the dynamics around some point $t',e'$ 

2618: shows the existence of small and fast fluctuations around this trajectory.

2619: Fluctuations are stochastic but their probability distribution 

2620: depends upon the slow variables $t',e'$ only.}

2621: \label{deterstoch}

2622: \end{center}

2623: \end{figure}

2624: 

2625: Expressions (\ref{sole23}) for the equation densities allow us to draw

2626: the resolution trajectories corresponding to the action of DPLL on a

2627: 3-XORSAT instance. Initially the instance is represented by a point

2628: with coordinates $(p=1,\alpha=\alpha_0)$ in Figure \ref{diag2+p}. As

2629: more and more variables are assigned the representative point moves

2630: away from the rightmost vertical axis. After a fraction $t$ of

2631: variables have been assigned the coordinates of the point are

2632: \begin{equation}

2633: p(t) = \frac{e_3}{e_2+e_3}=\frac{1-t}{1+2t}\ , \quad 

2634: \alpha(t) = \frac{e_2+e_3}{1-t}=\alpha_0(1-t)(1+2t) \ .

2635: \end{equation}

2636: Trajectories corresponding to various initial ratios are shown in

2637: Figure \ref{diag2+p}.

2638: For small ratios $\alpha_0 < \alpha _E$ trajectories remain

2639: confined in the sat phase, end in S of coordinates $(0,0)$, where a

2640: solution is found. At $\alpha_E$ ($=\frac 23$ for the UC heuristic), 

2641: the single branch trajectory hits

2642: tangentially the threshold line in T of coordinates $(\frac 14,\frac 23)$. 

2643: When $\alpha_0 >  \alpha _E$ the trajectories enter the Unsat phase,

2644: meaning that DPLL has turned a satisfiable instance (if $\alpha _0 <

2645: \alpha _c$) into an unsatisfiable one as a result of poor assignments.

2646: It is natural to expect that $\alpha_E$ is the highest ratio at which 

2647: DPLL succeeds in finding a solution without resorting to much backtracking.

2648: 

2649: 

2650: 

2651: \subsection{Dynamics of unit-equations and universality}\label{secuni}

2652: 

2653: The trajectories we have derived in the previous Section are correct 

2654: provided no contradiction emerges. But

2655: contradictions may happen as soon as there are $E_1=2$

2656: unit-equations, and are all the more likely than $E_1$ is large. 

2657: Actually the set of 1-equations form a 1-XORSAT instance which is

2658: unsatisfiable with a finite probability as soon as $E_1$ is of the order

2659: of $\sqrt N$ from the results of Section \ref{secsf1}.

2660: Assume now that $E_1 (T)\ll N$ after $T$ variables have been assigned, 

2661: what is the probability $\rho_T$ that no contradiction

2662: emerges when the $T^{th}$ variable is assigned by DPLL? This probability is

2663: clearly one when $E_1=0$. When $E_1\ge 1$ we pick up a 1-equation,

2664: say, $x_{6}=1$, and wonder whether the

2665: opposite 1-equation, $x_6=0$, is present among the $(E_1-1)$ 1-equations

2666:  left. As equations are uniformly distributed over the set

2667: of $N-T$ untouched variables 

2668: \begin{equation}\label{probanocttot}

2669: \rho_T = \left( 1 - \frac 1{2 (N-T)} \right) ^{\max( E_1(T) -1,0)}  

2670: \ .

2671: \end{equation}

2672: The presence of the $\max$ in the above equation ensures it remains

2673: correct even in the absence of unit-equations ($E_1=0$). 

2674: $E_1(T)$ is a stochastic variable. However from the 

2675: decoupling between fast and slow time scales sketched in Figure 

2676: \ref{deterstoch} the probability distribution of $E_1(T)$ 

2677: depends only on the slow time scale $t$. Let us call $\mu (E_1;t)$ 

2678: this probability. Multiplying (\ref{probanocttot}) over the times

2679: $T=0$ to $T=N-1$ we deduce the probability that DPLL has successfully 

2680: found a solution without ever backtracking, 

2681: \begin{equation} \label{rhosuc}

2682: \rho _{success} = \exp \left( - \int _0^1 \frac{dt}{2(1-t)} 

2683: \sum _{E_1\ge 1} \mu (E_1;t)\; (E_1-1) \right)

2684: \end{equation}

2685: in the large $N$ limit.

2686: 

2687: \begin{figure}

2688: \begin{center}

2689: \includegraphics[width=200pt,angle=0]{fig561.eps}

2690: \caption{Evolution of the number $E_1$ of 1-equations as one more variable

2691: is assigned. $n_2$ denotes the number of 2-equations reduced to

2692: 1-equations, $s_1$ the number of 1-equations satisfied. If $E_1\ge 1$ a

2693: variable is fixed through unit-propagation: $E_1$ decreases by one 

2694: plus $s_1$, and increases by

2695: $n_2$. In the absence of unit-equation ($E_1=0$) the number of

2696: 1-equations after the assignment is simply $E_1'=n_2$.  }

2697: \label{rw1clause}

2698: \end{center}

2699: \end{figure}

2700: 

2701: We are left with the calculation of $\mu$\cite{Fr96}. 

2702: Figure \ref{rw1clause} sketches the stochastic evolution of the number

2703: $E_1$ during one step.  The number of 1-equations produced

2704: from 2-equations, $n_2$, is a Poisson variable

2705: with average value, from (\ref{sole23}),

2706: \begin{equation}\label{valued}

2707: d(t) = \frac {2 \, e_2 (t)}{1-t} = 6\,\alpha_0\, t(1-t)\,

2708: \end{equation}

2709: when $N\to \infty$. The number of 

2710: satisfied 1-equations, $s_1$, is negligible as long as $E_1$ remains

2711: bounded. The probability that the number of 1-equations

2712: goes from $E_1$ to $E_1'$ when $T\to T+1$ defines the entry of the

2713: transition matrix

2714: \begin{equation}\label{matrixmm}

2715: M (E'_1,E_1;t) = \sum _{n_2 \ge 0} e^{-d(t)} \frac{d(t)^{n_2}}{n_2!}

2716: \delta _{E'_1-(E_1+n_2-\delta_{E_1})}\ .

2717: \end{equation}

2718: from which a master equation for the probability of

2719: $E_1$ at time $T$ may be written. 

2720: On time scales $1\ll \Delta T\ll N$ this master equation

2721: converges to the equilibrium distribution $\mu$  \cite{Fr96,Co01},

2722: conveniently expressed in terms of the generating function

2723: \begin{equation} \label{defgtstat}

2724: G( x;t ) = \sum _{E_1\ge 0} \mu(E_1;t)\; x^{E_1} = 

2725: \frac{(1-d(t))(x-1)}{x\; e^{d(t) \; (1-x)} -1} \;

2726:  \ . 

2727: \end{equation}

2728: The above is a sensible result for $d(t)\le 1$ but does not make sense when 

2729: $d(t)>1$ since a probability cannot be negative! The reason is that we

2730: have derived (\ref{defgtstat}) under the implicit condition that no

2731: contradiction was encountered. This assumption cannot hold when the

2732: average rate of 1-equation production, $d(t)$, is larger that one, the

2733: rate at which 1-equations are satisfed by unit-propagation.

2734: From (\ref{valued}) we see, when $\alpha > \alpha _E=\frac 23$, the

2735: trajectory would cross the

2736: \begin{equation}\label{defcon}

2737: \alpha_D(p) = \frac {1} {2(1-p)}

2738: \end{equation}

2739: on which $d=1$ for some time $t_D<1$. A contradiction is very

2740: likely to emerge before the crossing. 

2741: 

2742: 

2743: When $\alpha <\alpha _E$ $d$ remains smaller than unity at any time. 

2744: In this regime the probability of

2745: success reads, using (\ref{rhosuc}) and (\ref{defgtstat}),

2746: \begin{equation} \label{nobtsuc}

2747: \rho_{success} = \exp \left(\frac {3\alpha}{4} - \frac 12 

2748: \sqrt{\frac{3\alpha}{2-3\alpha}}\; \tanh^{-1}

2749: \bigg[ \sqrt{\frac{3\alpha}{2-3\alpha}}\bigg] \right)\ .

2750: \end{equation}  

2751: $\rho_{success}$ is a decreasing

2752: function of the ratio $\alpha$, down from unity for $\alpha =0$ to zero for

2753: $\alpha =\alpha_{E}$.

2754: The present analysis of the UC heuristic can be easily transposed to the GUC

2755: heuristic. Details are not given here but can be found in 

2756: \cite{Ac02,Co01}. The result is an expression

2757: for $\rho_{success}$ larger than its UC counterpart (\ref{nobtsuc}),

2758: and vanishing in $\alpha _E\simeq 0.7507$. Interestingly the way 

2759: $\rho_{success}$ vanishes when $\alpha$ reaches $\alpha_E$,

2760: \begin{equation}

2761: - \ln \rho_{success} (\alpha_E - \epsilon) \sim \epsilon ^{-\frac 12}

2762: \qquad (\epsilon \to 0^+)

2763: \end{equation}

2764: is the same for both heuristics. This similarity extends to a whole

2765: class of heuristics which can be described by the flow of equation

2766: densities only and based on unit-propagation \cite{De04}. The

2767: probability that DPLL finds a solution without backtracking

2768: to a 3-XORSAT instance of size $N$ satisfies finite-size scaling at the 

2769: dynamical critical point,

2770: \begin{equation}\label{ffss}

2771: - \ln \rho_{success} (\alpha_E - \epsilon, N) \sim N^{\frac 16} \;

2772: \Phi\big( \epsilon \, N^{\frac 13} \big) \ ,

2773: \end{equation}

2774: where the scaling function $\Phi$ is independent of the heuristics

2775: and can be calculated exactly \cite{De04}. The exponent

2776: characterizing the width of the critical region is the one associated

2777: to percolation in random graphs (\ref{valpsi}). A consequence of

2778: (\ref{ffss}) is that, right at $\alpha_E$, $\rho_{success} \sim \exp 

2779: (- Cst\times N^{\frac 16})$ decreases as a stretched exponential of

2780: the size. The value of the exponent, and its robustness against the

2781: splitting heuristics can be understood from the following

2782: argument  \cite{De04}.

2783: 

2784: Let us represent 1- and 2- equations by a graph $G$ over the set

2785: of $N-T$ vertices (one for each variable $x_i$) with $E_1$ marked 

2786: vertices (one for each unit-equation $x_i=0,1$), and $E_2$

2787: signed edges ($x_i + x_j=0,1$), see Section \ref{secrg}.

2788: $d$ is simply the average degree of vertices in $G$.

2789: Unit-propagation corresponds to removing

2790: a marked vertex (and its attached edges), after having marked its

2791: neighbours; the process is iterated until the connected component is

2792: entirely removed (no vertex is marked). Meanwhile, new edges 

2793: have been created from the reduction of 3-equations into 2-equations.

2794: Then a vertex is picked up according to the heuristic and marked,

2795: and unit-propagation resumes. The

2796: success/failure transition coincides with the percolation

2797: transition on $G$:  $d=1$ as expected.

2798: From random graph theory \cite{Bo89} the

2799: percolation critical window is of width 

2800: $|d -1| \sim N^{-1/3}$. As $d$ is

2801: proportional to the ratio $\alpha _0$ (\ref{valued})

2802: we find back $\psi= \frac 13$.

2803: The time spent by resolution trajectories in the critical

2804: window is $\Delta t \sim \sqrt {|d -1|} \sim N^{-1/6}$,

2805: corresponding to $\Delta T = N \, \Delta t \sim N^{5/6}$ eliminated

2806: variables. As the largest components have size $S\sim N^{2/3}$ the

2807: number of such components eliminated is $C=\Delta T/S\sim N^{1/6}$.

2808: What is the probability $q$ that a large component is removed without

2809: encountering a contradiction? During the removal of the component the

2810: number of marked vertices `freely' diffuses, and reaches $E_1\sim \sqrt

2811: {S}\sim N^{1/3}$. The probability that no contradiction occurs

2812: is, from (\ref{probanocttot}), $q\sim (1-\frac {Cst}N)^{E_1\times S}$,

2813: a finite quantity. Thus $\rho_{success} \sim q^C \sim

2814: \exp(-N^{1/6})$. The presence of numerous, smaller components does 

2815: not affect this scaling. 

2816: 

2817: 

2818: 

2819: \subsection{Exponential phase:  massive backtracking}

2820: 

2821: For ratios $\alpha _0>\alpha _E$ DPLL is very likely to find a

2822: contradiction. Backtracking enters into play, and is responsible for

2823: the drastic slowing down of the algorithm (Figure \ref{dpllxorsat}).

2824: 

2825: The history of the search process can be represented by a search tree,

2826: where the nodes represent the variables assigned, and the descending

2827: edges their values (Figure \ref{fig-tree}). 

2828: The leaves of the tree correspond to solutions (S), or

2829: to contradictions (C). The analysis of the $\alpha<\alpha_E$ regime

2830: leads us to the conclusion that search trees look like

2831: Figure \ref{fig-tree}A at small ratios\footnote{A small amount of

2832:   backtracking may be necessary to find the solution since

2833:   $\rho_{success}<1$ \cite{Fr96}, but the overall picture of a single

2834:   branch is not qualitatively affected.}. 

2835: Consider now the case of unsatisfiable formulas 

2836: ($\alpha_0> \alpha_c$) where all leaves carry contradictions after

2837: DPLL halts (Figure \ref{fig-tree}C). 

2838: DPLL builds the tree in a sequential manner, adding nodes

2839: and edges one after the other, and completing branches through

2840: backtracking steps. We can think of the same search tree built in a 

2841: parallel way\cite{Co01}. At time (depth $T$) our tree is 

2842: composed of $L(T)\le 2^T$ branches, each carrying a partial assignment 

2843: over $T$

2844: variables. Step $T$ consists in assigning one more variable to each

2845: branch, according to DPLL rules, that is, through unit-propagation or

2846: split. Possible consequences are: emergence of a contradiction and end

2847: of the branch, simplification of the attached formulas and the branch keeps

2848: growing. 

2849: 

2850: \begin{figure}

2851: \begin{center}

2852: \includegraphics[width=130pt,angle=-90]{fig571.eps}

2853: \caption{Search trees in three regimes of Section \ref{secdpll}: 

2854: {\bf A.} linear, satisfiable ($\alpha < \alpha_E$); {\bf  B.} 

2855: exponential, satisfiable ($\alpha_E<\alpha<\alpha_c$);

2856: {\bf C.} exponential, unsatisfiable ($\alpha > \alpha _c$). Leaves

2857: are marked with S (solutions) or C (contradictions). G is the highest

2858: node to which DPLL backtracks, see Figure \ref{diag2+p}.}

2859: \label{fig-tree}

2860: \end{center}

2861: \end{figure}

2862: 

2863: The number of branches $L(T)$ is a stochastic variable. Its average

2864: value can be calculated as follows \cite{Mon05}. Let us define the

2865: average number $L(\vec E;T)$ of branches with equation populations $\vec

2866: E=(E_1,E_2,E_3)$ at depth $T$. Initially $L(\vec E;0)=1$ for

2867: $\vec E=(0,0,\alpha_0 N)$, 0 otherwise.

2868: Call $M(\vec E',\vec E;T)$ the average

2869: number of branches with population $\vec E'$ generated from a branch with

2870: population $\vec E$ once the $T^{th}$ variable is assigned. Transition 

2871: matrix $M$ is an extension of (\ref{matrixmm}) 

2872: to the whole population vector $\vec E$ and not only $E_1$.

2873: We have $0\le M\le 2$, the extreme values corresponding to a

2874: contradiction and to a split respectively. We claim that 

2875: \begin{equation}\label{evol5}

2876: L(\vec E';T+1) = \sum _{\vec E} M(\vec E',\vec E;T) \; L(\vec E;T) \ .

2877: \end{equation}

2878: Evolution equation (\ref{evol5}) is somewhat suspicious since it looks like

2879: the approximation (\ref{statq}) we have done in the analysis of

2880: RWSAT. Yet a major difference exists which makes (\ref{evol5})

2881: exact \cite{Mon05}. Drawing randomly many times the same instance, as

2882: we are doing, is in principle forbidden but not along {\em one} branch 

2883: for the very reason the analysis of Section \ref{secuc} was

2884: correct. Actually what we have done in Section \ref{secuc} is to draw

2885: randomly at time $T$ the equations containing the $T^{th}$

2886: variable. But this is correct since those equations are immediately

2887: simplified into shorter equations and their remaining content remains

2888: unknown \cite{Ka02}. The situation seems more complicated in the case

2889: of the whole tree since the same equation can appear at different

2890: depth along distinct branches. Indeed the number of branches produced

2891: from two distinct branches after assignment of one variable are correlated

2892: variables. But thanks to the linearity of expectation those

2893: correlations do not matter and (\ref{evol5}) is correct.

2894: 

2895: Transition matrix $M$ can be explicitely written down. It is more

2896: convenient to write (\ref{evol5}) for the generating function of the

2897: number of branches, $B(\vec x;T) = \sum _{\vec E} L(\vec E;T)

2898: x_1^{E_1}\,x_2^{E_2}\,x_3^{E_3}$, with the result

2899: \begin{equation} \label{evol6}

2900: B(\vec x;T+1) = \frac 1{f_1} B(\vec f;T) + (2-\frac 1{f_1})

2901: B(0,f_2,f_3;T) - 2 B(\vec 0;T)\ ,

2902: \end{equation}

2903: where $\vec f$ is the vector with components

2904: \begin{equation}

2905: f_1= x_1 + \frac{\frac 12-x_1}{N-T}, f_2 = x_2 +  \frac{2(

2906:   x_2-x_1)}{N-T}, f_3 = x_3 +  \frac{3( x_2-x_3)}{N-T} \ .

2907: \end{equation}

2908: The three terms on the r.h.s. of (\ref{evol6}) correspond, from left

2909: to right: unit-propagation (the branch keeps growing), variable

2910: splitting (2 branches are created from the previous one), branches

2911: carrying empty instances (satisfied instance).

2912: Equation (\ref{evol6}) together with the initial condition $B(\vec

2913: x;0)= x_3^{\alpha _0N}$ completely defines the average dynamics of the 

2914: search tree. We sketch the main steps of its resolution below\cite{Co01}:

2915: \begin{enumerate}

2916: \item To count the  number of branches irrespectively of the number of 

2917: unit-equations we should consider the value $x_1=1$. However, as long 

2918: as branches grow the number $E_1$ of unit-equations cannot be large, 

2919: and remains bounded. We can therefore choose $x_1=\frac 12$ which

2920: simplifies (\ref{evol6}) without affecting the large size scaling of

2921: $L$ and $B$. This technical trick is reminiscent of Knuth's kernel

2922: method \cite{Kn66}.

2923: \item For large $N$ it is reasonable to expect that the number of

2924:   branches grows exponentially with the depth, or, equivalently,

2925: \begin{equation}

2926: \sum _{E_1} L(E_1,E_2,E_3;T) \sim e ^{N \; \lambda (e_2,e_2;t) + o(N)}

2927: \end{equation}

2928: where $e_2,e_3$ are the densities of equations as usual. From point 1

2929: the Legendre transform of $\lambda$

2930: \begin{equation}\label{degpp}

2931: \gamma (x_2,x_3;t) = \max _{e_2,e_3} \big[ \lambda (e_2,e_2;t) + 

2932: e_2 \ln x_2 + e_3 \ln x_3 \big]

2933: \end{equation}

2934: fulfills the partial differential equation (PDE)

2935: \begin{equation}

2936: \frac{\partial \gamma} {\partial t}  = \ln 2 + \frac{1-2x_2}{1-t} 

2937: \frac{\partial \gamma}{\partial x_2} +  \frac{3(x_2-x_3)}{1-t} 

2938: \frac{\partial \gamma}{\partial x_3} \ .

2939: \end{equation}

2940: with the initial condition $\gamma (x_2,x_3;t) = \alpha _0 \ln x_3$.

2941: \item The first order PDE can be solved exactly with the

2942:   characteristic method. The output, after Legendre inversion

2943:  through (\ref{degpp}), is the entropy $\lambda (e_2,e_3;t)$ 

2944: of branches at reduced depth $t$.  Let us call $\lambda ^*(t)$ the

2945:   maximum value of $\lambda$ over the equation densities for a fixed

2946:   fraction $t$ of assigned variables.

2947: \item $\lambda ^*(t)$ is a function growing from $\lambda ^*=0$ at

2948: $t=0$, reaching a maximum value $\lambda ^*_M$ in $t_M$, and

2949: decreasing for larger times $t\le 1$. $t_M$ is the depth in the tree

2950: of Figure \ref{fig-tree}C where most contradictions are found; the

2951: number of C leaves is, to exponential order, $e^{N\lambda^*_M}$. We

2952: conclude that the size of the tree we were looking for is

2953: \begin{equation}  

2954: \tau _{split} = \lambda^*_M \ ,

2955: \end{equation}

2956: compare with (\ref{mesure}). For large $\alpha \gg \alpha_c$ one finds

2957: $\tau_{split}\sim \ln 2 /(6\alpha)$ in agreement with \cite{beame}. 

2958: The calculation can be extended to highers values of $K$.

2959: \end{enumerate}

2960: 

2961: The above calculation holds for the unsatisfiable, exponential phase.

2962: How can we understand the satisfiable but exponential regime

2963: $\alpha_E<\alpha_0<\alpha_c$? The resolution trajectory crosses the

2964: Sat/Unsat critical line at some point G shown in Figure

2965: \ref{diag2+p}. Immediately after $G$ the instance left by DPLL is

2966: unsatisfiable. A subtree with all its leaves carrying contradictions

2967: will develop below G (Figure \ref{fig-tree}B). The size $\tau_{split}

2968: ^G$ of this

2969: subtree can be easily calculated from the above theory. The only 

2970: change is the initial condition over $\gamma$: 

2971: $\gamma (x_2,x_3;0) = \alpha_G (p_G \ln

2972: x_3 + (1-p_G) \ln x_2)$ where $(p_G,\alpha_G)$ are the coordinates of

2973: G which can be calculated from $\alpha_0$ and the knowledge of the

2974: critical Sat/Unsat line. Once this subtree has been built DPLL

2975: backtracks to G, flips the attached variable and will finally end up

2976: with a solution. Hence the (log of the)  number of splits necessary will be

2977: typically equal to $\tau_{split} = (1-t_G)\, \tau_{split}^G$ \cite{Co01}. 

2978: 

2979: % conclusion

2980: 

2981: \section{Conclusions}\label{secconclusion}

2982: 

2983: Previous Sections have allowed us to illustrate rather general

2984: techniques and ideas to deal with random systems. It does not come as

2985: a surprise that other problems than XORSAT e.g. the satisfaction of

2986: Boolean constraints, graph coloring, the covering of vertices, ...  

2987: have been successfully studied with these tools. Many of those

2988: problems, when given 

2989: an input distribution based on random graphs, actually share a lot 

2990: of common features with XORSAT. The reader 

2991: is referred to 

2992: \cite{Mo97,Bi00,Me02,Me03b,Se06,Ac05} (satisfiability), 

2993: \cite{Mu02,Se06} (coloring), 

2994: \cite{We01b,We01} (vertex cover), ... 

2995: for entry points to the literature. Let us also mention that 

2996: many other interesting optimization problems, not directly related to

2997: random graphs,  have been studied with the techniques of Sections 4

2998: and 5, and the results sometimes rigorously proven 

2999: e.g. matching \cite{Or85,Me87b,Al01}, 

3000: traveling salesman \cite{Me86}, 

3001: number partitioning \cite{Me98,Me01}, 

3002: graph partitioning \cite{Fu85}, ...

3003: Finally, from a historical point of view, one should not forget

3004: that statistical mechanics tools have found numerous and beautiful

3005: applications in the study of the learning and storage

3006: properties of neural networks\cite{Am89,Va01}, all the more 

3007: so the random satisfiability problem can be recast as an Ising

3008: perceptron problem \cite{Kr89}.

3009: 

3010: The study of random optimization problems is obviously interesting

3011: from a probabilistic point of view. As far as computer science is

3012: concerned they can be seen as useful benchmarks for testing and

3013: improving resolution procedures. A successful example is the

3014: traduction of the cavity equations of Section 5 into an algorithm for

3015: solving given instances of the satisfiability problem

3016: \cite{Me02}. This algorithm, called Survey Propagation, extends to 

3017: the clustered phase the Belief Propagation procedure of wide-spread use in 

3018: statistical inference, and is a very efficient procedure

3019: to find solutions to 

3020: 3-Satisfiability slightly below threshold. Another application of

3021: statistical physics ideas is the conception of new heuristics for DPLL

3022: capable of proving the unsatisfiability of formulas with 700 hundreds

3023: variables at threshold \cite{De03}. 

3024: 

3025: Despite those successes important question remain open. First is there

3026: a relationship between clustering and hardness of resolution? 

3027: This question is reminiscent of a very general issue in statistical physics,

3028: namely the relationship between dynamical and static properties of 

3029: disordered or glassy systems \cite{Cu93}. 

3030: The onset of clustering, or more

3031: precisely of strong correlations between variables over the space of solutions

3032: drastically worsens the performances of 

3033: sampling algorithms e.g. Monte Carlo procedures \cite{Mo05,Se06}. However, in 

3034: practical applications, one

3035: looks for a solution rather than for the sampling of 

3036: the solution space... From this point of view knowing whether 

3037: solutions are clustered or not

3038:  does not seem to be of crucial relevance. Actually 

3039: a local and polynomial search strategy capable

3040: of finding solutions well above the clustering threshold has been

3041: explicitely found for various optimizations problems \cite{Jo07}.

3042: 

3043: Another open question is what happens at large

3044: $K$, that is, when constraints involve more and more variables. 

3045: The performances of all known algorithms, be they local search

3046: procedures or DPLL solvers, seem to deteriorate. Worst-case bound 

3047: indicate that the large $K$ case is very difficult \cite{Im99}. 

3048: From statistical

3049: mechanics point of view problems look like more and more the random

3050: energy model \cite{Me87} as $K$ increases, but can we beat the

3051: worst-case bounds on average? Finally let us mention a recent work by

3052: Feige \cite{Fe02} which, for the first time, showed that the

3053: complexity of solving random SAT (or XORSAT) model  had a fundamental

3054: interest in worst-case approximation theory. Consider 3-SAT

3055: instances with ratio $\alpha \gg \alpha_c$. Most of them have GS

3056: energy close to $\alpha N/2$, but a very tiny fraction of those

3057: instances have energy smaller than, say, $\epsilon N$ where $\epsilon\ll

3058: \alpha$ is fixed. Is there a polynomial algorithm capable of

3059: recognizing all such atypical formulas from the vast majority of

3060: typical instances? Insights from statistical physics suggest that,

3061: the answer is positive for SAT (if we want most satisfiable instances

3062: to be detected and not all of them) while XORSAT seems to be much

3063: harder\cite{Al07}! Actually, to the knowledge of the author, no local

3064: search algorithm (based on random walk, variable assigment, Monte

3065: Carlo, message-passing, cooling procedure, ...) is efficient for 

3066: solving XORSAT. This makes the

3067: study of this problem even more valuable from a computer science point

3068: of view.

3069: 

3070:  

3071: 

3072: 

3073: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

3074: 

3075: \appendix

3076: 

3077: \section{A primer on large deviations}\label{applargedev}

3078: 

3079: Large deviation theory is the field of probability which deals with

3080: very unlikely events \cite{De93}. 

3081: You are given a fair (unbiased) coin and

3082: toss it $N$ times. The number $H$ of head draws has  probability

3083: \begin{equation} \label{ld1}

3084: p_N(H) = \frac 1{2^N} {N \choose H} \ .

3085: \end{equation}

3086: When $N$ gets large $H$ is highly concentrated around $H^*=N/2$

3087: with small relative fluctuations of the order of $O(\sqrt N)$. 

3088: Yet we can ask for the

3089: probability of observing a fraction $h=H/N$ equal to say, 25\%, of

3090: heads, far away from the likely value $h^*=50\%$. To calculate this

3091: probability we use Stirling's asymptotic expression for the binomial

3092: coefficient in (\ref{ld1}) to obtain

3093: \begin{equation} \label{ld2}

3094: p_N(H=h \, N) = e^{ - N \omega(h) +o(N)} \ ,

3095: \end{equation}

3096: where

3097: \begin{equation}

3098: \omega (h) = \ln 2 + h \ln h + (1-h)\ln (1-h)\ 

3099: \end{equation}

3100: is called rate function. The

3101: meaning of (\ref{ld2}) is that events with value of $h\ne h^*$ are

3102: exponentially rare in $N$, and $\omega(h)$ give the decay (rate) exponent.

3103: The answer to our question is $e^{-N

3104: \omega(.25)} \sim e^{-0.13 \,N}$ when $N$ is large. Some comments are:

3105: \begin{itemize}

3106: \item  $\omega(h)$ is strictly positive, except in $h = h^*=\frac 12$

3107:   where it vanishes. This is the only value for the fraction of head draws

3108: with non exponentially small--in--$N$ probability.

3109: \item Let $h=h^* + \delta h$ where $\delta h$ is small. Using

3110:   $\omega (h^*) = \omega '(h^*) =0$ we have

3111: \begin{equation}

3112: P_N \big( H= (h^* + \delta h) N \big) = \exp \big[ -N\, \frac 12

3113:   \omega ''(h^*) \,(\delta h)^2 + \ldots \big] \ ,

3114: \end{equation}

3115: that is, $\delta h$ is Gaussianly distributed with zero mean and

3116: variance $(N\omega''(h^*))^{-1}=(4N)^{-1}$. Hence central limit

3117: theorem is found back from the parabolic behaviour of the

3118: rate function around its minimum\footnote{Non standard behaviour

3119:   e.g. fluctuations of the order of $N^{\nu}$ with $\nu\ne \frac 12$

3120: as found in Levy flights correspond to non-analyticies of $\omega$ 

3121: in $h^*$ or the vanishing  of the second derivative.}.

3122: \item $\omega$ is here a convex function of its argument. This

3123:   property is true rate functions describing independent

3124:   events. Indeed, suppose we have $H$ positive (according to some

3125:   criterion e.g. being a head for a coin) events among a set of $N$

3126:   events, 

3127: then another set of $N'$ events among which $H'$ are positive. If the 

3128: two sets are uncorrelated 

3129: \begin{equation}

3130: p_{N+N'} (H+H') \ge p_N (H) \times p_{N'} (H')

3131: \end{equation}

3132: since the same total number $H+H'$ of positive events could be

3133: observed in another

3134: combination of $N+N'$ events. Taking the logarithm 

3135: and defining $h=H/N$, $h'=H'/N$, $u=N/(N+N')$ we obtain

3136: \begin{equation}

3137: \omega (u\, h+ (1-u)\, h') \le u\, \omega(h) + (1-u)\, \omega(h') \ ,

3138: \end{equation}

3139: for any $u\in [0;1]$. Hence the representative curve of $\omega$ lies

3140: below the chord joining any two points on this curve, and $\omega$ is

3141: convex. Non-convex rate functions are found in presence

3142: of strong correlations\footnote{Consider the following experiment. 

3143: You are given three

3144:   coins: the first one is fair (coin A), the second and third coins,

3145:   respectively denoted by B and C,  are biased and

3146:   give head with probabilities, respectively, $\frac 14$ and $\frac

3147:   34$. First draw coin A once. If the outcome is head pick up coin B,

3148:   otherwise pick up coin C. Then draw your coin $N$ times. What is the

3149:   rate function associated to the fraction $h$ of heads?}.

3150: \end{itemize}

3151: 

3152: 

3153: \section{Inequalities of first and second moments}\label{appmoment}

3154: 

3155: Let ${\cal N}$ be a random variable taking values on the positive integers, and

3156: call $p_{\cal N}$ its probability. We denote by $\langle{\cal N}\rangle$ and 

3157: $\langle {\cal N} ^2\rangle$ the first and second moments of 

3158: ${\cal N}$ (assumed to be finite), and write 

3159: \begin{equation}

3160: p({\cal N}\ge 1) =\sum _{{\cal N}=1,2,3, \ldots} p_{\cal N} = 1 -p_0

3161: \end{equation}

3162: the probability that ${\cal N}$ is not equal to zero.

3163: Our aim is to show the inequalities

3164: \begin{equation} \label{ineq}

3165: \frac{\langle {\cal N}\rangle^2}{\langle {\cal N}^2\rangle} \le  

3166: p({\cal N}\ge 1) \le {\langle {\cal N}\rangle} \ .

3167: \end{equation}

3168: The right inequality, call 'first moment inequality', is straightforward:

3169: \begin{equation}

3170: {\langle {\cal N}\rangle} = \sum _{\cal N} {\cal N}\; p_{\cal N} = 

3171: \sum _{{\cal N}\ge 1} {\cal N} \; p_{\cal N} \ge

3172: \sum _{{\cal N}\ge 1}  p_{\cal N} =  p({\cal N}\ge 1) .

3173: \end{equation}

3174: Consider now the linear space made of vectors 

3175: ${\bf v}=(v_0,v_1,v_2,\ldots\}$ whose components are labelled by positive 

3176: integers, with the scalar product

3177: \begin{equation}

3178: {\bf v} \cdot {\bf v}' = \sum _{\cal N} p_{\cal N}\; v_{\cal N}\; 

3179: v_{\cal N}' \ .

3180: \end{equation}

3181: Choose now $v_{\cal N}={\cal N}$, and $v'_0=0,v'_{\cal N}=1$ for 

3182: ${\cal N}\ge 1$. Then

3183: \begin{equation}

3184: {\bf v} \cdot {\bf v} = \langle {\cal N}^2\rangle \ , \ 

3185: {\bf v} \cdot {\bf v}' = \langle {\cal N}\rangle \ , \

3186: {\bf v}' \cdot {\bf v}' = p({\cal N}\ge 1) \ .

3187: \end{equation}

3188: The left inequality in (\ref{ineq}) is simply the Cauchy-Schwarz inequality 

3189: for ${\bf v},{\bf v}'$: $({\bf v} \cdot {\bf v}')^2 \le  ({\bf v} \cdot {\bf v} ) \times ({\bf v}' \cdot {\bf v} ')$. 

3190: 

3191: \section{Corrections to the saddle-point calculation of $\langle 

3192: {\cal N}^2\rangle$} \label{appfluctu}

3193: 

3194: In this Appendix we show that $\langle {\cal N}^2\rangle$ is

3195: asymptotically equivalent to $\langle {\cal N}\rangle^2$, where ${\cal N}$ is the number of solutions of a 3-XORSAT formula with ratio $\alpha 

3196: < \alpha_2 \simeq 0.889$. This

3197: requires to take care of the finite-size corrections around the

3198: saddle-point calculations of Section \ref{secld2}. Let $Z=(z_1,z_2,

3199: \ldots, z_N)$ denotes a configuration of variables at distance $d$

3200: from the zero configuration {\em i.e.} $dN$ variables $z_i$ are equal

3201: to 1, the other  $(1-d)N$ variables are null. 

3202: Let $q(d,N)$ be the probability that $Z$ satisifies 

3203: the equation $z_i+z_j+z_k=0$ where $(i,j,k)$ is a random triplet of

3204: distinct integers (unbiased distribution):

3205: \begin{eqnarray}

3206: q(d,N)&=& \frac 1{{N\choose 3}} \left[ {(1-d)N \choose 3} + (1-d)N \;

3207: {dN \choose 2} \right] \\

3208: &=& q(d) \left( 1+ \frac{h(d)}N\right) + 

3209: \ldots \quad \mbox{where} \quad h(d) =\frac{

3210: 6 d  (2d-1)}{3 d^2+(1-d)^2}\ ,\nonumber 

3211: \end{eqnarray}

3212: and $q(d)$ is defined in (\ref{qk}) with $K=3$. Terms of the order of $N^{-2}$ 

3213: have been discarded. 

3214: 

3215: Using formula (\ref{n2p}) with $q(d)$ substituted with $q(d,N)$ 

3216: and the Stirling formula for the asymptotic behaviour of

3217: combinatorial coefficients we have

3218: \begin{eqnarray} \label{truc1}

3219: \langle {\cal N}^2\rangle \sim \sum _{d=0,\frac 1N, \frac 2N, \ldots} 

3220: \frac{\sqrt{2\pi N}}{\sqrt{2\pi Nd}\sqrt{2\pi N(1-d)}}\;

3221: e^{N\, A(d,\alpha) + \alpha\, h(d)}

3222: \end{eqnarray}

3223: where $A(d,\alpha)$ is defined in (\ref{upperom2}), and $\sim$

3224: indicates a true asymptotic equivalence (no multiplicative factor

3225: omitted). The r.h.s. of (\ref{truc1}) is the Riemann sum associated 

3226: to the integral

3227: \begin{eqnarray} \label{truc2}

3228: \langle {\cal N}^2\rangle \sim \int _0^1 \frac{N \, dd}

3229: {\sqrt{2\pi Nd(1-d)}}\;

3230: e^{N\, A(d,\alpha) + \alpha\, h(d)} \ .

3231: \end{eqnarray}

3232: We now estimate the integral through the saddle-point method.

3233: For $\alpha < \alpha _2 \simeq 0.889$ the dominant contribution to the integral comes from the vicinity of

3234: $d^*=\frac 12$. There are quadratic fluctuations 

3235: around this saddle-point, with a variance equal to $N$ times the 

3236: inverse of (the modulus of) 

3237: the second derivative $A_{dd}$ of $A$ with respect to $d$.

3238: Carrying out the Gaussian integral over those fluctuations we obtain

3239: \begin{equation} \label{truc3}

3240: \langle {\cal N}^2\rangle \sim  \frac{N \;

3241: e^{N\, A(d^*,\alpha) + \alpha\, h(d^*)} }{\sqrt{2\pi N d^*(1-d^*)}}\;

3242: \sqrt{\frac{2\pi}{N\, |A_{dd}(d^*,\alpha)|}}

3243: \sim  \langle {\cal N}\rangle ^2

3244: \end{equation}

3245: since $h(d^*)=0$, $A_{dd}(d^*,\alpha)=-4$. Therefore, from the

3246: second moment inequality, $P_{SAT}\to 1$ when $N\to\infty$ at ratios

3247: smaller than $\alpha_2$.

3248: 

3249: %

3250: % ********** End of text entry *************

3251: %

3252: \begin{thebibliography}{99}

3253: 

3254: %b1 ###

3255: \bibitem{Ac01}

3256: {{D.} {Achlioptas}},

3257: {\em Theor. Comp. Sci.} {\bf 265} (2001), {159}.

3258: 

3259: \bibitem{Ac02}

3260: {{D.} {Achlioptas}},

3261: {{P.} {Beame}}, and {{M.} {Molloy}},

3262: {\em Journal of Computer and System Sciences} 

3263: {\bf 68} (2004), {238}.

3264: 

3265: \bibitem{Ac05}

3266: {{D.} {Achlioptas}}, {{A.} {Naor}} and

3267: {{Y.} {Perez}},

3268: {\em Nature} {\bf 435} (2005), {759}.

3269: 

3270: \bibitem{Al89}

3271: {{D.J.} {Aldous}},

3272: {\em Discrete Math.} {\bf 76} (1989), {167}.

3273: 

3274: \bibitem{Al01}

3275: {{D.J.} {Aldous}},

3276: {\em Rand. Struct. Algo.)} {\bf 48} (2001), {381}.

3277:  

3278: \bibitem{Al02}

3279: {{M.} {Alekhnovich}} and {{E.} {Ben-Sasson}},

3280: \textit{Analysis of the Random Walk Algorithm on Random 3-CNFs},

3281: preprint (2002).

3282: 

3283: \bibitem{Al07}

3284: {{F.} {Altarelli}}, {{R.} {Monasson}}

3285: and {{F.} {Zamponi}},

3286: {\em  J. Phys. A} {\bf 40} (2007), {867}.

3287: 

3288: \bibitem{Am89}

3289: {{D.J.} {Amit}},

3290: \textit{Modeling Brain Function},

3291: (Cambridge University Press, Cambridge, 1989).

3292: 

3293: \bibitem{De79}

3294: {{J.R.L.} {de Almeida}} and {{D.J.} 

3295: {Thouless}},

3296: {\em J. Phys. A} {\bf 11} (1978), {983}.

3297: 

3298: \bibitem{beame}

3299: {{P.} {Beame}}, {{R.} {Karp}},

3300: {{T.} {Pitassi}}, and {{M.} {Saks}},

3301: (Proceedings of the ACM Symp.\ on Theory of Computing, 1998, pp. 561).

3302: 

3303: \bibitem{Bi00}

3304: {{G.} {Biroli}}, {{R.} {Monasson}} and

3305: {{M.} {Weigt}},

3306: {\em Eur. Phys. J. B} {\bf 14} (2000), {551}.

3307: 

3308: \bibitem{Bo89}

3309: {{B.} {Bollobas}},

3310: \textit{Random Graphs}

3311: (Cambridge University Press, Cambridge, 2001).

3312: 

3313: \bibitem{Br93}

3314: {{A.Z.} {Broder}}, {{A.M.} {Frieze}} 

3315:  and {{E.} {Upfal}},

3316: (Proceedings of Symposium of Discrete Algorithms (SODA), Austin,

3317: 1993).

3318: 

3319: \bibitem{Ch90}

3320: {{M.T.} {Chao}} and {{J.} {Franco}},

3321: {\em Information Science} {\bf 51} (1990), 

3322: {289}; {\em SIAM Journal on Computing} 

3323: {\bf 15} (1986), {1106}.

3324: 

3325: \bibitem{Ch88}

3326: {{V.} {Chv{\`a}tal}} and {{E.} {Szmeredi}},

3327: {\em Journal of the ACM} {\bf 35} (1988), {759}.

3328: 

3329: \bibitem{Co01}

3330: {{S.} {Cocco}} and {{R.} {Monasson}},

3331: {\em Phys. Rev. Lett.} {\bf 86} (2001), {1658}; {\em Eur. Phys. J. B} {\bf 22}

3332: (2002) , {505}.

3333: 

3334: \bibitem{Co03}

3335: {{S.} {Cocco}}, {{O.}

3336: {Dubois}}, {{J.} {Mandler}} and {{R.} 

3337: {Monasson}},

3338: {\em Phys. Rev. Lett.} {\bf 90} (2003), {047205}.

3339: 

3340: \bibitem{Cr99}

3341: {{N.} {Creignou}} and {{H.} {Daud\'e}},

3342: {\em  Discrete Applied Mathematics} {\bf 96-97} (1999), {41}.

3343: 

3344: \bibitem{Cr03a}

3345: {{N.} {Creignou}} and {{H.} {Daud\'e}},

3346: {\em RAIRO: Theoretical Informatics and Applications} 

3347: {\bf 37} (2003), {127}.

3348: 

3349: \bibitem{Cr03b}

3350: {{N.} {Creignou}}, {{H.} {Daud\'e}}

3351: and {{O.} {Dubois}},

3352: {\em Combinatorics, Probability and Computing}

3353: {\bf 12} (2003), {113}.

3354: 

3355: \bibitem{Cu93}

3356: {{L.} {Cugliandolo}} and {{J.} {Kurchan}},

3357: {\em Phys. Rev. Lett.}

3358: {\bf 71} (1993), {173}.

3359: 

3360: \bibitem{dpll}

3361: {{M.} {Davis}} and {{H.} {Putnam}},

3362: {\em J.\ Assoc.\ Comput.\ Mach.} {\bf 7} (1960), {201};

3363: {{M.} {Davis}}, {{G.} {Logemann}}

3364:  and {{D.} {Loveland}},

3365: {\em Communications of the ACM} {\bf 5} (1962), {394}.

3366: 

3367: 

3368: \bibitem{De93}

3369: {{A.} {Dembo}} and {{O.} {Zeitouni}},

3370: \textit{Large deviations techniques and applications}

3371: (Springer-Verlag, New York, 1993).

3372: 

3373: \bibitem{De03}

3374: {{G.} {Dequen}} and {{O.} {Dubois}},

3375: (Proceedings of Theory and Applications of Satisfiability Testing, 6th 

3376: International Conference, SAT 2003. Santa Margherita Ligure, 2003, pp 486).

3377: 

3378: 

3379: \bibitem{De04}

3380: {{C.} {Deroulers}} and {{R.} {Monasson}},

3381: {\em Eur. Phys. J. B} {\bf 49} (2006), {339}.

3382: 

3383: \bibitem{Du02}

3384: {{O.} {Dubois}} and {{J.} {Mandler}},

3385: (Proc. of the 43rd

3386: annual IEEE symposium on Foundations of Computer Science, Vancouver,

3387: 2002).

3388: 

3389: \bibitem{Va01}

3390: {{A.} {Engel}} and

3391: {{C.} {Van den Broeck}},

3392: \textit{Statistical Mechanics of Learning},

3393: (Cambridge University Press, Cambridge, 2001).

3394: 

3395: \bibitem{Fe02}

3396: {{U.} {Feige}},

3397: (Proceedings of 34th STOC conference, 2002, pp 534).

3398: 

3399: \bibitem{Fr96}

3400: {{A.} {Frieze}} and {{S.} {Suen}},

3401: {\em Journal of Algorithms} {\bf 20} (1996), {312}.

3402: 

3403: \bibitem{Fu85}

3404: {{Y.} {Fu}} and {{P.W.} {Anderson}},

3405: {\em J. Phys. A} {\bf 19} (1986), {1605}.

3406: 

3407: \bibitem{We01}

3408: {{A.K.} {Hartmann}} and {{M.} {Weigt}},

3409: {\em Theor. Comp. Sci.} {\bf 265} (2001), {199}.

3410: 

3411: \bibitem{Im99}

3412: {{R.} {Impagliazzo}} and {{R.} {Paturi}},

3413: (Proceedings of the 

3414: {IEEE} Conference on Computational Complexity, 1999, pp 237).

3415: 

3416: \bibitem{Ka02}

3417: {{A.C.} {Kaporis}}, {{L.M..}

3418: {Kirousis}}, and {{Y.C..} {Stamatiou}},

3419: \textit{How to prove conditional randomness using the principle of deferred

3420: decisions}, technical report, Computer technology Institute, Patras

3421:  (2002). 

3422: 

3423: \bibitem{Kn66}

3424: {{D.} {Knuth}},

3425: \textit{The Art of Computer Programming; vol 1: fundamental

3426:   algorithms, section 2.2.1},

3427: (Addison-Wesley, Ney York, 1968).

3428: 

3429: \bibitem{Kr89}

3430: {{W.} {Krauth}} and

3431: {{M.} {M\'ezard}},

3432: {\em J. Phys. (France) } {\bf 50} (1989), {3057}.

3433: 

3434: \bibitem{Se06}

3435: {{F.} {Krzakala}} {\em at al.}

3436: \textit{Gibbs States and the Set of Solutions of Random Constraint

3437:   Satisfaction Problems} (preprint, 2006).

3438: 

3439: \bibitem{Jo07}

3440: {{F.} {Krzakala}} and {{J.} {Kurchan}},

3441: \textit{(A landscape analysis of constraint satisfaction problems}

3442: (preprint, 2007).

3443: 

3444: \bibitem{Me98}

3445: {{S.} {Mertens}},

3446: {\em Phys. Rev. Lett.} {\bf 81} (1998), {4281};

3447: {\em Phys. Rev. Lett.} {\bf 84} (2000), {1347}.

3448: 

3449: \bibitem{Me01}

3450: {{S.} {Mertens}},

3451: {\em Theor. Comp. Sci.} {\bf 265} (2001), {79}.

3452: 

3453: \bibitem{Me03b}

3454: {{M.} {Mertens}},

3455: {{M.} {M\'ezard}} and {{R} {Zecchina}},

3456: {\em Rand. Struct. Algo.} {\bf 28} (2006), {340}.

3457: 

3458: \bibitem{Me87b}

3459: {{M.} {M\'ezard}} and {{G.} {Parisi}},

3460: {\em J. Phys. (Paris)} {\bf 48} (1987), {1451}.

3461: 

3462: \bibitem{Me86}

3463: {{M.} {M\'ezard}} and {{G.} {Parisi}},

3464: {\em J. Phys. (Paris)} {\bf 47} (1986), {1285}.

3465: 

3466: \bibitem{Me87}

3467: {{M.} {M\'ezard}}, {{G.} {Parisi}} and 

3468: {{M.} {Virasoro}},

3469: \textit{Spin glasses and beyond}

3470: (World Scientific, Singapore, 1987).

3471: 

3472: \bibitem{Me01y}

3473: {{M.} {M\'ezard}} and {{G.} {Parisi}},

3474: {\em Eur. Phys. J. B}

3475: {\bf 20} (2001), {217};

3476: {\em J. Stat. Phys}

3477: {\bf 111} (2003), {111}.

3478:   

3479: 

3480: \bibitem{Me02}

3481: {{M.} {M\'ezard}} and {{R} {Zecchina}},

3482: {\em Phys. Rev. E} {\bf 56} (2002), {066126}.

3483: 

3484: \bibitem{Me03}

3485: {{M.} {M\'ezard}}, {{F.}

3486: {Ricci-Tersenghi}}, and {{R.} {Zecchina}},

3487: {\em J. Stat. Phys.} {\bf 111} (2003), {505}.

3488: 

3489: \bibitem{Mi92}

3490: {{D.} {Mitchell}}, {{B.} {Selmann}}

3491:  and {{H.} {Levesque}},

3492: {\em Proc.\ of the Tenth Natl.\ Conf.\ on Artificial 

3493: Intelligence (AAAI-92)} (1992) , {440}.

3494: 

3495: \bibitem{Mon95}

3496: {{R.} {Monasson}},

3497: {\em Phys. Rev. Lett.}

3498: {\bf 75} (1995), {2847}.

3499: 

3500: \bibitem{Mo97}

3501: {{R.} {Monasson}} and {{R.} {Zecchina}},

3502: {\em Phys. Rev. E} {\bf 56} (1997), {1357}.

3503: 

3504: \bibitem{Mon98}

3505: {{R.} {Monasson}},

3506: {\em J. Phys. A} {\bf 31} (1998), {513}.

3507: 

3508: \bibitem{Mon05}

3509: {{R.} {Monasson}},

3510: {\em Lecture Notes in Computer Science} {\bf 3624} (2005), {402}.

3511: 

3512: \bibitem{Mo07}

3513: {{A.} {Montanari}} and {{D.} {Shah}},

3514: (Proceedings of Symposium of Discrete Algorithms (SODA), New Orleans,

3515: 2007).

3516: 

3517: \bibitem{Mo05}

3518: {{A.} {Montanari}} and {{G.} {Semerjian}},

3519: {\em Phys. Rev. Lett.}

3520: {\bf 94} (2005), {247201}.

3521: 

3522: 

3523: \bibitem{Mo95}

3524: {{R.} {Motwani}} and {{P.} {Raghavan}},

3525: \textit{Randomized algorithms}

3526: (Cambridge University Press, Cambridge, 1995).

3527: 

3528: \bibitem{Mu02}

3529: {{R.} {Mulet}}, {{A.} {Pagnani}},

3530: {{M.} {Weigt}} and {{R} {Zecchina}},

3531: {\em  Phys. Rev. Lett.} {\bf 89} (2002), {268701}.

3532: 

3533: \bibitem{Or85}

3534: {{H.} {Orland}},

3535: {\em J. Phys. (Paris) Lett.} {\bf 46} (1985), {L763}; 

3536: {{M.} {M\'ezard}} and {{G.} {Parisi}},

3537: {\em J. Phys. (Paris) Lett.} {\bf 46} (1985), {L771}.

3538: 

3539: \bibitem{Pa83}

3540: {{C.} {Papadimitriou}} and {{K.} 

3541: {Steiglitz}},

3542: \textit{Combinatorial Optimization: Algorithms and Complexity}

3543: (Dover, 1998).

3544: 

3545: \bibitem{Pa92}

3546: {{C.H.} {Papadimitriou}},   

3547: (Proceedings of the 32nd Annual IEEE Symposium on Foundations of

3548: Computer Science, 1991, pp 163).

3549: 

3550: \bibitem{Pa86}

3551: {{G.} {Parisi}} and {{M.} {Virasoro}},

3552: {\em J. Phys. (Paris)} {\bf 50} (1986), {3317}.

3553: 

3554: \bibitem{We02}

3555: {{F.} {Ricci-Tersenghi}},

3556: {{M.} {Weigt}} and {{R.} {Zecchina}},

3557: {\em Phys. Rev. E} {\bf 63} (1999), {026702}.

3558: 

3559: \bibitem{Sc00}

3560: {{U.} {Sch\"oning}},

3561: {\em  Algorithmica} {\bf 32} (2002), {615}.

3562: 

3563: \bibitem{Se03}

3564: {{G.} {Semerjian}} and {{R.} {Monasson}},

3565: {\em Phys. Rev. E} {\bf 67} (2003), {066103}; 

3566: {{W.} {Barthel}}, {{A.} {Hartmann}}

3567:  and {{M.} {Weigt}},

3568: {\em Phys. Rev. E} {\bf 67} (2003), {066104}.

3569: 

3570: \bibitem{Wo95}

3571: {{N.} {Wormald}},

3572: {\em The Annals of Applied Probability} {\bf 5} (1995), {1217}.

3573: 

3574: 

3575: \bibitem{We01b}

3576: {{M.} {Weigt}} and {{A.K.} {Hartmann}}

3577: {\em Phys. Rev. Lett.} {\bf 84} (2000), {6118};

3578: {\em Phys. Rev. Lett.} {\bf 86} (2001), {1658}.

3579: 

3580: \end{thebibliography}

3581: \end{document}

3582: 

3583: