1: \section{Introduction}
2: \label{intro}
3: A crucial part of assessing the performance of a structured P2P system (aka DHT) is
4: evaluating how it copes with churn. Extensive simulation is currently the prevalent tool for
5: gaining such knowledge. Examples include the work of Li \textit{et al.} \cite{dhtcomparison:infocom05},
6: Rhea \textit{et al.} \cite{rhea04handling}, and Rowstron \textit{et al.} \cite{rowstron04depend}. There has also been some theoretical analyses done, albeit less frequently. For instance, Liben-Nowell \textit{et al.} \cite{nowell02analysis} prove a lower bound on the
7: maintenance rate required for a network to remain connected
8: in the face of a given churn rate. Aspnes \textit{et al.} \cite{aspnes02FaultTolerant} give upper and lower
9: bounds on the number of messages needed to locate a node/data item
10: in a DHT in the presence of node or link failures.
11: The value of theoretical studies of this nature is that they provide
12: insights neutral to the details of any particular DHT.
13:
14: We have chosen to adopt a slightly different approach to theoretical
15: work on DHTs. We concentrate not on establishing bounds, but rather on
16: a more precise prediction of the relevant quantities in such
17: dynamically evolving systems. Our approach is based mainly on the Master-Equation approach used in the
18: analysis of physical systems. We have previously introduced our approach in
19: in \cite{KEAH1,KEAH2} where we presented
20: a detailed analysis of the Chord system \cite{chord:ton}.
21: In this paper, we show that the approach is applicable to other systems as well.
22: We do this by comparing the periodic stabilization maintenance technique
23: of Chord with the
24: correction-on-change maintenance technique of DKS \cite{onana03dks}.
25:
26: Due to space limitations, we assume reader familiarity
27: with Chord and DKS, including such terminology as successors, finger starts
28: and finger nodes {\it etc}.
29:
30: The rest of the paper is organised as follows. In Section \ref{mastereq}, we introduce the Master-Equation
31: approach. In Section \ref{related}, we mention some related work. In section \ref{lookup} we begin by briefly
32: reviewing some of
33: our previously published results on predicting the performance of the Chord network
34: as a function of the failed pointers in the system in the case that the nodes
35: use a periodic maintenance scheme. We then show some new results on how this complicated
36: equation can be simplified to get quick predictions for varying number
37: of peers and varying number of links per node. We relegate some of the details of
38: this analysis to Appendix \ref{A1}. In section \ref{correction-on-change},
39: we explain how to use the Master-Equation approach to analyse the
40: reactive maintenance strategy of interest and present
41: our results on how this strategy
42: compares with the periodic case analysed earlier. We summarise our results
43: in Section \ref{summary}.
44:
45:
46: %Also something about the rates (failure, join and stabilisation, $alpha$, $r$) for the
47: %periodic case. The rates used in the reactive case are explained in the
48: %appropriate section.
49:
50: \section{The Master-Equation Approach for Structured Overlays}
51: \label{mastereq}
52: In a complicated system like a P2P network, in which
53: there are many participants,
54: and in which there are many inter-leaved processes happening in time,
55: predicting the state of the network (or of any quantity of interest) can
56: at best be done by specifying the
57: probability distribution function (PDF) of the
58: quantity in the steady state (when the system, though changing
59: continually in time, is stationary on average).
60: For example, one quantity of interest for us when analysing such a network, is the fraction of failed links between nodes, in the steady state.
61: This quantity does not take
62: some deterministic value in the steady state. Instead it is specified by a PDF,
63: which can then be used to determine the average value.
64: The problem is thus to calculate the PDF (and then to understand how it
65: affects the performance of the network, as explained below).
66:
67:
68: In general this is not
69: an easy task, since the probability is affected by a number of inter-leaved
70: processes in any time-varying system. In \cite{KEAH1,KEAH2}, we demonstrated
71: how we could analyse a P2P network like Chord
72: \cite{chord:ton}, using a Master-Equation based approach. This approach
73: is generally used in physics to understand a system evolving in time,
74: by means of equations specifying the time-evolution of the probabilities of finding the
75: system in a specific state. These equations require as an input,
76: the rates of various processes affecting the state of the system. For example,
77: in a peer-to-peer network, these processes could be the join and failure rates
78: of the member nodes, the rate at which each node performs maintenance as well as
79: the rate at which lookups are done in the network
80: (the latter rate is relevant only if the lookups affect the state of the
81: network in some way). Given these rates, the equation for the time-evolution
82: of the probability of the quantity of interest can be written by
83: keeping track of how these rates affect this quantity (such as the number
84: of failed pointers in the system) in an infinitesimal interval of time,
85: when only a limited number of processes (typically one) can be expected to
86: occur simultaneously.
87:
88: With this approach, we
89: were able to quantify very accurately the probabilities of any connection in the network
90: (either fingers or successors) having failed. We then
91: demonstrated how we could use this information
92: to predict the performance of the network---the number of hops {\it including} time outs
93: which a lookup takes on average --- as a function of the rates
94: (of join, failure and stabilization) of all the processes happening in the network, as
95: well as of all the parameters specifying the network (such as
96: how many pointers a node has on average). The analysis was done for a specific
97: maintenance strategy, called periodic maintenance (or eager maintenance)
98:
99:
100: In this paper, we generalise our approach so as to be able to compare networks
101: using different maintenance strategies. In particular, we compare our earlier results
102: for periodic maintenance with a reactive maintenance strategy proposed in
103: \cite{ghodsi05hicss}. Combining this with some of our previous results, we are
104: also, as a by product, able to compare the performance of networks specified by
105: different numbers of peers, different number of pointers per node and/or
106: different maintenance strategies. As we show below, which system is better
107: depends both on the value of the parameters as well as the level of churn.
108: The approach we propose is thus a useful tool for the quantitative and fair
109: comparison of networks specified by different parameters and using different algorithms.
110:
111: \section{Related Work}
112: \label{related}
113: In \cite{aberer04tok}, an analysis, very similar in spirit to the one done in this paper, is carried out in
114: the context of P-Grid \cite{pgrid}. An equation is written for system performance in the
115: state of dynamic equilibrium for various maintenance strategies. However for each maintenance strategy,
116: the analysis has to be entirely redone. In contrast, a master equation description
117: provides a foundation for the theoretical analysis of overlays,
118: which does not have to be entirely rebuilt each time any given algorithm is changed.
119: As we show in this paper, we can carry over a lot of our earlier analysis, when the maintenance scheme
120: is changed from a periodic to a reactive one. In addition, the master equation description
121: can be made arbitrarily precise to include non-linear effects as well. And as we show, non linear effects
122: are important when churn is high.
123:
124:
125:
126:
127: \section{The Lookup Equation for Chord}
128: \label{lookup}
129:
130: We quantify the performance of the network, by the number of hops required
131: on average from the originator of the query to the node with the answer. This
132: is just the total number of nodes contacted per query
133: (or equivalently, the total
134: number of pointers used per query) {\it including }
135: the total number of failed pointers used
136: en route. This latter quantity
137: (which arises because of the churn in the network)
138: is the reason that the hop count per query increases with high dynamism and is
139: hence an important quantity to understand. In the case
140: of the periodic maintenance scheme, this quantity is a function of
141: $(1- \beta) r$ where $r$ is the ratio of the stabilisation rate to the
142: join (or failure) rate and $1-\beta$
143: is the fraction of times a node stabilises its finger, when
144: performing maintenance, as mentioned in Section \ref{intro}.
145: We demonstrate how this quantity can be calculated in Section
146: \ref{correction-on-change}, in the context of the reactive
147: maintenance policy,
148: which is a simple generalisation of how it is calculated
149: earlier in\cite {KEAH1,KEAH2 },
150: for the periodic maintenance scheme.
151: In this section, we briefly review our earlier results on how the
152: performance of the network (as exemplified by the average hopcount per query),
153: can be determined once the fraction of failed pointers is known.
154:
155:
156: The key to predicting the performance of the network is to
157: write a recursive equation
158: for the expected cost $C_{t}(r, \beta)$
159: (also denoted $C_{t}$) for a given node
160: to reach some target, $t$ keys away from it.
161: (For example, $C_1$ is the cost of looking up the adjacent key which is $1$
162: key away).
163:
164:
165:
166: The Lookup Equation for the expected cost of reaching a general distance $t$
167: is then derived by following closely the Chord
168: protocol which is a greedy strategy
169: designed to reduce the distance to the query at every step
170: without overshooting the target . A lookup for $t$
171: thus proceeds by first finding
172: the closest preceding finger. The node that this finger points
173: to is then asked to continue the query, if it is alive.
174: If this node is dead, the originator of the query
175: uses the next closest preceding finger and the query proceeds in this manner.
176:
177: For the purposes of the analysis,
178: it is easier to think in terms of the closest preceding {\it start}.
179: Let us hence define $\xi$ to be the {\emph start} of the
180: finger (say the $k^{th}$) that most closely precedes $t$.
181: Hence $\xi = 2^{k-1} + n$ and
182: $t = \xi+m$, i.e. there are $m$ keys between the sought target $t$
183: and the start of the most closely preceding
184: finger. With that, we can write a recursion relation
185: for $C_{\xi+m}$ as follows:
186:
187: \begin{equation}
188: %\vspace*{-0.5cm}
189: \label{eq:cost}
190: \begin{split}
191: &C_{\xi+m} = C_{\xi} \left[1-a(m)\right] \\
192: &+ (1-f_k) a(m)\left[1 + \sum_{i=0}^{m-1} bc(i,m)C_{m-i}\right]
193: \\
194: &+ f_k a(m) \biggl[ 1 + \sum_{i=1}^{k-1} h_k(i) \\
195: &\sum_{l=0}^{\xi/2^i-1}bc(l,\xi/2^i)(1+(i-1) +C_{\xi_i-l+m}) + O(h_k(k)) \biggr]
196: % &+ \biggl[ f_k a(m) \\
197: % &+f_k a(m)\sum_{i=1}^{k-1} h_k(i) \sum_{l=1}^{\xi/2^i}bc(l,m)(1+C_{\xi_i(k)+1-l+m}) + 2h_k(k)\biggr]
198: \end{split}
199: %\vspace*{-0.5cm}
200: \end{equation}
201:
202: where $\xi_i \equiv \sum_{m=1,i} \xi/2^{m}$ and $h_k(i)$ is the
203: probability that a node is forced to use its $k-i^{th}$ finger owing to the
204: death of its $k^{th}$ finger.
205:
206: The probabilities $a,bc$ can be derived from the internode
207: interval distribution
208: \cite{KEAH1,KEAH2} which is just the distribution of distances between adjacent nodes. Given a ring of $\cal K$ keys and $N$ nodes (on average), where nodes can join and leave independently, the probability that two adjacent nodes are a distance $x$ apart on the ring is simply
209: $P(x) = \rho^{x-1}(1-\rho)$ where $\rho = \frac{{\cal K}-N}{\cal K}$.
210: Using this distribution, its easy to estimate the probability that
211: there is definitely atleast one node in an interval of length $x$.
212: This is: $a(x)\equiv {1-\rho^x}$. The probability
213: that the {\it first} node encountered from any key is at a distance $i$ from that key is then $b_i \equiv {\rho^{i}(1-\rho)}$.
214: Hence the conditional probability that the first node from a given key is at a distance $i$ {\it given} that
215: there is atleast one node in the interval is $ bc(i,x)\equiv b(i)/a(x)$.
216:
217:
218: The probability $h_k(i)$ is easy to compute given the probability $a$
219: as well as the probabilities $f_k$'s of the $k^{th}$ finger being dead.
220:
221: \begin{equation}
222: \label{eq:hki}
223: \begin{split}
224: h_k(i) = & a(\xi/2^{i}) (1-f_{k-i}) \\
225: \times &\Pi_{s=1,i-1} (1-a(\xi/2^{s}) + a(\xi/2^s)f_{k-s}), i<k \\
226: h_k(k) = & \Pi_{s=1,k-1} (1-a(\xi/2^{s}) + a(\xi/2^s)f_{k-s})
227: \end{split}
228: \end{equation}
229:
230: Eqn.\ref{eq:hki} accounts for all the
231: reasons that a node may have to use its $k-i^{th}$ finger
232: instead of its $k^{th}$ finger. This could happen because the
233: intervening fingers were either dead or not distinct (fingers $k$ and $k-1$ are not distinct if
234: they have the same entry in the finger table. Though the {\it starts} of the two fingers are
235: different, if there is no node in the interval between the {\it starts}, the entry in the finger table will be the same).
236: The probabilities $h_k(i)$ satisfy the constraint $\sum_{i=1}^{k} h_k(i)=1$.
237: $h_k(k)$, is the probability that a node
238: cannot use any earlier entry in its finger table,in which case it has to fall back on its successor list instead.
239: We indicate this case by the last term in Eq. \ref{eq:cost} which is
240: $O(h_k(k))$. In practise, the probability for this
241: is extremely small except for targets very close to $n$.
242: Hence this does not
243: significantly affect the value of general lookups and we ignore it
244: for the moment.
245:
246:
247: The cost for general lookups is
248: $$
249: L(r,\beta) = \frac{\Sigma_{i=1}^{{\cal K} -1} C_i(r,\beta)}{\cal K}
250: $$
251:
252: The lookup equation is solved recursively numerically, using the expressions for
253: $a$, $bc$, $h_k (i)$ and $C_1$. In Fig. \ref{fig:w},
254: we have plotted the theoretical prediction of Equation \ref{eq:cost}
255: versus what we get from simulating Chord. Here we have used
256: $N \sim 1000$ and $K= 2^{20}$. As can be seen the
257: the theoretical results match the simulation results very well.
258:
259: \begin{figure}[t]
260: \centering
261: % \includegraphics[height=9cm, angle=270]{wd}
262: % \includegraphics[height=9cm, angle=270]{i}
263: % \includegraphics[height=8cm, angle=270]{f}
264: \includegraphics[height=8cm, angle=270]{l_jan25}
265:
266: %\begin{table}[t]
267: %\centering
268: % \vspace*{-0.3cm}
269: \caption{Theory and Simulation for $L(r,\beta)$}
270: \label{fig:w}
271: \end{figure}
272:
273: In Fig.~\ref{fig:lookup_theory} we also show the theoretical predictions
274: for some larger values of $N$.
275:
276:
277:
278: \begin{figure}[t]
279: \centering
280: \includegraphics[height=8cm, angle=270]{lookup_theory_jan25}
281: \caption{Lookup cost, theoretical curve, for $N=1000, 2000,4000,8000, 160000$ peers. The rationale for the fits is explained later in the text.}
282: \label{fig:lookup_theory}
283: \end{figure}
284:
285:
286: On general grounds, it is easy to argue from the structure of
287: Equation \ref{eq:cost}, that the dependence of the average lookup
288: on churn comes entirely
289: from the presence of the terms $f_k$.
290: Since $f_k \sim f$ is independent of $k$ for large fingers, we can
291: approximate the average lookup length by the
292: functional form $L (r, \beta) = A + {B}f + C f^{2} + \cdots $.
293: The coefficients $A, B, C$ {\it etc} can be recursively computed
294: by solving the lookup equation
295: to the required order in $f$. They depend only on $N$ the number of nodes,
296: $1- \rho $ the
297: density of peers and $b$ the base or equivalently the
298: size of the finger table of each node.
299: The advantage of writing the lookup length this
300: way is that churn-specific details such as how new
301: joinees construct a finger table
302: or how exactly
303: stabilizations are done in the system, can be isolated
304: in the expression for $f$.
305: If we were to change our stabilization strategy, as we will
306: demonstrate below, we could immediately
307: estimate the lookup length by plugging in the new expression
308: for $f$ in the above relation.
309:
310: Another advantage of having a simple expression such as the above,
311: is that if we can estimate $A,B, C \cdots$ accurately, we can make use
312: of the expression for $L$ to estimate the churn
313: (or the value of $r$) in the system,
314: hence using a local measure to estimate a global quantity.
315: The logic in doing so is the inverse of the reasoning we have used so far.
316: So far, we have used the churn
317: as the input for finding $f_k$ and hence $L$. But we can also reverse
318: the logic and try and estimate churn, if we know the value of the
319: average lookup length $L$. If $L$ has the above simple expression,
320: then given $A$ and $B$ to $O(f)$, we have
321: $f= \frac{L-A}{B}$. From the expression for $f$
322: (see section \ref{correction-on-change} for how to evaluate $f$),
323: we can now get the value of $r$. Hence any
324: peer can make an estimate of the churn that the system
325: is facing if it knows how long its lookups are taking on average, and
326: if it has an estimate of $N$.
327:
328:
329: To get $A$, we need to consider Eqn \ref{eq:cost} with no churn
330: (all $f_k$'s set to zero).
331: %To get $B$, we need to analyze the lookup equation to $O(f)$ and so on.
332: In Appendix \ref{A1}, we study
333: the lookup equation ~\ref{eq:cost} in some detail to understand
334: the behaviour without churn and obtain the value of $A$ for any base $b$.
335: This is useful on several counts. First,
336: the value of $A$ is needed to predict the lookup costs as explained above.
337: Secondly, if $b$ changes ( a system of base $b$ has a finger table of size
338: ${\cal M}=(b-1)log_b ({\cal K})$), all else remaining the same, the only
339: major change in the lookup cost is due to the change in $A$. So estimating
340: $A$ precisely has the benefit that we
341: can predict the lookup cost for {\it any}
342: base $b$. Thirdly, the analysis confirms that Equation \ref{eq:cost}
343: does indeed reproduce well known results for
344: the lookup hop count in Chord, such
345: as for example, that the average lookup cost is $0.5*\log(N)$
346: without churn \cite{chord:ton}. Infact as demonstrated in Appendix \ref{A1},
347: for any $N$, the average lookup cost as predicted by Eq. \ref{eq:cost}
348: is indeed $0.5*\log(N)$ plus some $\rho$-dependent corrections which
349: though small are accurately predicted.
350:
351: %An added benefit of the analysis is
352: %that we can also predict what the average lookup without churn will be for
353: %any base (Chord has base $2$ and accordingly has a finger table
354: %size of $log_2({\cal K})$.
355:
356: A simple estimate for $B$ and $C$ can be made in the following manner.
357: Let every finger be dead with some finite probability $f$.
358: Each lookup encounters on average $A$ fingers,
359: where $A$ is the average lookup length {\it without} churn.
360: Each of these fingers could be alive
361: (in which case it contributes a cost of $1$),
362: dead with a probability $f$ in which case it contributes a cost of $2$ if the
363: next finger chosen is alive (with probability $1-f$) and so on. Its trivial to
364: verify that this estimates the look-up cost to be $A(1+f + f^2 + \cdots)$.
365: Comparing with our expression for $L$, this gives an estimate of
366: $B= A, C=A, \cdots$.
367:
368: In general if $L=A+B*g(f)$,
369: then if we scale $L$ by plotting $(L-A)/B$ for varying $N$,
370: we should get an estimate of $g(f)$. Note that $f$ depends on $\rho$ and
371: ${\cal M}$ the number of fingers. In addition if $g(f)= a_1f+a_2f^2 + \cdots$, the coefficients $a_1$,$a_2$, {\it etc}
372: can also depend on $\rho$.
373: However for $1-\rho <<1$, these dependences on $\rho$ are small and
374: the curves for different $N$ collapse onto the same curve on scaling.
375: In Fig. ~\ref{fig:lookup_theory_scaled} we have scaled the curves ploted
376: in Fig. ~\ref{fig:lookup_theory} in the above manner, using $B=A$.
377: The values of $A$ used are derived from the analysis of the previous section.
378: As can be seen the curves collapse onto one curve which is well
379: approximated by the function $g(f)=f+3*f^2$, giving $a_1=1$ and $a_2=3$.
380: The fits in Fig ~\ref{fig:lookup_theory} are also according to this functional form.
381: It should be emphasized however that this approximation for $g(f)$ is
382: good only for $1-\rho <<1$. For higher values of peer density, the curves for
383: different $N$ will not collapse onto one curve and
384: any $\rho$-dependence of the coefficients $a_i$'s will show up as well.
385:
386:
387: \begin{figure}[t]
388: \centering
389: \includegraphics[height=8cm, angle=270]{lookup_theory_scaled}
390: \caption{Scaled Lookup cost, for $N=1000, 2000,4000,8000, 160000$ peers.}
391: \label{fig:lookup_theory_scaled}
392: \end{figure}
393:
394:
395: We can use the above functional form to predict how lookups would behave if we change the base
396: $b$ (the size of the routing table) of the system. In Fig
397: ~\ref{fig:lookup_theory_base} we plot the functional form
398: $A(b)(1+f(b)+3f(b)^2)$ for $b=2,4,16$. The coefficient $A(b)$ is
399: accurately predicted by Eq. \ref{eq:constnochurn1}(in Appendix \ref{A1}),
400: with the definition of $\xi(i+1)$ taken appropriately. $f(b)$ is affected by the base $b$
401: because the number of fingers increases with $b$.
402:
403: As can be seen, when churn is low, a large $b$ is an advantage and
404: significantly improves the lookup length. However when churn is high,
405: the flip side of having a larger routing table is that it needs
406: more maintenance. Hence beyond some value of churn, the
407: larger the value of $b$, the larger the lookup latency.
408:
409: This is similar to the spirit of the numerical investigations done in ~\cite{li03comparing}.
410: However when comparing different bases for Chord, Li {\it et al} ~\cite{li03comparing} find
411: that while base $2$ is the best for high churn (as we find here), base $8$ is the best for low churn.
412: Increasing the base beyond this does not seem to improve the cost. The discrepancy between this finding
413: and ours is due to the details of the periodic maintenance scheme which we use.
414: In our case, we have taken the simplest scenario in which each node needs to stabilise ${\cal M}$ fingers
415: and the order in which this is done is random. In practice
416: only $\sim \log N$ of the ${\cal M}$ fingers are distinct, so only $\sim \log N$ stabilisations need be done
417: by each node. In addition, in ~\cite{li03comparing}, finger stabilisations are done {\it only} if the
418: finger is pinged and found to be dead.
419:
420: \begin{figure}[t]
421: \centering
422: \includegraphics[height=8cm, angle=270]{lookup_theory_base}
423: \caption{Lookup cost, for $N=1000$ peers for base $b=2,4,16$.}
424: \label{fig:lookup_theory_base}
425: \end{figure}
426:
427:
428:
429: \section{'Correction-on-Change' Maintenance Strategy}
430: \label{correction-on-change}
431:
432: In this section, we analyse a different maintenance strategy
433: using the master-equation formalism. The strategy we have analysed
434: so far is periodic stabilisation of successors as well as fingers.
435: We now consider a strategy where a node periodically stabilises its
436: successors but does not do so for its fingers. Instead, for maintaining
437: its fingers, it relies on other nodes for updates \cite{ghodsi05hicss}.
438: Whenever a node $n$ detects that its first successor $n.s_1$ is wrong
439: (failed or incorrect), it sends out messages to all the nodes
440: that are pointing
441: to its wrong first successor, so that they can update
442: their affected finger. The node sending messages can either do so
443: by broadcasting these messages to all affected nodes simultaneously, or by
444: scheduling messages periodically at some rate. We analyse the latter option
445: in this paper, since it provides a more intuitive and broader framework for
446: the comparison of the two schemes
447:
448:
449: For a system with {\it id}-size ${\cal K}$,
450: there are of the order of ${\cal M}=\log_{2}{\cal K}$ fingers
451: pointing to any node (there can be more than this if node spacings are
452: smaller than average. However, as we argue below, for our purpose this is not important).
453: Of course, not all $\cal M$ of these fingers are distinct. Several of these
454: fingers belong to node $n$ itself.
455: However to keep the analysis simple (and in keeping with the
456: spirit of our analysis of the periodic stabilisation scheme), we assume that
457: every node that detects a wrong successor needs to send out exactly
458: $\cal M$ messages (even if some of these 'messages' are sent to itself).
459:
460: To find out where the nodes that point to $n.s_1$ are located, $n$
461: needs to do a lookup. For example, to find the node
462: with the $k^{th}$ finger pointing to $n.s_1$, $n$ can do a lookup for the id $n-2^{k-1}$.
463: On obtaining the first successor (lets call it node $p$)
464: of this id, it would immediately know
465: if the $k^{th}$ finger of $p$ indeed needs to be updated. We think
466: of each lookup as a 'correction message'.
467: If there is more than one node that needs its $k^{th}$ successor updated
468: (because for example, the successors of $p$ also happen to point to $n.s_1$),
469: $n$ could leave the responsibility of informing these other nodes to
470: $p$. We could take into account the probability that a correction action
471: leads to more than $\cal M$ messages. But for the moment we ignore
472: this point (We could argue that once it is $p$'s responsibilities to check
473: that its successors know about $n.s_1$, it could piggy-back this information
474: when it does a successor stabilisation, which does not affect the number
475: of messages sent).
476:
477: Whenever a node receives a message updating its
478: information about a finger, it immediately corrects the appropriate entry
479: in its routing table.
480:
481: In the following, we demonstrate how we can analyse such a strategy.
482: We would like to ultimately compare its performance to
483: periodic stabilisation in the face of churn. To make such a comparisn
484: meaningful, we need to quantify the concept of 'maintenance-effort'
485: per node, and compare the two schemes at a given level of churn and at the same
486: value of the maintenance effort per node.We elaborate on this a little later
487: in Section \ref{compare}.
488:
489: %A simple way is to look at it as just the total number of
490: %messages sent per node. Here we
491: %are simply adding up all the successor-stabilisation actions performed
492: %by the node as well as all the correction messages sent,
493: %in order to get the total number of messages sent per node.
494: %This is not strictly correct since a correction-message takes of
495: %the order of $\log_2(N)$ hops while a successor-stabilisation takes
496: %only one. But we keep it this way for the sake of simplicity
497: %(and also because we have analysed periodic stabilistion this way).
498: %To be more accurate, we may have to take into account the
499: %different natures of successor stabilisation and correction messages.
500: %We may also need to take into account the amount of {\it information}
501: %sent per message instead of just the total number of messages.
502: %However in the following, we take the simplest route.
503:
504: %Another complication here is that in the case of periodic stabilisation,
505: %successor and finger stabilisations are independent and can happen
506: %at independent rates. However for the 'correction-on-change' strategy
507: %described above, the rate at which correction-messages are sent,
508: %is dependent on the rate at which successor stabilisation is done.
509: %We will quantify this below.
510:
511: Another point to note is how to quantify system performance. We have
512: previously done it in terms of lookup hops. But a more
513: correct way might be to ask for the latency for {\it consistent} lookups
514: (since some of the lookups could be inconsistent). However we have checked that
515: , within our analytical framework, this does
516: not change the results qualiltatively.
517:
518: \subsection{Analysis of the Correction-on-change strategy }
519: \label{analysis}
520:
521: To generalise the analysis to meet the situation when some
522: nodes are sending messages while others are not, we say that a node
523: can be in state $S_1$ or $S_2$. In state $S_1$,
524: a node can stabilise its first successor at rate $\alpha \lambda_s$,
525: fail at rate $\lambda_f$ and assist in
526: joins at rate $\lambda_j$ as before. In state $S_2$,
527: a node can stabilise its first successor at rate $a \lambda_s$,
528: fail at rate $\lambda_f$, assist in joins at rate $\lambda_j$
529: and in addition, send correction messages
530: (which is essentially equivalent to doing one lookup ) at
531: rate $ \lambda_M \equiv c \lambda_s$. As we show in Section
532: \ref{compare}, if we want to compare the two maintenance strategies
533: in a fair manner then the most general values that these
534: parameters can take is $\alpha=1$ and $a+c=1$.
535:
536:
537: Let $N_{S_1}$ be the number of nodes in state $S_1$ and
538: $N_{S_2}$ the number of nodes in state $S_2$. Clearly
539: $N_{S_1} + N_{S_2} = N$, the total number of nodes in the system.
540:
541: We can further partition $S_2$ into $S_{2}^{1}$,$S_{2}^{2}$,
542: $S_{2}^{3}$, $\cdots$, $S_{2}^{\cal M}$.
543: $S_{2}^{1}$ is the state of the node which has yet to send its first
544: correction message, $S_{2}^{2}$ the state of the node which has
545: sent its first correction message but is yet to send its second, {\it etc}.
546:
547: Consider the gain and loss terms for $N_{S_1}$. These are summarised in
548: table \ref{tab:ns1}.
549:
550: \begin{table}
551: \centering
552: \begin{tabular}{|l|l|} \hline
553: $N_{S_1}(t+\Delta t)$ & Probability of Occurence \\ %\hline
554: $= N_{S_1}(t)-1$ & $c_{1.1}=(\lambda_f N_{S_1} \Delta t)$ \\ %\hline
555: $= N_{S_1}(t)+1$ & $c_{1.2}=(\lambda_j N \Delta t)$ \\ %\hline
556: $= N_{S_1}(t)+1$ & $c_{1.3}=(\lambda_M N_{S_{2}^{\cal M}}\Delta t)$ \\
557: $= N_{S_1}-1$ & $c_{1.4}=(\alpha \lambda_s N_{S_1} \Delta t) w_1$\\ %\hline
558: $= N_{S_1}(t)$ & $1 - (c_{1.1} + c_{1.2} + c_{1.3} + c_{1.4})$\\ \hline
559: \end{tabular}
560: \caption{Gain and loss terms for $N_{S_1}$ the number of nodes in state $S_1$.}
561: \label{tab:ns1}
562: \end{table}
563:
564:
565: Term $c_{1.1}$ is the probability that an $S_1$ node is lost because it
566: failed. Term $c_{1.2}$ is the probability that a join occurs
567: thus adding to the number of $S_1$ nodes in the system (since a new joinee
568: is always an $S_1$-type node). Term $c_{1.3}$ is the probability that
569: an $S_{2}^{\cal M}$ node sent its last message at rate $\lambda_{M}$ and
570: converted into an $S_1$ node. The last term $c_{1.4}$ is the probability
571: that an $S_1$-type node did a stabilisation at rate $\alpha \lambda_s$,
572: found a wrong first successor with probability $w_1$ and hence
573: converted into an $S_2$ node. $w_1$ is the fraction of wrong successor
574: pointers of an $S_1$-type node.
575:
576: Defining $\lambda_s/\lambda_f = r$ and $\lambda_M/\lambda_f = cr$
577: the steady state equation predicted by table \ref{tab:ns1} is:
578: \begin{equation}
579: P_{S_1}(1+ \alpha rw_1) = 1+ cr P_{S_{2}^{\cal M}}
580: \label{eq:PS1}
581: \end{equation}
582:
583: where $P_{S_1} = N_{S_1}/N$.
584:
585: We can write a similar equation $N_{S_2}$ which however does not give
586: us any new information since $N_{S_1} +N_{S_2} = N$.
587:
588: Writing a gain-loss equation for each of the $N_{S_{2}^{i}}$'s in turn,
589: we obtain,
590:
591: \begin{equation}
592: P_{S_{2}^{1}} = \frac{P_{S_1}(\alpha rw_1 - arw_{1}^{\prime})}{1+cr+arw_{1}^{\prime}} + \frac{arw_{1}^{\prime}}{1+cr+arw_{1}^{\prime}}
593: \end{equation}
594:
595: and
596:
597: \begin{equation}
598: P_{S_{2}^{i}} = P_{S_{2}^{1}} \left(\frac{cr}{1+cr+arw_{1}^{\prime}} \right)^{i-1}
599: \end{equation},
600: for $2 \le i \le {\cal M}$.
601:
602: Here $w_1$ is the fraction of $S_1$ nodes with wrong pointers and
603: $w_{1}^{\prime}$ is the fraction of $S_2$ nodes with wrong pointers.
604: We have made a simplification here in assuming that the fraction of wrong pointers
605: of $S_2$ nodes is the same, irrespective of the state of the $S_2$ node. In practice
606: (especially if $a=0$),
607: this will not be the case. However
608: for the parameter ranges we are interested in ($r >>1$), this is not crucial.
609:
610: Clearly $\sum_{1}^{\cal M} P_{S_{2}^{i}} = P_{S_2}$. A quantity of interest
611: in our analysis is
612:
613: \begin{equation}
614: P_{S_{2}^{\cal M}}/P_{S_2} = 1 - \frac{(1-g_1^{{\cal M}-1})}{1-g_1^{\cal M}}
615: \label{eq:order}
616: \end{equation}
617:
618: %Using this, we can write the following equation:
619:
620: %\begin{equation}
621: %1-P_{S_1} = rw_1P_{S_1} - g \left( arw_{1}^{\prime} + P_{S_1} \left( rw_1 - arw_{1}^{\prime} \right) \right)
622: %\label{eq:ps1}
623: %\end{equation}
624:
625: where $g_1= \frac{cr}{(1+cr+arw_{1}^{\prime})}$.
626:
627: To solve for $P_{S_1}$ {\it etc}, we need to solve for
628: $w_1$ and $w_{1}^{\prime}$.
629:
630: However, consider first the equation for $W_T$ -- the {\it total} number of wrong
631: successor pointers in the system (irrespective of whether the pointer belongs
632: to an $S_1$ or an $S_2$ type node. The gain and loss terms for $W_T$ are shown in
633: table \ref{tab:wrong_total}. $w = W_T/N$ is the fraction of wrong succesor
634: pointers in the system.
635:
636: \begin{table}[t]
637: \caption{Gain and loss terms for $W_T$: the total number of wrong first successor
638: pointers in the system.}
639: \label{tab:wrong_total}
640: \centering
641: \begin{tabular}{|l|l|} \hline
642: Change in $W_T$ & \minorchange{Probability of Occurrence} \\ %\hline
643: $W_T(t+\Delta t) = W_T(t)+1$ & $c_{2.1}=(\lambda_j N \Delta t) (1-w)$ \\ %\hline
644: $W_T(t+\Delta t) = W_1(t)+1$ & $c_{2.2}=(\lambda_f N \Delta t) (1-w)^2 $ \\ %\hline
645: $W_T(t+\Delta t) = W_1(t)-1$ & $c_{2.3}=(\lambda_f N \Delta t)$ \\ %\hline
646: $W_1(t+\Delta t) = W_1(t)-1$ & $c_{2.4}=(\alpha \lambda_s \Delta t) {N_{S_1}} w_1 + (a \lambda_s \Delta t) {N_{S_2}} w_{1}^{\prime}$\\ %\hline
647: $W_1(t+\Delta t) = W_1(t)$ & $1 - (c_{2.1} + c_{2.2} + c_{2.3} + c_{2.4})$\\
648: \hline
649: \end{tabular}
650: %\vspace*{-0.35cm}
651: \end{table}
652:
653: This gives the following equation
654:
655: \begin{equation}
656: (3+\alpha r) w_1P_{S_1} + (3 +ar) w_{1}^{\prime}P_{S_2} =2
657: \label{eq:WT}
658: \end{equation}
659:
660:
661: The gain and loss terms $W_{1}^{\prime}$.
662: -- the number of $S_2$ nodes with wrong successor pointers --
663: are written in much the same way
664: except for a few small changes.
665: Table \ref{tab:wrong} details the changes that occur in $W_{1}^{\prime}$.
666: in time $\Delta t$.
667:
668: \begin{table}[t]
669: \caption{Gain and loss terms for $W_{1}^{\prime}$: the number of wrong first successor
670: pointers of $S_2$-type nodes.}
671: \label{tab:wrong}
672: \centering
673: \begin{tabular}{|l|l|} \hline
674: Change in $W_1$ & \minorchange{Probability of Occurrence} \\ %\hline
675: $W_{1}^{\prime}(t+\Delta t) = W_{1}^{\prime}(t)+1$ & $c_{2.1}=(\lambda_j {N_{S_2}} \Delta t) (1- w_{1}^{\prime})$. \\ %\hline
676: $W_{1}^{\prime}(t+\Delta t) = W_{1}^{\prime}(t)+1$ & $c_{2.2}=\lambda_f {N_{S_2}} (1-w_{1}^{\prime})^2 P_{S_2} $ \\ %\hline
677: & $+ (1-w_1)(1-w_{1}^{\prime}) P_{S_1}) \Delta t$ \\ %\hline
678: $W_{1}^{\prime}(t+\Delta t) = W_{1}^{\prime}(t)-1$ & $c_{2.3}=\lambda_f {N_{S_{2}}} {(w_{1}^{\prime}}^2 P_{S_2} + w_1w_{1}^{\prime} P_{S_1}) \Delta t $ \\
679: $W_{1}^{\prime}(t+\Delta t) = W_{1}^{\prime}(t)-1$ & $c_{2.4}= a \lambda_s {N_{S_2}} w_{1}^{\prime} \Delta t $\\ %\hline
680: $W_{1}^{\prime}(t+\Delta t) = W_{1}^{\prime}(t)-1$ & $c_{2.5}=\lambda_M {N_{S_2}^{\cal M}} w_{1}^{\prime} \Delta t $\\ %\hline
681: $W_1(t+\Delta t) = W_1(t)$ & $1 - (c_{2.1} + c_{2.2} + c_{2.3} + c_{2.4} +c_{2.5})$\\
682: \hline
683: \end{tabular}
684: %\vspace*{-0.35cm}
685: \end{table}
686:
687: The terms here are much the same as derived earlier except that
688: we now have to keep track of whether the node that is failing
689: (in terms $c_{2.2}$ and $c_{2.3}$) is a $S_1$ or an $S_2$-type node.
690: In addition term $c_{2.5}$ is the probability that an $S_{2}^{\cal M}$-type
691: node has a wrong successor pointer, but sends a message and hence
692: turns into an $S_1$ node with a wrong pointer.
693:
694: Table \ref{tab:wrong} gives us the following equation for
695: $w_{1}^{\prime}$ in the steady state
696:
697: \begin{equation}
698: 2= w_{1}^{\prime} \left(3+a r + cr\frac{P_{S_{2}^{\cal M}}}{P_{S_2}} \right)
699: +( w_1 - w_{1}^{\prime})P_{S_1}
700: \label{eq:Wprime}
701: \end{equation}
702:
703:
704: We can write a similar equation for $w_{1}$ which however does not contain
705: any new information since $w_1$ and $w_{1}^{\prime}$ satisfy equation
706: \ref{eq:WT}.
707:
708: So in effect we have three equations, Eqn. \ref{eq:PS1}, Eq. \ref{eq:WT} and
709: \ref{eq:Wprime} for three unknowns $P_{S_1}$, $w_1$ and $w_{1}^{\prime}$.
710: In practice this set of equations is very hard to solve exactly because of the
711: appearance of terms such as $g_1^{\cal M}$ in Eq. \ref{eq:order}.
712:
713: In the following we will solve the set of equation to $O(1/r)$ by expanding
714: Eq. \ref{eq:order} to first order in $w_{1}^{\prime}$. In this case,
715:
716: \begin{equation}
717: P_{S_{2}^{\cal M}}/P_{S_2} = \frac{1}{\cal M} - \left(\frac{{\cal M}-1}{2 {\cal M}}\right)\frac{1+arw_{1}^{\prime}}{cr}
718: \label{eq:order1byr}
719: \end{equation}
720:
721: We can now solve the set of three coupled equations to get a quartic equation for
722: $w_{1}^{\prime}$ as a function of $a,\alpha,{\cal M} $ and $r$. Only one of the
723: roots of the quartic equation is a true solution satisfying all the conditions above.
724: The details of the calculations though straight forward are tedious and not shown here.
725:
726: To calculate the cost of lookups, we still need to calculate the
727: probability that a finger is dead. The loss and gain terms for this calculation are
728: almost exactly the same as carried out earlier, in \cite{KEAH1,KEAH2}
729: (except for term $c_{3.2}$) and are shown in table \ref{tab:f}.
730:
731: \begin{table}
732: \caption{The relevant gain and loss terms for $F_k$, the number of nodes whose $k{th}$ fingers are pointing to a failed node for $k > 1$.}
733: \label{tab:f}
734: \centering
735: \begin{tabular}{|l|l|} \hline
736: $F_k(t+\Delta t)$ & \minorchange{Probability of Occurence} \\ %\hline
737: $= F_k(t)+1$ & $c_{3.1}=(\lambda_j{N} \Delta t) \sum_{i=1}^{k}p_{\minorchange{\it join}}(i,k)f_i$
738: \\ %\hline
739: $= F_k(t)-1$ & $c_{3.2}= \frac{f_k}{\sum_k f_k} (\lambda_M {N_{S_2}}(1- w_{1}^{\prime}) A(w_1,w_{1}^{\prime}) \Delta t)$ \\ %\hline
740: $= F_k(t)+1$ & $c_{3.3}= (1-f_k)^2 [1-p_{1}(k)] (\lambda_f {N} \Delta t)$ \\ %\hline
741: $= F_k(t)+2$ & $c_{3.4}= (1-f_k)^2 (p_{1}(k)-p_{2}(k)) (\lambda_f{N} \Delta t)$ \\ %\hline
742: $= F_k(t)+3$ & $c_{3.5}= (1-f_k)^2 (p_{2}(k)-p_{3}(k)) (\lambda_f {N} \Delta t)$ \\ %\hline
743: $= F_k(t)$ & $1 - (c_{3.1} + c_{3.2} + c_{3.3}+ c_{3.4}+ c_{3.5})$\\ \hline
744: \end{tabular}
745: \end{table}
746:
747: The term $c_{3.2}$ is the probability that a message is sent ($\lambda_M N_{S_2}$)
748: times the probability that a $k^{th}$ pointer gets
749: this message (with probability
750: $f_k/\sum f_k$ since {\it only} nodes with wrong pointers
751: get the messages), times the probability that the message is not outdated
752: ($1-w_{1}^{\prime}$), times the probability that the predecessor of the node
753: which has to receive the message has a correct successor pointer. This
754: last quantity is denoted by
755: $A(w_1,w_{1}^{\prime})= 1- (w_1 P_{S_1} + w_{1}^{\prime} P_ {S_2})$, since the predecessor
756: could have been an $S_1$ or an $S_2$ type node.
757:
758: An estimate for $\sum f_k$ is simply $\sim {\cal M} N_{S_2}/N$. Substituting this
759: in term $c_{3.2}$, this term becomes $= \lambda_M N \Delta t (f_k/{\cal M}) (1-w_{1}^{\prime}) A(w_1,w_{1}^{\prime})$
760:
761: Solving for $f_k$ in the steady state, and substituting for $w_{1}^{\prime}$, we
762: get $f_k$ as a function of the parameters. As mentioned earlier a quick and precise
763: estimate of the lookup length is then
764: obtained by taking $L= A(1+f+3f^2)$.
765:
766:
767: \subsection{Comparison of Correction-on-change and Periodic Stabilisation}
768: \label{compare}
769:
770: In order to compare how the two strategies perform
771: under churn, we need to make sure that
772: we are comparing lookup latencies for the same number of total maintenance
773: messages sent.
774:
775: Let us assume that the maximum rate for sending messages per node is $C$.
776: In the case of periodic stabilisation, this implies
777: that the rate of doing successor stabilisations $\lambda_{s_1}$ and finger
778: stabilisations $\lambda_{s_2}$ must in total not exceeed $C$.
779: This implies that $\lambda_{s_1}/C + \lambda_{s_2}/C \le 1$.
780: If we assume that all nodes always send messages up to their maximum capacity, then
781: clearly $\lambda_{s_1}/C + \lambda_{s_2}/C = 1$.
782: Suppose we define $r\equiv C/\lambda_j$ and $r_1 \equiv \lambda_{s_1}/ \lambda_j, r_2 \equiv \lambda_{s_2}/\lambda_j$. Then for a given value of
783: $r$, $r_1+r_2=r$. Hence if finger
784: stabilisations are done at rate $(1-\beta) r$, the successor stabilisations
785: need to be done at rate $\beta r$, where
786: the parameter $\beta$ can be varied from $0$ to $1$.
787:
788: In the case of correction-on-change, we need to impose the same
789: maximum rate $C$
790: no matter which state the nodes are in.
791: In this case, let
792: $\lambda_{S_1}$ be the rate of successor stabilisation in state $S_1$,
793: $\lambda_{S_2}$ the rate of successor stabilisation in state $S_2$ and
794: $\lambda_{S_3}$ be the rate of sending messages in state $S_2$.
795: Clearly $\lambda_{S_1}=C$ and $\lambda_{S_2} + \lambda_{S_3} =C$.
796: Defining $r$ as before, we get
797: $\lambda_{s_1}/ \lambda_j = r$ and
798: $\lambda_{s_2}/ \lambda_j + \lambda_{s_3}/\lambda_j = r$.
799: Hence comparing with our parameters
800: $\alpha=1$ and $a+c=1$.
801:
802: In Fig. ~\ref{fig:coc}, we have plotted the function
803: $L= A(1+f+3f^2)$ with the value of the lookup length
804: without churn $A=5.846$ for $N=1000$ nodes,
805: for $a=0$ (and $c=1$) and
806: for $\beta=0.4$. $f$ is calculated separately for the two maintenance techniques.
807:
808:
809: \begin{figure}[t]
810: \centering
811: \includegraphics[height=8cm, angle=270]{reactive1}
812: \caption{Comparison of the Lookup cost for the two maintenance
813: strategies, for $N=1000$.}
814: \label{fig:coc}
815: \end{figure}
816:
817: As can be seen, correction-on-change is better than periodic stabilisation
818: when churn is low but not when churn is high. On
819: comparing lookup lengths for several different $a$, it becomes evident
820: (see yFig. ~\ref{fig:coc1}) that
821: $a\sim 0.2$ is the optimum value for the correction-on-change strategy.
822:
823: \begin{figure}[t]
824: \centering
825: \includegraphics[height=8cm, angle=270]{coc}
826: \caption{Comparison of the Lookup cost for different values of the parameter $a$, as explained in the text.}
827: \label{fig:coc1}
828: \end{figure}
829:
830:
831: So interestingly, for nodes in state $S_2$, it is not the
832: best strategy to increase $c$ as much as possible. Its a better strategy
833: to spend some of the bandwidth on maintaining a correct successor.
834:
835: \section{Summary}
836: \label{summary}
837: In summary, we have demonstrated the usefulness of the
838: master-equation approach for understanding churn in overlay networks.
839: Our analysis can take into account most details of the
840: algorithms used by these networks, to provide predictions for
841: how the performance depends on the parameters.
842: There are several directions in which we can extend the present analysis.
843: One of the more important ones is to model congestion on the links.
844: This could affect the performance of the two compared
845: maintenance strategies differently. The periodic case
846: may not be as affected as much as the reactive case, which could suffer from
847: congestion collapse.
848:
849:
850: {\bf Acknowledgments} We would like to thank Ali Ghodsi for several very useful discussions.
851:
852:
853:
854: {\footnotesize
855: \bibliographystyle{amsplain}
856: \bibliography{P2P_v2}
857: }
858:
859:
860: \section{Appendix}
861: \label{A1}
862:
863: Equation \ref{eq:cost} with the churn-dependent terms set to zero becomes:
864:
865: \begin{equation}
866: \label{eq:costnochurn}
867: C_{\xi+m} = C_{\xi} \left[1-a(m)\right] + a(m) + \sum_{i=0}^{m-1}b(i)C_{m-i}
868: \end{equation}
869:
870: After some rewriting of this, it is easily seen that the cost for {\it any}
871: key $i+1$ can be written as the following recursion relation:
872:
873: \begin{equation}
874: \label{eq:constnochurn1}
875: C_{i+1} = \rho C_i + (1-\rho) +(1-\rho)C_{i+1-\xi(i+1)}
876: \end{equation}
877:
878: Here we have used the definition of $a$ and $b$ from the internode-interval distribution and the notation $\xi(i+1)$ refers to the {\it start} of the
879: finger most closely preceding $i+1$. For instance, for $i+1=4$,
880: $\xi(i+1)=2$ and for $i+1=11$, $\xi(i+1)=8$ etc.
881:
882: \begin{figure}[t]
883: \centering
884: \includegraphics[height=8cm, angle=270]{nochurn}
885: \caption{Theory and Simulation for the lookup cost without churn for a key space of size ${\cal K}=2^{14}$ for varying $N$. Plotted as reference is the curve $0.5 \log_2(N)$. Note that on the y axis we have actually plotted
886: $L-1$ for convenience.}
887: \label{fig:nochurn}
888: \end{figure}
889:
890: \begin{figure}[t]
891: \centering
892: \includegraphics[height=8cm, angle=270]{ici}
893: \caption{The average cost $C_i$ (the number hops for looking up an
894: item $i$ keys away) in a network of ${\cal N}=1000$ nodes and
895: ${\cal K}= 2^{20} $ keys without churn obtained from the
896: recurrence relation (\ref{eq:constnochurn1}). The average lookup length $L$ is also plotted as a reference.}
897: \label{fig:ici}
898: \end{figure}
899:
900:
901: We are interested in solving the recursion relation and computing $L= \frac{1}{{\cal K}}\sum_{i=1}^{{\cal K}-1}C_i$.
902: To do this, we decompose this sum into the following partial sums:
903: \begin{equation}
904: \begin{split}
905: s_0 &= C_1 = 1 \\
906: s_1 &= C_2 \\
907: s_2 &= C_3 + C_4 \\
908: s_3 &= C_5 + C_6 + C_7 + C_8 \\
909: \ldots \\
910: s_{\cal M} &= C_{2^{{\cal M}-1}+1}+\ldots+C_{{\cal K}-1}
911: \end{split}
912: \end{equation}
913: Substituting the expressions for the $C$'s in the above, we find:
914: \begin{equation}
915: \begin{split}
916: s_0 &= 1 \\
917: s_1 &= \frac{\rho}{1-\rho}[C_1-C_2]+ 1 + s_0 \\
918: s_2 &= \frac{\rho}{1-\rho}[C_2-C_4]+ 2 + [s_0+s_1] \\
919: \ldots \\
920: s_i &= \frac{\rho}{1-\rho}[C_{2^{i-1}}-C_{2^i}]+ 2^{i-1} + \sum_{j=0}^{j-1} s_j
921: \end{split}
922: \end{equation}
923: By substituting serially the expressions for $s_j$ (where $0 \leq j \leq i-1$), the expression for $s_i$
924: (for $i \geq 2$) becomes:
925: \begin{equation}
926: \begin{split}
927: s_i &= \frac{\rho}{1-\rho} [2^{i-2}C_1 - C_{2^i} - \sum_{j=1}^{i-2} s^{i-2-j}C_{2^j}] \\
928: &+ 2^{i}+ (i-1)2^{i-2}
929: \end{split}
930: \end{equation}
931: Hence
932: \begin{equation}
933: \begin{split}
934: \sum_{i=0}^{\cal M} s_i & = -\rho + [2^{{\cal M}+1}-1] + {\cal M}2^{{\cal M}-1} - [2^{{\cal M}}-1] \\
935: &+ \frac{\rho}{1-\rho} \biggl[(2^{{\cal M}-1}-1)C_1 - \sum_{i=2}^{{\cal}M -1}C_{2^i} - C_{{\cal K}-1} \\
936: &- (2^{{\cal M}-2}-1)C_2 - (2^{{\cal M}-3}-1)C_4 - \dots \biggr]
937: \end{split}
938: \end{equation}
939: Therefore
940: \begin{equation}
941: \begin{split}
942: \sum_{i=0}^{\cal M} s_i & = -\rho + 2^{\cal M} + {\cal M}2^{{\cal M}-1} \\
943: &+ \frac{\rho}{1-\rho} \biggl[(2^{{\cal M}-1}-1)C_1 - \sum_{i=2}^{{\cal}M -1}C_{2^i} - C_{{\cal K}-1} \\
944: -& \sum_{j=2}^{{\cal M}-2} (2^{{\cal M}-j}-1)C_{2^{j-1}} \biggr]
945: \end{split}
946: \end{equation}
947: The equation for the average lookup length without churn is thus,
948: \begin{equation}
949: \begin{split}
950: L &= \frac{\sum s}{\cal K} \\
951: &= -\frac{\rho}{\cal K} +1 + \frac{1}{2}{\cal M} \\
952: &+ \frac{\rho}{1-\rho} \biggl[ \frac{2^{{\cal M}-1}-1}{\cal K}C_1
953: - \frac{1}{\cal K} \sum_{i=2}^{{\cal M}-1}C_{2^i}
954: - \frac{1}{\cal K} C_{{\cal K} -1} \\
955: -& \sum_{j=2}^{{\cal M}-2} \frac{2^{{\cal M}-j}-1}{\cal K} C_{2^{j-1}} \biggr]
956: \end{split}
957: \end{equation}
958:
959: If we can take the limit ${\cal K} \to \infty$, we can throw away some of the terms.
960: \begin{equation}
961: \begin{split}
962: \lim_{{\cal K}\to\infty} L &= 1 + \frac{1}{2}{\cal M} \\
963: &+ \frac{\rho}{1-\rho} \biggl[\frac{C_1}{2}
964: - \frac{1}{\cal K} \sum_{i=1}^{{\cal M}-1}C_{2^i}
965: +\frac{C_2}{\cal K}
966: - \frac{1}{\cal K} C_{{\cal K} -1} \\
967: -& \sum_{j=2}^{{\cal M}-2} \frac{2^{{\cal M}-j}}{\cal K} C_{2^{j-1}}
968: + \sum_{j=2}^{{\cal M}-2} \frac{C_{2^{j-1}}}{\cal K} \biggr]\\
969: \approx & 1 + \frac{1}{2}{\cal M} + \frac{\rho}{1-\rho}
970: \left[\frac{C_1}{2}- \frac{C_2}{4}- \frac{C_4}{8} \ldots
971: - \frac{C_{2^{{\cal M}-3}}}{2^{{\cal M}-2}}\right]
972: \end{split}
973: \end{equation}
974: Since $C_1 = 1$, we can write
975: \begin{equation}
976: \label{eq:Kinf}
977: \begin{split}
978: L &= 1 + \frac{1}{2}{\cal M} - \frac{\rho}{2(1-\rho)}
979: \biggl[\frac{C_2 -1}{2} + \frac{C_4 -1 }{4}+ \ldots \\
980: +& \frac{C_{2^{{\cal M}-3}}-1}{2^{{\cal M}-3}}\biggr]
981: \end{split}
982: \end{equation}
983: From the recursion relation for the $C_i$'s, it is easy to see that
984: \begin{equation}
985: (C_i-1) = (1-\rho) g_i^{(1)}(\rho)+(1-\rho)^2 g_i^{(2)}(\rho) + \ldots
986: \end{equation}
987: where the $g_i$'s are functions only of $\rho$.
988:
989: Hence if ($1-\rho$) is small ($\frac{N}{\cal K}\to 0$), we need only compute the $C_i$'s to
990: first order in ($1-\rho$) to get the leading order effect and second order in ($1-\rho$) to get the correction etc.
991:
992: Hence in general the, the expression for $L$ is:
993: \begin{equation}
994: \label{eq:exact}
995: \begin{split}
996: L &= 1 + \frac{1}{2}{\cal M} - \frac{\rho}{2}
997: \biggl[e_1(\rho)+(1-\rho)e_2(\rho)+(1-\rho)^2 e_3(\rho) \ldots \biggr]
998: \end{split}
999: \end{equation}
1000: Where $e_1(\rho)=\sum_{i=1}^{{\cal M}-3} g_{2^i}^{(1)}(\rho)$ etc.
1001:
1002: We evaluate this expression numerically by solving recursion relation (\ref{eq:constnochurn1}) and compare it with
1003: simulations done at zero churn. As can be seen the prediction of the equation is very accurate (Figure \ref{fig:nochurn}).
1004:
1005:
1006: Let us now compute $e_1(\rho)$ to see what the leading order effect is. We now need to solve recursion relation (\ref{eq:constnochurn1}) only to order $1-\rho$, which gives:
1007: \begin{equation}
1008: \begin{split}
1009: C_2-1 &= (1-\rho) \\
1010: C_4-1 &= (1-\rho)\left[1+ \rho +\rho^2 \right] \\
1011: C_8-1 &= (1-\rho)\left[1+ \rho +\rho^2+ \dots + \rho^6 \right] \\
1012: \ldots\\
1013: C_i-1 &= (1-\rho)\left[1+ \rho +\rho^2+ \dots + \rho^{i-2} \right] \\
1014: \end{split}
1015: \end{equation}
1016: Therefore,
1017: \begin{equation}
1018: \begin{split}
1019: L &= 1 + \frac{1}{2}{\cal M} + \frac{\rho}{2}
1020: \biggl[\frac{1}{2}+\frac{1+\rho+\rho^2}{4}+ \ldots \biggr]
1021: \end{split}
1022: \end{equation}
1023: Consider the expression inside the brackets. We are computing this in the approximation $\frac{N}{\cal K}=\epsilon \to 0$, i.e. $\rho =1 - \epsilon$, therefore $\rho^x = (1-\epsilon)^x \approx e^{-\epsilon x}$. If $x > \frac{1}{\epsilon}$, then $\rho^x \to 0$, therefore if $x > \frac{\cal K}{N}$, then $\rho^x \to 0$. Hence, the terms inside the brackets become:
1024: \begin{equation}
1025: \label{eq:T}
1026: \begin{split}
1027: \sum_{j=1}^{T} \frac{2^{j}-1}{2^j} + (2^T -1)\sum_{j=T+1}^{{\cal M}-3} \frac{1}{2}j
1028: \end{split}
1029: \end{equation}
1030: Where $T \equiv \ln_2{\cal K}-\ln_2{N}$ and we have put $\rho^x \approx 1$ for $x < \frac{\cal K}{N}$ and $\rho \to 0$ for $x > \frac{\cal K}{N}$. This is clearly an overestimation and so we expect the result to over estimate the exact expression
1031: \ref{eq:exact}.
1032:
1033: Expression \ref{eq:T} becomes:
1034:
1035: $$T-\left[1-(\frac{1}{2})^{{\cal M}-3}\right] + \left[1-(\frac{1}{2})^{{\cal M}-3-T}\right] \approx T$$
1036: Therefore:
1037: \begin{equation}
1038: \label{eq:chder}
1039: \begin{split}
1040: L & = 1+ \frac{1}{2}{\ln_2{\cal K}} - \frac{1}{2}\left[ \ln_2{\cal K} - \ln_2N\right] \\
1041: & \approx 1+ \frac{1}{2}\ln_2N
1042: \end{split}
1043: \end{equation}
1044: Which is the known result for the average lookup length of Chord.
1045:
1046: Another important parameter in the performance of DHTs in general is the
1047: base. By increasing the base, the number of fingers per node increases
1048: which leads to a shorter lookup path length. The effect of varying the
1049: base has been studied in \cite{onana03dks,dhtcomparison:infocom05}. So
1050: far, we have considered in this analysis base-$2$ Chord. We can likewise
1051: carry out this analysis for any base.
1052:
1053:
1054: In general, we have base-$b$ with $(b-1) log_b({\cal K})$ fingers per node.
1055: Consider as an example $b=4$. Here we can define the
1056: the partial sums again in the following manner:\\
1057: \begin{equation}
1058: \begin{split}
1059: \Delta_0 &= s_0 = C_1=1 \\
1060: \Delta_1 &= s_1 + s_2 + s_3 \\
1061: \Delta_2 &= s_4 + s_5 + s_6 \\
1062: \ldots &
1063: \end{split}
1064: \end{equation}
1065: where
1066: \begin{equation}
1067: \begin{split}
1068: s_1 &= C_2 = \rho C_1 + (1-\rho) + (1-\rho)C_1\\
1069: s_2 &= C_3 = \rho C_2 + (1-\rho) + (1-\rho)C_1\\
1070: s_3 &= C_4 = \rho C_3 + (1-\rho) + (1-\rho)C_1\\
1071: s_4 &= C_5 + C_6 + C_7 + C_8 \\
1072: s_5 &= C_9 + C_{10} + C_{11} + C_{12} \\
1073: s_6 &= C_{13} + C_{14} + C_{15} + C_{16}\\
1074: \ldots &
1075: \end{split}
1076: \end{equation}
1077: Therefore
1078: \begin{equation}
1079: \begin{split}
1080: \Delta_0 &= C_1 \\
1081: \Delta_1 &= \rho \left[\Delta_1 + C_1 -C_4 \right] + 3 (1-\rho) + 3 (1-\rho) \left[ \Delta_0 \right] \\
1082: \Delta_2 &= \rho \left[\Delta_2 + C_4 -C_{16} \right] + 12(1-\rho) + 3 (1-\rho) \left[ \Delta_0+\Delta_1 \right] \\
1083: \ldots &
1084: \end{split}
1085: \end{equation}
1086: In general for a base $b$, define $B \equiv b-1$ and $b^{\cal M}={\cal K}$.
1087: Then we have:
1088: \begin{equation}
1089: \begin{split}
1090: \Delta_j &= \frac{\rho}{1-\rho} \left[C_{b^{j-1}} - C_{b^j} \right] \\
1091: +& B(B+1)^{j-1} + B \left[ \Delta_0+ \Delta_1+ \dots + \Delta_{j-1} \right]
1092: \end{split}
1093: \end{equation}
1094:
1095: Following much the same procedure as before, we find
1096: \begin{equation}
1097: \begin{split}
1098: L =& \frac{1}{\cal K} \sum_{j=0}^{\cal M}\Delta_j \\
1099: \approx & 1+ \frac{B}{B+1} {\cal M} - \frac{B}{B+1} \frac{\rho}{1-\rho} \left[ \frac{C_b-1}{B+1} + \frac{C_{b^2}-1}{(B+1)^2} + \ldots \right]
1100: \end{split}
1101: \end{equation}
1102: for ${\cal K} \to \infty$ as the analogue of (\ref{eq:Kinf}).
1103: Again we can simplify and slightly overestimate the sum by assuming that $\rho^x \approx 0$ for $x > \frac{\cal K}{N}$ and $\rho^x \approx 1$ for $x < \frac{\cal K}{N}$. Then we get:
1104: \begin{equation}
1105: \begin{split}
1106: L \approx 1 + \frac{b-1}{b}\frac{\ln_2N}{\ln_2b}
1107: \end{split}
1108: \end{equation}
1109:
1110: This is the analogue of Eq. \ref{eq:chder} for any base $b$.
1111:
1112:
1113:
1114:
1115:
1116: %\end{multicols}
1117: %\clearpage
1118: %\input{fixsucc.tex}
1119: %\input{fixfingers.tex}
1120:
1121:
1122: \end{document}
1123:
1124:
1125: