1: %%%%%%%%%
2: % Akshay Kashyap, Hawthorne, NY
3: % began August 11, 2004. 6:15pm.
4: % last updated: Nov 17, 04, 11:18am.
5: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
6: \documentclass[10pt]{article}
7: %\evensidemargin 0.6in
8: \oddsidemargin 0.25in
9: \topmargin 0.0in
10: \textwidth 6.0in
11: \headheight 0.0in
12: \headsep 0.0in
13: \textheight 9.0in
14: \footskip 0.2in
15: %\footheight 1in
16: \renewcommand{\baselinestretch}{1.0}
17:
18: \DeclareSymbolFont{AMSb}{U}{msb}{m}{n}
19: \DeclareMathSymbol{\N}{\mathbin}{AMSb}{"4E}
20: \DeclareMathSymbol{\Z}{\mathbin}{AMSb}{"5A}
21: \DeclareMathSymbol{\R}{\mathbin}{AMSb}{"52}
22: \DeclareMathSymbol{\Q}{\mathbin}{AMSb}{"51}
23: \DeclareMathSymbol{\I}{\mathbin}{AMSb}{"49}
24: \DeclareMathSymbol{\C}{\mathbin}{AMSb}{"43}
25:
26: \usepackage{times,psfrag,graphicx,color,authblk}
27: \usepackage{amsfonts,amsmath,amssymb} %{amsthm}
28: \usepackage{psfrag,graphics,epsfig}
29: %\usepackage{notes}
30: %\pagestyle{empty}
31:
32: \makeatother
33: \author{Akshay Kashyap}
34: \affil{\small{Dept. of ECE, UIUC, Urbana, IL. Tel: 217-766-2537, Email: {\tt{kashyap@uiuc.edu}}.}}
35: \author{Luis Alfonso Lastras-Monta\~{n}o}
36: \affil{\small{IBM T.J. Watson Research Center, Yorktown Heights, NY. Email: \tt{lastrasl@us.ibm.com}.}}
37: \author{Cathy Xia}
38: \author{Zhen Liu}
39: \affil{\small{IBM T.J. Watson Research Center, Hawthorne, NY. Emails: \tt{cathyx@us.ibm.com}},
40: \tt{zhenl@us.ibm.com}.}
41: \begin{document}
42:
43: \input{mymacros.tex}
44:
45: %\twocolumn
46:
47: \title{Distributed source coding in dense sensor networks}
48: %\author{}
49: \date{}
50: \maketitle{}
51: \begin{abstract}
52: We study the problem of the reconstruction of a Gaussian field defined in $[0,1]$ using $N$
53: sensors deployed at regular intervals. The goal is to quantify the total data rate required
54: for the reconstruction of the field with a given mean square distortion.
55: We consider a class of two-stage mechanisms which a) send information to allow the
56: reconstruction of the sensor's samples within sufficient accuracy, and then b) use these
57: reconstructions to estimate the entire field. To implement the first stage, the heavy
58: correlation between the sensor samples suggests the use of distributed coding schemes to
59: reduce the total rate. We demonstrate the existence of a distributed block coding scheme that
60: achieves, for a given fidelity criterion for the reconstruction of the field, a total
61: information rate that is bounded by a constant, independent of the number $N$ of sensors.
62: %The rate of this scheme is within a constant, independent of $N$, of the minimum information
63: %rate required by an encoder that has access to all the sensor measurements simultaneously.
64: The constant in general depends on the autocorrelation function of the field and the desired
65: distortion criterion for the sensor samples. We then describe a scheme which can be implemented
66: using only scalar quantizers at the sensors, without any use of distributed source coding,
67: and which also achieves a total information rate that is a constant, independent of the number of
68: sensors. While this scheme operates at a rate that is greater than the rate achievable through
69: distributed coding and entails greater delay in reconstruction, its simplicity makes it attractive
70: for implementation in sensor networks.
71: \end{abstract}
72:
73: \section{Introduction}\seclbl{intro}
74:
75:
76: In this paper, we consider a sensor network deployed for the purpose of sampling
77: and reconstructing a spatially varying random process. For the sake of concreteness,
78: let us assume that the area of interest is represented by the line segment $[0,1]$,
79: and that the for each $s\in [0,1]$, the value of the random process is $X(s)$. For
80: example, $X(s)$ may denote the value of some environmental variable, such as temperature,
81: at point $s$.
82:
83: A sensor network, for the purpose of this paper, is a system of sensing devices (sensors) capable of
84: \ben
85: \item taking measurements from the environment that they are deployed in, and
86: \item communicating the sensed data to a fusion center
87: for processing.
88: \een
89: The task of the fusion center is to obtain a reconstruction $\{\tilde{X}(s), s\in [0,1]\}$
90: of the spatially varying process, while meeting some distortion criteria.
91:
92: There has been great interest recently in performing such sensing tasks with small, low
93: power sensing devices, deployed in large numbers in the region of
94: interest~\cite{neuhoff},~\cite{servetto},~\cite{neupra}~\cite{ishwaretaljourn}.
95: This interest is motivated by the
96: commercial availability
97: of increasingly small and low-cost sensors which have a wide array of sensing
98: and communication functions built in (see, for example,~\cite{mica2dot}), and yet
99: must operate with small, difficult to replace batteries.
100:
101: Compression of the sensed data is of vital importance in a sensor network.
102: Sensors in a wireless sensor network operate under severe power constraints, and
103: communication is a power intensive operation. The rate at which sensors must
104: transmit data to the fusion center in order to enable a satisfactory reconstruction
105: is therefore a key quantity of interest. Further, in any communication
106: scheme in which there is an upper bound (independent of the number of sensors)
107: on the amount of data that the fusion center can receive per unit time, there is another
108: obvious reason why the compressibility of sensor data is important - the average
109: rate that can be guaranteed between any sensor and the fusion center varies inversely
110: with the number of sensors. Therefore, any scheme in which the per-sensor rate decreases
111: slower than inversely with the number of sensors will build backlogs of data at sensors
112: for large enough number of sensors.
113:
114: Environmental variables typically vary slowly as a function of space and it is reasonable
115: to assume that samples at locations close to each other will be highly correlated.
116: The theory of distributed source coding (\cite{slepianwolf},~\cite{zberger},~\cite{tavya})
117: shows that if the sensors have knowledge of this correlation, then it is possible
118: to reduce the data-rate at which the sensors need to communicate, while still maintaining
119: the property that the information conveyed by each sensor depends only on that sensor's
120: measurements. Research on practical techniques
121: (\cite{discus},~\cite{colemanlc},~\cite{stankovic},~\cite{litublum},~\cite{chenhejagmohan})
122: for implementing distributed source coding typically focuses on two correlated sources,
123: with good solutions for the many sources problem still to be developed.
124: Thus, in our work,
125: we attack the problem at hand using the available theoretical tools which have their origins
126: in~\cite{slepianwolf}.
127:
128: This approach has been taken earlier in~\cite{neuhoff} and~\cite{servetto}, which investigate
129: whether it is possible to use such distributed coding schemes to reduce the {\em per-sensor}
130: data rate by deploying a large {\em number} of sensors at closely spaced locations in the area of interest.
131: In particular, it is investigated whether it is possible to construct coding schemes in
132: which the per-sensor rate decreases inversely with the number of sensors.
133: The conclusion of~\cite{neuhoff}, however, is that if the sensors quantize the samples
134: using scalar quantizers, and then encode them, the sum of the data rates of all sensors
135: increases as the number of sensors increases (even with distributed coding), and therefore
136: the per-sensor rate cannot be traded off with the number of sensors in the manner described above.
137:
138: Later, though, it was demonstrated in~\cite{mydcc05} that there exists a distributed coding
139: scheme
140: %which indeed does achieve the desired tradeoff, in that
141: which achieves a sum rate that is a constant independent of the number of sensors used (so long as there is a
142: large enough number of sensors). The per-sensor rate of such a scheme therefore
143: decreases inversely with the number of sensors, which is the trade-off of sensor
144: number with per-sensor rate that was desired, but shown unachievable with scalar quantization,
145: in~\cite{neuhoff}. Results similar to those of~\cite{mydcc05} for the case when a field of
146: infinite size is sampled densely have since appeared in~\cite{neupra}.
147: However, a question that still appears to be unresolved is whether it is possible to
148: achieve a per-sensor rate that varies inversely with the number of sensors using a
149: {\em simple} sensing (sampling, coding, and reconstruction) scheme.
150:
151: This paper is an
152: expanded version of~\cite{mydcc05}. We describe the distributed coding scheme of~\cite{mydcc05}
153: in detail, and then study another sampling and coding scheme which achieves the desired
154: decrease of per-sensor rate with the number of sensors. The two main properties of this scheme
155: are that (1) it does not make use of distributed coding and therefore does not require
156: the sensors to have any knowledge of the correlation structure of the spatial variable of interest,
157: and (2) it can in fact be implemented using only scalar quantizers at the sensors for the purpose
158: of coding the samples. The scheme utilizes the fact that the sensors are synchronized, which is
159: already assumed in the models of~\cite{neuhoff},~\cite{servetto},~\cite{neupra}, and is easily
160: achievable in practice. Since scalar quantizers are easily implementable in sensors with very low
161: complexity, this paper shows that it is possible achieve per-sensor rates that decrease
162: inversely with the number of sensors with simple, practical schemes.
163:
164: A brief outline of this paper is as follows: We pose the problem formally and establish
165: notation in~\secref{problem}. We study the achievability of the above tradeoff with
166: a distributed coding scheme in~\secref{dsc}, and compare the rate of
167: this coding scheme with that of a reference centralized coding scheme in~\secref{ref}.
168: We describe the simple coding scheme mentioned above in~\secref{p2p}. Some numerical results
169: are presented in~\secref{numerical}. We make some concluding remarks in~\secref{conclusion}.
170:
171: \subsection{Problem statement}\seclbl{problem}
172:
173: \subsubsection{Model for the spatial process}\seclbl{assfield}
174: We take a discrete time model, and assume that the spatial process of interest is modeled
175: by a (spatially) stationary, real-valued Gaussian random process, $X^{(i)}(s)$ at each time $i$, where
176: $s$ is the space variable. The focus of this paper is the sampling and reconstruction of
177: a finite section of the process, which we assume without loss of generality to be the interval
178: $[0,1]$. We follow conventional usage in referring to the spatial process
179: $X^{(i)}=\{X^{(i)}(s),s\in [0,1]\}$ as the {\em field} at time $i$.
180:
181: We assume that the field $X^{(i)}$ at time $i$ is independent of the field $X^{(j)}$ for any
182: $j\neq i$, and has identical statistics at all times. (In what follows, we omit the time index
183: when we can do so without any ambiguity.) For simplicity, we assume that $X$ is centered,
184: $\cE[X(s)] = 0$, and that the variance of $X(s)$ is unity, for all $s\in [0,1]$.
185: The autocorrelation function of the field is denoted as
186: \[
187: \rho(\tau) = \cE\left[ X(s) X(s + \tau)\right].
188: \]
189: Following common usage, we sometimes refer to $\rho$ as the {\em correlation structure} of the
190: field.
191: Clearly, $\rho(0) = 1$, and $\rho(\tau) \leq 1$ for any $\tau$.
192: We need only mild assumptions on the field $X$:
193: \ben
194: \item We assume that $X$ is mean-square continuous, which is equivalent to the continuity
195: of $\rho$ at $0$ (see, for example,~\cite{hajekrpnotes}).
196: \item We assume that there is a neighborhood of $0$ in which $\rho$ is
197: non-increasing.
198: \een
199:
200: Note that all results in this paper extend to fields in higher dimensions. We restrict the exposition
201: to one-dimensional fields for clarity and to avoid the tedious notation required for
202: higher dimensional fields.
203:
204: \subsubsection{Assumptions on the sensor network}\seclbl{assnet}
205:
206: We assume that $N$ sensors are placed at regular intervals in the segment $[0,1]$, with sensor
207: $k$ being placed at $s_k = \frac{2k-1}{2N}$ for $k=1,2,\ldots,N$. Sensors are assumed
208: to be synchronized, and at each time $i$, sensor $k$ can observe the value $X^{(i)}(s_k)$
209: of the field at its location, for each $k$. Sensor $k$ encodes a block of $m$ observations,
210: $[X^{(1)}(s_k),X^{(2)}(s_k),\ldots,X^{(m)}(s_k)]$ into an index $I_k$ chosen from the set
211: $\{1,2,\ldots,\lfloor e^{mR_k}\rfloor\}$, where $R_k$ is the rate of sensor $k$, which we
212: state in the units of nats per discrete time unit. We assume that the blocklength $m$ is the
213: same at all sensors. The messages of the sensors are assumed to be communicated to the fusion
214: center over a shared, rate constrained, noiseless channel. The fusion center then uses the received
215: data to produce a reconstruction $\tilde{X}^{(i)}(s)$
216: of the field.
217:
218: A {\em coding scheme} is a specification of the sampling and encoding method used at all sensors,
219: as well as the reconstruction method used at the fusion center.
220:
221: \subsubsection{Error criterion}
222:
223: We refer to $\cE(X^{(i)}(s) - \tilde{X}^{(i)}(s))^{2}$ as the mean square error (MSE)
224: of the reconstruction of the field at point $s$ and time $i$. We measure the error
225: in the reconstruction as the average (over a blocklength) integrated MSE, which is
226: defined as
227: \bq
228: \eqnlbl{costfn}
229: J_{MSE}(m) = \frac{1}{m}\sum_{i=1}^m
230: \int_0^1 \cE\left(\xsupi(s)-\tilde{X}^{(i)}(s)\right)^2ds.
231: \eq
232: We study coding schemes in which, for all large enough blocklengths $m$ and a specified positive
233: constant $D_{net}$, the fusion center is able reconstruct the field with an integrated
234: MSE of less than $D_{net}$, that is, schemes for which
235: \bq
236: \eqnlbl{errorcrit}
237: \lim_{m \rightarrow \infty}J_{MSE}(m)\leq D_{net}.
238: \eq
239:
240: \subsubsection{Sum rate}\seclbl{sumrate}
241:
242: In this paper, we describe coding schemes in which for any given value of $D_{net}$ in~\eqnref{errorcrit},
243: the sum rate, $\sum_{k=1}^{N} R_{k}$, is bounded above by some constant
244: $\bar{R}$ independent of the number $N$ of sensors. The bound $\bar{R}$ may in general
245: depend on $D_{net}$. This allows the per-sensor rate can be traded off with the number of sensors,
246: so that for all $N$ large enough, the rate of each sensor is no more than a constant multiple of $\frac{1}{N}$.
247: %\marginpar{\color{red}Should we mention that this is {\em the} desired scaling. We repeat some of this
248: %in the beginning of~\secref{dsc}}
249:
250: \subsection{Contributions}
251:
252: Our main contributions are:
253: \ben
254: \item We prove the existence of a distributed coding scheme in which, under the assumption
255: that the correlation structure is known at each sensor, a sum rate that is
256: independent of the number of sensors $N$ can be achieved.
257: \item We design a simple coding scheme which can be implemented using scalar quantization at
258: sensors, which does not require the sensors to have any information about the correlation structure,
259: and which makes use of the fact that the sensors are synchronized to achieve a sum rate that is a
260: constant independent of $N$.
261: \een
262:
263: The latter scheme has the advantage of being simple enough to be implementable even with
264: extremely resource-constrained sensors. However, the sum-rate achievable through this scheme is
265: in general greater than the sum-rate achievable through distributed coding. Also, unlike distributed coding,
266: this scheme entails a delay that increases with the number of sensors in the network.
267:
268: %\subsection{Related work}\seclbl{relwork}
269:
270: \section{Distributed coding}\seclbl{dsc}
271:
272: In this section we describe a distributed coding scheme which achieves the desired scaling.
273:
274: \subsection{Encoding and decoding}
275: The scheme consists of $N$ encoders, $\{f_k\}_{k=1}^{N}$, where $f_k$ is the encoder at
276: sensor $k$, and $N$ decoders, $\{g_k\}_{k=1}^{N}$ at the fusion center. For each $k$,
277: the rate of $f_k$ is assumed to be $R_k$, and $f_k$ maps the block
278: \[
279: [X^{(1)}(s_k),X^{(2)}(s_k),\ldots,X^{(m)}(s_k)]
280: \]
281: of samples to an index $I_k$ chosen
282: from $\{1,2,\ldots,\lfloor e^{mR_k}\rfloor\}$, which is then communicated to the fusion center.
283: While the output of encoder $k$ may {\em not} depend on the {\em realizations} of the
284: observations at any other sensor $i\neq k$, it is assumed that all sensors have
285: knowledge of the statistics of the field (in particular, the function $\rho$ is assumed
286: known at each sensor\footnote{In practice, the sensors need only know the vector
287: $\left[\rho\left(\frac{1}{N}\right),\rho\left(\frac{2}{N}\right),\ldots,\rho\left(\frac{N-1}{N}\right)\right]$.})
288: and utilize this information to compress their samples. The decoders
289: may use the messages received from all encoders to produce their reconstruction:
290: \begin{eqnarray}
291: \nno \tilde{X}^{(1,\cdots,m)}(s_{k}) =
292: g_{k}(f_{1}(X^{(1,\cdots,m)}(s_{1})),\cdots,f_{{N}}(X^{(1,\cdots,m)}(s_{N}))),
293: %\eqnlbl{distributed}
294: \end{eqnarray}
295: where $X^{(1,\cdots,m)}(s_k)$ is shorthand for $[X^{(1)}(s_k),X^{(2)}(s_k),\ldots,X^{(m)}(s_k)]$,
296: for $k=1,\ldots,N$ and similarly for $\tilde{X}$.
297:
298: \subsection{Reconstructing the continuous field}\seclbl{dscrec}
299:
300: The reconstruction of the field for those values of $s \in [0,1]$ where there are
301: no sensors is done in a two-step fashion as follows.
302: In the first step, the estimates $\tilde{X}(s_k)$ of sensor samples are obtained as described above.
303: Then, the value of the field between sensor locations is found by interpolation.
304:
305: The interpolation $\tilde{X}(s)$ for $s\notin \{s_k | k=1,\ldots,N\}$ is based on the minimum
306: MSE estimator for $X(s)$ given the value of the sample closest to $s$. Formally,
307: for any $s$, define $n(s) = \frac{2k+1}{2N}$ if $s\in[\frac{k}{N},\frac{k+1}{N})$ as the
308: location of the sample closest to $s$. Then, given $X(n(s))$, the minimum MSE estimate for
309: $X(s)$ is given by $\cE[X(s) | X(n(s))] = \rho(s-n(s))X(n(s))$. The reconstruction of the field at the fusion center
310: is obtained by replacing $X(n(s))$ in this estimate with the quantized version $\tilde{X}(n(s))$,
311: \bq
312: \eqnlbl{interpfn}
313: \tilde{X}(s)= \rho(s-n(s))\tilde{X}(n(s)).
314: \eq
315: While this two-step reconstruction procedure is not optimal in general, it suffices for
316: our purposes.
317:
318: \subsection{Error analysis}\seclbl{eadsc}
319: Define
320: \bq
321: \eqnlbl{jprimemse}
322: J_{MSE}'(m) =\frac{1}{N}\sum_{k=1}^{N}
323: \frac{1}{m}\sum_{i=1}^{m}\cE\left(X^{(i)}(s_k)-\tilde{X}^{(i)}(s_k)\right)^2.
324: \eq
325: Using the upper bound found in equation~\eqnref{ubf} (\appref{bounds})
326: on the error of the coding scheme described above, we see that
327: $\lim_m J_{MSE}(m)\leq D_{net}$ is met if $\lim_mJ_{MSE}'(m)\leq D'(N)$,
328: where
329: %\marginpar{\color{red}NOT REQD: This error analysis uses only second order properties
330: %of the field.}
331: \bq
332: \eqnlbl{dprime}
333: D'(N) = \left(\sqrt{D_{net}-\left(1-\rho(\frac{1}{2N})^2\right)^2}
334: - \sqrt{\rho^2(\frac{1}{2N})(1-\rho^2(\frac{1}{2N}))}\right)^2,
335: \eq
336: given that $N$ is large enough so that $1-\rho^2\left(\frac{1}{2N}\right) < D_{net}$.
337: It is easy to see that $D'(N)$ approaches $D_{net}$ from below as $N\rightarrow\infty$.
338:
339: %for any $\epsilon$ in the interval $(0,D_{net})$, there is a
340: %$N_1 > 0$ such that for $N \geq N_1$, $D'(N) \geq D_{net} - \epsilon$.
341:
342: \subsection{Sum rate}
343:
344: We now study the sum rate of the distributed coding scheme discussed above. We begin
345: with finding the encoding rates required for achieving
346: \bq
347: \eqnlbl{discreteconstr}
348: \lim_mJ_{MSE}'(m)\leq D,
349: \eq
350: for some constant $D$.
351:
352: The rate region $\cR(D)$ is defined as the set of all $N-$tuples of rates
353: $(R_1,R_2,\ldots,R_{N})$ for which there exist encoders
354: $f_k$ and decoders $g_k$, for $k=1,\ldots,N$, such that~\eqnref{discreteconstr} can be met.
355: If a rate vector belongs to the rate region, we say that the corresponding set of rates
356: is achievable.
357:
358: The rate-distortion problem in~\eqnref{discreteconstr} is a Gaussian version of the
359: Slepian-Wolf distributed coding problem~\cite{slepianwolf}.
360: Until recently, the rate region for this problem was not known for even $2$ sources.
361: An achievable region for two discrete sources first appeared
362: in~\cite{berger77}, and was extended to continuous sources in~\cite{zberger}.
363: The extension to a general number of Gaussian sources appears in~\cite{pvmsc}. The two-source
364: Gaussian distributed source coding problem was recently solved in~\cite{tavya},
365: where the achievable region of~\cite{berger77} was found to be tight. The rate region
366: is still not known for more than $2$ sources. We use the achievable region found in~\cite{pvmsc}.
367:
368: Though the result is stated in~\cite{pvmsc} for individual distortion constraints on the sources, the
369: extension to a more general distortion constraint is straightforward. We state the achievable
370: region for distributed source coding in the form most useful to us in~\thmref{tungberg} below.
371: In the statement of the theorem, we use $A \mch B \mch C$ to denote a Markov-chain relationship
372: between random variables $A, B$ and $C$, that is, conditioned on $B$, $A$ is independent of
373: $C$. Also, for any $S \subset \{1,\ldots,N\}$, $\bl{X}_S$ denotes the vector of those sources
374: the indexes of which lie in the set $S$ and $S^c$ denotes the complement of the set $S$.
375: %\marginpar{\color{red}REQD? How should Tung-Berger be stated in general? Can we make do with just stating
376: %second order statistics of the random variables involved? For example, can $\bl{Z}$ be {\em any}
377: %random variable with mean zero and variance $pI$?}
378: \begin{thm}
379: \thmlbl{tungberg}
380: $\cR(D)\supset\cR_{in}(D)$, where $\cR_{in}(D)$ is the set of $N-$tuples of rates for which
381: there exists a vector $\bl{U} \in \R^N$ of random variables that satisfies the following conditions.
382: \ben
383: \item $\forall~S\subseteq\{1,2,\ldots,N\}$,~~~~$\bl{U}_S \mch \bl{X}_S \mch \bl{X}_{S^c} \mch \bl{U}_{S^c}$.
384: \item $\forall~S\subseteq\{1,2,\ldots,N\}$,~~~~$\sum_{i\in S}R_i \geq I(\bl{X}_S;\bl{U}_S|\bl{U}_{S^c})$.
385: \item $\exists~~\tilde{\bl{X}}(\bl{U})$ such that
386: \bq
387: \eqnlbl{thmdiscon}
388: \frac{1}{N}\sum_{i=1}^{N} \cE\left[\left(X(s_i)-\tilde{X}(s_i)(\bl{U})\right)^2\right] \leq D.
389: \eq
390: \een
391: \end{thm}
392: Note that each of the rate-constraints in~\thmref{tungberg} forms some part
393: of the boundary of the achievable region $\cR_{in}$ (see, for example,~\cite{pvmsc}). In particular,
394: the constraint on the sum rate is not implied by any other set of constraints.
395:
396: Constructing a vector $\bl{U}$ satisfying the conditions of~\thmref{tungberg} corresponds to the
397: usual construction of a forward channel for proving achievability in a rate-distortion problem.
398: For each $i$, $U_i$ can be thought of as the encoding of $X(s_i)$.
399:
400: We now construct a $\bl{U}$ that would suffice for our purposes. Consider a random vector $\bl{Z}\in \R^N$
401: that is independent of $\bl{X}$, and has a Gaussian distribution with mean $0$ and covariance matrix
402: $pI$, where $I$ is the identity matrix. Then $\bl{U} = \bl{X} + \bl{Z}$ satisfies the Markov chain constraints
403: %\marginpar{\color{red}NOT REQD\\For non-Gaussian $\bl{X}$, this would only be the best error achievable
404: %by {\em linear} estimators,
405: %but that does not change our argument.}
406: of~\thmref{tungberg}. To find a good bound on the sum rate, we now find a lower bound on the
407: variance $p$ for which there exists an estimator $\tilde{\bl{X}}(\bl{X} +\bl{Z})$ which satisfies
408: condition~\eqnref{thmdiscon}. Since $\bl{X}+\bl{Z}$ is jointly Gaussian with $\bl{X}$, the estimator
409: which minimizes the MSE in~\eqnref{thmdiscon} is the linear estimator,
410: \bq
411: \eqnlbl{optest}
412: \tilde{\bl{X}}(\bl{X}+\bl{Z}) = \Sigma_{\bl{X}(\bl{X}+\bl{Z})}
413: \Sigma_{\bl{X}+\bl{Z}}^{-1}\left(\bl{X}+\bl{Z}\right),
414: \eq
415: where $\Sigma_{\bl{X}(\bl{X}+\bl{Z})} = \cE[\bl{X}(\bl{X}+\bl{Z})^T]$ and $\Sigma_{\bl{X}} = \cE[\bl{X}\bl{X}^T]$.
416: Let $p_{\max}(N,D,\rho)$ be the largest value of $p$ for which the MSE achieved by this estimator
417: satisfies~\eqnref{thmdiscon}. We prove below that for large enough $N$, $p_{\max}$ grows faster than
418: linearly with $N$.
419:
420: \begin{lem}
421: \lemlbl{lingrowth}
422: Let $\rho(\tau)$ be a symmetric autocorrelation function such that $\lim_{t \rightarrow 0} \rho(t) = 1$
423: and a threshold $\theta > 0$ exists for which
424: \begin{enumerate}
425: \item $1 \geq \rho(\tau) \geq \rho(\theta)>0$ if $\tau \in (0,\theta)$ and
426: \item the inequality $1 - \rho^{2}(\theta)/(1+\theta) \leq D$ holds.
427: \end{enumerate}
428: Then
429: \begin{eqnarray*}
430: \liminf_{N \rightarrow \infty} \frac{1}{N}p_{\max}(N,D,\rho) \geq \theta^{2}.
431: \end{eqnarray*}
432: \end{lem}
433: \emph{Note:} The second condition can be met for all $D>0$ since $1 - \rho^{2}(\theta)/(1+\theta) \rightarrow 0 $
434: as $\theta \rightarrow 0$.\\
435: \emph{Proof:}
436: We call a value of $p$ allowable if the expected reconstruction error
437: in~\eqnref{thmdiscon}, with $\bl{U} = \bl{X} + \bl{Z}$, is less than $D$.
438: We find the largest $p$ for the error criterion: $\cE[(\tilde{X}(s_i)-X(s_i))^2] \leq D$
439: for each $i \in \{1,\ldots,N\}$, which is more stringent than the average error requirement
440: of~\eqnref{thmdiscon}.
441:
442: Let us consider the estimation of $X(s_1)$. Since $\tilde{X}(s_i)$ is the best linear estimate of
443: $X(s_i)$ from the data $\bl{X+Z}$, any other linear estimator cannot result in a smaller
444: expected MSE. We take advantage of this observation and choose a linear estimator that although
445: suboptimal, is simple to analyze and yet suffices to establish the lemma.
446:
447: Our estimator for $X(s_1)$ shall be the scaled average
448: $\alpha \sum_{ 1 \leq i \leq N \theta} X(s_i) + Z_{i}$,
449: where $\alpha$ is a parameter to be optimized shortly. To estimate $X(s_i)$ for $i \neq 0$,
450: simply substitute the samples used with those whose indexes lie in the set
451: $\{i+1,\cdots,i+N\theta\}$ (or, for samples at the right edge of the interval $[0,1]$,
452: $\{i-N\theta,\cdots,i-1\}$; this does not lead to any change in what follows because of
453: the stationarity of the field).
454:
455: %\marginpar{\color{red}NOT REQD\\All of this uses only second order properties of the random variables
456: %involved, and so holds irrespective of whether the field is Gaussian or not.}
457: It is not difficult to see that
458: \begin{eqnarray}
459: \nno \lefteqn{ \cE \left(X(s_1) - \alpha \sum_{1 \leq i \leq N \theta} X(s_{i}) + Z_{i} \right)^{2} } \\
460: \nno &=& \cE \left[ X(s_{1})^{2} \right]
461: - 2\alpha \sum_{1 \leq i \leq N \theta} \rho(i/N) + \alpha^{2} \cE \left( \sum_{1 \leq i \leq N \theta} X(s_{i}) \right)^{2}
462: + \alpha^{2} \cE \left( \sum_{1 \leq i \leq N \theta} Z_{i} \right)^{2} \\
463: \nno &\leq& 1 - 2 \alpha (N \theta-1) \rho(\theta) + \alpha^{2} N^{2} \theta^{2} + \alpha^{2} N \theta p \\
464: &=& \left[ 1 - 2 \alpha N \theta \rho(\theta) + \alpha^{2} N^{2} \theta^{2} + \alpha^{2} N \theta p \right] + 2 \alpha \rho(\theta),
465: \eqnlbl{notdifficult}
466: \end{eqnarray}
467: where we have used the inequality $1 \geq \rho(\tau) \geq \rho(\theta)$ for $\tau \in (0,\theta)$ and the
468: fact that the greatest integer not greater than $N \theta$ is at least $N \theta - 1$.
469: The value of $\alpha$ that makes the bracketed expression in~\eqnref{notdifficult} smallest is equal to
470: $\alpha^{*} = \frac{\rho(\theta)}{N \theta + p}$ (we do not optimize the entire expression for
471: simplicity). Substitution of this value yields
472: \begin{eqnarray*}
473: 1 - \frac{\rho^{2}(\theta) }{ 1 + p/(N\theta)}\left( 1 - \frac{2}{N \theta}\right).
474: \end{eqnarray*}
475: Now let $\epsilon > 0$ be sufficiently small so that $\theta^{2} - \epsilon \theta(1 + \theta) > 0$, and let
476: $N$ be sufficiently large so that $\frac{2}{N \theta} < \epsilon$.
477: %\begin{eqnarray*}
478: %\frac{2}{N \theta} < \epsilon
479: %\end{eqnarray*}
480: We can always do this since $\theta$ only depends on $D$ and on the autocorrelation function.
481: %\leq 1 - \frac{1}{2}\frac{\rho^{2}(\theta) }{ 1 + p/(N\theta)}$
482: %as an upper bound in (\ref{eq:notdifficult}), where the latter is a consequence of the assumption
483: %that $N \theta \geq 4$.
484: Now suppose that $p/N = \theta^{2} - \epsilon \theta(1 + \theta)$, then
485: \begin{eqnarray*}
486: 1 - \frac{\rho^{2}(\theta) }{ 1 + p/(N\theta)}\left( 1 - \frac{2}{N \theta}\right) &\leq & 1 - \frac{\rho^{2}(\theta) }{ 1 + p/(N\theta)}( 1 - \epsilon) \\
487: & = & 1 - \frac{\rho^{2}(\theta)}{1 + \theta} \leq D.
488: \end{eqnarray*}
489: The above implies that for $N$ sufficiently large,
490: $\frac{1}{N}p_{\max}(N,D,\rho) \geq \theta^{2} - \epsilon \theta(1 + \theta)$.
491: %\begin{eqnarray*}
492: %\frac{1}{N}p_{\max}(N,D,\rho) \geq \theta^{2} - \epsilon \theta(1 + \theta).
493: %\end{eqnarray*}
494: Taking the liminf, we obtain that for all sufficiently small $\epsilon>0$,
495: \begin{eqnarray*}
496: \liminf_{N \rightarrow \infty} \frac{1}{N}p_{\max}(N,D,\rho) \geq \theta^{2} - \epsilon \theta(1 + \theta).
497: \end{eqnarray*}
498: %Taking the limit as $\epsilon \rightarrow 0$, we obtain the desired lemma conclusion. $\hfill\diamond$
499: Since $\epsilon > 0$ can be arbitrarily small, we obtain the desired conclusion. $\hfill\diamond$
500: \smallskip
501:
502: The purpose of this Lemma is only to establish that $p_{\max}(N,D,\rho)$ grows at least linearly with $N$.
503: The constants presented were chosen for simplicity of presentation.
504:
505: The following is our main result on the rate of distributed coding:
506: %\marginpar{\color{red}REQD? If allowing non-Gaussian sources, we do not need to restrict to a Gaussian
507: %$\bl{Z}$ any more in~\thmref{tungberg}. We must use the same $\bl{Z}$ here. Continued...}
508: \begin{prop}
509: \proplbl{sumrate}
510: The sum rate of the distributed coding scheme described above is bounded above by
511: a constant, independent of $N$.
512: %$\frac{1}{2\theta^2}$, where $\theta$ is as in~\lemref{lingrowth}.
513: %Since the rate for each sensor is the same due to symmetry, the per-sensor rate therefore
514: %decreases inversely with the number $N$ of sensors for all large enough $N$.
515: %decays as $\frac{1}{2N\theta^2}$ for all large enough $N$.
516: \end{prop}
517: {\em Proof:} Consider a vector Gaussian channel with input $\bl{W}\in\R^{N}$ and output $\bl{Y}\in\R^N$,
518: $\bl{Y} = \bl{W} + \bl{Z}$, where $\bl{Z}$ is as above,~and
519: where the power constraint on the input is given by $\cE[\bl{W}^T\bl{W}] \leq N$.
520: Since $\bl{Z}$ is distributed $N(0,pI)$, the capacity of this
521: channel,
522: \[
523: \max_{\bl{W}} I(\bl{W};\bl{W}+\bl{Z})~\mathrm{subject~to~} \cE[\bl{W}^T\bl{W}] \leq N,
524: \]
525: %\marginpar{\color{red}If it turns out that
526: %\thmref{tungberg} can be stated in the form that it is now in, with only second order properties
527: %involved, then we still choose Gaussian $\bl{Z}$ to get the channel with the least capacity.}
528: is equal to $\frac{N}{2}\log\left(1 + \frac{1}{p}\right)$ (see, for example,~\cite{tcbook}).
529:
530: Let $\epsilon > 0$ be any number smaller than $D_{net}$.
531: We know from~\secref{eadsc} that there is an $N_1$ such that for $N \geq N_1$,
532: $D'(N) \geq D_{net} -\epsilon$. Further, from~\lemref{lingrowth}, we know that there exists some
533: $N_2 \geq 0$ and a constant
534: %$\tilde{\theta} > 0$
535: $\theta > 0$ such that for
536: $N \geq N_2$, $p_{\max}(N,D_{net}-\epsilon,\rho) \geq {\theta}^2 N$. Clearly,
537: $p_{\max}(N,D,\rho)$ is a non-decreasing function of $D$, and therefore for $N\geq \max\{N_1, N_2\}$,
538: $p_{\max}(N,D'(N),\rho) \geq p_{\max}(N,D_{net}-\epsilon,\rho)$. It then follows that for
539: $N \geq \max\{N_1, N_2\}$,
540: \[
541: I(\bl{X};\bl{X}+\bl{Z}) \leq \frac{N}{2}\log\left(1 + \frac{1}{{\theta}^2 N}\right).
542: \]
543: Then, using the inequality $\log(1+x) \leq x$, and using the result of~\thmref{tungberg} to
544: substitute $\sum_{k=1}^{N} R_k$ for $I(\bl{X};\bl{X}+\bl{Z})$, we see that
545: \[
546: \sum_{k=1}^{N} R_k = \frac{1}{2{\theta}^2}
547: \]
548: is achievable.
549: $\hfill\diamond$
550: \smallskip
551:
552: The constants in~\propref{sumrate} have been chosen for simplicity. In general, the rates
553: achievable by distributed coding are smaller than the bound found in~\propref{sumrate}.
554:
555: \section{Comparison with a reference scheme}\seclbl{ref}
556:
557: %\marginpar{\color{red}REQD: We are finding a {\em lower bound} on the rate of a scheme. But we know
558: %that for centralized coding, Gaussian sources are the hardest to code. So, here the assumption
559: %for Gassianity is required. Otherwise, we would have to find a field that has the smallest rate
560: %distortion for a given MSE.}
561: In this section, we compare the rate of the distributed coding scheme discussed in~\secref{dsc}
562: with a reference scheme, which for reasons that will become apparent below, we call as
563: {\em centralized} coding.
564:
565: The scheme consists of {\em one} centralized encoder $f$, which has access to samples taken
566: at all sensors at times $\{1,\ldots,m\}$,
567: and $N$ decoders, $\{g_k\}_{k=1}^{N}$ at the fusion center. The encoder maps
568: the samples of the sensors, $X^{(1,\ldots,m)}(s_{1},\ldots,s_{N})$, into an index chosen
569: from the set $\{1,2,\ldots,\lfloor e^{m R^*_N}\rfloor\}$, where $R^*_N$ is the rate
570: of the centralized scheme, and communicates this index to the fusion center.
571: The decoder $g_k$ at the fusion center reconstructs the samples from sensor $k$ from the
572: messages received from the centralized encoder,
573: \begin{eqnarray}
574: \nno \tilde{X}^{(1,\cdots,m)}(s_{k}) = g_{k}(f(X^{(1,\ldots,m)}(s_{1},\ldots,s_{N}))),
575: % \eqnlbl{jointencoding}
576: \end{eqnarray}
577: for $k=1,\ldots,N$.
578:
579: At the fusion center, the reconstruction of the field $\tilde{X}(s)$ is obtained in the
580: same two-step manner described in~\secref{dscrec}: the fusion center constructs estimates
581: $\tilde{X}(s_k)$ of the samples $X(s_k)$, for $k=1,\ldots,N$ from the messages received
582: from the sensors, and then interpolates between samples using~\eqnref{interpfn}.
583:
584: Let $R^*_N(D_{net})$ be the smallest rate for which there exists an encoder $f$ and
585: decoders $\{g_k\}_{k=1}^N$ such that the integrated MSE~\eqnref{costfn} achieved by the above
586: scheme satisfies the constraint~\eqnref{errorcrit}. Then, it is clear that $R^*_N(D_{net})$
587: is a lower bound on the rates of all
588: schemes which use the two-step reconstruction procedure of~\secref{dscrec}. In this section we
589: bound the excess rate of the distributed coding scheme of~\secref{dsc} over the rate
590: $R^*_N(D_{net})$ of the centralized scheme.
591:
592: \subsection{Error analysis}\seclbl{eacomp}
593:
594: Using the lower bound in~\appref{bounds}, equation~\eqnref{lbf}, on the error~\eqnref{costfn} in
595: terms of $J_{MSE}'(m)$ of~\eqnref{jprimemse} we conclude that for $N$ large enough,
596: if $J_{MSE}(m) \leq D_{net}$, then $J_{MSE}'(m) \leq D''(N)$, where
597: %\marginpar{\color{red}NOT REQD:\\ This error analysis again uses only second order properties.}
598: \bqas
599: D''(N) = \frac{2\left(1-\rho^2\left(\frac{1}{2N}\right)\right) +
600: 2\sqrt{\left(1-\rho^2\left(\frac{1}{2N}\right)\right)
601: \left(1-\rho^2\left(\frac{1}{2N}\right) + D_{net}\right)} + D_{net}}{\rho^2\left(\frac{1}{2N}\right)}
602: \eqas
603: Note that $D''(N)$ approaches $D_{net}$ from above as $N\rightarrow\infty$.
604: %Note that for any $\epsilon > 0$, there is a $N_0$ large enough so that for $N > N_0$, $D''(N) \leq D_{net} + \epsilon$.
605:
606: \subsection{Bounding the rate loss}
607: Now, consider
608: \bq
609: \eqnlbl{rdach}
610: \bl{V}^* = \arg\min_{p(\bl{V}|\bl{X})} I(\bl{X};\bl{V}),
611: \mathrm{subject~to~}\frac{1}{N}\cE\left[\|\bl{X}-\bl{V}\|^2_2\right] \leq D''(N).
612: \eq
613: From~\secref{eacomp}, it is clear that the rate of the centralized coding scheme, $R^*_N(D_{net})$
614: satisfies, for any $N$,
615: \[
616: R^*_N(D_{net}) \geq I(\bl{X};\bl{V}^*).
617: \]
618:
619: %\marginpar{\color{red}We are again using the same $\bl{Z}$ as in~\thmref{tungberg}, and so we must recompute the
620: %capacity if the $\bl{Z}$ is not Gaussian.}
621: We now use techniques similar to those in~\cite{zamirloss} to bound the redundancy of
622: distributed coding over the rate of joint coding. Let $\bl{Z}$ be as in~\propref{sumrate}.
623: Expanding $I(\bl{X};\bl{X}+\bl{Z},\bl{V})$ in two ways, we get
624: $I(\bl{X};\bl{X}+\bl{Z}) + I(\bl{X};\bl{V}|\bl{X}+\bl{Z})
625: = I(\bl{X};\bl{V})+ I(\bl{X};\bl{X}+\bl{Z}|\bl{V})$,
626: so that
627: \bqa
628: \eqnlbl{redpt} I(\bl{X};\bl{X}+\bl{Z})-I(\bl{X};\bl{V})&\leq&I(\bl{X};\bl{X}+\bl{Z}|\bl{V})\\
629: \nno&=&I(\left(\bl{X}-\bl{V}\right);\left(\bl{X}-\bl{V}\right)+\bl{Z}|\bl{V}).
630: \eqa
631: Since $\bl{V}\mch (\bl{X}-\bl{V}) \mch (\bl{X}-\bl{V})+\bl{Z}$, we have
632: $I(\left(\bl{X}-\bl{V}\right);\left(\bl{X}-\bl{V}\right)+\bl{Z}|\bl{V})
633: \leq I(\left(\bl{X}-\bl{V}\right);\left(\bl{X}-\bl{V}\right)+\bl{Z})$.
634: Subject to the constraint in~\eqnref{rdach},
635: $I(\left(\bl{X}-\bl{V}\right);\left(\bl{X}-\bl{V}\right)+\bl{Z})$
636: is upper bounded by the
637: capacity of a parallel Gaussian channel, with noise $\bl{Z}$ and input $\bl{W} = \bl{X}-\bl{V}$,
638: the power constraint on which is given by $\frac{1}{N}\cE[\|\bl{W}\|^2] \leq D''(N)$. The capacity of
639: this channel is~\cite{tcbook} $C = \frac{N}{2}\log\left(1 + \frac{D''(N)}{p}\right)$, and
640: therefore
641: %\[
642: %I(\left(\bl{X}-\bl{V}\right);\left(\bl{X}-\bl{V}\right)+\bl{Z}) \leq \frac{N}{2}\log\left(1 + \frac{D''(N)}{p}\right).
643: %\]
644: %Now,
645: from~\eqnref{redpt} and the definition~\eqnref{rdach} of $\bl{V}$ as the
646: rate-distortion achieving random vector, we get
647: \bqas
648: \nno I(\bl{X};\bl{X}+\bl{Z})-R_N^*(D_{net}) &\leq& \frac{N}{2}\log\left(1 + \frac{D''(N)}{p}\right).\\
649: &\leq& \frac{N}{2}\frac{D''(N)}{p},
650: \eqas
651: where the second inequality follows because $\log(1+x) \leq x$.
652: From~\secref{eacomp}, we know that for any $\epsilon > 0$, there is a $N_1$ large enough so that
653: for all $N \geq N_1$, $D''(N) \leq D_{net} + \epsilon$, and we can choose the variance
654: $p$ of the entries of $\bl{Z}$ to be at least $N\theta^2$, where ${\theta}$ is as
655: in~\lemref{lingrowth}, while still ensuring that $\bl{X}+\bl{Z}$ meets the requirements on
656: the auxiliary random variable $\bl{U}$ of~\thmref{tungberg}.
657: Therefore, substituting $\sum_{i=1}^{N} R_i$ for $I(\bl{X};\bl{X}+\bl{Z})$, and using~\lemref{lingrowth}
658: and the result of~\secref{eacomp} we get that for any $\epsilon > 0$,
659: there is an $N_1$ large enough so that for all $N \geq N_1$,
660: \bqa
661: \eqnlbl{redbd}
662: \sum_{i=1}^{N} R_i - R_N^*(D_{net}) &\leq& \frac{D_{net} + \epsilon}{2{\theta}^2}.
663: \eqa
664:
665: We conclude that the rate of the distributed coding scheme of~\secref{dsc} is no more than
666: a constant (independent of $N$) more than the rate of a centralized coding scheme with the
667: same reconstruction procedure.
668: Again, the constant in~\eqnref{redbd} has been chosen for
669: simplicity of presentation and is in general much larger than the actual excess of the
670: rate of the distributed coding scheme (see~\secref{numerical}).
671:
672:
673: \section{Point-to-point coding}\seclbl{p2p}
674:
675: The distributed coding scheme studied in~\secref{dsc} shows that the tradeoff of sensor
676: numbers to sensor accuracy is achievable. However, it may not be feasible to implement
677: complicated distributed coding schemes in simple sensors. In this section we show that
678: if the sensors are synchronized and if a delay that increases linearly with the number
679: of sensors is tolerable, then the desired tradeoff can be achieved by a simple scheme in
680: which encoding can be performed at sensors without any knowledge of the correlation
681: structure of the field.
682:
683: In this scheme, we partition the interval $[0,1]$ into $K$ equal sized sub-intervals,
684: $[0,\frac{1}{K}],(\frac{1}{K},\frac{2}{K}]$,$\ldots$,$(\frac{K-1}{K},1]$. We specify $K$ later, but assume that
685: $N>K$ sensors are placed uniformly in $[0,1]$. We assume that $K$ divides $N$ for simplicity (so that
686: there are an integer number, $\frac{N}{K}$, of samples in each interval).
687:
688: Since the somewhat involved
689: notation may obscure the simple idea behind the scheme, we explain it before describing the scheme in detail.
690: We consider time in blocks of duration $\frac{N}{K}$ units each. The scheme
691: operates overall with a blocklength of $m = m'\frac{N}{K}$, that is, $m'$ blocks, for some integer $m'$.
692: %At any time, exactly one sensor from each sub-interval is active, and
693: Each sensor %in a sub-interval
694: is active exactly once in any time interval that is $\frac{N}{K}$ units in duration.
695: A sensor samples the field at its location only at those times when it is active.
696: Each sensor uses a point-to-point code of blocklength $m'$ and rate $R_p$ nats per {\em active}
697: time unit. The code is chosen appropriately so as to meet the distortion constraint. However,
698: since the sensor is active only in $m'$ out of $m'\frac{N}{K}$ time units, the rate of the
699: code {\em per time-step} is only
700: $\frac{K}{N}R_p$ nats. We show below that the desired distortion can be achieved with a rate $R_p$
701: that is independent of $N$ and therefore the desired scaling can be achieved by the above scheme.
702:
703: We now describe the scheme in detail. Consider the time instants $\left\{1,2,\ldots,m'\frac{N}{K}\right\}$.
704: Each sensor uses a code of blocklength $m = m'\frac{N}{K}$, which is constructed from
705: a code of blocklength $m'$, as follows. For each $j$ in $\{1,2,\ldots,\frac{N}{K}\}$ and each
706: $l$ in $\{0,1,\ldots,K-1\}$, sensor ${\frac{N}{K}l+j}$ (which is the $j$-th sensor from
707: the left in the sub-interval $\left(\frac{l}{K},\frac{l+1}{K}\right]$, and is at
708: location $s_{\frac{N}{K}l+j}$) samples the field only at times
709: $\cT_{l,j} = \{j,j+\frac{N}{K},j+\frac{2N}{K},\ldots,j+\frac{(m'-1)N}{K}\}$. It uses a code
710: of rate $R_p$, to be specified below, to map the $m'$ samples $\{X^{(i)}(s_{\frac{N}{K}l+j}), i \in \cT_{l,j}\}$ to an
711: element of the set $\{1,2,\ldots,\lfloor e^{m'R_p}\rfloor\}$. The rate per-time unit of each
712: sensor is therefore $\frac{1}{m'\frac{N}{K}}m'R_p = \frac{K}{N}R_p$ nats.
713:
714: The fusion center consists of $N$ decoders, one for each sensor. Decoder $k$ constructs
715: estimates of the samples encoded by sensor $k$ {\em using only messages received from
716: sensor $k$}. Then, for each time
717: $i = \frac{N}{K}l + j$ in $\{1,\ldots,m'\frac{N}{K}\}$, the fusion center has reconstructions
718: \[
719: \left[\tilde{X}^{(i)}(s_j), \tilde{X}^{(i)}(s_{\frac{N}{K}+j}), \tilde{X}^{(i)}(s_{\frac{2N}{K}+j}),
720: \ldots, \tilde{X}^{(i)}(s_{\frac{(K-1)N}{K}+j})\right],\] that is, one reconstruction for each sub-interval.
721:
722: For any $s\in[0,1]$, we denote the location of the (unique) sensor active within the
723: interval $(\frac{l}{K},\frac{l+1}{K}]$ to which $s$ belongs by $r^{(i)}(s)$. For each time
724: instant $i$, the fusion center reconstructs the field for $s\neq r^{(i)}(s)$ as
725: \[
726: \tilde{X}^{(i)}(s) = \rho(s-r^{(i)}(s))\tilde{X}^{(i)}(r^{(i)}(s)),
727: \]
728: where $\tilde{X}^{(i)}(r^{(i)}(s))$ is the decoded sample at the fusion center of the sensor
729: at $r^{(i)}(s)$ at time $i$.
730:
731: %\marginpar{\color{red}CAN DO: This error analysis {\em does} use the assumption of Gaussianity. However,
732: %we can do a coarser analysis as in~\appref{bounds} to make it depend only on second order
733: %properties.}
734: We show in~\appref{eap2p} that
735: \bqa
736: \nno \lefteqn{\frac{1}{m}\sum_{i=1}^{m} \int_0^1 \cE[(X^{(i)}(s)-\tilde{X}^{(i)}(s))^2] ds} \\
737: \eqnlbl{p2pbd} &\leq&(1-\rho^2(\frac{1}{K})) +
738: \frac{1}{N}\sum_{k=1}^{N}
739: \left\{\frac{1}{m'} \sum_{i_k\in\cT_k}\cE[\left(X^{(i_k)}(s_k)-\tilde{X}^{(i_k)}(s_k)\right)^2]\right\}
740: \eqa
741: where, with some abuse of notation, we use $\cT_k$ to denote the set of time steps in which
742: sensor $k$ is active. Note that the cardinality of $\cT_k$ is $m'$ for each $k$.
743:
744: We now choose $K$ large enough so that $(1-\rho^2(\frac{1}{K})) < D_{net}$ and choose
745: \bq
746: \eqnlbl{p2pd}
747: D_K = D_{net} - (1-\rho^2(\frac{1}{K})).
748: \eq
749: %\marginpar{\color{red}NOT REQD\\
750: %Here we are upper bounding the rate, and so using Gaussian sources (hardest to code) is acceptable.}
751: The $m'$-blocklength code used at sensor $k$ for the times that it is active is a code that
752: achieves the rate-distortion bound for the distortion constraint
753: \[
754: \frac{1}{m'} \sum_{i_k\in\cT_k}\cE[\left(X^{(i_k)}(s_k)-\tilde{X}^{(i_k)}(s_k)\right)^2 \leq D_K,
755: \]
756: as $m'\rightarrow\infty$. It is
757: well known that the rate of this code is $R_p = \frac{1}{2}\log\frac{1}{D_K}$ nats per time step.
758: It is clear from~\eqnref{p2pbd} and~\eqnref{p2pd} that this scheme achieves the required distortion.
759: Since the rate of each sensor in the overall scheme is $\frac{K}{N}R_p$
760: nats per time step we have therefore constructed a scheme
761: in which the bit rate of each sensor is
762: \bq
763: \eqnlbl{p2prate}
764: -\frac{K}{N}\frac{1}{2}\log\left[D_{net} - (1-\rho^2(\frac{1}{K}))\right]
765: \eq
766: nats per time step. We can now choose $K$ to minimize the sum-rate
767: $-\frac{K}{2}\log\left[D_{net} - (1-\rho^2(\frac{1}{K}))\right]$.
768:
769: Further, it is well known (see~\cite[Section 5.1]{toby}) that using scalar quantization,
770: each sensor can achieve distortion $D_K$ at rate $\frac{1}{2}\log\frac{1}{D_K} + \delta$,
771: where $\delta$ is a small constant. For example, for Max-Lloyd quantizers
772: (see~\cite[Section 5.1]{toby}), $\delta$ is less than $1$ bit.
773:
774: Therefore, we conclude that it is indeed possible to achieve the desired tradeoff between
775: sensor numbers and the per-sensor rate even when the sensors encode their measurements using
776: appropriate scalar quantizers, given that we also make use of the synchronization between sensors
777: to activate sensors appropriately. This is in contrast to the conclusions of~\cite{neuhoff},
778: where full use of synchronization is not made, and therefore it is found that the above tradeoff
779: is not achievable with scalar quantization.
780:
781: \section{Numerical examples}\seclbl{numerical}
782:
783: In this section we give numerical examples of the rates of the coding schemes discussed
784: in~\secref{dsc},~\secref{ref} and~\secref{p2p}. The two fields we consider as
785: examples are (1) a (spatially) band-limited Gaussian field, for which $\rho(\tau) = \sinc(\tau)$,
786: where $\sinc(\tau)=\frac{\sin(\pi\tau)}{\pi\tau}$, and (2) a Gauss-Markov field, for which
787: $\rho(\tau) = \exp\{-|\tau|\}$.
788:
789: For these fields, we numerically find the largest value $p_{\max}$ of the variance $p$ of $\bl{Z}$ for
790: which the error for the estimator in~\eqnref{optest} is no more than the distortion $D'(N)$
791: of~\eqnref{dprime}, with $D_{net} =0.1$. The resulting values are shown in~\figref{pdvar}. We see
792: that for large values of $N$, $p_{\max}$ is indeed approximately linear in $N$.
793: \begin{figure}
794: \centerline{\includegraphics*[width=6cm]{matlab_codeV2/newfigs/pmax_sinc.eps}
795: \includegraphics*[width=6cm]{matlab_codeV2/newfigs/pmax_exp.eps}}
796: \caption{\figlbl{pdvar}Linear increase of $p_{max}$ for large $N$:
797: $\rho(\tau) = \sinc(\tau)$ (left) and $\rho(\tau) = \exp\{-|\tau|\}$ (right). $D_{net} =0.1$.}
798: \end{figure}
799:
800: We compute the achievable sum rate of the distributed source coding scheme, which is equal
801: to $I(\bl{X};\bl{X}+\bl{Z})$ from~\thmref{tungberg}, with the $p_{\max}$ found above as
802: the variance of the entries of $\bl{Z}$. These rates are shown in~\figref{sumrates}.
803: For reference, we also show the lower bound on the
804: rate of the centralized coding scheme computed in~\secref{ref}.
805: \begin{figure}
806: \centerline{\includegraphics*[width=7cm]{matlab_codeV2/newfigs/rates_sinc.eps}
807: \includegraphics*[width=7cm]{matlab_codeV2/newfigs/rates_exp.eps}}
808: \caption{\figlbl{sumrates}Rates of joint and distributed coding (in nats per snapshot) vs.
809: number of sensors $N$: $\rho(\tau) = \sinc(\tau)$ (left) and $\rho(\tau) = \exp\{-|\tau|\}$
810: (right). $D_{net} =0.1$.}
811: \end{figure}
812:
813: In comparison, on minimizing the rate~\eqnref{p2prate} of the point-to-point coding
814: scheme of~\secref{p2p}, we find that best sum rate for $\rho(\tau) = \sinc(\tau)$ is
815: $11.77$ nats for $K=7$ intervals, and that the best sum rate for $\rho(\tau) = \exp(-|\tau|)$
816: is $46.92$ nats with $K=24$ intervals, which is significantly greater than the sum-rate of
817: the distributed coding scheme found above. However, part of the reason
818: for the large sum-rate of the point-to-point coding scheme is that our analysis exaggerates
819: an edge-effect for the sake of simplicity:
820: In~\secref{p2p} we estimated the value of the field at point $s$ at time $i$ using
821: the sample that the fusion center has at time $i$ from the sub-interval that $s$ lies in.
822: We could instead have used the sample {\em closest} to $s$ that is available at the fusion
823: center at time $i$, similar to what is done in~\secref{dsc} and~\secref{ref}. However, this
824: would have meant dealing with the first and the last sub-interval differently, and therefore
825: we did not follow the analysis outlined above. Without this edge effect, the rates of the
826: point-to-point coding scheme are approximately half the rates found above, which are still
827: considerably larger than the sum-rates of the distributed coding scheme.
828:
829:
830: \section{Conclusions}\seclbl{conclusion}
831:
832: We have studied the sum rate of distributed coding for the reconstruction of
833: a random field using a dense sensor network. We have shown the existence of a distributed
834: coding scheme which achieves a sum rate that is a constant independent of the number of
835: sensors. Such a scheme is interesting because it allows us to achieve a per-sensor rate
836: that decreases inversely as the number of sensors, and therefore to achieve small per-sensor
837: rates using a large number of sensors.
838:
839: In obtaining bounds on the sum rate of distributed coding, we made full use to the
840: heavy correlation between samples of the field taken at positions that are close
841: together. When the
842: number of sensors is large, the redundancy in their data can be utilized by coding more
843: and more coarsely: this corresponds to more noisy samples, and is manifested in the
844: growth of the noise $p_{\max}$ in the forward channel in~\secref{dsc}. We believe that
845: this technique of bounding the sum rate is of independent interest.
846:
847: We have also shown that contrary to what has been suggested in~\cite{neuhoff} and~\cite{neupra},
848: it is indeed possible to design a scheme that achieves a constant sum rate with sensors that
849: are scalar quantizers, even {\em without} the use of distributed coding. This scheme, however, requires
850: that we make appropriate use of the synchronization between the sensors, results in
851: a delay in reconstruction which increases linearly with the number of sensors, and achieves
852: rates that may be significantly higher than the rates achieved by distributed coding. The scheme
853: is nevertheless interesting because its low complexity makes it easy to implement.
854:
855: \section*{Acknowledgement}
856: The first author thanks Prof. R. Srikant for many insightful comments on this work,
857: and for his encouragement to work on this paper while the first author was at UIUC.
858:
859: \appendix
860: %\marginpar{\color{red}NOT REQD\\This error analysis uses only second order properties of the field,
861: %and therefore remains unchanged upon relaxing the assumption of Gaussianity.}
862: \section{Bounds on $J_{MSE}(m)$ for the schemes in~\secref{dsc} and~\secref{ref}}\applbl{bounds}
863:
864: We can write the error in reconstruction at any $s\in [0,1]$ as
865: \bqa
866: \nno X(s) - \tilde{X}(s) &=& X(s) - \rho(s-n(s))\tilde{X}(n(s))\\
867: \nno &=& \left[ X(s) - \rho(s - n(s)) X(n(s)) \right]
868: + \left[ \rho(s - n(s))\left(X(n(s)) - \tilde{X}(n(s))\right) \right]\\
869: \eqnlbl{ersplit}
870: &=& E_{S}(s) + E_{Q}(s),
871: \eqa
872: where $E_{S}(s) = X(s) - \rho(s - n(s)) X(n(s))$ and
873: $E_Q(s) = \rho(s - n(s))\left(X(n(s)) - \tilde{X}(n(s))\right)$.
874: Note that in the schemes described in~\secref{dsc} and~\secref{ref}, the encodings
875: of all samples are used to obtain the estimate $\tilde{X}(n(s))$, and therefore
876: $\tilde{X}(n(s))$ is in general not independent of $X(s_k)$, for $s_k\neq n(s)$. As
877: a result, $E_S(s)$ and $E_Q(s)$ are in general not independent. In this appendix,
878: we find upper and lower bounds on $J_{MSE}(m)$ that hold for the schemes of~\secref{dsc}
879: and~\secref{ref}.
880:
881: Using the Cauchy-Schwarz inequality (for any two appropriately integrable random variables
882: $A$ and $B$, $|\cE[AB]| \leq \sqrt{\cE[A^2]\cE[B^2]}$), it is easy to see that
883: \bqa
884: \eqnlbl{ub}
885: \cE\left(E_S(s) + E_Q(s)\right)^2
886: &\leq&
887: \cE\left(E_S(s)\right)^2 + \cE\left(E_Q(s)\right)^2
888: + 2\sqrt{\cE\left(E_S(s)\right)^2\cE\left(E_Q(s)\right)^2}\\
889: \eqnlbl{lb}
890: \cE\left(E_S(s) + E_Q(s)\right)^2
891: &\geq&
892: \cE\left(E_Q(s)\right)^2
893: - 2\sqrt{\cE\left(E_S(s)\right)^2\cE\left(E_Q(s)\right)^2}.
894: \eqa
895: Now, note that $\cE\left(E_S(s)\right)^2 = (1-\rho^2(s-n(s))$. Therefore,
896: \bqa
897: \nno \cE\left(E_S(s)\right)^2\cE\left(E_Q(s)\right)^2 &=&
898: \rho^2(s-n(s))\left(1-\rho^2(s-n(s))\right)\cE\left(X(n(s))-\tilde{X}(n(s))\right)^2.
899: \eqa
900: For $N$ large enough so that both $\rho^2\left(\frac{1}{2N}\right) \geq \frac{1}{2}$ and $1/(2N)$
901: lies in the interval around
902: $0$ in which $\rho$ is non-increasing (so that for $s\in\left(\frac{k}{N},\frac{k+1}{N}\right)$
903: $\rho^2(s-n(s))(1-\rho^2(s-n(s)) \leq\rho^2(\frac{1}{2N})(1-\rho^2(\frac{1}{2N}))$,
904: which holds because the function $h(x)=x(1-x)$ is decreasing in $[\frac{1}{2}, 1]$), we get
905: that
906: \bqa
907: \eqnlbl{sqrtbd}
908: \cE\left(E_S(s)\right)^2\cE\left(E_Q(s)\right)^2 &\leq&
909: \rho^2\left(\frac{1}{2N}\right)\left(1-\rho^2\left(\frac{1}{2N}\right)\right)\cE\left(X(n(s))-\tilde{X}(n(s))\right)^2.
910: \eqa
911:
912: From~\eqnref{costfn} and~\eqnref{ersplit}, we have
913: \bq
914: \eqnlbl{ersplitav}
915: J_{MSE}(m) = \frac{1}{m}\sum_{i=1}^m
916: \int_0^1 \cE\left(E_{S}^{(i)}(s) + E_{Q}^{(i)}(s) \right)^2ds.
917: \eq
918: Therefore, integrating~\eqnref{ub} and~\eqnref{lb} over $[0,1]$, using~\eqnref{sqrtbd} and
919: Jensen's inequality (and the concavity of the function $y(x) = \sqrt{x}$), and averaging over
920: the time index, we get
921: \bqa
922: \eqnlbl{ubf}
923: J_{MSE}(m) &\leq& \left\{1-\rho^2\left(\frac{1}{2N}\right)\right\} + J_{MSE}'(m) +
924: 2\sqrt{\rho^2(\frac{1}{2N})(1-\rho^2(\frac{1}{2N}))J_{MSE}'(m)},\\
925: \eqnlbl{lbf}
926: J_{MSE}(m) &\geq& \rho^2(\frac{1}{2N})J_{MSE}'(m) -
927: 2\sqrt{\rho^2(\frac{1}{2N})(1-\rho^2(\frac{1}{2N}))J_{MSE}'(m)},
928: \eqa
929: where $J_{MSE}'(m)$ is as in~\eqnref{jprimemse}.
930:
931: %\marginpar{\color{red}REQD/CAN DO: Here we have made use of Gaussianity, but we can relax that
932: %by doing a coarser analysis as in~\appref{bounds} above.}
933: \section{Error analysis for the point-to-point coding scheme}\applbl{eap2p}
934:
935: With some abuse of notation, we can still write the error in reconstruction as
936: \[
937: X(s) - \tilde{X}(s) = E_S(s) + E_Q(s),
938: \]
939: where now
940: \bqa
941: \nno E_S(s) &=& X(s) - \rho(s-r(s))X(r(s)), ~\mathrm{and}\\
942: \nno E_Q(s) &=& \rho(s-r(s))\left(X(r(s))-\tilde{X}(r(s))\right).
943: \eqa
944: In the point-to-point coding scheme, the fusion center estimates the samples of each sensor
945: using only the messages that it receives from that particular sensor.
946: Note that $E^{(i)}_S(s)$ is the error in the optimal MSE estimate of $X(s)$ given $X^{(i)}(r(s))$.
947: It is well known that if $\{X(s),s\in [0,1]\}$ is a Gaussian process, the error $E^{(i)}_S(s)$ in is
948: independent of $X^{(i)}(r^{(i)}(s))$.
949: Further, due to the independence of the field $X^{(i)}$ and
950: the field $X^{(j)}$ for any $j\neq i$, $E^{(i)}_S(s)$ is independent
951: of $X^{(j)}(r^{(j)}(s))$ for all $j$, and hence also of the reconstructions
952: $\tilde{X}^{(j)}(r^{(j)}(s))$ and the error terms $E^{(i)}_Q(s)$. Therefore,
953: for any $i$,
954: \[
955: \cE[(X^{(i)}(s)-\tilde{X}^{(i)}(s))^2] = \cE[(E_S^{(i)}(s))^2] + \cE[(E_Q^{(i)}(s))^2].
956: \]
957: Now, for $K$ large enough, $\cE[(E_S^{(i)}(s))^2] = 1-\rho^2(s-r^{(i)}(s)) \leq 1-\rho^2(\frac{1}{K})$
958: for every $s\in [0,1]$. Also, since $\rho^2(s) \leq 1$ for all $s \in [0,1]$,
959: \bqa
960: \nno \cE[(E_Q^{(i)}(s))^2] % &=& \rho^2(s-r^{(i)}(s))\cE[\left(X^{(i)}(r^{(i)}(s))-\tilde{X}^{(i)}(r^{(i)}(s))\right)^2] \\
961: \nno &\leq& \cE[\left(X^{(i)}(r^{(i)}(s))-\tilde{X}^{(i)}(r^{(i)}(s))\right)^2].
962: \eqa
963: So, we get
964: \bqa
965: \nno \int_0^1 \cE[(X^{(i)}(s)-\tilde{X}^{(i)}(s))^2]ds &=& \sum_{l=0}^{K-1}\int_{\frac{l}{K}}^{\frac{l+1}{K}}\cE[(X^{(i)}(s)-\tilde{X}^{(i)}(s))^2]ds\\
966: \nno &\leq& \sum_{l=0}^{K-1}\int_{\frac{l}{K}}^{\frac{l+1}{K}}(1-\rho^2(\frac{1}{K})) + \cE[\left(X^{(i)}(r^{(i)}(s))-\tilde{X}^{(i)}(r^{(i)}(s))\right)^2]ds\\
967: \nno &=& (1-\rho^2(\frac{1}{K})) + \frac{1}{K}\sum_{l=0}^{K-1}\cE[\left(X^{(i)}(r^{(i)}(\frac{l+1}{K}))-\tilde{X}^{(i)}(r^{(i)}(\frac{l+1}{K}))\right)^2],
968: \eqa
969: where we note that by our notation, $r^{(i)}(\frac{l+1}{K})$ is the location of the (unique) sensor active at
970: time step $i$ in the interval $(\frac{l}{K},\frac{l+1}{K}]$.
971:
972: Now summing over the time index we get,
973: \bqa
974: \nno \lefteqn{\frac{1}{m}\sum_{i=1}^{m} \int_0^1 \cE[(X^{(i)}(s)-\tilde{X}^{(i)}(s))^2]ds}\\
975: \nno &\leq& (1-\rho^2(\frac{1}{K}))
976: + \frac{1}{Km}\sum_{i=1}^{m}\sum_{l=0}^{K-1}\cE[\left(X(r^{(i)}(\frac{l+1}{K}))-\tilde{X}(r^{(i)}(\frac{l+1}{K}))\right)^2].
977: \eqa
978: Rearranging the sum on the right and substituting $m = \frac{m'N}{K}$ we get
979: \bqas
980: \lefteqn{\frac{1}{m}\sum_{i=1}^{m} \int_0^1 \cE[(X^{(i)}(s)-\tilde{X}^{(i)}(s))^2]ds}\\
981: &\leq& (1-\rho^2(\frac{1}{K}))
982: + \frac{1}{m'N}\sum_{k=1}^{N}\sum_{i_k\in\cT_k}\cE[\left(X^{(i_k)}(s_k)-\tilde{X}^{(i_k)}(s_k)\right)^2],\\
983: &=& (1-\rho^2(\frac{1}{K}))
984: + \frac{1}{N}\sum_{k=1}^{N}
985: \left\{\frac{1}{m'} \sum_{i_k\in\cT_k}\cE[\left(X^{(i_k)}(s_k)-\tilde{X}^{(i_k)}(s_k)\right)^2]\right\}
986: \eqas
987: where $\cT_k$ is the set of time steps in which sensor $k$ is active.
988:
989:
990:
991: \bibliographystyle{IEEEtran}
992: \begin{thebibliography}{10}
993: \providecommand{\url}[1]{#1}
994: \csname url@rmstyle\endcsname
995: \providecommand{\newblock}{\relax}
996: \providecommand{\bibinfo}[2]{#2}
997: \providecommand\BIBentrySTDinterwordspacing{\spaceskip=0pt\relax}
998: \providecommand\BIBentryALTinterwordstretchfactor{4}
999: \providecommand\BIBentryALTinterwordspacing{\spaceskip=\fontdimen2\font plus
1000: \BIBentryALTinterwordstretchfactor\fontdimen3\font minus
1001: \fontdimen4\font\relax}
1002: \providecommand\BIBforeignlanguage[2]{{%
1003: \expandafter\ifx\csname l@#1\endcsname\relax
1004: \typeout{** WARNING: IEEEtran.bst: No hyphenation pattern has been}%
1005: \typeout{** loaded for the language `#1'. Using the pattern for}%
1006: \typeout{** the default language instead.}%
1007: \else
1008: \language=\csname l@#1\endcsname
1009: \fi
1010: #2}}
1011:
1012: \bibitem{neuhoff}
1013: D.~Marco, E.~J. Duarte-Melo, M.~Liu, and D.~L. Neuhoff, ``On the many-to-one
1014: transport capacity of a dense wireless sensor network and the compressibility
1015: of its data,'' in \emph{Lecture notes in Computer Science}, L.~J. Guibas and
1016: F.~Zhao, Eds.\hskip 1em plus 0.5em minus 0.4em\relax Springer, 2003, pp.
1017: 1--16.
1018:
1019: \bibitem{servetto}
1020: A.~Scaglione and S.~D. Servetto, ``On the interdependence of routing and data
1021: compression in multi-hop sensor networks,'' in \emph{Proc. of IEEE MOBICOM},
1022: 2002.
1023:
1024: \bibitem{neupra}
1025: D.~Neuhoff and S.~Pradhan, ``An upper bound to the rate of ideal distributed
1026: lossy source coding of densely sampled data,'' in \emph{Proc. of IEEE
1027: ICASSP}, 2006.
1028:
1029: \bibitem{ishwaretaljourn}
1030: P.~Ishwar, A.~Kumar, and K.~Ramchandran, ``On distributed sampling in dense
1031: sensor networks: a ``bit-conservation'' principle,'' \emph{submitted to IEEE
1032: Journal on Selected Areas in Communication}, July 2003.
1033:
1034: \bibitem{mica2dot}
1035: {Crossbow Technologies}, ``{MICA2DOT} datasheet,'' {Available} online at
1036: {\small {\tt http://www.xbow.com/Products/productsdetails.aspx?sid=73}}.
1037:
1038: \bibitem{slepianwolf}
1039: D.~Slepian and J.~Wolf, ``Noiseless coding of correlated information sources,''
1040: \emph{Transactions on Information Theory}, vol. IT-19, pp. 471--480, July
1041: 1973.
1042:
1043: \bibitem{zberger}
1044: R.~Zamir and T.~Berger, ``Multiterminal source coding with high resolution,''
1045: \emph{IEEE Transactions on Information Theory}, vol.~45, pp. 106--117,
1046: January 1999.
1047:
1048: \bibitem{tavya}
1049: A.~B. Wagner, S.~Tavildar, and P.~Viswanath, ``Rate-region of the quadratic
1050: {Gaussian} two-terminal source-coding problem,'' February 2006, submitted to
1051: IEEE Transactions on Information Theory.
1052:
1053: \bibitem{discus}
1054: S.~S. Pradhan and K.~Ramchandran, ``Distributed source coding using syndromes
1055: ({DISCUS}): design and construction,'' \emph{IEEE Transactions on Information
1056: Theory}, March 2003.
1057:
1058: \bibitem{colemanlc}
1059: T.~P. Coleman, A.~H. Lee, M.~Medard, and M.~Effros, ``Low-complexity approaches
1060: to slepian-wolf near-lossless distributed data compression,'' \emph{IEEE
1061: Transactions on Information Theory}, submitted for publication.
1062:
1063: \bibitem{stankovic}
1064: V.~Stankovic, A.~Liveris, Z.~Xiong, and C.~Georghiades, ``On code design for
1065: the general {S}lepian-{W}olf problem and for lossless multiterminal
1066: communication networks,'' \emph{IEEE Transactions on Information Theory},
1067: submitted for publication.
1068:
1069: \bibitem{litublum}
1070: J.~Li, Z.~Tu, , and R.~S. Blum, ``Slepian-wolf coding for nonuniform sources
1071: using turbo codes,'' in \emph{Proc. of IEEE/ACM Data Compression Conference},
1072: March 2004.
1073:
1074: \bibitem{chenhejagmohan}
1075: J.~Chen, D.~He, , and A.~Jagmohan, ``Slepian-wolf code design via
1076: source-channel correspondence,'' in \emph{Proc. of IEEE International
1077: Symposium on Information Theory}, July 2006.
1078:
1079: \bibitem{mydcc05}
1080: A.~Kashyap, L.~A. {Lastras-Monta\~{n}o}, C.~Xia, and Z.~Liu, ``Distributed
1081: coding in dense sensor networks,'' in \emph{Proc. of IEEE/ACM Data
1082: Compression Conference}, 2005.
1083:
1084: \bibitem{hajekrpnotes}
1085: B.~Hajek, \emph{An Exploration of Random Processes for Engineers}.\hskip 1em
1086: plus 0.5em minus 0.4em\relax Available online at {\small {\tt
1087: http://www.ifp.uiuc.edu/}$\sim${\tt hajek/Papers/randomprocesses.html}},
1088: 2006.
1089:
1090: \bibitem{berger77}
1091: T.~Berger, ``Multiterminal source coding,'' in \emph{The Information Theory
1092: Approach to Communication}, G.~Longo, Ed.\hskip 1em plus 0.5em minus
1093: 0.4em\relax Springer, 1977.
1094:
1095: \bibitem{pvmsc}
1096: P.~Viswanath, ``Sum rate of multiterminal {Gaussian} source coding,'' in
1097: \emph{{DIMACS} Series in Discrete Mathematics and Theoretical Computer
1098: Science}, 2002.
1099:
1100: \bibitem{tcbook}
1101: T.~M. Cover and J.~A. Thomas, \emph{Elements of Information Theory}.\hskip 1em
1102: plus 0.5em minus 0.4em\relax New York: Wiley, 1991.
1103:
1104: \bibitem{zamirloss}
1105: R.~Zamir, ``Rate loss in the {Wyner-Ziv} problem,'' \emph{IEEE Transactions on
1106: Information Theory}, vol.~42, no.~6, pp. 2073--2084, November 1996.
1107:
1108: \bibitem{toby}
1109: T.~Berger, \emph{Rate Distortion Theory}.\hskip 1em plus 0.5em minus
1110: 0.4em\relax Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1971.
1111:
1112: \end{thebibliography}
1113: \end{document}
1114:
1115: %%%%%%%%%%%%%%%%%%%%%
1116: