1: \documentclass[12pt]{article}
2: %\documentclass[12pt,oneside]{IEEEtran}
3: \usepackage{epsfig,amsmath,amssymb,amsfonts,amstext,amsthm}
4: \usepackage{latexsym,graphics,epsf,epsfig,cite}
5: %\usepackage[dvips]{graphicx}
6: %\usepackage{psfrag}
7: %\input{/homes/huawang/huawang/research/paper/ece459/macros1}
8: %\input{/homes/mantrava/tex/renew_conf}
9: %\usepackage{epsfig,psfrag,color,amssymb,amsmath,amsthm,amsfonts,amstext}
10:
11:
12: \topmargin=-0.5in \headsep=0.5in \oddsidemargin=-0.1in
13: %\evensidemargin=-0.25in
14: \textwidth=6.25in
15: %\columnsep=0.4in
16: \textheight=8.5in
17: \parskip=1.5ex
18: \parindent=2ex
19:
20: \footnotesep=3.0ex
21: %\bibliographystyle{plain}
22:
23: %\pagestyle{fancyplain}
24: %\lhead{A. Mantravadi: }
25: %\rhead{}
26: %\cfoot{\thepage}
27: %\headrulewidth 0pt
28:
29:
30: %\renewcommand{\baselinestretch}{1.2}
31: \newcommand{\transprob}{ P_N({\bf {x}e}|{{\bf{\hat{x}}}_m})}
32: \newcommand{\vecxhatm}{{\bf{\hat{x}}}_m}
33: \newcommand{\vecx}{{\bf{x}}}
34: \newcommand{\vecy}{{\bf{y}}}
35: \newcommand{\vecxhat}{{\bf{\hat{x}}}}
36: \newcommand{\xhat}{\hat{x}}
37: \newcommand{\xhatm}{hat{x}_m}
38: \newcommand{\be}{\begin{eqnarray}}
39: \newcommand{\ee}{\end{eqnarray}}
40: \newcommand{\mx}{\mathcal{X}}
41: \newcommand{\my}{\mathcal{Y}}
42: \newcommand{\mmu}{\mathcal{U}}
43: \newcommand{\mms}{\mathcal{S}}
44: \newcommand{\bx}{{\bf X}}
45: \newcommand{\by}{{\bf Y}}
46: \newcommand{\bw}{{\bf W}}
47: \newcommand{\tf}{\tilde{f}}
48: \newcommand{\tphi}{\tilde{\phi}}
49: \newcommand{\fe}{\quad \text{for every} \quad}
50: \newcommand{\myvec} [1] { {\bf #1} }
51: \newcommand{\pp}{\hspace*{1cm}}
52: \newcommand{\qq}{\vspace*{-0.1in}}
53: \newcommand{\Lbr}{\left [}
54: \newcommand{\Rbr}{\right ]}
55: \newcommand{\lbr}{\left \{}
56: \newcommand{\rbr}{\right \}}
57: \newcommand{\lp}{\left (}
58: \newcommand{\rp}{\right )}
59: \newcommand{\E}{{\mathbb E}}
60: \newcommand{\Prob}{{\Bbb P}}
61: \newcommand{\PROB}{{\Bbb P}}
62: \newcommand{\Qrob}{{\Bbb Q}}
63: \newcommand{\QROB}{{\Bbb Q}}
64: \newcommand{\vx}{{{\mathbf{x}}}}
65: \newcommand{\vy}{{{\mathbf{y}}}}
66: \newcommand{\vz}{{{\mathbf{z}}}}
67: \newcommand{\vw}{{{\mathbf{w}}}}
68: \newcommand{\vu}{{{\mathbf{u}}}}
69: \newcommand{\va}{{\mathbf{A}}}
70: \newcommand{\vi}{{\mathbf{I}}}
71: \newcommand{\vd}{{\mathbf{D}}}
72: \newcommand{\vk}{{\mathbf{K}}}
73: \newcommand{\vl}{{\mathbf{\Lambda}}}
74: \newcommand{\vq}{{\mathbf{Q}}}
75: \newcommand{\vp}{{\mathbf{P}}}
76: \newcommand{\vo}{{\mathbf{0}}}
77: \newcommand{\diag}{{\text{diag}}}
78: \newcommand{\cov}{{\text{Cov}}}
79: \newcommand{\meane}{{\mathbb{E}}}
80: \newcommand{\listl}{{l=1,\;\dots,\;L}}
81: \newcommand{\mD}{\bf D}
82: \newcommand{\mB}{\bf B}
83:
84: \newtheorem{result}{\indent \em Result}
85: \newtheorem{conj}{\textbf{\textsl{Conjecture}}}
86: \newtheorem{assump}{\textbf{\textsl{Assumption}}}
87: \newtheorem{definition}{\textbf{\textsl{Definition}}}
88: \newtheorem{corollary}{\textbf{\textsl{Corollary}}}
89: \newtheorem{lemma}{\textbf{\textsl{Lemma}}}
90: \newtheorem{theorem}{\textbf{\textsl{Theorem}}}
91: \newtheorem{proposition}{\textbf{\textsl{Proposition}}}
92: \newtheorem{remark}{\textbf{\textsl{Remark}}}
93: \newtheorem{apprx}{\textbf{\textsl{Approximation}}}
94: \newtheorem{example}{\textbf{\textsl{Example}}}
95: \newtheorem{claim}{\textbf{\textsl{Claim}}}
96: \newenvironment{myproof}{\textbf{\textsl{Proof.}}}{\hfill$\bf
97: \Box$\medskip}
98: \newcommand{\df}{\stackrel{{\rm def}}{=}}
99:
100:
101:
102: \begin{document}
103:
104: \vspace*{-2cm}
105: \begin{center}
106: \em{Submitted to the IEEE Transactions on Information Theory, Oct.
107: 2005}
108: \end{center}
109:
110: \begin{center}
111: \baselineskip 1.3ex {\Large \bf Vector Gaussian Multiple Description with Individual and Central
112: Receivers\footnote{This research was sponsored in part by
113: NSF CCR-0325924 and a Vodafone US Foundation Fellowship.}} \\
114:
115: \vspace{0.15in} Hua Wang and Pramod Viswanath \footnote{The
116: authors are with the Department of Electrical and Computer
117: Engineering and the Coordinated Science Laboratory, University of
118: Illinois at Urbana-Champaign, Urbana IL~~61801; e-mail: {\tt
119: \{huawang,pramodv\}@uiuc.edu}}
120: \end{center}
121:
122: \begin{abstract}
123: $L$ multiple descriptions of a vector Gaussian source for
124: individual and central receivers are investigated. The sum rate of
125: the descriptions with covariance distortion measure constraints,
126: in a positive semidefinite ordering, is exactly characterized. For
127: two descriptions, the entire rate region is characterized. Jointly
128: Gaussian descriptions are optimal in achieving the limiting rates.
129: The key component of the solution is a novel information-theoretic
130: inequality that is used to lower bound the achievable multiple
131: description rates.
132:
133: \end{abstract}
134: \vspace{-0.2in} \baselineskip\normalbaselineskip
135: %\baselineskip 4.2ex
136: %----------Introduction--------------------------------------------
137: \qq
138: \section{Introduction} \label{sec:intro}
139: \qq
140:
141: In the multiple description problem, an information source is
142: encoded into $L$ packets and these packets are sent through
143: parallel communication channels. There are several receivers, each
144: of which can receive a subset of the packets and needs to
145: reconstruct the information source based on the received packets.
146: In the most general case, there are $2^L -1$ receivers and the
147: packets received in each receiver correspond to one of $2^L -1$
148: subsets of $\{1, \; \dots, \; L\}$. A long standing open problem
149: in the literature \cite{Ozarow80,ElGamal82,Ahlswede85,Zhang87,Zamir99,FWFu02,Venkat03,Pradhan04,HYFeng05,Puri05} is to characterize the information-theoretic
150: rate region subject to the specified distortion constraints.
151: Practical multiple description codes have been discussed
152: in
153: \cite{Vaishampayan93,Vaishampayan94,Vaishampayan98,Vaishampayan01,Goyal01,Diggavi02,Goyal02,CTian04} and
154: recent work \cite{Ishwar03,JChen05} has considered the multiple description problem in the context of the distributed source coding scenario.
155: Optimal descriptions of even the Gaussian source with quadratic
156: distortion measures have not been fully characterized. In the special case
157: of two descriptions of a scalar Gaussian source with quadratic
158: distortion measures, however, the entire rate region has been
159: characterized in \cite{Ozarow80}.
160:
161:
162: Our focus is on $L$
163: descriptions of a memoryless {\em vector} Gaussian source forwhere
164: $L$ individual and a single common receiver (cf.
165: Figure~\ref{fig:md}). Each receiver needs to reconstruct the
166: original source such that the empirical covariance matrix of the
167: difference is less than, in the sense of a positive semidefinite
168: ordering, a ``distortion'' matrix. In this setting, the
169: symmetric rate multiple description problem of a scalar Gaussian
170: source with symmetric distortion constraints has been characterized
171: in \cite{Venkat03,Pradhan04,Puri05}, but a complete understanding of
172: all other rate-distortion settings is open.
173:
174: \begin{figure}%[h]
175: \begin{center}
176: % \centerline{\epsfig{file=md.eps, scale=0.5}}
177: \scalebox{1.2}{ \input{md.pstex_t} }
178: %\input{md.pstex_t}
179: \caption{MD problem with only individual reconstructions and
180: central reconstruction} \label{fig:md}
181: \end{center}
182: \end{figure}
183:
184:
185: Our main result is an {\em exact} characterization of the sum rate for
186: any specified $L+1$ distortion matrix
187: constraints. With $L=2$, we characterize the entire
188: rate region. Our contribution is two fold:
189: \begin{itemize}
190: \item First, we derive a novel information-theoretic inequality that
191: provides a lower bound to the sum of the
192: description rates. The key step is to avoid using the entropy power
193: inequality, which was a central part of the proof of two descriptions
194: of the scalar Gaussian source in \cite{Ozarow80}: the vector entropy
195: power inequality is tight only with a certain
196: covariance alignment condition, which arbitrary distortion matrix
197: requirements do not necessarily allow.
198: %(the symmetric rate, symmetric
199: %individual distortion requirements do indeed allow as seen in
200: %\cite{Venkat03}).
201: \item Second, we show that jointly Gaussian descriptions actually
202: achieve the lower bound not by resorting to a direct calculation
203: and comparison, which appears to be difficult for $L > 2$, but
204: instead by arguing
205: the equivalence of certain optimization problems. %we give necessary
206: %and sufficient condition for the inner bound to meet the outer
207: %bound and show the existence of an optimal set of Gaussian
208: %quantizers for which the necessary and sufficient condition holds.
209: \end{itemize}
210:
211:
212:
213: Consider another two description problem of a pair of jointly Gaussian
214: memoryless sources as depicted in
215: Figure~\ref{fig:md_special}. There are two encoders that describe this
216: source to three receivers: receiver $i$ gets the description of
217: encoder $i$, with $i=1,2$ and the third receiver receives both the
218: descriptions. Suppose receiver $i$ is interested in reconstructing
219: the $i$th marginal of the jointly Gaussian source, with $i=1,2$.
220: The third receiver is interested in reconstructing the entire vector
221: source. This description problem is closely related to the vector
222: Gaussian description problem that is the main focus of this paper.
223: We exploit this connection and characterize the rate region where
224: the reconstructions have a constraint on the covariance of error
225: at each of the receivers (in the sense of a positive semidefinite
226: order).
227:
228:
229: \begin{figure}%[h]
230: \begin{center}
231: % \centerline{\epsfig{file=md.eps, scale=0.5}}
232: \scalebox{1.2}{ \input{md2.pstex_t} }
233: %\input{md.pstex_t}
234: \caption{Multiple Descriptions with separate distortion
235: constraints.} \label{fig:md_special}
236: \end{center}
237: \end{figure}
238:
239: We have organized the results in this paper as follows. In Section~\ref{sec2} we give a formal
240: description of the problem and summarize our main result. The
241: derivation of a lower bound is in Section~\ref{sec3}. In Section~\ref{sec4} we
242: provide an upper bound and provide conditions for the achievable sum
243: rate to meet the lower bound. We see in Section~\ref{sec5} that the
244: conditions are indeed satisfied in the special case of a scalar
245: Gaussian source. The solution in the case of the more complicated vector Gaussian
246: source is in Section~\ref{sec6}. The solution to the multiple
247: description problem depicted in Figure~\ref{fig:md_special} is the
248: topic of Section~\ref{sec71}. Finally,
249: while the characterization of the rate region of general multiple
250: descriptions of the Gaussian source (with each receiver having access
251: to some subset of the descriptions) is
252: still open, we can use the insights derived via our sum rate
253: characterization to solve this problem for a nontrivial set of
254: covariance distortion constraints; this is done in Section~\ref{sec72}.
255:
256:
257: A note about the notation in this paper: we use lower case
258: letters for scalars, lower case and bold face for vectors, upper
259: case and bold face for matrices. The superscript $t$ denotes
260: matrix transpose. We use $\vi$ and $\mathbf{0}$ to denote the
261: identity matrix and the
262: all zero matrix respectively, and $\diag\{p_1, \dots,
263: p_n\}$ to denote a diagonal matrix with the diagonal entries equal
264: to $p_1, \dots, p_n$. The partial order $ \succ$ ($\succcurlyeq$)
265: denotes positive definite (semidefinite) ordering: $\va \succ
266: \mB$ ($\va \succcurlyeq \mB$) means that $\va -\mB$ is a positive
267: definite (semidefinite) matrix. We write $\mathcal{N}(\mu, \vq)$
268: to denote a Gaussian random vector with mean $\mu$ and covariance
269: $\vq$. All logarithms in this paper are to the natural base.
270:
271:
272:
273:
274: \section{Problem Setting and Main Results}
275: \label{sec2}
276: %In this section we first give a formal definition of the problem
277: %of multiple description with individual and central receivers.
278: %Then we give an outer bound and a lower bound on the rate region.
279: %We focus on describing the structure of the bounds and relegate
280: %the proofs to later sections.
281:
282: \subsection{Problem Setting}
283:
284: The information source $\{\vx[m]\}$ is an i.i.d.\ random process
285: with the marginal distribution $\mathcal{N}(0, \mathbf{K}_x)$,
286: i.e., a collection of i.i.d.\ Gaussian random vectors. Denoting
287: the dimension of $\{\vx[m]\}$ by $N$, we suppose that $\vk_x$ is
288: an $N \times N$ positive definite matrix. There are $L$ encoding
289: functions at the source, encoder $l$ encodes a source sequence,
290: of length $n$, $\vx^n = (\vx[1],\;\dots , \; \vx[n])^t$ to a
291: source code $C^{(n)}_l = f_l^{(n)}(\vx^n)$, for $l=1\ldots L$.
292: This code $C^{(n)}_l$
293: is sent through $l$th communication channel at the rate $R_l =
294: \frac{1}{n}\log|C^{(n)}_l|$. There are $L$ individual receivers and
295: one central receiver.
296:
297: For $l=1, \; \dots \; L$, the $l$th individual receiver uses its
298: information (the output of the $l$th channel) to generate an
299: estimate $\hat{\vx}_l^n$
300: %\meane\left[\vx^n|f_l^{(n)}(\vx^n)\right], \quad l=1,\;\dots,\;L
301: $=g_l^{(n)}\left(f_l^{(n)}(\vx^n)\right)$ of the source sequence
302: $\vx^n$. The central receiver uses the output of all the $L$
303: channels to generate an estimate $\hat{\vx}_0^n$
304: % = g_0^{(n)}\left(f_1^{(n)}(\vx^n), \; \dots, \;
305: % f_L^{(n)}(\vx^n)\right)$
306: of the source sequence $\vx^n$. Since we are interested in
307: covariance constraints, the decoder maps can be restricted to be
308: the minimal mean square error (MMSE) estimate of the source
309: sequence based on the received codewords. So,
310: \begin{equation}
311: \begin{split}
312: \hat{\vx}_l^{n} & = \meane\left[\vx^n|f_l^{(n)}(\vx^n)\right], \quad l=1,\;\dots,\;L \\
313: \hat{\vx}_0^{n} & = \meane\left[\vx^n|f_1^{(n)}(\vx^n), \; \dots,
314: \; f_L^{(n)}(\vx^n)\right].
315: \end{split}
316: \end{equation}
317: Suppose the reconstructed sequences satisfy the covariance
318: constraints
319: \begin{equation}
320: \begin{split}
321: \frac{1}{n}\sum\limits_{m=1}^n
322: \meane\Big[(\vx[m]-\hat{\vx}_l[m])^t(\vx[m]-\hat{\vx}_l[m])\Big] &
323: \preccurlyeq \mathbf{D}_l,
324: \quad l=1,\;\dots,\;L, \\
325: \frac{1}{n}\sum\limits_{m=1}^n
326: \meane\Big[(\vx[m]-\hat{\vx}_0[m])^t(\vx[m]-\hat{\vx}_0[m])\Big] &
327: \preccurlyeq \mathbf{D}_0,
328: \end{split}
329: \end{equation}
330: then we say that multiple descriptions with distortion constraints
331: $(\vd_1,\;\dots,\;\vd_L,\;\vd_0)$ are achievable at the rate
332: tuple $(R_1, \; \dots, \; R_L)$.
333:
334: %\begin{equation*}
335: %\begin{split}
336: %g^{(n)}_l : \; & C^{(n)}_l \longrightarrow \mathcal{R}^n, \quad
337: %l=1, \; \dots, \; L \\
338: %g^{(n)}_0 : \; & C^{(n)}_1 \times \dots \times C^{(n)}_L
339: %\longrightarrow \mathcal{R}^n
340: %\end{split}
341: %\end{equation*}
342: %such that the mean square distortions between the reconstruction
343: %sequences
344: %\begin{equation*}
345: %\begin{split}
346: %\hat{\vx}_l^n = & g_l^{(n)}\left(f_l^{(n)}(\vx^n)\right), \quad l=1, \; \dots, \; L \\
347: %\hat{\vx}_0^n = & g_0^{(n)}\left(f_1^{(n)}(\vx^n), \; \dots, \;
348: %f_L^{(n)}(\vx^n)\right),
349: %\end{split}
350: %\end{equation*}
351: %and the source sequence $\vx^n$ satisfy
352:
353: The closure of the set of all achievable rate tuples is called the
354: rate region and is denoted by $\mathcal{R}_*(\vk_x, \;
355: \vd_1,\;\dots,\;\vd_L,\;\vd_0)$.
356: Throughout this paper, we suppose that $\mathbf{0} \prec \vd_0 \prec
357: \vd_l \prec \vk_x,\; \forall
358: l=1,\dots,L$.\footnote{That $\vd_0 \preccurlyeq \vd_l$, is without
359: loss of generality is seen by applying the data processing
360: inequality for mmse estimation errors; having more access to
361: information can only reduce the covariance of the error in a
362: positive semidefinite sense. Similarly, $\vk_x \preccurlyeq \vd_0$
363: is also not interesting; here we simplify this condition and take
364: $\vd_0 \prec \vk_x$.}
365: %When $\mathbf{0} \preccurlyeq \vd_0
366: %\preccurlyeq \vd_l \preccurlyeq \vk_x$ for some $l$, the solution
367: %can be obtained by adding a small perturbation to the distortions
368: %and taking limits as the small perturbation goes to zero.
369:
370:
371: \subsection{Sum Rate}\label{sec:sumrate}
372:
373: Our main result is the precise characterization of the sum rate of
374: multiple descriptions for individual and central receivers.
375: \begin{theorem}
376: For distortion constraints
377: $(\vd_1,\;\dots,\;\vd_L,\;\vd_0)$, the sum rate %for multiple
378: %description with individual and central receivers for an i.i.d
379: %$\mathcal{N}(0,\vk_x)$ Gaussian source
380: is
381: \begin{equation}\label{eq:sumrate}
382: \displaystyle %\sum\limits_{l=1}^LR_l =
383: \sup_{\vk_z \succ
384: \mathbf{0}}\quad
385: \frac{1}{2}\log\left (\frac{|\vk_x||\vk_x+\mathbf{K}_z|^{(L-1)}|\mathbf{D}_0+\mathbf{K}_z|}
386: {|\mathbf{D}_0|\prod\limits_{l=1}^L|\mathbf{D}_l+\mathbf{K}_z|}\right ).
387: \end{equation}
388: \end{theorem}
389: This sum rate is achieved by a {\em jointly Gaussian random
390: multiple description scheme}: let $\vw_1,\; \cdots, \; \vw_L$ be
391: zero mean jointly Gaussian random vectors independent of $\vx$,
392: with the positive definite covariance matricex $(\vw_1,\;
393: \cdots, \; \vw_L)$ denoted by $\vk_w$. Defining
394: \[
395: \vu_l = \vx + \vw_l, \quad l=1, \; \dots, \; L,
396: \]
397: we consider $\mathbf{K}_w$ such that
398: \begin{equation}\label{eq:covariance}
399: \begin{split}
400: \cov[\vx|\vu_l] \overset{\text{def}}{=} &
401: \meane\Big[(\vx-\meane[\vx|\vu_l])^t(\vx-\meane[\vx|\vu_l])\Big]
402: \preccurlyeq \mathbf{D}_l,
403: \quad l=1,\;\dots,\;L, \\
404: \cov[\vx|\vu_1,\;\dots,\;\vu_L] \overset{\text{def}}{=} &
405: \meane\Big[(\vx-\meane[\vx|\vu_1,\;\dots,\;\vu_L])^t(\vx-\meane[\vx|\vu_1,\;\dots,\;\vu_L])\Big]
406: \preccurlyeq \mathbf{D}_0.
407: \end{split}
408: \end{equation}
409: To construct the code book for the $l$th description, draw
410: $e^{nR_l}$ $\vu^n_l$ vectors randomly according to the marginal of
411: $\vu_l$. The encoders observe the source sequence $\vx^n$,
412: look for codewords $(\vu^n_1, \; \dots, \; \vu^n_L)$ that are
413: jointly typical with $\vx^n$ and send the index of the resulting $\vu^n_l$
414: through the $l$th channel, respectively. The $l$th
415: individual
416: receiver uses this index and generates a reproduction sequence
417: $\meane[\vx^n|\vu^n_l]$ for $l=1\ldots L$, the central receiver uses all the
418: $L$ indices to generate a reproduction sequence
419: $\meane[\vx^n|\vu^n_1,\;\dots,\;\vu^n_L]$. For every $\vk_w$
420: satisfying \eqref{eq:covariance}, the rate tuple
421: $(R_1,\;\dots,\;R_L)$ satisfying
422: \begin{equation}\label{eq:innerboundinu}
423: \sum\limits_{l \in S}R_l \ge \sum\limits_{l \in S}h(\vu_l) -
424: h(\vu_l, l \in S|\vx) = \frac{1}{2}\log\frac{\prod\limits_{l \in
425: S}|\vk_x+\vk_{w_l}|}{|\vk_{w_S}|}, \quad \forall S \subseteq \{1,
426: \; \dots, \; L\}
427: \end{equation}
428: is achievable by using this coding scheme, where $\vk_{w_S}$ is
429: the covariance matrix for all $\vw_l, l \in S$, and $\vk_{w_l} =
430: \meane[\vw_l^t\vw_l]$. In particular, the achievable sum rate is
431: \begin{equation}\label{eq:achievablesumrate}
432: \frac{1}{2}\log\frac{\prod\limits_{l=1
433: }^L|\vk_x+\vk_{w_l}|}{|\vk_{w}|}.
434: \end{equation}
435:
436: %{\tt What property should $R_l$ satisfy? Either you should explain
437: %the scheme entirely or give a very quick summary without explaining
438: %the random drawings and so forth.}
439:
440: We denote this ensemble of descriptions, throughout this paper, as
441: the jointly Gaussian description scheme and the time sharing
442: between them as the jointly Gaussian description strategy. We show
443: that jointly Gaussian description schemes are optimal in achieving
444: the sum rate \eqref{eq:sumrate}.
445:
446: %It is shown in \cite{Venkat03} that the achievable rate region
447: %based on this coding scheme is
448: %\begin{equation}\label{eq:innerboundinu}
449: %\sum\limits_{l \in S}R_l \ge \sum\limits_{l \in S}h(\vu_l) -
450: %h(\vu_l, l \in S) = \frac{1}{2}\log\frac{\prod\limits_{l \in
451: %S}|\vk_x+\vk_{w_l}|}{|\vk_{w_S}|}, \quad \forall S \subseteq \{1,
452: %\; \dots, \; L\},
453: %\end{equation}
454: %where $\vk_{w_S}$ is the covariance matrix for all $\vk_{w_l}, l
455: %\in S$. In particular, the achievable sum rate by using this
456: %Gaussian strategy is
457: %\begin{equation}\label{eq:sumrateinnerboundinu}
458: %\sum\limits_{l=1}^LR_l \ge
459: %\frac{1}{2}\log\frac{\prod\limits_{l=1}^L|\vk_x+\vk_{w_l}|}{|\vk_w|}.
460: %\end{equation}
461:
462: %We characterize the relation between covariance matrix $\vk_w$
463: %that achieves the sum rate {\tt what do you mean? the maximum
464: %achievable sum rate? This paragraph needs a rewrite -- one idea is
465: %not to have this much detail.} and the optimizing $\vk_z$ for
466: %\eqref{eq:sumrate}. In particular, we show that to achieve the
467: %optimal sum rate, $(\vw_1,\; \cdots, \; \vw_L)$ needs to satisfy
468: %$\meane[\vw_i^t\vw_j] = \meane[\vw_j^t\vw_i] \preccurlyeq
469: %\mathbf{0}$ for $i \neq j$, and in some interesting cases, instead
470: %of optimizing over all $\vk_z \succ \mathbf{0}$, we can get the
471: %optimizing $\vk_z$ for \eqref{eq:sumrate} by solving certain
472: %matrix equations.
473:
474: \subsection{Rate Region for Two Description Problem}
475:
476: \begin{figure}[h]
477: \begin{center}
478: \scalebox{0.6}{ \input{region.pstex_t} }
479: \caption{Rate region for two description problem}
480: \label{fig:region}
481: \end{center}
482: \end{figure}
483:
484: For two descriptions, we can characterize the entire
485: rate region.
486: \begin{theorem}
487: Given distortion constraints $(\vd_1,\;\vd_2,\;\vd_0)$, the rate
488: region for the two description problem for an i.i.d.\
489: $\mathcal{N}(0,\vk_x)$ vector Gaussian source is
490: \begin{equation}\label{eq:innerboundl2}
491: \mathcal{R}_*(\vk_x,\;\vd_1,\;\vd_2,\;\vd_0) = \left\{
492: \begin{array}{l}
493: (R_1,\;R_2): \\
494: \displaystyle R_l \ge \frac{1}{2}\log\frac{|\vk_x|}{|\vd_l|}, \quad l=1,\;2 \\
495: \displaystyle R_1+R_2 \ge \sup_{\vk_z \succ \mathbf{0}}
496: \frac{1}{2}\log\frac{|\vk_x||\vk_x+\mathbf{K}_z||\mathbf{D}_0+\mathbf{K}_z|}
497: {|\mathbf{D}_0||\mathbf{D}_1+\mathbf{K}_z||\vd_2+\vk_z|}
498: \end{array}
499: \right\}.
500: \end{equation}
501: \end{theorem}
502: We show that if the distortion constraints
503: $(\vd_1,\;\vd_2,\;\vd_0)$ satisfy $ \vd_0 + \vk_x - \vd_1 - \vd_2
504: \succ \mathbf{0}$ and $\vd_0^{-1} + \vk_x^{-1} -
505: \vd_1^{-1}-\vd_2^{-1} \succ \mathbf{0}$, we can get the optimizing
506: $\vk_z$ by solving a matrix Riccati equation. An illustration of
507: the rate region is shown in Figure~\ref{fig:region}. In this case,
508: if we let $\vk_{w_l} = [\vd_l^{-1}-\vk_x^{-1}]^{-1}$ for
509: $l=0,1,2$, then the optimizing $\vk_z$ is
510: \[
511: \vk_z = \vk_x(\vk_x-\va^*)^{-1}\vk_x-\vk_x,
512: \]
513: where
514: \[
515: \va^* =
516: (\vk_{w_1}-\vk_{w_0})^{\frac{1}{2}}\left[(\vk_{w_1}-\vk_{w_0})^{-\frac{1}{2}}
517: (\vk_{w_2}-\vk_{w_0})(\vk_{w_1}-\vk_{w_0})^{-\frac{1}{2}}\right]^{\frac{1}{2}}(\vk_{w_1}-\vk_{w_0})^{\frac{1}{2}}
518: -\vk_{w_0}.
519: \]
520: Letting $R_{sum}$ denote the optimal sum rate, the two corner
521: points in Figure~\ref{fig:region} are
522: $$
523: B_1 =
524: \left(\frac{1}{2}\log\frac{|\vk_x|}{|\vd_1|},
525: R_{sum}-\frac{1}{2}\log\frac{|\vk_x|}{|\vd_1|}\right), \quad \mbox{and}$$
526: $$
527: B_2 = \left(R_{sum}-\frac{1}{2}\log\frac{|\vk_x|}{|\vd_2|},\frac{1}{2}\log\frac{|\vk_x|}{|\vd_2|}\right).$$
528:
529: %{\tt Please give the exact sum rate in this case and mark the
530: %important points on the axes of the figure by giving exact
531: %values.}
532:
533: \section{Lower Bound}\label{sec:outerbound}
534: \label{sec3}
535: By fairly procedural steps, we have the following lower bound to the
536: sum rate of the multiple descriptions:
537: \begin{equation}
538: \begin{split}
539: n\sum\limits_{l=1}^LR_l \ge & \sum\limits_{l=1}^LH(C_l) =
540: \sum\limits_{l=1}^LH(C_l)-H(C_1,\;\dots,\;C_L|\vx^n) \\
541: = & \sum\limits_{l=1}^LH(C_l)-H(C_1,\cdots,C_L) +
542: H(C_1,\;\dots,\;C_L) - H(C_1,\;\dots,\;C_L|\vx^n) \\
543: = & I(C_1;C_2;\dots;C_L) +
544: I(C_1,\;\dots,\;C_L; \vx^n),
545: \end{split}
546: \end{equation}
547: where we have defined
548: \[
549: I(C_1;C_2;\dots;C_L) \df
550: \sum\limits_{l=1}^LH(C_l)-H(C_1,\;\dots,\;C_L) =
551: \sum\limits_{l=2}^LI(C_l;C_1\dots C_{l-1}),
552: \]
553: and called it the symmetric mutual information between
554: $C_1,\;\dots,\;C_L$. Note that $I(C_1;C_2;\dots;C_L) \geq 0$ and is also well
555: defined even when $C_1,\;\dots,\;C_L$ are continuous random variables. Our
556: main result is the
557: following information theoretic inequality which gives a lower
558: bound to the sum of symmetric mutual information between
559: $(C_1,C_2,\dots,C_L)$ and mutual information between
560: $C_1,C_2,\dots,C_L$ and $\vx^n$ for given covariance constraints.
561: \begin{lemma}\label{lemma:inform}
562: Let $\vx^n = (\vx[1],\;\dots,\;\vx[n])$, where $\vx[m]$'s are
563: i.i.d.\ $\mathcal{N}(\mathbf{0},\vk_x)$ Gaussian random vectors for
564: $m=1,\; \dots,\; n$. Let $C_1,\;\dots,\;C_L$ be random variables
565: jointly distributed with $\vx^n$. Let
566: $\hat{\vx}_0^n = \meane[\vx^n|C_1,\;\dots,\;C_L]$ and
567: $\hat{\vx}_l^n = \meane[\vx^n|C_l]$ for $l=1,\;\dots,\;L$. Given
568: positive definite matrices $\vd_1,\;\dots,\;\vd_L,\;\vd_0$, if
569: \begin{equation}\label{eq:lemmacov}
570: \begin{split}
571: \frac{1}{n}\sum\limits_{m=1}^n \meane[(\vx[m] -
572: \hat{\vx}_l[m])^t(\vx[m] - \hat{\vx}_l[m])] & \preccurlyeq
573: \mathbf{D}_l,
574: \quad l=1,\;\dots,\;L, \\
575: \frac{1}{n}\sum\limits_{m=1}^n \meane[(\vx[m] -
576: \hat{\vx}_0[m])^t(\vx[m] - \hat{\vx}_0[m])] & \preccurlyeq
577: \mathbf{D}_0,
578: \end{split}
579: \end{equation}
580: then
581: \begin{equation}\label{eq:inequality}
582: I(C_1;C_2;\dots;C_L)+ I(C_1,\;\dots,\;C_L;
583: \vx^n) \ge \sup_{\vk_z \succ \mathbf{0}}
584: \frac{n}{2}\log\frac{|\vk_x||\vk_x+\mathbf{K}_z|^{(L-1)}|\mathbf{D}_0+\mathbf{K}_z|}
585: {|\mathbf{D}_0|\prod\limits_{l=1}^L|\mathbf{D}_l+\mathbf{K}_z|}.
586: \end{equation}
587: Furthermore, there exists a jointly Gaussian distribution of
588: $(C_1,\dots,C_L,\vx^n)$ such that the inequality in
589: \eqref{eq:inequality} is tight.
590: %if the inequalities in
591: %\eqref{eq:lemmacov} are tight and $(C_1,\dots,C_L,\vx^n)$ are c.
592: \end{lemma}
593: %\begin{proof}
594: %See Appendix \ref{app:inform}.
595: %\end{proof}
596:
597: This is a fundamental information-theoretic inequality which
598: involves only the joint distribution\footnote{This inequality holds
599: even when $C_1,C_2,\dots,C_L$ are not simply functions of $\vx^n$
600: and can also be continuous random variables.} between
601: $C_1,C_2,\dots,C_L$ and $\vx^n$ and bounds on mean square error
602: estimation of $\vx^n$ from $C_1,C_2,\dots,C_L$; we delegate the
603: proof of this result to Appendix~\ref{app:inform}. We can now use
604: Lemma \ref{lemma:inform} to derive a lower bound to the sum rate
605: \begin{equation}\label{eq:outer}
606: \sum\limits_{l=1}^LR_l \ge \sup_{\vk_z \succ \mathbf{0}}
607: \frac{1}{2}\log\frac{|\vk_x||\vk_x+\mathbf{K}_z|^{(L-1)}|\mathbf{D}_0+\mathbf{K}_z|}
608: {|\mathbf{D}_0|\prod\limits_{l=1}^L|\mathbf{D}_l+\mathbf{K}_z|}.
609: \end{equation}
610:
611: By letting $L=1$ in the lemma above, we can derive a simple lower
612: bound to the rate of the individual descriptions as well:
613: \begin{equation}\label{eq:individualbound}
614: \begin{split}
615: R_l & \ge \frac{1}{n}H(C_l) = \frac{1}{n}\big(H(C_l)-H(C_l|\vx^n)\big) \\
616: & = \frac{1}{n} I(\vx^n;C_l) \\
617: & \ge \frac{1}{2}\log\frac{|\vk_x|}{|\mathbf{D}_l|}, \quad
618: \listl.
619: \end{split}
620: \end{equation}
621: This bound is actually the point-to-point rate-distortion function
622: for individual receivers, since each individual receiver only
623: faces a point-to-point compression problem.
624:
625: %Thus we have the following outer bound on the rate region for a
626: %give distortion constraint $(\vd_1,\;\dots,\;\vd_L,\;\vd_0)$.
627: %\begin{equation}\label{eq:outerbound}
628: %\mathcal{R}_{out}(\vk_x, \; \vd_1,\;\dots,\;\vd_L,\;\vd_0) =
629: %\left\{
630: %\begin{array}{l}
631: %(R_1,\;\dots,\;R_L): \\
632: %\displaystyle R_l \ge \frac{1}{2}\log\frac{|\vk_x|}{|\vd_l|}, \quad l=1,\;\dots,\;L \\
633: %\displaystyle \sum\limits_{l=1}^LR_l \ge \sup_{\vk_z \succ
634: %\mathbf{0}}
635: %\frac{1}{2}\log\frac{|\vk_x||\vk_x+\mathbf{K}_z|^{(L-1)}|\mathbf{D}_0+\mathbf{K}_z|}
636: %{|\mathbf{D}_0|\prod\limits_{l=1}^L|\mathbf{D}_l+\mathbf{K}_z|}
637: %\end{array}
638: %\right\}.
639: %\end{equation}
640: Note that for any positive definite $\vk_z$,
641: \[
642: \frac{1}{2}\log\frac{|\vk_x||\vk_x+\mathbf{K}_z|^{(L-1)}|\mathbf{D}_0+\mathbf{K}_z|}
643: {|\mathbf{D}_0|\prod\limits_{l=1}^L|\mathbf{D}_l+\mathbf{K}_z|}
644: \]
645: is a lower bound to the sum rate of the multiple descriptions. Two
646: special choices of $\vk_z$ are of particular interest:
647: \begin{itemize}
648: \item Letting $\vk_z = \epsilon \vi$ and 0 $\epsilon \rightarrow
649: 0^+$, we have the following lower bound:
650: \begin{equation}\label{eq:kz0}
651: \displaystyle \sum\limits_{l=1}^LR_l \ge
652: \frac{1}{2}\log\frac{|\vk_x|^L}{|\vd_1|\dots|\vd_L|}.
653: \end{equation}
654: This bound is actually the summation of the bounds on the individual
655: rates.
656:
657: \item Letting some eigenvalues of $\mathbf{K}_z$ goes to infinity, we
658: have the following lower bound:
659: \begin{equation}\label{eq:kzinf}
660: \displaystyle \sum\limits_{l=1}^LR_l \ge
661: \frac{1}{2}\log\frac{|\vk_x|} {|\mathbf{D}_0|}.
662: \end{equation}
663: This bound is the point-to-point rate-distortion function when we
664: only have the central distortion constraint.
665: \end{itemize}
666: We will see later that for some distortion constraints
667: $(\vd_1,\;\dots,\;\vd_L,\;\vd_0)$, \eqref{eq:kz0} and
668: \eqref{eq:kzinf} can be tight.
669: %{\tt What is the point of these two remarks? That is, how do they
670: % connect to the previous discussion and where will they be picked up
671: % later in the paper?}
672: % --------------------------------------------------
673: \section{Upper Bound}\label{sec:innerbound}
674: \label{sec4}
675:
676: In the previous section we gave a lower bound to the sum rate. Now
677: we give a upper bound to the sum rate by using the jointly
678: Gaussian description scheme described in Section
679: \ref{sec:sumrate}.
680:
681: \subsection{Jointly Gaussian Multiple Description Scheme}
682:
683: First we give a sketch of the achievable rate region by using
684: jointly Gaussian description scheme. Given the source sequence
685: $\vx^n$, as long as we can find a combination of codewords
686: $(\vu^n_1, \; \dots, \; \vu^n_L)$ that are jointly typical with
687: $\vx^n$, all the receivers can generate reproduction sequences that
688: satisfy their given distortion constraints. An intuitive way to
689: understand \eqref{eq:innerboundinu} is the following: since
690: $(\vu^n_1, \; \dots, \; \vu^n_L)$ are jointly typical with
691: $\vx^n$, then for any $S \subseteq \{1, \; \dots, \; L\}$, we have
692: that $\vu^n_l, l \in S$ are jointly typical with $\vx^n$. Now the
693: probability that a randomly generated combination of codewords
694: $\vu^n_l, l \in S$ are jointly typical with $\vx^n$ is roughly
695: \[
696: \frac{e^{nh(\vu_l, l \in S|
697: \vx)}}{\prod\limits_{l \in S}e^{nh(\vu_l)}},
698: \]
699: and the number of possible combination of codewords $\vu^n_l, l
700: \in S$ are $\prod\limits_{l \in S}e^{nR_l}$. Thus, as long as
701: \begin{equation}\label{eq:raterandom}
702: \sum\limits_{l \in S}R_l \ge \sum\limits_{l \in S}h(\vu_l) -
703: h(\vu_l, l \in S|\vx),
704: \end{equation}
705: we can find a combination of codewords $\vu^n_l, l \in S$ that are
706: jointly typical with $\vx^n$. Rigorously speaking, we need to show that as
707: long as \eqref{eq:raterandom} is satisfied, then for any given
708: source sequence $\vx^n$ we can find a combination of codewords
709: $(\vu^n_1, \; \dots, \; \vu^n_L)$ such that $\vu^n_l, l \in S$ are
710: jointly typical with $\vx^n$ for all $S \subseteq \{1, \; \dots,
711: \; L\}$. The {\it second moment method}\cite{Alon00} is commonly
712: used to address this aspect, and a proof can be found in
713: \cite{Venkat03}.
714:
715: Evaluating \eqref{eq:raterandom} based on the jointly Gaussian
716: distribution of $\vx$ and $\vu_1,\;\dots,\;\vu_L$, we get that all
717: the rate tuples $(R_1,\;\dots,\;R_L)$ satisfying
718: \begin{equation}
719: \sum\limits_{l \in S}R_l \ge \sum\limits_{l \in S}h(\vu_l) -
720: h(\vu_l, l \in S|\vx) = \frac{1}{2}\log\frac{\prod\limits_{l \in
721: S}|\vk_x+\vk_{w_l}|}{|\vk_{w_S}|}, \quad \forall S \subseteq \{1,
722: \; \dots, \; L\}
723: \end{equation}
724: are achievable by the jointly Gaussian description scheme. In
725: particular, we have that the achievable sum rate is
726: \begin{equation}\label{eq:achievablerate}
727: \sum\limits_{l=1}^Lh(\vu_l)-h(\vu_1,\;\dots,\;\vu_L|\vx) =
728: \frac{1}{2}\log\frac{\prod\limits_{l=1}^L|\vk_x+\vk_{w_l}|}{|\vk_w|}.
729: \end{equation}
730: The resulting distortions $(\vd^*_1,\;\dots,\;\vd^*_L,\;\vd^*_0)$
731: by using jointly Gaussian description scheme can be calculated as
732: \begin{equation}\label{eq:distortion2}
733: \begin{split}
734: \vd^*_l = & \cov[\vx|\vu_l] = [\vk_x^{-1} + \mathbf{K}_{w_l}^{-1}]^{-1}, \quad l=1,\;\dots,\; L , \\
735: \vd^*_0 = & \cov[\vx|\vu_1,\;\dots,\;\vu_L] = [\vk_x^{-1} +
736: (\vi,\; \dots, \; \vi)\mathbf{K}_w^{-1}(\vi,\; \dots, \;
737: \vi)^t]^{-1}.
738: \end{split}
739: \end{equation}
740:
741:
742: \subsection{Combinatorial Property of the Achievable Region}
743:
744: The achievable region given in \eqref{eq:raterandom} has useful
745: combinatorial properties; in particular it belongs to the class of {\it
746: contra-polymatroids}\cite{Welsh}. Certain rate regions of the multiple
747: access channel \cite{Tse98} and distributed source coding problems
748: \cite{Viswanath04} are also known to have this specific combinatorial
749: property. To see this, let
750: \[
751: \phi(S) \overset{\text{def}}{=} \sum\limits_{l \in S}h(\vu_l) -
752: h(\vu_l, l \in S|\vx), \quad S \subseteq \{1, \; \dots, \; L\}.
753: \]
754: We can readily verify that
755: \begin{equation}
756: \begin{split}
757: \phi(S\cup\{t\}) & \ge \phi(S), \quad \forall t \in \{1, \; \dots,
758: \; L\}, \\
759: \phi(S\cup T)+\phi(S\cap T) & \ge \phi(S) + \phi(T).
760: \end{split}
761: \end{equation}
762: By definition, we conclude that the achievable rate region of a
763: jointly Gaussian multiple description scheme is a contra-polymatroid.
764: The key advantage of this combinatorial propety is that we can exactly
765: characterize the vertices of the achievable rate region
766: \eqref{eq:raterandom}. Letting $\pi$ to be a
767: permutation on $\{1, \; \dots, \; L\}$, define
768: \[
769: b_i^{(\pi)} \overset{\text{def}}{=}
770: \phi(\{\pi_1,\pi_2,\dots,\pi_i\})-\phi(\{\pi_1,\pi_2,\dots,\pi_{i-1}\}),
771: \quad i=1, \; \dots, \; L,
772: \]
773: and $\mathbf{b}^{(\pi)} = \left(b_1^{(\pi)}, \dots,
774: b_L^{(\pi)}\right)$. Then the $L!$ points $\{\mathbf{b}^{(\pi)},
775: \pi \text{ a permutation}\}$ are the vertices of the
776: contra-polymatroid \eqref{eq:raterandom}.
777:
778:
779:
780:
781: %We can take the union of achievable rate region over all $\vk_w$
782: %satisfying \eqref{eq:covariance} and then taking the convex hull
783: %to get a large achievable region.
784:
785:
786: %Evaluating \eqref{eq:covariance} using standard Gaussian MMSE
787: %results, we have
788: %\begin{equation}\label{eq:distortion2}
789: %\begin{split}
790: %& \cov[\vx|\vu_l] = [\vk_x^{-1} + \mathbf{K}_{w_l}^{-1}]^{-1}, \quad l=1,\;\dots,\; L \\
791: %& \cov[\vx|\vu_1,\;\dots,\;\vu_L] = [\vk_x^{-1} + (\vi,\; \dots,
792: %\; \vi)\mathbf{K}_w^{-1}(\vi,\; \dots, \; \vi)^t]^{-1}.
793: %\end{split}
794: %\end{equation}
795: \subsection{Comparison of Upper Bound and the Lower Bound}
796:
797: Our goal is to show that the jointly Gaussian description scheme
798: achieves the lower bound to the sum rate. In general it does not
799: seem facile to do a direct calculation and comparison. We forgo
800: this strategy and, instead, provide an alternative
801: characterization of the achievable sum rate which is much easier
802: to compare with the lower
803: bound. %, and give necessary and sufficient conditions for the upper
804: %bound to match the lower bound.
805:
806: Similar to the derivation of the lower bound (in Appendix~\ref{app:inform}),
807: we consider an
808: $\mathcal{N}(0, \mathbf{K}_z)$ Gaussian random vector $\vz$,
809: independent of $\vx$ and all $\vw_l$'s. Defining $\vy=\vx+\vz$,
810: we have the following achievable sum rate:
811: %{\tt I made the inequality in the first step an equality so that you
812: % can make the logical conclusion.}
813: \begin{eqnarray}\label{eq:suminnerbound25}
814: \sum\limits_{l=1}^LR_l & = &
815: \sum\limits_{l=1}^Lh(\vu_l)-h(\vu_1,\;\dots,\;\vu_L|\vx) \nonumber \\
816: & = & \sum\limits_{l=1}^Lh(\vu_l)-h(\vu_1,\;\dots,\;\vu_L) +
817: h(\vu_1,\;\dots,\;\vu_L) - h(\vu_1,\;\dots,\;\vu_L|\vx) \nonumber \\
818: & = & \sum\limits_{l=1}^Lh(\vu_l)-h(\vu_1,\cdots,\vu_L) +
819: I(\vu_1,\;\dots,\;\vu_L; \vx) \nonumber \\
820: & \overset{(a)}{\ge} &
821: \sum\limits_{l=1}^Lh(\vu_l)-h(\vu_1,\cdots,\vu_L) +
822: I(\vu_1,\;\dots,\;\vu_L; \vx) -
823: \left(\sum\limits_{l=1}^Lh(\vu_l|\vy)-h(\vu_1,\;\dots,\;\vu_L|\vy)\right) \nonumber \\
824: & = & \sum\limits_{l=1}^L\big(h(\vy)-h(\vy|\vu_l)\big)-h(\vy)
825: +h(\vy|\vu_1,\;\dots,\;\vu_L) + h(\vx)-h(\vx|\vu_1,\;\dots,\;\vu_L) \nonumber \\
826: & = & h(\vx)+(L-1)h(\vy)-\sum\limits_{l=1}^Lh(\vy|\vu_l)+
827: h(\vy|\vu_1,\;\dots,\;\vu_L)-h(\vx|\vu_1,\;\dots,\;\vu_L)
828: \nonumber \\
829: & = &
830: \frac{1}{2}\log\frac{\Big|\vk_x\Big|\Big|\vk_x+\mathbf{K}_z\Big|^{(L-1)}\Big|\cov[\vx|\vu_1,\;\dots,\;\vu_L]+\mathbf{K}_z\Big|}
831: {\Big|\cov[\vx|\vu_1,\;\dots,\;\vu_L]\Big|\prod\limits_{l=1}^L\Big|\cov[\vx|\vu_l]+\mathbf{K}_z\Big|},
832: \end{eqnarray}
833: where the last step is from a procedural Gaussian MMSE calculation.
834:
835: Note that if we have
836: \begin{equation}\label{eq:independent}
837: \sum\limits_{l=1}^Lh(\vu_l|\vy)-h(\vu_1,\;\dots,\;\vu_L|\vy) = 0,
838: \end{equation}
839: then (a) in \eqref{eq:suminnerbound25} is actually an equality.
840: Thus, if our choice of $\mathbf{K}_w$ and $\mathbf{K}_z$ satisfy
841: the following two conditions:
842: \begin{itemize}
843: \item \eqref{eq:independent} is true.
844:
845: \item distortion constraints are met with equality, i.e.,
846: \begin{equation}\label{eq:distortion}
847: \begin{split}
848: & \cov[\vx|\vu_l] = \mathbf{D}_l, \quad l=1,\;\dots,\; L, \\
849: & \cov[\vx|\vu_1,\;\dots,\;\vu_L] = \mathbf{D}_0,
850: \end{split}
851: \end{equation}
852: \end{itemize}
853: then the upper bound matches the lower bound and we have characterized the
854: sum rate. In the following we examine under what
855: circumstances the above two conditions are true.
856:
857: First, we give a necessary and sufficient condition for
858: \eqref{eq:independent} to be true, delegating the proof to
859: Appendix~\ref{app:nec_suff_condition}.
860: \begin{proposition}
861: \label{prop:nec_suff_condition}
862: There exists some choice of positive definite $\vk_z$ such that
863: \eqref{eq:independent} is true if and only if $\vk_w$, the
864: covariance matrix of $(\vw_1,\; \cdots, \; \vw_L)$, takes the
865: following form
866: \begin{equation}\label{eq:kw}
867: \mathbf{K}_w =
868: \begin{pmatrix}
869: \mathbf{K}_{w_1} & -\mathbf{A} & -\mathbf{A} & \dots & -\mathbf{A} \\
870: -\va & \mathbf{K}_{w_2} & -\va & \dots & -\va \\
871: \hdotsfor{5} \\
872: -\va & \dots & -\va & \mathbf{K}_{w_{L-1}} & -\va \\
873: -\va & \dots & -\va & -\va & \mathbf{K}_{w_L}
874: \end{pmatrix},
875: \end{equation}
876: where $ \mathbf{0} \prec \va \prec \vk_x$.
877: \end{proposition}
878:
879:
880: Next, we look at the conditions for \eqref{eq:distortion} to be
881: true. From \eqref{eq:distortion2}, we have
882: \begin{equation}\label{eq:distortion1}
883: \begin{split}
884: \mathbf{D}_l^{-1} & = \cov[\vx|\vu_l]^{-1} = \vk_x^{-1} + \mathbf{K}_{w_l}^{-1}, \quad l=1,\;\dots,\; L \\
885: \mathbf{D}_0^{-1} & = \cov[\vx|\vu_1,\;\dots,\;\vu_L]^{-1} =
886: \vk_x^{-1} + (\vi,\; \dots, \; \vi)\mathbf{K}_w^{-1}(\vi,\; \dots,
887: \; \vi)^t.
888: \end{split}
889: \end{equation}
890: $(\vi,\; \vi, \; \dots, \;
891: \vi)\mathbf{K}_w^{-1}(\vi,\; \vi, \; \dots, \; \vi)^t$, is calculated in the following lemma; the proof is available in Appendix~\ref{app:inverse}.
892: \begin{lemma}\label{lemma:inverse}
893: Let
894: \[
895: \vk_w = \begin{pmatrix}
896: \vk_{w_1} & -\va & -\va & \dots & -\va \\
897: -\va & \vk_{w_2} & -\va & \dots & -\va \\
898: \hdotsfor{5} \\
899: -\va & \dots & -\va & \vk_{w_{L-1}} & -\va \\
900: -\va & \dots & -\va & -\va & \vk_{w_L}.
901: \end{pmatrix}.
902: \]
903: If $\vk_w \succ \mathbf{0}$ and $\va \succeq \mathbf{0}$, then
904: \[
905: (\vi,\; \vi, \; \dots, \; \vi)\mathbf{K}_w^{-1}(\vi,\; \vi, \;
906: \dots, \; \vi)^t = \left[\left(\sum\limits_{l=1}^L
907: (\vk_{w_l}+\va)^{-1}\right)^{-1}-\va\right]^{-1}.
908: \]
909: \end{lemma}
910: Using this lemma, from \eqref{eq:distortion1} we arrive at
911: \begin{equation}
912: \left[(\vd_0^{-1}-\vk_x^{-1})^{-1}+\va\right]^{-1} =
913: \sum\limits_{l=1}^L\left[(\vd_l^{-1}-\vk_x^{-1})^{-1}+\va\right]^{-1}.
914: \end{equation}
915: Defining
916: \begin{equation}
917: \vk_{w_0} = (\vd_0^{-1}-\vk_x^{-1})^{-1},
918: \end{equation}
919: \eqref{eq:distortion1} is equivalent to
920: \begin{equation}\label{eq:core}
921: \left[\vk_{w_0}+\va\right]^{-1} =
922: \sum\limits_{l=1}^L\left[\vk_{w_l}+\va\right]^{-1}.
923: \end{equation}
924: Thus, if there exists a positive definite solution $\va$ to
925: \eqref{eq:core}, and the corresponding $\vk_w$ is positive definite,
926: then the distortion constraints are met with equality, i.e.,
927: \eqref{eq:distortion} holds. It turns out that as long as $\va$ is a solution to
928: \eqref{eq:core}, the resulting $\vk_w$ is always positive definite; we state this formally below, delegating the proof to Appendix \ref{app:positivedefinite}.
929: \begin{lemma}\label{lemma:positivedefinite}
930: If for some $\vk_{w_0} \succ \mathbf{0}$ and $\va \succ
931: \mathbf{0}$ \eqref{eq:core} is true, then the covariance matrix
932: $\vk_w$ defined in \eqref{eq:kw} is positive definite.
933: \end{lemma}
934: We summarize the state of affairs in the following theorem.
935: \begin{theorem}\label{th:main}
936: Given distortion constraints $(\vd_1, \; \dots, \; \vd_L, \vd_0)$,
937: let
938: \begin{equation}
939: \vk_{w_l} = (\vd_l^{-1}-\vk_x^{-1})^{-1}, \quad l=0, \; 1, \;
940: \dots, \; L.
941: \end{equation}
942: If there exists an solution $\va^*$ to \eqref{eq:core} and
943: $\mathbf{0} \prec \va^* \prec \vk_x$, then the jointly Gaussian
944: description scheme with $\vk_w$ defined in \eqref{eq:kw} with $\va
945: = \va^*$ achieves the optimal sum rate, and the optimal $\vk_z$
946: for lower bound \eqref{eq:outer} is $\mathbf{K}_z =
947: \vk_x(\vk_x-\va^*)^{-1}\vk_x-\vk_x$.
948: \end{theorem}
949:
950: Thus we show that if the given distortion constraints $(\vd_1, \;
951: \dots, \; \vd_L, \vd_0)$ satisfy the condition for Theorem
952: \ref{th:main}, then the jointly Gaussian description scheme
953: achieves the optimal sum rate and we can calculate the optimal
954: $\vk_w$ by solving a matrix equation. However, for arbitrarily
955: given distortion constraints, \eqref{eq:core} may not have a
956: solution $\va^*$ such that $\mathbf{0} \prec \va^* \prec \vk_x$.
957: In this case, we can show that there exists a jointly Gaussian
958: description scheme that achieves the sum rate lower bound, and
959: resulting in distortions $(\vd^*_1, \; \dots, \; \vd^*_L,
960: \vd^*_0)$ such that $\vd^*_l \preccurlyeq \vd_l$ for
961: $l=0,1,\dots,L$. In the following we first study the relatively
962: simpler case of scalar Gaussian source, and then move to discuss
963: the vector Gaussian source.
964:
965: \section{Scalar Gaussian Source}\label{section:scalar}
966: \label{sec5}
967: Here we suppose that the information source is an i.i.d.\ sequence
968: of $\mathcal{N}(0,\sigma_x^2)$ scalar Gaussian random
969: variables. Let individual distortion constraints be
970: $(d_1,\;\dots,\;d_L)$ and the central distortion constraints be
971: $d_0$, where $0 < d_0 < d_l < \sigma_x^2$ for $l=1,\; \dots, \;
972: L$. We consider the jointly Gaussian description scheme with the
973: following covariance matrix for $w_1, \; \dots, \; w_l$.
974: \begin{equation}\label{eq:scalarkw}
975: \vk_w = \begin{pmatrix}
976: \sigma_1^2 & -a & -a & \dots & -a \\
977: -a & \sigma_2^2 & -a & \dots & -a \\
978: \hdotsfor{5} \\
979: -a & \dots & -a & \sigma_{L-1}^2 & -a \\
980: -a & \dots & -a & -a & \sigma_L^2
981: \end{pmatrix}.
982: \end{equation}
983:
984: Consider the condition for Theorem \ref{th:main} to hold: to meet
985: the individual distortion constraint with equality, we need
986: \begin{equation}\label{eq:sigmal}
987: \sigma_l^2 = (d_l^{-1}-\sigma_x^{-2})^{-1} =
988: \frac{d_l\sigma_x^2}{\sigma_x^2-d_l}, \quad l=1,\;\dots,\;L.
989: \end{equation}
990: Let
991: \begin{equation}
992: \sigma_0^2 \overset{\text{def}}{=} (d_0^{-1}-\sigma_x^{-2})^{-1} =
993: \frac{d_0\sigma_x^2}{\sigma_x^2-d_0},
994: \end{equation}
995: we need
996: \begin{equation}
997: \left[\sigma_0^2+a\right]^{-1} =
998: \sum\limits_{l=1}^L\left[\sigma_l^2+a\right]^{-1}
999: \end{equation}
1000: to have a solution $a^* \in (0, \sigma_x^2)$, to meet the central
1001: distortion constraint with equality. Towards this, define
1002: \begin{equation}
1003: f(a)\overset{\text{def}}{=} \frac{1}{\sigma_0^2+a} -
1004: \sum\limits_{l=1}^L\frac{1}{\sigma_l^2+a},
1005: \end{equation}
1006: and we have
1007: \begin{equation}
1008: \begin{split}
1009: f(0) & = \frac{1}{\sigma_0^2}-
1010: \sum\limits_{l=1}^L\frac{1}{\sigma_l^2} =
1011: \frac{1}{d_0}+\frac{L-1}{\sigma_x^2}-\sum\limits_{l=1}^L\frac{1}{d_l}, \\
1012: f(\sigma_x^2) & = \frac{1}{\sigma_0^2+\sigma_x^2}-
1013: \sum\limits_{l=1}^L\frac{1}{\sigma_l^2+\sigma_x^2} =
1014: \frac{1}{\sigma_x^4}\left(\sum\limits_{l=1}^Ld_l-d_0-(L-1)\sigma_x^2\right).
1015: \end{split}
1016: \end{equation}
1017: Using induction, we can show that
1018: \begin{equation}
1019: \left(\sum\limits_{l=1}^L\frac{1}{d_l}-\frac{L-1}{\sigma_x^2}\right)^{-1}
1020: \ge \sum\limits_{l=1}^Ld_l-(L-1)\sigma_x^2.
1021: \end{equation}
1022: Thus we have
1023: \begin{equation*}
1024: \begin{split}
1025: f(0) \le 0 & \Rightarrow f(\sigma_x^2) \le 0, \\
1026: f(\sigma_x^2) \ge 0 & \Rightarrow f(0) \ge 0.
1027: \end{split}
1028: \end{equation*}
1029: Then given distortions $(d_1, \; \dots, \; d_L, \; d_0)$, $f(0)$
1030: and $f(\sigma_x^2)$ falls into the following three cases.
1031:
1032:
1033: % {\tt I think the logic is not clear at this point. In
1034: %particular, it
1035: % is not clear why the induction step leads to these three cases. The
1036: % ``thus'' in the following sentence is not justified as written.}
1037:
1038: {\bf Case 1:} $f(0) > 0$ and $f(\sigma_x^2) <0$.
1039:
1040: In this case, since $f(a)$ is a continuous function, there exists
1041: an $a^* \in (0,\sigma_x^2)$ such that $f(a^*) = 0$. In this case
1042: the condition for Theorem \ref{th:main} holds and from Theorem
1043: \ref{th:main} we know that jointly Gaussian description scheme
1044: with covariance matrix for $w_1, \; \dots, \; w_l$ being
1045: \eqref{eq:scalarkw} with $a = a^*$ achieves the optimal sum rate.
1046:
1047: {\bf Case 2:} $f(0) \le 0$. Alternatively,
1048: $\frac{1}{d_0}+\frac{L-1}{\sigma_x^2}-\sum\limits_{l=1}^L\frac{1}{d_l}
1049: \le 0$.
1050:
1051: In this case, the condition for Theorem \ref{th:main} does not
1052: hold. But the jointly Gaussian description scheme can still achieve
1053: the sum rate. To see this, choosing $a=0$ in $\mathbf{K}_w$ we
1054: can meet individual distortions with equality and get a central
1055: distortion $d_0'$. From \eqref{eq:distortion1} we have
1056: \begin{equation}
1057: \begin{split}
1058: \frac{1}{d_0'} & = \frac{1}{\sigma_x^2}+ (1 \; 1 \; \dots \;
1059: 1)K_w^{-1} (1 \; 1 \; \dots
1060: \; 1)^t \\
1061: & = \frac{1}{\sigma_x^2}+\sum\limits_{l=1}^L\frac{1}{\sigma_l^2} = \sum\limits_{l=1}^L\frac{1}{d_l} - \frac{L-1}{\sigma_x^2} \\
1062: & \geq \frac{1}{d_0}.
1063: \end{split}
1064: \end{equation}
1065: Hence we have achieved distortion $(d_1,\dots,d_L,d'_0)$ where $d_0'
1066: \le d_0$, and from \eqref{eq:achievablerate} the achievable sum
1067: rate is
1068: \begin{equation}
1069: \sum\limits_{l=1}^LR_l \ge
1070: \frac{1}{2}\log\frac{\sigma_x^{2L}}{d_1d_2\cdots d_{L}},
1071: \end{equation}
1072: which equals the sum of our bounds on individual rates.
1073:
1074: {\bf Case 3:} $f(\sigma_x^2) \ge 0$, Alternatively, $
1075: \sum\limits_{l=1}^Ld_l - d_0 - (L-1)\sigma_x^2 \geq 0 $.
1076:
1077: In this case, the conditions for Theorem \ref{th:main} do not
1078: hold as well. But the jointly Gaussian description strategy still
1079: achieves the sum rate. To see this,
1080: note that we can find a $d'_L$ such that $0 < d'_L \le d_L$ and
1081: \begin{equation}\label{eq:s3}
1082: \sum\limits_{l=1}^{L-1}d_l + d'_L- d_0 - (L-1)\sigma_x^2 = 0,
1083: \end{equation}
1084: and we choose $a = \sigma_x^2$, $\sigma_l^2 =
1085: (d_l^{-1}-\sigma_x^{-2})^{-1}$ for $l=1,\cdots,L-1$, and
1086: $\sigma_L^2 = ({d'}_L^{-1}-\sigma_x^{-2})^{-1}$ in $K_w$. Defining
1087: $\sigma_0^2 = (d_0^{-1}-\sigma_x^{-2})^{-1}$, \eqref{eq:s3}
1088: is equivalent to the following equation:
1089: \begin{equation}\label{eq:s4}
1090: \left[\sigma_0^2+\sigma_x^2\right]^{-1} =
1091: \sum\limits_{l=1}^L\left[\sigma_l^2+\sigma_x^2\right]^{-1}.
1092: \end{equation}
1093: From Lemma \ref{lemma:positivedefinite}, our choice of $K_w$ is
1094: positive definite. Thus the resulting distortions are $(d_1, \;
1095: \dots, \; d_{L-1}, \; d'_L, d_0)$, where $0 < d'_L \le d_L$.
1096:
1097: Using the determinant equation \begin{equation}
1098: \begin{vmatrix}
1099: \sigma_1^2 & -\sigma_x^2 & -\sigma_x^2 & -\sigma_x^2 & \dots & -\sigma_x^2\\
1100: -\sigma_x^2 & \sigma_2^2 & -\sigma_x^2 & -\sigma_x^2 & \dots & -\sigma_x^2 \\
1101: -\sigma_x^2 & -\sigma_x^2 & \sigma_3^2 & -\sigma_x^2 & \dots & -\sigma_x^2 \\
1102: \hdotsfor{6} \\
1103: -\sigma_x^2 & \dots & -\sigma_x^2 & -\sigma_x^2 & \sigma_{L-1}^2 & -\sigma_x^2 \\
1104: -\sigma_x^2 & \dots & -\sigma_x^2 & -\sigma_x^2 & -\sigma_x^2 & \sigma_L^2
1105: \end{vmatrix}
1106: =
1107: \Big(1-\sum\limits_{l=1}^L\frac{\sigma_x^2}{\sigma_l^2+\sigma_x^2}\Big)\prod\limits_{l=1}^L(\sigma_l^2+\sigma_x^2)
1108: \end{equation}
1109: and \eqref{eq:s4}, we have an achievable sum rate
1110: %{\tt I made the inequality an equality for the logic to hold.}
1111: \begin{equation}
1112: \sum\limits_{l=1}^LR_l = \frac{1}{2}\log
1113: \frac{\sigma_x^2}{d_0}.
1114: \end{equation}
1115: We conclude that in this case the point-to-point rate-distortion
1116: bound for the central receiver is achievable.
1117:
1118: In summary, we have shown that the jointly Gaussian description
1119: scheme achieves the lower bound on the sum rate. Further, the sum
1120: rate can be calculated either trivially (by choosing $a^*=0$ in
1121: case II or $a^*=1$ in case III) or by solving a polynomial
1122: equation in a single variable (case I).
1123:
1124:
1125: \section{Vector Gaussian Source}
1126: \label{sec6} The essence of our proof of the optimality of jointly
1127: Gaussian description scheme for scalar Gaussian sources is the use
1128: of the {\em
1129: intermediate value theorem} for scalar continuous
1130: functions. However, there is no natural extension of this theorem
1131: for vector valued functions. To avoid this problem, we first
1132: explicitly solve the two description problem and characterize the
1133: optimality of jointly Gaussian description scheme. Next, we show
1134: that the jointly Gaussian description scheme is optimal for $L
1135: \ge 2$ by showing an equivalence of certain optimization
1136: problems. In the last part of this section, we show that
1137: the jointly Gaussian description strategy can achieve the optimal
1138: rate region for the two description problem.
1139:
1140: \subsection{Explicit Solutions for Some Cases of Two Description
1141: Problem}\label{sec:vector2}
1142:
1143: With only two descriptions, we can explicitly solve
1144: \eqref{eq:core}, thus generalizing the corresponding solution for
1145: the scalar Gaussian source, derived in \cite{Ozarow80}.
1146: %when the distortion constraint matrices satisfy conditions
1147: %analogous to that described in the case of scalar Gaussian source.
1148:
1149: Suppose the distortion constraints are denoted by $(\vd_1,\;
1150: \vd_2, \; \vd_0)$ and let
1151: \[
1152: \vk_w = \begin{pmatrix}
1153: \vk_{w_1} & -\va^* \\
1154: -\va^* & \vk_{w_2}
1155: \end{pmatrix}.
1156: \]
1157: We now solve \eqref{eq:distortion1}, which is equivalent to
1158: \eqref{eq:core}, for $\vk_{w_1}$, $\vk_{w_2}$ and $\va^*$. From
1159: \eqref{eq:distortion1} we get
1160: \begin{equation}
1161: \vk_{w_l} = (\vd_l^{-1}-\vk_x^{-1})^{-1}, \quad l=1,\;2,
1162: \end{equation}
1163: and
1164: \begin{equation}
1165: \vd_0^{-1} = \vk_x^{-1}+ (\vi \; \vi)\vk_w^{-1}(\vi \; \vi)^t.
1166: \end{equation}
1167: Expanding out $\vk_w^{-1}$ using Lemma \ref{lemma:blockinverse} in
1168: Appendix \ref{app:matrix}, we get
1169: \begin{equation}
1170: \vd_0^{-1}-\vk_x^{-1} =
1171: \vk_{w_1}^{-1}+(\vi+\vk_{w_1}^{-1}\va^*)(\vk_{w_2}-\va^*\vk_{w_1}^{-1}\va^*)^{-1}(\vi+\va^*\vk_{w_1}^{-1}).
1172: \end{equation}
1173: Taking inverse on both sides, we have
1174: \begin{equation}\label{eq:47}
1175: (\vd_0^{-1} - \vk_x^{-1})^{-1} =
1176: \vk_{w_1}-(\vk_{w_1}+\va^*)(\vk_{w_1}+\vk_{w_2}+2\va^*)^{-1}(\vk_{w_1}+\va^*).
1177: \end{equation}
1178: Defining $\vk_{w_0}$ as
1179: \begin{equation}
1180: \vk_{w_0} \overset{\text{def}}{=} [\vd_0^{-1} - \vk_x^{-1}]^{-1} ,
1181: \end{equation}
1182: \eqref{eq:47} is equivalent to
1183: \begin{equation}\label{eq:49}
1184: \vk_{w_1}-\vk_{w_0} =
1185: (\vk_{w_1}+\va^*)(\vk_{w_1}+\vk_{w_2}+2\va^*)^{-1}(\vk_{w_1}+\va^*).
1186: \end{equation}
1187: Defining
1188: \[
1189: \mathbf{X} \overset{\text{def}}{=} \vk_{w_1}+\va^*,
1190: \]
1191: \eqref{eq:49} is equivalent to
1192: \begin{equation}
1193: \vk_{w_1}-\vk_{w_0} =
1194: \mathbf{X}(2\mathbf{X}+\vk_{w_2}-\vk_{w_1})^{-1}\mathbf{X},
1195: \end{equation}
1196: which is further equivalent to
1197: \begin{equation}
1198: \mathbf{X}(\vk_{w_1}-\vk_{w_0})^{-1}\mathbf{X} =
1199: 2\mathbf{X}+\vk_{w_2}-\vk_{w_1}.
1200: \end{equation}
1201: This is a version of the so-called {\em algebraic Riccati equation};
1202: the corresponding Hamiltonian is readily seen to be positive
1203: semidefinite and we can even write down the following explicit solution:
1204: \begin{equation}
1205: \begin{split}
1206: \mathbf{X} = & \vk_{w_1}-\vk_{w_0}
1207: \\
1208: & +(\vk_{w_1}-\vk_{w_0})^{\frac{1}{2}}\left[(\vk_{w_1}-\vk_{w_0})^{-\frac{1}{2}}
1209: (\vk_{w_2}-\vk_{w_0})(\vk_{w_1}-\vk_{w_0})^{-\frac{1}{2}}\right]^{\frac{1}{2}}(\vk_{w_1}-\vk_{w_0})^{\frac{1}{2}}.
1210: \end{split}
1211: \end{equation}
1212: Thus
1213: \begin{equation}
1214: \va^* =
1215: (\vk_{w_1}-\vk_{w_0})^{\frac{1}{2}}\left[(\vk_{w_1}-\vk_{w_0})^{-\frac{1}{2}}
1216: (\vk_{w_2}-\vk_{w_0})(\vk_{w_1}-\vk_{w_0})^{-\frac{1}{2}}\right]^{\frac{1}{2}}(\vk_{w_1}-\vk_{w_0})^{\frac{1}{2}}
1217: -\vk_{w_0}.
1218: \end{equation}
1219: Now, if $\mathbf{0} \prec \va^* \prec \vk_x$ then we can appeal to
1220: Theorem~\ref{th:main} and arrive at the explicit jointly Gaussian
1221: description scheme parameterized by $\vk_w$ that achieves the sum
1222: rate. Analogous to the scalar case (cf.\ \cite{Ozarow80}), we
1223: have the following sufficient condition for when this is true; the
1224: proof is available in Appendix~\ref{app:twod}.
1225: \begin{proposition}\label{prop:twod}
1226: If the distortion constraints $(\vd_1,\;\vd_2,\;\vd_0)$ satisfy
1227: \begin{equation}\label{eq:vdfor2}
1228: \begin{split}
1229: & \vd_0 + \vk_x - \vd_1 - \vd_2 \succ \mathbf{0} \\
1230: {\textup{and}} \quad & \vd_0^{-1} + \vk_x^{-1} -
1231: \vd_1^{-1}-\vd_2^{-1} \succ \mathbf{0},
1232: \end{split}
1233: \end{equation}
1234: then $\mathbf{0} \prec \va^* \prec \vk_x$.
1235: \end{proposition}
1236: We now complete the proof by considering the cases that are not
1237: covered by the conditions in Proposition~\ref{prop:twod}.
1238: \begin{itemize}
1239: \item When
1240: \[
1241: \vd_0^{-1} + \vk_x^{-1} - \vd_1^{-1}-\vd_2^{-1} \preccurlyeq
1242: \mathbf{0},
1243: \]
1244: we can choose $\va^* = \mathbf{0}$ to achieve the sum of
1245: point-to-point individual rate-distortion functions. Thus in this
1246: case, the sum rate is equal to this natural lower bound.
1247: \item When
1248: \[
1249: \vd_0 + \vk_x - \vd_1 - \vd_2 \preccurlyeq \mathbf{0},
1250: \]
1251: we can choose $\va^* = \vk_x$ to achieve the point-to-point rate
1252: distortion-function for central receiver, also a natural lower
1253: bound.
1254:
1255: \item When neither $\vd_0 + \vk_x - \vd_1 - \vd_2$ nor $\vd_0^{-1}
1256: + \vk_x^{-1} - \vd_1^{-1}-\vd_2^{-1}$ is positive or negative
1257: semidefinite (this case cannot happen in the scalar case), we cannot
1258: use Theorem~\ref{th:main}, and the trivial choice of $\va^* =
1259: \mathbf{0}$ or $\va^* = \vk_x$ does not meet the lower bound. In
1260: the next subsection we will address this case and prove that
1261: the jointly Gaussian
1262: description scheme indeed achieves the lower bound on the sum rate
1263: for $L \ge 2$.%, and
1264: %hence we can conclude the optimality of jointly Gaussian
1265: %description scheme for this case of the two description problem.
1266:
1267: %{\tt What happens when these two constraints are only partially
1268: % met and partially unmet?}
1269: \end{itemize}
1270:
1271: %{\tt Need to explain this part in more detail.}
1272:
1273: If we let the source to be scalar Gaussian, our result reduces to
1274: Ozarow's solution of the two description problem for a scalar Gaussian
1275: source\cite{Ozarow80}: this is because the last case described above
1276: does not happen in the scalar case.
1277:
1278: \subsection{Solutions for $L \ge 2$}\label{sec:vecl}
1279: While we exactly characterized the optimal jointly Gaussian
1280: description scheme and used this characterization in arguing that
1281: it achieves the fundamental lower bound to the sum rate, such
1282: exact calculations do not appear to be as immediate when $L>2$.
1283: So, we eschew this somewhat brute-force approach and resort to a
1284: more subtle proof that involves exploring the structure of the
1285: solution to an optimization problem. First, note that by a linear
1286: transformation at the encoders and the decoders, we have the
1287: following result on rate region for multiple description with
1288: individual and central receivers.
1289: \begin{proposition}
1290: %The rate region for Gaussian vector source with covariance $\vk_x$
1291: %and distortion constraints $(\vd_1, \; \dots, \; \vd_L, \; \vd_0)$
1292: %is the same as the rate region for Gaussian vector source with
1293: %covariance $\vi$ and distortion constraints
1294: %$(\vk_x^{-\frac{1}{2}}\vd_1\vk_x^{-\frac{1}{2}}, \; \dots, \;,
1295: %\vk_x^{-\frac{1}{2}}\vd_L\vk_x^{-\frac{1}{2}}, \;
1296: %\vk_x^{-\frac{1}{2}}\vd_0\vk_x^{-\frac{1}{2}})$, i.e.,
1297: \begin{equation}
1298: R_*(\vk_x,\;\vd_1, \; \dots, \;, \vd_L, \;
1299: \vd_0) = R_*(\vi,\;\vk_x^{-\frac{1}{2}}\vd_1\vk_x^{-\frac{1}{2}},
1300: \; \dots, \;, \vk_x^{-\frac{1}{2}}\vd_L\vk_x^{-\frac{1}{2}}, \;
1301: \vk_x^{-\frac{1}{2}}\vd_0\vk_x^{-\frac{1}{2}}).
1302: \end{equation}
1303: \end{proposition}
1304: Thus, throughout this subsection we will suppose, for notation
1305: simplicity, that $\vk_x = \vi$.
1306:
1307: Given distortion constraints $(\mathbf{D}_1,\; \dots \, \;
1308: \mathbf{D}_L,\; \mathbf{D}_0)$, let
1309: \begin{equation}
1310: \vk_{w_l} = (\vd_l^{-1}-\vi)^{-1}, \quad l=0, \; 1, \; \dots, \;
1311: L,
1312: \end{equation}
1313: and define
1314: \begin{eqnarray}
1315: f(\va) \overset{\text{def}}{=} & \left[\vk_{w_0}+\va\right]^{-1} -
1316: \sum\limits_{l=1}^L\left[\vk_{w_l}+\va\right]^{-1}, \\
1317: F(\va) \overset{\text{def}}{=} & \log |\vk_{w_0}+\va| -
1318: \sum\limits_{l=1}^L \log |\vk_{w_l}+\va|.
1319: \end{eqnarray}
1320: Note that
1321: \begin{equation}
1322: \frac{dF(\va)}{d\va} = f(\va).
1323: \end{equation}
1324: Consider the following optimization problem:
1325: \begin{equation}\label{eq:optimal}
1326: \max\limits_{\mathbf{0} \preccurlyeq \va \preccurlyeq
1327: \vi}\quad\quad F(\va).
1328: \end{equation}
1329: Now, since $F(\va)$ is a continuous map and $\mathbf{0} \preccurlyeq
1330: \va \preccurlyeq \vi$ is a compact set, there exists an optimal
1331: solution $\va^*$ to \eqref{eq:optimal} where $\va^*$ satisfies the
1332: Karush-Kuhn-Tucker (KKT) conditions: there exist $\vl_1 \succcurlyeq
1333: \mathbf{0}$ and $\vl_2 \succcurlyeq \mathbf{0}$ such that
1334: \begin{eqnarray}
1335: f(\va^*)+\vl_1-\vl_2 & = & \mathbf{0} \label{eq:kkt1}\\
1336: \vl_1 \va^* & = & \mathbf{0}\label{eq:kkt2} \\
1337: \vl_2 (\va^*-\vi) & = & \mathbf{0}\label{eq:kkt3}.
1338: \end{eqnarray}
1339: %For any given distortion constraint $(\mathbf{D}_1,\; \dots \, \;
1340: %\mathbf{D}_L,\; \mathbf{D}_0)$,
1341: Now $\va^*$ falls into the following four cases.
1342:
1343: {\bf Case 1:} $\mathbf{0} \prec \mathbf{A}^* \prec \mathbf{I}$.
1344: Alternatively, 0 and 1 are not eigenvalues of $\va^*$. In this
1345: case, $\vl_1 = {\bf 0}$ and $\vl_2 ={\bf 0}$; thus the KKT
1346: conditions in \eqref{eq:kkt1} reduce to
1347: \[
1348: f(\va^*) = \mathbf{0}.
1349: \]
1350: Equivalently,
1351: \begin{equation}
1352: \left[\vk_{w_0}+\va^*\right]^{-1} =
1353: \sum\limits_{l=1}^L\left[\vk_{w_l}+\va^*\right]^{-1}.
1354: \end{equation}
1355: From Theorem~\ref{th:main}, the jointly Gaussian description
1356: scheme with covariance matrix for $\vw_1, \; \dots, \; \vw_L$
1357: being
1358: \begin{equation}\label{eq:kwvector}
1359: \vk_w = \begin{pmatrix}
1360: \vk_{w_1} & -\va^* & -\va^* & \dots & -\va^* \\
1361: -\va^* & \vk_{w_2} & -\va^* & \dots & -\va^* \\
1362: \hdotsfor{5} \\
1363: -\va^* & \dots & -\va^* & \vk_{w_{L-1}} & -\va^* \\
1364: -\va^* & \dots & -\va^* & -\va^* & \vk_{w_L}
1365: \end{pmatrix}
1366: \end{equation}
1367: achieves the lower bound to the sum rate. Thus in this case, we have
1368: characterized the optimality of the jointly Gaussian
1369: description scheme parameterized by \eqref{eq:kwvector} in terms of
1370: achieving the sum rate.
1371:
1372: {\bf Case 2:} $\mathbf{0} \preccurlyeq \va^* \prec \vi$. Alternatively,
1373: some eigenvalues of $\va^*$ are 0, but no eigenvalues of $\va^*$ are
1374: 1. Thus $\vl_2 = {\bf 0}$ and the KKT conditions in \eqref{eq:kkt1}
1375: reduce to
1376: \begin{equation}\label{eq:kkt1_again}
1377: (\vk_{w_0}+\va^*)^{-1} -
1378: \sum\limits_{l=1}^L(\vk_{w_l}+\va^*)^{-1}+\vl_1 = \mathbf{0},
1379: \end{equation}
1380: for some $\vl_1 \succcurlyeq \mathbf{0}$ satisfying $\vl_1 \va^* =
1381: \mathbf{0}$. The {\em key idea} now is to see that the distortion constraint
1382: on the central receiver is too loose and we can in fact achieve a {\em
1383: lesser} distortion (in the sense of positive semidefinite ordering)
1384: for the same sum rate. We first identify this lower distortion:
1385: defining
1386: $$
1387: \vk_{w_0}^* = \left(\vk_{w_0}^{-1}+\vl_1\right)^{-1},
1388: $$
1389: consider the smaller distortion matrix on the central receiver
1390: $$
1391: \vd_0^* = \left( {\vk_{w_0}^*}^{-1}+\vi\right)^{-1} = \left(\vi+
1392: \vk_{w_0}^{-1}+\vl_1\right)^{-1} = (\vd_0^{-1} + \vl_1)^{-1} \prec
1393: \vd_0.
1394: $$
1395: This new distortion matrix on the central receiver satisfies two
1396: key properties, that we state as a lemma (whose proof is available
1397: in Appendix \ref{app:enhanced0}).
1398: \begin{lemma}\label{lemma:enhanced0}
1399: \begin{eqnarray}
1400: (\vk_{w_0}+\va^*)^{-1}+\vl_1 & = & (\vk_{w_0}^*+\va^*)^{-1},
1401: \label{eq:enhanced_1}\\
1402: \frac{|\vd_0+\vk_z|}{|\vd_0|} & = &
1403: \frac{|\vd_0^*+\vk_z|}{|\vd_0^*|}.\label{eq:enhanced0_1}
1404: \end{eqnarray}
1405: \end{lemma}
1406: Comparing \eqref{eq:kkt1_again} with \eqref{eq:enhanced_1}, we have
1407: \begin{equation}\label{eq:vk}
1408: \left[\vk^*_{w_0}+\va^*\right]^{-1} =
1409: \sum\limits_{l=1}^{L}\left[\vk_{w_l}+\va^*\right]^{-1}.
1410: \end{equation}
1411: Now, the corresponding $\vk_z = (\vi-\va^*)^{-1}-\vi$ is singular.
1412: If it hadnt been, then by Theorem~\ref{th:main} we could have
1413: concluded that jointly Gaussian description scheme achieves the
1414: lower bound to the sum rate. We now address this technical
1415: difficulty.
1416:
1417:
1418: Our first observation is that there exists $\delta >0$ such that for
1419: all $\epsilon \in
1420: (0,\delta)$ we have $\mathbf{0} \prec \va+\epsilon \vi \prec \vi$,
1421: and $0 \prec \vk_{w_0}^*-\epsilon \vi$, $\mathbf{0} \prec
1422: \vk_{w_l}-\epsilon \vi$, and we can rewrite \eqref{eq:vk} as
1423: \begin{equation}\label{eq:continues1}
1424: \big[(\vk_{w_0}^*-\epsilon \vi)+(\va^*+\epsilon \vi)\big]^{-1} =
1425: \sum\limits_{l=1}^L\big[(\vk_{w_l}-\epsilon \vi)+(\va^*+\epsilon
1426: \vi)\big]^{-1}.
1427: \end{equation}
1428: Thus if the distortion constraints were
1429: $(\vd_1(\epsilon), \; \dots , \; \vd_L(\epsilon), \;
1430: \vd_0(\epsilon))$ with
1431: \begin{equation*}
1432: \begin{split}
1433: \vd_l(\epsilon) & = \left[(\vk_{w_l}-\epsilon
1434: \vi)^{-1}+\vi\right]^{-1}, \quad \listl, \\
1435: \vd_0(\epsilon) & = \left[(\vk_{w_0}^*-\epsilon
1436: \vi)^{-1}+\vi\right]^{-1},
1437: \end{split}
1438: \end{equation*}
1439: then $ \va^*+\epsilon \vi$ is a solution to
1440: \eqref{eq:continues1}. This situation corresponds to that discussed in
1441: Case I; we can conclude that sum rate for this modified distortion
1442: multiple description problem is
1443: \begin{equation}
1444: %\begin{split}
1445: %R_l(\epsilon) & \ge \frac{1}{2}\log \frac{1}{|\vd_l(\epsilon)|} \\
1446: % \sum\limits_{l=1}^LR_l(\epsilon) \ge
1447: \frac{1}{2}\log\frac{|\vi+\vk_z(\epsilon)|^{(L-1)}|\vd_0(\epsilon)+\vk_z(\epsilon)|}{|\vd_0(\epsilon)|\prod\limits_{l=1}^L
1448: |\vd_l(\epsilon)+\vk_z(\epsilon)|},
1449: %\end{split}
1450: \end{equation}
1451: where $\vk_z(\epsilon) = \left[\vi-(\va^*+\epsilon
1452: \vi)\right]^{-1}-\vi$.
1453: We would like to let $\epsilon$ approach zero and consider the
1454: limiting multiple description problem. In particular, we show that
1455: \begin{eqnarray}
1456: \vd_l(\epsilon) & \rightarrow & \vd_l, \quad \listl, \label{eq:deps1} \\
1457: \vd_0(\epsilon) & \rightarrow & \vd_0^*, \label{eq:deps2}
1458: \end{eqnarray}
1459: as $\epsilon\rightarrow 0$ in Appendix~\ref{app:depsilon}. Further, we
1460: show that
1461: \begin{equation}
1462: \vk_z(\epsilon) \rightarrow (\vi-\va^*)^{-1}-\vi,\label{eq:eps3}
1463: \end{equation}
1464: as $\epsilon\rightarrow 0$ in Appendix~\ref{app:kzepsilon}.
1465: Thus we can conclude that the sum rate approaches, using
1466: \eqref{eq:enhanced0_1},
1467: %Lemma~\ref{lemma:enhanced0},
1468: \begin{equation}
1469: %\begin{split}
1470: %R_l & \ge \frac{1}{2}\log \frac{1}{|\vd_l|} \\
1471: \frac{1}{2}\log\frac{|\vi+\vk_z|^{(L-1)}|\vd_0+\vk_z|}{|\vd_0|\prod\limits_{l=1}^L
1472: |\vd_l+\vk_z|},
1473: %\end{split}
1474: \end{equation}
1475: as $\epsilon \rightarrow 0$; here $\vk_z = (\vi-\va^*)^{-1}-\vi$.
1476: We observe that this sum rate is achievable using the jointly
1477: Gaussian multiple scheme. Further, this sum rate is identical to
1478: the lower bound to sum rate for the original distortions $(\vd_1,
1479: \; \dots , \; \vd_L, \; \vd_0)$. Thus we conclude the optimality
1480: of the jointly Gaussian description scheme in this case as well.
1481:
1482: {\bf Case 3:} $\mathbf{0} \prec \va^* \preccurlyeq \vi$.
1483: Alternatively, some eigenvalues of $\va^*$ are 1, but no
1484: eigenvalues of $\va^*$ are 0. In this case, the $\vl_1 = {\bf 0}$
1485: and the KKT conditions in \eqref{eq:kkt1} reduce to
1486: \begin{equation}
1487: %\begin{cases}
1488: (\vk_{w_0}+\va^*)^{-1} -
1489: \sum\limits_{l=1}^L(\vk_{\vw_l}+\va^*)^{-1}-\vl_2 = 0,
1490: %\end{cases}
1491: \end{equation}
1492: for some $\vl_2 \succcurlyeq 0$ satisfying $\vl_2 (\va^*-\vi) = {\bf
1493: 0}$. Defining
1494: %\begin{lemma}\label{lemma:enhancedl}
1495: $$
1496: \vk_{w_l}^* = \left[
1497: (\vk_{\vw_l}+\vi)^{-1}+\vl_2\right]^{-1}-\vi, $$
1498: we have, as in \eqref{eq:enhanced_1}, that
1499: \begin{equation}\label{eq:enh1}
1500: (\vk_{\vw_l}+\va^*)^{-1}+\vl_2 = (\vk_{\vw_l}^*+\va^*)^{-1}.
1501: \end{equation}
1502: %\end{lemma}
1503: The observation
1504: \[
1505: (\vk_{\vw_l}+\va^*)^{-1}+\vl_2 =
1506: \left[(\vk_{\vw_l}+\vi)+(\va^*-\vi)\right]^{-1}+\vl_2,
1507: \]
1508: combined with the proof of \eqref{eq:enhanced_1} suffices to justify
1509: \eqref{eq:enh1}. Now, from \eqref{eq:enh1},
1510: \begin{equation}\label{eq:case3}
1511: (\vk_{w_0}+\va^*)^{-1} -
1512: \sum\limits_{l=1}^{L-1}(\vk_{w_l}+\va^*)^{-1}-(\vk_{w_L}^*+\va^*)^{-1}
1513: = {\bf 0}.
1514: \end{equation}
1515: As in the previous case, the key step is to identify smaller
1516: distortion matrices at each of the individual receivers (ordered in the positive semidefinite sense) that is
1517: achievable at the same sum rate:
1518: $$\vd_l^* =\left[{\vk_{w_l}^*}^{-1}+\vi\right]^{-1}, \quad l = 1, \;
1519: \ldots, \;
1520: L.$$
1521: To see that this is indeed
1522: a smaller distortion matrix, observe that since $\vk_w$ is
1523: positive definite, it follows that $\vk_{w_l}^* \succ 0$ and
1524: \begin{equation}
1525: \begin{split}
1526: \vd_l^* & = \left[{\vk_{w_l}^*}^{-1}+\vi\right]^{-1} \\
1527: & = \left[\left(\left(
1528: (\vk_{w_l}+\vi)^{-1}+\vl_2\right)^{-1}-\vi\right)^{-1}+\vi\right]^{-1}
1529: \\
1530: & = \left[ \vi - (\vk_{w_l}+\vi)^{-1}-\vl_2\right] \\
1531: & = \left[ \vi + \vk_{w_l} \right]^{-1} -\vl_2 \\
1532: & = \vd_l - \vl_2, \quad l = 1, \; \ldots, \;
1533: L.
1534: \end{split}
1535: \end{equation}
1536: Since $\vl_2 \succcurlyeq {\bf 0}$, it follows that $\mathbf{0}
1537: \prec \vd_l^* \preccurlyeq \vd_l, \; l=1, \; \ldots, \; L$. Define
1538: \begin{equation}
1539: \begin{split}
1540: \vd_l(\epsilon) & = \left[(\vk_{w_l}+\epsilon
1541: \vi)^{-1}+\vi\right]^{-1}, \quad l=0,\;1,\;\dots,\;L-1, \\
1542: \vd_L(\epsilon) & = \left[(\vk_{w_L}^*+\epsilon
1543: \vi)^{-1}+\vi\right]^{-1},
1544: \end{split}
1545: \end{equation}
1546: then there exists $\delta >0$ such that for all $\epsilon \in
1547: (0,\delta)$ we have $ \mathbf{0} \prec \va^*-\epsilon \vi \prec
1548: \vi$, and $\mathbf{0} \prec \vd_l(\epsilon) \prec \vi$. We can
1549: rewrite \eqref{eq:case3} as
1550: \begin{equation}\label{eq:continues2}
1551: \left[(\vk_{w_0}+\epsilon \vi)+(\va^*-\epsilon \vi)\right]^{-1} =
1552: \sum\limits_{l=1}^{L-1}\left[(\vk_{w_l}+\epsilon
1553: \vi)+(\va^*-\epsilon \vi)\right]^{-1}+\left[(\vk_{w_L}^*+\epsilon
1554: \vi)+(\va^*-\epsilon \vi)\right]^{-1}.
1555: \end{equation}
1556: Thus if the distortion constraints were $(\vd_1(\epsilon), \;
1557: \dots , \; \vd_L(\epsilon), \; \vd_0(\epsilon))$, then $
1558: \va^*-\epsilon \vi$ is a solution to \eqref{eq:continues2}. This
1559: situation corresponds to that discussed in Case I; we conclude
1560: that the sum rate for this modified distortion multiple
1561: description problem is
1562: \begin{equation}
1563: %\begin{split}
1564: %R_l(\epsilon) & \ge \frac{1}{2}\log \frac{1}{|\vd_l(\epsilon)|} \\
1565: \frac{1}{2}\log\frac{|\vi+\vk_z(\epsilon)|^{(L-1)}|\vd_0(\epsilon)+\vk_z(\epsilon)|}{|\vd_0(\epsilon)|\prod\limits_{l=1}^L
1566: |\vd_l(\epsilon)+\vk_z(\epsilon)|},
1567: %\end{split}
1568: \end{equation}
1569: where $\vk_z(\epsilon) = \left[\vi-(\va^*-\epsilon
1570: \vi)\right]^{-1}-\vi$. We would like to let $\epsilon$ approach
1571: zero and consider the limiting multiple description problem.
1572: Similar to equations~\eqref{eq:deps1} and~\eqref{eq:deps2}, we
1573: have
1574: \begin{equation}
1575: \begin{split}
1576: \vd_l(\epsilon) & \rightarrow \vd_l,\quad l=1,\ldots L, \\
1577: \vd_0(\epsilon) & \rightarrow \vd_0^*.
1578: \end{split}
1579: \end{equation}
1580: Further, we show that
1581: \begin{equation}\label{eq:kzinflemma}
1582: \lim_{\epsilon \rightarrow
1583: 0}\frac{|\vi+\vk_z(\epsilon)|^{(L-1)}|\vd_0(\epsilon)+\vk_z(\epsilon)|}{|\prod\limits_{l=1}^L
1584: |\vd_l(\epsilon)+\vk_z(\epsilon)|} = 1
1585: \end{equation}
1586: in Appendix \ref{app:kzinflemma}. We can now conclude that the
1587: sum rate approaches
1588: \begin{equation}
1589: %\begin{split}
1590: %R_l & \ge \frac{1}{2}\log \frac{1}{|\vd_l|} \\
1591: \frac{1}{2}\log\frac{1}{|\vd_0|}
1592: %\end{split}
1593: \end{equation}
1594: as $\epsilon$ approaches 0.
1595: %Thus the corner point
1596: %\begin{equation}
1597: %\begin{split}
1598: %R_l &= \frac{1}{2}\log\frac{1}{|\vd_l|}, \; l=1,\cdots,L-1 \\
1599: %R_L &= \frac{1}{2}\log\frac{1}{|\vd_0|} -
1600: %\frac{1}{2}\log\frac{1}{|\vd_1|\dots|\vd_{L-1}|}
1601: %\end{split}
1602: %\end{equation}
1603: In other words, the point-to-point rate-distortion function for central
1604: receiver with distortion $\vd_0$ can be achieved by using the jointly
1605: Gaussian description scheme, and the resulting distortion is
1606: $(\vd_1, \; \dots , \; \vd_L^*, \; \vd_0)$ where $\mathbf{0} \prec
1607: \vd_L^* \preccurlyeq \vd_L$. In conclusion, the jointly Gaussian description
1608: scheme is also optimal in this case.
1609: %From similar argument, we can achieve all the corner points of the
1610: %point-to-point lower bound and satisfy the distortion constraint.
1611: %Thus by time sharing, the entire point-to-point lower bound can be
1612: %achieved.
1613:
1614: {\bf Case 4:} $\mathbf{0} \preccurlyeq \va^* \preccurlyeq \vi$.
1615: i.e., both 0 and 1 are eigenvalues of $\va^*$. In this case, the
1616: KKT conditions are: there exist $\vl_1 \succcurlyeq 0$ and $
1617: \vl_2 \succcurlyeq 0$ such that equations \eqref{eq:kkt1},
1618: \eqref{eq:kkt2} and \eqref{eq:kkt3} hold. We can combine equations
1619: \eqref{eq:enhanced_1} and \eqref{eq:enh1} to get
1620: \begin{equation}\label{eq:case4}
1621: (\vk_{w_0}^*+\va^*)^{-1} =
1622: \sum\limits_{l=1}^{L-1}(\vk_{w_l}+\va^*)^{-1}+(\vk_{w_L}^*+\va^*)^{-1},
1623: \end{equation}
1624: where
1625: \begin{equation*}
1626: \begin{split}
1627: \vk_{w_0}^* & = \left(\vk_{w_0}^{-1}+\vl_1\right)^{-1}, \\
1628: \vk_{w_L}^* & = \left[ (\vk_{w_L}+\vi)^{-1}+\vl_2\right]^{-1}-\vi.
1629: \end{split}
1630: \end{equation*}
1631:
1632: As in cases 2 and 3, we want to show the optimality of the jointly
1633: Gaussian multiple
1634: description scheme through a limiting procedure. We do this by first
1635: perturbing $\va^*$ so that it has no eigenvalue equal to
1636: 0 or 1 as follows.
1637:
1638: Without loss of generality, suppose that $\va^*$ has $p$ eigenvalues
1639: equal to 0 and $q$ eigenvalues equal 1, where $p>0$ and $q>0$, and
1640: there exists $N \times N$ orthogonal matrix $\vq$ such that
1641: \[
1642: \vq \va^* \vq^t =
1643: \diag\{\underset{p}{\underbrace{0,\;\dots,\;0}},\underset{q}{\underbrace{\;1,\;\dots,\;1}},\;a_{p+q+1},\;\dots,\;a_N\},
1644: \]
1645: with $0 < a_{p+q+1} <1, \; \dots, 0 < a_{N} <1$. We need to
1646: perturb the eigenvalues of $\va^*$ away from both 0 and 1. Towards this,
1647: we define two $N \times N$ diagonal matrices:
1648: \begin{equation*}
1649: \begin{split}
1650: \mathbf{E}_1 & =
1651: \diag(\underset{p}{\underbrace{1,\;\dots,\;1}},\underset{N-p}{\underbrace{\;0,\;\dots,\;0,\;0,\;\dots,\;0}}),
1652: \\
1653: \mathbf{E}_2 & =
1654: \diag(\underset{p}{\underbrace{0,\;\dots,\;0}},\underset{q}{\underbrace{\;1,\;\dots,\;1}},\;0,\;\dots,\;0),
1655: \end{split}
1656: \end{equation*}
1657: Also define
1658: \begin{equation*}
1659: \begin{split}
1660: \va^*(\epsilon_1,\epsilon_2) & = \va^* + \vq^t (\epsilon_1
1661: \mathbf{E}_1 - \epsilon_2 \mathbf{E}_2)\vq, \\
1662: \vk_z(\epsilon_1,\epsilon_2) & =
1663: (\vi-\va^*(\epsilon_1,\epsilon_2))^{-1}-\vi, \\
1664: \vk_{w_l}(\epsilon_1,\epsilon_2) & = \vk_{w_l} - \vq^t (\epsilon_1
1665: \mathbf{E}_1 - \epsilon_2 \mathbf{E}_2)\vq , \quad l=1,\;\dots,\;L-1,\\
1666: \vk_{w_L}(\epsilon_1,\epsilon_2) & = \vk_{w_L}^* - \vq^t
1667: (\epsilon_1
1668: \mathbf{E}_1 - \epsilon_2 \mathbf{E}_2)\vq, \\
1669: \vk_{w_0}(\epsilon_1,\epsilon_2) & = \vk_{w_0}^* - \vq^t
1670: (\epsilon_1 \mathbf{E}_1 - \epsilon_2 \mathbf{E}_2)\vq.
1671: \end{split}
1672: \end{equation*}
1673: Further, defining
1674: \begin{equation}
1675: \vd_l(\epsilon_1,\epsilon_2) = (\vi +
1676: \vk_{w_l}(\epsilon_1,\epsilon_2))^{-1}, \quad \listl,
1677: \end{equation}
1678: there exists $\delta >0$ such that for all $\epsilon_1 \in
1679: (0,\delta)$ and $\epsilon_2 \in (0,\delta)$ we have $ \mathbf{0}
1680: \prec \va^*(\epsilon_1,\epsilon_2) \prec \vi$, and $\mathbf{0}
1681: \prec \vd_l(\epsilon_1,\epsilon_2) \prec \vi$.
1682: Now, we can rewrite \eqref{eq:case4} as
1683: \begin{equation}\label{eq:continues3}
1684: \Big[\vk_{w_0}(\epsilon_1,\epsilon_2)+\va^*(\epsilon_1,\epsilon_2)\Big]^{-1}
1685: =
1686: \sum\limits_{l=1}^{L}\Big[\vk_{w_l}(\epsilon_1,\epsilon_2)+\va^*(\epsilon_1,\epsilon_2)\Big]^{-1}.
1687: \end{equation}
1688: Thus if the distortion
1689: constraints were $(\vd_1(\epsilon_1, \epsilon_2), \; \dots , \;
1690: \vd_L(\epsilon_1,\epsilon_2), \; \vd_0(\epsilon_1,\epsilon_2))$,
1691: then
1692: $\va^*(\epsilon_1,\epsilon_2)$ is a solution to
1693: \eqref{eq:continues3}. This situation corresponds to that
1694: discussed in Case I; we conclude that the sum rate for this
1695: modified distortion multiple description problem is
1696: \begin{equation}
1697: \frac{1}{2}\log\frac{|\vi+\vk_z(\epsilon_1,\epsilon_2)|^{(L-1)}|\vd_0(\epsilon_1,\epsilon_2)+
1698: \vk_z(\epsilon_1,\epsilon_2)|}{|\vd_0(\epsilon_1,\epsilon_2)|\prod\limits_{l=1}^L
1699: |\vd_l(\epsilon_1,\epsilon_2)+\vk_z(\epsilon_1,\epsilon_2)|},
1700: \end{equation}
1701: where $\vk_z(\epsilon_1,\epsilon_2) =
1702: \left[\vi-\va^*(\epsilon_1,\epsilon_2)\right]^{-1}-\vi$. We would
1703: like to let $\epsilon_1$ and $\epsilon_2$ approach zero and
1704: consider the limiting multiple description problem. Similar to
1705: equations~\eqref{eq:deps1} and~\eqref{eq:deps2}, when $\epsilon_1$
1706: and $\epsilon_2$ approach 0, we get
1707: \begin{equation}
1708: \begin{split}
1709: \vd_l(\epsilon_1,\epsilon_2) & \rightarrow \vd_l, \quad l=1,\;\dots,\;L-1, \\
1710: \vd_L(\epsilon_1,\epsilon_2) & \rightarrow \vd_L^*, \\
1711: \vd_0(\epsilon_1,\epsilon_2) & \rightarrow \vd_0^*,
1712: \end{split}
1713: \end{equation}
1714: where $\vd_L^* = \vd_L-\vl_2$ as in case 3 and $\vd_0^* =
1715: [\vd_0^{-1}+\vl_1^{-1}]^{-1}$ as in case 2. Further, we show that
1716: \begin{equation}\label{eq:case4e1e2}
1717: %R_l & \ge \frac{1}{2}\log \frac{1}{|\vd_l|} \\
1718: \lim_{\epsilon_2 \rightarrow 0}\lim_{\epsilon_1 \rightarrow 0}
1719: \frac{1}{2}\log\frac{|\vi+\vk_z(\epsilon_1,\epsilon_2)|^{(L-1)}|\vd_0(\epsilon_1,\epsilon_2)+
1720: \vk_z(\epsilon_1,\epsilon_2)|}{|\vd_0(\epsilon_1,\epsilon_2)|\prod\limits_{l=1}^L
1721: |\vd_l(\epsilon_1,\epsilon_2)+\vk_z(\epsilon_1,\epsilon_2)|}
1722: = \frac{1}{2}\log \frac{1}{|\vd_0|}
1723: \end{equation}
1724: in Appendix \ref{app:case4e1e2}. We conclude that the sum
1725: rate approaches
1726: \begin{equation}
1727: %\begin{split}
1728: %R_l & \ge \frac{1}{2}\log \frac{1}{|\vd_l|} \\
1729: \frac{1}{2}\log\frac{1}{|\vd_0|}
1730: %\end{split}
1731: \end{equation}
1732: as $\epsilon_1$ and $\epsilon_2$ approach 0.
1733: %Thus the corner point
1734: %\begin{equation}
1735: %\begin{split}
1736: %R_l &= \frac{1}{2}\log\frac{1}{|\vd_l|}, \; l=1,\cdots,L-1 \\
1737: %R_L &= \frac{1}{2}\log\frac{1}{|\vd_0|} -
1738: %\frac{1}{2}\log\frac{1}{|\vd_1|\dots|\vd_{L-1}|}
1739: %\end{split}
1740: %\end{equation}
1741: Thus the point-to-point rate-distortion function for central
1742: receiver with distortion $\vd_0$ can be achieved by using the jointly
1743: Gaussian description scheme, and the resulting distortions are
1744: $(\vd_1, \; \dots , \; \vd_L^*, \; \vd_0^*)$ where $\mathbf{0}
1745: \prec \vd_L^* \preccurlyeq \vd_L$ and $\mathbf{0} \prec \vd_0^*
1746: \preccurlyeq \vd_0$. In other words, the jointly Gaussian multiple description
1747: scheme is also optimal in this case.
1748:
1749: To summarize, we see that the jointly Gaussian description scheme
1750: achieves the limiting sum rate. The limiting sum rate is the
1751: solution to an optimization problem. For some specific distortion
1752: constraints, the sum rate can be characterized as the solution to a
1753: matrix polynomial equation (Case I).
1754:
1755: \subsection{Rate Region for Two Descriptions}
1756:
1757: Applying the result in Section \ref{sec:vecl} to the case of $L=2$,
1758: i.e., the two description problem, we can see that jointly
1759: Gaussian description scheme achieves the optimal sum rate. This
1760: resolves the case left out in Section \ref{sec:vector2}. It also turns
1761: out that in the two description problem, we can show that jointly
1762: Gaussian description strategy achieves the entire rate region.
1763: This is the main result of this subsection.
1764:
1765: From Section \ref{sec:outerbound} we have a outer bound to the
1766: rate region for the two description problem
1767: \begin{equation}
1768: \mathcal{R}_{out}(\vk_x,\;\vd_1,\;\vd_2,\;\vd_0) = \left\{
1769: \begin{array}{l}
1770: (R_1,\;R_2): \\
1771: \displaystyle R_l \ge \frac{1}{2}\log\frac{|\vk_x|}{|\vd_l|}, \quad l=1,\;2 \\
1772: \displaystyle R_1+R_2 \ge \sup_{\vk_z \succ \mathbf{0}}
1773: \frac{1}{2}\log\frac{|\vk_x||\vk_x+\mathbf{K}_z||\mathbf{D}_0+\mathbf{K}_z|}
1774: {|\mathbf{D}_0||\mathbf{D}_1+\mathbf{K}_z||\vd_2+\vk_z|}
1775: \end{array}
1776: \right\}.
1777: \end{equation}
1778:
1779: %From section \ref{sec:innerbound} we see that the rate pair
1780: %$(R_1,R_2) \in \mathcal{R}_{in}(\vk_x,\;\vd_1,\;\vd_2,\;\vd_0)$,
1781: %with $\mathcal{R}_{in}(\vk_x,\;\vd_1,\;\vd_2,\;\vd_0)$ defined as
1782: %\begin{equation}
1783: %\mathcal{R}_{in}(\vk_x,\;\vd_1,\;\vd_2,\;\vd_0) = \left\{
1784: %\begin{array}{l}
1785: %(R_1,\;R_2): \\
1786: %\displaystyle R_l \ge \frac{1}{2}\log\frac{|\vk_x+\vk^*_{w_l}|}{|\vk_{w_l}|}, \quad l=1,\;2 \\
1787: %\displaystyle R_1+R_2 \ge
1788: %\frac{1}{2}\log\frac{|\vk_x+\vk^*_{w_1}||\vk_x+\vk^*_{w_2}|}{|\vk_{w}|}
1789: %\end{array}
1790: %\right\},
1791: %\end{equation}
1792: %where
1793: %\[
1794: %\vk_w = \begin{pmatrix}
1795: % \vk^*_{w_1} & -\va^* \\
1796: % -\va^* & \vk^*_{w_2}
1797: % \end{pmatrix}.
1798: %\]
1799: %satisfying
1800: %\begin{equation}
1801: %\begin{split}
1802: %[\vk_{w_l}^{*-1} + \vk_x^{-1}]^{-1} & \preccurlyeq \vd_l^{-1},
1803: %\quad
1804: %l=1,\;2, \\
1805: %[\vk_x^{-1}+ (\vi \; \vi)\vk_w^{-1}(\vi \; \vi)^t]^{-1} &
1806: %\preccurlyeq \vd_0^{-1}.
1807: %\end{split}
1808: %\end{equation}
1809:
1810: Following the discussion in Section \ref{sec:vecl}, we show in the
1811: following that the jointly Gaussian description strategy (jointly
1812: Gaussian multiple description schemes and the time sharing between
1813: them) achieves the outer bound to the rate region.
1814:
1815: Let
1816: \[
1817: \vk_{w_l} = (\vd_l^{-1} - \vk_x^{-1})^{-1}, \quad l=0, \; 1,\;2
1818: \]
1819: and
1820: \[
1821: F(\va) = \log|\vk_{w_0}+\va| -
1822: \log|\vk_{w_1}+\va|-\log|\vk_{w_2}+\va|.
1823: \]
1824: Now consider the optimization problem:
1825: \begin{equation}\label{eq:optimal_2}
1826: \max\limits_{\mathbf{0} \preccurlyeq \va \preccurlyeq
1827: \vk_x}\quad\quad F(\va).
1828: \end{equation}
1829: As in Section \ref{sec:vecl}, the optimal solution $\va^*$ falls
1830: into four cases.
1831:
1832: {\bf Case 1:} $\mathbf{0} \prec \mathbf{A}^* \prec \vk_x$. In
1833: this case, we know from Section \ref{sec:innerbound} that the rate
1834: pair $(R_1,R_2)$ satisfying
1835: \begin{equation}
1836: \left\{
1837: \begin{array}{l}
1838: (R_1,\;R_2): \\
1839: \displaystyle R_l \ge \frac{1}{2}\log\frac{|\vk_x+\vk_{w_l}|}{|\vk_{w_l}|}, \quad l=1,\;2 \\
1840: \displaystyle R_1+R_2 \ge
1841: \frac{1}{2}\log\frac{|\vk_x+\vk_{w_1}||\vk_x+\vk_{w_2}|}{|\vk_{w}|}
1842: \end{array}
1843: \right\}
1844: \end{equation}
1845: is achievable using the jointly Gaussian multiple description scheme
1846: with the covariance matrix of $\vw_1, \; \vw_2$ being
1847: \[
1848: \vk_w = \begin{pmatrix}
1849: \vk_{w_1} & -\va^* \\
1850: -\va^* & \vk_{w_2}
1851: \end{pmatrix}.
1852: \]
1853: Denoting the resulting distortions as $(\vd_1, \; \vd_2, \; \vd_0)$,
1854: we readily calculate
1855: \[
1856: \frac{1}{2}\log\frac{|\vk_x+\vk_{w_l}|}{|\vk_{w_l}|} =
1857: \frac{1}{2}\log|
1858: \vk_x||\vk_{w_l}^{-1}+\vk_x^{-1}| =
1859: \frac{1}{2}\log\frac{|\vk_x|}{|\vd_l|}
1860: \]
1861: for $l=1,\;2$. From the discussion in Section \ref{sec:vecl},
1862: we know that the lower bound to sum rate is achieved using this
1863: jointly Gaussian description scheme. Thus, in this case, the jointly
1864: Gaussian description scheme achieves the rate region. As an aside, we
1865: note in
1866: this case that, $\va^*$ satisfies
1867: \[
1868: \left[\vk^*_{w_0}+\va^*\right]^{-1} =
1869: \left[\vk_{w_1}+\va^*\right]^{-1}+\left[\vk_{w_2}+\va^*\right],
1870: \]
1871: and, from the discussion in Section \ref{sec:vector2}, that a
1872: sufficient condition for this case to happen is \eqref{eq:vdfor2}.
1873:
1874: {\bf Case 2:} $\mathbf{0} \preccurlyeq \mathbf{A}^* \prec \vk_x$.
1875: This case is similar to case 1: the jointly Gaussian description
1876: scheme with covariance matrix for $\vw_1, \; \vw_2$ being
1877: \[
1878: \vk_w = \begin{pmatrix}
1879: \vk_{w_1} & -\va^* \\
1880: -\va^* & \vk_{w_2}
1881: \end{pmatrix}.
1882: \]
1883: achieves the lower bound on the rate region. We note that in this case the
1884: resulting distortions are $(\vd_1, \; \vd_2, \; \vd_0^*)$, with
1885: $\vd_0^* \preccurlyeq \vd_0$. Further, we know from the discussion in
1886: \ref{sec:vector2}, that a sufficient condition for this
1887: case to happen is
1888: \[
1889: \vd_0^{-1} + \vk_x^{-1} - \vd_1^{-1}-\vd_2^{-1} \preccurlyeq
1890: \mathbf{0}.
1891: \]
1892:
1893: {\bf Case 3:} $\mathbf{0} \prec \mathbf{A}^* \preccurlyeq \vk_x$.
1894: In this case, we know from the discussion in Section \ref{sec:vecl}
1895: that for another two description problem with distortions
1896: $(\vd_1,\;\vd_2^*,\;\vd_0)$ such that $\vd_2^* \preccurlyeq
1897: \vd_2$, the jointly Gaussian description scheme with covariance
1898: matrix for $\vw_1, \; \vw_2$ being
1899: \[
1900: \vk_w = \begin{pmatrix}
1901: \vk_{w_1} & -\va^* \\
1902: -\va^* & \vk^*_{w_2}
1903: \end{pmatrix}
1904: \]
1905: achieves the lower bound to sum rate
1906: $\left(\frac{1}{2}\log\frac{|\vk_x|}{|\vd_0|}\right)$ to the
1907: original distortions $(\vd_1,\;\vd_2,\;\vd_0)$. We can see, from the
1908: contra-polymatroid structure of the achievable region of jointly
1909: Gaussian description scheme, that the corner point
1910: $$B_1 =
1911: \left(\frac{1}{2}\log\frac{|\vk_x|}{|\vd_1|},\frac{1}{2}\log\frac{|\vk_x|}{|\vd_0|}-\frac{1}{2}\log\frac{|\vk_x|}{|\vd_1|}\right)$$
1912: in Figure \ref{fig:region} is achievable by this jointly Gaussian
1913: description scheme.
1914:
1915:
1916: Now observe that the discussion in case 3 of Section
1917: \ref{sec:vecl} is symmetric with respect to the individual receivers.
1918: Thus, by
1919: exchanging the role of receiver 1 and receiver 2, we can achieve
1920: the other corner point
1921: $$B_2= \left(\frac{1}{2}\log\frac{|\vk_x|}{|\vd_0|}-\frac{1}{2}\log\frac{|\vk_x|}{|\vd_2|},\frac{1}{2}\log\frac{|\vk_x|}{|\vd_2|}\right)$$
1922: in Figure \ref{fig:region} by another appropriate jointly Gaussian description
1923: scheme. Finally, time sharing between these two jointly Gaussian
1924: multiple description schemes allows us to achieve the lower bound
1925: on the rate region. As an aside, we note, as a consequence of the
1926: discussion in Section \ref{sec:vector2}, that a sufficient
1927: condition for this case to happen is
1928: \[
1929: \vd_0+\vk_x - \vd_1 - \vd_2 \preccurlyeq \mathbf{0}.
1930: \]
1931:
1932: {\bf Case 4:} $\mathbf{0} \preccurlyeq \mathbf{A}^* \preccurlyeq
1933: \vk_x$. In this case, we know, from the discussion in Section
1934: \ref{sec:vecl}, that for another two description problem
1935: with distortions $(\vd_1,\;\vd_2^*,\;\vd^*_0)$ such that $\vd_2^*
1936: \preccurlyeq \vd_2$ and $\vd_0^* \preccurlyeq \vd_0$, the jointly
1937: Gaussian description scheme with covariance matrix for $\vw_1, \;
1938: \vw_2$ being
1939: \[
1940: \vk_w = \begin{pmatrix}
1941: \vk_{w_1} & -\va^* \\
1942: -\va^* & \vk^*_{w_2}
1943: \end{pmatrix}
1944: \]
1945: achieves the lower bound to sum rate
1946: $\left(\frac{1}{2}\log\frac{|\vk_x|}{|\vd_0|}\right)$ to the
1947: original distortions $(\vd_1,\;\vd_2,\;\vd_0)$. Using an argument
1948: entirely analogous to that applied that the jointly Gaussian
1949: description strategy achieves the rate region.
1950:
1951: To summarize: the jointly Gaussian description strategy
1952: achieves the rate region for the two description problem. For a class
1953: of distortion constraints, the corner points of the rate region can
1954: be characterized by solving a matrix polynomial equation, as already
1955: seen in Section \ref{sec:vector2}.
1956:
1957: \section{Discussions}
1958:
1959: Although multiple description for individual and central receivers
1960: is a special case of the most general multiple description
1961: problem, the solution to this problem sheds substantial insight
1962: to the issue-at-large. In this section, we discuss two instances
1963: of other multiple description problems that can be resolved using
1964: the insights developed so far. In particular, we discuss the
1965: problem of two descriptions with separate distortion constraints
1966: and the general multiple description problem for some special sets
1967: of distortion constraints.
1968:
1969: \subsection{Two Description with Separate Distortion Constraints}
1970: \label{sec71} The problem of two descriptions with separate
1971: distortion constraints is ilustrated in Figure
1972: \ref{fig:md_special}.
1973: %{\tt Perhaps repeat the figure 2 from section 1 here.}
1974: Suppose the vector Gaussian source $\vx[m]=(\vx_1[m],\vx_2[m])$,
1975: the dimension of $\vx_1[m]$ is $N_1$ and the dimension of
1976: $\vx_2[m]$ is $N_2$. This implies that the dimension of $\vx[m]$ is $N =
1977: N_1+N_2$. Let $\vk_x = \meane[\vx[m]^t\vx[m]]$, $\vk_{x_1} =
1978: \meane[\vx_1[m]^t\vx_1[m]]$, and $\vk_{x_2} =
1979: \meane[\vx_2[m]^t\vx_2[m]]$. There are two encoders at the source
1980: providing two descriptions of $\vx[m]$. There are three receivers:
1981: the individual receivers 1 and 2 are only interested in generating
1982: reproduction of $\vx_1[m]$ with mean square distortion constraint
1983: $\vd_1$ (an $N_1 \times N_1$ positive definite matrix) from
1984: description 1 and $\vx_2[m]$ with mean square distortion
1985: constraint $\vd_2$ (an $N_2 \times N_2$ positive definite matrix)
1986: from description 2, respectively. The central receiver uses both
1987: descriptions to generate a reproduction of $\vx[m]$ with the error covariance
1988: meeting a distortion constraint $\vd_0$ (an $N \times N$ positive definite
1989: matrix) from both descriptions.
1990:
1991: This situation is closely related to the two description problem and
1992: we can harness our results thus far to completely characterize the
1993: rate region of the problem at hand.
1994: \begin{theorem}
1995: The rate region of two description with separate distortion
1996: constraints is
1997: \begin{equation}\label{eq:innerboundseparate}
1998: \mathcal{R}(\vd_1,\;\vd_2,\;\vd_0) =
1999: %Conv\left(
2000: \bigcup_{\Upsilon(\vd'_1,\;\vd'_2)}
2001: \mathcal{R}_*(\vd'_1,\;\vd'_2,\;\vd_0),
2002: %\right),
2003: \end{equation}
2004: where $\Upsilon(\vd'_1,\;\vd'_2)$ is defined as
2005: \begin{equation}
2006: \Upsilon(\vd'_1,\;\vd'_2) \overset{\text{def}}{=} \left\{
2007: \begin{array}{l}
2008: (\vd'_1,\;\vd'_2): \; (\vd'_1)_{\{1, \dots,N_1\}} \preccurlyeq
2009: \vd_1, \; (\vd'_2)_{\{N_1+1, \dots,N\}} \preccurlyeq \vd_2
2010: \end{array}
2011: \right\}.
2012: \end{equation}
2013: \end{theorem}
2014: \begin{proof}
2015: It is clear that any rate pair $(R_1,R_2) \in
2016: \mathcal{R}_*(\vd'_1,\;\vd'_2,\;\vd_0)$ for some
2017: $(\vd'_1,\;\vd'_2) \in \Upsilon(\vd'_1,\;\vd'_2)$ is in the rate
2018: region for the two description with separate distortion
2019: constraints, and so
2020: \[ \mathcal{R}_*(\vd'_1,\;\vd'_2,\;\vd_0)
2021: \subseteq \mathcal{R}(\vd_1,\;\vd_2,\;\vd_0).
2022: \]
2023: On the other hand, although receiver 1 (2) is only interested in
2024: reconstructing $\vx_1$ ($\vx_2$), they can actually reconstruct the
2025: entire source
2026: $\vx$ based on their received descriptions. Hence, any coding
2027: scheme for the two description with separate distortion
2028: constraints will result in some achievable distortions
2029: $(\vd'_1,\;\vd'_2,\;\vd'_0)$ with $(\vd'_1,\;\vd'_2) \in
2030: \Upsilon(\vd'_1,\;\vd'_2)$ and $\vd_0' \preccurlyeq \vd_0$. Thus
2031: any rate pair $(R_1,R_2) \in \mathcal{R}(\vd_1,\;\vd_2,\;\vd_0)$
2032: achieved by this coding scheme is in the rate region
2033: $\mathcal{R}_*(\vd'_1,\;\vd'_2,\;\vd_0)$ for the two description
2034: problem. Thus
2035: \[
2036: \mathcal{R}(\vd_1,\;\vd_2,\;\vd_0) \subseteq
2037: %Conv\left(
2038: \bigcup_{\Upsilon(\vd'_1,\;\vd'_2)}
2039: \mathcal{R}_*(\vd'_1,\;\vd'_2,\;\vd_0).
2040: %\right),
2041: \]
2042: From equivalence of the two regions in
2043: \eqref{eq:innerboundseparate}, the proof is now complete.
2044: \end{proof}
2045:
2046: \subsection{General Gaussian Multiple Description Problem for
2047: Special Choices of Distortion Constraints}
2048: \label{sec72}
2049: Consider the general Gaussian multiple description problem with
2050: source covariance $\vk_x$ and $2^L-1$ distortion constraints $D_S$
2051: for each $S \subseteq \{1, \; \dots, \; L\}$.
2052:
2053: Following arguments similar to that used in arriving at the lower bound
2054: \eqref{eq:outer} for sum rate, we have an outer bound on the rate
2055: region:
2056: \begin{equation}\label{eq:generalregion}
2057: \mathcal{R}_{out}(\vk_x, \; \vd_1,\;\dots,\;\vd_L,\;\vd_0) =
2058: \left\{
2059: \begin{array}{l}
2060: (R_1,\;\dots,\;R_L): \\
2061: \sum\limits_{l \in S}R_l \ge
2062: \frac{1}{2}\log\frac{|\vk_x||\vk_x+\mathbf{K}_z|^{(|S|-1)}|\mathbf{D}_S+\mathbf{K}_z|}
2063: {|\mathbf{D}_S|\prod\limits_{l \in S}|\mathbf{D}_l+\mathbf{K}_z|},
2064: \quad \forall S \subseteq \{1, \; \dots, \; L\}
2065: \end{array}
2066: \right\}.
2067: \end{equation}
2068: Following arguments similar to those used in arriving at the
2069: upper bound \eqref{eq:suminnerbound25} for the sum rate, we can use a
2070: jointly Gaussian description scheme with covariance matrix of
2071: $\vw_l$'s ($\vk_w$) taking the form \eqref{eq:kw},
2072: any tuple $(R_1,\;\dots,\;R_L)$ satisfying
2073: \begin{equation}
2074: \left\{
2075: \begin{array}{l}
2076: (R_1,\;\dots,\;R_L): \\
2077: \sum\limits_{l \in S}R_l \ge
2078: \frac{1}{2}\log\frac{\Big|\vk_x\Big|\Big|\vk_x+\mathbf{K}_z\Big|^{(|S|-1)}\Big|\cov[\vx|\vu_l,\;l
2079: \in S]+\mathbf{K}_z\Big|} {\Big|\cov[\vx|\vu_l,\;l \in
2080: S]\Big|\prod\limits_{l \in
2081: S}\Big|\cov[\vx|\vu_l]+\mathbf{K}_z\Big|},\quad \forall S
2082: \subseteq \{1, \; \dots, \; L\}
2083: \end{array}
2084: \right\}
2085: \end{equation}
2086: is achievable. Thus if we can find a $\vk_w$ of the form in \eqref{eq:kw}
2087: such that all of the $2^L-1$ distortion constraints are met with
2088: equality, i.e.,
2089: \begin{equation}
2090: \vd_S = \cov[\vx|\vu_l,\;l \in S] = [\vk_x^{-1} + (\vi,\; \dots,
2091: \; \vi)\mathbf{K}_{w_S}^{-1}(\vi,\; \dots, \; \vi)^t]^{-1} , \quad
2092: \forall S \subseteq \{1, \; \dots, \; L\},
2093: \end{equation}
2094: where $\vk_{w_S}$ is the covariance matrix for all $\vk_{w_l}, l
2095: \in S$, then the achievable region matches the outer bound and we
2096: would have characterized the rate region of the multiple
2097: description problem.
2098:
2099:
2100: From the above discussion, we see that for some choice of
2101: distortion constraints of the multiple description problem, we can
2102: indeed do this: First choose $L+1$ distortions
2103: $(\vd_1,\;\vd_2,\;\dots,\;\vd_L,\;\vd_0)$ such that they satisfy
2104: the condition for Theorem \ref{th:main} for the multiple
2105: description problem with individual and central receivers. Next we
2106: can solve for the $\vk_w$ which is the covariance matrix of
2107: $(\vw_1, \; \dots, \; \vw_L)$ for the sum-rate-achieving jointly
2108: Gaussian description scheme. For any other $S \subseteq \{1, \;
2109: \dots, \; L\}$, this scheme results in distortion $\vd_S =
2110: [\vk_x^{-1} + (\vi,\; \dots, \; \vi)\mathbf{K}_{w_S}^{-1}(\vi,\;
2111: \dots, \; \vi)^t]^{-1}$. Finally we choose these $\vd_S$'s as the
2112: other distortion constraints. Now we have a general multiple
2113: description problem with $2^L-1$ distortion constraints $D_S$ for
2114: each $S \subseteq \{1, \; \dots, \; L\}$, and hence we can find a
2115: $\vk_w$ of form \eqref{eq:kw} such that all of the $2^L-1$
2116: distortion constraints are met with equality. Thus
2117: \eqref{eq:generalregion} is actually the rate region and it can be
2118: achieved by a jointly Gaussian description scheme.
2119:
2120: %However, this is a very
2121: %special case and in general this Gaussian strategy can not achieve
2122: %the rate region for arbitrarily given distortion constraints.
2123: %{\tt How do you know that your outer bound is not tight in general? Do
2124: % you have a counterexample? If so, it will be nice to include it.}
2125:
2126:
2127: %%\section{Conclusion}
2128: %%
2129: %%We have presented the sum rate for vector Gaussian multiple
2130: %%description with individual and central receivers. In the case of
2131: %%two description problem we have characterized the rate region. We
2132: %%showed the connection between the problem of multiple description
2133: %%with individual and central receivers and the problem of multiple
2134: %%description with separate distortion constraints. We also obtained
2135: %%the rate region for the general Gaussian multiple description
2136: %%problem for some specially chosen distortion constraints.
2137:
2138: %\section{Conclusion}
2139: %In this paper, we study the MD problem for a Gaussian source when
2140: %only the individual descriptions and central description are of
2141: %interest.
2142: \vspace{1cm}
2143:
2144: \appendix
2145: \noindent{\Large{\bf{Appendix}}}
2146:
2147: \section{Useful Matrix Lemmas}\label{app:matrix}
2148:
2149: In this appendix we provide some useful results in matrix analysis
2150: that are extensively used in this paper.
2151:
2152: \begin{lemma}[Matrix Inversion Lemma]\cite[Theorem 2.5]{Zhang99}
2153: Let $\va$ be an $m \times m$ nonsingular matrix and $\mathbf{B}$
2154: be an $n \times n$ nonsingular matrix and let $\mathbf{C}$ and
2155: $\mathbf{D}$ be $m \times n$ and $n \times m$ matrices,
2156: respectively. If the matrix $\va+\mathbf{CBD}$ is nonsingular,
2157: then
2158: \[
2159: (\va+\mathbf{CBD})^{-1} =
2160: \va^{-1}-\va^{-1}\mathbf{C}(\mathbf{B}^{-1}+
2161: \mathbf{D}\mathbf{A}^{-1}\mathbf{C})^{-1}\mathbf{D}\va^{-1}
2162: \]
2163: \end{lemma}
2164: %{\tt What is this notation $\mathbb{M}_n$?}
2165:
2166:
2167: \begin{lemma}\label{lemma:blockinverse}\cite[Theorem 2.3]{Zhang99}
2168: Suppose that the partitioned matrix
2169: \[
2170: \mathbf{M} =
2171: \begin{pmatrix}
2172: \mathbf{A} & \mathbf{B} \\
2173: \mathbf{C} & \mathbf{D}
2174: \end{pmatrix}
2175: \]
2176: is invertible and that the inverse is conformally partitioned as
2177: \[
2178: \mathbf{M}^{-1} =
2179: \begin{pmatrix}
2180: \mathbf{X} & \mathbf{Y} \\
2181: \mathbf{U} & \mathbf{V}
2182: \end{pmatrix}.
2183: \]
2184: If $\va$ is a nonsingular principal sub-matrix of $\mathbf{M}$,
2185: then
2186: \begin{equation}
2187: \begin{split}
2188: \mathbf{X} = & \va^{-1} +
2189: \va^{-1}\mathbf{B}(\vd-\mathbf{C}\va^{-1}\mathbf{B})^{-1}\mathbf{C}\va^{-1},
2190: \\
2191: \mathbf{Y} = &
2192: -\va^{-1}\mathbf{B}(\vd-\mathbf{C}\va^{-1}\mathbf{B})^{-1}, \\
2193: \mathbf{U} = & -
2194: (\vd-\mathbf{C}\va^{-1}\mathbf{B})^{-1}\mathbf{C}\va^{-1}, \\
2195: \mathbf{V} = & (\vd-\mathbf{C}\va^{-1}\mathbf{B})^{-1}.
2196: \end{split}
2197: \end{equation}
2198: \end{lemma}
2199:
2200: \begin{lemma}\cite[Theorem 6.13]{Zhang99}
2201: Let $\mathbf{E} \in \mathbb{M}_n$ be a positive definite matrix
2202: and let $\mathbf{F}$ be an $n \times m$ matrix. Then for any $m
2203: \times m$ positive definite matrix $\mathbf{G}$,
2204: \begin{equation}
2205: \begin{pmatrix}
2206: \mathbf{E} & \mathbf{F} \\
2207: \mathbf{F}^t & \mathbf{G}
2208: \end{pmatrix}
2209: \succ 0 \Longleftrightarrow \mathbf{G} \succ
2210: \mathbf{F}^t\mathbf{E}^{-1}\mathbf{F}.
2211: \end{equation}
2212: \end{lemma}
2213:
2214: \begin{lemma}\cite[Theorem 6.8 and 6.9]{Zhang99}
2215: Let $\va$ and $\mathbf{B}$ be positive definite matrices such that
2216: $\va \succ \mathbf{B} \; (\va \succcurlyeq \mathbf{B})$. Then,
2217: \begin{equation}
2218: \begin{split}
2219: & |\va| \succ |\mathbf{B}| \quad ( |\va| \succcurlyeq
2220: |\mathbf{B}| ),
2221: \\
2222: & \va^{-1} \prec \mathbf{B}^{-1} \quad ( \va^{-1}
2223: \preccurlyeq
2224: \mathbf{B}^{-1}), \\
2225: & \va^{1/2} \succ \mathbf{B}^{1/2} \quad (\va^{1/2}
2226: \succcurlyeq \mathbf{B}^{1/2}).
2227: \end{split}
2228: \end{equation}
2229: \end{lemma}
2230:
2231: \section{Proof of Lemma \ref{lemma:inform}}\label{app:inform}
2232:
2233: Define an i.i.d.\ random process $\{\vz[m]\}$, $m=1,\; \dots,\; n$
2234: of $\mathcal{N}(0, \mathbf{K}_z)$ Gaussian random vectors, where
2235: $\vz[m]$, $m=1,\; \dots,\; n$ are independent of $\vx^n$ and
2236: $C_l$, $l=1,\; \dots,\; L$. Form a random process
2237: $\vy^n=(\vy[1],\;\dots,\;\vy[n])^t$ by
2238: \[
2239: \vy[m] = \vx[m]+\vz[m], \quad m=1,\; \dots,\; n.
2240: \]
2241: It follows that $\{\vy[m]\}$ is an i.i.d.\ random process
2242: %$m=1,\; \dots,\; n$
2243: of $\mathcal{N}(0, \mathbf{K}_y)$ Gaussian random vectors, where
2244: $\mathbf{K}_y = \vk_x + \mathbf{K}_z$. Then
2245: \begin{equation}\label{eq:inform}
2246: \begin{split}
2247: I(C_1;\;& C_2; \;\dots;\;C_L)+I(C_1,\;\dots,\;C_L; \vx^n) \\
2248: = & \sum\limits_{l=1}^LH(C_l)-H(C_1,\cdots,C_L) +
2249: I(C_1,\;\dots,\;C_L; \vx^n) \\
2250: \ge & \sum\limits_{l=1}^LH(C_l)-H(C_1,\cdots,C_L) +
2251: I(C_1,\;\dots,\;C_L; \vx^n) \\
2252: & - \Big(\sum\limits_{l=1}^LH(C_l|\vy^n)-H(C_1,\;\dots,\;C_L|\vy^n)\Big) \\
2253: = &
2254: \sum\limits_{l=1}^L\left(h(\vy^n)-h(\vy^n|C_l)\right)-h(\vy^n)
2255: +h(\vy^n|C_1,\;\dots,\;C_L) + h(\vx^n)-h(\vx^n|C_1,\;\dots,\;C_L) \\
2256: = & h(\vx^n)+(L-1)h(\vy^n)-\sum\limits_{l=1}^Lh(\vy^n|C_l)+
2257: h(\vy^n|C_1,\;\dots,\;C_L)-h(\vx^n|C_1,\;\dots,\;C_L).%\label{eq:rewrite_sum}
2258: \end{split}
2259: \end{equation}
2260: Since $\vx^n$ and $\vy^n$ are Gaussian vectors, for the first two
2261: terms in \eqref{eq:inform}, we have
2262: \begin{equation}\label{eq:99}
2263: \begin{split}
2264: h(\vx^n) = & \frac{1}{2}\log(2\pi e)^{Nn}|\vk_x|^n, \\
2265: h(\vy^n) = & \frac{1}{2}\log(2\pi e)^{Nn}|\vk_y|^n =
2266: \frac{1}{2}\log(2\pi e)^{Nn}|\vk_x+\vk_z|^n.
2267: \end{split}
2268: \end{equation}
2269: We also have the following bound on $h(\vy^n|C_l)$ for
2270: $l=1,\;\dots,\;L$:
2271: \begin{equation}\label{eq:100}
2272: \begin{split}
2273: h(\vy^n|C_l) & \le \sum\limits_{m=1}^nh(\vy[m]|C_l) \\
2274: & \le \sum\limits_{m=1}^n \frac{1}{2}\log(2\pi
2275: e)^N\big|\cov[\vy[m]|C_l]\big| \\
2276: & \le \frac{1}{2}\log(2\pi
2277: e)^{Nn}+\frac{n}{2}\log\left|\frac{1}{n}\sum\limits_{m=1}^n
2278: \cov[\vy[m]|C_l]\right| \\
2279: & = \frac{1}{2}\log(2\pi
2280: e)^{Nn}+\frac{n}{2}\log\left|\frac{1}{n}\sum\limits_{m=1}^n
2281: \cov[(\vx[m]+\vz[m])|C_l]\right| \\
2282: & = \frac{1}{2}\log(2\pi
2283: e)^{Nn}+\frac{n}{2}\log\left|\frac{1}{n}\sum\limits_{m=1}^n
2284: \cov[\vx[m]|C_l]+\vk_z\right| \\
2285: & \le \frac{1}{2}\log(2\pi
2286: e)^{Nn}+\frac{n}{2}\log\left|\vd_l+\vk_z\right| \\
2287: & = \frac{1}{2}\log(2\pi e)^{Nn}\left|\vd_l+\vk_z\right|^n.
2288: \end{split}
2289: \end{equation}
2290: Next we bound the last two terms of \eqref{eq:inform} as follows.
2291: \begin{equation}\label{eq:12}
2292: \begin{split}
2293: h(\vy^n|C_1,& \;\dots,\;C_L)-h(\vx^n|C_1,\;\dots,\;C_L) \\
2294: & =
2295: h(\vy^n|C_1,\;\dots,\;C_L)-h(\vx^n|\vz^n,\;C_1,\;\dots,\;C_L)\\
2296: & = h(\vy^n|C_1,\;\dots,\;C_L)-h(\vy^n|\vz^n,\;C_1,\;\dots,\;C_L)
2297: \\
2298: & = I(\vy^n;\vz^n|C_1,\;\dots,\;C_L).
2299: \end{split}
2300: \end{equation}
2301: %%Now we would like to think te that $I(\vy^n;\vz^n|C_1,\;\dots,\;C_L)$ is the capacity of a
2302: %%channel with input $\vz^n$ and additive noise
2303: %%$\meane[\vx^n|C_1,\;\dots,\;C_L]$.
2304: %{\tt This is not correct: you'll have to be more careful. See how
2305: % Bergmans' uses the EPI in the proof of scalar broadcast channel. You
2306: % need to first use for every realization of $C_1\ldots C_L$ and then
2307: % use a concavity argument to finally derive what you want.}
2308: %%Since there is a covariance
2309: %%constraint on the noise, the worst case noise for this channel is
2310: %%Gaussian \cite[Lemma II.2]{Diggavi01}.
2311: Letting
2312: \begin{equation}
2313: \vk_c[m] \stackrel{\rm def}{=} \cov[\vx[m]-\hat{\vx}_0[m]],\label{eq:covdefn}
2314: \end{equation}
2315: we have
2316: \begin{equation}\label{eq:awgnchannel}
2317: \begin{split}
2318: I(\vy^n;\vz^n|C_1,\;\dots,\;C_L) & =
2319: h(\vz^n|C_1,\;\dots,\;C_L) - h(\vz^n|\vy^n,C_1,\;\dots,\;C_L) \\
2320: & = h(\vz^n) - h(\vz^n|\vy^n-\hat{\vx}_0^n,C_1,\;\dots,\;C_L) \\
2321: & \ge h(\vz^n) - h(\vz^n|\vy^n-\hat{\vx}_0^n) \\
2322: & = \sum\limits_{m=1}^n
2323: \big(h(\vz[m])-h(\vz[m]|\vz[1],\;\dots,\;\vz[m-1],\vy^n-\hat{\vx}_0^n)\big)
2324: \\
2325: & \ge \sum\limits_{m=1}^n
2326: \big(h(\vz[m])-h(\vz[m]|\vy[m]-\hat{\vx}_0[m])\big) \\
2327: & = \sum\limits_{m=1}^nI(\vz[m];\vx[m]-\hat{\vx}_0[m]+\vz[m]) \\
2328: & \overset{(a)}{\ge}
2329: \sum\limits_{m=1}^n\frac{1}{2}\log\frac{|\vk_c[m]+\vk_z[m]|}{|\vk_c[m]|}
2330: \\ &
2331: \overset{(b)}{\ge}\frac{n}{2}\log\frac{|\vd_0+\vk_z|}{|\vd_0|},
2332: \end{split}
2333: \end{equation}
2334: where (a) is from \eqref{eq:covdefn} and \cite[Lemma II.2]{Diggavi01}.
2335: The justfication for (b) is from
2336: the convexity of
2337: $\log\frac{|\mathbf{A}+\mathbf{B}|}{|\mathbf{B}|}$ in $\mathbf{A}$
2338: and \eqref{eq:lemmacov}. From \eqref{eq:12} and
2339: \eqref{eq:awgnchannel} we have
2340: \begin{equation}\label{eq:103}
2341: h(\vy^n|C_1,\;\dots,\;C_L)-h(\vx^n|C_1,\;\dots,\;C_L) \ge
2342: \frac{n}{2}\log\frac{|\vd_0+\vk_z|}{|\vd_0|}.
2343: \end{equation}
2344: Combining \eqref{eq:inform}, \eqref{eq:99} and \eqref{eq:103}, we
2345: have
2346: \begin{equation}\label{eq:104}
2347: I(C_1;\; C_2; \;\dots;\;C_L)+I(C_1,\;\dots,\;C_L; \vx^n) \ge
2348: \frac{n}{2}\log\frac{|\vk_x||\vk_x+\mathbf{K}_z|^{(L-1)}|\mathbf{D}_0+\mathbf{K}_z|}
2349: {|\mathbf{D}_0|\prod\limits_{l=1}^L|\mathbf{D}_l+\mathbf{K}_z|}.
2350: \end{equation}
2351: Taking the supremum over all positive definite
2352: $\mathbf{K}_z$, we can sharpen the lower bound in \eqref{eq:104}:
2353: \begin{equation}
2354: \sum\limits_{l=1}^LI(C_1;C_2;\dots;C_L)+ I(C_1,\;\dots,\;C_L;
2355: \vx^n) \ge \sup_{\vk_z \succ \mathbf{0}}
2356: \frac{n}{2}\log\frac{|\vk_x||\vk_x+\mathbf{K}_z|^{(L-1)}|\mathbf{D}_0+\mathbf{K}_z|}
2357: {|\mathbf{D}_0|\prod\limits_{l=1}^L|\mathbf{D}_l+\mathbf{K}_z|}.
2358: \end{equation}
2359:
2360:
2361: \section{Proof of Proposition~\ref{prop:nec_suff_condition}}
2362: \label{app:nec_suff_condition}
2363: Conditioned on $\vy$, the collection of random variables $(\vu_1,\;\dots, \; \vu_L)$ are
2364: jointly Gaussian and thus we have
2365: \begin{equation}
2366: \sum\limits_{l=1}^Lh(\vu_l|\vy)-h(\vu_1,\;\dots,\;\vu_L|\vy) =
2367: \frac{1}{2}\log\frac{\prod\limits_{l=1}^L
2368: |\cov[\vu_l|\vy]|}{\big|\cov[\vu_1,\;\dots,\;\vu_L|\vy]\big|}.
2369: \end{equation}
2370: From MMSE of $\vu_l$ from $\vy$ we have
2371: \begin{equation}
2372: \cov[\vu_l|\vy] =
2373: \vk_x+\mathbf{K}_{w_l}-\vk_x(\vk_x+\mathbf{K}_z)^{-1}\vk_x, \quad
2374: l=1,\;\dots,\; L
2375: \end{equation}
2376: and
2377: \begin{equation}
2378: \cov(\vu_1,\;\dots,\;\vu_L|\vy)
2379: =
2380: \mathbf{J}\otimes \vk_x + \vk_w
2381: - \mathbf{J}\otimes \left(\vk_x(\vk_x+\mathbf{K}_z)^{-1}\vk_x\right),
2382: \end{equation}
2383: where $\mathbf{J}$ is an $L \times L $ matrix of all ones and
2384: $\otimes$ is the Kronecker Product.
2385:
2386:
2387: %\begin{equation}
2388: %\begin{split}
2389: % \cov(\vu_1,\;& \dots,\;\vu_L|\vy) \\
2390: % = &
2391: % \begin{pmatrix}
2392: % \vk_x+\mathbf{K}_{w_1} & \vk_x-\mathbf{A} & \vk_x-\mathbf{A} & \dots & \vk_x-\mathbf{A} \\
2393: % \vk_x-\va & \vk_x+\mathbf{K}_{w_2} & \vk_x-\va & \dots & \vk_x-\va \\
2394: % \hdotsfor{5} \\
2395: % \vk_x-\va & \dots & \vk_x-\va & \vk_x+\mathbf{K}_{w_{L-1}} & \vk_x-\va \\
2396: % \vk_x-\va & \dots & \vk_x-\va & \vk_x-\va & \vk_x+\mathbf{K}_{w_L}
2397: % \end{pmatrix} \\
2398: % & - \vk_x(\vk_x+\mathbf{K}_z)^{-1}\vk_x
2399: % \begin{pmatrix}
2400: % \vi & \vi & \vi & \dots & \vi \\
2401: % \vi & \vi & \vi & \dots & \vi \\
2402: % \hdotsfor{5} \\
2403: % \vi & \dots & \vi & \vi & \vi \\
2404: % \vi & \dots & \vi & \vi & \vi
2405: % \end{pmatrix}.
2406: %\end{split}
2407: %\end{equation}
2408: By Fischer inequality (the block matrix version of Hadamard
2409: inequality, see \cite[Theorem 6.10]{Zhang99}) we know that
2410: ${\prod\limits_{l=1}^L |\cov[\vu_l|\vy]|} =
2411: {\big|\cov[\vu_1,\;\dots,\;\vu_L|\vy]\big|}$ if and only if the
2412: off-diagonal block matrices of $\cov[\vu_1,\;\dots,\;\vu_L|\vy]$
2413: are all zero matrices. Thus we have
2414: \begin{equation*}
2415: \sum\limits_{l=1}^Lh(\vu_l|\vy)-h(\vu_1,\;\dots,\;\vu_L|\vy) = 0
2416: \end{equation*}
2417: if and only if
2418: \begin{equation}\label{eq:kz}
2419: \vk_x-\va = \vk_x(\vk_x+\vk_z)^{-1}\vk_x,
2420: \end{equation}
2421: or equivalently, if and only if
2422: \begin{equation}
2423: \mathbf{K}_z = \vk_x(\vk_x-\va)^{-1}\vk_x-\vk_x.
2424: \end{equation}
2425: To get a valid $\mathbf{K}_z \succ \mathbf{0}$, we need the
2426: additional condition $ \mathbf{0} \prec \va \prec \vk_x$.
2427:
2428: \section{Proof of Lemma \ref{lemma:inverse}}\label{app:inverse}
2429:
2430: First we assume $\va \succ 0$, and hence
2431: \begin{equation}
2432: \begin{split}
2433: & \left[\va^{-1}+\left(\vi \; \vi \; \dots \; \vi\right)\vk_w^{-1}\left(\vi \; \vi \; \dots \;
2434: \vi\right)^t \right]^{-1} \\
2435: = & \va - \va \left(\vi \; \vi \; \dots
2436: \vi\right)\left[\vk_w+\left(\vi \; \vi \; \dots \; \vi\right)^t
2437: \va \left(\vi \; \vi
2438: \; \dots \; \vi\right)\right]^{-1}\left(\vi \; \vi \; \dots \; \vi\right)^t \va \\
2439: = & \va- \va \left(\vi \; \vi \; \dots \vi\right) \Big[\diag \{\vk_{w_1}+\va,\;
2440: \vk_{w_2}+\va, \; \dots \; \vk_{w_L}+\va\} \Big]^{-1}\left(\vi \;
2441: \vi \;
2442: \dots \vi\right)^t \va \\
2443: = & \va - \va \sum\limits_{l=1}^L [\vk_{w_l}+\va]^{-1} \va. \\
2444: \end{split}
2445: \end{equation}
2446: Thus,
2447: \begin{equation}
2448: \begin{split}
2449: (\vi \; \vi \; & \dots \; \vi)\vk_w^{-1}\left(\vi \; \vi \; \dots
2450: \; \vi\right)^t \\
2451: = & \left[\va - \va \sum\limits_{l=1}^L (\vk_{w_l}+\va)^{-1}
2452: \va\right]^{-1}-\va^{-1} \\
2453: = & \va^{-1} - \va^{-1} \va \left[ -\left(\sum\limits_{l=1}^L
2454: (\vk_{w_l}+\va)^{-1}\right)^{-1} + \va \va^{-1}\va\right]^{-1}\va\va^{-1} - \va^{-1} \\
2455: = & \left[\left(\sum\limits_{l=1}^L
2456: (\vk_{w_l}+\va)^{-1}\right)^{-1}-\va\right]^{-1}.
2457: \end{split}
2458: \end{equation}
2459: When $\va$ is singular, we can choose $\delta >0$ such that
2460: $\va+\epsilon \vi \succ 0$ for $\epsilon \in (0,\delta)$, and thus
2461: we can apply the previous argument and let $\epsilon \rightarrow
2462: 0^+$ in the end.
2463:
2464:
2465: \section{Proof of Lemma
2466: \ref{lemma:positivedefinite}}\label{app:positivedefinite}
2467:
2468: We use induction. First consider the matrix
2469: \[
2470: \mathbf{\Delta}_2 = \begin{pmatrix}
2471: \vk_{w_1} & -\va \\
2472: -\va & \vk_{w_2}
2473: \end{pmatrix}.
2474: \]
2475: We have
2476: \begin{equation}
2477: \begin{split}
2478: \mathbf{\Delta}_2 \succ \mathbf{0} \; \Longleftrightarrow \;
2479: &\vk_{w_2} \succ \va\vk_{w_1}^{-1}\va \\
2480: \; \Longleftrightarrow \;
2481: & \vk_{w_2}+\va \succ
2482: \va\vk_{w_1}^{-1}\va+\va \\
2483: \; \Longleftrightarrow \; & (\vk_{w_2}+\va)^{-1} \prec
2484: (\va\vk_{w_1}^{-1}\va+\va)^{-1} \\
2485: \; \Longleftrightarrow \; &
2486: (\vk_{w_2}+\va)^{-1} \prec
2487: \va^{-1} - \left(\vk_{w_1}+\va\right)^{-1} \\
2488: \; \Longleftrightarrow \; &
2489: (\vk_{w_1}+\va)^{-1}+(\vk_{w_2}+\va)^{-1} \prec \va^{-1}
2490: \\
2491: \; {\Longleftarrow} \; &
2492: \sum\limits_{l=1}^L\left(\vk_{w_l}+\va\right)^{-1} \prec
2493: \va^{-1} \\
2494: \; \overset{(a)}{\Longleftrightarrow} \; & (\vk_{w_0}+\va)^{-1}
2495: \prec \va^{-1}
2496: \\
2497: \; \Longleftrightarrow \; & \vk_{w_0}+\va \succ \va \\
2498: \; \Longleftrightarrow \; & \vk_{w_0} \succ 0,
2499: \end{split}
2500: \end{equation}
2501: where (a) is from \eqref{eq:core}.
2502:
2503: Next we define
2504: \[
2505: \mathbf{\Delta}_k = \begin{pmatrix}
2506: \vk_{w_1} & -\va & -\va & \dots & -\va \\
2507: -\va & \vk_{w_2} & -\va & \dots & -\va \\
2508: \hdotsfor{5} \\
2509: -\va & \dots & -\va & \vk_{w_{k-1}} & -\va \\
2510: -\va & \dots & -\va & -\va & \vk_{w_k}
2511: \end{pmatrix}
2512: \]
2513: and suppose $\mathbf{\Delta}_k \succ 0$ for $k=3,\;\dots,\;l-1$.
2514: Then
2515: \begin{equation}
2516: \begin{split}
2517: \mathbf{\Delta}_l \succ 0 \; \Longleftrightarrow \; & \vk_{w_l}
2518: \succ
2519: \va(\vi,\;\vi,\;\dots,\;\vi)\Delta_{l-1}^{-1}(\vi,\;\vi,\;\dots,\;\vi)^t\va \\
2520: \; \Longleftrightarrow \; & \vk_{w_l} \succ
2521: \va\left[\left(\sum\limits_{k=1}^{l-1}(\vk_{W_k}+\va)^{-1}\right)^{-1}-\va\right]^{-1}\va
2522: \\
2523: \; \Longleftrightarrow \; & \vk_{w_l}+\va \succ
2524: \va\left[\left(\sum\limits_{k=1}^{l-1}(\vk_{w_k}+\va)^{-1}\right)^{-1}-\va\right]^{-1}\va
2525: +\va \\
2526: \; \Longleftrightarrow \; & (\vk_{w_l}+\va)^{-1} \prec
2527: \left[\va\left[\left(\sum\limits_{k=1}^{l-1}(\vk_{w_k}+\va)^{-1}\right)^{-1}-\va\right]^{-1}\va
2528: +\va\right]^{-1} \\
2529: \; \Longleftrightarrow \; & (\vk_{w_l}+\va)^{-1} \prec \va^{-1}
2530: -
2531: \left[\left(\sum\limits_{k=1}^{l-1}(\vk_{w_k}+\va)^{-1}\right)^{-1}-\va+\va\right]^{-1}
2532: \\
2533: \; \Longleftrightarrow \; & (\vk_{w_l}+\va)^{-1} \prec \va^{-1}
2534: -
2535: \sum\limits_{k=1}^{l-1}(\vk_{w_k}+\va)^{-1} \\
2536: \; {\Longleftarrow} \; & \sum\limits_{k=1}^{L}(\vk_{w_k}+\va)^{-1}
2537: \prec
2538: \va^{-1} \\
2539: \; \overset{(b)}{\Longleftrightarrow} \; & (\vk_{w_0}+\va)^{-1} \prec \va^{-1} \\
2540: \; \Longleftrightarrow \; & \vk_{w_0}+\va \succ \va \\
2541: \; \Longleftrightarrow \; & \vk_{w_0} \succ 0,
2542: \end{split}
2543: \end{equation}
2544: where (b) is from \eqref{eq:core}.
2545:
2546:
2547: \section{Proof of Proposition \ref{prop:twod}}\label{app:twod}
2548: First we prove that
2549: \[ \vd_0^{-1} + \vk_x^{-1} -
2550: \vd_1^{-1}-\vd_2^{-1} \succ \mathbf{0} \Rightarrow \va^* \succ
2551: \mathbf{0}.
2552: \]
2553: \begin{proof}
2554: We have
2555: \begin{equation}
2556: \va^* =
2557: (\vk_{w_1}-\vk_{w_0})^{\frac{1}{2}}\left[(\vk_{w_1}-\vk_{w_0})^{-\frac{1}{2}}
2558: (\vk_{w_2}-\vk_{w_0})(\vk_{w_1}-\vk_{w_0})^{-\frac{1}{2}}\right]^{\frac{1}{2}}(\vk_{w_1}-\vk_{w_0})^{\frac{1}{2}}
2559: -\vk_{w_0}.
2560: \end{equation}
2561: Thus
2562: \begin{equation}
2563: \begin{split}
2564: & \va^* \succ \mathbf{0} \\
2565: \; \Longleftrightarrow \; &
2566: (\vk_{w_1}-\vk_{w_0})^{\frac{1}{2}}\left[(\vk_{w_1}-\vk_{w_0})^{-\frac{1}{2}}
2567: (\vk_{w_2}-\vk_{w_0})(\vk_{w_1}-\vk_{w_0})^{-\frac{1}{2}}\right]^{\frac{1}{2}}(\vk_{w_1}-\vk_{w_0})^{\frac{1}{2}}
2568: \succ \vk_{w_0} \\
2569: \; \Longleftrightarrow \; &
2570: \left[(\vk_{w_1}-\vk_{w_0})^{-\frac{1}{2}}
2571: (\vk_{w_2}-\vk_{w_0})(\vk_{w_1}-\vk_{w_0})^{-\frac{1}{2}}\right]^{\frac{1}{2}}
2572: \succ
2573: (\vk_{w_1}-\vk_{w_0})^{-\frac{1}{2}}\vk_{w_0}(\vk_{w_1}-\vk_{w_0})^{-\frac{1}{2}}
2574: \\
2575: \; {\Longleftarrow} \; & (\vk_{w_1}-\vk_{w_0})^{-\frac{1}{2}}
2576: (\vk_{w_2}-\vk_{w_0})(\vk_{w_1}-\vk_{w_0})^{-\frac{1}{2}} \\
2577: & \quad \succ
2578: (\vk_{w_1}-\vk_{w_0})^{-\frac{1}{2}}\vk_{w_0}(\vk_{w_1}-\vk_{w_0})^{-1}\vk_{w_0}(\vk_{w_1}-\vk_{w_0})^{-\frac{1}{2}}
2579: \\
2580: \; \Longleftrightarrow \; & \vk_{w_2}-\vk_{w_0} \succ
2581: \vk_{w_0}\left(\vk_{w_1}-\vk_{w_0}\right)^{-1}\vk_{w_0} \\
2582: \; \Longleftrightarrow \; & \vk_{w_2}-\vk_{w_0} \succ
2583: \vk_{w_0}\left(-\vi +
2584: (\vk_{w_1}-\vk_{w_0})^{-1}\right)\vk_{w_1} \\
2585: \; \Longleftrightarrow \; & \vk_{w_2}-\vk_{w_0} \succ -\vk_{w_0}
2586: +
2587: \vk_{w_0}(\vk_{w_1}-\vk_{w_0})^{-1}\vk_{w_1} \\
2588: \; \Longleftrightarrow \; & \vk_{w_2} \succ
2589: \vk_{w_0}(\vk_{w_1}-\vk_{w_0})^{-1}\vk_{w_1} \\
2590: \; \Longleftrightarrow \; & \vk_{w_2} \succ
2591: \vk_{w_0}\vk_{w_0}^{-1}(\vk_{w_0}^{-1}-\vk_{w_1}^{-1})^{-1}\vk_{w_1}^{-1}\vk_{w_1}
2592: \\
2593: \; \Longleftrightarrow \; & \vk_{w_2} \succ
2594: (\vk_{w_0}^{-1}-\vk_{w_1}^{-1})^{-1} \\
2595: \; \Longleftrightarrow \; & \vk_{w_1}^{-1} + \vk_{w_2}^{-1}
2596: \prec
2597: \vk_{w_0}^{-1} \\
2598: \; \Longleftrightarrow \; & \vd_0^{-1} + \vk_x^{-1} -
2599: \vd_1^{-1}-\vd_2^{-1} \succ \mathbf{0}.
2600: \end{split}
2601: \end{equation}
2602: \end{proof}
2603: The proof of
2604: \[
2605: \vd_0 + \vk_x - \vd_1 - \vd_2 \succ \mathbf{0} \Rightarrow \va^*
2606: \prec \vk_x
2607: \]
2608: is similar and hence is omitted.
2609:
2610:
2611:
2612:
2613: \section{Proof of Lemma
2614: \ref{lemma:enhanced0}}\label{app:enhanced0}
2615:
2616: \begin{equation}
2617: \begin{split}
2618: \left[(\vk_{w_0}+\va^*)^{-1}+\vl_1\right]^{-1} & =
2619: \left[(\vk_{w_0}+\va^*)^{-1}(\vi+
2620: (\vk_{w_0}+\va^*)\vl_1)\right]^{-1} \\
2621: & \overset{(a)}{=} (\vi+\vk_{w_0}\vl_1)^{-1}(\vk_{w_0}+\va^*) \\
2622: & = (\vi+\vk_{w_0}\vl_1)^{-1}(\vk_{w_0}+\va^* -
2623: (\vi+\vk_{w_0}\vl_1)\va^*)+\va^* \\
2624: & \overset{(b)}{=} (\vi+\vk_{w_0}\vl_1)^{-1}\vk_{w_0}+\va^* \\
2625: & = \left(\vk_{w_0}^{-1}(\vi+\vk_{w_0}\vl_1)\right)^{-1} +\va^* \\
2626: & = \left(\vk_{w_0}^{-1}+\vl_1\right)^{-1} + \va^*,
2627: \end{split}
2628: \end{equation}
2629: where (a) and (b) are from $\vl_1 \va^* = \mathbf{0}$.
2630: \begin{equation}
2631: \begin{split}
2632: \frac{|\vd_0^*+\vk_z|}{|\vd_0^*|} & = |\vi+{\vd_0^*}^{-1}\vk_z| \\
2633: & = |\vi+(\vd_0^{-1} + \vl_1)\vk_z| \\
2634: & = |\vi+\vd_0^{-1}\vk_z + \vl_1\vk_z|\\
2635: & = |\vi+\vd_0^{-1}\vk_z+\vl_1\left((\vi-\va^*)^{-1}-\vi\right)| \\
2636: & \overset{(c)}{=} |\vi+\vd_0^{-1}\vk_z+\vl_1(\vi-\va^*)\left((\vi-\va^*)^{-1}-\vi\right)| \\
2637: & = |\vi+\vd_0^{-1}\vk_z| \\
2638: & = \frac{|\vd_0+\vk_z|}{|\vd_0|},
2639: \end{split}
2640: \end{equation}
2641: where (c) is from $\vl_1 \va^* = \mathbf{0}$.
2642:
2643: \section{Proof of Equations~\eqref{eq:deps1} and \eqref{eq:deps2}}\label{app:depsilon}
2644:
2645: We first prove the following lemma.
2646: \begin{lemma}\label{lemma:depsilon}
2647: Let $\vd$ be an $N \times N$ matrix such that $\mathbf{0} \prec
2648: \vd \prec \vi$. Let $\vk = (\vd^{-1}-\vi)^{-1}$. Choose $\epsilon
2649: >0$ such that $\vk - \epsilon \vi \succ \mathbf{0}$. Define
2650: \[
2651: \vd(\epsilon) \overset{\text{def}}{=} \left[(\vk-\epsilon
2652: \vi)^{-1}+\vi\right]^{-1}.
2653: \]
2654: Then, there exist constants $b_1 \ge b_2 >0$, such that
2655: \begin{equation*}
2656: \vd - b_1 \epsilon \vi +o(\epsilon) \prec \vd(\epsilon) \prec \vd
2657: - b_2 \epsilon \vi +o(\epsilon)
2658: \end{equation*}
2659: \end{lemma}
2660: \begin{proof}
2661: There exists an $N \times N$ orthogonal matrix $\vq$ such that
2662: \[ \vq \vk \vq^t = \diag\{k_1,\;\dots,\;k_N\}, \]
2663: where $k_i>0$ are eigenvalues of $\vk$. We have
2664: \begin{equation*}
2665: \begin{split}
2666: \vq \vd \vq^t & = \vq (\vk^{-1}+\vi)^{-1} \vq^t \\
2667: & =
2668: \diag\left\{\frac{k_1}{1+k_1},\;\dots,\;\frac{k_N}{1+k_N}\right\}, \\
2669: \end{split}
2670: \end{equation*}
2671: and
2672: \begin{equation*}
2673: \begin{split}
2674: \vq \vd(\epsilon) \vq^t & = \vq \Big[(\vk-\epsilon
2675: \vi)^{-1}+\vi\Big]^{-1} \vq^t \\
2676: & = \Big[(\diag\{k_1,\;\dots,\;k_N\}-\epsilon \vi)^{-1}+\vi\Big]^{-1} \\
2677: & =
2678: \diag\left\{\frac{k_1-\epsilon}{1+k_1-\epsilon},\;\dots,\;\frac{k_N-\epsilon}{1+k_N-\epsilon}\right\}\\
2679: & =
2680: \diag\left\{\frac{k_1}{1+k_1}-\frac{\epsilon}{(1+k_1)^2}+o(\epsilon
2681: ),\;\dots,\;\frac{k_N}{1+k_N}-\frac{\epsilon}{(1+k_N)^2}+o(\epsilon
2682: )\right\}.
2683: \end{split}
2684: \end{equation*}
2685: We now have
2686: \begin{equation*}
2687: \vq \vd \vq^t - b_1 \epsilon \vi +o(\epsilon) \prec \vq
2688: \vd(\epsilon) \vq^t \prec \vq \vd \vq^t - b_2 \epsilon \vi
2689: +o(\epsilon),
2690: \end{equation*}
2691: where $b_1 \ge b_2 > 0 $ are some constants. Hence
2692: \begin{equation*}
2693: \vd - b_1 \epsilon \vi +o(\epsilon ) \prec \vd(\epsilon) \prec
2694: \vd - b_2 \epsilon \vi +o(\epsilon ).
2695: \end{equation*}
2696: \end{proof}
2697: Equations \eqref{eq:deps1} and \eqref{eq:deps2} are a direct
2698: consequence of this lemma.
2699:
2700: \section{Proof of Equation~\eqref{eq:eps3}}\label{app:kzepsilon}
2701:
2702: We first prove the following lemma.
2703: \begin{lemma}\label{lemma:kzepsilon}
2704: Let $\va$ be an $N \times N$ matrix such that $\mathbf{0}
2705: \preccurlyeq \va \prec \vi$. Let $\vk_z = (\vi-\va)^{-1}-\vi$.
2706: Choose $\epsilon
2707: >0$ such that $\va + \epsilon \vi \prec \vi$. Define
2708: \[
2709: \vk_z(\epsilon) \overset{\text{def}}{=} \left[\vi-(\va + \epsilon
2710: \vi)\right]^{-1}-\vi.
2711: \]
2712: Then, there exist constants $c_1 \ge c_2 >0$ such that
2713: \begin{equation*}
2714: \vk_z - c_1 \epsilon \vi +o(\epsilon ) \prec \vk_z(\epsilon)
2715: \prec \vk_z - c_2 \epsilon \vi +o(\epsilon ).
2716: \end{equation*}
2717: \end{lemma}
2718: \begin{proof}
2719: There exists an $N \times N$ orthogonal matrix $\vq$ such that
2720: \[ \vq \va \vq^t = \diag\{a_1,\;\dots,\;a_N\} \]
2721: where $a_i>0$ are the eigenvalues of $\va$. We have
2722: \begin{equation*}
2723: \begin{split}
2724: \vq \vk_z \vq^t & = \vq ((\vi-\va)^{-1}-\vi) \vq^t \\
2725: & =
2726: \diag\left\{\frac{a_1}{1-a_1},\;\dots,\;\frac{a_N}{1-a_N}\right\}, \\
2727: \end{split}
2728: \end{equation*}
2729: and
2730: \begin{equation*}
2731: \begin{split}
2732: \vq \vk_z(\epsilon) \vq^t & = \vq ((\vi-(\va + \epsilon \vi))^{-1}-\vi) \vq^t \\
2733: & =
2734: \diag\left\{\frac{a_1+\epsilon}{1-a_1-\epsilon},\;\dots,\;\frac{a_N+\epsilon}{1-a_N-\epsilon}\right\}\\
2735: & =
2736: \diag\left\{\frac{a_1}{1-a_1}-\frac{(2a_1-1)\epsilon}{(1-a_1)^2}+o(\epsilon
2737: ),\;\dots,\;\frac{a_N}{1-a_N}-\frac{(2a_N-1)\epsilon}{(1-a_N)^2}+o(\epsilon
2738: )\right\}.
2739: \end{split}
2740: \end{equation*}
2741: We now have
2742: \begin{equation*}
2743: \vq \vk_z \vq^t - c_1 \epsilon \vi +o(\epsilon) \prec \vq
2744: \vk_z(\epsilon) \vq^t \prec \vq \vk_z \vq^t - c_2 \epsilon \vi
2745: +o(\epsilon),
2746: \end{equation*}
2747: where $c_1 \ge c_2> 0 $ are some constants. Hence
2748: \begin{equation*}
2749: \vk_z - c_1 \epsilon \vi +o(\epsilon ) \prec \vk_z(\epsilon)
2750: \prec \vk_z - c_2 \epsilon \vi +o(\epsilon).
2751: \end{equation*}
2752: \end{proof}
2753: Equation \eqref{eq:eps3} is a direct result of this lemma.
2754:
2755: \section{Proof of equation \eqref{eq:kzinflemma}}\label{app:kzinflemma}
2756:
2757: We first prove the following lemma.
2758: \begin{lemma}\label{lemma:kzinf}
2759: Let $\va$ be an $N \times N$ matrix such that $\mathbf{0} \prec
2760: \va \preccurlyeq \vi$. Choose $\epsilon
2761: >0$ such that $\va - \epsilon \vi \succ \vo$. Define
2762: \[
2763: \vk_z(\epsilon) \overset{\text{def}}{=} \left[\vi-(\va - \epsilon
2764: \vi)\right]^{-1}-\vi.
2765: \]
2766: Then, for any $\mathbf{E}$ and $\mathbf{F}$ such that $\vo \prec
2767: \mathbf{E} \preccurlyeq \vi$ and $\vo \prec \mathbf{F}
2768: \preccurlyeq \vi$, we have
2769: \begin{equation*}
2770: \lim_{\epsilon \rightarrow
2771: 0}\frac{|\mathbf{E}+\vk_z(\epsilon)|}{|\mathbf{F}+\vk_z(\epsilon)|}
2772: = 1.
2773: \end{equation*}
2774: \end{lemma}
2775: \begin{proof}
2776: There exists an $N \times N$ orthogonal matrix $\vq$ such that
2777: \[ \vq \va \vq^t = \diag\{a_1,\;\dots,\;a_N\}, \]
2778: where $0 < a_i \le 1$ are eigenvalues of $\va$. Without loss of
2779: generality, we suppose $a_1=1, \;\dots,\; a_p=1, \; a_{p+1}<1,
2780: \;\dots,\; a_N <1$.
2781:
2782: We have
2783: \begin{equation*}
2784: \begin{split}
2785: \vq \vk_z(\epsilon) \vq^t & = \vq ((\vi-(\va - \epsilon \vi))^{-1}-\vi) \vq^t \\
2786: & = \diag\left\{\frac{1-\epsilon}{\epsilon},\;\dots,\;
2787: \frac{1-\epsilon}{\epsilon}, \;
2788: \frac{a_{p+1}-\epsilon}{1-a_{p+1}+\epsilon}, \;
2789: \frac{a_N-\epsilon}{1-a_N+\epsilon}\right\},
2790: \end{split}
2791: \end{equation*}
2792: and since
2793: \begin{equation*}
2794: \frac{|\mathbf{I}+\vk_z(\epsilon)|}{|\vk_z(\epsilon)|} \ge
2795: \frac{|\mathbf{E}+\vk_z(\epsilon)|}{|\mathbf{F}+\vk_z(\epsilon)|}
2796: \ge \frac{|\vk_z(\epsilon)|}{|\mathbf{I}+\vk_z(\epsilon)|},
2797: \end{equation*}
2798: we have
2799: \begin{equation*}
2800: \lim_{\epsilon \rightarrow
2801: 0}\frac{|\mathbf{E}+\vk_z(\epsilon)|}{|\mathbf{F}+\vk_z(\epsilon)|}
2802: = 1.
2803: \end{equation*}
2804: \end{proof}
2805: Equation \eqref{eq:kzinflemma} is a direct consequence of this lemma.
2806:
2807: \section{Proof of Equation
2808: \eqref{eq:case4e1e2}}\label{app:case4e1e2}
2809:
2810: We would like to have a property similar to
2811: \eqref{eq:enhanced0_1}, as $\epsilon_1$ approaches zero, and a
2812: property similar to \eqref{eq:kzinflemma}, as $\epsilon_2$
2813: approaches zero. To see this is the case, we need the following
2814: lemma.
2815: \begin{lemma}
2816: \begin{equation*}
2817: \vl_1 \vk_z(\epsilon_1=0,\epsilon_2) = \mathbf{0}
2818: \end{equation*}
2819: \end{lemma}
2820: \begin{proof}
2821: Since
2822: \[ \vq \vl_1 \vq^t \vq \va^* \vq^t = \mathbf{0} \]
2823: and
2824: \begin{equation*}
2825: \begin{split}
2826: \vq \va^* \vq^t & =
2827: \diag(\underset{p}{\underbrace{0,\;\dots,\;0}},\underset{q}{\underbrace{\;1,\;\dots,\;1}},\;a_{p+q+1},\;\dots,\;a_s)
2828: \\
2829: \vq \va^* \vq^t - \epsilon_2 \mathbf{E}_2 & =
2830: \diag(\underset{p}{\underbrace{0,\;\dots,\;0}},\underset{q}{\underbrace{\;1-\epsilon_2,
2831: \;\dots,\;1-\epsilon_2}},\;a_{p+q+1},\;\dots,\;a_s),
2832: \end{split}
2833: \end{equation*}
2834: we have that
2835: \[
2836: \vq \va^* \vq^t (\vq \va^* \vq^t - \epsilon_2 \mathbf{E}_2) =
2837: \mathbf{0}.
2838: \]
2839: Thus
2840: \begin{equation*}
2841: \begin{split}
2842: \vq \vl_1 \vk_z(\epsilon_1=0,\epsilon_2)\vq^t & = \vq \vl_1 \vq^t
2843: \vq \left((\vi- \va^* + \vq^t
2844: \epsilon_2 \mathbf{E}_2 \vq )^{-1} - \vi\right) \vq^t \\
2845: & = \vq \vl_1 \vq^t \left((\vi - \vq \va^* \vq^t + \epsilon_2
2846: \mathbf{E}_2)^{-1} -\vi\right) \\
2847: & = \vq \vl_1 \vq^t (\vi - \vq \va^* \vq^t + \epsilon_2
2848: \mathbf{E}_2) \left((\vi - \vq \va^* \vq^t + \epsilon_2
2849: \mathbf{E}_2)^{-1} -\vi\right) \\
2850: & = \mathbf{0}.
2851: \end{split}
2852: \end{equation*}
2853: \end{proof}
2854: Using this lemma, we can show a property similar to
2855: \eqref{eq:enhanced0_1} as $\epsilon_1$ approaches zero. First note
2856: that similar to case 2, we have
2857: \begin{equation*}
2858: \vd_0^{-1}+\vl_1 - e_2 \epsilon_2 \vi +o(\epsilon_2) \prec
2859: \vd_0^{-1}(\epsilon_1, \epsilon_2) \prec \vd_0^{-1}+\vl_1 + e_1
2860: \epsilon_1 \vi +o(\epsilon_1)
2861: \end{equation*}
2862: where $e_1 >0$ and $e_2 > 0$ are constants. Hence we have
2863: \begin{equation*}
2864: \begin{split}
2865: \frac{|\vd_0(\epsilon_1=0, \epsilon_2)+\vk_z(\epsilon_1=0,
2866: \epsilon_2)|}{|\vd_0(\epsilon_1=0, \epsilon_2)|} & =
2867: |\vi+\vd^{-1}_0(\epsilon_1=0, \epsilon_2)\vk_z(\epsilon_1=0, \epsilon_2)| \\
2868: & \ge |\vi+(\vd_0^{-1} + \vl_1 - e_2 \epsilon_2
2869: \vi)\vk_z(\epsilon_1=0,
2870: \epsilon_2)|\\
2871: & = |\vi+\vd_0^{-1}\vk_z(\epsilon_1=0, \epsilon_2)-e_2
2872: \epsilon_2\vk_z(\epsilon_1=0, \epsilon_2)| \\
2873: & = \frac{|\vd_0+\vk_z(\epsilon_1=0, \epsilon_2)-e_2
2874: \epsilon_2\vd_0\vk_z(\epsilon_1=0, \epsilon_2)|}{|\vd_0|}.
2875: \end{split}
2876: \end{equation*}
2877: Similarly, we have
2878: \begin{equation*}
2879: \frac{|\vd_0(\epsilon_1=0, \epsilon_2)+\vk_z(\epsilon_1=0,
2880: \epsilon_2)|}{|\vd_0(\epsilon_1=0, \epsilon_2)|} \le
2881: \frac{|\vd_0+\vk_z(\epsilon_1=0, \epsilon_2)|}{|\vd_0|}.
2882: \end{equation*}
2883: Thus
2884: \begin{equation}
2885: \begin{split}
2886: \lim_{\epsilon_2 \rightarrow 0}\lim_{\epsilon_1 \rightarrow 0}
2887: & \frac{1}{2}\log\frac{|\vi+\vk_z(\epsilon_1,\epsilon_2)|^{(L-1)}|\vd_0(\epsilon_1,\epsilon_2)+
2888: \vk_z(\epsilon_1,\epsilon_2)|}{|\vd_0(\epsilon_1,\epsilon_2)|\prod\limits_{l=1}^L
2889: |\vd_l(\epsilon_1,\epsilon_2)+\vk_z(\epsilon_1,\epsilon_2)|} \\
2890: & = \lim_{\epsilon_2 \rightarrow 0}
2891: \frac{1}{2}\log\frac{|\vi+\vk_z(\epsilon_1=0,\epsilon_2)|^{(L-1)}|\vd_0+
2892: \vk_z(\epsilon_1=0,\epsilon_2)|}{|\vd_0|\prod\limits_{l=1}^L
2893: |\vd_l(\epsilon_1=0,\epsilon_2)+\vk_z(\epsilon_1=0,\epsilon_2)|} \\
2894: & = \frac{1}{2}\log \frac{1}{|\vd_0|},
2895: \end{split}
2896: \end{equation}
2897: where the last step is similar to \eqref{eq:kzinflemma}.
2898:
2899: \bibliographystyle{IEEEtran}
2900: %\bibliography{hua}
2901: \begin{thebibliography}{10}
2902: \providecommand{\url}[1]{#1} \csname url@rmstyle\endcsname
2903: \providecommand{\newblock}{\relax}
2904: \providecommand{\bibinfo}[2]{#2}
2905: \providecommand\BIBentrySTDinterwordspacing{\spaceskip=0pt\relax}
2906: \providecommand\BIBentryALTinterwordstretchfactor{4}
2907: \providecommand\BIBentryALTinterwordspacing{\spaceskip=\fontdimen2\font
2908: plus \BIBentryALTinterwordstretchfactor\fontdimen3\font minus
2909: \fontdimen4\font\relax}
2910: \providecommand\BIBforeignlanguage[2]{{%
2911: \expandafter\ifx\csname l@#1\endcsname\relax
2912: \typeout{** WARNING: IEEEtran.bst: No hyphenation pattern has been}%
2913: \typeout{** loaded for the language `#1'. Using the pattern for}%
2914: \typeout{** the default language instead.}%
2915: \else \language=\csname l@#1\endcsname \fi #2}}
2916:
2917: \bibitem{Ozarow80}
2918: L.~Ozarow, ``On a source-coding problem with two channels and
2919: three
2920: receivers,'' \emph{Bell Syst. Tech. J.}, vol.~59, no.~10, pp. 1909--1921,
2921: Dec. 1980.
2922:
2923: \bibitem{ElGamal82}
2924: A.~E. Gamal and T.~M. Cover, ``Achievable rates for multiple
2925: descriptions,''
2926: \emph{IEEE Trans. Inform. Theory}, vol.~28, no.~6, pp. 851--857, Nov. 1982.
2927:
2928: \bibitem{Ahlswede85}
2929: R.~Ahlswede, ``The rate-distortion region for multiple
2930: descriptions without
2931: excess rate,'' \emph{IEEE Trans. Inform. Theory}, vol.~31, no.~6, pp.
2932: 721--726, Nov. 1985.
2933:
2934: \bibitem{Zhang87}
2935: Z.~Zhang and T.~Berger, ``New results in binary multiple
2936: descriptions,''
2937: \emph{IEEE Trans. Inform. Theory}, vol.~33, no.~4, pp. 502--521, July 1987.
2938:
2939: \bibitem{Zamir99}
2940: R.~Zamir, ``Gaussian codes and shannon bounds for multiple
2941: descriptions,''
2942: \emph{IEEE Trans. Inform. Theory}, vol.~45, no.~7, pp. 2629--2635, Nov. 1999.
2943:
2944: \bibitem{FWFu02}
2945: F.~W. Fu and R.~W. Yeung, ``On the rate-distortion region for
2946: multiple
2947: descriptions,'' \emph{IEEE Trans. Inform. Theory}, vol.~48, no.~7, pp.
2948: 2012--2021, July 2002.
2949:
2950: \bibitem{Venkat03}
2951: R.~Venkataramani, G.~Kramer, and V.~K. Goyal, ``Multiple
2952: description coding
2953: with many channels,'' \emph{IEEE Trans. Inform. Theory}, vol.~49, no.~9, pp.
2954: 2106--2114, Sept. 2003.
2955:
2956: \bibitem{Pradhan04}
2957: S.~S. Pradhan, R.~Puri, and K.~Ramchandran, ``n-channel symmetric
2958: multiple
2959: descroption-part {I}: (n,k) source-channel erasure codes,'' \emph{IEEE Trans.
2960: Inform. Theory}, vol.~50, no.~1, pp. 47--61, Jan. 2004.
2961:
2962: \bibitem{HYFeng05}
2963: H.~Feng and M.~Effros, ``On the rate loss of multiple description
2964: source
2965: codes,'' \emph{IEEE Trans. Inform. Theory}, vol.~51, no.~2, pp. 671--683,
2966: Feb. 2005.
2967:
2968: \bibitem{Puri05}
2969: R.~Puri, S.~S. Pradhan, and K.~Ramchandran, ``n-channel symmetric
2970: multiple
2971: descroption-part {II}: an achievable rate-distortion region,'' \emph{IEEE
2972: Trans. Inform. Theory}, vol.~51, no.~4, pp. 1377--1392, Apr. 2005.
2973:
2974: \bibitem{Vaishampayan93}
2975: V.~A. Vaishampayan, ``Design of multiple description scalar
2976: quantizers,''
2977: \emph{IEEE Trans. Inform. Theory}, vol.~39, no.~3, pp. 821--834, May 1993.
2978:
2979: \bibitem{Vaishampayan94}
2980: V.~A. Vaishampayan and J.~Domaszewicz, ``Design of
2981: entropy-constrained
2982: multiple-description scalar quantizers,'' \emph{IEEE Trans. Inform. Theory},
2983: vol.~40, no.~1, pp. 245--250, Jan. 1994.
2984:
2985: \bibitem{Vaishampayan98}
2986: V.~A. Vaishampayan and J.~C. Batllo, ``Asymptotic analysis of
2987: multiple-description quantizers,'' \emph{IEEE Trans. Inform. Theory},
2988: vol.~44, no.~1, pp. 278--284, Jan. 1998.
2989:
2990: \bibitem{Vaishampayan01}
2991: V.~A. Vaishampayan, N.~Sloane, and S.~Servetto, ``Multiple
2992: description vector
2993: quantizers with lattice codebooks: design and analysis,'' \emph{IEEE Trans.
2994: Inform. Theory}, vol.~47, no.~4, pp. 1718--1734, July 2001.
2995:
2996: \bibitem{Goyal01}
2997: V.~K. Goyal, ``Multiple description coding: compression meets the
2998: network,''
2999: \emph{IEEE Signal Processing Mag.}, vol.~18, pp. 74--93, Sept. 2001.
3000:
3001: \bibitem{Diggavi02}
3002: S.~N. Diggavi, N.~J.~A. Sloane, and V.~A. Vaishampayan,
3003: ``Asymmetric multiple
3004: description lattice vector quantizers,'' \emph{IEEE Trans. Inform. Theory},
3005: vol.~48, no.~1, pp. 174--191, Jan. 2002.
3006:
3007: \bibitem{Goyal02}
3008: V.~K. Goyal, J.~A. Kelner, and J.~Kovacevic, ``Multiple
3009: description vector
3010: quantization with a coarse lattice,'' \emph{IEEE Trans. Inform. Theory},
3011: vol.~48, no.~3, pp. 781--788, Mar. 2002.
3012:
3013: \bibitem{CTian04}
3014: C.~Tian and S.~S. Hemami, ``Universal multiple description scalar
3015: quantizer:
3016: analysis and design,'' \emph{IEEE Trans. Inform. Theory}, vol.~50, no.~9, pp.
3017: 2737--2751, Sept. 2004.
3018:
3019: \bibitem{Ishwar03}
3020: P.~Ishwar, R.~Puri, S.~S. Pradhan, and K.~Ramchandran, ``On
3021: compression for
3022: robust estimation in sensor networks,'' in \emph{Proc. IEEE Int. Symp.
3023: Inform. Theory (ISIT)}, Yokohama, Japan, June-July 2003.
3024:
3025: \bibitem{JChen05}
3026: J.~Chen and T.~Berger, ``Robust distributed source coding,''
3027: submitted to {\em
3028: IEEE Trans. Inform. Theory}, 2005.
3029:
3030: \bibitem{Alon00}
3031: N.~Alon and J.~H. Spencer, \emph{The probabilistic Method, 2nd
3032: edition}.\hskip
3033: 1em plus 0.5em minus 0.4em\relax New York: Wiley, 2000.
3034:
3035: \bibitem{Welsh}
3036: D.~J.~A.~Welsh, {\em Matroid Theory}, Academic Press, London,
3037: 1976.
3038:
3039:
3040: \bibitem{Tse98}
3041: D.~Tse and S.~Hanly, ``Multi-access fading channels: part {I}:
3042: polymatroid
3043: structure,optimal resource allocation and throughput capacities,'' \emph{IEEE
3044: Trans. Inform. Theory}, vol.~44, no.~7, pp. 2796--2815, Nov. 1998.
3045:
3046: \bibitem{Viswanath04}
3047: P.~Viswanath, ``Sum rate of a class of gaussian multiterminal
3048: source coding
3049: problems,'' in \emph{Advances in Network Information Theory}, P. Gupta, G.
3050: Kramer and A. Wijngaarden editors, Rutgers, NJ, 2004, pp. 43--60.
3051:
3052: \bibitem{Zhang99}
3053: F.~Zhang, \emph{Matrix Theory: Basic Results and
3054: Techniques}.\hskip 1em plus
3055: 0.5em minus 0.4em\relax Springer, 1999.
3056:
3057: \bibitem{Diggavi01}
3058: S.~Diggavi and T.~M. Cover, ``Worst additive noise under
3059: covariance
3060: constraints,'' \emph{IEEE Trans. Inform. Theory}, vol.~47, no.~7, pp.
3061: 3072--3081, Nov. 2001.
3062:
3063:
3064: \end{thebibliography}
3065:
3066:
3067: %\end{thebibliography}
3068:
3069:
3070: \end{document}
3071: