1: \documentclass{article}
2: \usepackage[final]{graphicx}
3: \usepackage{geometry}
4: \usepackage{mathrsfs}
5: \usepackage{amssymb}
6: \usepackage{amsmath}
7:
8: \begin{document}
9:
10: \title{Encoding of Functions of Correlated Sources\thanks{This work
11: was supported in part by the National Competence Center in Research
12: on Mobile Information and Communication Systems (NCCR-MICS), a center
13: supported by the Swiss National Science Foundation under grant number
14: 5005-67322, and by a University of Florida Alumni Fellowship (2001-2005). }}
15: \author{Saravanan Vijayakumaran\thanks{The author is with the Laboratory
16: for Computer Communications and Applications, Ecole Polytechnique
17: F\'{e}d\'{e}rale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland.}}
18:
19: \maketitle
20: \abstract{In this correspondence, we describe the achievable rate region for
21: reliably recovering deterministic functions of correlated sources
22: which have a finite alphabet. The method of proof is almost the same as
23: that used to prove the Slepian-Wolf theorem.}
24: \section{Introduction}
25: Consider the problem of recovering a function $F(X,Y)$ of two
26: correlated sources $(X,Y)$ by encoding the sources separately (see
27: Fig.~\ref{fig:system}.) A problem of this class was first considered in
28: \cite{Korn79}, where the exact rate region for the modulo-two
29: adder source network was derived. In \cite{Han87}, necessary and
30: sufficient conditions were derived, for the achievable rate region
31: for recovering functions of correlated sources to coincide with
32: the Slepian-Wolf region \cite{Wolf73}.
33:
34: In this correspondence, we describe the achievable rate region for
35: reliably recovering deterministic functions of correlated sources
36: which have a finite alphabet. The method of proof is almost the same as
37: that used to prove the Slepian-Wolf theorem \cite{Wolf73}, \cite{Cove91}.
38:
39: \section{System Model}
40: The system model is essentially the same as the one described in \cite{Han87}. We repeat it here for
41: convenience and notational clarity.
42:
43: Let $X$ and $Y$ be a pair of correlated random variables defined on finite sample spaces $\mathscr{X}$
44: and $\mathscr{Y}$, respectively. Denote their joint probability distribution by
45: \begin{equation}
46: p_{XY}(x,y) = \Pr[X = x, Y = y], \hspace{10mm} x \in \mathscr{X}, y \in \mathscr{Y}.
47: \end{equation}
48: Conforming with the usual convention, we will use uppercase letters to denote random variables and lowercase letters
49: to denote fixed values the random variables may take. Let $(\mathbf{X},\mathbf{Y}) = (X^{n},Y^{n}) = ((X_1,Y_1),(X_2,Y_2),\ldots,(X_n,Y_n))$ be a sequence of
50: $n$ independent realizations of the pair of random variables $(X,Y)$. The distribution of $(\mathbf{X},\mathbf{Y})$ is given by
51: \begin{equation}
52: p_{XY}(\mathbf{x},\mathbf{y}) = \Pr[\mathbf{X} = \mathbf{x}, \mathbf{Y} = \mathbf{y}] = \prod_{i=1}^{n}p_{XY}(x_i,y_i), \hspace{10mm} \mathbf{x} \in \mathscr{X}^{n}, \mathbf{y} \in \mathscr{Y}^{n}
53: .
54: \end{equation}
55: The number of coordinates in $(\mathbf{X},\mathbf{Y})$ or $(\mathbf{x},\mathbf{y})$ will be clear from context.
56: \begin{figure}
57: \begin{center}
58: \includegraphics[width=0.85\textwidth]{fxysystem.eps}
59: \end{center}
60: \caption{Illustration of the system model.}
61: \label{fig:system}
62: \end{figure}
63:
64: Let $F:\mathscr{X} \times \mathscr{Y} \mapsto \mathscr{Z}$ be an arbitrary deterministic function.
65: We will denote the sequence $(F(X_1,Y_1),F(X_2,Y_2),\ldots,F(X_n,Y_n))$ by $F(\mathbf{X},\mathbf{Y})$.
66: We will sometimes find it convenient to denote the random variable $F(X,Y)$ by $Z$. Then $\mathbf{Z}=Z^{n}=F(\mathbf{X},\mathbf{Y})$.
67:
68: The sequence $(X_1,X_2,\ldots)$ is available at node $A$ and the sequence $(Y_1,Y_2,\ldots)$ is available at
69: node B. We wish to reliably recover the sequence $(Z_1,Z_2,\ldots)$ at node $C$, under the condition that
70: there is no communication between nodes $A$ and $B$. This situation is illustrated in Fig.~\ref{fig:system}. The channels from
71: node $A$ to node $B$ and node $A$ to node $C$ are assumed to be noiseless. So we have a distributed source
72: coding problem where the goal is to simultaneously minimize the required rates $R_1$ and $R_2$, which allow
73: reliable recovery of the sequence $(Z_1,Z_2,\ldots)$ at node $C$.
74:
75: We now present some definitions similar to ones presented in \cite[Section 14.4]{Cove91}.
76:
77: \textit{\textbf{Definition:}} A distributed source code $\mathscr{C}_{n}(F)$ for the random variable $F(X,Y)$
78: is a triplet of functions $(f_1,f_2,g)$,
79: \begin{eqnarray*}
80: &f_1&:\mathscr{X}^{n} \mapsto \{1,2,\ldots,2^{nR_1}\} \\
81: &f_2&:\mathscr{Y}^{n} \mapsto \{1,2,\ldots,2^{nR_2}\} \\
82: &g&:\{1,2,\ldots,2^{nR_1}\}\times \{1,2,\ldots,2^{nR_2}\} \mapsto \mathscr{Z}^{n} \\
83: \end{eqnarray*}
84: where $f_1,f_2$ correspond to the encoding functions and $g$ corresponds to the decoding function. Here $R_1,R_2$
85: are nonnegative real numbers and $n$ is a positive integer.
86:
87: \textit{\textbf{Definition:}} For a particular distributed source code $\mathscr{C}_{n}(F)$, the probability of
88: error is defined as
89: \begin{equation}
90: P_{e}^{(n)} = \Pr[g(f_1(\mathbf{X}),f_2(\mathbf{Y})) \neq F(\mathbf{X},\mathbf{Y})].
91: \end{equation}
92:
93: \textit{\textbf{Definition:}} A rate pair $(R_1,R_2)$ is said to be achievable for a function $F$ if there exists
94: a sequence of distributed source codes $\{\mathscr{C}_{n}(F): n \in \mathbb{N}\}$ with corresponding probabilities of
95: error $P_{e}^{(n)}$ such that $P_{e}^{(n)} \rightarrow 0$ as $n \rightarrow \infty$.
96:
97: \textit{\textbf{Definition:}} For a particular function $F$, the achievable rate region $\mathscr{R}(F)$ is the closure
98: of the set of all achievable rate pairs.
99:
100: \section{Main Result}
101: The following is the main result of this correspondence.
102:
103: \textit{\textbf{Theorem:}} The achievable rate region for a function $F$ of correlated random variables $(X,Y)$ is given by
104: \begin{equation*}
105: \mathscr{R}(F)=\left\{(R_1,R_2): R_1 \geq H(F(X,Y)|Y), R_2 \geq H(F(X,Y)|X), R_1+R_2 \geq H(F(X,Y))\right\}.
106: \end{equation*}
107:
108: The proof of this result is a simple application of the techniques used to prove the Slepian-Wolf theorem in \cite{Cove91}.
109: So we shamelessly adopt the conventions and notation in \cite[Chapter 14]{Cove91}, if not for any other reason but to illustrate
110: the simplicity of the proof. We need to borrow the following notation\footnote{See \cite[Section 14.2]{Cove91} for definitions and properties.}
111: before we proceed with the proof.
112:
113: Let $(U_1,U_2,\ldots,U_k)$ be a finite collection of discrete random variables with
114: a fixed joint distribution, $p(u_1,u_2,\ldots,u_n)$,
115: $(u_1,u_2,\ldots,u_n) \in \mathscr{U}_1 \times \mathscr{U}_2 \times \cdots \times \mathscr{U}_k$. The set of $\epsilon$-typical
116: $n$-sequences will be denoted by $A_{\epsilon}^{(n)}(U_1,U_2,\ldots,U_k)$. We will denote the set of $U_i$ $n$-sequences that are jointly
117: $\epsilon$-typical with a particular $U_j$ $n$-sequence, $\mathbf{u}_j$, by $A_{\epsilon}^{(n)}(U_i|\mathbf{u}_j)$.
118:
119:
120: \textit{\textbf{Proof of Achievability :}} For each $\mathbf{x} \in \mathscr{X}^{n}$, set $f_1(\mathbf{x})$ to a value chosen from the set
121: $\{1,2,\ldots,2^{nR_1}\}$ according to a uniform distribution. Similarly, for each $\mathbf{y} \in \mathscr{Y}^{n}$ set $f_2(\mathbf{y})$
122: to a value chosen from the set $\{1,2,\ldots,2^{nR_2}\}$ according to a uniform distribution. The encoding functions are revealed to the
123: corresponding encoder and the decoder, i.e., the decoder needs to know both $f_1$ and $f_2$ while encoder $i$ needs to know only $f_i$, $i=1,2$.
124:
125: The encoding operation consists of encoder $1$ and encoder $2$ sending the values of $f_1(\mathbf{X})$ and $f_2(\mathbf{Y})$, respectively,
126: to the decoder. Given the encoder outputs $(f_1(\mathbf{X}),f_2(\mathbf{Y}))=(i_0, j_0)$, the decoder outputs its estimate of $F(\mathbf{X},\mathbf{Y})$, $\hat{\mathbf{Z}}$,
127: to be $\mathbf{z}$ if there exists a unique sequence $\mathbf{z} \in \mathscr{Z}^{n}$ such that $(\mathbf{z},\mathbf{x},\mathbf{y}) \in A_{\epsilon}^{(n)}(Z,X,Y)$
128: for some $(\mathbf{x},\mathbf{y}) \in \mathscr{X}^{n} \times \mathscr{Y}^{n} $ such that $f_1(\mathbf{x})=i_0$ and $f_2(\mathbf{y})=j_0$. Note that
129: the pair $(\mathbf{x},\mathbf{y})$ need not be unique.
130:
131: The decoder operation is where the current coding scheme differs from Slepian-Wolf coding scheme. Of course, if $F$ is the identity function, i.e.,
132: $F(x,y)=(x,y), \forall (x,y) \in \mathscr{X} \times \mathscr{Y}$, then the above decoder coincides with the decoder in the Slepian-Wolf coding scheme.
133:
134: We now proceed with the analysis of the probability of error averaged over all possible encoder choices $f_1,f_2$. Let
135: $E=\{\hat{\mathbf{Z}} \neq \mathbf{Z}\}$ denote the decoding error event. Then we have $E = E_0 \cup E_1 \cup E_2 \cup E_{12}$ where
136: \begin{eqnarray*}
137: E_0 & =& \left\{\vphantom{\int} \exists \ \mbox{no} \ \mathbf{z} \in \mathscr{Z}^{n}: (\mathbf{z},\mathbf{x}^\prime,\mathbf{y}^\prime) \in A_{\epsilon}^{(n)}(Z,X,Y) \ \mbox{for some } (\mathbf{x}^\prime,\mathbf{y}^\prime) \ni f_1(\mathbf{x}^\prime) = f_1(\mathbf{X}), f_1(\mathbf{y}^\prime) = f_1(\mathbf{Y})\right\}, \\
138: E_1 & =& \left\{\vphantom{\int}\exists \mathbf{z} \in \mathscr{Z}^{n}:(\mathbf{z},\mathbf{x}^\prime,\mathbf{Y}) \in A_{\epsilon}^{(n)}(Z,X,Y) \ \mbox{for some } \mathbf{x}^\prime \ni f_1(\mathbf{x}^\prime) = f_1(\mathbf{X}), \mathbf{z} = F(\mathbf{x}^\prime,\mathbf{Y}) \neq F(\mathbf{X},\mathbf{Y})\right\}, \\
139: E_2 & =& \left\{\vphantom{\int}\exists \mathbf{z} \in \mathscr{Z}^{n}:(\mathbf{z},\mathbf{X},\mathbf{y}^\prime) \in A_{\epsilon}^{(n)}(Z,X,Y)\ \mbox{for some } \mathbf{y}^\prime \ni f_1(\mathbf{y}^\prime) = f_1(\mathbf{Y}), \mathbf{z} = F(\mathbf{X},\mathbf{y}^\prime) \neq F(\mathbf{X},\mathbf{Y})\right\}, \\
140: E_{12} & =& \left\{\vphantom{\int}\exists \mathbf{z} \in \mathscr{Z}^{n}:(\mathbf{z},\mathbf{x}^\prime,\mathbf{y}^\prime) \in A_{\epsilon}^{(n)}(Z,X,Y) \ \mbox{for some } (\mathbf{x}^\prime,\mathbf{y}^\prime) \ni f_1(\mathbf{x}^\prime) = f_1(\mathbf{X}), f_1(\mathbf{y}^\prime) = f_1(\mathbf{Y}),\right. \\
141: & & \hspace{105mm} \left. \vphantom{\int} \mathbf{z} = F(\mathbf{x}^\prime,\mathbf{y}^\prime) \neq F(\mathbf{X},\mathbf{Y})\right\}.
142: \end{eqnarray*}
143:
144: From the definition of jointly typical sequences \cite{Cove91}, it is easy to see that
145: \begin{equation}
146: \Pr[E_0] \leq \Pr[(\mathbf{z}, \mathbf{x}^\prime, \mathbf{y}^\prime) \in \mathscr{X}^{n} \times \mathscr{Y}^{n} \times \mathscr{Z}^{n}:(\mathbf{z},\mathbf{x}^\prime,\mathbf{y}^\prime) \notin A_{\epsilon}^{(n)}(Z,X,Y)] < \epsilon,
147: \end{equation}
148: for sufficiently large $n$.
149: We bound $\Pr[E_1]$ in the following manner.
150: \begin{eqnarray*}
151: \lefteqn{\Pr[E_1]} \\
152: & = & \Pr[\exists \mathbf{z} \in \mathscr{Z}^{n}:(\mathbf{z},\mathbf{x}^\prime,\mathbf{Y}) \in A_{\epsilon}^{(n)}(Z,X,Y) \ \mbox{for some } \mathbf{x}^\prime \ni f_1(\mathbf{x}^\prime) = f_1(\mathbf{X}), \mathbf{z} = F(\mathbf{x}^\prime,\mathbf{Y}) \neq F(\mathbf{X},\mathbf{Y})] \\
153: & \stackrel{(a)}{\leq} & \Pr[\exists \mathbf{z} \in \mathscr{Z}^{n}:(\mathbf{z},\mathbf{Y}) \in A_{\epsilon}^{(n)}(Z,Y), \ \mbox{for some } \mathbf{x}^\prime \ni f_1(\mathbf{x}^\prime) = f_1(\mathbf{X}), \mathbf{z} = F(\mathbf{x}^\prime,\mathbf{Y}) \neq F(\mathbf{X},\mathbf{Y})] \\
154: & = & \sum_{\mathbf{x},\mathbf{y}} p_{XY}(\mathbf{x},\mathbf{y})\Pr[\exists \mathbf{z} \in \mathscr{Z}^{n}:(\mathbf{z},\mathbf{y}) \in A_{\epsilon}^{(n)}(Z,Y) \ \mbox{for some } \mathbf{x}^\prime \ni f_1(\mathbf{x}^\prime) = f_1(\mathbf{x}), \mathbf{z} = F(\mathbf{x}^\prime,\mathbf{y}) \neq F(\mathbf{x},\mathbf{y})]\\
155: & \leq & \sum_{\mathbf{x},\mathbf{y}} p_{XY}(\mathbf{x},\mathbf{y})\Pr[(\mathbf{z},\mathbf{y}) \in A_{\epsilon}^{(n)}(Z,Y):\mbox{For some } \mathbf{x}^\prime \neq \mathbf{x}, f_1(\mathbf{x}^\prime) = f_1(\mathbf{x})]\\
156: & \stackrel{(b)}{\leq} & \sum_{\mathbf{x},\mathbf{y}} p_{XY}(\mathbf{x},\mathbf{y}) 2^{-nR_1}|A_{\epsilon}^{(n)}(Z|\mathbf{y})|\\
157: & \stackrel{(c)}{\leq} & 2^{-nR_1}2^{n(H(Z|Y)+2\epsilon)},
158: \end{eqnarray*}
159: where
160: \begin{itemize}
161: \item[(a)] follows from the fact that for any $(\mathbf{z}, \mathbf{x}^\prime, \mathbf{y}) \in \mathscr{Z}^{n} \times \mathscr{X}^{n} \times \mathscr{Y}^{n}$, $(\mathbf{z}, \mathbf{x}^\prime, \mathbf{y}) \in A_{\epsilon}^{(n)}(Z,X,Y)$ $ \Rightarrow (\mathbf{z}, \mathbf{y})
162: \in A_{\epsilon}^{(n)}(Z,Y)$,
163: \item[(b)] follows from the fact that we are averaging over all possible encoder choices for $f_1$ and the property that for a fixed $\mathbf{y} \in
164: \mathscr{Y}^{n}$, $|\{\mathbf{z} \in \mathscr{Z}^{n}:(\mathbf{z},\mathbf{y}) \in A_{\epsilon}^{(n)}(Z,Y)\}| = |A_{\epsilon}^{(n)}(Z|\mathbf{y})|$,
165: \item[(c)] follows from the fact that $|A_{\epsilon}^{(n)}(Z|\mathbf{y})| \leq 2^{n(H(Z|Y)+2\epsilon)}$ \cite[Theorem 14.2.2]{Cove91}.
166: \end{itemize}
167: The final bound on $\Pr[E_1]$ tends to zero as $n \rightarrow \infty$ if $R_1 > H(Z|Y) + 2\epsilon$.
168: Thus for sufficiently large $n$, $\Pr[E_1] < \epsilon$. Similarly, we can show that $\Pr[E_2] < \epsilon$ for sufficiently large $n$
169: if $R_2 > H(Z|X) + 2\epsilon$.
170:
171: Note that $E_1 \subset E_{12}$ and $E_2 \subset E_{12}$. It then follows that $E = E_0 \cup E_1 \cup E_2 \cup E_{12} = E_0 \cup E_1 \cup E_2 \cup (E_{12}\cap E_1^{c} \cap E_2^{c})$. We will find it easier to bound
172: $ E_{12}\cap E_1^{c} \cap E_2^{c}$ rather than bound $E_{12}$ directly. We bound $\Pr[E_{12}\cap E_1^{c} \cap E_2^{c}]$ in the following manner.
173: \begin{eqnarray*}
174: \lefteqn{\Pr[E_{12}\cap E_1^{c} \cap E_2^{c}]} \\
175: & = & \Pr[\exists \mathbf{z} \in \mathscr{Z}^{n}:(\mathbf{z}, \mathbf{x}^\prime, \mathbf{y}^\prime) \in A_{\epsilon}^{(n)}(Z,X,Y) \ \mbox{for some } \mathbf{x}^\prime \neq \mathbf{X},\mathbf{y}^\prime \neq \mathbf{Y} \ni f_1(\mathbf{x}^\prime) = f_1(\mathbf{X}), \\
176: & & \hspace{75mm} f_2(\mathbf{y}^\prime) = f_2(\mathbf{Y}), \mathbf{z} = F(\mathbf{x}^\prime,\mathbf{y}^\prime) \neq F(\mathbf{X},\mathbf{Y})] \\
177: & \stackrel{(a)}{\leq} & \Pr[\exists \mathbf{z} \in \mathscr{Z}^{n}:\mathbf{z} \in A_{\epsilon}^{(n)}(Z) \ \mbox{for some } \mathbf{x}^\prime \neq \mathbf{X},\mathbf{y}^\prime \neq \mathbf{Y} \ni f_1(\mathbf{x}^\prime) = f_1(\mathbf{X}), f_2(\mathbf{y}^\prime) = f_2(\mathbf{Y}),\\
178: & & \hspace{100mm} \mathbf{z} = F(\mathbf{x}^\prime,\mathbf{y}^\prime) \neq F(\mathbf{X},\mathbf{Y})] \\
179: & = & \sum_{\mathbf{x},\mathbf{y}} p_{XY}(\mathbf{x},\mathbf{y})\Pr[\exists \mathbf{z} \in \mathscr{Z}^{n}:\mathbf{z} \in A_{\epsilon}^{(n)}(Z) \ \mbox{for some } \mathbf{x}^\prime \neq \mathbf{x},\mathbf{y}^\prime \neq \mathbf{y} \ni f_1(\mathbf{x}^\prime) = f_1(\mathbf{x}), f_2(\mathbf{y}^\prime) = f_2(\mathbf{y}), \\
180: & & \hspace{100mm} \mathbf{z} = F(\mathbf{x}^\prime,\mathbf{y}^\prime) \neq F(\mathbf{x},\mathbf{y})] \\
181: & \leq & \sum_{\mathbf{x},\mathbf{y}} p_{XY}(\mathbf{x},\mathbf{y})\Pr[\mathbf{z} \in A_{\epsilon}^{(n)}(Z):\mbox{For some } \mathbf{x}^\prime \neq \mathbf{x},\mathbf{y}^\prime \neq \mathbf{y}, f_1(\mathbf{x}^\prime) = f_1(\mathbf{x}), f_2(\mathbf{y}^\prime) = f_2(\mathbf{y})] \\
182: & \stackrel{(b)}{\leq} & \sum_{\mathbf{x},\mathbf{y}} p_{XY}(\mathbf{x},\mathbf{y}) 2^{-nR_1}2^{-nR_2}|A_{\epsilon}^{(n)}(Z)|\\
183: & \stackrel{(c)}{\leq} & 2^{-n(R_1 +R_2)}2^{n(H(Z)+\epsilon)},
184: \end{eqnarray*}
185: where
186: \begin{itemize}
187: \item[(a)] follows from the fact that for any $(\mathbf{z}, \mathbf{x}^\prime, \mathbf{y}^\prime) \in \mathscr{Z}^{n} \times \mathscr{X}^{n} \times \mathscr{Y}^{n} $, $(\mathbf{z}, \mathbf{x}^\prime, \mathbf{y}^\prime) \in A_{\epsilon}^{(n)}(Z,X,Y)$ $ \Rightarrow \mathbf{z}
188: \in A_{\epsilon}^{(n)}(Z)$,
189: \item[(b)] follows from the fact that we are averaging over all possible encoder choices $f_1, f_2$ and from the definition of $A_{\epsilon}^{(n)}(Z)$,
190: \item[(c)] follows from the fact that $|A_{\epsilon}^{(n)}(Z)| \leq 2^{n(H(Z)+\epsilon)}$.
191: \end{itemize}
192: The final bound on $\Pr[E_{12}\cap E_1^{c} \cap E_2^{c}]$ can be made smaller than $\epsilon$ for sufficiently large $n$ if $R_1+R_2 > H(Z)+\epsilon$.
193:
194: Thus, we have $\Pr[E] \leq \Pr[E_0] + \Pr[E_1] + \Pr[E_2] + \Pr[E_{12}\cap E_1^{c} \cap E_2^{c}] < 4\epsilon$ for sufficiently large $n$. Since the
195: probability of error averaged over all codes is less than $4\epsilon$, there exists at least one code $\mathscr{C}_{n}^{*}(F)$ for which the average
196: probability of error is less than $4\epsilon$. Since $\epsilon$ was arbitrary, we can construct a sequence of codes such that $P_{e}^{(n)} \rightarrow 0$
197: as $n \rightarrow \infty$. The arbitrary choice of $\epsilon$ also implies that any rate pair $(R_1,R_2)$ satisfying $R_1 > H(F(X,Y)|Y),
198: R_2 > H(F(X,Y)|X), R_1+R_2 > H(F(X,Y))$ is achievable. Since the achievable rate region is the closure of all achievable rates, we have
199: \begin{equation*}
200: \mathscr{R}(F)\supset \left\{(R_1,R_2): R_1 \geq H(F(X,Y)|Y), R_2 \geq H(F(X,Y)|X), R_1+R_2 \geq H(F(X,Y))\right\}.
201: \end{equation*}
202: This completes the proof of the achievability.\hfill $\blacksquare$
203:
204: \textit{\textbf{Proof of Converse :}}
205: This proof is once again very similar to the proof of the converse to the Slepian-Wolf theorem \cite[Section 14.4.2]{Cove91}.
206:
207: Let $(R_1,R_2)$ be an achievable rate pair.
208: By definition, there exists a sequence of distributed source codes $\{\mathscr{C}_n(F): n \in \mathbb{N}\}$ and hence a sequence of function
209: triplets $\{(f_1^{(n)},f_2^{(n)},g^{(n)}):n \in \mathbb{N}\}$, with $P_{e}^{(n)} = \Pr[g(f_1(\mathbf{X}),f_2(\mathbf{Y})) \neq F(\mathbf{X},\mathbf{Y})]$
210: such that $P_{e}^{(n)} \rightarrow 0$ as $n \rightarrow \infty$.
211:
212: For notational convenience, define $I_0^{(n)} = f_1^{(n)}(\mathbf{X})$ and $J_0^{(n)} = f_2^{(n)}(\mathbf{Y})$. By Fano's inequality, we have
213: \begin{eqnarray}
214: H(F(\mathbf{X},\mathbf{Y})|I_0^{(n)},J_0^{(n)}) & \leq & P_{e}^{(n)}\log| \mathscr{Z}^{n}|+1 \nonumber \\
215: & = & P_{e}^{(n)}n\log| \mathscr{Z}|+1 = n\delta_n, \label{eqn:fano1}
216: \end{eqnarray}
217: where $\delta_n = P_{e}^{(n)}\log| \mathscr{Z}|$. We know that $\delta_n \rightarrow 0$ as $n \rightarrow \infty$. Since conditioning reduces entropy,
218: we also have
219: \begin{eqnarray}
220: H(F(\mathbf{X},\mathbf{Y})|\mathbf{Y},I_0^{(n)},J_0^{(n)}) & \leq & n\delta_n,\label{eqn:fano2} \\
221: H(F(\mathbf{X},\mathbf{Y})|\mathbf{X},I_0^{(n)},J_0^{(n)}) & \leq & n\delta_n,\label{eqn:fano3}
222: \end{eqnarray}
223:
224: Following the notation in \cite{Cove91}, we will write $U \rightarrow V \rightarrow W$ for some random variables $U,V,W$ to mean that
225: $U$ and $W$ are conditionally independent given $V$. For the problem under consideration, we have the following relations,
226: \begin{eqnarray*}
227: (I_0^{(n)},J_0^{(n)}) \rightarrow (\mathbf{X},\mathbf{Y}) \rightarrow F(\mathbf{X},\mathbf{Y}), \\
228: I_0^{(n)} \rightarrow (\mathbf{X},\mathbf{Y}) \rightarrow (F(\mathbf{X},\mathbf{Y}),\mathbf{Y}), \\
229: J_0^{(n)} \rightarrow (\mathbf{X},\mathbf{Y}) \rightarrow (F(\mathbf{X},\mathbf{Y}),\mathbf{X}).
230: \end{eqnarray*}
231: Application of the data processing inequality to each of the above relations and simple manipulations yield the following respective inequalities.
232: \begin{eqnarray}
233: H(I_0^{(n)},J_0^{(n)}|\mathbf{X},\mathbf{Y}) & \leq & H(I_0^{(n)},J_0^{(n)}|F(\mathbf{X},\mathbf{Y})) \label{eqn:data1}\\
234: H(I_0^{(n)}|\mathbf{X},\mathbf{Y}) & \leq & H(I_0^{(n)}|F(\mathbf{X},\mathbf{Y}),\mathbf{Y}) \label{eqn:data2}\\
235: H(J_0^{(n)}|\mathbf{X},\mathbf{Y}) & \leq & H(J_0^{(n)}|F(\mathbf{X},\mathbf{Y}),\mathbf{X})\label{eqn:data3}
236: \end{eqnarray}
237:
238: Then we have a chain of inequalities
239: \begin{eqnarray*}
240: n(R_1+R_2) & \geq & H(I_0^{(n)},J_0^{(n)}) = I(F(\mathbf{X},\mathbf{Y});I_0^{(n)},J_0^{(n)}) + H(I_0^{(n)},J_0^{(n)}|F(\mathbf{X},\mathbf{Y})) \\
241: &\stackrel{(a)}{\geq} & I(F(\mathbf{X},\mathbf{Y});I_0^{(n)},J_0^{(n)}) + H(I_0^{(n)},J_0^{(n)}|\mathbf{X},\mathbf{Y}) \\
242: &\stackrel{(b)}{=}& I(F(\mathbf{X},\mathbf{Y});I_0^{(n)},J_0^{(n)})\\
243: & = & H(F(\mathbf{X},\mathbf{Y})) - H(F(\mathbf{X},\mathbf{Y})|I_0^{(n)},J_0^{(n)})\\
244: &\stackrel{(c)}{\geq} & n H(F(X,Y)) - n\delta_n ,
245: \end{eqnarray*}
246: where
247: \begin{itemize}
248: \item[(a)] follows from (\ref{eqn:data1}),
249: \item[(b)] follows from the fact that $(I_0^{(n)},J_0^{(n)})$ is a function of $(\mathbf{X},\mathbf{Y})$,
250: \item[(c)] follows from the chain rule and the fact that $F(\mathbf{X},\mathbf{Y})$ consists of i.i.d. components, and from (\ref{eqn:fano1}).
251: \end{itemize}
252: Similarly, we can write
253: \begin{eqnarray*}
254: nR_1 & \geq & H(I_0^{(n)}) \geq H(I_0^{(n)}|\mathbf{Y}) \\
255: & = & I(F(\mathbf{X},\mathbf{Y});I_0^{(n)}|\mathbf{Y}) + H(I_0^{(n)}|F(\mathbf{X},\mathbf{Y}),\mathbf{Y}) \\
256: & \stackrel{(a)}{\geq} & I(F(\mathbf{X},\mathbf{Y});I_0^{(n)}|\mathbf{Y}) + H(I_0^{(n)}|\mathbf{X},\mathbf{Y}) \\
257: & \stackrel{(b)}{=} & I(F(\mathbf{X},\mathbf{Y});I_0^{(n)}|\mathbf{Y})\\
258: & = & H(F(\mathbf{X},\mathbf{Y})|\mathbf{Y}))-H(F(\mathbf{X},\mathbf{Y})|\mathbf{Y},I_0^{(n)},J_0^{(n)}) \\
259: &\stackrel{(c)}{\geq} & n H(F(X,Y)|Y) - n\delta_n ,
260: \end{eqnarray*}
261: where
262: \begin{itemize}
263: \item[(a)] follows from (\ref{eqn:data2}),
264: \item[(b)] follows from the fact that $I_0^{(n)}$ is a function of $\mathbf{X}$,
265: \item[(c)] follows from the chain rule and the fact that $H(F(X_i,Y_i)|Y_i)=H(F(X,Y)|Y)$ for $i=1,2,\ldots,n$, and from (\ref{eqn:fano2}).
266: \end{itemize}
267: Using similar techniques, we also get $nR_2 \geq n H(F(X,Y)|X) - n\delta_n$ by using (\ref{eqn:data3}) and (\ref{eqn:fano3}). Thus, for any $n$, we have
268: $R_1 \geq H(F(X,Y)|Y)-\delta_n$, $R_2 \geq H(F(X,Y)|X)-\delta_n$ and $R_1+R_2 \geq H(F(X,Y))-\delta_n$. Since $\delta_n \rightarrow 0$
269: as $n \rightarrow \infty$, we have that any rate pair is achievable only if $R_1 \geq H(F(X,Y)|Y)$, $R_2 \geq H(F(X,Y)|X)$ and $R_1+R_2 \geq H(F(X,Y))$. Thus,
270: \begin{equation*}
271: \mathscr{R}(F)\subset \left\{(R_1,R_2): R_1 \geq H(F(X,Y)|Y), R_2 \geq H(F(X,Y)|X), R_1+R_2 \geq H(F(X,Y))\right\}.
272: \end{equation*}
273: This completes the proof of the converse.\hfill $\blacksquare$
274:
275: \section{Concluding Remarks}
276: We have found the exact achievable rate region for the problem of reliably recovering a function of correlated sources by separate encoding of the sources.
277: The proof turns out to be a simple plug-and-play of the techniques in \cite{Cove91}. It is obvious that the achievable rate region found here reduces
278: to the Slepian-Wolf region when $F$ is the identity function. Although less obvious, it is not difficult to see that the result derived in this correspondence
279: conforms with the results of \cite{Korn79}, \cite{Han87}.
280: \bibliographystyle{ieee}
281: \bibliography{fxybib}
282: \end{document}
283: