1: \documentclass[10pt]{article}

2: \usepackage[dvips]{graphicx}

3: \usepackage{amsfonts,verbatim}

4: \usepackage{amsmath,amssymb}

5: \usepackage{amsmath,amsbsy}

6: \usepackage{epsf,epsfig,verbatim}

7: \baselineskip=1.0\normalbaselineskip

8: \renewcommand{\baselinestretch}{1.0}

9: \pagestyle{plain}

10: 

11: \usepackage{multirow}

12: \usepackage{slashbox}

13: \usepackage{hhline}

14: 

15: \newcommand{\prodin}{\prod_{i=1}^n}

16: \newcommand{\sumin}{\sum_{i=1}^n}

17: \newcommand{\ra}{\rightarrow}

18: \newcommand{\beq}{\begin{equation}}

19: \newcommand{\eeq}{\end{equation}}

20: \newcommand{\C}{{\bf \Large \mathcal{C}}}

21: \newcommand{\mc}{\mathcal}

22: \newcommand{\mb}{\mathbb}

23: \newcommand{\mf}{\mathbf}

24: \newcommand{\tl}{\tilde}

25: \newcommand{\h}{\hat}

26: \newcommand{\bm}{\boldmath}

27: \newcommand{\bs}{\boldsymbol}

28: \newcommand{\e}{\epsilon}

29: \newcommand{\dl}{\delta}

30: \newcommand{\g}{\gamma}

31: \newcommand{\Ae}{A_\epsilon^{(n)}}

32: \newcommand{\Ase}{A_\epsilon^{*(n)}}

33: 

34: 

35: \newtheorem{thm}{Theorem}

36: \newtheorem{dfn}{Definition}

37: \newtheorem{rem}{Remark}

38: \newtheorem{lem}{Lemma}

39: \newtheorem{cor}{Corollary}

40: \newtheorem{fact}{Fact}

41: \newtheorem{proof}{Proof}

42: 

43: 

44: \addtolength{\hoffset}{-3cm} \addtolength{\textwidth}{5.5cm}

45: \addtolength{\voffset}{-1.0cm} \addtolength{\textheight}{4cm}

46: 

47: \allowdisplaybreaks

48: 

49: \begin{document}

50: 

51: \title{A Graph-based Framework for Transmission of Correlated Sources

52:   over Broadcast Channels 

53: \footnote{This work was supported by NSF CAREER Grant

54:   CCF-0448115. This work was presented in part at the 39th Conference

55:   on Information Sciences and Systems (CISS), Baltimore, MD, March

56:   2005, Information Theory and Applications workshop 

57:   (ITA), San Diego, CA, February, 2006, and 

58: IEEE International Symposium on Information

59:   Theory (ISIT), Seattle, WA, July 2006.}} 

60: \author{Suhan Choi and  S.\ Sandeep Pradhan\\

61:       \{suhanc, pradhanv\}@eecs.umich.edu,\\

62: EECS Dept., Univ. of  Michigan, Ann Arbor, MI.}

63: 

64: \maketitle

65: 

66: 

67: \begin{abstract}

68: In this paper we consider the communication problem that involves

69: transmission of correlated sources over broadcast channels. We

70: consider a graph-based framework for this information transmission

71: problem. The system involves a source coding module and a channel

72: coding module. In the source coding module, the sources are

73: efficiently mapped into a nearly semi-regular bipartite graph, and

74: in the channel coding module, the edges of this graph are reliably

75: transmitted over a broadcast channel. We consider nearly

76: semi-regular bipartite graphs as discrete interface between source

77: coding and channel coding in this multiterminal setting. We provide

78: an information-theoretic characterization of (1) the rate of

79: exponential growth (as a function of the number of channel uses) of

80: the size of the bipartite graphs whose edges can be reliably

81: transmitted over a broadcast channel and (2) the rate of exponential

82: growth (as a function of the number of source samples) of the size

83: of the bipartite graphs which can reliably represent a pair of

84: correlated sources to be transmitted over a broadcast channel.

85: \end{abstract}

86: 

87: \setlength{\baselineskip}{18.6pt}

88: 

89: \section{Introduction}

90: 

91: With the emergence of new set of applications such as wireless sensor

92: networks, the problem of transmission of correlated information

93: sources over multiterminal channels has received a renewed attention. 

94: In this problem, many correlated information sources are accessed by a

95: set of transmitter terminals, and they wish to simultaneously transmit

96: some subset of them to another set of receiver terminals over a channel. 

97: In this paper we address the one-to-many communication system, where 

98: one transmitter terminal  has access to all the information sources,

99: and wish to transmit them to many receiver terminals. One such model

100: involving two receiver terminals was considered by Han and Costa  in 

101: \cite{han-costa87}, and  is described in the

102: following. Consider a pair of correlated discrete 

103: memoryless sources $(S,T)$ with some generic joint distribution

104: $p(s,t)$ and a pair of finite alphabets $\mc{S}$ and $\mc{T}$,

105: respectively.  The encoder observes long sequences of realizations of these

106: sources (of length say $n$),  and wishes to transmit them over a

107: broadcast channel which has one input $X$ and two outputs $Y_1$ and

108: $Y_2$, and  Receiver i has access to $Y_i$.  The channel behavior is

109: governed by a generic conditional distribution $p(y_1,y_2|x)$. The

110: channel is assumed to be discrete memoryless and is used without

111: feedback.   The encoder maps $n$-length source sequence pairs into

112: $n$-length channel input sequences. Each receiver  maps its

113: corresponding $n$-length channel output sequences into its

114: corresponding $n$-length source reconstruction sequences. The receivers

115: would like to produce a reconstruction sequence pair such that the

116: probability that this deviates from the original pair goes to zero as

117: blocklength $n$ becomes large. If it is possible to build such

118: sequences of mappings, then we say that the source pair is

119: transmissible over the broadcast channel. The goal is to find the set

120: of all sources that are transmissible over a given broadcast channel

121: or the set of all channels over which a given source pair is

122: transmissible. 

123: 

124: Two approaches have been proposed for this problem in the

125: literature. On is called the joint-source-channel coding approach

126: and the other is the separation-approach. The former approach

127: \cite{han-costa87}  addresses the problem directly by finding the

128: mappings for the given source pair and broadcast channel. A sufficient

129: condition for transmissibility of a source over a channel

130: has been given in \cite{han-costa87} for this

131: problem. In the separation-approach, we divide the encoding task into two

132: sub-tasks, and similarly the decoding task is accomplished in two

133: steps. In this approach, the $n$-length source pair is mapped into

134: three indexes (referred to as $W_0$, $W_1$ and $W_2$) 

135: coming from three finite sets of size say $\Delta_0$,

136: $\Delta_1$ and $\Delta_2$, respectively. The first index (common

137: message) is meant for both receivers, and the second and the third indexes

138: (private messages)  are meant for Receiver 1 and Receiver 2,

139: respectively. This is called source encoding. The goal is to remove

140: all the redundancy from the pair to produce three independent bit

141: streams. Then these three indexes are mapped (referred to as channel

142: coding) to $n$-length channel input sequences. On the other side of

143: the channel, 

144:  Receiver i first maps its $n$-length channel output

145: into a pair of indexes ($W_0$ and $W_i$) for $i=1,2$. Then they

146: independently produce source reconstruction sequences from the common

147: and the private messages. The first goal is to find the set (called as

148: the rate region) of all  

149: the rate tuples $(R_0,R_1,R_2)$, where $R_j=\frac{1}{n}\log \Delta_j$

150: for $j=0,1,2$, 

151: at which a  reliable representation of the given source pair can be

152: accomplished. The second goal is to find the set (called as the capacity

153: region) of all rate tuples 

154: $(R_0,R_1,R_2)$, at which a reliable communication of indexes over

155: the given channel can be accomplished. 

156: The source coding part works under the assumption that the channel is

157: noiseless, and the channel coding part works under the assumption that

158: the messages are independent. 

159: 

160: The channel coding part by itself was first introduced  by Cover

161: in \cite{cover72,cover75}.

162:  The capacity region has been found for many interesting classes of

163:  broadcast channels 

164: \cite{cover72,cover75,bergmans73,bergmans74,gallager74,

165: ahlswede-korner75,korner-marton77,elgamal79, 

166: marton77,pinsker78,marton79,

167: gelfand80,elgamal}.  The capacity region of certain class of

168: broadcast channels used in wireless communication systems 

169: have been obtained in 

170:  \cite{tse97,li01,caire03,viswanathtsc03,vishwanathjgda03,yu04,weingarten04}.

171: Marton \cite{marton79} (also see \cite{meulen75,hajek79,elgamal81})

172: established an inner bound to the capacity

173: region for the discrete memoryless broadcast channel, which

174: contains all the known achievable rate regions. 

175: Outer bounds to the capacity region have been obtained in

176: \cite{sato78,marton79,nair06}. See \cite{cover98} for a latest 

177: survey of the results on broadcast channels.

178: The source coding part was addressed by Gray and Wyner in

179: \cite{gray74}, where a complete characterization of the rate region 

180: was given. We refer this source coding problem as Gray-Wyner problem. 

181: However, it is well-known that the separation-approach is not

182: optimal for the one-to-many communication problem, unlike the case of

183: point-to-point information transmission 

184: problem.  In particular, a simple  example was given in

185: \cite{han-costa87} that 

186: showed that a triangular source can be sent reliably over a Blackwell

187: channel by  using a simple joint-source-channel coding scheme, but

188: there is no way of 

189: transmitting that source over that channel using the separation-approach.

190: 

191: 

192: Loosely speaking, in the separation-approach, there is an interface

193: between the source coding module and the channel coding module. 

194: Due to the structure of the system, i.e.,  a common message and a pair

195: of private messages, the interface can be thought of as 

196: a finite collection of products of finite sets. 

197: For example, in a system with $\Delta_i$ denoting the size of the

198: $i$th message for $i=0,1,2$, the interface can be thought of as

199: $\cup_{i=1}^{\Delta_0} (A_i \times B_i) $, where $A_i=\{(i-1)\Delta_1+1, 

200: (i-1)\Delta_1+2, \ldots, i\Delta_1\}$ and 

201: $B_i=\{(i-1)\Delta_2+1, 

202: (i-1)\Delta_2+2, \ldots, i\Delta_2\}$.

203: However, as seen above, this interface, a finite collection of

204: products of finite sets, 

205: is not an efficient representation of a pair of correlated sources for

206: transmission over broadcast channels\footnote{Another reason why this

207:   approach may be suboptimal is given in the following. In the

208:   characterization of the rate region in \cite{gray74}, it turns out

209:   that for certain choices of the triple $(R_0,R_1,R_2)$ that belongs

210:   to the boundary of the rate region, i.e., optimal triple, 

211: the private indexes produced by the joint

212:   encoder will \emph{not} be independent asymptotically. Hence, the

213:   channel coding module that follows, which works under the assumptions of

214:   independence, can not exploit this correlation, and is wasteful of

215:   resources.   For example, in (15a) in \cite{gray74}, if one chooses

216:   $W$ such that $(X,W,Y)$ do \emph{not} form a Markov chain, then the triple

217: $(R_0,R_1,R_2)=(I(X,Y;W),H(X|W),H(Y|W))$ has this property. }.

218: Note that the corresponding interface for a similar approach in the

219: point-to-point case is just a finite set. It is also well-known that

220: a similar separation-approach is not optimal for other multiuser

221: information transmission problems such as many-to-one communication

222: \cite{cover-elgamal-salehi80}. 

223: 

224: In our recent work \cite{pradhan-choi-isit,pradhan-choi06}, we have

225: reported a  bipartite graph-based framework for the problem of

226: transmission  of information in the many-to-one case. A similar

227: approach was also studied in \cite{ahlswede-han83}. In the present

228: work, we consider a similar approach to the one-to-many communication

229: scenario. The fundamental motivation for this comes from the

230: concept of typicality \cite{cover-thomas}.  Given a correlated

231: source pair $(S,T)$, a sequence in $\mc{S}^n$

232: is said to be typical (or individually  typical) with respect to

233: $p(s)$, if its empirical 

234: histogram is close to $p(s)$. Similarly one can define typical

235: sequences in $\mc{T}^n$. A sequence pair in $\mc{S}^n \times \mc{T}^n$

236: is said to be jointly typical if its empirical joint histogram is

237: close to $p(s,t)$. Using the law of large numbers, it follows that 

238: (a) there are roughly $2^{nH(S)}$ and $2^{nH(T)}$ individually typical

239: sequences in $\mc{S}^n$ and $\mc{T}^n$, respectively, where $H(\cdot)$

240: denotes entropy \cite{cover-thomas}, (b) there are roughly

241: $2^{nH(S,T)}$ jointly typical sequence pairs in $\mc{S}^n \times

242: \mc{T}^n$, (c) for every typical 

243: sequence in $\mc{S}^n$, there are roughly $2^{nH(T|S)}$ typical

244: sequences in $\mc{T}^n$ that are jointly typical and vice versa, 

245: (d) probability, under $p(s,t)$, of the set of jointly typical sequences

246: (called jointly typical set)

247: is close to $1$, and (e) the probability, under $p(s,t)$, of every

248: jointly typical sequence pair is roughly equal to

249: $2^{-nH(S,T)}$. These five properties lead one to associate a

250: bipartite graph on the jointly typical set, with vertexes

251: formed by individually typical sequences, and two vertexes are connected by

252: an edge if they are jointly typical. Such bipartite graphs, where the

253: degrees of the vertexes of one set is close to one constant, and the

254: degrees of that of the  other set is close to another constant, are referred to

255: as nearly semi-regular \cite{vanlint}.  Hence graphs can naturally

256: capture the behavior of the source pair. The details regarding the

257: source distribution can be dispensed with, and one can just work with

258: this bipartite graph. This may also lead to the

259: possibility of using them as discrete interface for one-to-many

260: communication. The source encoder would now act on the source pair and

261: produce correlated messages, or edges in a bipartite graph, and the

262: channel encoder would now work with correlated messages and reliably

263: transmit the edges in the graph over the broadcast channel. We would still

264: have a source coding module and a channel coding module. However, now

265: they would be interfaced using nearly semi-regular bipartite graphs

266: rather than just a finite collection of products of finite sets. Of

267: course, a finite collection of products of finite sets is a special

268: case of nearly semi-regular bipartite graphs. 

269: 

270: 

271: We now present a brief summary of the results presented in this paper,

272: for which we need some definitions. A nearly semi-regular bipartite 

273: graph is said to

274: have parameters $(\theta_1,\theta_2,\theta'_1,\theta'_2)$ if the $i$th

275: vertex set has size nearly equal to $\theta_i$ for $i=1,2$, and the

276: degrees of 

277: vertexes of the first set is nearly equal to $\theta'_2$ and vice

278: versa. With a slight abuse of notation, a 

279: nearly semi-regular bipartite  graph is said to

280: have parameters $(\theta_0,\theta_1,\theta_2,\theta'_1,\theta'_2)$

281: if it is the union of $\theta_0$ disjoint subgraphs each having

282: parameters $(\theta_1,\theta_2,\theta'_1,\theta'_2)$.  

283: A tuple of rates  $(R_0,R_1,R_2,R'_1,R'_2)$ is said to be achievable

284: for the given broadcast channel if \emph{there exists} a bipartite graph 

285:  with parameters $(2^{nR_0},2^{nR_1},2^{nR_2},2^{nR'_1},2^{nR'_2})$

286:  whose edges can be reliably transmitted by using the channel $n$

287:  times for large $n$. Similarly, a tuple of rates

288:  $(R_0,R_1,R_2,R'_1,R'_2)$ is said to be 

289: achievable for a given pair of correlated sources if \emph{there

290:   exists} a bipartite graph with parameters

291: $(2^{nR_0},2^{nR_1},2^{nR_2},2^{nR'_1},2^{nR'_2})$  which can reliably

292: represent $n$ realizations of the pair for large $n$. 

293: We provide information-theoretic partial 

294:  characterizations of the sets of achievable tuples for a 

295:  broadcast channel  and a correlated source pair. 

296: These are presented in Theorem 1-4. 

297: Having the significance of the proposed framework mentioned first, let

298: us look at the limitations of this optimistic framework as well. 

299: For that we need to look at the big picture. 

300: 

301: For the point-to-point case, to

302: check the transmissibility of a source over a channel, we just need to

303: check the non-emptiness of the intersection of two intervals

304: $[H,\infty)$ and $[0,C]$, where $H$ denotes the entropy of the source,

305:   and $C$ denotes the capacity of the channel. Note that only one parameter

306:   specifies the interval. For the one-to-many communication, the

307: conventional separation-approach gives a sufficient condition for checking the

308:   transmissibility: non-emptiness of the rate region of the

309:   source pair and that of the broadcast channel. We need three

310:   parameters $(R_0,R_1,R_2)$ to specify the rate region in both source

311:   coding as well as channel coding.  

312: Clearly, a  characterization that involves the fewest number of

313: parameters is what we want.  

314: 

315: 

316: In the graph-based framework, we consider rate regions for the source

317: and the channel, which are specified using five parameters:

318: $(R_0,R_1,R_2,R'_1,R'_2)$.  However, the non-empty intersection of

319: the rate region of the source pair with that of the channel still does not

320: guarantee successful transmission. This is because, graphs having the

321: same set of parameters may have different structures. It turns out

322: that these

323: graphs (that have the same set of parameters) can be partitioned into

324: equivalence classes, where all graphs 

325: in an equivalence class have the same structure. This structure of the

326: graphs has also been studied under the name of graph isomorphism

327: \cite{vanlint} in the literature.  We will address this

328: issue more formally in later sections.  Hence a graph with

329: parameters (say) $(2^{nR_0},2^{nR_1},2^{nR_2},2^{nR'_1},2^{nR'_2})$ 

330: which can reliably represent a source pair may not belong to the

331: equivalence class of a graph with the same set of parameters whose

332: edges can be transmitted reliably over the channel. 

333: In other words, to guarantee successful transmission of the source

334: over the channel, we need to

335: construct at least one pair of transmission systems (one for the source

336: component and one for the channel component) for every equivalence class.

337: Rather, in this work we have shown (a) the existence of an  equivalence

338: class for which a transmission system could be built in the source

339: coding component, and (b)  the existence of an  equivalence

340: class for which a transmission system could be built in the channel

341: coding component.

342: We plan to address this issue further in our future work. 

343: But we believe that the results given in this paper may be a first

344: step toward an optimal discrete interface for multiuser

345: communication. 

346: 

347: The outline of the remaining part of this paper is as follows. In

348: Section \ref{sec:bc-prelim}, we first provide a summary

349: of the results in a formal setting that are available in the

350: literature that are closely related to our work. Then, in 

351: Section \ref{sec:bc-problem}, we formulate the problem and consider

352: certain properties of bipartite graphs that are relevant to our

353: later discussion. Then channel coding part will be discussed in

354: Section \ref{sec:general-bc}, resulting in an achievable rate region

355: for the broadcast channel with correlated messages. We also consider

356: the special case of broadcast channels with one deterministic

357: component. Thereafter, the complementary source coding part, the representation

358: of correlated sources into graphs, will be described in

359: Section \ref{sec:bc-sc}. After that, an example and its

360: interpretation are provided in Section \ref{sec:bc-example}.

361: Finally, Section \ref{sec:bc-conclusion} provides some concluding

362: remarks.

363: 

364: \section{Preliminaries}\label{sec:bc-prelim}

365: 

366: In this section, we provide an overview of the most important prior

367: results in the literature on broadcast channels, and the related

368: source coding problem,  which are closely related to our work.

369: 

370: \subsection{Broadcast Channel with Independent private Messages} 

371: A broadcast channel is composed of one sender and many receivers. 

372: The objective is to broadcast information from a sender to the many

373: receivers. We consider broadcast channels with only two receivers

374: since multiple receivers cases can be similarly treated. Figure

375: \ref{fig:broadcast} shows a broadcast channel with one sender and

376: two receivers. The discrete memoryless stationary broadcast channel consists of

377: an input alphabet $\mathcal{X}$ and two output alphabets

378: $\mathcal{Y}_1$ and $\mathcal{Y}_2$ and a conditional distribution 

379: $p(y_1,y_2|x)$. The induced $n$-length conditional distribution is

380: given by $p(y_1^n,y_2^n|x^n)=\prod_{i=1}^n p(y_{1i},y_{2i}|x_i)$ when used

381: without feedback. In other words, a broadcast channel is an ordered

382: tuple $(\mc{X},\mc{Y}_1,\mc{Y}_2, p(y_1,y_2|x))$.

383: \begin{figure}[!h]

384: \centering \epsfig{file=broadcast.eps, clip=, width=.7\linewidth}

385: \centering \caption{\small Broadcast channel}\label{fig:broadcast}

386: \end{figure}

387: We consider the following definitions for transmission of

388: independent messages over a broadcast channel.

389: 

390: \begin{dfn}

391: A transmission system with parameters $(n,\Delta_0,\Delta_1,\Delta_2,\tau)$

392: for the given broadcast channel $(\mc{X},\mc{Y}_1,$ $\mc{Y}_2,

393: p(y_1,y_2|x))$ is an ordered tuple $(f,g_1,g_2)$, consisting of one

394: encoding mapping $f$ and two decoding mappings $g_1$ and $g_2$ where

395: \begin{itemize}

396: \item $f: \{1,2,\ldots,\Delta_0\} \times 

397: \{1,2,\ldots,\Delta_1\} \times \{1,2,\ldots,\Delta_2\}

398:   \rightarrow \mc{X}^n$, 

399: \item $g_i: \mc{Y}_i^n \rightarrow \{1,2,\ldots,\Delta_0\} \times 

400: \{1,2,\ldots,\Delta_i\}$, for $i=1,2$,

401: \item such that a performance measure given by the average probability

402:   of error satisfies: 

403:     \[

404:       \tau= \sum_{k=1}^{\Delta_0}  \sum_{i=1}^{\Delta_1} \sum_{j=1}^{\Delta_2}

405: \frac{1}{\Delta_0 \Delta_1 \Delta_2} Pr \left[ (g_1(Y_1^n), g_2(Y_2^n)) \neq

406: ((k,i),(k,j)) | X^n=f(k,i,j) \right].

407:     \]

408: \end{itemize}

409: \end{dfn}

410: 

411: \begin{dfn}

412:   A rate tuple $(R_0,R_1, R_2)$ is said to be \emph{achievable} for the

413:   given broadcast channel if for all $\e > 0$, and for sufficiently 

414:   large $n$, there exists a transmission system as defined above

415:   satisfying $\frac{1}{n}\log \Delta_i > R_i -\e$ for $i=0,1,2$ and with 

416:   the average probability of error $\tau < \e$.

417: \end{dfn}

418: 

419: \begin{dfn}

420:   The \emph{capacity region} $\mathcal{C}_{BCI}$ of the broadcast

421:   channel is the set of all achievable rate tuples

422:   $(R_0,R_1, R_2)$. 

423: \end{dfn}

424: 

425: The capacity region of general broadcast channels is still not known. But

426: Marton \cite{marton79} and Gelfand and Pinsker \cite{gelfand80} have

427: obtained an  achievable rate region for the general discrete

428: memoryless broadcast channel, 

429:  which is the largest known inner bound to the capacity region.

430: An achievable region of the discrete memoryless broadcast channel

431: \cite{csiszar-korner} is given by all rate tuples $(R_0,R_1,R_2)$

432: satisfying

433:   \begin{align}

434:     R_0 &< \min\{I(Z;Y_1),I(Z;Y_2)\},\\

435:     R_0 + R_1 &< I(Z,U;Y_1), \\

436:     R_0 + R_2 &< I(Z,V;Y_2), \\

437:     R_0 + R_1 + R_2 &< \min\{I(Z;Y_1),I(Z;Y_2)\}+I(U;Y_1|Z)+I(V;Y_2|Z)-I(U;V|Z)

438:   \end{align}

439: for some $p(z,u,v,x)$ on ${\mathcal Z} \times $ ${\mathcal U} \times

440: {\mathcal V} \times {\mathcal X}$, where $Z$, $U$  and $V$ are 

441: auxiliary random variables with finite alphabets ${\mathcal Z}$,

442: ${\mathcal U}$,  and ${\mathcal V}$, respectively  such

443: that $(Z,U,V) \rightarrow X \rightarrow (Y_1,Y_2)$ form a Markov

444: chain. 

445: 

446: \subsection{Gray-Wyner problem}

447:  

448: Consider a pair of correlated sources with a joint distribution

449: $p(s,t)$ and with finite alphabets ${\mathcal S}$ and ${\mathcal

450:   T}$. The sources are assumed to be stationary and memoryless. These

451: sources are denoted as an ordered tuple $({\mathcal S},{\mathcal

452:   T},p(s,t))$. 

453: 

454: \begin{dfn}

455: A transmission system with parameters

456: $(n,\Delta_0,\Delta_1,\Delta_2,\tau)$  for representing a pair of

457: correlated sources $({\mathcal S},{\mathcal T},p(s,t))$ is an ordered

458: tuple $(f_0,f_1,f_2,g_1,g_2)$ consisting of three encoding mappings

459: $f_0$, $f_1$ and $f_2$,  and two decoding mappings $g_1$ and $g_2$ where 

460: \begin{itemize}

461: \item $f_i: \mc{S}^n \times \mc{T}^n \rightarrow 

462:  \{1,2,\ldots,\Delta_i\},  \ \ \mbox{for } \ \ i=0,1,2$,

463: \item $g_1:  \{1,2,\ldots,\Delta_0\} \times \{1,2,\ldots,\Delta_1\}

464: \rightarrow  \mc{S}^n$, \ \ \ 

465:  $g_2:  \{1,2,\ldots,\Delta_0\} \times \{1,2,\ldots,\Delta_2\}

466: \rightarrow  \mc{T}^n,$

467: \item such that a performance measure given by the probability

468:   of error satisfies: 

469:     \[

470:       \tau= Pr \left[(g_1(f_0(S^n,T^n),f_1(S^n,T^n)),

471: g_2(f_0(S^n,T^n),f_2(S^n,T^n)) \neq (S^n,T^n)\right].

472:     \]

473: \end{itemize}

474: \end{dfn}

475: 

476: \begin{dfn}

477:   A rate tuple $(R_0,R_1, R_2)$ is said to be \emph{achievable} for the

478:   given correlated sources if for all $\e > 0$, and for sufficiently 

479:   large $n$, there exists a transmission system as defined above

480:   satisfying $\frac{1}{n}\log \Delta_i < R_i +\e$ for $i=0,1,2$, and with 

481:   the average probability of error $\tau < \e$.

482: \end{dfn}

483: 

484: \begin{dfn}

485:   The \emph{achievable region} $\mathcal{R}_{GWI}$ for the correlated

486:   sources is the set of all achievable rate tuples.

487: \end{dfn}

488: 

489: An information-theoretic characterization of this rate region

490: \cite{gray74,wyner75} is given in the

491: following. For a given pair of correlated sources a  tuple

492: $(R_0,R_1,R_2)$ is achievable  if and only if 

493:  \begin{align}

494:     R_0 &> I(S,T;Z), \\

495:     R_1 &> H(S|Z), \\

496:     R_2 &> H(T|Z), 

497:     \end{align}

498: for some distribution

499: $p(z,s,t)=p(s,t)p(z|s,t)$, where $Z$ is an auxiliary random variable with a

500: finite alphabet ${\mathcal Z}$. Using convexity arguments, it can be

501: shown that there is no loss of optimality if $|{\mathcal Z}| \leq

502: |{\mathcal S}||{\mathcal T}|$. 

503: 

504: This problem was  also considered in \cite{wyner75} in a slightly different

505: form. The minimum $R_0$ that belongs to the rate region such that the

506: corresponding $Z$ satisfies $T \rightarrow Z \rightarrow S$ a Markov

507: chain,  is called as

508: Wyner's common information $C(S,T)$. 

509: 

510: \subsection{Joint source-channel coding}

511: Consider the joint source-channel coding scheme studied in

512: \cite{han-costa87}. Suppose we are given a pair of correlated sources

513: without common part \cite{gacs,wit,wyner75}

514: \footnote{\cite{han-costa87} considered the general

515:   setting where sources may have non-zero common part.}   and a

516: broadcast channel. 

517: 

518: \begin{dfn}A transmission system with parameters $(n,\tau)$ for

519:   transmission of a pair of correlated sources $({\mathcal

520:   S},{\mathcal T}, p(s,t))$ and a broadcast channel 

521: $({\mathcal X},{\mathcal Y}_1,{\mathcal Y}_2,p(y_1,y_2|x))$ is an 

522: ordered tuple $(f,g_1,g_2)$ where

523: \begin{itemize}

524: \item $f:{\mathcal S}^n \times {\mathcal T}^n \rightarrow {\mathcal

525:   X}^n$ 

526: \item $g_1: {\mathcal Y}_1^n \rightarrow {\mathcal S}^n$, and 

527: $g_2: {\mathcal Y}_2^n \rightarrow {\mathcal T}^n$.

528: \item such that a performance measure given by the probability of

529:   decoding error satisfies

530: \[

531: \tau= \sum_{(s^n,t^n) \in {\mathcal S}^n \times {\mathcal T}^n}

532: p^n(s^n,t^n) Pr[(g_1(Y_1^n),g_2(Y_2^n))\neq (s^n,t^n)|X^n=f(s^n,t^n)].

533: \]

534: \end{itemize}

535: 

536: \end{dfn}

537: 

538: \begin{dfn}

539: A pair of correlated sources is said to be transmissible over a

540: broadcast channel if $\forall \e>0$, and for all sufficiently large

541: $n$, there exists a transmission system as defined above with

542: parameters $(n,\tau)$ such that $\tau < \e$.

543: \end{dfn}

544: 

545: The result of \cite{han-costa87} says that a pair of correlated

546: sources is transmissible over a broadcast channel if 

547: \begin{align}

548: H(S) &< I(S,W,U;Y_1)-I(T;W,U|S) \\

549: H(T) &< I(T,W,V;Y_2)-I(S;W,V|T) \\

550: H(S,T) &< \min \{I(W;Y_1),I(W;Y_2)\} +I(S,U;Y_1|W)+

551: I(T,V;Y_2|W)-I(S,U;T,V|W)

552: \end{align}

553: for some $p(w,u,v,x|s,t)$ on 

554: ${\mathcal W} \times {\mathcal U} \times {\mathcal V} \times {\mathcal

555:   X} \times  {\mathcal S} \times {\mathcal T}$, where $W,U,V$ are

556: auxiliary random variables with finite alphabets 

557: ${\mathcal W}$, ${\mathcal U}$ and ${\mathcal V}$ such that 

558: $(S,T) \rightarrow (WUV)\rightarrow X \rightarrow (Y_1,Y_2)$ form

559: a Markov chain. 

560: 

561: This problem has also been considered in different settings recently in

562: \cite{tuncel05,coleman06}.

563: 

564: 

565: \subsection{An Example of transmission of Correlated Sources over the Broadcast

566:   Channel}\label{sec:bc-ex} 

567: 

568: Let us consider an interesting example given in \cite{han-costa87}

569: showing the advantage of encoders that exploit the correlation

570: between sources. Consider the transmission of a set of correlated

571: sources $(S,T)$ with the joint distribution $p(s,t)$ given by

572: \begin{center}

573: 

574: \renewcommand{\arraystretch}{1.5}

575: 

576: \setlength{\fboxsep}{2pt}\setlength{\tabcolsep}{3pt}

577: 

578: \begin{tabular}{|c|c|c|}

579: \hline

580:  \setlength{\backslashbox{$T$}{$S$}} &0 & 1 \\ \hline

581:  0 & $~~~\frac{1}{3}~~~$ & 0 \\ \hline

582:  1 & $\frac{1}{3}$ & $~~~\frac{1}{3}~~~$ \\\hline

583: \end{tabular}

584: \end{center}

585: with finite alphabet $\mc{S}=\mc{T}=\{0,1\}$ over a Blackwell

586: channel with $\mc{X}=\{1,2,3\}$, $\mc{Y}_1=\mc{Y}_2=\{0,1\}$ where

587: the channel transition probabilities are specified by

588: $p(0,0|1)=p(0,1|2)=p(1,1|3)=1$. If we assign $X$=1, 2, and 3 to

589: $(S,T)$ = (0,0), (0,1), and (1,1), respectively, then $Y_1$ and

590: $Y_2$ determine $S$ and $T$ without error, respectively.

591: 

592: In the conventional separation-approach, 

593: first, factor $(S^n,T^n)$ into three independent messages

594:   $W_1,W_2$, and $W_0$.

595: Next, transmit $(W_0,W_1)$ and $(W_0,W_2)$ to receivers 1 and 2,

596:   respectively, over the broadcast channel using some, hitherto unknown,

597:   optimal coding scheme, so that $(W_0,W_1)$ and $(W_0,W_2)$ reliably

598:   determine $S^n$ and $T^n$, respectively.

599: Let the rate of $W_i$ be $R_i=\frac{1}{n}H(W_i)$ for

600: $i$ = 0, 1 and 2. Then, as the channel in consideration is

601: deterministic, the sum of these rates must be bounded as  $R_0

602: + R_1 + R_2 \leq \frac{1}{n} H(Y_1^n, Y_2^n) \leq \frac{1}{n} H(X^n)

603: \leq \log_2 3=H(S,T)$.  

604: 

605: According to \cite{wyner75}, the most efficient

606: decomposition of this kind with the constraint $R_0 + R_1 + R_2

607: \leq H(S,T)$ is attained when $R_0=C(S,T)$ In this case,

608: \begin{align}

609:   R_0 + R_1 + R_2 &= H(S,T) = \log_2 3 \textrm{(bits)}\\

610:   2R_0 + R_1 + R_2 &= \log_2 3 + I(S,T;Z) \geq  \log_2 3+C(S;T).

611: \end{align}

612: For this triangular source,

613: $C(S;T)=\log_2 3-h_2(\frac{2}{3}) = \frac{2}{3}$ where

614: $h_2(p)=-p\log_2 p-(1-p)\log_2(1-p)$. Therefore, $2R_0 + R_1 + R_2

615: \geq \log_2 3 +\frac{2}{3}\approx 2.252$ (bits). Consequently,

616: $R_0+R_1 > 1$ or $R_0+R_2 >1$. Since $H(Y_1) \leq 1$ and $H(Y_2)

617: \leq 1$ for any distribution, receiver 1 cannot reliably reproduce

618: $(W_0,W_1)$ or receiver 2 cannot reliably reproduce $(W_0,W_2)$.

619: Thus there is no way of reliably transmitting this triangular

620: source via the Blackwell channel by factoring the sources into $W_1,W_2$, and

621: $W_0$ as shown above.

622: 

623: \section{Problem Formulation}\label{sec:bc-problem}

624: 

625: \begin{figure}[h]

626: \centering \epsfig{file=bc-block.eps, clip, width=0.65\linewidth}

627: \centering \caption{\small Transmission of correlated sources over a

628: broadcast channel} \label{fig:bc-block}

629: \end{figure}

630: 

631: The problem we are addressing is the simultaneous transmission of

632: two correlated sources $(\mc{S},\mc{T},p(s,t))$ over a

633: broadcast channel 

634: $({\mathcal X},{\mathcal Y}_1,{\mathcal Y}_2,p(y_1,y_2|x))$, 

635: with one sender and

636: two receivers as shown in Figure \ref{fig:bc-block}. Here, the

637: encoder can access both sources $(S,T)$ and the receivers can not

638: communicate with each other. 

639: The encoder is given by a mapping $f: \mc{S}^n \times \mc{T}^n \ra

640: \mc{X}^n$. The decoders are given by mappings $g_1:\mc{Y}_1^n \ra

641: \mc{S}^n$ and $g_2:\mc{Y}_2^n \ra \mc{T}^n$. The performance measure

642: associated with this transmission system is the probability of

643: decoding error:

644: \begin{equation}

645:   Pr[(S^n,T^n)\neq(g_1(Y_1^n),g_2(Y_2^n))].

646: \end{equation}

647: 

648: 

649: \subsection{Basic Concepts}

650: 

651: \begin{figure}[h]

652: \centering \epsfig{file=bc-block-sc-cc.eps, clip,

653: width=0.9\linewidth} \centering \caption{\small The outputs of

654: sources are first mapped into edges in a bipartite graph, and the

655: edges coming from this graph are reliably transmitted over a

656: broadcast channel} \label{fig:bc-block-sc-cc}

657: \end{figure}

658: 

659: We consider a modular approach to this problem, which is shown in

660: Figure \ref{fig:bc-block-sc-cc}. The system has two modules: the

661: source coding module and the channel coding module. The outputs of

662: two correlated sources are first represented efficiently into a

663: triple of messages $(W_0,W_1,W_2)$ in the source coding module.

664: Then, these messages are reliably transmitted over the broadcast

665: channel in the channel coding module. 

666: In more detail, the source coding module produces three

667: messages $W_0, W_1$ and $W_2$, where $W_0$ is a common message to

668: both receivers which contains common information about both sources

669: $S$ and $T$, and $W_1$ and $W_2$ are private messages which contain

670: the individually remaining information about the source $S$ and $T$

671: after extracting the common information, respectively. The messages

672: $W_0,W_1$ and $W_2$ belong to integer sets

673: $\mathcal{W}_0=\{1, 2, \ldots, \Delta_0\}$, $\mathcal{W}_1=\{1, 2,

674: \ldots, \Delta_1\}$ and $\mathcal{W}_2=\{1, 2, \ldots, \Delta_2\}$,

675: respectively. In general, $W_0,W_1$ and $W_2$ are not independent.

676: Then, the channel

677: coding module wants to reliably transmit a message pair $(W_0,W_1)$

678: to receiver 1 and $(W_0,W_2)$ to receiver 2.

679: 

680: We assume that there is some kind of correlation between two

681: messages $W_1$ and $W_2$, i.e., private messages

682:  for the receivers can not be chosen independently. 

683: \begin{dfn}

684: \begin{itemize}

685:   \item The private messages of receivers are said to be \emph{correlated}, if

686: for every $w_0 \in \mc{W}_0$,   there exists a set $A(w_0)$ such that $A(w_0)

687: \subset \mathcal{W}_1 \times 

688: \mathcal{W}_2$,  and conditioned on $W_0=w_0$ the message

689:   pairs $(W_1,W_2) \in A(w_0)$ are equally likely with probability

690:   $\frac{1}{|A(w_0)|}$, and the message pairs $(W_1,W_2) \notin A(w_0)$ have

691:   probability zero.

692:   \item The private messages of the receivers are said to be

693:   \emph{independent}, 

694:   if $A(w_0) = \mathcal{W}_1 \times

695: \mathcal{W}_2$ for all $w_0 \in \mc{W}_0$.  In this case, the message pairs

696:     $(W_1,W_2)$ are equally likely with probability

697:     $\frac{1}{|\mathcal{W}_1

698: \times \mathcal{W}_2|}$.

699: \end{itemize}

700: \end{dfn}

701: We use bipartite graphs to model the correlation of the messages,

702: i.e., the set $A(w_0)$ is taken to be a bipartite graph for all $w_0

703: \in \mc{W}_0$. Let us first

704: define a bipartite graph and related mathematical terms before we

705: discuss the main problem.

706: 

707: \begin{dfn}

708: \begin{itemize}

709:   \item A \emph{bipartite graph} $G$ is defined as an ordered tuple

710: $G=(A_1,A_2,B)$ where $A_1$ and $A_2$ are two

711:   non-empty sets of vertexes, and $B$ is a set of edges where every edge

712:    of $B$ joins a vertex in $A_1$ to a vertex in $A_2$, i.e., $B \subseteq A_1 \times A_2$.

713:   \item If $G$ is a bipartite graph, let $V_1(G)$ and $V_2(G)$ denote  the

714:     first and the second vertex sets of $G$, respectively,  and

715:   $E(G)$ denote the edge set of $G$.

716:   \item If $(i,j) \in E(G)$, then $i$ and $j$ are said to be

717:   \emph{adjacent}.

718:   \item If each vertex in one set is adjacent to every vertex in the other

719:   set, then $G$ is said to be a \emph{complete} bipartite graph. In this case, $E(G) = V_1(G) \times

720:   V_2(G)$.

721:   \item The \emph{degree} of a

722: vertex $v \in V_i(G)$ in a graph $G$, denoted by

723: $\mathrm{deg}_{G,i}(v)$, is the number of edges

724: connected to $v$ for $i=1,2$.

725:   \item A \emph{subgraph} of a graph $G$ is a graph whose vertex and

726:   edge sets are subsets of those of $G$.

727: \end{itemize}

728: \end{dfn}

729: Since we consider a specific type of bipartite graphs in our

730: discussion, let us define those bipartite graphs as well. Although our main

731: results deal with nearly semi-regular graphs, for the purpose of

732: illustration, we consider semi-regular graphs for this section

733: alone.

734: 

735: \begin{dfn}\label{def:graph-4-parameters}

736: A bipartite graph $G$ is called \emph{semi-regular} \cite{vanlint}

737:  with parameters $(\theta_1, \theta_2, \theta_1',

738: \theta_2')$, if it satisfies:

739: \begin{itemize}

740:   \item $|V_i(G)|=\theta_i$ for $i$=1, 2,

741:   \item $\forall u \in V_1(G)$, $\mathrm{deg}_{G,1}(u)= \theta_2'$,

742:   \item $\forall v \in V_2(G)$, $\mathrm{deg}_{G,2}(v)= \theta_1'$.

743: \end{itemize}

744: \end{dfn}

745: 

746: As an example, two semi-regular bipartite graphs with parameters

747: $(4,6,2,3)$ are shown in Figure \ref{fig:semiregular-graph-ex}. Note

748: that there exist many semi-regular bipartite graphs with the same

749: set of parameters.

750: 

751: \begin{figure}[h]

752: \centering \epsfig{file=semiregular-graph-ex.eps, clip,

753: width=0.4\linewidth} \centering \caption{\small Examples of

754: semi-regular bipartite graphs with parameters $(4, 6, 2, 3)$}

755: \label{fig:semiregular-graph-ex}

756: \end{figure}

757: 

758: \begin{dfn}\label{def:graph-5-parameters}

759:  A bipartite graph $G$ is called \emph{nearly semi-regular} with parameters $(\Delta_1$, $\Delta_2$, $\Delta_1'$,

760: $\Delta_2'$, $\mu)$ for $\mu > 1$ if it satisfies:

761: \begin{itemize}

762:   \item $|V_i(G)|=\Delta_i $ for $i=1, 2$,

763:   \item $\forall u \in V_1(G)$, $\Delta_2' \mu^{-1} \leq

764:   \mathrm{deg}_{G,1}(u) \leq \Delta_2' \mu$, \ \ \ and \ \

765:   $\forall v \in V_2(G)$, $\Delta_1' \mu^{-1}  \leq \mathrm{deg}_{G,2}(v)  \leq \Delta_1' \mu$.

766: \end{itemize}

767: \end{dfn}

768: Note that $\mu$ is a slack parameter, which determines the range of

769: degrees of vertexes.

770: 

771: \begin{dfn}\label{def:bc-graph-5-parameters}

772:  A nearly semi-regular

773: bipartite graph $G$ is said to have parameters $(\Delta_0, \Delta_1,

774: \Delta_2, \Delta_1', \Delta_2', \mu)$ for $\mu > 1$, if it

775: satisfies the following conditions:

776: \begin{itemize}

777:   \item $G$ is the union  of $\Delta_0$ disjoint subgraphs

778:   $\tilde{G}_m$ for $m=1,2,\ldots,\Delta_0$ where $\tilde{G}_m$ is

779:   nearly semi-regular with parameters 

780: $(\Delta_1, \Delta_2, \Delta_1', \Delta_2', \mu)$ as in Definition 

781: \ref{def:graph-5-parameters}.

782:   \item $\forall m \in \{1,2,\ldots, \Delta_0\}$,

783:     $V_i(\tilde{G}_m)=\{(m-1)\Delta_i+1,(m-1)\Delta_i+2,\ldots,m\Delta_i\}$

784:     for 

785:     $i$=1, 2,

786:   \item $V_i(G)=\bigcup_{m=1}^{\Delta_0} V_i(\tilde{G}_m)$ for $i$=1, 2,

787:   \ \ and \ \  $E(G)=\bigcup_{m=1}^{\Delta_0} E(\tilde{G}_m)$,

788:  \end{itemize}

789: \end{dfn}

790: 

791: A triple of messages (random variables) $(W_0,W_1,W_2)$ can be

792: associated with a graph $G$ with parameters $(\Delta_0$, $\Delta_1$,

793: $\Delta_2$, $\Delta_1'$, $\Delta_2'$, $\mu)$ in the following way.

794: $W_i \in

795: \mc{W}_i$ and $\mc{W}_i=\{1,2,\ldots,\Delta_i\}$ for $i=0,1,2$, and

796: an edge $((m-1)\Delta_1+i,(m-1)\Delta_2+j)\in E(G)$ denotes a

797: realization of the triple

798: of messages $(W_0,W_1,W_2)=(m,i,j)$. In other words, 

799: $Pr[(W_0,W_1,W_2)=(m,i,j)]=\frac{1}{|E(G)|}$ if 

800: $((m-1)\Delta_1+i,(m-1)\Delta_2+j)\in E(G)$ and $0$ otherwise.

801: So, different correlation

802: structures of the messages can be modeled by varying the 

803: structure of the graph. Figure \ref{fig:bc-msg-graph-ex} illustrates

804: an example of a bipartite graph $G$ with parameters 

805: $(2, 4, 4, 2, 2, 1)$ composed of two

806: disjoint subgraphs. This graph can be associated with a triple of

807: messages $(W_0,W_1,W_2)$ where $W_i \in \mc{W}_i$ for $i=1,2,3$ such

808: that $\mc{W}_0=\{1,2\}$, and $\mc{W}_1=\mc{W}_2=\{1,2,3,4\}$. Note

809: that in the figure two subgraphs $\tl{G}_1$ and $\tl{G}_2$ have

810: different edge structures. This implies that the correlation of

811: $(W_1,W_2)$ depends on $W_0$. $W_0$ and $W_1$ are independent, 

812: and so are $W_0$ and $W_2$.

813: 

814: \begin{figure}[h]

815: \centering \epsfig{file=bc-msg-graph-ex.eps, clip,

816: width=0.2\linewidth} \centering \caption{\small An example of

817: bipartite graph $G$, with parameters $(2, 4, 4, 2, 2,1)$, composed of

818: two disjoint  subgraphs $\tl{G}_1$ and $\tl{G}_2$ where $\mc{W}_0=\{1,2\}$,

819: $\mc{W}_1=\mc{W}_2=\{1,2,3,4\}$.} \label{fig:bc-msg-graph-ex}

820: \end{figure}

821: 

822: 

823: \subsection{Equivalence classes of graphs}\label{sec:bc-equivalence-classes}

824: 

825: Let us consider the set of all semi-regular bipartite

826: graphs with fixed parameters

827: $(\theta_1,\theta_2,\theta_1',\theta_2')$. It is well-known that 

828: this set can

829: be partitioned into equivalence classes where equivalence relation is

830: permutation and relabeling of the vertexes in the graphs. In more

831: detail, one element (or graph) in a class can be obtained from the

832: other in the same class by permutation and relabeling of the

833: vertexes. However, if two elements (or graphs) belong to different

834: classes, they can not be obtained from each other by permutation and

835: relabeling since they have different correlation structures.

836: 

837: This means that if we have a transmission system which can reliably

838: transmit the edges of a graph $G$ (i.e., the correlation of the

839: message pairs are modeled using $G$) then this transmission system can

840: be used to reliably transmit edges of any graph  that belongs to the

841: equivalence class of $G$. Similarly, if one can construct a source

842: representation system that can represent a source pair using a graph

843: $G$ (i.e., the correlation of the index pairs produced by the source

844: encoder is modeled using $G$), then it can be used to represent the

845: source pair using any graph that belongs to the equivalence class of

846: $G$. 

847: 

848: 

849: \section{Broadcast Channels with Correlated

850: Messages}\label{sec:general-bc}

851: 

852: In this section we characterize transmissibility of certain

853: correlated messages over a stationary discrete memoryless broadcast

854: channel.

855: 

856: \subsection{Summary of Results}

857: 

858: \begin{figure}[h]

859: \centering \epsfig{file=bc-cc-block.eps, clip=, width=.6\linewidth}

860: \centering \caption{\small Transmission of correlated messages over

861: the broadcast channel} \label{fig:bc-cc-block}

862: \end{figure}

863: 

864: Although, ideally, we would want to use semi-regular graphs for

865: source representation and communication of information over broadcast

866: channels, for the sake

867: of analytical tractability, as is typical in Shannon theory, we will

868: allow some slack with regard to the degrees of the vertexes of these

869: graphs, and consider the asymptotic case when this slack is bounded in

870: some way.

871: 

872: \begin{dfn}\label{def:bc-cc-tx-system}

873: An $(n,\tau)$-transmission system for a nearly semi-regular

874: bipartite graph $G$ with parameters $(\Delta_0$, $\Delta_1$,

875: $\Delta_2$, $\Delta_1'$, $\Delta_2'$, $\mu)$ and a broadcast channel

876: $(\mc{X},\mc{Y}_1, \mc{Y}_2,p(y_1,y_2|x))$ is an ordered tuple

877: $(f,g_1,g_2)$, consisting of one encoding mapping $f$ and two

878: decoding mappings $g_1$ and $g_2$ where

879: \begin{itemize}

880: \item $f: E(G) \ra \mc{X}^n$, i.e,\\ $\forall ((m-1)\Delta_1+i,(m-1)\Delta_2+j) \in E(G)$, assign

881:   $x^n=f(m,i,j)$ where $m \in \{1,2,\ldots,\Delta_0\}$, $i \in \{1,2,\ldots,\Delta_1\}$, and $j \in

882:   \{1,2,\ldots,\Delta_2\}$,

883: \item $g_i: \mc{Y}_i^n \ra V_i(G)$ for $i=1,2$, i.e.,

884:   $g_i: ~\mc{Y}_i^n \ra

885:   \{1,2,\ldots,\Delta_0\}\times\{1,2,\ldots,\Delta_i\}$,

886: \item such that a performance measure given by the following average

887: probability of error satisfies: \beq \tau= \frac{1}{|E(G)|} \sum_{

888: ((m-1)\Delta_1+i,(m-1)\Delta_2+j) \in E(G)} \!\! Pr \left[

889: (g_1(Y_1^n),g_2(Y_2^n))\!\!\neq\!\! ((m,i),(m,j))

890: |X^n=f(m,i,j)\right]. \eeq

891: \end{itemize}

892: \end{dfn}

893: 

894: In Definition \ref{def:bc-cc-tx-system}, the messages $W_0$, $W_1$

895: and $W_2$ are assumed to have the following distribution:

896: \begin{itemize}

897:   \item Alphabet of $W_i$ is $\{1,2,\ldots,\Delta_i\}$ for $i=0,1,2$,

898:   \item $Pr\{(W_0,W_1,W_2)=(m,i,j)\}=\left\{

899:                                        \begin{array}{cl}

900:                                          \frac{1}{E(G)}, & \hbox{if $((m-1)\Delta_1+i,(m-1)\Delta_2+j)\in E(G)$,} \\

901:                                          0, & \hbox{else.}

902:                                        \end{array}

903:                                      \right.$

904: 

905: \end{itemize}

906: 

907: \noindent

908: \begin{rem}

909: \emph{In broadcast channels, the goal of the channel encoder is

910: to reliably transmit two pairs of messages $(W_0,W_1)$ and

911: $(W_0,W_2)$ to receiver 1 and receiver 2, respectively, over the

912: channel. In terms of graphs, it is to reliably transmit edges of a

913: graph which is associated with the triple of messages

914: $(W_0,W_1,W_2)$ over the channel.

915:  As an analogy, the conventional Shannon's

916: channel coding theorem in a point-to-point communication scenario

917: can be interpreted as finding the maximum number of codewords

918: (colors, if each codeword has a different color) that are

919: distinguishable at the noisy receiver. In the conventional broadcast

920: channel with one sender and two receivers, the goal is to

921: distinguish colors at the noisy receivers, where the first color,

922: which is common to two receivers, can come from one set and the

923: second and the third colors, which are private to each receiver, can

924: come from other two sets, respectively, and all possible combination

925: of triples in the three sets are allowed. A natural question to ask

926: is: if only a fraction of all possible combination of pairs of

927: colors is permitted depending on the common color, what is the

928: maximum size of the sets of these colors which can be reliably

929: distinguished at the receivers.}

930: \end{rem}

931: 

932: \begin{dfn}

933:   A tuple of rates $(R_0,R_1,R_2,R_1',R_2')$ is said to be

934:   \emph{achievable} for a given broadcast channel with 

935: correlated messages, if for all $\e > 0$, and for sufficiently

936: large $n$,

937:   there exists a bipartite graph $G$ with parameters $(\Delta_0,

938:   \Delta_1, \Delta_2, \Delta_1', \Delta_2',\mu)$ and an associated 

939:   $(n,\tau)$-transmission system as defined above satisfying:

940: $\frac{1}{n}\log \Delta_0 > R_0-\e$,  $\frac{1}{n}\log \Delta_i >

941: R_i-\e$, $\frac{1}{n}\log \Delta_i' > R_i'-\e$ for $i=1, 2$,

942: $\frac{1}{n}\log \mu < \e$ and the corresponding average probability

943: of error $\tau < \e$.

944: \end{dfn}

945: 

946: 

947: Note that in the above definition, we have taken an optimistic point

948: of view. As long as one can find a sequence of nearly semi-regular

949: graphs where the number of vertexes and the degrees are increasing

950: exponentially with given rates, such that the edges from these

951: graphs are reliably transmitted over the given broadcast channel, we

952: allow the corresponding rate tuple to belong to the achievable rate

953: region. The goal is to find the capacity region $\mc{C}_{BC}$ which

954: is the set of all achievable tuple of rates $(R_0, R_1, R_2, R_1',

955: R_2')$. In the following we provide an information-theoretic

956: characterization of an achievable rate region. This is an inner

957: bound to the capacity region ${\mathcal C_{BC}}$, and is also a

958: per-letter characterization. This is one of the main results of this

959: paper. 

960: 

961: \begin{thm}\label{thm:rate-region-bc-cor}

962: For a discrete memoryless broadcast channel $(\mc{X}, \mc{Y}_1,

963: \mc{Y}_2, p(y_1,y_2|x))$, $\mc{R}_{BC}^* \subset \mc{C}_{BC}$ where

964:   \begin{align}

965: \mc{R}_{BC}^* =\bigcup_{p(z,u,v,x)}\{(R_0, R_1, R_2, R_1', R_2'):~~~& \notag\\

966: R_0 &\leq \min\{I(Z;Y_1),I(Z;Y_2)\},\\

967:     R_1 &\leq I(U;Y_1|Z), \\

968:     R_2 &\leq I(V;Y_2|Z), \\

969:     R_1 + R_2' = R_1' + R_2& \leq I(U;Y_1|Z)+I(V;Y_2|Z)-I(U;V|Z)\}

970:   \end{align}

971: where $Z$, $U$ and $V$ are auxiliary random variables with finite

972: alphabets ${\mathcal Z}$, ${\mathcal U}$ and ${\mathcal V}$,

973: respectively, and  

974: $p(z,u,v,x,y_1,y_2)=p(z)p(u,v|z)p(x|z,u,v)p(y_1,y_2|x)$

975: satisfies a Markov chain $(Z,U,V) \ra X \ra (Y_1,Y_2)$.

976: \end{thm}

977: 

978: \begin{rem}

979: \emph{

980: When the private messages are

981:   independent, i.e., when all the elements in the set $\mc{W}_1 \times 

982:   \mc{W}_2$ can occur with non-zero and equal probability, the rate region

983:   becomes exactly the same as Marton's  \cite{marton79}. 

984:   However, when the private messages are correlated, i.e., only some

985:   elements in the set $\mc{W}_1 \times 

986:   \mc{W}_2$ can occur equally likely, the sum rate $R_1+R_2$ can be

987:   larger. As the amount of correlation between the messages increases

988:   the achievable rate region also becomes larger.  }

989: \end{rem}

990: 

991: \begin{rem}

992: \emph{

993:   The limitations of this theorem are as follows. Note that this

994:   theorem gives only a partial characterization of the set of all

995:   nearly semi-regular graphs whose edges can be reliably transmitted

996:   over a broadcast channel. In the formulation of the achievable rate

997:   region, we 

998:   have the freedom of choosing the correlation of the messages for

999:   every block-length $n$. The theorem characterizes the rate of

1000:   exponential growth (as a function of the number of channel uses) of 

1001:   size of certain nearly semi-regular graphs, such that edges

1002:   coming from any such graph can be reliably transmitted over the

1003:   broadcast channel. 

1004:    This obviously also means that it is possible to transmit

1005:   edges of every graph that belongs to the equivalence class of any of

1006:   these graphs. However, this fact does not mean that the edges of any

1007:   graph with those parameters can be reliably transmitted.}

1008: \end{rem}

1009: 

1010: \subsection{Proof of Theorem

1011: {\ref{thm:rate-region-bc-cor}}}\label{sec:proof-general-bc}

1012: 

1013: In this section we prove Theorem \ref{thm:rate-region-bc-cor} by

1014: using the method of random coding, random binning, the concept of joint

1015: typicality of sequence pairs and some concepts from the theory of

1016: random graphs \cite{jansbook}.  In addition to using the

1017: techniques given in \cite{elgamal81}, we devise a concept of a

1018: ``\emph{super-bin}'', which is a group of consecutive bins, to take

1019: into account the correlation between the messages.

1020: Given a broadcast channel with distribution $p(y_1,y_2|x)$,

1021: consider a fixed joint distribution $p(z,u,v,x)$ = $p(z)$

1022: $p(u,v|z)$ $p(x|z,u,v)$ where $z,u$ and $v$ are auxiliary random

1023: variables on $\mc{Z} \times \mc{U} \times \mc{V}$. Also, fix $\e

1024: >0$, an integer $n \geq 1$, and positive real numbers $R_0$,

1025: $R_1$, and $R_2$.

1026: 

1027: \vspace{10pt}

1028: 

1029: \noindent \textbf{Random sequences and bin generation}: Draw

1030: $2^{nR_0}$ sequences $Z^n(m)$ for  $m \in \{1,2,\ldots,2^{nR_0} \}$, of

1031: length $n$, independently with replacement from $\Ae(Z)$ each with probability

1032: $\frac{1}{|\Ae(Z)|}$ where $\Ae(Z)$ is the strongly $\e$-typical

1033: set with respect to the distribution $p(z)$ which is a marginal of

1034: the joint distribution $p(z,u,v)$ and $|A|$ denotes the

1035: cardinality of a set $A$.

1036: 

1037: For each $m \in \{1,2,\ldots, 2^{nR_0}\}$, draw

1038: $2^{n(I(U;Y_1|Z)-\e)}$ $n$-length sequences  

1039: $U^n(k,m)$ for $k \in \{1,2,\ldots, 2^{n(I(U;Y_1|Z)-\e)}\}$

1040: independently with replacement from  $A(U|Z^n(m))$ with probability

1041: $\frac{1}{|A(U|Z^n(m))|}$. Call this collection as $\C_1(m)$.

1042: Here, $A(U|z^n)$ is the set of

1043: $n$-sequences $u^n$ which are strongly jointly typical with the

1044: sequence $z^n$, i.e., for $z^n \in A_{\epsilon}^{(n)}(Z)$, the set 

1045: $A(U|z^n) \triangleq \{u^n|(u^n,z^n)\in \Ae(U,Z)\}$.

1046: Similarly, generate $2^{n(I(V;Y_2|Z)-\e)}$ 

1047: sequences $V^n(l,m)$ for $l \in$  $\{1,2,\ldots,$ $2^{n(I(V;Y_2|Z)-\e)}\}$ 

1048: independently with replacement from  $A(V|Z^n(m))$, where 

1049: for $z^n \in A_{\epsilon}^{(n)}$, the set  $A(V|z^n) \triangleq

1050: \{v^n|(v^n,z^n)\in \Ae(V,Z)\}$. Call this collection as $\C_2(m)$.

1051: Without loss of generality $2^{n(I(U;Y_1|Z)-\e)}$ and

1052: $2^{n(I(V;Y_2|Z)-\e)}$ are assumed to be integers.

1053: 

1054: Next, for each $m$ in $\{1,2,\ldots, 2^{nR_0}\}$, define random bins

1055: $B(i,m)$ and $C(j,m)$ for 

1056: $i \in \{1,2,\ldots, 2^{nR_1} \}$ and $j \in \{1,2,\ldots,

1057: 2^{nR_2} \}$ such that

1058: \begin{align}

1059: B(i,m) &= \{U^n(k,m)~|~ k \in [(i-1)\cdot 2^{n(I(U;Y_1|Z)-R_1-\e)}+1,

1060:   ~i\cdot2^{n(I(U;Y_1|Z)-R_1-\e)}] \} \label{bins1} \\ 

1061: C(j,m) &= \{V^n(l,m)~|~ l \in [(j-1)\cdot 2^{n(I(V;Y_2|Z)-R_2-\e)}+1,

1062: ~j\cdot2^{n(I(V;Y_2|Z)-R_2-\e)}] \}

1063: \label{bins2}

1064: \end{align}

1065: where without loss of generality $2^{n(I(U;Y_1|Z)-R_1-\e)}$ and

1066: $2^{n(I(V;Y_2|Z)-R_2-\e)}$ are considered to be integers, and

1067: $[a,b]$ denotes the set of integers from $a$ to $b$. 

1068: This imposes the following constraints on $R_1$ and $R_2$:

1069: $R_1 \leq I(U;Y_1|Z)-\e$, and  $R_2 \leq I(V;Y_2|Z)-\e$.

1070: 

1071: 

1072: \begin{figure}[h]

1073: \centering \epsfig{file=sub-graph.eps, clip=,

1074: width=.9\linewidth} \centering \caption{\small A subgraph

1075: $\tl{\mb{G}}_1$ ($m=1$) with parameters $(2^{nR_1}, 2^{nR_2},

1076: 2^{nR_1'}, 2^{nR_2'}, \mu)$. }

1077: \label{fig:bin-index-subgraph}

1078: \end{figure}

1079: 

1080: \noindent \textbf{Graph generation}: As shown in Figure

1081: \ref{fig:bin-index-subgraph}, for each $m$ in $\{1,2,\ldots,

1082: 2^{nR_0}\}$, a random graph 

1083: $\tl{\mb{G}}_m$ can be associated with the bins as follows. (1)

1084: $V_1(\tl{\mb{G}}_m)=\{(m-1)2^{nR_1}+1, \ldots, m2^{nR_1}\}$ and

1085: $V_2(\tl{\mb{G}}_m)=\{(m-1)2^{nR_2}+1, \ldots, m2^{nR_2}\}$, (2)

1086: $\forall (i,j) \in V_1(\tl{\mb{G}}_m) \times

1087: V_2(\tl{\mb{G}}_m)$, $(i,j) \in E(\tl{\mb{G}}_m)$ if and only if there

1088: exists in $B(i,m) \times C(j,m)$ at least one $\e$-strongly jointly

1089: typical sequence pair 

1090: that belongs to $A(U,V|Z^n(m))$, where 

1091: $A(U,V|z^n)$ is the set of pairs of sequences

1092: that are jointly $\e$-typical with the  sequence $z^n$, i.e., for $z^n

1093: \in \Ae(Z)$, the set  

1094: $A(U,V|z^n)$$ \triangleq \{(u^n,v^n)|(u^n,v^n,z^n)\in \Ae(U,V,Z)\}$.

1095: Let $\mb{G}$ denote the random graph that is the union of 

1096: $2^{nR_0}$ random graphs $\tl{\mb{G}}_m$ for $m

1097: \in \{1,2,\ldots,2^{nR_0}\}$.

1098: 

1099: \vspace{10pt}

1100: 

1101: \noindent \textbf{Codebook generation}: A random channel codebook $\mb{C}$

1102: can be generated from the graph $\mb{G}$ as follows. For every $m

1103: \in \{1,2,\ldots,2^{nR_0}\}$,  and every $(i,j) \in

1104: E(\tl{\mb{G}}_m)$, first find one pair of sequences $(U^n(k,m),V^n(l,m))

1105: \in A(U,V|Z^n(m)) \cap (B(i,m) \times C(j,m))$. Then draw a random 

1106: codeword $X^n(m,i,j)=f(m,i,j)$ uniformly from

1107: $A(X|U^n(k),V^n(l),Z^n(m))$, where 

1108: $A(X|u^n,v^n,z^n) \triangleq \{x^n| (x^n,u^n,v^n,z^n) \in \Ae (X,U,V,Z)\}$.

1109: Thus the size of the

1110: codebook is equal to the size of the edge set of the graph

1111: $\Bbb{G}$, i.e., $|\mb{C}|=|E(\mb{G})|$.

1112: 

1113: 

1114: \vspace{10pt} \noindent \textbf{Encoding error events due to the

1115: degree condition}: Before we proceed to the encoding and decoding

1116: procedure, we need to make sure that the generated codebooks satisfy

1117: certain properties. If vertexes of the graph $\mb{G}$ do not satisfy

1118: the degree conditions, the message pairs can not be transmitted with

1119: arbitrarily small probability of error. Let 

1120: \begin{equation}

1121: A=I(U;Y_1|Z)+I(V;Y_2|Z)-I(U;V|Z)-2 \epsilon

1122: \label{A},

1123: \end{equation}

1124: and choose $R'_1=A-R_2$ and $R'_2=A-R_1$. Since $0 \leq R'_i \leq R_i$

1125: for $i=1,2$, we have the following conditions on $R_1$ and $R_2$:

1126: \begin{equation}

1127: \max\{R_1,R_2\} \leq A \leq R_1+R_2.

1128: \label{AA}

1129: \end{equation}

1130: 

1131: In summary,  the nonnegative tuple $(R_1,R_2,R'_1,R'_2)$ that we use

1132: for the random coding  satisfies:

1133: \beq

1134: R_1 \leq I(U;Y_1|Z)-\e, \ \ \ R_1 \leq I(V;Y_2|Z)-\e, 

1135: \label{constraint1}

1136: \eeq

1137: \beq

1138: R_1+R'_2=R_2+R'_1= I(U;Y_1|Z)+I(V;Y_2|Z)-I(U;V|Z)-2 \e.

1139: \label{constraint2}

1140: \eeq

1141: 

1142: 

1143: 

1144: For a precise characterization of the error events, we need a function of 

1145: $\epsilon$, and in turn certain properties of typical sets. For any 

1146: triple $(U,V,Z)$ of finite-valued random variables, there

1147: exists \cite{csiszar-korner} a continuous positive function

1148: $\epsilon_1(\epsilon)$ (that depends on the triple)  

1149: such that (a) $\epsilon_1(\epsilon) \rightarrow 0$ as $\epsilon

1150: \rightarrow 0$ and (b) for all $\epsilon>0$ (sufficiently small), 

1151: there exists an integer $N_0(\epsilon)>0$ such that $\forall

1152: n>N_0(\epsilon)$ the following conditions hold simultaneously 

1153: \begin{itemize}

1154: \item  for all $z^n \in \Ae(Z)$,

1155: \beq

1156: 2^{n(H(U|Z) -\epsilon_1)} \leq |A(U|z^n)| \leq

1157: 2^{n(H(U|Z)+\epsilon_1)}

1158: \label{eqU}

1159: \eeq

1160: \beq

1161: 2^{n(H(V|Z) -\epsilon_1)} \leq |A(V|z^n)| \leq

1162: 2^{n(H(V|Z)+\epsilon_1)}

1163: \label{eqV}

1164: \eeq

1165: \beq

1166: 2^{n(H(U,V|Z) -\epsilon_1)} \leq |A(U,V|z^n)| \leq

1167: 2^{n(H(U,V|Z)+\epsilon_1)}

1168: \label{eqUV}

1169: \eeq

1170: \item  $\forall (u^n,z^n) \in \Ae(U,Z)$ 

1171: \beq

1172: 2^{n(H(V|U,Z) -\epsilon_1)} \leq |A(V|u^n,z^n)| \leq

1173: 2^{n(H(V|U,Z)+\epsilon_1)}

1174: \label{eqV|U}

1175: \eeq

1176: \item  $\forall (v^n,z^n) \in \Ae(V,Z)$ 

1177: \beq

1178: 2^{n(H(U|V,Z) -\epsilon_1)} \leq |A(U|v^n,z^n)| \leq

1179: 2^{n(H(U|V,Z)+\epsilon_1)}.

1180: \label{eqU|V}

1181: \eeq

1182: \end{itemize}

1183: 

1184: 

1185: Coming back to the encoding error event, an error will be

1186: declared if either one of the following events occur.

1187: Let $\e'>3\e_1>0$.

1188: \begin{itemize}

1189:   \item $E_1$: $\exists i \in V_1(\mb{G})$ such that $\left|

1190:   \frac{1}{n}\log{\rm deg}_{\mb{G},1}(i)-R_2' \right| > \e'$, 

1191:   \item $E_2$: $\exists j \in V_2(\mb{G})$ such that $\left|

1192:   \frac{1}{n}\log{\rm deg}_{\mb{G},2}(j)-R_1' \right| >   \e'$.

1193: \end{itemize}

1194: 

1195: Now we show that the probability of these error events can be made

1196: arbitrarily small under certain conditions. Let us define four events

1197: as follows. 

1198: \begin{itemize}

1199: \item $E_{0,1}$ : $\exists i \in V_1(\mb{G})$ such that

1200: ${\rm deg}_{\mb{G},1}(i) <

1201: 2^{n(R'_2-\e')}$

1202: \item $E_{0,2}$ : $\exists j \in V_2(\mb{G})$ such that ${\rm

1203: deg}_{\mb{G},2}(j) <

1204: 2^{n(R'_1-\e')}$

1205: \item $E^*_{0,1}$ : $\exists i \in V_1(\mb{G})$ such that ${\rm

1206: deg}_{\mb{G},1}(i) >

1207: 2^{n(R'_2+\e')}$

1208: \item $E^*_{0,2}$ : $\exists j \in V_2(\mb{G})$ such that ${\rm

1209: deg}_{\mb{G},2}(j) >

1210: 2^{n(R'_1+\e')}$

1211: \end{itemize}

1212: 

1213: Toward proving the required statements, we need a lemma given in

1214: \cite{krithivasan} about certain properties of typical sets.

1215: \begin{lem}\label{lem:bc-cc-0}

1216: For any triple of finite-valued random variables $(U,V,Z)$, 

1217: any $\epsilon>0$ (sufficiently small), any two positive

1218: real numbers $R_1$ and $R_2$ such that $R_1+R_2>I(U;V|Z)$,  

1219: and any  $z^n \in \Ae(Z)$ consider the following random experiment.

1220: Generate two collections of sequences $\C_U(z^n)$ and $\C_V(z^n)$ 

1221: of size $2^{nR_1}$ and $2^{nR_2}$ from $A(U|z^n)$ and

1222: $A(V|z^n)$, respectively, with uniform distribution and with

1223: replacement. Let $P_{\epsilon}(z^n,R_1,R_2)$ denote the probability that 

1224: $|(\C_U \times \C_V) \cap A(U,V|z^n)|=0$. Then $\forall \epsilon>0$, 

1225: $\forall R_1,R_2$ with $R_1+R_2>I(U;V|Z)$ and 

1226: $\forall z^n \in \Ae(Z)$, 

1227: \beq

1228: \lim_{n \rightarrow \infty} -\frac{1}{n} \log P_{\epsilon}(z^n,R_1,R_2)=

1229: \infty.

1230: \eeq

1231: \end{lem}

1232: 

1233: \noindent {\it Proof:} See Theorem 2.1 of \cite{krithivasan}.

1234: 

1235: The proofs of the next two lemmas use this result. 

1236: \begin{lem}\label{lem:bc-cc-1}

1237:   For any $\e>0$, and sufficiently large $n$:\\

1238: \begin{equation}

1239:  P\{E_{0,i}\} < \frac{\e}{12}, ~~\mbox{for $i=1,2$}

1240: \end{equation}

1241: \end{lem}

1242: 

1243: \noindent {\it Proof}: Refer to Appendix \ref{appendix:bc-lem-cc-1}.

1244: 

1245: 

1246: \begin{lem}\label{lem:bc-cc-2}

1247:   For any $\e>0$, and sufficiently large $n$:\\

1248: \begin{equation}

1249:  P\{E_{0,i}^*\} < \frac{\e}{12}, ~~\mbox{for $i=1,2$}

1250: \end{equation}

1251: \end{lem}

1252: 

1253: \noindent {\it Proof}: Refer to Appendix \ref{appendix:bc-lem-cc-2}.

1254: 

1255: Note that $E_1=E_{0,1} \cup E_{0,1}^*$ and $E_2=E_{0,2} \cup

1256: E_{0,2}^*$. Thus, according to the Lemma \ref{lem:bc-cc-1} and  

1257: \ref{lem:bc-cc-2}, it is easy to see that, for sufficiently large

1258: $n$, $P(E_1) < \frac{\e}{6}$, and  $P(E_2) < \frac{\e}{6}$,

1259: if $R_2'=I(U;Y_1|Z)-R_1+I(V;Y_2|Z)-I(U;V|Z)-2\e$ and

1260: $R_1'=I(U;Y_1|Z)-R_2+I(V;Y_2|Z)-I(U;V|Z)-2\e$, respectively.

1261: So, it is shown that with high probability we can obtain a nearly

1262: semi-regular bipartite graph $\mb{G}$ composed of $2^{nR_0}$

1263: disjoint subgraphs $\tl{\mb{G}}_m$ such that for each  $m$,  each vertex

1264: in $V_1(\tl{\mb{G}}_m)$ has degree nearly equal to $2^{nR'_2}$

1265: and each vertex in $V_2(\tl{\mb{G}}_m)$ has degree nearly equal to

1266: $2^{nR'_1}$. The size of $V_1(\tl{\mb{G}}_m)$ is $2^{nR_1}$ and 

1267: that of $V_2(\tl{\mb{G}}_m)$ is $2^{nR_2}$.

1268: 

1269: \vspace{10pt}\noindent \textbf{Choosing message correlation}: If

1270: none of the above two error events $E_1$ and $E_2$ occurs, choose $G=\mb{G}$. 

1271: Clearly  $G$ has parameters $(2^{nR_0}$,

1272: $2^{nR_1}$, $2^{nR_2}$, $2^{nR_1'}$, $2^{nR_2'}$, $2^{n\e'})$.

1273: If any of the

1274: above two error events occurs, then pick any graph with parameters

1275: $(2^{nR_0}$, $2^{nR_1}$, $2^{nR_2}$, $2^{nR_1'}$, $2^{nR_2'}$,

1276: $2^{n\e'})$ and call it as $G$.

1277: The distribution of $W_0$, $W_1$ and $W_2$ is chosen as follows.

1278: $Pr\{(W_0,W_1,W_2)=(m,i,j)\}=\frac{1}{E(G)}$ if

1279: $((m-1)2^{nR_1}+i,(m-1)2^{nR_2}+j)\in E(G)$ and

1280: $Pr\{(W_0,W_1,W_2)=(m,i,j)\}=0$ else. For this  graph $G$, and the given

1281: broadcast channel, using the above random codebook $\Bbb{C}$, we

1282: construct an $(n,\tau)$-transmission system, where $\tau$ will be

1283: specified in the sequel.

1284: 

1285: \vspace{10pt}\noindent \textbf{Encoding}: Sender transmits the

1286: codeword $X^n(m,i,j)$ over the channel to deliver two pair of

1287: messages $(m,i)$ and $(m,j)$ to receiver 1 and receiver 2,

1288: respectively.

1289: 

1290: \vspace{10pt}\noindent \textbf{Decoding}: Both Receiver 1 and 

1291: Receiver 2

1292: first find the unique index $\hat{m}$ such that $Z^n(\hat{m})$ is

1293: jointly typical with received sequence $Y_1^n$ and $Y_2^n$,

1294: respectively. Then, Receiver 1 finds the unique index $\hat{k}$ such

1295: that $U^n(\hat{k},\hat{m})$ is jointly typical with the received sequence

1296: $Y_1^n$ and $Z^n(\hat{m})$, i.e., $(U^n(\hat{k},\hat{m}), Y_1^n,

1297: Z^n(\hat{m})) \in \Ae(U,Y_1,Z)$. Similarly, Receiver 2 finds the

1298: unique index $\hat{l}$ such that 

1299: $(V^n(\hat{l},\hat{m}),

1300: Y_2^n, Z^n(\hat{m})) \in \Ae(V,Y_2,Z)$. Then, each receiver finds

1301: the decoded private messages $\hat{i}$ and $\hat{j}$ such that

1302: $U^n(\hat{k},\hat{m}) \in 

1303: B(\hat{i},\hat{m})$ and $V^n(\hat{l},\hat{m}) \in C(\hat{j},\hat{m})$,

1304: respectively. Otherwise, an error will be declared.

1305: 

1306: \vspace{10pt}\noindent \textbf{Probability of error analysis}: So,

1307: the probability of error $P(E)$ can be given by

1308: \begin{align}

1309:   P(E)&=P(E_1 \cup E_2)P(E|E_1 \cup E_2)+P(E \cap E_1^c \cap E_2^c)\\

1310:        &\leq P(E_1 \cup E_2)+P(E \cap E_1^c \cap E_2^c)

1311: \end{align}

1312: The second probability in the above equation can be bounded as given

1313: in the following lemma.

1314: 

1315: \begin{lem} \label{lem:bc-cc-3}

1316: For any $\epsilon>0$, and sufficiently large $n$, \beq P(E \cap

1317: E_1^c \cap E_2^c) < \frac{2 \epsilon}{3} \eeq provided

1318: \begin{align}

1319:   R_0 &\leq \min\{I(Z;Y_1),I(Z;Y_2) \}-\e. \\

1320: \end{align}

1321: \end{lem}

1322: {\it Proof}: Refer to Appendix \ref{appendix:bc-lem-cc-3}. \\

1323: Therefore, by applying the union bound we have 

1324: $P(E) \leq  P(E_1)+P(E_2)+P(E \cap E_1^c \cap E_2^c) \leq \e$.

1325: 

1326: 

1327: Since in every realization of random codebooks, we have chosen a

1328: nearly semi-regular graph $G$ with parameters $(2^{nR_0}$,

1329: $2^{nR_1}$, $2^{nR_2}$, $2^{nR_1'}$, $2^{nR_2'}$, $2^{n\e'})$, and

1330: averaged over the ensemble of random codebooks, the average

1331: probability of error is smaller than $\e$, there must exist a graph

1332: $G$ with parameters $(2^{nR_0}$, $2^{nR_1}$, $2^{nR_2}$,

1333: $2^{nR_1'}$, $2^{nR_2'}$, $2^{n\e'})$ and a codebook such that the

1334: average probability of error is smaller than $\e$. This is true only

1335: under the condition given by the statement of the theorem. Hence,

1336: the proof of Theorem \ref{thm:rate-region-bc-cor} has been

1337: completed. \hfill $\blacksquare$

1338: 

1339: 

1340: \subsection{The Capacity Region of a Broadcast Channel with One

1341: Deterministic Component with Correlated

1342: Messages}\label{sec:one-deterministic-bc-cor}

1343: 

1344: In this section, we consider the

1345: capacity region of broadcast channels with one deterministic

1346: component with correlated

1347: messages, $\mc{\tilde{C}}_{BC}$, when there is no common message,

1348: i.e., when $R_0=0$. In this case, as expected, we can provide a

1349: converse coding theorem.

1350: 

1351: \begin{thm}\label{thm:one-det-bc-cor}

1352: For a discrete memoryless semi-deterministic broadcast channel where

1353: $y_1=f_1(x)$ and the probability of $y_2$ given $x$ is $p(y_2|x)$,

1354: $\mc{\tilde{R}}_{BC}^* = \mc{\tilde{C}}_{BC}$ where

1355: \begin{align}

1356: \mc{\tilde{R}}_{BC}^* = \bigcup_{p(v,x)}\{(R_1, R_2, R_1',

1357:   R_2'):~~~~~& \notag\\ 

1358:   R_1 &\leq H(Y_1) \label{eq:12}\\

1359:   R_2 &\leq I(V;Y_2) \label{eq:13}\\

1360:   R_1 + R_2'= R_1'+R_2 &\leq H(Y_1|V)+I(V;Y_2)\}\label{eq:14}

1361: \end{align}

1362: for some $p(v,x)$ on ${\mathcal V} \times

1363: {\mathcal X}$, where $V$ is an  

1364: auxiliary random variable with finite alphabet  

1365: ${\mathcal V}$ such

1366: that $V \rightarrow X \rightarrow (Y_1,Y_2)$, 

1367: $|{\mathcal V}| \leq |{\mathcal X}|+2$.

1368: \end{thm}

1369: {\it Proof:} See Appendix \ref{appendix:one-det-bc-cor}.

1370: 

1371: 

1372: \section{Representation of Correlated Sources into Graphs}\label{sec:bc-sc}

1373: 

1374: In this section, we consider the problem of  representation of

1375: correlated sources into graphs for transmission over  broadcast

1376: channels, i.e., source 

1377: coding module  in broadcast channels with correlated sources.

1378: This problem can be interpreted as Gray-Wyner problem with correlated

1379: messages. In in this setup, the  source encoder can access both sources to be

1380: transmitted, but the source decoders can not collaborate with each

1381: other.  We consider two  correlated sources $(\mc{S},\mc{T},p(s,t))$.

1382: 

1383: \subsection{Summary of Results}

1384: 

1385: In this problem, the goal is to reliably represent two correlated

1386: sources into a triple of messages $(W_0,W_1,W_2)$ which can be associated

1387: with a nearly semi-regular bipartite graph $G$ with parameters

1388: $(\Delta_0, \Delta_1, \Delta_2, \Delta_1',\Delta_2', \mu)$ as

1389: defined in Definition \ref{def:bc-graph-5-parameters}. As shown in

1390: Figure \ref{fig:bc-sc-block}, the output of source encoder is the triple

1391: $W_0$, $W_1$ and $W_2$. We assume that two pairs of messages

1392: $(W_0,W_1)$ and $(W_0,W_2)$ are sent to receiver 1 and receiver 2,

1393: respectively, over the channel without error. From the received

1394: message pairs, the two source decoders wish to reliably reconstruct the

1395: original source sequences $S^n$ and $T^n$, respectively, without

1396: communicating with each other.

1397: For ease of exposition let is consider two simple functions

1398: $I_1$ and $I_2$ both having domain as $\Bbb{Z} \times \Bbb{Z}$ and 

1399: range as $\Bbb{Z}$ given by $I_1(a,b)=a$ and $I_2(a,b)=b$ for all

1400: $(a,b) \in \Bbb{Z} \times \Bbb{Z}$. 

1401: 

1402: \begin{figure}[h]

1403: \centering \epsfig{file=bc-sc-block.eps, clip=, width=.5\linewidth}

1404: \centering \caption{\small  Gray-Wyner problem with  correlated

1405: messages} \label{fig:bc-sc-block}

1406: \end{figure}

1407: 

1408: \begin{dfn}

1409: An $(n,\tau)$-transmission system for a nearly semi-regular

1410: bipartite graph $G$ with parameters $(\Delta_0$, $\Delta_1$,

1411: $\Delta_2$, $\Delta_1'$, $\Delta_2'$, $\mu)$ and a pair of

1412: correlated sources $({\mathcal S},{\mathcal T},p(s,t))$ 

1413: is an ordered

1414: tuple $(f,g_1,g_2)$, consisting of one encoding mapping $f$ and two

1415: decoding mappings $g_1$ and $g_2$ where

1416: \begin{itemize}

1417: \item $f: \mc{S}^n \times \mc{T}^n \ra E(G)$,

1418: \item $ g_1: V_1(G) \ra \mc{S}^n$, $g_2: V_2(G) \ra \mc{T}^n$,

1419:  \item such that a performance measure given by the

1420: probability of error satisfies:

1421: \begin{align}

1422: \tau&=Pr\{(g_1(I_1(f(S^n, T^n))),g_2(I_2(f(S^n, T^n)))) \!\neq\! (S^n,T^n) \}.

1423: \end{align}

1424: \end{itemize}

1425: \end{dfn}

1426: 

1427: \noindent Now we define achievable rates for this problem as follows.

1428: \begin{dfn}

1429:   A tuple of rates $(R_0,R_1,R_2,R_1',R_2')$ is said to be

1430:   \emph{achievable} for a pair of correlated sources

1431:   $(\mc{S},\mc{T},p(s,t))$ (for 

1432:   transmission over broadcast channels), if for all

1433: $\e > 0$, and for all sufficiently large $n$,

1434:   there exists a bipartite graph $G$ with parameters

1435:   $(\Delta_0,\Delta_1, \Delta_2, \Delta_1', \Delta_2',\mu)$ and an

1436:   associated

1437:   $(n,\tau)$-transmission system as defined above satisfying:

1438:   $\frac{1}{n}\log \Delta_i < R_i + \e$ for $i=0,1,2$,

1439:   $\frac{1}{n}\log \Delta_j' < R_j' + \e$ for $j=1, 2$,

1440:   $\frac{1}{n}\log \mu < \e$ 

1441:   and the corresponding probability of error $\tau < \e$.

1442: \end{dfn}

1443: 

1444: The goal is to find the achievable rate region $\mc{R}_{GW}$ which

1445: is the set of all achievable tuple of rates $(R_0, R_1, R_2, R_1',

1446: R_2')$. We have obtained an inner bound to the achievable rate region. It

1447: is one of the main results of this paper and is given by the following theorem.

1448: 

1449: \begin{thm}\label{thm:bc-sc-main}

1450: $\mc{R}_{GW}^* \subset \mc{R}_{GW}$ where

1451: \begin{align}

1452: \mc{R}_{GW}^* =\bigcup_{p(z|s,t)}\{(R_0, R_1, R_2, R_1', R_2')&: \notag\\

1453:  R_0 &\geq I(S,T;Z), \label{eq:bc-sc-main1}\\

1454:  R_1 &\geq H(S|Z),  \ \ \

1455:  R_2 \geq H(T|Z), \label{eq:bc-sc-main3}\\

1456:  R_1' &\geq H(S|T,Z), \ \ \

1457:  R_2' \geq H(T|S,Z), \label{eq:bc-sc-main5}

1458: \end{align}

1459: where $Z$ is an auxiliary random variable with a finite alphabet 

1460: $\mc{Z}$ such that $p(z,s,t)=p(s,t)p(z|s,t)$.

1461: 

1462: \end{thm}

1463: 

1464: \begin{rem}

1465: Note that we can obtain $R_0+R_1+R_2'= R_0+R_1'+R_2 \geq H(S,T)$ by

1466: combining the conditions in Theorem \ref{thm:bc-sc-main}. This

1467: means that in every nearly semi-regular bipartite graph 

1468: used to  represent the source, as expected,   

1469: the total number of edges  must be greater than or equal to $2^{nH(S,T)}$.

1470: \end{rem}

1471: 

1472: \subsection{Proof of Theorem

1473: \ref{thm:bc-sc-main}}\label{sec:bc-sc-thm-proof}

1474: 

1475: In this section, we present a proof of Theorem

1476: \ref{thm:bc-sc-main}. We use a random coding procedure and the notion of

1477: strongly jointly typical sequences.

1478: Let us consider a fixed finite set ${\mathcal Z}$, and a 

1479: joint probability distribution $p(z,s,t)$ on

1480: $\mc{Z} \times \mc{S} \times \mc{T}$.  Also fix $\e>0$, and real

1481: numbers $R_0$, $R_1$, $R_2$, $R'_i=R_i-I(S;T|Z)$ for $i=1,2$.

1482: Without loss of generality we assume that 

1483: $R_i>I(S;T|Z)$ for $i=1,2$.

1484: 

1485: 

1486: \vspace{10pt} \noindent \textbf{Random sequence generation}: First,

1487: draw $2^{nR_0}$ sequences $Z^n(m)$, $m \in \{1, 2$, $\ldots,

1488: 2^{nR_0}\}$, of length $n$, independently from the strongly

1489: $\e$-typical set $\Ae(Z)$. That is,

1490: $P\{Z_m^n=z^n\}=\frac{1}{|\Ae(Z)|}$ if $z^n \in \Ae(Z)$, and

1491: $P\{Z_m^n=z^n\}=0$ if $z^n \notin \Ae(Z)$. For every $m \in \{1,2,

1492: \ldots, 2^{nR_0}\}$, draw $2^{nR_1}$  sequence  from $\Ae(S|Z^n(m))$ 

1493: independently, equally likely and with replacement. Call this

1494: collection $B(m)$. Denote the $i$th sequence as $S^n(i,m)$. 

1495: By collecting  these $2^{nR_0}$ bins the first codebook $\Bbb{C}_1$ can be

1496: obtained. In other words, $\Bbb{C}_1=\{B(1),B(2), \ldots,

1497: B(2^{nR_0})\}$.

1498: 

1499: Similarly, for every  $m \in \{1,2,\ldots,2^{nR_0}\}$, 

1500: construct  a bin $C(m)$ containing $2^{nR_2}$ sequences from 

1501: $\Ae(T|Z^n(m))$. Let the $j$th sequence in $C(m)$ be denoted by $T^n(j,m)$.

1502: The second codebook $\Bbb{C}_2$ can be

1503: similarly generated, i.e., $\Bbb{C}_2=\{C(1),C(2), \ldots,

1504: C(2^{nR_0})\}$.

1505: 

1506: \vspace{10pt} \noindent \textbf{Graph generation}: Now we generate a

1507: bipartite graph $\Bbb{G}$ from the codebooks $\Bbb{C}_1$ and

1508: $\Bbb{C}_2$ as follows:

1509: \begin{itemize}

1510:   \item $\Bbb{G}$ is composed of $2^{nR_0}$ disjoint subgraphs

1511:   $\tl{\Bbb{G}}_m$ for $m \in \{1, 2, \ldots, 2^{nR_0}\}$,

1512:   \item $V_i(\Bbb{G})\!=\!\cup_{m=1}^{2^{nR_0}}V_i(\tl{\Bbb{G}}_m)$

1513:   for $i=1,2$, \ \ \ 

1514:   $E(\Bbb{G})=\cup_{m=1}^{2^{nR_0}}E(\tl{\Bbb{G}}_m)$,

1515:   \item $\forall m \!\in \!\{1, 2, \ldots, 2^{nR_0}\}$,

1516:   $V_1(\tl{\Bbb{G}}_m)\!=\!\{(m-1)2^{nR_1}-1, \ldots, m2^{nR_1}\}$,  

1517: $V_2(\tl{\Bbb{G}}_m)$=$\{(m-1)2^{nR_2}-1, \ldots, m2^{nR_2}\}$,

1518:   \item $\forall m \!\in \!\{1, 2, \ldots, 2^{nR_0}\}$,

1519:                $(i,j) \in E(\tl{\Bbb{G}}_m)$ if and only if 

1520: $(S^n(i,m)$, $T^n(j,m)$, $Z^n(m))$ $\in

1521:          \Ae(S,T,Z)$.

1522: \end{itemize}

1523: 

1524: \vspace{10pt} \noindent \textbf{Encoding error events}: Before we

1525: proceed further, let us make sure that the generated codebooks

1526: satisfy certain properties. If the vertexes of $\Bbb{G}$ do not

1527: satisfy certain degree requirements, we may not be able to reliably

1528: represent the sources using this graph. 

1529: For the triple $(S,T,Z)$, consider the function $\epsilon_1(\e)$ as 

1530: defined in (\ref{eqU})-(\ref{eqU|V}). Let $\e'>3\e_1>0$. 

1531: An encoding error will be declared if either one of the following

1532: events occurs. \\

1533: $E_1: \exists i \in V_1(\Bbb{G}) \mbox{ such } \mbox{ that }

1534: | \frac{1}{n} \log {\rm deg}_{\Bbb{G},1}-R'_2 | > \e'$, \\

1535: $E_2: \exists j \in V_2(\Bbb{G}) \mbox{ such } \mbox{ that }

1536: | \frac{1}{n} \log {\rm deg}_{\Bbb{G},2}-R'_1 | > \e'$. 

1537: 

1538: Using the following lemma it can be shown that the probability of

1539: these two events can be made arbitrarily small for large $n$. 

1540: 

1541: \begin{lem}\label{lem:bc-sc-1}

1542: For any $\e>0$, and sufficiently large $n$,

1543: we have 

1544: $P(E_i)< \frac{\e}{8}, \ \ {\rm for } \ \ i=1,2.$

1545: \end{lem}

1546: {\it Proof:} The proofs of these two results are similar,

1547: respectively, to those of Lemma

1548: \ref{lem:bc-cc-1} and Lemma \ref{lem:bc-cc-2}. Hence for conciseness

1549: we omit the proof.

1550: 

1551: So by Lemma \ref{lem:bc-sc-1}, $P(E_1)+P(E_2) < \frac{\e}{4}$. 

1552: Hence we can obtain a bipartite graph

1553: $\Bbb{G}$ where each vertex in $V_1(\Bbb{G})$ has degree nearly

1554: equal to $ 2^{nR'_2}$ and each vertex in $V_2(\Bbb{G})$ has

1555: degree nearly equal to $ 2^{nR'_1}$.

1556: 

1557: \vspace{10pt} \noindent \textbf{Choosing message correlation}: 

1558: If none of the above error events $E_1$ and $E_2$ occurs, then choose

1559: $G=\Bbb{G}$. If any of the above two error events occurs, then pick

1560: any graph with parameters

1561: $(2^{nR_0},2^{nR_1},2^{nR_2},2^{nR'_1},2^{nR'_2},2^{n\e'})$

1562: and call it as $G$ and no guarantee will be given regarding the

1563: probability of decoding error. 

1564: For this graph $G$, and the given correlated sources $S$ and $T$, using

1565: the above random codebooks $\Bbb{C}_1$ and $\Bbb{C}_2$, we construct

1566: an $(n,\tau)$-transmission system, where $\tau$ will be specified in

1567: the sequel.

1568: 

1569: \vspace{10pt} \noindent \textbf{Encoding}: If $(E_1\cup E_2)$ occurs,

1570: then the encoder is some arbitrary mapping $\mc{S}^n \times \mc{T}^n

1571: \rightarrow E(G)$. Otherwise, 

1572: for a given $(S^n,T^n)

1573: \!\in\! \Ae(S,T)$, find an index $m$, for $m \in \{1, 2,$ $\ldots,

1574: 2^{nR_0}\}$, such that $(S^n$, $T^n$, $Z^n(m))$ $\in \Ae(S,T,Z)$. If

1575: there is no such index $m$, let $m$ be a random index chosen uniformly

1576: from $\{1,2,\ldots,2^{nR_0}\}$. Also, find a pair of

1577: indexes $(i,j)$ where $i$ (and $j$, respectively) is the index such

1578: that $S^n(i,m)=S^n$ (and $T^n(j,m)=T^n$, respectively). If there is no

1579: such sequence pair let $(i,j)$ be a random edge from the corresponding

1580: sub-graph of $G$. The encoder sends $(m,i)$ and $(m,j)$ to

1581: receiver 1 and receiver 2, respectively.

1582: 

1583: \vspace{10pt} \noindent \textbf{Decoding}: 

1584: Given the received

1585: index pair $(m,i)$, receiver 1 declares $\h{S}^n=S^n(i,m)$. Similarly,

1586: given $(m,j)$, receiver 2 declares 

1587: $\h{T}^n=T^n(j,m)$.

1588: 

1589: \vspace{10pt} \noindent \textbf{Probability of error analysis}: Let

1590: $E$ denote the event $\{ (g_1(m,i),g_2(m,j)) \!\neq \!(S^n, T^n)\}$,

1591: that the reconstruction vector pair is not equal to the source

1592: vector pair. The probability of error $P(E)$ can be given by

1593: \begin{align}

1594:   P(E)&=P(E_1 \!\cup\! E_2)P(E|E_1 \! \cup \! E_2)\!+\!P(E \cap E_1^c

1595:        \cap E_2^c)\\ 

1596:        &\leq P(E_1 \!\cup\! E_2)+P(E \cap E_1^c \cap E_2^c).

1597: \end{align}

1598: 

1599: The second probability in the above equation can be bounded as

1600: given in the following lemma.

1601: 

1602: \begin{lem}\label{lem:bc-sc-3}

1603:   For any $\e>0$, and sufficiently large $n$,

1604:   \beq P(E \cap E_1^c \cap E_2^c) < \frac{3\e}{4} \eeq

1605:   provided

1606:   \beq

1607:     R_0 > I(S,T;Z) + \e_2(\e), \ \ 

1608:     R_1 > H(S|Z) + \e_1(\e), \ \ 

1609:     R_2 > H(T|Z) + \e_1(\e),

1610:   \eeq

1611:   where $\e_2(\e) \ra 0$ as $\e \ra 0$ and $\e_2(\e)>0$.

1612: \end{lem}

1613: {\it Proof}: Refer to Appendix \ref{appendix:bc-lem-sc-3}.

1614: 

1615: Thus, $P(E)  \leq P(E_1)\!+\!P(E_2)\!+\! P(E \cap E_1^c \cap E_2^c) < \e$.

1616: Therefore $P(E) < \e$ for sufficiently large $n$ and under the

1617: conditions given by the theorem. In every realization of random

1618: codebooks we have obtained a graph $G$ with the same set of parameters, and

1619: averaged over this ensemble, we have made sure that the probability

1620: of error is within the tolerance level of $\e$. Hence, the proof of

1621: the direct coding theorem is completed. \hfill $\blacksquare$

1622: 

1623: 

1624: \subsection{Outer bound}\label{sec:bc-sc-thm-proof}

1625: 

1626: In this section we provide a  partial converse for  Theorem

1627: \ref{thm:bc-sc-main}.

1628: 

1629: \begin{thm}\label{thm:bc-sc-main_c}

1630: $\mc{R}_{GW} \subset  \mc{R}_{GW}^{**}$ where

1631: \begin{align}

1632: \mc{R}_{GW}^{**} =\bigcup_{p(w|s,t)}\{(R_0, R_1, R_2, R_1', R_2')&: \notag\\

1633:  R_0 &\geq I(S,T;Z), \label{eq:bc-sc-main1a}\\

1634:  R_1 &\geq H(S|Z), \ \ \

1635:  R_2 \geq H(T|Z), \label{eq:bc-sc-main3a}\\

1636: R_1+R_2'&= R_1'+R_2 \geq H(S,T|Z)\label{eq:bc-sc-main6a}\},

1637: \end{align}

1638: where $Z$ is an auxiliary random variable such that $p(z,s,t)=p(s,t)p(z|s,t)$.

1639: \end{thm}

1640: {\it Proof:} Refer to Appendix \ref{appendix:bc-sc-main_c}.

1641: 

1642: 

1643: \subsection{An interpretation of the rate region}\label{sec:different}

1644: 

1645: We have shown that for sufficiently large

1646: block length $n$, a pair correlated sources $(S,T)$ can be

1647: represented into  a nearly semi-regular graph $G$ with parameters

1648: $(2^{nR_0}$, $2^{nR_1}$, $2^{nR_2}$, $2^{nR_1'}$, $2^{nR_2'}$,

1649: $2^{n\e'}$) as shown in Figure \ref{fig:bc-message-graph}.

1650: \begin{figure}[!h]

1651: \begin{center}

1652: \psfig{figure=sub-graph_source.eps,height=2.0in,width=3.6in}

1653: \end{center}

1654: \centering \caption{\small Graph $G$ with parameters $(2^{nR_0}$,

1655: $2^{nR_1}$, $2^{nR_2}$, $2^{nR_1'}$, $2^{nR_2'}$, $2^{n\e'})$

1656: composed of $2^{nR_0}$ subgraphs $\tl{G}_m$ with parameters 

1657: ($2^{nR_1}$, $2^{nR_2}$,

1658: $2^{nR_1'}$, $2^{nR_2'}$, $2^{n\e'}$)}\label{fig:bc-message-graph}

1659: \end{figure}

1660: Note that many different graphs can be used to represent a pair

1661: correlated sources without increasing the amount of ``global''

1662: redundancy, i.e., the sum rate satisfies:

1663: $R_0+R_1+R_2'=H(S,T)$. This can be seen from the fact that the total

1664: number of edges in 

1665: the graph satisfies:  $|E(G)|\approx 2^{n(I(S,T;Z)+H(S,T|Z))}=2^{nH(S,T)}$.

1666: Further, the rate of the common message can vary from $0$ to $H(S,T)$.

1667: As examples, let us consider some special cases as follows.

1668: \begin{itemize}

1669:   \item Case $A$: If $I(S,T;Z)=0$, then $R_0=0$, $R_1=H(S)$, $R_2=H(T)$,

1670:   $R_1'=H(S|T)$, $R_2'=H(T|S)$. Roughly speaking, this

1671:   corresponds to the typicality-graph of $(S,T)$.

1672:   \item Case $B$: If $I(S,T;Z)=H(S,T)$, then $R_0=H(S,T)$, $R_1=0$, $R_2=0$,

1673:   $R_1'=0$, $R_2'=0$.

1674: \end{itemize}

1675: In particular, consider the case when  $R_0=C(S;T)$, where $C(S;T)$

1676: denotes the common information of  Wyner

1677: \cite{wyner75}. 

1678: In this case, $R_0=C(S;T)$, $R_1=H(S|Z)$, $R_2=H(T|Z)$,

1679:   $R_1'=H(S|Z)$, $R_2'=H(T|Z)$ since $S$ and $T$ are conditionally

1680: independent given $Z$. So, each subgraph $\tl{G}_m$ becomes

1681:  nearly complete. This also means that the private messages for receiver 1

1682: and 2 become nearly independent. Note that a subgraph $\tl{G}_m$ can

1683: not be complete if $R_0 < C(S;T)$ because, by the definition,

1684: $C(S;T)$ is the infimum of $I(S,T;Z)$ such that $S \ra Z \ra T$.

1685: 

1686: In summary, Case $A$ can be thought of as situated at one end of the

1687: spectrum, and Case $B$ as situated on the other end of the spectrum.

1688: For every value of $I(S,T;Z)$, we get an equally efficient

1689: representation of the sources into a nearly semi-regular graph.

1690: Here, an efficient representation means that the cardinality

1691: of the edge set of a graph used for representation (of $n$ source

1692: samples) is nearly equal to  $2^{nH(S,T)}$.

1693: 

1694: 

1695: \section{Interpretation of the Example in Section

1696:   \ref{sec:bc-ex}}\label{sec:bc-example} 

1697: 

1698: In this section, we revisit the example of Section \ref{sec:bc-ex}.

1699: We analyze and

1700: interpret this example from the perspective  of graphs and the

1701: proposed coding scheme. This example deals with  transmission of

1702: two correlated sources over a deterministic broadcast channel. This

1703: example can be considered as a special case where the

1704: typicality-graph of the sources and that of channel outputs can

1705: match exactly. So, we can apply our coding theorem to this case.

1706: 

1707: Consider the pair of correlated sources of Section \ref{sec:bc-ex}:

1708: $\mc{S}=\mc{T}=\{0, 1\}$ with distribution  $p(0,0)=p(0,1)=p(1,1)=1/3$, and the

1709: Blackwell channel with ${\mathcal X}=\{1,2,3\}$, and ${\mathcal

1710:   Y}_i=\{0,1\}$ for $i=1,2$ with conditional distribution 

1711: $p(0,0|1)=p(0,1|2)=p(1,1|3)=1$.  Let $\alpha=h_2(1/3)$ and $\beta=\log 3

1712: -h_2(1/3)$. It follows that $H(S)=H(T)=\alpha$

1713: and $H(S|T)=H(T|S)=\beta$. Consider the rate tuple

1714: $(0,\alpha,\alpha,\beta,\beta)$. Clearly this point belongs to

1715: ${\mathcal R}^*_{GW}$ as this tuple corresponds to the

1716: typicality-graph of $({\mathcal S},{\mathcal T},p(s,t))$. 

1717: Note that the above broadcast channel is a deterministic broadcast

1718: channel. Using the arguments presented in Section

1719: \ref{sec:one-deterministic-bc-cor}, it can be shown that for any

1720: deterministic broadcast channel, for the special case when $R_0=0$,  

1721: the capacity region $\tilde{{\mathcal C}}_{BC}$ is given by 

1722: 

1723: \beq

1724: \bigcup_{p(x)} \{(R_1,R_2,R'_1,R'_2): \ \

1725: R_i \leq H(Y_i) \ \ \mbox{for} \ \ i=1,2, \ \  

1726: R_1+R'_2=R_2+R'_1 \leq H(Y_1,Y_2) \}.

1727: \eeq

1728: 

1729: Choosing $X$ equally likely over $\{1,2,3\}$, it follows that 

1730: the rate tuple $(\alpha,\alpha,\beta,\beta)$ belongs to 

1731: $\tilde{{\mathcal C}}_{BC}$, which, of course,  implies that 

1732: $(0,\alpha,\alpha,\beta,\beta)$ belongs to 

1733: ${\mathcal C}_{BC}$. In this case the distribution of the channel

1734: output $(Y_1,Y_2)$ is the same as the distribution of the sources

1735: $(S,T)$. Further, the graph corresponding to the tuple

1736: $(0,\alpha,\alpha,\beta,\beta)$ in source coding is the same as the

1737: graph corresponding to the tuple $(0,\alpha,\alpha,\beta,\beta)$ in channel

1738: coding.  

1739: 

1740: In summary: (1) there is a

1741: nonempty intersection between the achievable rate regions of the source

1742: coding module and the channel coding module, and (2) the graph $G$

1743: associated with the source coding module matches (is identical) with the

1744: graph $G$ of channel coding module. 

1745: Hence the given sources $(S,T)$

1746: can be reliably sent over the given broadcast channel.

1747: 

1748: \section{Conclusion}\label{sec:bc-conclusion}

1749: We have considered the problem of transmission of correlated sources over

1750: broadcast channels. We have considered a graph-based modular architecture

1751: involving two components: a channel coding component and a source

1752: coding component.  The graphs are used to model the correlation between the

1753: messages. Correlated sources 

1754: are first mapped into such graphs, and the edges coming from these

1755: graphs are reliably transmitted over a broadcast channel.

1756: 

1757: We have given a partial characterization of the set of all graphs

1758: that can be used to represent a given pair of correlated sources,

1759: and similarly given a partial characterization of the set of all

1760: graphs such that edges coming those graphs are reliably transmitted

1761: over a given broadcast channel.

1762: We have also considered  special cases such as 

1763:  deterministic broadcast channels and broadcast channels with one

1764:  deterministic component, where converse results are provided. We 

1765: have applied this analysis to the case of transmission of a triangular source

1766: over a Blackwell channel as an example. The goal of

1767: this work is to show that graphs may be used as discrete interface

1768: in this modular approach to multiterminal communication problems.

1769: 

1770: 

1771: \appendix

1772: 

1773: 

1774: \section*{Appendix}

1775: 

1776: \section{Proof of Lemma \ref{lem:bc-cc-1}}\label{appendix:bc-lem-cc-1}

1777: The event $E_{0,1}$ can be considered as

1778: \begin{equation}

1779: E_{0,1}=\bigcup_{i=1}^{2^{nR_1}} \bigcup_{m=1}^{2^{nR_0}} E_{0,1}(i,m)

1780: \end{equation}

1781: where  $E_{0,1}(i,m)$ is the event that 

1782: ${\rm deg}_{\mb{G},1}((m-1)2^{nR_1}+i) < 2^{n(R'_2-\e')}$.

1783: 

1784: Note that  each vertex in $V_1(\mb{G})$ is only connected with a

1785: subset of $V_2(\tl{\mb{G}}_m)$ for some $m$, $m \in

1786: \{1,2,\ldots,2^{nR_0}\}$.  

1787: We define super-bins $\mf{B}(p,m)$ and $\mf{C}(q,m)$ for

1788: $p\in [1,2^{n(R'_1-\epsilon')}]$ and $q\in[1,2^{n(R'_2-\epsilon')}]$, each of

1789: which is a union of $2^{n(R_1+R_2-A+\epsilon')}$ consecutive bins $B(i,m)$ and

1790: $C(j,m)$, respectively, i.e.,

1791: \begin{align}

1792:   \mf{B}(p,m)=\bigcup_{i=(p-1)2^{n(R_1+R_2-A+\epsilon')}+1}

1793: ^{p\,2^{n(R_1+R_2-A+\epsilon')}} B(i,m),\\

1794:   \mf{C}(q,m)=\bigcup_{j=(q-1)2^{n(R_1+R_2-A+\epsilon')}+1}

1795: ^{q2^{n(R_1+R_2-A+\epsilon')}} C(j,m)

1796: \end{align}

1797: The size of each super-bin $\mf{B}(p,m)$ and

1798: $\mf{C}(q,m)$ is $2^{n(I(U;Y_1|Z)-\epsilon-R'_1+\epsilon')}$ and

1799: $2^{n(I(V;Y_2|Z)-\epsilon-R'_2+\epsilon')}$, respectively. Recall that 

1800: $R_1+R'_2=R_2+R'_1=A$. 

1801: 

1802: Before we proceed further, let us observe that 

1803:  $B(i,m)$, $\mf{B}(p,m)$ and $\C_1(m)$ are

1804: collections of $2^{n(I(U;Y_1|Z)-\epsilon-R_1)}$,  

1805: $2^{n(I(U;Y_1|Z)-\epsilon-R'_1+\epsilon')}$ and 

1806: $2^{n(I(U;Y_1|Z)-\epsilon)}$, respectively, of random sequences.

1807: Then, the event $E_{0,1}(i,m)$ can be expressed as

1808: \begin{equation}

1809: E_{0,1}(i,m) \subset \bigcup_{q=1}^{2^{n(R'_2-\e')}} E_{0,1}(i,q,m)

1810: \end{equation}

1811: where  $E_{0,1}(i,q,m)$ is the event that $(B(i,m) \times

1812: \mf{C}(q,m)) \cap A(U,V|Z^n(m))$ is empty.  So, 

1813: by using the union bound, the probability of this event

1814: $P(E_{0,1}(i,m))$ can be bounded as:

1815: \begin{align}

1816:   P(E_{0,1}(i,m)) &\leq P \left( \bigcup_{q=1}^{2^{n(R'_2-\e')}}

1817:   E_{0,1}(i,q,m) \right)\\ 

1818:   &\leq \sum_{q=1}^{2^{n(R'_2-\e')}} P(E_{0,1}(i,q,m))\\

1819:   &\leq \sum_{q=1}^{2^{n(R'_2-\e')}} \sum_{z^n \in \Ae(Z)}

1820:   \frac{1}{|\Ae(Z)|} P(E_{0,1}(i,q,m)|Z^n(m)=z^n)

1821: \end{align}

1822: Therefore, for sufficiently large $n$, the probability of the event

1823: $E_{0,1}$ can be bounded by applying the union bound:

1824: \begin{align}

1825:  P(E_{0,1}) &\leq \sum_{i=1}^{2^{nR_1}} \sum_{m=1}^{2^{nR_0}} P(E_{0,1}(i,m))\\

1826:             &\leq \sum_{i=1}^{2^{nR_1}} \sum_{m=1}^{2^{nR_0}} 

1827:  \sum_{q=1}^{2^{n(R'_2-\e')}} \sum_{z^n \in \Ae(Z)}

1828:   \frac{1}{|\Ae(Z)|} P(E_{0,1}(i,q,m)|Z^n(m)=z^n) \\

1829:               &\overset{(a)}\leq 2^{n(R_0+R_1+R'_2-\e')} 2^{-nM}\\

1830:               &{\leq} \frac{\e}{12},

1831: \end{align}

1832: where (a) is from Lemma \ref{lem:bc-cc-0} because

1833: \beq

1834: \frac{1}{n} \log |B(i,m)||\mf{C}(q,m)|=I(U;V|Z)+\epsilon',

1835: \eeq 

1836:  and $M > 0$ is a

1837: sufficiently large number satisfying $M > R_0+R_1+R'_2-\e'$.

1838: 

1839: In a similar way, we can also show that $P(E_{0,2})

1840: <\frac{\e}{12}$ for sufficiently large $n$.

1841:  \hfill $\blacksquare$

1842: 

1843: 

1844: 

1845: \section{Proof of Lemma \ref{lem:bc-cc-2}}\label{appendix:bc-lem-cc-2}

1846: 

1847: The event $E_{0,1}^*$ can be expressed as

1848: \begin{equation}

1849: E_{0,1}^*=\bigcup_{i=1}^{2^{nR_1}} \bigcup_{m=1}^{2^{nR_0}} E_{0,1}^*(i,m)

1850: \end{equation}

1851: where  $E_{0,1}^*(i,m)$ is the event that 

1852: ${\rm deg}_{\mb{G},1}((m-1)2^{nR_1}+i) > 2^{n(R'_2+\e')}$.

1853: 

1854: Let $\mc{D}(i,m)=\{j:  |A(U,V|Z^n(m))\cap (B(i,m) \times C(j,m))|

1855: \neq 0 \}$. Then, 

1856: \begin{align}

1857:   |\mc{D}(i,m)|=\sum_{j=1}^{2^{nR_2}} \psi(j),

1858:   \mbox{  where} ~~\psi(j)=\left\{%

1859: \begin{array}{ll}

1860:     1, & \hbox{if} \ \ |A(U,V|Z^n(m))\cap (B(i,m) \times C(j,m))| \neq 0, \\

1861:     0, & \hbox{otherwise.} 

1862: \end{array}%

1863: \right.

1864: \end{align}

1865: 

1866: In particular,

1867: \begin{align}

1868:   P\{\psi(j)=1\}&= P \{ |A(U,V|Z^n(m)) \cap

1869:   (B(i,m) \times C(j,m))| \neq 0 \}\\

1870:  &= \sum_{z^n \in \Ae(Z)} \frac{1}{|\Ae(Z)|} P \{ |A(U,V|Z^n(m)) \cap

1871:   (B(i,m) \times C(j,m))| \neq 0 | Z^n(m)=z^n\}\\

1872:    &\overset{(a)}{\leq} \sum_{z^n \in \Ae(Z)} \frac{1}{|\Ae(Z)|}

1873: 2^{n(I(U;Y_1|Z)-R_1-\e)}

1874:   2^{n(I(V;Y_2|Z)-R_2-\e)} 2^{-n(I(U;V|Z)-3\e_1)}\\ 

1875:   &= 2^{n(A-R_1-R_2+3\e_1)}

1876: \end{align} 

1877: where (a) is obtained by applying the union bound, and

1878: from the property of strongly jointly $\e$-typical sequences

1879: \cite{cover-thomas}: for all $z^n \in \Ae(Z)$, for a randomly and

1880: independently chosen $U^n 

1881: \in A(U|z^n)$ and $V^n \in A(V|z^n)$, for sufficiently large

1882: $n$, the probability that

1883:  $(U^n, V^n) \in A(U,V|z^n)$ is bounded by

1884: \begin{equation}\label{eq:bc-cc-3}

1885: 2^{-n(I(U,V|Z)+3\e_1)} \leq P\{(U^n, V^n) \in A(U,V|z^n)\} \leq

1886: 2^{-n(I(U,V|Z)-3\e_1)}

1887: \end{equation} 

1888: where we have used (\ref{eqU}), (\ref{eqV}) and (\ref{eqUV}). 

1889: 

1890: 

1891: So, the expectation of $|\mc{D}(i,m)|$ can be bounded as follows.

1892: \begin{align}

1893:   E|\mc{D}(i,m)| &=\sum_{j=1}^{2^{nR_2}} P\{\psi(j)=1\}\\

1894:              &\leq 2^{nR_2}2^{n(A-R_1-R_2+3\e_1)}\\

1895:              &\leq 2^{n(R'_2+3\e_1)} \label{eq:bc-cc-5}

1896: \end{align}

1897: 

1898: Now,

1899: \begin{align}

1900:   P\{E_{0,1}^*(i,m)\}&=P\{|\mc{D}(i,m)|>

1901:                     \underbrace{2^{n(A-R_1+\e')}}_{\triangleq a}\}\\ 

1902:                     &\overset{(b)}{<}e^{-at}E\{ e^{t|\mc{D}|} \}

1903: \end{align}

1904: for any $t>0$ where $(b)$ follows from the Chernoff bound

1905: \cite{cover-thomas}.

1906: 

1907: Now we calculate an upper bound for $E\{ e^{t|\mc{D}|}\}$. Consider an

1908: arbitrary sequence $z^n \in \Ae(Z)$.   Let $u^n[l]$ be the

1909: $l$-th sequence (using some ordering) in $A(U|z^n)$.

1910: Recalling the definition given in (\ref{bins1}) and (\ref{bins2}),

1911: let us denote $|B(i,m)|=\beta$, and 

1912: $i_k=(i-1)\beta+k$ for $k \in \{1,2,\ldots,\beta\}$ for notational

1913: simplicity. Now consider the following sequence of arguments:

1914: \begin{align}

1915:   E\{ e^{t|\mc{D}(i,m)|}|Z^n(m)=z^n \} 

1916:   &= E \bigg\{ \prod_{j=1}^{2^{nR_2}} e^{t\psi(j)} \bigg| Z^n(m)=z^n 

1917:  \bigg\}\\

1918:   &= \sum_{l_1=1}^{|A(U|z^n)|}\!\!\!\!p\{U^n(i_1,m)=u^n[l_1]\} \!\!

1919:   \sum_{l_2=1}^{|A(U|z^n)|}

1920:   \!\!\!\!p\{U^n(i_2,m)=u^n[l_2]\}\ldots

1921:   \!\!\sum_{l_{\beta}=1}^{|A(U|z^n)|}\!\!\!\!p\{U^n(i_\beta,m)=u^n[l_\beta]\}

1922:   \notag\\ 

1923:   &~~~~~~~~~~~E \bigg\{ \prod_{j=1}^{2^{nR_2}} e^{t\psi(j)}\bigg|

1924:   ~\mbox{$U^n(i_\theta,m)=u^n[l_\theta]$, 

1925:   for $\theta=1,2,\ldots,\beta$ }, \mbox{and }  Z^n(m)=z^n \bigg\}\\

1926:   &= \sum_{l_1=1}^{|A(U|z^n)|}\sum_{l_2=1}^{|A(U|z^n)|}\ldots

1927:   \sum_{l_{\beta}=1}^{|A(U|z^n)|}\frac{1}{|A(U|z^n)|^{\beta}} 

1928:   \notag\\

1929:   &~~~~~~~~~~~E \bigg\{ \prod_{j=1}^{2^{nR_2}}

1930:   e^{t\psi(j)}\bigg|~\mbox{$U^n(i_\theta,m)=u^n[l_\theta]$, 

1931:   for $\theta=1,2,\ldots,\beta$ }, \mbox{ and } Z^n(m)=z^n

1932:   \bigg\}\\

1933:   &\overset{(c)}{=}

1934:   \sum_{l_1=1}^{|A(U|z^n)|}\sum_{l_2=1}^{|A(U|z^n)|}\ldots

1935:   \sum_{l_{\beta}=1}^{|A(U|z^n)|}\frac{1}{|A(U|z^n)|^{\beta}} 

1936:   \notag\\

1937:   &~~~~~~~~~~~\prod_{j=1}^{2^{nR_2}} E \bigg\{

1938:   e^{t\psi(j)}\bigg|~~\mbox{$U^n(i_\theta,m)=u^n[l_\theta]$, 

1939:   for $\theta=1,2,\ldots,\beta$ }, \mbox{ and } Z^n(m)=z^n

1940:   \bigg\}

1941: \end{align}

1942: where $(c)$ is from the fact that $\psi(j)$'s are independent when

1943: the outcomes of $U^n(i_1,m),U^n(i_2,m),\ldots,U^n(i_\beta,m)$ and

1944: $Z^n(m)$ are fixed. Let

1945: us denote 

1946: \beq

1947: p_j=P\{\psi(j)=1 \big|~\mbox{$U^n(i_\theta,m)=u^n[l_\theta]$,

1948:   for $\theta=1,2,\ldots,\beta$ }, \mbox{ and } Z^n(m)=z^n \}.

1949: \label{pj}

1950: \eeq

1951: 

1952: Then,

1953: \begin{align}

1954:   E \{e^{t\psi(j)} \big|~\mbox{$U^n(i_\theta,m)=u^n[l_\theta]$,

1955:   for $\theta=1,2,\ldots,\beta$ }, \mbox{ and } Z^n(m)=z^n  \}&=e^t

1956:   p_j+1\cdot (1-p_j)\\   

1957:  &=1-p_j(1-e^t)\\

1958:                     &\leq e^{-p_j(1-e^t)}~~~ \mbox{since $1-x \leq e^{-x}$} .

1959: \end{align}

1960: So,

1961: \begin{align}

1962: &\prod_{j=1}^{2^{nR_2}} E

1963: \{e^{t\psi(j)}|~\mbox{$U^n(i_\theta,m)=u^n[l_\theta]$,

1964:   for $\theta=1,2,\ldots,\beta$ }, \mbox{ and } Z^n(m)=z^n  \}\\

1965:   &\leq \prod_{j=1}^{2^{nR_2}} e^{p_j(e^t-1)}\\

1966: &={\rm exp} \bigg\{ (e^t-1) \sum_{j=1}^{2^{nR_2}} p_j\bigg\}\\

1967: &={\rm exp} \bigg\{ (e^t-1) E\left\{|\mc{D}(i,m)| \big|

1968: ~\mbox{$U^n(i_\theta,m)=u^n[l_\theta]$,

1969:   for $\theta=1,2,\ldots,\beta$ },  \mbox{ and } Z^n(m)=z^n \right\} \bigg\}

1970: \end{align}

1971: 

1972: Then,

1973: \begin{align}

1974:  E\{ e^{t|\mc{D}(i,m)|} | Z^n(m)=z^n \}

1975:  &\leq

1976: \sum_{l_1=1}^{|A(U|z^n)|}\!\!\!\!\ldots\!\!\!\!

1977: \sum_{l_{\beta}=1}^{|A(U|z^n)|}\!\!\!\!\frac{1}{|A(U|z^n)|^{\beta}}

1978:  \notag\\ 

1979:   &~~~~~~~~{\rm exp} \bigg\{ (e^t-1) E\left\{|\mc{D}(i,m)| \big|

1980: \mbox{$U^n(i_\theta,m)=u^n[l_\theta]$,

1981:   for $\theta=1,2,\ldots,\beta$ }, \mbox{ and } Z^n(m)=z^n  \right\}

1982:  \!\! \bigg\}\\ 

1983:   &\overset{(d)}{\leq} {\rm exp} \bigg\{ (e^t-1) 2^{n(R'_2+3\e_1)}  \bigg\}

1984: \end{align}

1985: where $(d)$ is obtained because $E\left\{|\mc{D}(i,m)| \big|

1986: ~\mbox{$U^n(i_\theta,m)=u^n[l_\theta]$,

1987:   for $\theta=1,2,\ldots,\beta$ }, \mbox{ and } Z^n(m)=z^n  \right\}$

1988: is bounded by $2^{n(R'_2+3 \e_1)}$ (the same as the bound on the

1989: unconditional expectation as in inequality 

1990: (\ref{eq:bc-cc-5}))  regardless of the particular sequence collection

1991: $u^n[l_1],u^n[l_2],\ldots,u^n[l_\beta]$ and $z^n$. To see this crucial step 

1992: clearly let us rewrite  $p_j$ (see (\ref{pj})), using the union bound

1993: on the outcome of the bin $C(j,m)$, as follows: 

1994: \beq

1995: p_j \leq  2^{n(I(V;Y_2|Z)-R_2-\epsilon)} 

1996:   P \left[  \bigcup_{\theta=1}^{\beta} F_{\theta}  \right],

1997: \eeq

1998: where $F_{\theta}$ is the event that a sequence drawn from $A(V|z^n)$ 

1999: belongs to $A(V|u^n[l_{\theta}],z^n)$. Since all sequences in

2000: $B(i,m)$, given that $Z^n(m)=z^n$, 

2001: are drawn from $A(U|z^n)$, we have using (\ref{eqV}) and (\ref{eqV|U})

2002: \beq

2003: p_j \leq  2^{n(I(V;Y_2|Z)-R_2-\epsilon)}  2^{n(I(U;Y_1|Z)-R_1-\epsilon)} 

2004: 2^{-n(I(U;V|Z)-3 \epsilon_1)}= 2^{n(A-R_1-R_2+3\e_1)}.

2005: \eeq

2006: 

2007: 

2008: Therefore, for $t>0$,

2009: \begin{align}

2010:   P\{E_{0,1}^*(i,m)\} &\leq e^{-at} {\rm exp} \bigg\{ (e^t-1)

2011:     2^{n(R'_2+3\e_1)}  \bigg\}\\ 

2012:     &={\rm exp} \bigg\{-at+(e^t-1)\underbrace{2^{n(R'_2+3\e_1)}}_{\triangleq

2013:     ~b}\bigg\}

2014: \end{align}

2015: 

2016: To get a tighter upper bound, let us denote $f(t)=-at+b(e^t-1)$,

2017: for $t>0$. Then, $f'(t)=-a+be^t$ and $f''(t)=be^t

2018: > 0$. So, $f(t)$ has the minimum value when $t=\ln

2019: \left(\frac{a}{b} \right)$.

2020: 

2021: Thus, $P\{E_{0,1}^*(i,m)\}$ is bounded as

2022: \begin{align}

2023:   P\{E_{0,1}^*(i,m)\}& \leq {\rm exp} \bigg\{-a\ln \left( \frac{a}{b}

2024:   \right)+a-b \bigg\} 

2025: \end{align}

2026: where $a=2^{n(R'_2+\e')}$ and $b=2^{n(R'_2+3\e_1)}$.

2027: 

2028: So,

2029: \begin{align}

2030:   P\{E_{0,1}^*(i,m)\} &\leq {\rm exp} \left\{-2^{n(R'_2+3\e_1)}

2031:   [2^{n(\e'-3\e_1)}\ln(2^{n(\e'-3\e_1)})-2^{n(\e'-3\e_1)}+1 ]

2032:   \right\}\\*

2033:   &= {\rm exp} \big\{-2^{n(R'_2+3\e_1)}\eta \big\} \\

2034:   &\leq  {\rm exp} \big\{-2^{n(R'_2+3\e_1)} \big\}

2035: \end{align}

2036: where

2037: $\eta=2^{n(\e'-3\e_1)}\ln(2^{n(\e'-3\e_1)})-2^{n(\e'-3\e_1)}+1$. Note

2038: that  $\eta >0$ since (i) $\e'>3\e_1$ and (ii) for $x>1$, $g(x)=x\ln(x)-x+1$ is

2039: increasing function of $x$ and $g(x)>0$. Further $g(e)=1$. 

2040: 

2041: Therefore, for sufficiently large $n$, by applying the union bound,

2042: \begin{align}

2043: P\{E_{0,1}^*\} &=P\Big\{ \bigcup_{i=1}^{2^{nR_1}} \bigcup_{m=1}^{2^{nR_0}} 

2044: E_{0,1}^*(i,m) \Big\}\\

2045:                &\leq \sum_{i=1}^{2^{nR_1}}\sum_{m=1}^{2^{nR_0}} 

2046:  P\{E_{0,1}^*(i)\}\\

2047:                &\leq 2^{n(R_0+R_1)}{\rm exp} \big\{-2^{n(R'_2+3\e_1)}

2048:  \big\} \\ 

2049:                &={\rm exp} \big\{n(R_0+R_1)\ln2-2^{n(R'_2+3\e_1)}

2050:   \big\} \\ 

2051:                &\overset{(e)}{<} \frac{\e}{12}

2052: \end{align}

2053: where (e) is from the fact that

2054: $n(R_0+R_1)\ln2$ is linearly increasing but $2^{n(R'_2+3\e_1)}$ is

2055: exponentially increasing as $n$ increases.

2056: 

2057: In a similar way, we can also show that

2058: $P\{E_{0,2}^*\}<\frac{\e}{12}$ for sufficiently large $n$.

2059:  \hfill $\blacksquare$

2060: 

2061: \section{Proof of Lemma \ref{lem:bc-cc-3}}\label{appendix:bc-lem-cc-3}

2062: Now let us calculate the probability $P(E \cap E_1^c \cap E_2^c)$.

2063: Without loss of generality, let us assume that the outcome of the 

2064: message triple is given by: $(W_0,W_1,W_2)=(m,i,j)$. Let 

2065: $(U^n(k_i,m),V^n(l_j,m))$ be the pair of sequences that are jointly 

2066: typical in the bin $B(i,m)$ and $C(j,m)$.

2067: Consider the following error events.

2068: \begin{itemize}

2069:   \item[$E_3$:] $(Z^n(m),U^n(k_i,m),V^n(l_j,m),X^n, Y_1^n, Y_2^n) \notin

2070:   \Ae(Z,U,V,X,Y_1,Y_2)$ 

2071:   \item[$E_4$:] Decoding step fails at receiver 1, i.e., $ \exists

2072:   ~\hat{m} \neq m$ such that $(Z^n(\hat{m}), Y_1^n) \in 

2073:   \Ae(Z,Y_1)$ or

2074:   $ \exists ~\hat{k} \neq k_i$ such that $(U^n(\hat{k},m), Y_1^n, Z^n(m))

2075:   \in A_{\e}^{(n)}(U,Y_1,Z)$ 

2076:   \item[$E_5$:] Decoding step fails at receiver 2, i.e., $ \exists

2077:   ~\hat{m} \neq m$ such that $(Z^n(\hat{m}), Y_2^n) \in 

2078:   \Ae(Z,Y_2)$ or

2079:   $ \exists ~\hat{l} \neq l_j$ such that $(V^n(\hat{l},m), Y_2^n, Z^n(m))

2080:   \in A_{\e}^{(n)}(V,Y_2,Z)$ 

2081: \end{itemize}

2082: 

2083: It is easy to see that, for sufficiently large $n$, 

2084: $P(E_3) \leq \frac{2\e}{9}$, using  the Markov lemma

2085: \cite{berger,cover-thomas};

2086: $P(E_4) \leq \frac{2\e}{9}$, if $R_0 \leq I(Z;Y_1)-\e$ and  

2087: $P(E_5) \leq \frac{2\e}{9}$, if $R_0 \leq I(Z;Y_2)-\e$.

2088: Recall that for all $m \in \{1,2,\ldots,2^{nR_0}\}$, 

2089: $|\C_1(m)|=2^{n(I(U;Y_1|Z)-\e)}$ and 

2090: $|\C_2(m)|=2^{n(I(V;Y_2|Z)-\e)}$.

2091: 

2092: So,

2093: \begin{align}

2094:   P(E \cap E_1^c \cap E_2^c) &= P(\cup_{i=3}^5 E_i \cap E_1^c \cap

2095:   E_2^c)\\

2096:   &\leq \sum_{i=3}^5 P\left(E_i \cap E_1^c \cap E_2^c \right) <

2097:   \frac{2\e}{3}

2098: \end{align}

2099: Hence, the proof of Lemma \ref{lem:bc-cc-3} has been completed.

2100: \hfill $\blacksquare$

2101: 

2102: 

2103: \section{Proof of Theorem \ref{thm:one-det-bc-cor}}

2104: \label{appendix:one-det-bc-cor}

2105: 

2106: Here we provide the converse part, as the direct part follows from 

2107: Theorem \ref{thm:rate-region-bc-cor}. The proof is very similar to 

2108: Marton's proof \cite{marton79} of the

2109: outer bound of the capacity region of semi-deterministic broadcast

2110: channels. Consider any sequence (indexed by $n$) of $(n,\tau(n))$ 

2111: transmission systems for a sequence of bipartite graphs $G_n$,

2112: respectively, with parameters

2113: $(1,2^{nR_1},2^{nR_2},2^{nR'_1},2^{nR'_2},\mu(n))$ and the

2114: broadcast channel with one deterministic component such that 

2115: $\tau(n) \rightarrow 0$, and $\frac{1}{n} \log \mu(n) \rightarrow 0$

2116: as $n \rightarrow \infty$. 

2117: 

2118: By Fano's Inequality,

2119: \begin{align}

2120:   H(W_1,W_2|Y_1^n,Y_2^n) ~&\leq ~ \tau(n) \cdot n(R_1+R_2) +

2121:   H(\tau(n)) ~\leq~n\e_n \\ 

2122:   H(W_1|Y_1^n) ~&\leq ~ \tau(n) \cdot n R_1 + H(\tau(n)) ~\leq~n\e_n \\

2123:   H(W_2|Y_2^n) ~&\leq ~ \tau(n) \cdot n R_2 + H(\tau(n)) ~\leq~n\e_n

2124: \end{align} where $\e_n \rightarrow 0$ as $\tau(n) \rightarrow 0 $.

2125: We can bound the rate $R_1$ as

2126: \begin{align}

2127:   n R_1 &= H(W_1) = I(W_1;Y_1^n)+H(W_1|Y_1^n)\\

2128:         &\leq I(W_1;Y_1^n)+ n\e_n 

2129:         \leq H(Y_1^n) + n \e_n \\

2130:         &\leq \sum_{i=1}^{n} H(Y_{1i}) + n\e_n\\

2131:         \Rightarrow R_1 &\leq \frac{1}{n}\sum_{i=1}^{n} H(Y_{1i}) +

2132:         \e_n. \label{eq:15} 

2133: \end{align}

2134: $R_2$ can be bounded as

2135: \begin{align}

2136:   nR_2 &= H(W_2) = I(W_2;Y_2^n)+H(W_2|Y_2^n)\\

2137:         &\leq I(W_2;Y_2^n)+ n\e_n 

2138:         =\sum_{i=1}^{n} I(W_2;Y_{2i}|Y_{2}^{(i-1)}) + n\e_n\\

2139: 	 &\leq \sum_{i=1}^{n}

2140:         [H(Y_{2i})-H(Y_{2i}|W_2,Y_2^{i-1},Y_{1(i+1)}^n)] + n\e_n\\ 

2141:         &=\sum_{i=1}^{n} I(V_i;Y_{2i}) + n\e_n, ~~\text{by defining

2142:         $V_i=(W_2,Y_2^{i-1},Y_{1(i+1)}^n)$}\\ 

2143: \Rightarrow R_2 &\leq \frac{1}{n}\sum_{i=1}^{n} I(V_i;Y_{2i}) +

2144: \e_n. \label{eq:16}

2145: \end{align}

2146: 

2147: Using the fact that (a) the channel is discrete memoryless and is used

2148: without feedback, and (b) $Y_1$ is a deterministic function of the 

2149: channel input, it can be easily shown that $V_i \rightarrow X_i

2150: \rightarrow (Y_{1i},Y_{2i})$. 

2151: Before we bound the sum of rates $R_1+R_2'$, let us recall the

2152: following identity from \cite{gel80}: for any two sequence of 

2153: random variables $A^n$ and $B^n$, 

2154: \beq

2155: \sum_{i=1}^n I(A^{(i-1)};B_i|B_{(i+1)}^n)=

2156: \sum_{i=1}^n I(B_{(i+1)}^n;A_i|A^{(i-1)}).

2157: \label{specialequality}

2158: \eeq

2159: 

2160: The number of all possible pairs of messages $(W_1,W_2)$ which have

2161: non-zero probability is lower

2162: bounded by $2^{n(R_1+R_2')}\mu^{-1}=2^{n(R_1'+R_2)}\mu^{-1}$.

2163: So, we can bound the sum rate $n(R_1+R_2')$ as

2164: \begin{align}

2165:   n(R_1+R_2') &\leq H(W_1,W_2) + \log \mu\\

2166:         &=H(W_2)+H(W_1|W_2) + \log \mu\\

2167:         &=I(W_2;Y_2^n)+\underbrace{H(W_2|Y_2^n)}_{\leq n\e_n}

2168:         +I(W_1;Y_1^n|W_2)+\underbrace{H(W_1|Y_1^n,W_2)}_{\leq

2169:         H(W_1|Y_1^n) \leq ~n\e^n} + \log \mu\\ 

2170:         &\leq I(W_1;Y_1^n|W_2)+I(W_2;Y_2^n)+2n\e_n + \log \mu\\

2171:         &= H(Y_1^n)-I(W_2;Y_1^n)+I(W_2;Y_2^n)+2n\e_n + \log \mu\\

2172:         &\overset{(a)}{\leq} H(Y_1^n) -\left[ \sum_{i=1}^n (I(W_2,

2173:         Y_{1(i+1)}^n; Y_{1i}) - I(Y_{1(i+1)}^n;Y_{1i})) \right] +

2174:         \sum_{i=1}^n I(W_2,Y_2^{(i-1)}; Y_{2i})+2n\e_n + \log \mu\\

2175:         &\overset{(b)}= \sum_{i=1}^n H(Y_{1i})+ \sum_{i=1}^n \left[

2176:         I(V_i;Y_{2i})-I(V_i;Y_{1i}) \right] + 2n\e_n + \log \mu\\

2177:         &= \sum_{i=1}^n \left[H(Y_{1i}|V_i)+I(V_i;Y_{2i}) \right] +

2178:         2n\e_n + \log \mu\\ 

2179:        \Rightarrow R_1 + R_2' &\leq \frac{1}{n}\sum_{i=1}^{n}

2180: (H(Y_{1i}|V_i)+I(V_i;Y_{2i})) + 2\e_n + \frac{1}{n}\log \mu \label{eq:17}

2181: \end{align}

2182: where\\

2183: $(a)$ follows from the fact that $I(W_2;Y_{2i}|Y_{2}^{(i-1)}) \leq 

2184: I(W_2,Y_{2}^{(i-1)};Y_{2i})$. \\

2185: $(b)$ from the fact that $H(Y_1^n)+\sum_{i=1}^n I(Y_{1(i+1)}^n;Y_{1i})=

2186: \sum_{i=1}^n H(Y_{1i})$, and (\ref{specialequality}).

2187: 

2188: To complete the proof, let us define a new random variable $Q$

2189: uniformly distributed over the set $Q=\{1,2,\ldots,n\}$ with

2190: probability $\frac{1}{n}$ as shown in \cite{cover-thomas,gelfand80}. By

2191: defining $V=V_Q$, $Y_1=Y_{1Q}$, $Y_2=Y_{2Q}$, it can be shown that

2192: the inequalities (\ref{eq:12}), (\ref{eq:13}), and (\ref{eq:14})

2193: are equivalent to (\ref{eq:15}), (\ref{eq:16}), and (\ref{eq:17}),

2194: respectively. \hfill $\blacksquare$

2195: 

2196: 

2197: 

2198: \section{Proof of Lemma \ref{lem:bc-sc-3}}\label{appendix:bc-lem-sc-3}

2199: Let us calculate the probability $P(E \cap E_1^c \cap E_2^c)$. If

2200: previous error events $E_1$ or $E_2$ do not occur, we define other

2201: error events as follows.

2202: \begin{itemize}

2203:   \item[$E_3$]: $(S^n,T^n) \notin \Ae$,

2204:   \item[$E_4$]: $\nexists m \in \{1,2,\ldots,2^{nR_0}\}$ such that

2205:   $(S^n,T^n,Z^n(m))\in \Ae(S,T,Z)$,

2206:   \item[$E_5$]: $\nexists i \in \{1,2,\ldots,2^{nR_1}\}$ such that

2207:     $S^n(i,m)=S^n$ and 

2208: $S^n(i,m) \in B(m)$,

2209:   \item[$E_6$]: $\nexists j \in \{1,2,\ldots,2^{nR_2}\}$ such that

2210:     $T^n(j,m)=T^n$ and 

2211: $T^n(j,m) \in C(m)$,

2212: \end{itemize}

2213: Then,

2214: \begin{align}

2215:   P(E \cap E_1^c \cap E_2^c)&=P\left( \cup_{i=3}^6 E_i \cap (E_1^c \cap

2216:                                    E_2^c) \right)\\ 

2217:      &\leq  P\left[E_3 \cap (E_1^c \cap E_2^c) \right] +

2218: \sum_{i=4}^6 P\left(E_i \cap E^c_3 \cap  (E_1^c \cap E_2^c) \right) 

2219: \end{align}

2220: 

2221: By the property of jointly typical sequences \cite{cover-thomas},

2222: $P[E_3 \cap (E_1^c \cap E_2^c)]< \frac{\e}{8}$ for sufficiently large

2223: $n$. Using arguments of Chapter 13 of \cite{cover-thomas}, it can be

2224: shown that  for sufficiently large $n$, if $ R_0 > I(S,T;Z) + \e_2(\e)$ where

2225: $\e_2(\e) \ra 0$ as $\e \ra 0$, then 

2226: $P[E_4 \cap E_3^c \cap (E_1^c \cap E_2^c)] \leq \frac{\e}{8}$. 

2227:  See Chapter 13 of \cite{cover-thomas}

2228: for a characterization of $\e_2$. Now

2229: \begin{align}

2230:   P[E_5 \cap E_3^c \cap E_4^c \cap (E_1^c \cap E^c_2)]

2231:   &\overset{(a)}{=}P[\forall S^n(i,m) \in B(m), S^n(i,m) \neq S^n | S^n

2232:   \in A(S|Z^n(m))]\\ 

2233:          &\overset{(b)}{\leq} \left[1-2^{-n(H(S|Z)+\e_1(\e))}

2234:   \right]^{2^{nR_1}}\\ 

2235:          &\overset{(c)}{\leq} {\rm exp}\{-2^{n(R_1-H(S|Z)-\e_1(\e))}\}\\

2236:          &< \frac{\e}{8}

2237: \end{align}

2238: for sufficiently large $n$, if $R_1>H(S|Z)+\e_1(\e)$ where\\

2239: (a) is from the fact that $(S^n,T^n,Z^n(m))\in \Ae(S,T,Z)$ implies

2240: $S^n \in A(S|Z^n(m))$,\\

2241: (b) is obtained using the fact that each $S^n(i,m)$ has the same chance of

2242: equaling $S^n$ and independently chosen, and 

2243: the fact that  $|A(S|Z^n(m))|\leq 2^{n(H(S|Z)+\e_1(\e))}$,\\

2244: (c) follows from $(1-x)^n \leq e^{-xn}$ for $0 \leq x \leq 1$ and

2245: $n>0$.

2246: 

2247: So,

2248: \begin{align}

2249:   P[E_5 \cap E_3^c \cap (E_1^c\cap E^c_2)]&=P[E_5 \cap E_3^c \cap E_4

2250:                                    \cap (E_1^c \cap E^c_2)]

2251:                                    + P[E_5 \cap E_3^c \cap E_4^c \cap

2252:                                    (E_1^c \cap E^c_2)]\\ 

2253:                                    &\leq P[E_4 \cap E_3^c \cap (E_1^c

2254:                                    \cap E^c_2)]

2255:                                    + P[E_5 \cap E_3^c \cap E_4^c \cap 

2256:                                    (E_1^c \cap E_2^c)]\\

2257:                                    &< \frac{\e}{4}

2258: \end{align}

2259: 

2260: Similarly, it can be shown that $P[E_6 \cap E_3^c \cap (E_1^c \cap E^c_2)] <

2261: \frac{\e}{4}$ if $R_2>H(T|Z)+\e_1(\e)$ for sufficiently large $n$.

2262: Hence $P[E \cap(E_1^c \cap E_2^c)] \leq \frac{3 \e}{4}$.

2263: \hfill $\blacksquare$

2264: 

2265: 

2266: \section{Proof of Theorem \ref{thm:bc-sc-main_c}}

2267: \label{appendix:bc-sc-main_c}

2268: 

2269: Let $f(n),g_1(n),g_2(n)$ be a fixed sequence of encoder and

2270: decoders. Also, let $f(S^n,T^n)=(W_0,W_1,W_2)$. 

2271: We can bound the rate $R_0$ as

2272: \begin{align}

2273:   nR_0 &\geq H(W_0)\geq I(S^n,T^n;W_0)\\

2274:                &=\sum_{i=1}^n [H(S_i,T_i|S^{i-1},T^{i-1})-

2275:                H(S_i,T_i|\underbrace{S^{i-1},T^{i-1},W_0}_{\triangleq

2276:                Z_i})]\\ 

2277:               &= \sum_{i=1}^n [H(S_i,T_i)- H(S_i,T_i|Z_i)]

2278:               = \sum_{i=1}^n I(S_i,T_i;Z_i) \label{eq:bc-sc-12}

2279: \end{align}

2280: Using the constraints on the degrees of the vertexes in the graph

2281: associated with messages, we can also have

2282: \begin{align}

2283: n(R_2+R'_1)=  n(R_1+R_2') &\overset{(a)}{\geq} H(W_1,W_2|W_0) - \log \mu\\

2284:               &= I(S^n,T^n;W_1,W_2|W_0)+H(W_1,W_2|S^n,T^n,W_0) - \log \mu\\

2285:               &\overset{(b)}{=} I(S^n,T^n;W_1,W_2|W_0) - \log \mu\\

2286:               &=H(S^n,T^n|W_0)-H(S^n,T^n|W_1,W_2,W_0) - \log \mu\\

2287:               &\overset{(c)}{\geq} H(S^n,T^n|W_0)-n\e_n - \log \mu \\

2288:               &= \sum_{i=1}^n

2289:               H(S_i,T_i|\underbrace{S^{i-1},T^{i-1},W_0}_{=Z_i})-n\e_n

2290:               - \log 

2291:               \mu,\\

2292:               &= \sum_{i=1}^n H(S_i,T_i|Z_i)-n\e_n - \log \mu,

2293:               \label{eq:bc-sc-10} 

2294: \end{align}

2295: where

2296: (a) is from the constraints on the degrees of the graph,

2297: (b) is obtained since $W_1$ and $W_2$ is a function of $S^n$ and

2298: $T^n$,

2299: (c) follows from Fano's inequality. 

2300: Also, we can also bound the rate $R_1$ as

2301: \begin{align}

2302:   nR_1 &\geq H(W_1) \geq H(W_1|W_0)

2303:               = I(S^n;W_1|W_0)+H(W_1|S^n,W_0) \\

2304:               &\overset{(a)}{\geq} I(S^n;W_1|W_0)

2305:               = H(S^n|W_0)-H(S^n|W_1,W_0)\\

2306:               &\overset{(b)}{\geq} H(S^n|W_0)-n\e_n 

2307:               = \sum_{i=1}^n

2308:               H(S_i|S^{i-1},W_0)-n\e_n,\\

2309:               &\overset{(c)}{\geq} \sum_{i=1}^n

2310:               H(S_i|\underbrace{S^{i-1},T^{i-1},W_0}_{=Z_i})-n\e_n,\\

2311:               &= \sum_{i=1}^n H(S_i|Z_i)-n\e_n, \label{eq:bc-sc-13}

2312: \end{align}

2313: where

2314: (a) is obtained since $H(W_1|S^n,W_0) \geq 0$,

2315: (b) follows from Fano's inequality, and 

2316: (c) follows from adding conditioning.

2317: 

2318: Similarly, we also can obtain

2319: \begin{align}

2320:   nR_2 &\geq \sum_{i=1}^n H(T_i|Z_i) - n\e_n. \label{eq:bc-sc-14}

2321: \end{align}

2322: 

2323: Therefore, using arguments similar to those of \cite{wyner75},

2324: we can have the partial converse by dividing the

2325: inequalities (\ref{eq:bc-sc-10}),

2326: (\ref{eq:bc-sc-12}), (\ref{eq:bc-sc-13}), and (\ref{eq:bc-sc-14})

2327:  by $n$, and taking the limit as $n\ra\infty$.

2328:  \hfill $\blacksquare$

2329: 

2330: 

2331: 

2332: \begin{thebibliography}{10}

2333: \bibitem{han-costa87}

2334: T.~S. Han and H.~M. Costa, ``Broadcast channels with arbitrarily correlated

2335:   sources,'' \emph{IEEE Trans. Inform. Theory}, vol. IT-33, no.~5, pp.

2336:   641--650, Sep. 1987.

2337: 

2338: \bibitem{cover72}

2339: T.~M. Cover, ``Broadcast channels,'' \emph{IEEE Trans. Inform. Theory}, vol.

2340:   IT-18, no.~1, pp. 2--14, Jan. 1972.

2341: 

2342: \bibitem{cover75}

2343: ------, ``An achievable rate region for the broadcast channel,'' \emph{IEEE

2344:   Trans. Inform. Theory}, vol. IT-21, no.~4, pp. 399--404, Jul. 1975.

2345: 

2346: \bibitem{bergmans73}

2347: P.~P. Bergmans, ``Random coding theorems for the broadcast channels with

2348:   degraded components,'' \emph{IEEE Trans. Inform. Theory}, vol. IT-15, pp.

2349:   197--207, Mar. 1973.

2350: 

2351: \bibitem{bergmans74}

2352: ------, ``A simple converse for broadcast channels with additive white

2353:   {G}aussian noise,'' \emph{IEEE Trans. Inform. Theory}, vol. IT-20, pp.

2354:   279--280, Mar. 1974.

2355: 

2356: \bibitem{gallager74}

2357: R.~G. Gallager, ``Capacity and coding for degraded broadcast channels,''

2358:   \emph{Probl. Pered. Inform.}, vol.~10, no.~3, pp. 3--14, Jul.-Sep. 1974,

2359:   ;translated in \emph{Probl. Inform. Transm.}, pp. 185-193, Jul.-Sep. 1974.

2360: 

2361: \bibitem{ahlswede-korner75}

2362: R.~Ahlswede and J.~{K\"{o}rner}, ``Source coding with side information and a

2363:   converse for degraded broadcast channels,'' \emph{IEEE Trans. Inform.

2364:   Theory}, vol. IT-21, pp. 629--637, Nov. 1975.

2365: 

2366: \bibitem{korner-marton77}

2367: J.~{K\"{o}rner} and K.~Marton, ``General broadcast channels with degraded

2368:   message sets,'' \emph{IEEE Trans. Inform. Theory}, vol. IT-23, pp. 60--64,

2369:   Jan. 1977.

2370: 

2371: \bibitem{elgamal79}

2372: A.~{El Gamal}, ``The capacity of a class of broadcast channels,'' \emph{IEEE

2373:   Trans. Inform. Theory}, vol. IT-25, pp. 166--169, Mar. 1979.

2374: 

2375: \bibitem{marton77}

2376: K.~Marton, ``The capacity region of deterministic broadcast channels,'' in

2377:   \emph{Trans. Int. Symp. Inform. Theory}, Paris-Cachan, France, 1977.

2378: 

2379: \bibitem{pinsker78}

2380: M.~S. Pinsker, ``Capacity of noiseless broadcast channels,'' \emph{Probl.

2381:   Pered. Inform.}, vol.~14, no.~2, pp. 28--34, Apr.-Jun. 1978, translated in

2382:   \emph{Probl. Inform. Transm.}, pp. 97-102, Apr.-June 1978.

2383: 

2384: \bibitem{marton79}

2385: K.~Marton, ``A coding theorem for the discrete memoryless broadcast channel,''

2386:   \emph{IEEE Trans. Inform. Theory}, vol. IT-25, no.~3, pp. 306--311, May 1979.

2387: 

2388: \bibitem{gelfand80}

2389: S.~I. Gelfand and M.~S. Pinsker, ``Capacity of a broadcast channel with one

2390:   deterministic component,'' \emph{Probl. Pered. Inform.}, vol.~16, no.~1, pp.

2391:   24--34, Jan.-Mar. 1980, ; translated in \emph{Probl. Inform. Transm.}, vol.

2392:   16, no. 1, pp. 17-25, Jan.-Mar. 1980.

2393: 

2394: \bibitem{elgamal}

2395: A.~{El Gamal}, ``The capacity of the product and sum of two reversely degraded

2396:   broadcast channels,'' \emph{Probl. Pered. Inform.}, vol.~16, pp. 3--23,

2397:   Jan.-Mar. 1980.

2398: 

2399: \bibitem{tse97}

2400: D.~Tse, ``Optimal power allocation over parallel {G}aussian broadcast

2401:   channels,'' \emph{Proc. Int. symp. on Inform. Theory (ISIT)}, July 1997.

2402: 

2403: \bibitem{li01}

2404: L.~Li and A.~Goldsmith, ``Capacity and optimal resource allocation for fading

2405:   broadcast channels-{P}art {I}: Ergodic capacity,'' \emph{IEEE Trans. on

2406:   Inform. Theory}, vol.~47, pp. 1083--1102, March 2001.

2407: 

2408: \bibitem{caire03}

2409: G.~Caire and S.~Shamai, ``{O}n the achievable throughput of a multiantenna

2410:   gaussian broadcast channel hannel,'' \emph{IEEE Trans. on Inform. Theory},

2411:   vol.~49, no.~7, pp. 1691--1706, July 2003.

2412: 

2413: \bibitem{viswanathtsc03}

2414: P.~Viswanath and D.~N.~C. Tse, ``{S}um capacity of the vector {G}aussian

2415:   channel and uplink-downlink duality,'' \emph{IEEE Trans. on Inform. Theory},

2416:   vol.~49, no.~8, pp. 1912--1921, August 2003.

2417: 

2418: \bibitem{vishwanathjgda03}

2419: S.~Vishwanath, N.~Jindal, and A.~Goldsmith, ``{D}uality, achievable rates and

2420:   sum-rate capacity of {G}aussian {M}{I}{M}{O} broadcast channel,'' \emph{IEEE

2421:   Trans. on Inform. Theory}, vol.~49, pp. 2658--2668, October 2003.

2422: 

2423: \bibitem{yu04}

2424: W.~Yu and J.~M. Cioffi, ``Sum capacity of {G}aussian vector broadcast

2425:   channels,'' \emph{IEEE Trans. on Inform. Theory}, vol.~50, no.~9, pp.

2426:   1875--1892, September 2004.

2427: 

2428: \bibitem{weingarten04}

2429: H.~Weingarten, Y.~Steinberg, and S.~{Shamai(Shitz)}, ``The capacity region of

2430:   the {Gaussian MIMO} broadcast channel,'' \emph{Proc. Conf. on Inform.

2431:   Sciences and Systems (CISS)}, Mar. 2004.

2432: 

2433: \bibitem{meulen75}

2434: E.~C. {Van der Meulen}, ``Random coding theorems for the general discrete

2435:   memoryless broadcast channel,'' \emph{IEEE Trans. Inform. Theory}, vol.

2436:   IT-21, no.~2, pp. 180--190, Mar. 1975.

2437: 

2438: \bibitem{hajek79}

2439: B.~E. Hajek and M.~B. Pursley, ``Evaluation of an achievable rate region for

2440:   the broadcast channel,'' \emph{IEEE Trans. Inform. Theory}, vol. IT-25,

2441:   no.~1, pp. 36--46, Jan. 1979.

2442: 

2443: \bibitem{elgamal81}

2444: A.~{El Gamal} and E.~{Van der Meulen}, ``A proof of marton's coding theorem for

2445:   the discrete memoryless broadcast channel,'' \emph{IEEE Trans. Inform.

2446:   Theory}, vol. IT-27, no.~1, pp. 120--122, Jan. 1981.

2447: 

2448: \bibitem{sato78}

2449: H.~Sato, ``{A}n outer bound on the capacity region of broadcast channel,''

2450:   \emph{IEEE Trans. on Inform. Theory}, vol.~24, pp. 374--377, May 1978.

2451: 

2452: \bibitem{nair06}

2453: C.~Nair and A.~E. Gamal, ``An outer bound to the capacity region of the

2454:   broadcast channel,'' in \emph{Proc. IEEE Int. Symp. on Inform. Theory

2455:   (ISIT)}, July 2006.

2456: 

2457: \bibitem{cover98}

2458: T.~M. Cover, ``Comments on broadcast channels,'' \emph{{I}{E}{E}{E} Trans. on

2459:   Inform. Theory}, vol.~44, no.~6, pp. 2524--2530, October 1998.

2460: 

2461: \bibitem{gray74}

2462: R.~M. Gray and A.~D. Wyner, ``Source coding for a simple network,'' \emph{Bell

2463:   Syst. Tech. J.}, vol.~53, pp. 1681--1721, Nov. 1974.

2464: 

2465: \bibitem{cover-elgamal-salehi80}

2466: T.~M. Cover, A.~{El Gamal}, and M.~{Salehi}, ``Multiple-access channel with

2467:   arbitrarily correlated sources,'' \emph{IEEE Trans. Inform. Theory}, vol.

2468:   IT-26, no.~6, pp. 648--657, Nov. 1980.

2469: 

2470: \bibitem{pradhan-choi-isit}

2471: S.~S. Pradhan, S.~Choi, and K.~Ramchandran, ``Achievable rates for

2472:   multiple-access channels with correlated messages,'' in \emph{Proc. IEEE Int.

2473:   Symp. on Inform. Theory (ISIT)}, Jun. 2004.

2474: 

2475: \bibitem{pradhan-choi06}

2476: ------, ``A graph-based framework for transmission of correlated sources over

2477:   multiple access channel,'' \emph{\emph{submitted to} IEEE Trans. Inform.

2478:   Theory}, Jan. 2006.

2479: 

2480: \bibitem{ahlswede-han83}

2481: R.~Ahlswede and T.~S. Han, ``On source coding with side information via a

2482:   multiple-access channel and related problems in multi-user information

2483:   theory,'' \emph{IEEE Trans. Inform. Theory}, vol. IT-29, pp. 396--412, May

2484:   1983.

2485: 

2486: \bibitem{cover-thomas}

2487: T.~M. Cover and J.~A. Thomas, \emph{Elements of Information Theory}.\hskip 1em

2488:   plus 0.5em minus 0.4em\relax New York:Wiley, 1991.

2489: 

2490: \bibitem{vanlint}

2491: J.~H. van Lint and R.~M. Wilson, \emph{A course in combinatorics}.\hskip 1em

2492:   plus 0.5em minus 0.4em\relax Cambrigde University Press, 2003.

2493: 

2494: \bibitem{csiszar-korner}

2495: I.~Csisz\'{a}r and {J. K\"{o}rner}, \emph{Information Theory: Coding Theorems

2496:   for Discrete Memoryless Systems}.\hskip 1em plus 0.5em minus 0.4em\relax

2497:   Academic Press, New York, 1981.

2498: 

2499: \bibitem{wyner75}

2500: A.~D. Wyner, ``The common information of two dependent random variables,''

2501:   \emph{IEEE Trans. Inform. Theory}, vol. IT-21, no.~2, pp. 163--179, Mar.

2502:   1975.

2503: 

2504: \bibitem{gacs}

2505: P.~{G\'{a}cs} and J.~{K\"{o}rner}, ``Common information is much less than

2506:   mutual information,'' \emph{Problems of Control and Information Theory},

2507:   vol.~2, pp. 149--162, 1973.

2508: 

2509: \bibitem{wit}

2510: H.~S. Witsenhausen, ``On sequences of pairs of dependent random variables,''

2511:   \emph{SIAM J. Appl. Math.}, vol.~28, pp. 100--113, Jan. 1975.

2512: 

2513: \bibitem{tuncel05}

2514: E.~Tuncel, ``Lossless joint source-channel coding across broadcast channels

2515:   with decoder side information,'' in \emph{Proc. IEEE Int. Symp. on Inform.

2516:   Theory (ISIT)}, September 2005.

2517: 

2518: \bibitem{coleman06}

2519: T.~Coleman, E.~Martinian, and E.~Ordentlich, ``Joint source-channel decoding

2520:   for transmitting correlated sources over broadcast channels,'' in \emph{Proc.

2521:   IEEE Int. Symp. on Inform. Theory (ISIT)}, July 2006.

2522: 

2523: \bibitem{jansbook}

2524: S.~Janson, T.~Luczak, and A.~Ruci\'{n}ski, \emph{Random Graphs}.\hskip 1em plus

2525:   0.5em minus 0.4em\relax John Wiley and Sons, Inc., 2000.

2526: 

2527: \bibitem{krithivasan}

2528: D.~Krithivasan and S.~S. Pradhan, ``On certain large deviation analysis of

2529:   sampling from typical sets,'' Univ. of Michigan, Ann Arbor, Communication and

2530:   Signal Processing Laboratory (CSPL) Report No. 341, July 2006.

2531: 

2532: \bibitem{berger}

2533: T.~Berger, \emph{Multiterminal Source Coding. In: The Information Theory

2534:   Approach to Communications (ed. G. Longo), CISM Courses and Lecture Notes No.

2535:   229}.\hskip 1em plus 0.5em minus 0.4em\relax Springer, Wien-New York, 1977.

2536: 

2537: \bibitem{gel80}

2538: S.~Gel'fand and M.~Pinsker, ``Coding for channel with random parameters,''

2539:   \emph{Problems of Control and Information Theory}, vol.~9, pp. 19--31, 1980.

2540: 

2541: \end{thebibliography}

2542: 

2543: 

2544: 

2545: \end{document}

2546: