1: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
2: %Manuscript Formats Journal or Conference or Thesis
3: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
4: %\documentclass[journal]{IEEEtran}
5: \documentclass[12pt, onecolumn, draftclsnofoot,peerreview]{IEEEtran}
6: %\documentclass[12pt,a4paper]{article}
7: %\documentclass[10pt,conference,letterpaper]{IEEEtran}
8: %\documentclass[12pt,draftcls, onecolumn,journal]{IEEEtran}
9: %\documentclass[10pt,twocolumn,journal]{IEEEtran}
10: \usepackage{times}
11: \usepackage[final]{graphicx}
12: \usepackage[reqno]{amsmath}
13: \usepackage{amsfonts}
14:
15: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
16: % Include packages
17: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
18: %\usepackage{cite}
19: %\usepackage{pslatex}
20: %\usepackage{moreverb}
21: \usepackage{epsfig}
22: %\usepackage{amsmath}
23: \usepackage{amssymb}
24: %\usepackage{booktabs}
25: %\usepackage{array}
26: %\usepackage{multirow}
27: %\usepackage{threeparttable}
28: %\usepackage{url}
29: %\usepackage{dsfont}
30:
31: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
32: % Some user-defined commands
33: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
34: \newtheorem{proposition}{Proposition}
35: \newtheorem{theorem}{Theorem}
36: \newtheorem{lemma}{Lemma}
37: \newtheorem{definition}{Definition}
38: \newtheorem{corollary}{Corollary}
39: \renewcommand{\QED}{\QEDopen}
40: \renewcommand{\thefootnote}{\fnsymbol{footnote}}
41: \def \tb#1#2{\mathop{#1 \vphantom{\sum}}\limits_{\displaystyle #2}}
42: \def \tbsmall #1#2{\mathop{#1 \vphantom{\sum}}\limits_{\scriptstyle #2}}
43: \newcommand{\mr}{\multirow{4}*}
44: \DeclareMathAlphabet{\mathpzc}{OT1}{pzc}{m}{it}
45:
46: %\renewcommand{\baselinestretch}{0.972}
47: \begin{document}
48:
49:
50: % paper title
51: \title{Fingerprinting with Minimum Distance Decoding}
52:
53: \author{
54: \authorblockN{Shih-Chun Lin, Mohammad Shahmohammadi and Hesham El Gamal*}
55: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
56: %Footnotes (Thanks finical support)
57: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
58: \thanks{S. C. Lin is with Department of Electrical Engineering,
59: National Taiwan University, Taipei, Taiwan 10617. The work of S.
60: C. Lin was supported by ``Graduate Students Study Abroad Program''
61: of National Science Council, Taiwan, R.O.C.}
62: \thanks{M. Shahmohammadi and H. El Gamal are with Department of
63: Electrical and Computer Engineering, The Ohio State University,
64: Columbus, OH, 43210. This work was partly performed while Hesham
65: El Gamal was visiting Nile University, Cairo, Egypt. The authors
66: acknowledge the generous funding of the National Science
67: Foundation, USA
68: }
69: \thanks{E-mail: \{lins, shahmohm, helgamal\}@ece.osu.edu.} }
70:
71: % make the title area
72: \maketitle \IEEEpeerreviewmaketitle
73:
74: \begin{abstract}
75: This work adopts an information theoretic framework for the design
76: of collusion-resistant coding/decoding schemes for digital
77: fingerprinting. More specifically, the minimum distance decision
78: rule is used to identify $1$ out of $t$ pirates. Achievable rates,
79: under this detection rule, are characterized in two distinct
80: scenarios. First, we consider the averaging attack where a random
81: coding argument is used to show that the rate $1/2$ is achievable
82: with $t=2$ pirates. Our study is then extended to the general case
83: of arbitrary $t$ highlighting the underlying complexity-performance
84: tradeoff. Overall, these results establish the significant
85: performance gains offered by minimum distance decoding as compared
86: to other approaches based on orthogonal codes and correlation
87: detectors which can support only a sub-exponential number of users
88: (i.e., a zero rate). In the second scenario, we characterize the
89: achievable rates, with minimum distance decoding, under any
90: collusion attack that satisfies the marking assumption. For $t=2$
91: pirates, we show that the rate $1-H(0.25)\approx 0.188$ is
92: achievable using an ensemble of random linear codes. For $t\geq 3$,
93: the existence of a {\em non-resolvable} collusion attack, with
94: minimum distance decoding, for any non-zero rate is established.
95: Inspired by our theoretical analysis, we then construct
96: coding/decoding schemes for fingerprinting based on the celebrated
97: Belief-Propagation framework. Using an explicit repeat-accumulate
98: code, we obtain a vanishingly small probability of misidentification
99: at rate $1/3$ under averaging attack with $t=2$. For collusion
100: attacks which satisfy the marking assumption, we use a more
101: sophisticated accumulate repeat accumulate code to obtain a
102: vanishingly small misidentification probability at rate $1/9$ with
103: $t=2$. These results represent a marked improvement over the best
104: available designs in the literature.
105: \end{abstract}
106:
107: % --------------------- EDICS ---------------------------
108: \vspace{-4mm}
109: \begin{center}
110: {\underline{\bf \small EDICS}} \hspace{3mm} {\small WAT-FING}
111: \end{center}
112: \vspace{-6mm}
113:
114: \section{Introduction}
115: Digital fingerprinting is a paradigm for protecting copyrighted data
116: against illegal distribution \cite{Boneh}. In a nutshell, a
117: \emph{distributor}, i.e., the provider of copyrighted data, wishes
118: to distribute its data $\mathbb{D}$ among a number of licensed
119: \emph{users}. Each licensed copy is identified with a mark, which
120: will be referred to as a \emph{fingerprint} in the sequel, composed
121: of a set of redundant digits embedded inside the copyrighted data.
122: The locations of the redundant digits are kept \emph{hidden} from
123: the users and are only known to the distributor. Their positions,
124: however, remain the same for all users. If any user re-distributes
125: its data in an unauthorized manner, it will be easily identified by
126: its fingerprint. However, several users may collude to form a
127: \emph{coalition} enabling them to produce an unauthorized copy which
128: is difficult to trace. In the literature, the colluding members are
129: typically referred to as \emph{pirates} or \emph{colluders}. Hence,
130: the need arises for the design of collusion-resistant digital
131: fingerprinting techniques. Our work develops an information
132: theoretic framework for the design of low complexity {\em
133: pirate-identification} schemes.
134:
135: To enable a succinct development of our results, we first consider
136: the widely studied \emph{averaging attack}~\cite{Liu}. The
137: colluders, in this strategy, average their media contents to
138: produce the forged copy. An explicit fingerprinting code
139: construction for this attack was proposed in \cite{Liu}. In this
140: construction, however, the maximum number of users $M$, grows only
141: polynomially with the fingerprinting code-length $n$ (more
142: precisely $M=O(n^2)$). Clearly, this rate of growth corresponds to
143: a zero rate in the information theoretic sense. This motivates our
144: pursuit for a fingerprinting scheme which supports an
145: exponentially growing number of users, with the code-length, while
146: allowing for low complexity pirate-identification strategies.
147: Towards this goal, we use a random coding argument to establish
148: the existence of a rate $0.5$ {\em linear} fingerprinting code
149: which achieves a vanishingly small probability of
150: misidentification when 1) Only $t=2$ pirates are involved in the
151: averaging attack and 2) The low complexity minimum distance (MD)
152: decoder is used to identify one of the two pirates. The enabling
153: observation is the intimate connection between the scenario under
154: consideration and the binary erasure channel (BEC). This result is
155: then extended to the general case with an arbitrary coalition size
156: $t$ where the tradeoff between complexity and performance is
157: highlighted.
158:
159: Building on our analysis for the averaging attack, we then proceed
160: to fingerprinting strategies which are resistant to more general
161: forging techniques. More specifically, we adopt the \emph{marking
162: assumption} first proposed in~\cite{Boneh}. In this framework, the
163: pirates attempt to identify the positions occupied by the
164: fingerprinting digits by comparing their copies. Afterwards, they
165: can {\em only} modify the identified coordinates, in any desired
166: way, to minimize the probability of traceability. The validity of
167: the marking assumption hinges on the assumption that any
168: modification to the data content $\mathbb{D}$ will damage it
169: permanently. This prevents the users from modifying any location
170: in which they do not identify as a fingerprinting digit since it
171: {\em may be} a data symbol. Boneh and Shaw~\cite{Boneh} were the
172: first to construct fingerprinting codes that are resistant to
173: attacks that satisfy the marking assumption. This approach was
174: later extended in \cite{Barg_code} using the idea of separating
175: codes \cite{Spt}. To the best of our knowledge, the best available
176: explicit binary fingerprinting codes are the {\em low rate} codes
177: presented in~\cite{Barg_code}. For example, for $t=2$, the best
178: available code has a rate$\approx 0.0092$. More recently, upper
179: and lower bounds on the binary fingerprinting capacity for $t=2$
180: and $t=3$ were derived in~\cite{Barg_cap}. The decoder used in
181: \cite{Barg_cap}, however, was based on exhaustive search, and
182: hence, would suffer from an exponentially growing complexity in
183: the code length. This prohibitive complexity motivates our
184: proposed approach. In this paper, we show that using linear
185: fingerprinting codes and MD decoding, one can achieve rates less
186: than $0.188$ when the coalition size is $t=2$. Unfortunately, the
187: proposed approach does not scale for $t\geq 3$. This negative
188: result calls for a more sophisticated identification technique
189: inspired by the analogy between our set-up and multiple access
190: channels. Our results in this regard will be reported elsewhere.
191:
192: Since the complexity of the {\em exact} MD decoder can be
193: prohibitive when the code-length is long, we develop a low
194: complexity belief-propagation (BP) identification approach
195: \cite{richardson2001cld}\cite{pishronik2004dld}. This detector
196: only requires a linear complexity in $n$, and offer remarkable
197: performance gain over the best known explicit constructions for
198: fingerprinting \cite{Barg_code}\cite{Liu}. For example, we propose
199: a modified iterative decoder tailored for the averaging attack
200: with $t=2$. Using this decoder along with an explicit
201: repeat-accumulate (RA) fingerprinting code, we achieve a
202: vanishingly small probability of misidentification for rates up to
203: $1/3$. For the marking assumption set-up, we achieve a vanishingly
204: small misidentification probability for rates up to $1/9$ using
205: the recently proposed class of low rate accumulate repeat
206: accumulate (ARA) codes~\cite{divsalar}. It is worth noting that
207: these results represent a marked improvement over the state of the
208: art in the literature. Furthermore, one would expect additional
209: performance enhancement by optimizing the degree sequences of the
210: codes (which is beyond the scope of this work).
211:
212: The rest of the paper is organized as follows. In
213: Section~\ref{Sec_Not}, we introduce the mathematical notations and
214: formally define our problem setup. Then we explore the theoretical
215: limits of fingerprinting using the MD decoder in
216: Sections~\ref{Sec_Theorem}~and~\ref{Sec_Theorem_2}. The simulation
217: results based on the BP framework are presented in
218: Section~\ref{Sec_BP}. Finally, Section~\ref{Sec_conclu} offers some
219: concluding remarks.
220:
221: \section{Notations and Problem statement} \label{Sec_Not}
222: Throughout the paper, random variables and their realizations are
223: denoted by capital letters and corresponding smaller case letters,
224: respectively. Deterministic vectors are denoted by bold-face
225: letters. We denote the entropy function by $H(\cdot)$, with the
226: argument being the probability mass function. Furthermore, for
227: simplicity, we abbreviate $H(p,1-p)$ by $H(p)$, where $1 \geq p
228: \geq 0$. For two functions of $n$, we write $a(n)\doteq b(n)$ if:
229: $\lim \limits_{n\rightarrow\infty }\frac{1}{n}\frac{a(n)}{b(n)}=
230: 1$, for example, ${n \choose d}\doteq 2^{nH(\frac{d}{n})}$. The
231: Hamming distance between two vectors $\mathbf{x}_1,\mathbf{x}_2$
232: is denoted by $d_H(\mathbf{x}_1,\mathbf{x}_2)$. Without loss of
233: generality, we assume that the number of users is $M$, and hence,
234: a coalition $U$ of size $t$ is a subset of $\{1,2,\ldots,M\}$
235: where $|U| = t$. The goal of the coalition, in a nutshell, is to
236: produce a forged fingerprint, $\mathbf{y}$, such that the
237: distributor will not be able to trace it back to any of its
238: members. In the following, we first introduce the notation that
239: will be used for a general attack satisfying the marking
240: assumption and then specify our notations for the averaging attack
241: scenario. It should be noted that our formulation follows in the
242: footsteps of \cite{Barg_cap}. For completeness, however, we repeat
243: it here. As mentioned in \cite{Boneh}, deterministic
244: fingerprinting under the marking assumption is not possible in
245: general. Therefore, the distributor needs to employ some kind of
246: randomization which leads to a collection of binary codes $(F,G)$
247: composed of $K$ pairs of encoding and decoding functions as:
248: \begin{eqnarray}
249: & & f_k:\{1,2,\ldots,M\}\rightarrow \{0,1\}^n \\
250: & & g_k:\{0,1\}^n\rightarrow\{1,2,\ldots,M\} \nonumber \\
251: & & k = 1,2,\ldots,K, \nonumber
252: \end{eqnarray}
253: where the code rate $R$ is $\frac{\log_2 M}{n}$ and the secret key,
254: $k$ is a random variable employed to randomize the codebook. This
255: way, the exact codebook utilized for fingerprinting is kept hidden
256: from the users. It should be noted that, adhering to common
257: conventions in cryptography, the family of encoding and decoding
258: functions as well as the probability distribution of the secret key,
259: $p(k)$, are known to all users. Finally, it is clear from the
260: definition of $g_k$ that the objective of the distributer, in our
261: formulation, is to identify only one of the colluders correctly.
262:
263: For simplicity of presentation, let's assume that $t=2$ then the
264: fingerprints corresponding to the coalition of users (also referred
265: to as pirates or colluders), $u_1,u_2$ are denoted by
266: $\{\mathbf{x}_1,\mathbf{x}_2\}$. The marking assumption implies that
267: position $i$ is \emph{undetectable} to the two colluders if
268: $x_{1i}=x_{2i}$, otherwise it is called
269: \emph{detectable}~\cite{Boneh}. Those undetectable coordinates can
270: not be changed by the pirates, and hence, the set of all possible
271: forged copies is give by
272:
273: \begin{equation}
274: E(U) = \{\mathbf{y}\in \{0,1\}^n\mid y_i=x_{1i},\forall{i}\quad
275: undetectable \}.
276: \end{equation}
277: %If we relax the condition that $\mathbf{y}$ has to be a binary vector, and allow the users to erase or use other alphabets than those of the fingerprints we can capture
278: In general, a coalition $U$ may utilize a random strategy that
279: satisfies the marking assumption to produce $\mathbf{y}$. That is,
280: if ${V(\mathbf{y}\mid \mathbf{x}_1,\mathbf{x}_2)}$ is the
281: probability that $\mathbf{y}$ is created, given the coalition
282: $\{\mathbf{x}_1,\mathbf{x}_2\}$, then we have:
283: \begin{equation}
284: {V(\mathbf{y}\mid \mathbf{x}_1,\mathbf{x}_2)} = 0 \qquad \textrm{for
285: all } \mathbf{y}\not\in E(U). \\ \label{Eq_Adm}
286: \end{equation}
287: In this paper, we focus on the maximum probability of
288: misidentification over the set of all strategies which satisfy
289: (\ref{Eq_Adm}) (denoted by $\mathcal{V}$ in the sequel). Similar
290: to~\cite{Barg_cap}, we average the probability of misidentification
291: over all possible coalitions leading to the following performance
292: metric:
293: \begin{equation} \label{Pm_Def}
294: \overline{P}_m(F,G):=\frac{1}{{M \choose t}}\sum_{U}{\max_{V \in
295: \mathcal{V}}P_m(U,F,G,V)},
296: \end{equation}
297: where
298: \[
299: P_m(U,F,G,V) := \mathbb{E}_K \Big( \sum_{\mathbf{y}\in
300: E(U),g_k(\mathbf{y})\notin U} V(\mathbf{y}\mid f_k(U))\Big ).
301: \]
302: In the case of an averaging attack, we employ the typical assumption
303: of mapping the binary fingerprints into the antipodal alphabets
304: $\{-1,1\}$ where the encoder now is defined as~\cite{Liu}
305: \begin{equation} %\label{Eq_xxx}
306: f:\{1,2,\ldots,M\}\rightarrow \{-1,+1\}^n.
307: \end{equation}
308: As anticipated from the name, the forged copy is now given by:
309: \begin{equation} \label{Eq_avg_attack}
310: \mathbf{y}=\frac{1}{t}\sum_{i=1}^t\mathbf{x}_i,
311: \end{equation}
312: where the addition is over real field. The decoder is now defined as
313: \begin{equation} \label{Eq_g_avg}
314: g:\{\mathpzc{A}_\mathbf{y}\}^n\rightarrow\{1,2,\ldots,M\},
315: \end{equation}
316: where $\mathpzc{A}_\mathbf{y}$ is the alphabets of $\mathbf{y}$, for
317: example, it is $\{-1,0,+1\}$ when $t=2$. Misidentification will
318: happen if $g(\mathbf{y}) \notin U$. Note that for $t=2$, if
319: $g(\mathbf{y}) \in U$, i.e., we trace one colluder correctly then we
320: can always trace another colluder correctly according to
321: (\ref{Eq_avg_attack}). In this special case, the performance metric
322: in (\ref{Pm_Def}) reduces to
323: \begin{equation}
324: \overline{P}^a_m:=\frac{1}{{M \choose t}} \sum_{U}(g(\mathbf{y})
325: \notin U).
326: \end{equation}
327:
328: \section{The Averaging Attack} \label{Sec_Theorem} In this section, we investigate the
329: theoretical achievable rate of fingerprinting code with the minimum
330: distance (MD) decoder under the averaging attack. First, we need the
331: following definition.
332: \begin{definition} \label{def_capacity}
333: We say that the capacity of of an ensemble of fingerprinting
334: codebooks ${\cal E}$ is $R_{\cal E}$ under MD decoding if
335:
336: \begin{enumerate}
337:
338: \item For $M=2^{nR}$ with $R<R_{\cal E}$, the average probability of misidentification over the ensemble $P_m$
339: using MD decoding goes exponentially to zero as the codelength $n$
340: goes to infinity.
341: \item Conversely, for $M=2^{nR}$ with $R>R_{\cal
342: E}$, there exists a constant $\delta>0$ such that $P_m>\delta$ for
343: sufficiently large block lengths.
344: \end{enumerate}
345: \end{definition}
346: Note that this converse in the previous definition is applicable
347: only to a specific family of codes similar to the approach taken
348: in~\cite{richardson2001cld,pishronik2004dld}. We also call a rate
349: is MD-achievable if only the first part in Definition
350: \ref{def_capacity} is met. We are now ready to prove our first result. \\
351:
352: \begin{theorem} \label{Theorem_Avg_iid}
353: The fingerprinting capacity of the i.i.d codebook ensemble when
354: $t=2$ is $R_{\cal E}=0.5$ (under the averaging attack and the MD
355: decoder).
356: \\
357: \end{theorem}
358: \begin{proof} The encoder and decoder come as follows. \\
359: \noindent \textbf{Encoder:} The encoder chooses codewords
360: uniformly and independently from all $2^n$ different vectors
361: belonging to $\{0,1\}^n$, transfers the fingerprinting codeword
362: alphabets from $\{0,1\}$ to $\{-1,+1\}$, and assigns the
363: fingerprints to the users.
364: \\ \\
365: \noindent \textbf{Decoder:} With the given forged fingerprint
366: $\mathbf{y}$, the decoder treats the position $i$ where
367: $\mathbf{y}_i=0$ as an erased position, and the others as unerased
368: positions. Let $\mathpzc{E}$ be the set of erasure positions and
369: $\overline{\mathpzc{E}} := [1:n] \setminus \mathpzc{E}$. Also let
370: $\mathbf{y}_{\overline{\mathpzc{E}}}$ denote those components of
371: $\mathbf{y}$ which are indexed by $\overline{\mathpzc{E}}$. The
372: decoder will search the codebook to find the codeword which agrees
373: with $\mathbf{y}$ in all unerased positions
374: $\mathbf{y}_{\overline{\mathpzc{E}}}$. Once the decoder finds such
375: a codeword, the decoder declares it as the pirate. A
376: misidentification occurs when the codeword of an innocent user
377: $\mathbf{z}$ is consistent with $\mathbf{y}$.
378: \\
379: Achievability: For a small $\varepsilon$, we say the assigned
380: fingerprints $\mathbf{x}_1,\mathbf{x}_2$ are \emph{close} if
381: $d_H(\mathbf{x}_1,\mathbf{x}_2)\leq n(\frac{1}{2}+\varepsilon)$,
382: here the fingerprinting alphabets are $\{0,1\}$ before
383: transformation. As shown in Appendix \ref{App_close_pair}, we know
384: that with high probability, $(\mathbf{x}_1,\mathbf{x}_2)$ are a
385: close pair. Thus, given a small $\epsilon>0$,
386: \begin{equation} %\label{Eq_xxx}
387: |\mathpzc{E}| \leq n(\frac{1}{2}+\epsilon),
388: \end{equation}
389: since the erasures happen when the bits of $(\mathbf{x}_1,\mathbf{x}_2)$ are different.
390: For the given forged fingerprint $\mathbf{y}$, $\mathbf{z}$ must
391: agree with $\mathbf{y}$ in all $n-|\mathpzc{E}|$ unerased
392: positions, and can be $-1$ or $+1$ in the rest $|\mathpzc{E}|$
393: erased positions. The probability of choosing such codeword is
394: upper-bounded by
395: \begin{equation} %\label{Eq_xxx}
396: 2^{n*(1/2+\epsilon)}/2^n.
397: \end{equation}
398: By using the union bound, we know that for $R<1/2-\epsilon$, the
399: probability of misidentification $P_m$ tends to zero exponentially
400: fast for sufficiently large
401: codeword length $n$.
402: \\ \\
403: Converse: From (\ref{Eq_random_dis}) in the Appendix, we know that
404: $P(|\mathpzc{E}| \geq n/2)>P(|\mathpzc{E}|=n/2)=\delta$, where
405: $\delta$ is non-vanishing with respect to codeword length $n$. For a
406: fingerprinting codeword $\mathbf{x}$, we form
407: $\mathbf{x}_{\overline{\mathpzc{E}}}$ as the components of
408: $\mathbf{x}$ which are indexed by $\overline{\mathpzc{E}}$. And we
409: arrange all $\mathbf{x}_{\overline{\mathpzc{E}}}$ in the
410: fingerprinting codebook as rows of a $2^{nR} \times
411: (n-|\mathpzc{E}|)$ array $\mathbf{X}_{\overline{\mathpzc{E}}}$. The
412: misidentification happens if $\mathbf{y}_{\overline{\mathpzc{E}}}$
413: equals to more than two rows of
414: $\mathbf{X}_{\overline{\mathpzc{E}}}$. With $R>1/2$, $|\mathpzc{E}|
415: \geq n/2$, and sufficiently large $n$,
416: \begin{equation} %\label{Eq_xxx}
417: 2^{nR}-2>2^{(n-|\mathpzc{E}|)}-1.
418: \end{equation}
419: And the misidentification will happen with probability at least
420: $1/3$. From above, we know that if $R>1/2$, the misidentification
421: probability will be larger than $\delta/3$ for sufficiently large
422: $n$ which
423: concludes the proof. \\
424: \end{proof}
425:
426: Intuitively, the i.i.d generated codebook will result in
427: $|\mathpzc{E}| \approx n/2$ number of erased positions with high
428: probability \cite{Barg_cap}. Then the ``channel'' between one of
429: the pirates $\mathbf{x}_1$ and the forged fingerprint $\mathbf{y}$
430: can be approximated by a binary erasure channel (BEC) with erasure
431: probability 1/2. From \cite{Book_Cover}, we know that the capacity
432: using the MD decoder of this channel is 1/2. However, in the
433: two-pirate fingerprinting system, there are always two codewords
434: $\mathbf{x}_1$ and $\mathbf{x}_2$ in the codebook which meet the
435: MD decoding criteria. This is the fundamental difference between
436: this system and the classical BEC channel. In the BEC channel,
437: with high probability, only one codeword will meet the MD decoding
438: criteria. As will be presented in Section \ref{Sec_Avg_Simu}, this
439: difference will have an important implication on the design of
440: Belief Propagation decoders for fingerprinting. The following
441: result shows that restricting ourselves to the class of linear
442: fingerprinting does not entail any performance loss (at least from
443: an information theoretic perspective) \\
444: \begin{theorem} \label{Theorem_Avg_linear}
445: The fingerprinting capacity of the binary linear ensemble with
446: $t=2$ is $R_{\cal E}=0.5$ (under the averaging attack and the MD
447: decoder). \\
448: \end{theorem}
449: \begin{proof}
450: We consider the ensemble of binary linear codes of length $n$ and
451: dimension $n-l$ defined by the $l \times n $ parity check matrix
452: $H$, where each entry of $H$ is an
453: i.i.d Bernoulli random variable with parameter $1/2$. The code rate $R=1-l/n$.\\
454:
455: \noindent \textbf{Encoder:} The encoder chooses one codebook from
456: this linear code ensemble, transfers the fingerprinting codeword
457: alphabets from $\{0,1\}$ to $\{-1,+1\}$, and assigns the
458: fingerprints to the users.
459: \\ \\
460: \noindent \textbf{Decoder:} With the given forged fingerprint
461: $\mathbf{y}$, again the decoder treats the position $i$ where
462: $\mathbf{y}_i=0$ as an erased position, and the others as unerased
463: positions. The decoder will also transfer the alphabets of
464: unerased positions from $\{-1,+1\}$ back to $\{0,1\}$. Let
465: $H_\mathpzc{E}$ denote the submatrix of $H$ that consists of those
466: columns of $H$ which are indexed by the set of erasures
467: $\mathpzc{E}$. In a similar manner, let $\mathbf{x}_\mathpzc{E}$
468: denote those components of the pirate's fingerprint which are
469: indexed by $\mathpzc{E}$, and
470: $\mathbf{x}_{\overline{\mathpzc{E}}}$ denote those components
471: which are indexed by $\overline{\mathpzc{E}}$. In the following,
472: we assume that the fingerprinting codeword alphabets are
473: transferred back to $\{0,1\}$ and the addition is module-2. Note
474: that the true pirates $\mathbf{x}_1$ and $\mathbf{x}_2$ will
475: result in the same
476: $\mathbf{x}_{\overline{\mathpzc{E}}}=\mathbf{y}_{\overline{\mathpzc{E}}}$,
477: where $\mathbf{y}_{\overline{\mathpzc{E}}}$ is defined as in
478: Theorem \ref{Theorem_Avg_iid}. From the parity check equations,
479: \begin{equation} \label{Eq_BEC_syndrome}
480: H_\mathpzc{E}\mathbf{x}^{\mathrm{T}}_\mathpzc{E}=\mathbf{s}^{\mathrm{T}},
481: \end{equation}
482: where
483: $\mathbf{s}^{\mathrm{T}}:=H_{\overline{\mathpzc{E}}}\mathbf{y}^{\mathrm{T}}_{\overline{\mathpzc{E}}}$
484: is called the syndrome. The syndrome is known at the decoder. The
485: decoder solves these linear equations to find
486: $\mathbf{x}_\mathpzc{E}$, combines it with the known
487: $\mathbf{x}_{\overline{\mathpzc{E}}}=\mathbf{y}_{\overline{\mathpzc{E}}}$,
488: and declares one of the results as the pirate.
489: \\ \\
490: Achievability: We know that (\ref{Eq_BEC_syndrome}) has at least
491: two solutions corresponding to the true pirates $\mathbf{x}_1$ and
492: $\mathbf{x}_2$. The rank of $l \times |\mathpzc{E}|$ matrix
493: $H_\mathpzc{E}$ must equal to $|\mathpzc{E}|-1$ to make sure that
494: there is only two solutions. The decoder will declare an innocent
495: user as the pirate if there are more than two solutions, iff
496: $H_\mathpzc{E}$ has rank less than $|\mathpzc{E}|-1$. This happens
497: with probability
498: \begin{equation} \label{Eq_rankless}
499: 1-\frac{M_b(l,|\mathpzc{E}|,|\mathpzc{E}|-1)}{2^{l|\mathpzc{E}|}-M_b(l,|\mathpzc{E}|,|\mathpzc{E}|)},
500: \end{equation}
501: where $M_b(l_1,m_1,k_1)$ denote the number of binary matrices with
502: dimension $l_1 \times m_1$ and rank $k_1$.
503:
504: To make (\ref{Eq_rankless}) approach zero as $n$ increases, the
505: second term in (\ref{Eq_rankless}) must approach one as $n$
506: goes up. To show this, we first assume that $|\mathpzc{E}|+n
507: \epsilon_1 \leq l$, where $\epsilon_1>0$ is a small number. And
508: according to (\ref{Eq_NumEminus1}) in Appendix \ref{App_Matrix}
509: and \cite{di2002finite}, the second term in (\ref{Eq_rankless})
510: equals
511: \begin{equation} \label{Eq_ranklessdev1}
512: \frac{M_b(|\mathpzc{E}|-1,l,|\mathpzc{E}|-1)(2^{|\mathpzc{E}|}-1)}{2^{l|\mathpzc{E}|}-M_b(|\mathpzc{E}|,l,|\mathpzc{E}|)}.
513: \end{equation}
514: From \cite{di2002finite}, for $j=0 \ldots |\mathpzc{E}|-1 $
515: \begin{align} \label{Eq_NumFullRank}
516: &M_b(|\mathpzc{E}|-j,l,|\mathpzc{E}|-j)=
517: \prod_{p=0}^{|\mathpzc{E}|-j-1}(2^l-2^p).
518: \end{align}
519: Using this formula in (\ref{Eq_ranklessdev1}) and dividing the
520: nominator and denominator by
521: $M_b(|\mathpzc{E}|-1,l,|\mathpzc{E}|-1)$, this term equals
522: \begin{equation} \label{Eq_ranklessdev2}
523: \frac{2^{|\mathpzc{E}|}-1}{2^{(|\mathpzc{E}|-1)}+2^l[-1+\prod_{p=0}^{|\mathpzc{E}|-2}1/(1-2^{p-l})]}.
524: \end{equation}
525: Note that $n\epsilon_1 \leq l-|\mathpzc{E}|$, each $2^{p-l}$
526: approaches zero exponentially fast with $n$. By using Taylor
527: series on $1/(1-2^{p-l})$, and with some simplifications, the
528: denominator becomes
529: \begin{equation} %\label{Eq_xxx}
530: 2^{(|\mathpzc{E}|-1)}+\sum_{p=0}^{|\mathpzc{E}|-2}2^{p}+2^{|\mathpzc{E}|}*h.o.t.=2^{|\mathpzc{E}|}*(1+h.o.t.)-1,
531: \end{equation}
532: where the higher order terms of the Taylor series are denoted by $h.o.t$ and approach zero exponentially fast. Using this result in
533: (\ref{Eq_ranklessdev2}), our claim is valid and
534: (\ref{Eq_rankless}) approaches zero as $n \rightarrow \infty$
535: if $|\mathpzc{E}|+n \epsilon_1 \leq l$.
536:
537: As shown in Appendix \ref{App_Theorem_Linear}, $|\mathpzc{E}| \leq
538: n(1/2+\epsilon)$ with high probability, we know that if
539: $n(1/2+\epsilon)+n \epsilon_1 \leq l$, or
540: $R<1/2-(\epsilon+\epsilon_1)$, the probability of
541: misidentification can be made arbitrary small.
542: \\ \\
543: Converse: From (\ref{Eq_linear_dis}) in Appendix, we know that
544: $P(|\mathpzc{E}| \geq n/2)>P(|\mathpzc{E}|=n/2)=\delta$, where
545: $\delta$ is non vanishing with respect to codeword length $n$.
546: With $R>1/2$ and sufficiently large $n$, $P(|\mathpzc{E}|-1 > l)
547: \geq \delta$. In this case, the rank of $H_{\mathpzc{E}}$ is less
548: than $|\mathpzc{E}|-1$ and the syndrome decoder will find at least
549: three solutions of equation (\ref{Eq_BEC_syndrome}). The
550: misidentification will happen with probability at least $1/3$
551: since. From above, we know that if $R>1/2$, the probability will
552: be larger than $\delta/3$ for sufficiently large $n$ and it
553: concludes the proof. \\
554: \end{proof}
555:
556: %\subsubsection{More than two-pirate case}
557: Next, our approach is generalized to coalitions with $t>2$. The
558: key to the following corollary is to treat all alphabets other
559: than $\pm 1$ in $\mathpzc{A}_{\mathbf{y}}$ of (\ref{Eq_g_avg}) as
560: erasures.
561: \\
562: \begin{corollary} %\label{Theorem_xxx}
563: The rate $\frac{1}{2^{(t-1)}}$ is MD-achievable for fingerprinting
564: under
565: average attack with a coalition of size $t$. \\
566: \end{corollary}
567: \begin{proof}
568: The encoder/decoder are the same as the ones in Theorem
569: \ref{Theorem_Avg_iid} except for the choices of erasure positions as
570: described previously. Note that $\mathbf{y}_i \neq \pm 1$ whenever
571: the pirates' fingerprints bits are not the same at position $i$.
572: Similar to \cite{Barg_cap}, we know that with high probability, the
573: i.i.d generated codebooks will meet
574: \[
575: |\mathpzc{E}| \leq n\left \{1-\frac{1}{2^{(t-1)}}+\epsilon \right
576: \}.
577: \]
578: Then, following in the footsteps of the proof of Theorem
579: \ref{Theorem_Avg_iid} we obtain our result.
580: \end{proof}
581: \[\;\]
582:
583: The advantage of the MD decoder, used to obtain the previous result,
584: is the universality for all $t$. However, for each $t$, we can
585: obtain higher rates by tailoring our encoder/decoder to this
586: specific case. To illustaret the idea, let's consider the $t=3$
587: case. Now, $\mathpzc{A}_{\mathbf{y}}=\{ \pm 1, \pm \frac{1}{3} \}$
588: and one can achieve better performance by exploiting the information
589: contained in the positions with $\mathbf{y}_i = \pm \frac{1}{3}$.
590: \\
591: \begin{theorem} \label{Theorem_3tJT}
592: The rate
593: $H(\frac{1}{8},\frac{1}{8},\frac{3}{8},\frac{3}{8})-H(\frac{1}{4},\frac{1}{2},\frac{1}{4})=0.3113$
594: is achievable for fingerprinting under average attack with $t=3$.
595: \\
596: \end{theorem}
597:
598: \begin{proof} The encoder is the same as Theorem
599: \ref{Theorem_Avg_iid}. As for the decoder, we first define $X$ as a random variable with
600: $P(X=\pm 1)=1/2$, and the random variable $Y=(X+X_2+X_3)/3$, where
601: $X_2,X_3$ has the same distribution as $X$ and $(X,X_2,X_3)$ are
602: independent. The transition matrix of $P(Y|X)$ is
603: \begin{table}[ht]
604: \begin {center}
605: \begin{tabular}{c|cccc}
606: $_{X} \backslash ^{Y} $&-1&-1/3&1/3&1\\ \hline
607: -1&$\frac{1}{4}$&$\frac{1}{2}$&$\frac{1}{4}$&0\\
608: 1&0&$\frac{1}{4}$&$\frac{1}{2}$&$\frac{1}{4}$. \\
609: \end{tabular}
610: \end{center}
611: \end{table}
612:
613: Typically, we need a maximum likelihood (ML) decoder designed for
614: the transition matrix $P(Y|X)$. Note that when $t=2$, this decoder
615: reduces to the one specified in Theorem \ref{Theorem_Avg_iid}.
616: However, it is hard to investigate the performance of the ML
617: decoder, and we use the jointly-typical decoder defined in
618: \cite{Book_Cover} as a lower-bound for the achievable rate of this
619: decoder. Given a forged fingerprint $\mathbf{y}$, the decoder
620: search the codebook to find the codeword such that this codeword
621: and $\mathbf{y}$ are jointly-typical with respect to $P(X,Y)$ .
622: Once the decoder finds such a codeword, the decoder declares it as
623: the pirate.
624: \\ \\
625: Achievability : Without loss of generality, we can assume that the
626: pirates indices are $(1,2,3)$. An event $E_i$ occurs when the
627: $i$th codeword and $\mathbf{y}$ are jointly typical, and the event
628: $E^c_i$ is its complement. Then the probability of
629: misidentification $P_m$ is upper-bounded by
630: \[
631: P_m \leq P(E^c_1)+P(E^c_2)+P(E^c_3)+\sum_{i \neq 1,2,3} P(E_i).
632: \]
633:
634: From \cite[Theorem 15.2.1]{Book_Cover}, the first three terms can be made less than any arbitrary small $\epsilon>0$ for sufficiently large $n$. And
635: the last term is upper-bounded by
636: \[
637: (M-3) 2^{-n(I(X;Y)-4\epsilon)},
638: \]
639: So if $R<I(X;Y)-4\epsilon$, $P_m$ can be made arbitrary small for
640: sufficiently large $n$. According to the transition matrix of
641: $P(Y|X)$, we know that
642: \[
643: I(X;Y)=H(\frac{1}{8},\frac{1}{8},\frac{3}{8},\frac{3}{8})-H(\frac{1}{4},\frac{1}{2},\frac{1}{4}),
644: \]
645: which concludes the proof
646: \end{proof}
647:
648:
649:
650: \section{The Marking Assumption} \label{Sec_Theorem_2}
651: Having studied the special case of averaging attack, we now
652: proceed to the case when the coalition can employ \emph{any}
653: strategy as long as the marking assumption is satisfied. The
654: following result establishes the achievable rate of random
655: fingerprinting codes with MD decoding \\
656: \begin{theorem} \label{Theorem_Random}
657: For all rates less than $1-H(0.25)$ there exists an MD-achievable
658: fingerprinting code, when $t=2$. \\
659: \end{theorem}
660: \begin{proof}
661: We use a random coding argument to prove our result. We construct
662: the following ensemble of binary random codes as in Theorem
663: \ref{Theorem_Avg_iid}: Binary random vectors (fingerprints) of
664: length $n$ are assigned to the $M=2^{nR}$ users where each
665: coordinate is chosen independently with equal probability of being
666: $0,1$. For a small $\varepsilon$, we say the assigned fingerprints
667: $\mathbf{x}_1,\mathbf{x}_2$ are close if
668: $d_H(\mathbf{x}_1,\mathbf{x}_2)\leq n(\frac{1}{2}+\varepsilon)$.
669: If the pair $(\mathbf{x}_1,\mathbf{x}_2)$ is close we denote it by
670: $\mathbf{x}_1\overset{C}{\leftrightarrow} \mathbf{x}_2$, otherwise
671: for a non-close pair we write:
672: $\mathbf{x}_1\overset{N}{\leftrightarrow} \mathbf{x}_2$. Given a
673: forged fingerprint $\mathbf{y}$, the average probability of
674: misidentification over this ensemble can be upper bounded by:
675: \[
676: P_m(\mathbf{y}|\mathbf{x}_1 \overset{C}{\leftrightarrow}
677: \mathbf{x}_2)+P(\mathbf{x}_1 \overset{N}{\leftrightarrow}
678: \mathbf{x}_2),
679: \]
680: where $P_m(\mathbf{y}|\mathbf{x}_1
681: \overset{C}{\leftrightarrow}\mathbf{x}_2)$ is the
682: misidentification probability when $\mathbf{y}$ is produced by a
683: close pair $(\mathbf{x}_1,\mathbf{x}_2)$ and $P(\mathbf{x}_1
684: \overset{N}{\leftrightarrow} \mathbf{x}_2)$ is the probability
685: that the pirates did not constitute a close pair. Both probability
686: are averaged over the random coding ensemble. By the following
687: argument, we will show that these probabilities goes exponentially
688: to zero as $n$ goes to infinity hence the proof.
689:
690: In Appendix \ref{App_close_pair} we have proved that
691: $P(\mathbf{x}_1 \overset{N}{\leftrightarrow} \mathbf{x}_2)$ goes
692: to zero as $n$ goes to infinity. Now we turn to
693: $P_m(\mathbf{y}|\mathbf{x}_1 \overset{C}{\leftrightarrow}
694: \mathbf{x}_2)$. Since
695: $d_H(\mathbf{x}_1,\mathbf{x}_2)<n(\frac{1}{2} + \varepsilon )$,
696: the Hamming distance of the forged copy $\mathbf{y}$ with at least
697: one of the pirates must be less than $h(n) := n(\frac{1}{4} +
698: \frac{\varepsilon}{2})$ due to the marking assumption. Without
699: loss of generality, we assume this pirate to be $\mathbf{x}_1$.
700: Using minimum Hamming distance decoding, misidentification occurs
701: if there is another binary vector $\mathbf{z}$ of length $n$ in
702: the codebook such that $d_H(\mathbf{y},\mathbf{z}) \leq
703: d_H(\mathbf{y},\mathbf{x}_1)$. The total probability of this event
704: in the random ensemble is upper-bounded by
705: \[
706: \frac{M\sum_{i=1}^{h(n)}{n\choose i}}{2^n}\doteq
707: M*2^{-n(1-H(0.25))},
708: \]
709: where the union bound is used. The probability of
710: misidentification in a random code of size $M=2^{nR}$ is at most
711: \[
712: 2^{-n(1-H(0.25)-R)}.
713: \]
714: The above probability goes exponentially to zero as $n\rightarrow
715: \infty$ for all rates $R<1-H(0.25)$. $\;$ \\
716: \end{proof}
717: Intuitively, with a high probability, the forged copy will be
718: produced by a pair of {\em close} pirates. Therefore, the minimum
719: Hamming distance between the pirates $\mathbf{x}_1$ and the forged
720: copy $\mathbf{y}$ is approximately $n/4$ implying that we can
721: treat the "channel" between them as a binary symmetric channel
722: (BSC) with crossover probability $1/4$ (whose capacity is
723: $1-H(0.25)$ \cite{Book_Cover}). Next, we extend our result to
724: binary linear codes \\
725: % \textit{\emph{Converse:}}
726: % \begin{theorem} \label{Convers}
727: %For all rates more than $1-H(0.25)$, there exists a collusion strategy for which the probability of misidentification of the %minimum distance decoding is bounded away from zero, when $t=2$.
728: %\\
729:
730: \begin{theorem} \label{Theorem_Linear}
731: For all rates less than $1-H(0.25)$, there exists a \emph{linear}
732: MD-achievable fingerprinting code, when $t=2$.
733: \\
734: \end{theorem}
735: \begin{proof}
736: Consider the ensemble of binary linear codes with binary parity
737: generator matrix $G$ where elements of $G$ are chosen equally and
738: independently from $\{0,1\}$ similar to Theorem
739: \ref{Theorem_Avg_linear}. The size of matrix $G$ is $(n-l) \times
740: n$, with rate $R=(n-l)/n$ and the codeword length $n$. It should
741: also be noted that in the following all matrix multiplications and
742: additions are done in module-2 unless otherwise stated. In order
743: to randomize the codebook, the distributor employs the following
744: strategy: Generating the secret key vectors as independent binary
745: random vectors of length $n$, whose coordinates are chosen to be
746: $0,1$ independently with probability 1/2. We denote the vector
747: indexed by secret key $k$ as $\mathbf{k}$. The vector $\mathbf{k}$
748: is added in the binary domain to the codeword, and the resulting
749: vector is assigned to the corresponding user. Note that this
750: operation will not change the detectable positions, where the
751: codewords are the different. With forged copy $\mathbf{y}$, the
752: decoder subtracts $\mathbf{k}$ and performs MD decoding. As we
753: mentioned earlier, the secret key is unknown to the users and is
754: only known to the distributor.
755:
756: Similar to the proof of Theorem \ref{Theorem_Random}, we can
757: upper-bound the probability of misidentification as
758: \begin{equation}
759: P_m(\mathbf{y}|\mathbf{x}_1 \overset{C}{\leftrightarrow}
760: \mathbf{x}_2)+P(\mathbf{x}_1 \overset{N}{\leftrightarrow}
761: \mathbf{x}_2).
762: \end{equation}
763: In Appendix \ref{App_Theorem_Linear} we have established that over
764: the ensemble of linear random codes described above,
765: $P(\mathbf{x}_1 \overset{N}{\leftrightarrow} \mathbf{x}_2)$ also
766: goes to zero as the code length goes to infinity. Now let us
767: consider $P_m(\mathbf{y}|\mathbf{x}_1 \overset{C}{\leftrightarrow}
768: \mathbf{x}_2).$ The codes assigned to the users which are the result of the addition of a secret key to a linear code can be written as:
769: \begin{equation} \label{coset}
770: \mathbf{u}G + \mathbf{k}
771: \end{equation}
772: where $\mathbf{u}$ is an information message vector. Notice that the ensemble defined by (\ref{coset}) is the same as ensemble of \emph{coset codes} introduced in \cite{Gallager}. In our proof, we need the following lemmas for the coset codes ensemble that are proved in \cite{Gallager}.
773: \newtheorem{lemma 1}{Lemma}
774: \begin{lemma} \label{Lm1}
775: The probability of any binary vector $\mathbf{v}$ being a codeword in the ensemble defined by (\ref{coset}) is equal to $2^{-n}$.
776: \end {lemma}
777: \begin{lemma} \label{Lm2}
778: Let $\mathbf{v_1}$, $\mathbf{v_2}$ be the codewords corresponding to two different information sequences $\mathbf{u_1}$, $\mathbf{u_2}$. Then over the ensemble of codes, $\mathbf{v_1}$, $\mathbf{v_2}$ are statistically independent.
779: \end {lemma}
780: Similar to the proof of Theorem \ref{Theorem_Random}, again due to the marking assumption we can
781: assume $d_H(\mathbf{y,x}_1) < h(n).$ Using MD decoding, misidentification occurs
782: if there is another binary vector $\mathbf{z}$ of length $n$ in
783: the codebook such that $d_H(\mathbf{y},\mathbf{z}) \leq
784: d_H(\mathbf{y},\mathbf{x}_1)$. The total number of binary vectors for which $d_H(\mathbf{y},\mathbf{z}) \leq
785: d_H(\mathbf{y},\mathbf{x}_1)$ can be upper bounded by: $\sum_{i=1}^{h(n)}{n\choose i} \doteq 2^{nH(0.25)}$. By Lemma \ref{Lm1} and Lemma \ref{Lm2} over the ensemble each of such vectors $\mathbf{z}$ is independent of $\mathbf{x_1}$ with probability $2^{-n}$. Therefore, the total probability of this event in the ensemble is upper-bounded by:
786: \[
787: M*2^{-n(1-H(0.25))},
788: \]
789: where again the union bound is used. The probability of
790: misidentification in a random coset code of size $M=2^{nR}$ is at most
791: \[
792: 2^{-n(1-H(0.25)-R)}.
793: \]
794: The above probability goes exponentially to zero as $n\rightarrow
795: \infty$ for all rates $R<1-H(0.25)$. $\;$ \\
796: \end{proof}
797:
798:
799:
800:
801:
802:
803:
804:
805:
806:
807:
808:
809: % where Similar to the proof of Theorem
810: % \ref{Theorem_Random}, again due to the marking assumption we can
811: % assume $d_H(\mathbf{y,x}_1) < h(n).$ Thus we can write
812: % $\mathbf{y}$ as $\mathbf{y=x}_1 \oplus \mathbf{t}$, where
813: % $\mathbf{t}$ is a binary vector with Hamming weight less than
814: % $h(n)$, and $\oplus $ is the per dimension module-2 addition. We
815: % refer to $\mathbf{t}$ as \emph{noise}. Define $T$ to be the set of
816: % all binary vectors with Hamming weights less than $h(n).$ It is
817: % obvious that
818: % \begin{equation}
819: % |T|=\sum_{i=1}^{h(n)}\left(\begin{array}{c} n \\
820: % i \end{array}\right) \doteq 2^{nH(0.25)}.
821: % \end{equation}
822: % Upon receiving a forged copy the decoder employs the following
823: % strategy. First the \emph{syndrome} $\mathbf{s}$ is computed as
824: % \[
825: % \mathbf{s}=H\mathbf{y}=H(\mathbf{x}_1 \oplus
826: % \mathbf{t})=H\mathbf{t},
827: % \]
828: % We again reiterate the fact that all matrix multiplications are
829: % done in the binary domain i.e. module-2. For all members of $T$,
830: % the decoder computes their corresponding syndromes. If there is
831: % only one member whose syndrome is equal to $\mathbf{s}$, the
832: % decoder subtracts the corresponding noise from the forged copy and
833: % declares the result to be the pirate codeword $\mathbf{x}_1$.
834: % Otherwise, if there are more than one members of $T$ resulting in
835: % the same syndrome, the one with the smallest Hamming weight is
836: % decided to be the true noise. Given $\mathbf{t}$ and $H$, an error
837: % occurs where there is another vector $\mathbf{t}'$ in $T$ with a
838: % smaller Hamming weight such that $H(\mathbf{t\oplus t}')=0$. Thus
839: % the probability of misidentification $\bar{P}_{m|H,\mathbf{t}}$
840: % given a parity check matrix $H$ and a noise vector $\mathbf{t}$ is
841: % the probability of the event that there is a vector $\mathbf{t}'
842: % \in T$ such that $H(\mathbf{t\oplus t}')=0$. It can be
843: % upper-bounded as follows
844: % \begin{equation}
845: % \bar{P}_{m|H,\mathbf{t}} \leq \sum_{\mathbf{t}' \in T\\
846: % , \mathbf{t' \neq t}} \Phi \{ H(\mathbf{t\oplus t}')=0 \},
847: % \end{equation}
848: % where $\Phi(\cdot)$ is the indicator function. Thus, given $H$
849: % \begin{equation}
850: % \bar{P}_{m|H}(\mathbf{y}|\mathbf{x}_1 \overset{C}{\leftrightarrow}
851: % \mathbf{x}_2) <
852: % \sum_{\mathbf{t} \in T} P(\mathbf{t}) \sum_{\mathbf{t}' \in T\\
853: % , \mathbf{t' \neq t}} \Phi \{ H(\mathbf{t\oplus t}')=0 \}.
854: % \end{equation}
855: % Now we try to calculate the average misidentification probability
856: % for closed-pairs over the ensemble of random linear codes
857: % $P_m(\mathbf{y}|\mathbf{x}_1 \overset{C}{\leftrightarrow}
858: % \mathbf{x}_2)$ as
859: % \begin{equation} \label{Eq_Linea_Pe1}
860: % E_{H} \left[ \bar{P}_{m|H}(\mathbf{y}|\mathbf{x}_1
861: % \overset{C}{\leftrightarrow}
862: % \mathbf{x}_2)\right] < \sum_{\mathbf{t} \in T} P(\mathbf{t}) E_H \left[ \sum_{\mathbf{t}' \in T\\
863: % , \mathbf{t' \neq t}} \Phi \{ H(\mathbf{t\oplus t}')=0 \} \right].
864: % \end{equation}
865: % Note that for all $\mathbf{t}' \in T$, $E_H[\Phi\{
866: % H(\mathbf{t\oplus t}')=0 \}]$ is the probability of the binary
867: % vector $\mathbf{v=t\oplus t}'$ being a codeword in a code chosen
868: % randomly from the ensemble. We know that this probability is
869: % independent of $\mathbf{v}$ and is equal to $2^{-l}$
870: % \cite{Gallager}. Thus
871: % \begin{equation} \label{Eq_Linea_Pe2}
872: % E_H \left[ \sum_{\mathbf{t}' \in T\\
873: % , \mathbf{t' \neq t}} \Phi \{ H(\mathbf{t\oplus t}')_2=0 \}
874: % \right]=\sum_{\mathbf{t}' \in T\\
875: % , \mathbf{t' \neq t}} E_H \left[ \Phi \{ H(\mathbf{t\oplus t}')=0 \}
876: % \right]=(|T|-1)2^{-l}.
877: % \end{equation}
878: % Using (\ref{Eq_Linea_Pe1}) and (\ref{Eq_Linea_Pe2}), we can
879: % upper-bound $P_m$ as
880: % \begin{equation}
881: % P_m < (|T|-1)2^{-l} \sum_{\mathbf{t} \in T} P(\mathbf{t}) <
882: % (|T|-1)2^{-l} \doteq 2^{n(H(0.25)-\frac{l}{n})}.
883: % \end{equation}
884: % Recall that $R=1-l/n$, it is straightforward to see that for all
885: % rates less than $1-H(0.25)$, the average probability of
886: % misidentification goes exponentially to zero with the codelength.
887: % Thus, there is at least one codebook in the ensemble with the same
888: % property. $\;$ \\
889: % \end{proof}
890:
891: When the coalition size, $t$ is larger than two, the minimum
892: distance decoding will fail due to the following argument. Let
893: $t=3$ and assume that the forged copy is produced by
894: \[
895: \mathbf{y=x}_1 + \mathbf{x}_2 + \mathbf{x}_{3},
896: \]
897: where the additions are modulo-2. It is easy to check that this
898: attack satisfies the marking assumption. For $t > 3$ the coalition
899: can consider only three of the pirates, ignore the rest and apply
900: this attack. Following the footsteps in the proof of Theorem
901: \ref{Theorem_3tJT}, it is easy to see that the MD-achievable rate
902: is zero. Indeed, it can also be shown that the resulting ``BSC
903: channel" has crossover probability $1/2$, and this negative result
904: is obtained \cite{Book_Cover}.
905:
906: \section{Belief Propagation for Fingerprinting} \label{Sec_BP} Implementing the exact minimum distance decoder may require
907: prohibitive complexity (especially for large codeword lengths).
908: This motivates our approach of using the BP framework to
909: approximate the MD decoder. More specifically, in this section, we
910: present explicit constructing of graph-based codes, along with the
911: corresponding BP decoders, which are tailored for the
912: fingerprinting application.
913: \subsection{Averaging attack}
914: \label{Sec_Avg_Simu} As remarked earlier, the two-pirate averaging
915: attack will produce a ``channel'' {\em almost equivalent} to the
916: classical BEC. This inspires the use of graphical codes based on
917: the Repeat Accumulate (RA) framework \cite{divsalar1998ctt}, such
918: as the nonsystematic irregular RA code of \cite{pfister2005cae}
919: and the irregular ARA code of \cite{Pfister2007ARA}, which were
920: shown to be capacity achieving for the BEC. In our simulations, we
921: use the original regular RA codes of \cite{divsalar1998ctt} due to
922: their simplicity and good performance for low rate scenarios. It
923: is worth noting that all the techniques discussed in the sequel
924: can be applied directly to the irregular codes presented in
925: \cite{pfister2005cae,Pfister2007ARA}. For the sake of completeness
926: we review briefly the encoding procedure for regular RA codes:
927: first, the information bits are repeated a constant number of
928: times (by a regular repetition code) and interleaved. The
929: interleaved bits are then accumulated to generate the code
930: symbols. Similarly, one can employ the standard BP iterative
931: decoding approach \cite{luby2001eec} to identify the pirates.
932: However, we argue next that significant performance improvement
933: can be obtained via a key modification to the iterative
934: decoder\footnote{in the following, the fingerprinting codeword
935: alphabets are $\{0,1\}$ after decoder transformation and the
936: addition is module-2.}.
937:
938: It is well known that the standard iterative algorithm will fail if
939: a stopping set exists in the erased positions \cite{di2002finite}.
940: Unfortunately, a stopping set always will exist in the erased
941: positions produced by averaging attack. To see this, it is more
942: convenient to represent the RA code using the appropriate bipartite
943: Tanner graph containing a set of variables
944: $\mathpzc{V}=\{v_1,v_2,\ldots\}$ and a set of check nodes. The
945: reader are referred to
946: \cite{divsalar1998ctt,pfister2005cae,Pfister2007ARA} for more
947: details on the graphical representation of RA codes. A stopping set
948: $\mathpzc{S}$ is, therefore, a subset of $\mathpzc{V}$, such that
949: all neighbors of $\mathpzc{S}$ are connected to $\mathpzc{S}$ at
950: least twice. The standard BP algorithm
951: can now be stated as the follows. \\
952:
953: \noindent \textit{[Standard BP]}:
954: \begin{enumerate}
955: \item Find a check node that satisfies the following
956: \begin{itemize}
957: \item This check node is not labelled as ``finished''.
958: \item The values of all but one of the variable nodes connected to the check node are known.
959: \end{itemize}
960: Set the value of the
961: unknown erased one to be the module-2 addition of the other variable nodes. And label
962: that check node as ``finished''.
963: \item \textit{Repeat} step 1 until all check nodes are labeled as ``finished''
964: or the decoding cannot continue further. If the latter happens,
965: declare the decoding fail.
966: \\
967: \end{enumerate}
968: It is now easy to see that, in the stopping set, every check node
969: is connected to at least two erased variable nodes and the decoder
970: will halt at this point. The following result establishes the
971: limitation of the standard BP decoder in our fingerprinting
972: scenario \\
973:
974: \begin{proposition} %\label{Propo_xxx}
975: Let $\mathpzc{V}_{B1}$ and $\mathpzc{V}_{B2}$ be the set of values
976: of the variable node set $\mathpzc{V}$ corresponding to pirate
977: fingerprints $\mathbf{x}_1$ and $\mathbf{x}_2$, respectively. And
978: let $\mathpzc{V}_d$ be the set of variable nodes where the
979: corresponding values in $\mathpzc{V}_{B1}$ and $\mathpzc{V}_{B2}$
980: are different. Then $\mathpzc{V}_d$ is a stopping set.
981: \\
982: \end{proposition}
983:
984: \begin{proof}
985: This statement is proved by contradiction. First we assume that
986: $\mathpzc{V}_d$ is not a stopping set. It means that $\exists j \in
987: \bigcup_{i \in \mathpzc{V}_d}N(i)$ where the check node $j$ has only
988: one neighbor $i'$ in $\mathpzc{V}_d$. Here we denote the neighbor of
989: node $i$ in the graph as $N(i)$. For the neighboring variable nodes
990: of this check node, we have
991: \begin{equation} \label{Eq_Stop_dev1}
992: \left\{
993: \begin{array}{ll}
994: \mathpzc{V}_{B1}(i)=\mathpzc{V}_{B2}(i) & \forall i \neq i', i \in N(j) \\
995: \mathpzc{V}_{B1}(i')=\mathpzc{V}_{B2}(i')+1 & i' \in N(j). \\
996: \end{array} \right.
997: \end{equation}
998: However, from the check equation of this check node
999: \begin{equation} \label{Eq_Stop_dev2}
1000: \sum_{i \in N(j)} \mathpzc{V}_{B1}(i)=\sum_{i \in N(j)}
1001: \mathpzc{V}_{B2}(i)=0,
1002: \end{equation}
1003: where the addition is module-2. It is obvious that
1004: (\ref{Eq_Stop_dev1}) contradicts with (\ref{Eq_Stop_dev2}) since
1005: the total number of variable nodes such that $\mathpzc{V}_{B1}(i)
1006: \neq \mathpzc{V}_{B2}(i), i \in N(j)$ should be even. Thus
1007: $\mathpzc{V}_d$ is a stopping set. \\
1008: \end{proof}
1009: Since, under averaging attack, the bits of the forged fingerprint
1010: will be erased whenever the pirate fingerprints are different in
1011: the Tanner graph, then $\mathpzc{V}_d$ will be always contained in
1012: the erased positions and the iterative decoder will fail. The
1013: modification, presented next, will break the stopping set
1014: $\mathpzc{V}_d$, and hence, allow the iterative decoder to proceed
1015: forward. The key observation is that for every erased position in
1016: $\mathpzc{V}_d$, the pirate fingerprints can only be represented
1017: by only two combinations $\{0,1\}$ or $\{1,0\}$. It allows us to
1018: choose one variable node in this stopping set, and set its value
1019: to $1$. The modified forged
1020: fingerprint will then be ``closer'' to one of the pirate fingerprints. In summary, the decoder becomes\\
1021:
1022: \noindent \textit{[Modified BP for fingerprinting]}:
1023:
1024: \begin{enumerate}
1025: \item Perform the standard BP algorithm, remove all the ``finished'' labels and \textit{Go to} step $2$
1026: \item Choose a proper variable node in $\mathpzc{V}_d$ (different from previous choices), and set
1027: its value to $1$. If the decoder has executed this step more
1028: than $N_{max}$ times, declare a decoding failure and exit.
1029: \item Run the standard BP on the new graph. If the decoder fails,
1030: reset the variable nodes to their original values and \textit{Go
1031: to} step $2$.
1032: \\
1033: \end{enumerate}
1034:
1035: \begin{figure}
1036: \centering \epsfig{file=./figures/Var_Choose.eps ,
1037: width=0.8\textwidth} \caption{Proper variable node to be chosen in
1038: step 2 of proposed modified BP algorithm for two-pirate averaging
1039: attack.} \label{Fig_var_choose}
1040: \end{figure}
1041:
1042: In step $2$, we must make sure that the chosen variable node
1043: breaks the stopping set $\mathpzc{V}_d$. The neighboring variables
1044: nodes of a degree-3 check node in RA code are good choices. From
1045: the check equations in (\ref{Eq_Stop_dev2}), the erased variables
1046: nodes will appear in pair. If we set the value of one of the two
1047: erased neighbor variable node $v_i$ as 1, this degree-3 check node
1048: is connected to $\mathpzc{V}_d \setminus v_i$ with only one edge.
1049: Then $\mathpzc{V}_d \setminus v_i$ is not a stopping set. We also
1050: need to choose the variable node which will affect as much other
1051: variable nodes in $\mathpzc{V}_d \setminus v_i$ as possible by
1052: setting its value. Since all check nodes of RA code are degree-3,
1053: we choose such variable node $v_i$ in the degree-2
1054: variable-node-chain of RA code, as shown in Fig.
1055: \ref{Fig_var_choose}. The check node is depicted as $\boxplus$,
1056: the unerased variable nodes as black circle and the erased ones as
1057: hollow circle. Furthermore, each variable node which will benefit
1058: from guessing $v_{i}$ is shown as hollow circle with the letter
1059: ``A'' in the figure. The key observation is that, for node $v_i$,
1060: the two neighboring accumulator output nodes, i.e., $v_{i-1}$ and
1061: $v_{i+1}$, correspond to non-erased bits. This implies that that
1062: setting the value of $v_i$ will at least affect $6$ other variable
1063: nodes of rate $1/3$ RA code.
1064:
1065: \begin{figure}
1066: \centering \epsfig{file=./figures/FP_RA_BEC_Rate.eps ,
1067: width=0.8\textwidth} \caption{Probability of misidentification
1068: under two-pirate averaging attack using RA codes with different
1069: rates and modified BP algorithm without variable node selection.}
1070: \label{Fig_FP_RA_BEC_Rate}
1071: \end{figure}
1072:
1073: \begin{figure}
1074: \centering \epsfig{file=./figures/FP_RA_BEC.eps ,
1075: width=0.8\textwidth} \caption{Probability of misidentification
1076: under two-pirate averaging attack using rate 1/3 RA code and
1077: modified BP algorithm with different $N_{max}$.}
1078: \label{Fig_FP_RA_BEC}
1079: \end{figure}
1080:
1081: \begin{figure}
1082: \centering \epsfig{file=./figures/FP_RA_BEC_Len.eps,
1083: width=0.8\textwidth} \caption{Probability of misidentification
1084: under two-pirate averaging attack using rate 1/3 RA code and
1085: modified BP algorithm with different code lengths $n$.}
1086: \label{Fig_FP_RA_BEC_Len}
1087: \end{figure}
1088:
1089: Now, we are ready to report our simulation results. First, we show
1090: the performance of proposed algorithm with different rate RA codes
1091: without variable node selection in Fig~\ref{Fig_FP_RA_BEC_Rate}
1092: (i.e., we select the first unerased variable node in the RA
1093: degree-2 variable-node-chain and set $N_{max}=1$). Here, the
1094: number of information bits $n/R=16384$ is fixed for all rates, to
1095: make the number of users $M$ the same. We observe that, without
1096: selecting the variable node as shown in Fig~\ref{Fig_var_choose},
1097: the probability of misidentification $\bar{P}^a_m$ is high for
1098: rate $1/3$. This performance can be improved by the proposed
1099: algorithm for variable node selection and increasing $N_{max}$ as
1100: depicted in Fig.~\ref{Fig_FP_RA_BEC}. Finally, in
1101: Fig.~\ref{Fig_FP_RA_BEC_Len} we report $\bar{P}^a_m$ with
1102: different code length $n$ and $N_{max}=2$.
1103:
1104: Finally, we note that our algorithm is similar, in spirit, to the
1105: proposed guessing algorithm in \cite{pishronik2004dld}. The critical
1106: difference is that the structure of our problem ensures that the
1107: guessed bit always corresponds to one of the pirates, and hence, we
1108: do not need to worry about the possibility of contradictions as the
1109: iteration proceeds.
1110: \subsection{The Marking Assumption: The Memoryless Attack}
1111: In this subsection, we report our simulation results for the
1112: two-pirate memoryless attack. In this attack, when the pirates
1113: encounter a detectable position, they choose $0,1$ independently
1114: and with equal probability to form the forged copy. We use rates
1115: $1/8$, $1/9$ and $1/10$ ARA codes based on the low rate
1116: protographs presented in \cite{divsalar}. The protographs of the
1117: codes are depicted in Fig~\ref{Fig_Prot}. For a formal description
1118: of the ARA codes, we refer the interested readers to
1119: \cite{divsalar}, \cite{Pfister2007ARA} and references therein.
1120: Decoding is done iteratively using the BP framework with a maximum
1121: number of iterations equal to $60$. Here, the decoder treats the
1122: forged fingerprint as the output of a BSC with crossover
1123: probability equal to $0.25$. In Fig~\ref{Pm_Fig}, the probability
1124: of misidentification $\bar{P}_m$ is depicted versus different code
1125: lengths for different rates. As shown in the figure, it is clear a
1126: vanishing small misidentification probability is achievable for
1127: rate $1/9$ which is about an order of magnitude higher than the
1128: best result available in the literature for explicit
1129: fingerprinting codes.
1130: \begin{figure}
1131: \centering \epsfig{file=./all_prot.eps , width=0.8\textwidth}
1132: \caption{Protographs of rate 1/8, 1/9, 1/10 ARA codes.}
1133: \label{Fig_Prot}
1134: \end{figure}
1135:
1136: \begin{figure}
1137: \centering \epsfig{file=./All_codes_ver2.eps ,
1138: width=0.8\textwidth} \caption{Probability of misidentification for
1139: ARA codes with different rates and code lengths, under two-pirate
1140: memoryless attack.} \label{Pm_Fig}
1141: \end{figure}
1142:
1143: \section{Conclusion} \label{Sec_conclu}
1144: This paper developed an information theoretic framework for the
1145: design of low complexity coding/decoding techniques for
1146: fingerprinting. More specifically, we established the superior
1147: performance of the minimum distance decoder and validated our
1148: theoretical claims via explicit construction of BP
1149: encoding/decoding schemes. In the averaging attack scenario, our
1150: framework was inspired by the equivalence between our problem and
1151: the BEC. We also showed that the worst case attack, under the
1152: marking assumption, is equivalent to a BSC with a cross-over
1153: probability equal to $1/4$. Our approach for the averaging attack
1154: can handle arbitrary coalition sizes, whereas it was shown that
1155: the MD decoder recover from marking assumption attacks only with
1156: coalitions composed of two pirates. This negative result motives
1157: our current investigations on more sophisticated approaches for
1158: pirate tracing using the intimate connection between collusion in
1159: digital fingerprinting and multiple access channels.
1160:
1161:
1162:
1163: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1164: %APPENDIX
1165: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1166: %\useRomanappendicesfalse
1167: \appendices
1168: \section{On non-close pairs in random ensemble}
1169: We will examine the probability of non-close pairs for random
1170: i.i.d and linear codebook ensembles, and show that these events
1171: will not happen with high probability.
1172: \subsection{i.i.d codebook ensemble} \label{App_close_pair}
1173: For a codebook $C$ in the i.i.d ensemble and $1\leq d\leq n$,
1174: define the number of unordered pairs of codewords
1175: $(\mathbf{x}_i,\mathbf{x}_j)$ with $i \neq j$ in $C$ at distance
1176: $d$ apart as
1177: \begin{equation}
1178: S_c(d):=\sum_{i=1}^{M} \sum_{j=1}^{i-1} \Phi \{
1179: d_{H}(\mathbf{x}_i,\mathbf{x}_j)=d\},
1180: \end{equation}
1181: where $\Phi (\cdot )$ is the indicator function. In \cite{Forney},
1182: it is established that with probability going to one as
1183: $n\rightarrow \infty $
1184: \begin{equation} \label{Eq_random_dis}
1185: S_c(d) \doteq \left\{ \begin{array}{ll}
1186: 2^{n(2R+H(\frac{d}{n})-1)} & \textrm{ $n\delta_{GV}(2R)<d<n(1-\delta_{GV}(2R)) $}\\
1187: 0 & \textrm{otherwise,}\\
1188: \end{array} \right.
1189: \end{equation}
1190: where $\delta_{GV}(\cdot) $ is the Gilbert-Varshamov distance
1191: which for $0<R<1$, $\delta_{GV}(R) $ is defined as the root
1192: $\delta<0.5$ of the equation $H(\delta)=1-R$. And $\delta_{GV}(R)
1193: $ is zero for $R \geq 1$. Using (\ref{Eq_random_dis}), we can
1194: write the probability of non-close pairs in the codes of the
1195: random ensemble as
1196: \begin{equation}
1197: \frac{\sum_{d>n(1/2+\epsilon)}^{n(1-\delta_{GV}(2R))}2^{n(2R+H(d/n)-1)}}{2^{2nR}}<\frac{n2^{n(2R-1+H(\frac{1}{2}+\epsilon))}}{2^{2nR}}.
1198: \end{equation}
1199: which goes exponentially to zero as $n \rightarrow \infty.$
1200: \subsection{Random binary linear codebook ensemble} \label{App_Theorem_Linear}
1201: For a code $C$ in the linear ensemble and $1 \leq d \leq n$ by
1202: the symmetry of linear codes we can write
1203: \begin{equation}
1204: S_c(d) = \sum_{i=1}^{M} \sum_{j=1}^{i-1} \Phi \{
1205: d_{H}(\mathbf{x}_i,\mathbf{x}_j)=d\}=\frac{1}{2}\sum_{i=1}^{M}
1206: \sum_{j \neq i} \Phi \{
1207: d_{H}(\mathbf{x}_i,\mathbf{x}_j)=d\}=\frac{M}{2}N_c(d)\doteq
1208: 2^{nR} N_c(d),
1209: \end{equation}
1210: where $N_c(d) := \sum_{j \neq i} \Phi \{
1211: d_{H}(\mathbf{x}_i,\mathbf{x}_j)=d\}$. In \cite{Forney}, it is
1212: shown that with probability going to one as $n \rightarrow \infty$
1213: \begin{equation} \label{Eq_linear_dis}
1214: N_c(d) \doteq \{
1215: \begin{array}{cc} 2^{n(R+H(d/n)-1)} , &n
1216: \delta_{GV}(R) <d<n(1-\delta_{GV}(R)) \\ 0 , & \; otherwise.
1217: \end{array}
1218: \end{equation}
1219: Therefore, the average probability of a pair being non-close can
1220: be written as
1221: \begin{equation}
1222: \frac{\sum_{d>n(1/2+\epsilon)}^{n(1-\delta_{GV}(R))}2^{n(2R+H(d/n)-1)}}{2^{2nR}}<\frac{n2^{n(2R-1+H(\frac{1}{2}+\epsilon))}}{2^{2nR}},
1223: \end{equation}
1224: which again goes exponentially to zero as $n \rightarrow \infty.$
1225:
1226: \section{Computation of $M_b(l,|\mathpzc{E}|,|\mathpzc{E}|-1)$} \label{App_Matrix}
1227: We will show that for $l \geq |\mathpzc{E}|$
1228: \begin{equation} \label{Eq_NumEminus1}
1229: M_b(l,|\mathpzc{E}|,|\mathpzc{E}|-1)=M_b(|\mathpzc{E}|-1,l,|\mathpzc{E}|-1)(2^{|\mathpzc{E}|}-1).
1230: \end{equation}
1231:
1232: To this end, by symmetry,
1233: \[
1234: M_b(l,|\mathpzc{E}|,|\mathpzc{E}|-1)=M_b(|\mathpzc{E}|,l,|\mathpzc{E}|-1).
1235: \]
1236: And from Appendix A of \cite{di2002finite} and $|\mathpzc{E}| \leq
1237: l$, the RHS equals to
1238: \begin{align} \notag
1239: &M_b(|\mathpzc{E}|,l,|\mathpzc{E}|-1)=M_b(|\mathpzc{E}|-1,l,|\mathpzc{E}|-1)2^{|\mathpzc{E}|-1}\\
1240: \notag&+M_b(|\mathpzc{E}|-1,l,|\mathpzc{E}|-2)(2^{l}-2^{|\mathpzc{E}|-2}).
1241: \notag
1242: \end{align}
1243: From Appendix A of \cite{di2002finite}, we also have the following
1244: recursive formula for $j=1 \ldots |\mathpzc{E}|-2 $
1245: \begin{align} \notag
1246: &M_b(|\mathpzc{E}|-j,l,|\mathpzc{E}|-1-j)=M_b(|\mathpzc{E}|-1-j,l,|\mathpzc{E}|-1-j)2^{|\mathpzc{E}|-1-j}\\
1247: \notag&+M_b(|\mathpzc{E}|-1-j,l,|\mathpzc{E}|-2-j)(2^{l}-2^{|\mathpzc{E}|-2-j}).
1248: \end{align}
1249: And $M_b(|\mathpzc{E}|,l,|\mathpzc{E}|-1)$ equals to
1250: \begin{align} \label{Eq_NumEminus1_dev_1}
1251: \sum_{j=1}^{|\mathpzc{E}|-1} & \left \{
1252: M_b(|\mathpzc{E}|-j,l,|\mathpzc{E}|-j)2^{|\mathpzc{E}|-j}
1253: \prod_{p=1}^{j-1}(2^l-2^{|\mathpzc{E}|-1-p}) \right \} \\ \notag +
1254: & M_b(1,l,0)*(2^l-1)
1255: \prod_{p=1}^{|\mathpzc{E}|-2}(2^l-2^{|\mathpzc{E}|-1-p}),
1256: \end{align}
1257: where $M_b(1,l,0)=1$.
1258:
1259: Finally, using (\ref{Eq_NumFullRank}) in
1260: (\ref{Eq_NumEminus1_dev_1}),
1261: \[
1262: M_b(|\mathpzc{E}|,l,|\mathpzc{E}|-1)=\sum_{j=1}^{|\mathpzc{E}|}M_b(|\mathpzc{E}|-1,l,|\mathpzc{E}|-1)2^{|\mathpzc{E}|-j},
1263: \]
1264: And it is easy to check that the above formula equals to
1265: (\ref{Eq_NumEminus1}).
1266:
1267:
1268: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1269: %Reference
1270: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1271: % Old \bibliographystyle{IEEEbib}
1272: \bibliographystyle{IEEEtran}
1273: % \bibliographystyle{unsrt}
1274: \bibliography{FP}
1275:
1276: \end{document}
1277: