1:
2:
3: %----------------------------------------------------------------
4: %%%%%%%%%%%%%%%%%%%%5Check-
5:
6: % check whether to use pseudo-additivity or nonextensive additivity
7:
8:
9:
10: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
11: % INSTITUTE OF PHYSICS PUBLISHING %
12: % %
13: % `Preparing an article for publication in an Institute of Physics %
14: % Publishing journal using LaTeX' %
15: % %
16: % LaTeX source code `ioplau2e.tex' used to generate `author %
17: % guidelines', the documentation explaining and demonstrating use %
18: % of the Institute of Physics Publishing LaTeX preprint files %
19: % `iopart.cls, iopart12.clo and iopart10.clo'. %
20: % %
21: % `ioplau2e.tex' itself uses LaTeX with `iopart.cls' %
22: % %
23: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
24: %
25: %
26: % First we have a character check
27: %
28: % ! exclamation mark " double quote
29: % # hash ` opening quote (grave)
30: % & ampersand ' closing quote (acute)
31: % $ dollar % percent
32: % ( open parenthesis ) close paren.
33: % - hyphen = equals sign
34: % | vertical bar ~ tilde
35: % @ at sign _ underscore
36: % { open curly brace } close curly
37: % [ open square ] close square bracket
38: % + plus sign ; semi-colon
39: % * asterisk : colon
40: % < open angle bracket > close angle
41: % , comma . full stop
42: % ? question mark / forward slash
43: % \ backslash ^ circumflex
44: %
45: % ABCDEFGHIJKLMNOPQRSTUVWXYZ
46: % abcdefghijklmnopqrstuvwxyz
47: % 1234567890
48: %
49: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
50: %
51: \documentclass[12pt]{iopart}
52: \newcommand{\gguide}{{\it Preparing graphics for IOP journals}}
53:
54: %==============================
55: %Mine
56:
57: \usepackage{amssymb}
58: \usepackage{amsthm}
59:
60: %------------------theorm env---------------
61: \newtheorem{theorem}{Theorem}[section]
62: \newtheorem{lemma}[theorem]{Lemma}
63: \newtheorem{proposition}[theorem]{Proposition}
64: \newtheorem{corollary}[theorem]{Corollary}
65: \newtheorem{definition}[theorem]{Definition}
66: \newtheorem{remark}[theorem]{Remark}
67: % \def\QED{\mbox{\rule[0pt]{1.5ex}{1.5ex}}}
68: % \def\proof{\noindent\hspace{2em}{\it Proof: }}
69: % \def\endproof{\hspace*{\fill}~\QED\par\endtrivlist\unskip}
70: %--------------------------------------------
71: \newcommand{\ud}{\mathrm{d}}
72: %=====================================
73:
74:
75: %Uncomment next line if AMS fonts required
76: %\usepackage{iopams}
77: \begin{document}
78:
79: %\title[R\'{e}nyi's Recipe and Nonextensitivity]{R\'{e}nyi's
80: %Recipe and Nonextensitivity: A Characterization Theorem for Tsallis
81: %Entropy}
82:
83: \title[]{Uniqueness of Nonextensive entropy under \\ R\'{e}nyi's Recipe}
84:
85: \author{Ambedkar Dukkipati\footnote{Corresponding author}, M Narasimha
86: Murty and Shalabh Bhatnagar}
87:
88: \address{Department of Computer Science and Automation,
89: Indian Institute of Science, Bangalore-560012, India.}
90: \ead{\mailto{ambedkar@csa.iisc.ernet.in},
91: \mailto{mnm@csa.iisc.ernet.in}, \mailto{shalabh@csa.iisc.ernet.in}}
92:
93:
94: %----------------------------------------
95: \begin{abstract}
96: By replacing linear
97: averaging in Shannon entropy with Kolmogorov-Nagumo
98: average (KN-averages) or quasilinear mean and further
99: imposing the additivity
100: constraint, R\'{e}nyi proposed the first formal generalization of
101: Shannon entropy. Using this recipe of R\'{e}nyi, one can prepare only
102: two information measures:
103: Shannon and R\'{e}nyi entropy. Indeed, using this formalism
104: R\'{e}nyi characterized these additive entropies in terms of
105: axioms of quasilinear mean. As additivity is a characteristic
106: property of Shannon entropy, pseudo-additivity of the form $x \oplus_{q}
107: y = x + y + (1-q)x y$ is a characteristic property of
108: nonextensive (or Tsallis)
109: entropy.
110: One can apply R\'{e}nyi's recipe in the nonextensive case by
111: replacing the linear averaging in
112: Tsallis entropy with KN-averages and thereby imposing the
113: constraint of
114: pseudo-additivity.
115: In this paper we show that nonextensive entropy is unique
116: under the R\'{e}nyi's recipe, and there by give a
117: characterization.
118: \end{abstract}
119:
120: %Uncomment for PACS numbers title message
121: \pacs{ 65.40.Gr, 89.70.+c, 02.70.Rr}
122: % Keywords required only for MST, PB, PMB, PM, JOA, JOB?
123: %\vspace{2pc}
124: %\noindent{\it Keywords}: Article preparation, IOP journals
125: % Uncomment for Submitted to journal title message
126: %\submitto{\JPA}
127: % Comment out if separate title page not required
128: \maketitle
129:
130: %=========================Introduction===========================
131: \section{Introduction}
132:
133: In recent years, interest in generalized information measures
134: has increased dramatically, after the introduction of
135: {\em nonextensive entropy} in Physics
136: in 1988 by
137: Tsallis~\cite{Tsallis:1988:GeneralizationOfBoltzmannGibbsStatistics}.
138: One can get this nonextensive entropy or Tsallis entropy by
139: generalizing the information of single
140: event in the definition of Shannon entropy, by replacing
141: logarithm with so
142: called $q$-logarithm,
143: which is defined as
144: $\ln_{q} x = \frac{x^{1-q}-1}{1-q}$. Tsallis entropy does not
145: satisfy the additivity property which is a characteristic
146: property of Shannon entropy. Instead, it satisfies
147: pseudo-additivity of the form
148: $x \oplus_{q} y = x + y + (1-q)xy$ and this
149: definition of entropy
150: (also known as nonextensive entropy) led
151: to the field of nonextensive statistical mechanics in
152: Physics. In this paper we use the term pseudo-addition to
153: represent the binary operation $x \oplus_{q} y = x + y +
154: (1-q)xy$ for any $q \in \mathbb{R}$ and $q > 0$.
155:
156: Tsallis entropy is considered as a useful
157: measure in describing the thermostatistical properties of a
158: certain class of physical systems that entail long-range
159: interactions, long-term memories and multi-fractal structures.
160: Tsallis entropy is also studied in information theory and
161: Shannon-Khinchin axioms have been generalized to
162: nonextensive case. While
163: canonical distributions resulting from maximization of
164: Shannon entropy are exponential in nature, in the
165: Tsallis case, these result in power-law distributions. To a great extent, the success of Tsallis proposal is due to
166: the ubiquity of power law distributions in nature.
167:
168: Indeed, the starting point of the theory of generalized measures of
169: information is due to Alfred
170: R{\'{e}}nyi~\cite{Renyi:1960:SomeFundamentalQuestionsOfInformationTheory,Renyi:1961:OnMeasuresOfEntropyAndInformation}.
171: By using Kolmogorov-Nagumo averages (KN-average)
172: R\'{e}nyi introduced a
173: generalized information measure, known as $\alpha$-entropy or
174: R\'{e}nyi entropy, the first formal well-known generalization
175: of Shannon entropy.
176: {\em KN-average} or quasilinear mean (we use these two
177: terms interchangeably) is of
178: the form
179: ${\langle x
180: \rangle}_{\psi} = \psi^{-1} \left (\sum_{k} p_{k}
181: \psi(x_{k})\right)$,
182: where $\psi$ is an arbitrary continuous
183: and strictly monotone function.
184: Replacing linear
185: averaging in Shannon entropy with KN-averages and further
186: imposing the additivity
187: constraint -- a characteristic property of underlying
188: information associated with single event, which is
189: logarithmic -- leads to {\em R\'{e}nyi
190: entropy}.
191: Using this recipe of R\'{e}nyi, one can prepare only
192: two information measures:
193: Shannon and R\'{e}nyi entropy. Using this formalism
194: R\'{e}nyi characterized these additive entropies in terms of
195: axioms of KN-averages.
196:
197: One can apply R\'{e}nyi's recipe in the nonextensive case by
198: replacing the linear averaging in
199: Tsallis entropy with KN-averages and thereby imposing the
200: constraint of
201: pseudo-additivity.
202: A natural question arises: what are all the pseudo-additive
203: information measures one can prepare with this recipe? We
204: prove that only Tsallis entropy is possible in this case,
205: which allows us to characterize
206: Tsallis entropy based on axioms of KN-averages.
207:
208: % Tsallis and R{\'{e}}nyi entropy measures are two possible
209: % different generalization of the Shannon entropy but are not
210: % generalizations of each other.
211:
212: To understand these generalizations, the so called Hartley
213: function~\cite{Hartley:1928:TransmissionOfInformation} of a
214: single stochastic event plays a fundamental role. We discuss
215: Hartley function in
216: \S~\ref{Section:KN-avearagesAndInformationMeasures} along
217: with a brief discussion on quasilinear mean and R\'{e}nyi
218: entropy. The main results of this paper, on uniqueness of Tsallis
219: entropy under R\'{e}nyi's recipe and a result on
220: characterization of Tsallis entropy are presented in
221: \S~\ref{Section:RenyisRecipieAndTsallisEntropy} and
222: \S~\ref{Section:AcharacterizationTheoremForTsallisEntropy}
223: respectively.
224:
225: %====================================================================
226: \section{KN-averages and Information measures}
227: \label{Section:KN-avearagesAndInformationMeasures}
228:
229: \subsection{Hartley Function and Shannon Entropy}
230:
231: Let $X$ be a discrete random variable (r.v) defined on some
232: probability space, which takes only $n$ values, $n < \infty$.
233: We denote the set of all such random
234: variables by $\mathcal{X}$. Corresponding
235: to the $n$-tuple $(x_{1}, \ldots, x_{n})$ of values which $X$
236: takes, probability mass function (pmf) of
237: $X$ is denoted by $p = (p_{1}, \ldots p_{n})$, where $p_{k}
238: \geq 0$ for $k = 1, \ldots n$ and $\sum_{k=1}^{n} p_{k}
239: =1$. Expectation of r.v $X$ is denoted by $EX$ or $\langle X
240: \rangle$; in this paper we use both the notations,
241: interchangeably.
242:
243: Shannon entropy, a logarithmic measure of information on $X$ denoted by $S(X)$,
244: reads~\cite{Shannon:1948:MathematicalTheoryOfCommunication_BellLabs}
245: \begin{equation}
246: \label{Equation:DefinitionOfShannonEntropy}
247: S(X) = - \sum_{k=1}^{n} p_{k} \ln p_{k} \enspace,
248: \end{equation}
249: and measures the average lack of information that is
250: inherent in $p$.
251:
252: This motivation to quantify information in terms of logarithmic
253: functions is due to
254: Hartley~\cite{Hartley:1928:TransmissionOfInformation}, who
255: first used a logarithmic function to define uncertainty
256: associated with a finite set.
257: This is known as Hartley information measure.
258: The Hartley information measure of a
259: finite set $A$ with $n$ elements is defined as
260: $H(A) = \log_{b} n$.
261: If the base of the logarithm is $2$, then the uncertainty is
262: measured in {\em bits}, and in the case of natural logarithm,
263: the unit is nats. Throughout this paper we use only natural
264: logarithm as a convention.
265:
266: One can give a more general definition of Hartley information
267: measure, which is a special case of Shannon entropy as
268: follows. Define a function $H:
269: \{x_{1}, \ldots, x_{n} \} \rightarrow \mathbb{R}$ of the
270: values taken by r.v $X \in \mathcal{X}$ with corresponding
271: p.m.f $p = (p_{1}, \ldots p_{n})$
272: as~\cite{AczelDaroczy:1975:OnMeasuresOfInformationAndTheirCharacterization}
273: \begin{equation}
274: \label{Equation:HartleyFunctionForRV}
275: H(x_{k}) = \ln \frac{1}{p_{k}} \enspace,\:\: \forall k = 1, \ldots n.
276: \end{equation}
277: $H$ is also known as entropy of a single event and plays an
278: important role in all classical measures of information. It can be
279: interpreted either as a measure of how unexpected the event was,
280: or as measure of the information yielded by the event.
281: Hartley function satisfies: (i) H is {\em
282: nonnegative}: $H(x_{k}) \geq 0$ (ii) H is {\em additive}:
283: $H(x_{i}x_{j}) = H(x_{i}) + H(x_{j})$ (iii) H is {\em
284: normalized}: $H(x_{k}) = 1$, whenever $p_{k} = \frac{1}{e}$
285: (in the case of logarithm with
286: base $2$, the same satisfied for $p_{k} = \frac{1}{2}$). These properties
287: are both necessary and
288: sufficient~\cite{AczelDaroczy:1975:OnMeasuresOfInformationAndTheirCharacterization}.
289:
290: Now, Shannon
291: entropy~(\ref{Equation:DefinitionOfShannonEntropy}) can be
292: written as expectation of Hartley
293: function as
294: \begin{equation}
295: \label{Equation:Definition_ShannonEntropy}
296: S (X) = {\langle H \rangle} = \sum_{k=1}^{n} p_{k} H_{k} \enspace,
297: \end{equation}
298: where $H_{k} = H(x_{k}),\: \forall k = 1, \ldots n$, with the
299: understanding that ${\langle H \rangle} = {\langle H(X)
300: \rangle}$.
301:
302: The characteristic additive property of Shannon entropy
303: \begin{equation}
304: \label{Equation:AdditivityOfShannonEntropy}
305: S(X \times Y) = S(X) + S(Y) \enspace,
306: \end{equation}
307: for two independent random variables $X$ and
308: $Y$ now follows as a consequence of the additivity property of
309: Hartley function.
310:
311: There are two postulates involved in defining Shannon entropy
312: as expectation of Hartley function. One is the additivity of
313: information which is the characteristic property of Hartley
314: function, and the other is
315: that if different amounts of information occur with different
316: probabilities, the total information will be the
317: average of the individual informations weighted by the
318: probabilities of their occurrences.
319:
320: The basic idea behind R\'{e}nyi's generalization is any
321: putative candidate for an entropy should be a mean and there
322: by use a well known
323: idea in mathematics
324: that the linear mean, though most widely used, is not the only
325: possible way of averaging, however, one can define the mean with
326: respect to an arbitrary
327: function. Here we briefly discuss
328: generalized averages and its properties which are essential for
329: the results we present in this paper.
330:
331: %-----------------------------------------------------------------
332: \subsection{Kolmogorov-Nagumo Averages or Quasilinear Mean}
333:
334: In the general theory of means, quasilinear mean of a random variable
335: $X$ is defined as{\footnote{Kolmogorov~\cite{Kolmogorov:1930:SurLaNotionDeLaMoyenne} and Nagumo~\cite{Nagumo:1930:UberEineKlasseVonMittlewerte}
336: first characterized the quasilinear mean ${\langle x
337: \rangle}_{\psi}$ for a vector $(x_{1}, \ldots,
338: x_{n})$ as ${\langle x \rangle}_{\psi} =
339: \psi^{-1}\left(\sum_{k=1}^{n} \frac{1}{n} \psi(x_{k})\right)$
340: where $\psi$ is a continuous and strictly monotone
341: function. De Finetti~\cite{DeFinetti:1931:SulConcettoDiMedia}
342: extended their result to the case of simple (finite)
343: probability distributions. The version of the quasilinear mean
344: representation theorem referred to in
345: \S~\ref{Section:AcharacterizationTheoremForTsallisEntropy} is
346: due to Hardy, Littlewood and
347: P{\'{o}}lya~\cite{HardyLittlewoodPolya:1934:Inequalities}, which
348: followed closely the approach of de
349: Finetti. Acz{\'{e}}l~\cite{Aczel:1948:OnMeanValues} proved a
350: characterization of the quasilinear mean using functional
351: equations.
352: Ben-Tal~\cite{Ben-Tal:1977:OnGeneralizedMeansAndGeneralizedConvexFucntions}
353: showed that quasilinear means are ordinary arithmetic means
354: under suitably defined addition and scalar multiplication
355: operations.
356: Norris~\cite{Norris:1976:GeneralMeansAndStatisticalTheory} did
357: a survey of quasilinear means and its more restrictive forms in
358: Statistics. More recent survey of generalized means can be
359: found
360: in~\cite{OstasiewiczOstasiewicz:2000:MeansAndTheirAppliacations}.
361: Applications of quasilinear means can be found in economics
362: (for example,
363: \cite{EpsteinZin:1989:SubstitutionRisk_SecondaryRef}) and
364: decision theory (for example,
365: \cite{KrepsPorteus:1978:TemporalResolution_SecondaryRef}).
366: Recently Czachor and
367: Naudts~\cite{CzachorNaudts:2002:ThermostatisticsBasedOnKolmogorov-NagumoAverages}
368: studied generalized thermostatistics based on quasilinear means.}%ENDfootnote
369: \begin{equation}
370: \label{Equation:Definition_KNaverages}
371: E_{\psi}X = {\langle X \rangle}_{\psi} = \psi^{-1} \left( \sum_{k=1}^{n}
372: p_{k} \psi\left(x_{k} \right) \right) \enspace,
373: \end{equation}
374: where $\psi$ is continuous and strictly monotonic (increasing
375: or decreasing) in which
376: case it has an inverse $\psi^{-1}$ which satisfies the same
377: conditions. In the context of generalized means, $\psi$ is
378: referred to as Kolmogorov-Nagumo
379: function or KN-function.
380: If, in particular, $\psi$ is linear, then
381: (\ref{Equation:Definition_KNaverages}) reduces to the
382: expression of linear averaging,
383: $EX = {\langle X \rangle} = \sum_{k=1}^{n} p_{k} x_{k}$.
384:
385: The following theorem qualifies quasilinear means.
386: %THEOREM:KN-average as a Mean----
387: \begin{theorem}
388: \label{Theorem:KN:KNaverageAsMean}
389: If $\psi$ is continuous and strictly monotone in
390: $a \leq x \leq b$, $a \leq x_{k} \leq b,\:\:\:
391: k = 1, \ldots n$, $p_{k} > 0 $ and $\sum_{k=1}^{n} p_{k} =1 $,
392: then
393: $\exists$ unique $x_{0} \in (a,b)$ such that
394: \begin{displaymath}
395: \psi(x_{0}) = \sum_{k=1}^{n} p_{k} \psi(x_{k})
396: \end{displaymath}
397: and $x_{0}$ is greater than some and less than
398: others of the $x_{k}$ unless all $x_{k}$ are zero.
399: \end{theorem}
400:
401: Thus, the mean ${\langle \, . \,\rangle}_{\psi}$ is determined when the
402: function $\psi$ is given. We may ask whether the converse is
403: true: if ${\langle X \rangle}_{\psi_{1}} ={\langle
404: X \rangle}_{\psi_{2}} $ for all $X \in \mathcal{X}$, is
405: $\psi_{1}$
406: necessarily the same function as $\psi_{2}$?
407: First we give the following definition.
408: %DEFINITION:Equivalent Mean-----
409: \begin{definition}
410: \label{Definition:KNequivalentFunctions}
411: Continuous and strictly monotone functions $\psi_{1}$ and $\psi_{2}$ are
412: said to be {\em KN-equivalent} if ${\langle X \rangle}_{\psi_{1}} =
413: {\langle X \rangle}_{\psi_{2}}$ for all $X \in \mathcal{X}$.
414: \end{definition}
415: Note that when we compare two means, it is to be understood
416: that the underlying probabilites are same. The following
417: theorem characterizes KN-equivalent functions.
418: %THEOREM:Condition for KN-equivalent Functions
419: \begin{theorem}
420: \label{Theorem:ConditionForKNequivalentFuntions}
421: In order that two continuous and strictly monotone functions
422: $\psi_{1}$ and $\psi_{2}$ are KN-equivalent, it is necessary and sufficient
423: that
424: \begin{displaymath}
425: \psi_{1} = \alpha \psi_{2} + \beta \enspace,
426: \end{displaymath}
427: where $\alpha$ and $\beta$ are constants and $\alpha \neq 0$.
428: \end{theorem}
429:
430: \begin{corollary}
431: Let $\psi$ be a KN-function then ${\langle X \rangle}_{\psi} =
432: {\langle X \rangle}_{-\psi}$ .
433: \end{corollary}
434: Hence, when ever required, without loss of generality, one
435: can assume that $\psi$ is an increasing function.
436: The following theorem characterizes additivity of quasilinear means.
437: \begin{theorem}
438: \label{Theorem:AdditivityOfKNaverages}
439: Let $\psi$ be a KN-function and $c$ be a real constant then
440: ${\langle X + c\rangle}_{\psi} = {\langle X \rangle}_{\psi} +
441: c$ i.e.,
442: \begin{displaymath}
443: \psi^{-1} \left( \sum_{k=1}^{n}
444: p_{k} \psi\left(x_{k} + c \right) \right) = \psi^{-1} \left( \sum_{k=1}^{n}
445: p_{k} \psi\left(x_{k} \right) \right) + c
446: \end{displaymath}
447: if and only if $\psi$ is either linear or exponential.
448: \end{theorem}
449: Proof of
450: Theorems~\ref{Theorem:KN:KNaverageAsMean},
451: \ref{Theorem:ConditionForKNequivalentFuntions} and
452: \ref{Theorem:AdditivityOfKNaverages}
453: can be found in the book on inequalities by Hardy, Littlewood,
454: P{\'{o}}lya~\cite{HardyLittlewoodPolya:1934:Inequalities}.
455:
456: %-----------------------------------------------------
457: \subsection{R\'{e}nyi Entropy}
458:
459: In the definition of Shannon entropy
460: (\ref{Equation:Definition_ShannonEntropy}), if the standard
461: mean
462: of Hartley function $H$
463: is replaced with the quasilinear
464: mean~(\ref{Equation:Definition_KNaverages}), one can obtain a
465: generalized measure of information of r.v $X$ with respect to
466: a KN-function $\psi$ as
467: \begin{equation}
468: \label{Equation:QuasilinearEntropy}
469: S_{\psi}(X) = \psi^{-1} \left(\sum_{k=1}^{n} p_{k} \psi \left(
470: \ln \frac{1}{p_{k}} \right) \right) = \psi^{-1}
471: \left(\sum_{k=1}^{n} p_{k} \psi \left(
472: H_{k} \right) \right) \enspace,
473: \end{equation}
474: where $\psi$ is a KN-function. We refer to
475: (\ref{Equation:QuasilinearEntropy}) as quasilinear entropy
476: with respect to the KN-function $\psi$.
477: If we impose the constraint of additivity on $S_{\psi}$, then
478: $\psi$ should
479: satisfy~\cite{Renyi:1960:SomeFundamentalQuestionsOfInformationTheory}
480: \begin{equation}
481: \label{Equation:AdditivityEquationForKNaverages}
482: {\langle X + c \rangle}_{\psi} = {\langle X \rangle}_{\psi} +
483: c \enspace,
484: \end{equation}
485: for any random variable $X \in \mathcal{X}$ and a constant
486: $c$.
487:
488: R\'{e}nyi employed this formalism to define a
489: one-parameter family
490: of measures of information ($\alpha$-entropies) as follows:
491: %Equation: Definition of Renyi entropy
492: \begin{equation}
493: \label{Equation:Definition_RenyiEntropy}
494: S_{\alpha}(X) = \frac{1}{1-\alpha} \ln \left(\sum_{k=1}^{n}
495: p_{k}^{\alpha} \right) \enspace,
496: \end{equation}
497: where the KN-function $\psi$ is chosen in
498: (\ref{Equation:QuasilinearEntropy}) as
499: $\psi(x) = e^{(1-\alpha)x}$ whose choice is motivated by
500: Theorem~\ref{Theorem:AdditivityOfKNaverages}. If we choose
501: $\psi$ as a
502: linear function in quasilinear
503: entropy~(\ref{Equation:QuasilinearEntropy}), what we get is
504: Shannon entropy.
505: R\'{e}nyi entropy is a
506: one-parameter generalization of Shannon entropy in the sense
507: that the limit $\alpha \rightarrow 1$ in
508: (\ref{Equation:Definition_RenyiEntropy}) retrieves Shannon
509: entropy.
510:
511: %applications
512: Despite its formal origin R\'{e}nyi entropy proved important
513: in a variety of practical applications in coding
514: theory~\cite{AczelDaroczy:1975:OnMeasuresOfInformationAndTheirCharacterization},
515: statistical
516: inference~\cite{ArimitsuArimitsu:2000:TsallisStatisticsAndTurbulence_SecondaryRef,ArimitsuArimitsu:2001:AnalysisOfTurbulence_SecondaryRef}, quantum
517: mechanics~\cite{MaassenUffink:1988:GeneralizedEntropicUncertaintyRelations},
518: chaotic dynamics
519: systems~\cite{HalseyJensenKadanoffProcacciaShraiman:1986:FractalMeasuresAndThierSingularities}.
520: Thermodynamic properties of systems with multi-fractal
521: structures have been studied by extending the notion of
522: Gibbs-Shannon entropy into a more general framework - R\'{e}nyi
523: entropy~\cite{JizbaArimitsu:2004:ObservabilityOfRenyiEntropy}.
524:
525: %=============================================================
526: \section{R\'{e}nyi's Recipe and Tsallis Entropy}
527: \label{Section:RenyisRecipieAndTsallisEntropy}
528:
529: %--------------------------------------------------
530: \subsection{Tsallis Entropy}
531:
532: Due to an increasing interest in long-range correlated systems
533: and non-equilibrium phenomena there has recently been much
534: focus on the Tsallis (or nonextensive)
535: entropy. Although, first introduced by Havrda and Charvat
536: \cite{HavrdaCharvat:1967:QuantificationMethodOfClassificationProcess}
537: in the context of cybernetics theory
538: and later studied by
539: Dar{\'{o}}czy~\cite{Daroczy:1970:GeneralizedInformationFunctions},
540: it was
541: Tsallis~\cite{Tsallis:1988:GeneralizationOfBoltzmannGibbsStatistics}
542: who exploited its nonextensive features and placed it in a
543: physical setting. Hence it is also known as
544: Harvda-Charvat-Dar\'{o}czy-Tsallis entropy. Throughout this
545: paper we refer to this as Tsallis or nonextensive
546: entropy. Tsallis entropy of a r.v $X \in \mathcal{X}$ with p.m.f
547: $p=(p_{1}, \ldots p_{n})$ is defined as
548: \begin{equation}
549: \label{Equation:Definition_TsallisEntropy}
550: S_{q}(X) = \frac{1 - \sum_{k=1}^{n} p_{k}^{q}}{q-1} \enspace,
551: \end{equation}
552: where $q >0$ is called the nonextensive index.
553: %($q$ is positive in
554: %order to ensure the concavity of $S_{q}$).
555: Tsallis entropy too, like R\'{e}nyi entropy, is a
556: one-parameter generalization of
557: Shannon entropy in the sense that $q \rightarrow 1$ in
558: (\ref{Equation:Definition_TsallisEntropy}) retrieves Shannon
559: entropy. Tsallis entropy is
560: concave for all $q > 0$, but R\'{e}nyi entropy is concave only
561: for $0 < \alpha < 1 $. The index $q$ characterizes the
562: degree of
563: nonextensivity reflected in the pseudo-additivity property
564: \begin{equation}
565: \label{Equation:PseudoAdditivityOfTsallisEntropy}
566: S_{q}(X \times Y) = S_{q}(X) \oplus_{q} S_{q}(Y) = S_{q}(X) + S_{q}(Y) +
567: (1-q) S_{q}(X) S_{q}(Y) \enspace,
568: \end{equation}
569: where $X,Y \in \mathcal{X}$ are two independent random variables.
570:
571:
572: %----------------------------------------------------------
573: \subsection{Nongeneralizability of Tsallis Entropy}
574:
575: Though the derivation of Tsallis entropy, when it was proposed
576: in 1988~\cite{Tsallis:1988:GeneralizationOfBoltzmannGibbsStatistics} is slightly different, one can understand this
577: generalization using $q$-logarithm
578: function (see~(\ref{Equation:Definition_q-Logorithm})), where
579: one would first generalize logarithm in the
580: Hartley information with $q$-logarithm and define $q$-Hartley
581: function $\widetilde{H}: \{x_{1}, \ldots, x_{n}\} \rightarrow
582: \mathbb{R}$ of r.v $X$ as
583: ~\cite{Tsallis:1999:NonextensiveStatisticalMechanics}
584: \begin{equation}
585: \label{Equation:Definition_q-HartleyInformationMeasure}
586: \widetilde{H}_{k}=\widetilde{H}(x_{k}) = \ln_{q}
587: \frac{1}{p_{k}}\enspace, \quad k=1,\ldots n \enspace.
588: \end{equation}
589: The $q$-logarithm
590: in~(\ref{Equation:Definition_q-HartleyInformationMeasure}) is
591: defined as
592: \begin{equation}
593: \label{Equation:Definition_q-Logorithm}
594: \ln_{q}(x) = \frac{x^{1-q}-1}{1-q} \enspace,
595: \end{equation}
596: which satisfies pseudo-additivity of the form
597: $\ln_{q}(xy)=\ln_{q}x \oplus_{q}
598: \ln_{q}y$ and in the limit $q \to 1$, we have $\ln_{q} x \to \ln x$.
599: Now Tsallis entropy
600: (\ref{Equation:Definition_TsallisEntropy})
601: can be defined as the expectation of $q$-Hartley function $\widetilde{H}$
602: as
603: \begin{equation}
604: \label{Equation:Definition_TsallisEntropy_2}
605: S_{q}(X) = {\left\langle \widetilde{H} \right\rangle} \enspace.
606: \end{equation}
607: Note that the characteristic pseudo-additivity property of Tsallis
608: entropy~(\ref{Equation:PseudoAdditivityOfTsallisEntropy})
609: is a consequence of additivity property of Hartley
610: function.
611:
612: Before we present the main results of this paper, we briefly
613: discuss the context of quasilinear means where there is a
614: relation between Tsallis and R\'{e}nyi entropy.
615: The $q$-Hartley function can be written as
616: \begin{displaymath}
617: \widetilde{H}_{k} = \ln_{q} \frac{1}{p_{k}} = \phi_{q}(H_{k})\enspace,
618: \end{displaymath}
619: where
620: \begin{equation}
621: \label{Equation:KN:ModfiedKNfunction}
622: \phi_{q}(x) = \frac{e^{(1-q)x} -1}{1 - q} =
623: \ln_{q}(e^{x}) \enspace.
624: \end{equation}
625: Note that $\phi_{q}$ is KN-equivalent to $e^{(1-q)x}$
626: (by Theorem~\ref{Theorem:ConditionForKNequivalentFuntions}), the
627: KN-function used in R\'{e}nyi entropy. Hence
628: Tsallis entropy is related to R\'{e}nyi entropies as
629: \begin{equation}
630: \label{Equation:RelationBetweenTsallisAndRenyi_ViaKN}
631: S_{q}^{\mbox{T}} = \phi_{q}(S_{q}^{\mbox{R}}) \enspace,
632: \end{equation}
633: where $S_{q}^{\mbox{T}}$ and $S_{q}^{\mbox{R}}$ denote the
634: Tsallis and R\'{e}nyi entropy respectively with a real number
635: $q$ as a parameter.
636: Hence, Tsallis entropy and R\'{e}nyi entropy are monotonic
637: functions of each other and, as a result, both must be
638: maximized by the same probability distribution.
639:
640: Now a natural question that arises is
641: whether one could generalize Tsallis
642: entropy using R\'{e}nyi's recipe i.e., by replacing linear average in
643: (\ref{Equation:Definition_TsallisEntropy_2}) by KN-averages
644: and impose the
645: condition of pseudo-additivity. It is equivalent to determining
646: the KN-function $\psi$ for which so called $q$-quasilinear
647: entropy defined as
648: \begin{equation}
649: \label{Equation:Definition_q-QuasilinearEntropy}
650: \widetilde{S}_{\psi} (X) = {\left\langle \widetilde{H}
651: \right\rangle}_{\psi} = \psi^{-1}
652: \left[ \sum_{k=1}^{n} p_{k} \psi \left( \widetilde{H}_{k}
653: \right) \right] \enspace,
654: \end{equation}
655: where $\widetilde{H}_{k} = \widetilde{H}(x_{k})\: \forall k =
656: 1, \ldots n$, satisfies the pseudo-additive property.
657:
658: First, we present the following result which characterizes the
659: pseudo-additivity of quasilinear means.
660: %THEOREM:Nonextensive Additivity of Two Random Variables
661: \begin{theorem}
662: \label{Theorem:NonextensiveAditivityOfTwoRandomVariables}
663: Let $X,Y \in \mathcal{X}$ be two independent random
664: variables. Let $\psi$ be any KN-function. Then
665: \begin{equation}
666: \label{Equation:NonextensiveAdditivityOfKN-averages_Condition_Form1}
667: {\langle X \oplus_{q} Y \rangle}_{\psi} = {\langle X \rangle}_{\psi} \oplus_{q}{\langle Y \rangle}_{\psi}
668: \end{equation}
669: if and only if $\psi$ is linear.
670: \end{theorem}
671: %PROOF....
672: \proof
673: Let $p$ and $r$ be the p.m.fs of random variables $X, Y \in
674: \mathcal{X}$ respectively.
675: The proof of
676: sufficiency is simple which follows from
677: \begin{displaymath}
678: {\langle X \oplus_{q} Y \rangle}_{\psi} = {\langle X
679: \oplus_{q} Y \rangle} = \sum_{i=1}^{n} \sum_{j=1}^{n}
680: p_{i}r_{j} (x_{i} \oplus_{q} y_{j}) \enspace,
681: \end{displaymath}
682: and by the definition of $\oplus_{q}$, we have
683: {\setlength\arraycolsep{0pt}
684: \begin{eqnarray}
685: {\langle X \oplus_{q} Y \rangle} &=& \sum_{i=1}^{n} \sum_{j=1}^{n}
686: p_{i}r_{j} (x_{i} + y_{j} + (1-q) x_{i} y_{j}) \nonumber\\
687: & = & \sum_{i=1}^{n} p_{i} x_{i} + \sum_{j=1}^{n} r_{j} y_{j}
688: + (1-q) \sum_{i=1}^{n} p_{i} x_{i} \sum_{j=1}^{n} r_{j} y_{j}\enspace.
689: \nonumber
690: \end{eqnarray}}
691:
692: To prove the converse, we need to determine all forms of $\psi$ which
693: satisfy
694: \begin{equation}
695: \label{Equation:NonextensiveAdditivityOfKN-averages_Condition_Form2}
696: \psi^{-1} \left(\sum_{i=1}^{n} \sum_{j=1}^{n} p_{i}r_{j}
697: \psi \left( x_{i} \oplus_{q} y_{j}
698: \right) \right)
699: = \psi^{-1} \left(\sum_{i=1}^{n} p_{i} \psi \left( x_{i}
700: \right) \right) \oplus_{q} \psi^{-1} \left(\sum_{j=1}^{n}
701: r_{j} \psi \left( y_{j} \right) \right) \enspace.
702: \end{equation}
703:
704: Since~(\ref{Equation:NonextensiveAdditivityOfKN-averages_Condition_Form2})
705: must hold for arbitrary p.m.fs $p$,$r$ and for arbitrary
706: numbers
707: $\{x_{1}, \ldots, x_{n}\}$ and $\{y_{1}, \ldots, y_{n}\}$, one
708: can choose $y_{j} = c$ independently of $j$. Then
709: (\ref{Equation:NonextensiveAdditivityOfKN-averages_Condition_Form2})
710: yields
711: \begin{equation}
712: \label{Equation:NonextensiveAdditivityOfKN-averages_Condition_Form3}
713: \psi^{-1} \left(\sum_{i=1}^{n} p_{k}
714: \psi \left( x_{i} \oplus_{q} c \right) \right) =
715: \psi^{-1} \left(\sum_{i=1}^{n} p_{k} \psi \left(
716: x_{i} \right) \right) \oplus_{q} c \enspace.
717: \end{equation}
718: That is, $\psi$ should satisfy
719: \begin{equation}
720: \label{Equation:NonextensiveAdditivityOfKN-averages_Condition_Form4}
721: {\langle X \oplus_{q} c \rangle}_{\psi} = {\langle X
722: \rangle}_{\psi} \oplus_{q} c \enspace,
723: \end{equation}
724: for any $X \in \mathcal{X}$ and any constant $c$. This can be
725: rearranged as
726: \begin{displaymath}
727: {\langle (1 + (1-q) c) X + c \rangle}_{\psi} =
728: (1 + (1-q) c) {\langle X \rangle}_{\psi} + c
729: \end{displaymath}
730: by using the definition of $\oplus_{q}$.
731: Since $q$ is independent of other quantities, $\psi$ should
732: satisfy an equation of the form
733: \begin{equation}
734: \label{Equation:NonextensiveAdditivityOfKN-averages_Condition_Form5}
735: {\langle dX + c \rangle}_{\psi} = d {\langle X \rangle}_{\psi}
736: + c \enspace,
737: \end{equation}
738: where $d \neq 0$ (by writing $d =(1+(1-q)c)$).
739: Finally $\psi$ must satisfy
740: \begin{equation}
741: \label{Equation:NonextensiveAdditivityOfKN-averages_Condition_Sub1}
742: {\langle X + c \rangle}_{\psi} = {\langle X \rangle}_{\psi} + c
743: \end{equation}
744: and
745: \begin{equation}
746: \label{Equation:NonextensiveAdditivityOfKN-averages_Condition_Sub2}
747: {\langle dX \rangle}_{\psi} = d {\langle X \rangle}_{\psi} \enspace,
748: \end{equation}
749: for any $X \in \mathcal{X}$ and any constants $d$, $c$.
750: From Theorem~\ref{Theorem:AdditivityOfKNaverages}, the condition
751: (\ref{Equation:NonextensiveAdditivityOfKN-averages_Condition_Sub1})
752: is satisfied only when $\psi$ is linear or exponential.
753:
754: To complete the theorem we have to show that
755: KN-averages do not satisfy condition
756: (\ref{Equation:NonextensiveAdditivityOfKN-averages_Condition_Sub2})
757: when $\psi$ is exponential.
758: For a particular choice of
759: $\psi(x) = e^{(1- \alpha)x}$, assume that
760: \begin{equation}
761: \label{Equation:ToGetTheContradiction_ForTheTheorem}
762: {\langle d X \rangle}_{\psi} = d {\langle X
763: \rangle}_{\psi} \enspace,
764: \end{equation}
765: where
766: \begin{displaymath}
767: {\langle d X \rangle}_{\psi_{1}} = \frac{1}{1-\alpha} \ln
768: \left( \sum_{k=1}^{n} p_{k} e^{(1-\alpha) d x_{k}} \right) \enspace,
769: \end{displaymath}
770: and
771: \begin{displaymath}
772: d {\langle X \rangle}_{\psi_{1}} = \frac{d}{1-\alpha} \ln
773: \left( \sum_{k=1}^{n} p_{k} e^{(1-\alpha) x_{k}} \right) \enspace.
774: \end{displaymath}
775: Now define a KN-function $\psi'$ as $\psi'(x) = e^{(1-
776: \alpha)dx}$, for which
777: \begin{displaymath}
778: {\langle X \rangle}_{\psi'} = \frac{1}{d(1-\alpha)} \ln
779: \left( \sum_{k=1}^{n} p_{k} e^{(1-\alpha) d x_{k}} \right) \enspace.
780: \end{displaymath}
781: Condition
782: (\ref{Equation:ToGetTheContradiction_ForTheTheorem}) implies
783: \begin{displaymath}
784: {\langle X \rangle}_{\psi} = {\langle X \rangle}_{\psi'} \enspace,
785: \end{displaymath}
786: and by
787: Theorem~\ref{Theorem:ConditionForKNequivalentFuntions},
788: $\psi$ and $\psi'$ are
789: KN-equivalent which gives a contradiction.
790:
791: \endproof
792: %ENDPROOF.....
793:
794: One can observe that the above proof avoids solving
795: functional equations as in the case of
796: Theorem~\ref{Theorem:AdditivityOfKNaverages} (see
797: \cite{AczelDaroczy:1975:OnMeasuresOfInformationAndTheirCharacterization}).
798: Instead it makes
799: use of basic results of KN-averages.
800: The following corollary is the immediate consequence of
801: Theorem~\ref{Theorem:NonextensiveAditivityOfTwoRandomVariables}.
802: %Theorem: Nongeneralizability of Tsallis Entropy------
803: \begin{corollary}
804: \label{Corollary:NongenralizabilityOfTsallisEntropy}
805: $q$-quasilinear entropy $\widetilde{S}_{\psi}$ (defined as
806: in~(\ref{Equation:Definition_q-QuasilinearEntropy})) with respect to
807: a KN-function $\psi$ satisfies pseudo-additivity if
808: and only if $\widetilde{S}_{\psi}$ is Tsallis entropy.
809: \end{corollary}
810: \proof
811: Let $X,Y \in \mathcal{X}$ be two independent random variables
812: and let
813: $p,r$ be their corresponding pmfs.
814: By the pseudo-additivity constraint, $\psi$ should satisfy
815: \begin{equation}
816: \label{Equation:KNtsallis_PseudoAdditivity_Condition_Form1}
817: \widetilde{S}_{\psi}(X \times Y) = \widetilde{S}_{\psi}(X) \oplus_{q}
818: \widetilde{S}_{\psi}(Y)
819: \end{equation}
820: From the property of $q$-logarithm that $\ln_{q} x y = \ln_{q}x
821: \oplus_{q} \ln_{q}y$, we need
822: {\setlength\arraycolsep{0pt}
823: \begin{eqnarray}
824: \label{Equation:KNtsallis_PseudoAdditivity_Condition_Form2}
825: \psi^{-1} && \left(\sum_{i=1}^{n} \sum_{j=1}^{n} p_{i}r_{j} \psi
826: \left( \ln_{q} \frac{1}{p_{i}r_{j}} \right) \right) \nonumber\\
827: && = \psi^{-1} \left(\sum_{i=1}^{n} p_{i} \psi \left( \ln_{q}
828: \frac{1}{p_{i}} \right) \right) \oplus_{q}
829: \psi^{-1} \left(\sum_{j=1}^{n} r_{j} \psi \left( \ln_{q}
830: \frac{1}{r_{j}} \right) \right) \enspace.
831: \end{eqnarray}
832: Equivalently, we need
833: {\setlength\arraycolsep{0pt}
834: \begin{eqnarray}
835: \psi^{-1} && \left(\sum_{i=1}^{n} \sum_{j=1}^{n} p_{i}r_{j}
836: \psi \left( \widetilde{H}_{i}^{p} \oplus_{q} \widetilde{H}_{j}^{r}
837: \right) \right) \nonumber \\
838: && = \psi^{-1} \left(\sum_{i=1}^{n} p_{i} \psi \left(
839: \widetilde{H}_{i}^{p} \right) \right) \oplus_{q}
840: \psi^{-1} \left(\sum_{j=1}^{n} r_{j} \psi
841: \left(\widetilde{H}_{j}^{r} \right) \right) \enspace, \nonumber
842: \end{eqnarray}
843: where $\widetilde{H}^{p}$ and $\widetilde{H}^{r}$ represent
844: the $q$-Hartley functions corresponding to probability distributions $p$
845: and $r$ respectively.
846: That is, $\psi$ should satisfy
847: \begin{displaymath}
848: {\langle \widetilde{H}^{p} \oplus_{q} \widetilde{H}^{r}
849: \rangle}_{\psi} = {\langle \widetilde{H}^{p} \rangle}_{\psi}
850: \oplus_{q} {\langle \widetilde{H}^{r} \rangle}_{\psi} \enspace.
851: \end{displaymath}
852: Also from
853: Theorem~\ref{Theorem:NonextensiveAditivityOfTwoRandomVariables},
854: $\psi$ is linear and hence $\widetilde{S}_{\psi}$ is Tsallis.
855: \endproof
856: Corollary~\ref{Corollary:NongenralizabilityOfTsallisEntropy}
857: shows that using the R\'{e}nyi's recipe in the nonextensive
858: case one can prepare only Tsallis entropy, while in the
859: classical there are two possibilities.
860:
861: %=============================================================
862: \section{A Characterization Theorem for Tsallis Entropy}
863: \label{Section:AcharacterizationTheoremForTsallisEntropy}
864:
865: The importance of R\'{e}nyi's formalism to generalize Shannon
866: entropy is a characterization of Shannon entropy in terms of
867: axiom of quasilinear
868: means~\cite{Renyi:1960:SomeFundamentalQuestionsOfInformationTheory}.
869: By the result,
870: Theorem~\ref{Theorem:NonextensiveAditivityOfTwoRandomVariables},
871: that we presented in this paper, one can give a
872: characterization of
873: Tsallis entropy in terms of axioms of quasilinear means. For such a
874: characterization one would assume that entropy is the expectation
875: of a function of underlying r.v. In the classical case, the
876: function is Hartley function, while in the nonextensive case
877: it is $q$-Hartlay function.
878:
879: Since characterization of quasilinear means is given in terms of
880: cumulative distribution of a random variable, we use the
881: following definitions and notation.
882:
883: Let $F:{\mathbb{R}} \rightarrow
884: {\mathbb{R}}$ denote the cumulative distribution function of
885: random variable $X \in \mathcal{X}$. Corresponding to a
886: KN-function $\psi: {\mathbb{R}} \rightarrow {\mathbb{R}}$,
887: generalized mean of $F$ (or $X$) can be written as
888: \begin{equation}
889: \label{Equation:KN-averagesInTermsOfCumulativeDistribution}
890: E_{\psi}(F)= E_{\psi}(X) = {\langle X \rangle}_{\psi} =
891: \psi^{-1}\left(\int \psi \, \ud
892: F \right) \enspace,
893: \end{equation}
894: which is continuous analogue to
895: (\ref{Equation:Definition_KNaverages}) and it is axiomized by
896: Kolmogorov, Nagumo and De Finetti (see
897: \cite[Theorem 215]{HardyLittlewoodPolya:1934:Inequalities}) as
898: follows.
899:
900:
901: %Theorem: Axioms of Kolmogorov Nagumo Averages
902: \begin{theorem}
903: \label{Theorem:AxiomsForKN-averages}
904: Let $\mathcal{F}_{I}$ be the set of all cumulative
905: distribution functions defined on some interval $I$ of the
906: real line ${\mathbb{R}}$. A functional $\kappa:
907: {\mathcal{F}}_{I} \rightarrow {\mathbb{R}}$ satisfies the
908: following axioms:
909: \begin{description}
910: \item[axiom 1:] $\kappa(\delta_{x}) = x$, where $\delta_{x} \in
911: {\mathcal{F}}_{I}$ denotes the step function at
912: $x$ (\textit{Consistency with certainty}) ,
913:
914: \item[axiom 2:] $F,G \in
915: {\mathcal{F}}_{I}$, if $F \leq G $ then $\kappa(F) \leq
916: \kappa(G)$; the equality holds if and only if $F = G$
917: (\textit{Monotonicity}) and,
918:
919: % \item[axiom 2:] (\textit{Substitution}) $F,G \in
920: % {\mathcal{F}}_{I}$, if $E(F) = E(G)$ then
921: % $\forall \beta \in (0,1) \:\: \exist \gamma \in (0,1)$ such
922: % that $ E(\beta F + (1-\beta)H) = E( \gamma
923: % G + (1-\gamma)H)$, for any $H \in {\mathcal{F}}_{I}$
924:
925: \item[axiom 3:] $F,G \in
926: {\mathcal{F}}_{I}$, if $\kappa(F) = \kappa(G)$ then
927: $ \kappa(\beta F + (1-\beta)H) = \kappa( \beta
928: G + (1-\beta)H)$, for any $H \in {\mathcal{F}}_{I}$
929: (\textit{Quasilinearity})
930:
931: \end{description}
932: if and only if
933: there is a continuous strictly monotone function $\psi$ such
934: that
935: \begin{displaymath}
936: \kappa(F) =
937: \psi^{-1}\left(\int \psi \, \ud F \right) \enspace.
938: \end{displaymath}
939: \end{theorem}
940:
941: The modified axioms for quasilinear mean can be found in
942: \cite{Chew:1983:AgeneralizationOfTheQuasilinearMean,Fishburn:1986:ImplicitMeanValues,OstasiewiczOstasiewicz:2000:MeansAndTheirAppliacations}).
943: Now we give our characterization theorem for Tsallis entropy
944: that is similar to the
945: characterization of Shannon entropy given by
946: R\'{e}nyi~\cite{Renyi:1960:SomeFundamentalQuestionsOfInformationTheory}.
947: \begin{theorem}
948: \label{Theorem:CharacterizationOfTsallisEntropy}
949: Let $X \in \mathcal{X}$ be a random variable. An information measure
950: defined as a (generalized) mean $\kappa$ of $q$-Hartley function of
951: $X$ is Tsallis entropy if and only if
952: \begin{enumerate}
953: \item $\kappa$ satisfies axioms of quasilinear means given in
954: Theorem~\ref{Theorem:AxiomsForKN-averages} and,
955:
956: \item If $X,Y \in \mathcal{X}$ are two random variables which
957: are independent, then
958: \begin{displaymath}
959: \kappa(X \oplus_{q} Y) =
960: \kappa(X) \oplus_{q} \kappa(Y) \enspace.
961: \end{displaymath}
962: \end{enumerate}
963: \end{theorem}
964: Theorem~\ref{Theorem:CharacterizationOfTsallisEntropy} is a
965: direct consequence of
966: Theorems~\ref{Theorem:NonextensiveAditivityOfTwoRandomVariables}
967: and \ref{Theorem:AxiomsForKN-averages}.
968: This characterization of Tsallis entropy only replaces the
969: additivity constraint in the characterization of Shannon
970: entropy given by R\'{e}nyi in
971: ~\cite{Renyi:1960:SomeFundamentalQuestionsOfInformationTheory},
972: with pseudo-additivity, which further does not make use
973: of the postulate $\kappa(H) + \kappa(-H)=0$. (This postulate is needed to
974: distinguish Shannon entropy from R\'{e}nyi entropy). This
975: is possible because Tsallis entropy is unique by means of
976: KN-averages and under pseudo-additivity.
977:
978:
979: % \proof
980: % From the Theorem~\ref{Theorem:AxiomsForKN-averages} we have
981: % \begin{displaymath}
982: % E(H) = {\langle H \rangle}_{\psi} =
983: % \psi^{-1}\left(\int \psi \, \ud F \right) \enspace,
984: % \end{displaymath}
985: % where $\psi$ is strictly monotone and continuous. From the
986: % postulate (2) and
987: % Theorem~\ref{Theorem:NonextensiveAditivityOfTwoRandomVariables} we
988: % have the remaining proof.
989: % \endproof
990:
991: %====================================================================
992: \section{Conclusions}
993: \label{Section:Conclusions}
994:
995: Passing an information measure through R\'{e}nyi formalism --
996: procedure followed by R\'{e}nyi to generalize Shannon entropy
997: -- allows one to study the possible generalizations and
998: characterize information measure in the context in terms of
999: axioms of quasilinear means. In this paper we studied this
1000: technique for nonextensive entropy and showed that Tsallis
1001: entropy is unique under R\'{e}nyi's recipe.
1002: Considering the attempts to study generalized thermostatistics
1003: based on
1004: KN-averages (for example
1005: \cite{CzachorNaudts:2002:ThermostatisticsBasedOnKolmogorov-NagumoAverages}),
1006: the results presented in this paper further the
1007: relation between entropic measures and generalized averages.
1008:
1009: \section*{References}
1010:
1011: \bibliographystyle{unsrt}
1012: \bibliography{papi}
1013:
1014:
1015: \end{document}
1016:
1017:
1018:
1019:
1020:
1021: