1: \begin{abstract}

2: The weak law of large numbers implies that, under mild assumptions on the source, the Renyi entropy per produced symbol converges (in probability) towards the Shannon entropy rate. 

3: 

4: This paper quantifies the speed of this convergence for sources with independent (but not iid) outputs, generalizing and improving the result of  Holenstein and Renner (IEEE Trans. Inform. Theory, 2011). 

5: \begin{enumerate}[(a)]

6: \item we characterize sources with \emph{slowest convergence} (for given entropy): their outputs are mixtures of a uniform distribution and a unit mass.

7: \item based on the above characterization, we establish faster convergences

8: in \emph{high-entropy} regimes.

9: \end{enumerate}

10: 

11: We discuss how these improved bounds may be used to better quantify security of outputs of random number generators.

12: 

13: In turn, the characterization of "worst" distributions can be used 

14: to derive sharp "extremal" inequalities between Renyi and Shannon entropy.

15: 

16: The main technique is \emph{non-convex programming}, used to characterize distributions of possibly large exponential moments under certain entropy.

17: 

18: 

19: % In other words, 

20: %5high-entropy sources

21: %This corresponds to the well established practice of prefering sources with %higher-entropy rate. 

22:  

23: % that provide

24: % quantitative bounds for the entropy rate of sources producing independent symbols (a variant of the asymptotic equipartition property). 

25: 

26: %, by which

27: %we characterize the "slowest" source (of a certain entropy): it combines a unit mass and a uniform distribution. As a byproduct we get the same characterization in related problems of maximizing Renyi entropy.

28: \end{abstract}

29: