1: \begin{abstract}

2:   Standard probabilistic sparse coding assumes a Laplace prior, %

3:   a linear mapping from latents to observables, and Gaussian observable distributions. 

4:   We here derive a solely entropy-based learning objective for the parameters of standard sparse coding.

5:   The novel variational objective has the following features: 

6:   (A)~unlike MAP approximations, it uses non-trivial posterior approximations for probabilistic inference;

7:   (B)~unlike for previous non-trivial approximations, the novel objective is fully analytical; and 

8:   (C)~the objective allows for a novel principled form of annealing. %

9:   The objective is derived by first showing that the standard ELBO objective converges to

10:   a sum of entropies, which matches similar recent results for generative models with Gaussian priors.

11:   The conditions under which the ELBO becomes equal to entropies are then shown to have analytical solutions,

12:   which leads to the fully analytical objective. %

13:   Numerical experiments are used to demonstrate the feasibility of learning with such entropy-based ELBOs. We investigate

14:   different posterior approximations including Gaussians with correlated latents and deep amortized approximations. 

15:   Furthermore, we numerically investigate entropy-based annealing which results in improved learning. %

16:   Our main contributions are theoretical, however, and they are twofold: 

17:   (1)~for non-trivial posterior approximations, we provide the (to the knowledge of the authors) first analytical ELBO objective for standard probabilistic sparse coding; and (2)~we provide the first demonstration on how a recently shown convergence of the ELBO to entropy sums can be used for learning.  

18: \end{abstract}

19: