1: \begin{abstract}

2:   We propose a novel regularizer to improve the training of Generative Adversarial Networks (GANs). 

3: The motivation is that when the discriminator \D spreads out its model capacity in the right way, the learning signals given to the generator \G are more informative and diverse.  

4: These in turn help \G to explore better and discover the real data manifold while avoiding large unstable jumps due to the erroneous extrapolation made by \D. 

5: Our regularizer guides the rectifier discriminator \D to better allocate its model capacity, by encouraging the binary activation patterns on selected internal layers of \D to have a high joint entropy. 

6: %The proposed regularizer encourages the binary activation patterns on selected internal layers of a rectifier neural net to have a high joint entropy. 

7: %Applying this input-dependent model capacity regularizer to a rectifier \D leads to the desired informative and diverse learning signals for \G. 

8: Experimental results on both synthetic data and real datasets demonstrate improvements in stability and  convergence speed of the GAN training, as well as higher sample quality.  The approach also leads to higher classification accuracies in semi-supervised learning.

9: \end{abstract}