1: \begin{abstract}

2:  A Differentiable Neural Computer (DNC) \cite{DNC} is a neural network with an external memory which allows for iterative content modification via read, write and delete operations.

3:  We show that information theoretic properties of the memory contents play an important role in the performance of such architectures. We introduce a novel concept of \textit{memory demon} \footnote {The name is inspired by the concept of Maxwell's Demon who decreases the \textit{entropy} of gas in a box by letting all the high-velocity molecules accumulate on one side and all the low-velocity ones on the other.} to DNC architectures which modifies the memory contents implicitly via additive input encoding. The goal of the memory demon is to maximize the expected sum of mutual information of the consecutive external memory contents. 

4:  

5: Github codes \href{here}{https://github.com/azarafrooz/dnc-with-demon}

6:  

7:  %Moreover, for a practical and large scale implementation, we integrate the proposed architecture with Sparse Access Memory.

8:  

9: %The experiments show that DNC architectures with memory demon show significant improvement over the state of the art of such architectures, in terms of both training convergence for various algorithmic tasks as well as mean error rate on the bAbI question answering dataset. Moreover, the introduction of a memory demon is a step forward towards understanding the role of internal noise in DNC. 

10:   \end{abstract}

11: