1: \begin{abstract}

2: We explore the robustness of recurrent neural networks when the computations within the network are noisy.

3: One of the motivations for looking into this problem is to reduce the high power cost of conventional computing of

4: neural network operations through the use of analog neuromorphic circuits.

5: Traditional GPU/CPU-centered deep learning architectures exhibit bottlenecks in power-restricted applications,

6: such as speech recognition in embedded systems.

7: The use of specialized neuromorphic circuits, where analog signals passed through memory-cell arrays are sensed to

8: accomplish matrix-vector multiplications, promises large power savings and speed gains but brings with it the

9: problems of limited precision of computations and unavoidable analog noise.

10: 

11: In this paper we propose a method, called {\em Deep Noise Injection training}, to train RNNs to obtain a set of

12: weights/biases that is much more robust against noisy computation during inference.

13: We explore several RNN architectures, such as vanilla RNN and long-short-term memories (LSTM),

14: and show that after convergence of Deep Noise Injection training the set of trained weights/biases has

15: more consistent performance over a wide range of noise powers entering the network during inference.

16: Surprisingly, we find that Deep Noise Injection training improves overall performance of some networks even

17: for numerically accurate inference.

18: 

19: \end{abstract}

20: