1: \begin{abstract}

2: %    We study the capabilities of recurrent neural networks to capture patterns

3: %    over sequences that, in terms of formal languages, correspond to the 

4: %    \emph{group-free regular languages}. Such patterns commonly occur in

5: %    applications such as stock prediction and reinforcement learning, and they

6: %    are the expressivity of well-known temporal logics with many applications in

7: %    formal methods and AI. They are the languages that can be recognised by

8: %    information processing systems with finite memory whose components implement

9: %    the fundamental functionality of storing one bit of information while at the

10: %    same time do not implement groups.

11: % 

12: %    We identify a class of recurrent neural networks whose expressivity matches

13: %    exactly the group-free regular languages. These are the recurrent networks where

14: %    neurons are arranged in a cascade architecture, and where activation functions

15: %    satisfy some simple conditions that we identify. The cascade architecture plays

16: %    a central role in avoiding spurious groups, and so we emphasise its

17: %    importance by calling them \emph{Recurrent Neural Cascades} (RNCs).

18: % 

19: %    The methodology we develop is based on algebraic automata theory, and it is of

20: %    an independent interest. It amounts to analysing which semigroup and group

21: %    a recurrent network is able to implement. The analysis is purely

22: %    qualitative, focusing on properties such as continuity and monotonicity, and

23: %    without resorting to quantitative considerations such as the precision of the

24: %    weights. We believe our methodology has a potential that extends beyond our

25: %    current results. It may be used, e.g., to identify new classes of

26: %    recurrent networks capturing specific groups.

27: % \end{abstract}

28: