1: \begin{abstract}

2: 

3:     There has been much recent interest in designing symmetry-aware neural networks (NNs) exhibiting relaxed equivariance. 

4:     Such NNs aim to interpolate between being exactly equivariant and being fully flexible, affording consistent performance benefits. 

5:     In a separate line of work, certain structured parameter matrices---those with displacement structure, characterized by \emph{low displacement rank} (LDR)---have been used to design small-footprint NNs. 

6:     Displacement structure enables fast function and gradient evaluation, but permits accurate approximations via compression primarily to classical convolutional neural networks (CNNs). 

7:     In this work, we propose a general framework---based on a novel construction of symmetry-based structured matrices---to build approximately equivariant NNs with significantly reduced paramter counts.  Our framework integrates the two aforementioned lines of work via the use of so-called Group Matrices (GMs), a forgotten precursor to the modern notion of regular representations of finite groups. 

8:     GMs allow the design of structured matrices---resembling LDR matrices---which generalize the linear operations of a classical CNN from cyclic groups to general finite groups and their homogeneous spaces. 

9:     We show that GMs can be employed to extend all the elementary operations of CNNs to general discrete groups. 

10:     Further, the theory of structured matrices based on GMs provides a generalization of LDR theory focussed on matrices with cyclic structure, providing a tool for implementing approximate equivariance for discrete groups. 

11:     We test GM-based architectures on a variety of tasks in the presence of relaxed symmetry. 

12:     We report that our framework consistently performs competitively compared to approximately equivariant NNs, and other structured matrix-based compression frameworks, sometimes with a one or two orders of magnitude lower parameter count.

13:  

14: \end{abstract}

15: