1: \documentclass[a4paper,12pt]{article}

2: \pdfoutput=1

3: \usepackage{amsmath}

4: \usepackage{amssymb}

5: \usepackage{amsthm}

6: \usepackage{graphicx}

7: \usepackage{caption}

8: \usepackage{subfig}

9: %\usepackage{fancyref}

10: %\usepackage{apacite}

11: \usepackage{color}

12: 

13: \usepackage[utf8]{inputenc}

14: 

15: \usepackage[top=3cm, bottom=5cm, left=2cm, right=2cm]{geometry}

16: %\usepackage[normalsections,normalfloats,normalleading,normalindent]{savetrees}

17: 

18: 

19: \newtheorem{thm}{Theorem}

20: \newtheorem{lem}{Lemma}[thm]

21: \newtheorem{prop}{Property}[thm]

22: \theoremstyle{remark}

23: \newtheorem{rem}{Remark}[section]

24: \newtheorem{thmrem}{Theorem remark}[thm]

25: \newtheorem{lemrem}{Lemma remark}[lem]

26: \theoremstyle{definition}

27: \newtheorem{defi}{Definition}[section]

28: 

29: \newcounter{todo}[section]

30: \newcommand{\TODO}[1]{\addtocounter{todo}{1}\textcolor{red}{TODO

31: \arabic{section}.\arabic{todo}: #1}}

32: 

33: \newenvironment{example}{{\bf Example : }}{$\hfill\square$}

34: 

35: \newcommand{\dfp}[2]{\partial_{#1} f^\prime_{#2}}

36: \def\demi{\frac{1}{2}}

37: \def\irdeu{\frac{1}{\sqrt{2}}}

38: 

39: \def\D{\mathcal{D}}

40: \def\B{\mathcal{B}}

41: \def\M{\mathcal{M}}

42: 

43: \def\G{\Gamma}

44: \def\MG{\mathcal{M}_\Gamma}

45: \def\Mp{\mathcal{M}_2}

46: 

47: \def\Mall{\mathcal{M}_{all}}

48: \def\Vall{V_{all}}

49: 

50: \def\g{\gamma}

51: \def\Mg{\mathcal{M_\gamma}}

52: \def\Vg{V_\gamma}

53: 

54: \def\sp{s^\prime}

55: \def\up{u^\prime}

56: 

57: \def\NatO{N_{\alpha,\|.\|}^{t_0,t}}

58: \def\Na{N_{\alpha,\|.\|}^{-\infty,t}}

59: \def\NbtO{N_{\beta,\|.\|}^{t_0,t}}

60: \def\Nb{N_{\beta,\|.\|}^{-\infty,t}}

61: 

62: \def\Vp{V_{all}^\prime}

63: \def\Vpt{V_{all}^{\prime T}}

64: 

65: \def\fp{f^\prime}

66: \def\np{n^\prime}

67: \def\wp{w^\prime}

68: 

69: 

70: \author{Léonard Gérard and Jean-Jacques Slotine}

71: \title{Neuronal networks and controlled symmetries, \\ a generic framework}

72: \date{}

73: 

74: \setlength{\parskip}{8pt}

75: 

76: \begin{document}

77: 

78: \maketitle

79: 

80: \begin{abstract}

81: The extraordinary computational power of the brain may be related in

82: part to the fact that each of the smaller neural networks that compose

83: it can behave transiently in many different ways, depending on its

84: inputs. We use contraction theory to extend earlier work on synchrony

85: and symmetry, and exploit input continuity of contracting systems to

86: ensure robust control of spatial and spatio-temporal symmetries of the

87: output with the input.

88: \end{abstract}

89: 

90: \section{Introduction}

91: The brain is often described as being composed, in part, of a very large

92: number of small ``identical'' functional

93: units~\cite{Mountcastle1978}. Cortical computation, for instance, is

94: commonly viewed as being organized around cortical

95: columns~\cite{Koerner1999,Kupper2006,Milo2002,Hayon2005}. In this

96: context, a frequent suggestion is that the overall computational power

97: of the brain may be related to some sort of combinatorial

98: complexity~\cite{Malsburg1995}, and to the fact that each part of the

99: brain is reused for different computations.

100: 

101: At the level of individual units, although high behavioral variety

102: could be explained by some learning process ~\cite{Robert1993,Bienenstock1982,Abarbanel2002,Malenka2004,Izhikevich2004}

103: or internal change,it is unlikely that such changes occur in very short

104: periods of time. Instead, the most efficient source of behavioral variety

105: could be simply to have a high input dependency, exploiting the

106: nonlinearity of biological neural networks. In other words, depending

107: on its input, a functional unit could behave in very diverse ways,

108: though remaining stable and robust against noise and small variations

109: in the input.

110: 

111: Intuitively, it is also known that sensory ``input'' enrolls only 5\%

112: or so of the connections to the thalamus~\cite{Sherman2002}, and that

113: a similarly small percentage describes connections from the thalamus

114: to input cortical layers~\cite{Binzegger2004}. Recent research

115: suggests that this ``paucity of input'' between different regions of

116: the brain is actually quite general~\cite{Kennedy2007}.

117: 

118: Inspired by this, we will try to draw a generic framework allowing to

119: observe neural systems under some ``controlling'' input. To this

120: effect, we introduce ``input

121: continuity'' analysis which will provide a way to describe the

122: properties of a unit's output knowing the properties of its

123: input. This framework has broader possible uses and applications than

124: used in this paper and will be discussed in the appendix. Here the

125: main use of the ``input continuity`` will be its link with contracting

126: systems~\cite{Lohmiller1998}, giving us a simple way to change the

127: behavior of systems as will be shown with some toy examples in Section

128: \ref{sec:control} also displaying some interesting results about

129: symmetries and contracting systems, results exposed in Section

130: \ref{sec:symmetries} of the paper.

131: 

132: Indeed the study of symmetries is important in dynamical systems

133: \cite{Spong2005,Golubitsky1999} and more specifically in neural

134: networks. It strongly influences, as we will see, synchrony and

135: polysynchrony (concurrent synchronization), concepts playing an

136: important role in neurobiology

137: ~\cite{Boergers2003,Pinsky1995,Kazanovich2003}. To this matter we will

138: try to cover Lie continuous symmetries and spatio-temporal symmetries,

139: giving some interesting tools to ensure this symmetries in the

140: output. Most of those are based on contracting systems. Let us first

141: recall different contraction theorem and properties.

142: 

143: \section{Contraction}

144: Essentially, a nonlinear time-varying dynamic system will be called {\it contracting} if initial conditions or temporary disturbances are forgotten exponentially fast, i.e., if trajectories of the perturbed system return to their nominal behavior with an exponential convergence rate.  It turns out that relatively simple algebraic conditions can be given for this stability-like property to be verified, and that this property is preserved through basic system combinations.

145: 

146: A nonlinear contracting system has the following properties~\cite{Lohmiller1998,Lohmiller2000,Slotine2002,Wang2004}

147: \begin{itemize}

148: 	\item global exponential convergence and stability are guaranteed

149: 	\item convergence rates can be explicitly computed as eigenvalues of well-defined symmetric matrices

150: 	\item robustness to variations in dynamics can be easily quantified

151: \end{itemize}

152: 

153: \subsection{Basic results}

154: Our general dynamical systems will be in $\mathbb{R}^n$, deterministic, with $f$ a smooth non linear function.

155: \begin{equation}

156: \label{eq:main}

157: 	\dot x = f(x,t)

158: \end{equation}

159: 

160: The basic theorem of contraction analysis, derived in~\cite{Lohmiller1998}, can be stated as:

161: \begin{thm}[Contraction]

162: 	Denote the Jacobian matrix of $f$ with respect to its first variable by $\frac{\partial f} {\partial x}$. If there exists a square matrix $\Theta(x,t)$ such that $\Theta(x,t)^T\Theta(x,t)$ is uniformly positive definite and the matrix

163: 	\[

164: 	F = \left(\dot\Theta + \Theta \frac{\partial f} {\partial x}

165: 	\right) \Theta^{-1}

166: 	\]

167: 	is uniformly negative definite, then all system trajectories converge exponentially to a single trajectory, with convergence rate $|\sup_{x,t}\lambda_\mathrm{max}(F)|>0$. The system is said to be \emph{contracting}, $F$ is called its \emph{generalized Jacobian}, and $\theta(x,t)^T\Theta(x,t)$ its contraction \emph{metric}.

168: \end{thm}

169: 

170: It can be shown conversely that the existence of a uniformly positive definite metric \[M(x,t)=\Theta(x,t)^T\Theta(x,t)\] with respect to which the system is contracting is also a necessary condition for global exponential convergence of trajectories~\cite{Lohmiller1998}.  Furthermore, all transformations $\Theta$ corresponding to the same $M$ lead to the same eigenvalues for the symmetric part $F_s $ of $F$~\cite{Slotine2002}, and thus to the same contraction rate $|\sup_{x,t}\lambda_\mathrm{max}(F_s)|$.

171: 

172: \begin{rem}

173: 	In the linear time-invariant case, a system is globally contracting if and only if it is strictly stable, and $F$ can be chosen as a normal Jordan form of the system with $\Theta$ the coordinate transformation to that form~\cite{Lohmiller1998}.

174: \end{rem}

175: 

176: \begin{rem}

177: 	Contraction analysis can also be derived for discrete-time systems and for classes of hybrid systems~\cite{Lohmiller2000}.

178: \end{rem}

179: 

180: Finally, it can be shown that contraction is preserved through basic system combinations, such as parallel combinations, hierarchies, and certain types of negative feedback, see~\cite{Lohmiller1998} for details.

181: 

182: % {\bf Conventions throughout this paper}: For simplicity we will always

183: % assume a {\it constant} contraction metric. Furthermore, the norm $\|

184: % . \|$ will refer to the weighted euclidian norm $\| \Theta \ . \|$.

185: 

186: \subsection{Contraction toward a linear subspace}

187: The main theorem of~\cite{Pham2006} gives us the ability to prove contraction of all solutions to a subspace $\M$ of the state space. It is a powerful tool that we will use in the symmetry studies, and can be stated as

188: \begin{thm}

189: 	\label{thm:cuong_convergence}

190: 	Consider a linear flow-invariant subspace $\M$ of the system ($f(\M)\subset\M$) and the associated orthonormal projection matrix $U^TU$ (we have $V^TV + U^TU = I_n$ and $x\in \M \iff V x = 0$). All trajectories of the system converge exponentially to $\M$ if the system

191: 	\begin{equation}

192: 	\label{eq:cuong_sysy}

193: 	\dot{y}=V f(V^T y,t)

194: 	\end{equation}

195: 	is contracting with respect to a constant metric. If furthermore we denote the contraction rate for (\ref{eq:cuong_sysy}) by $\lambda>0$, then the convergence to $\M$ will be exponential with rate $\lambda$.

196: \end{thm}

197: We will call the above condition, $V^TfV$ contracting for a constant metric, \emph{ contraction toward $\M$}.

198: 

199: \begin{rem}

200: 	\label{rem:condMg}

201: 	The theorem uses mainly two independent hypotheses

202: 	\begin{itemize}

203: 		\item Contraction condition of Equation \ref{eq:cuong_sysy} : $f$ contracts toward $\M$

204: 		\item Invariance condition of $\M$ : $f(\M) \subset \M$

205: 	\end{itemize}

206: \end{rem}

207: 

208: 

209: \subsection{Contraction yields robustness}

210: It can be shown that (see section 3.7 in \cite{Lohmiller1998} for a proof and generalization)

211: \begin{thm}[Contraction and robustness]

212: 	Consider a contracting system $\dot x = f(x,t)$, with a constant metric $\Theta$ and contraction rate $\lambda$. Let $P_1(t)$ be a trajectory of the system, and let $P_2(t)$ be a trajectory of the \emph{disturbed} system

213: 	\[

214: 	\dot x = f(x,t) + d(x,t)

215: 	\]

216: 	Then the distance $R(t)$ between $P_1(t)$ and $P_2(t)$ verifies $ R(t)\leq \sup_{x,t}\|d(x,t)\| / \lambda$ after exponential transients of rate $\lambda$.

217: \end{thm}

218: 

219: \section{Symmetries and contraction}

220: \label{sec:symmetries}

221: The symmetries of a neural network, defined in a broad sense, can reflect important properties. There are many different ways to express symmetries, such as symmetry of the input, the output, or the system, all of which are usually interdependent.

222: 

223: \subsection{Generic $\g$ operator}

224: Consider a dynamical system  $\dot x = f(x,t)$. A linear operator $\g$ acting

225: over the state space defines two usual ``symmetries'' :

226: \begin{itemize}

227: 	\item symmetry of the system state: if $x=\g x$, we will say that $x$ is

228: $\g$-symmetric

229: 	\item symmetry of the dynamical system : if $\g f=f \g$, we will say

230: that $f$ is $\g$-equivariant \cite{Golubitsky1999}

231: \end{itemize}

232: Note that any linear operator belongs to $GL$, the general linear group, see for example \cite{Spong2005} also using linear operators as symmetries.

233: 

234: The following simple result shows that a contracting dynamical system ``transfers'' its symmetries to its state trajectories.

235: 

236: \begin{lem}

237: 	\label{lem:cont_geq}

238: 	If $f$ is $\g$-equivariant and contracting, all solutions converge exponentially to a unique $\g$-symmetric trajectory $x(t)$.

239: \end{lem}

240: \begin{proof}

241: 	Since the system is contracting all solutions converge exponentially to a single solution $x(t)$. But $\g x(t)$ is also a solution :

242: 	\[

243: 	\frac{d}{dt}\left( \g x(t) \right) = \g \dot{x}(t) = \g f(x,t) = f(\g

244: x,t)

245: 	\]

246: 	hence $x(t) \to \g x(t)$ exponentially.

247: \end{proof}

248: 

249: \subsubsection*{A simple example: permutations}

250: \label{sec:discrete_operator}

251: 

252: Let us illustrate $\g$-symmetry in the simple discrete case of a permutation operator.

253: 

254: Consider $x \in E = \mathbb{R}^n$, and write it as $(x_1,x_2,\dots,x_n)$, the

255: action of a permutation $\g$ on $E$ is defined by $\g x =

256: (x_{\g(1)},\dots,x_{\g(n)})$.

257: 

258: Decompose $\g$ into disjoint non-trivial cycles, \[\g=\sigma_0 \circ \sigma_1

259: \dots \sigma_p\] and decompose the space accordingly as \[E = \mathbb{R}^n =

260: E_{\sigma_0} \times E_{\sigma_1} \times \dots \times E_{\sigma_p} \times

261: E_{I_q}\] with $E_{\sigma_i}$ the space of action of $\sigma_i$.

262: 

263: $\g$ symmetry of the state space describes {\it concurrent synchronization}: in each subspace $E_{\sigma_i}$ the solution is synchronous, thus yielding $p$ co-existing synchronous assemblies, as illustrated in Figure \ref{fig:toy_5}.

264: 

265: \begin{figure}[htb]

266: \fbox{	\begin{minipage}{0.65\linewidth}

267: 		\begin{align*}

268: 			\g &= (0 1) (2 3) (4)\\

269: 			E &= E_{(0 1)} \times E_{(2 3)} \times\mathbb{R}\\

270: 			x &= ( x_0, x_1, x_2, x_3, x_4 )\\

271: 			\g x &= ( x_1, x_0, x_3, x_2, x_4 )

272: 		\end{align*}

273: 		We have $f$ $\g$-equivariant, indeed $\g f = f \g$ :

274: 		\begin{align*}

275: 			f_2(x_0, x_1, x_2, x_3, x_4) &= f_1 ( x_1, x_0, x_3,x_2, x_4 )\\

276: 			f_1(x_0, x_1, x_2, x_3, x_4) &= f_2 ( x_1, x_0, x_3,x_2, x_4 )\\

277: 			f_4(x_0, x_1, x_2, x_3, x_4) &= f_3 ( x_1, x_0, x_3,x_2, x_4 )\\

278: 			f_3(x_0, x_1, x_2, x_3, x_4) &= f_4 ( x_1, x_0, x_3,x_2, x_4 )\\

279: 			f_5(x_0, x_1, x_2, x_3, x_4) &= f_5 ( x_1, x_0, x_3,x_2, x_4 )

280: 		\end{align*}

281: 		And $\g$ symmetry of $x$ would be synchrony of $x_0$ with $x_1$ and $x_2$ with $x_3$ :

282: 		\begin{equation*}

283: 			\left.

284: 			\begin{aligned}

285: 				x_0 = x_1  \\

286: 				x_2 =x_3

287: 			\end{aligned}

288: 			\right\}   \iff \ x = \g x

289: 		\end{equation*}

290: 	\end{minipage}

291: 	\begin{minipage}{0.34\linewidth}

292: 		\includegraphics[width=0.9\linewidth]{figs/toy_5}

293: 	\end{minipage}}

294: 	\caption{Toy example which could model a three layered network.}

295: 	\label{fig:toy_5}

296: \end{figure}

297: 

298: \subsection {Spatio-temporal symmetries}

299: A straightforward extension of spatial symmetries are {\it

300: spatio-temporal symmetries}. Inspired by the theory developed by

301: Golubitsky et al. ~\cite{Golubitsky1999,Buono2001,Golubitsky2000,Josic2006}, we define a \emph{spatio-temporal symmetry $h=(\gamma,T)$}

302: using a spatial symmetry $\gamma$ and a period $T$, according to

303: \[

304: h x(t) = \gamma x(t+T)

305: \]

306: Unlike the $H/K$ theorem of Golubitsky et al. ~\cite{Buono2001,Josic2006}, we

307: do not restrict ourselves to permutation for the spatial symmetry, but, to some

308: linear operator $\gamma$ such as there exists an integer $p_\gamma$, classically

309: called the order of $\gamma$, such that $\gamma^{p_\gamma}=Id$.

310: 

311: \begin{example} Consider a 3-ring with the $h$-symmetry :

312: \[

313: \g \begin{pmatrix}x_1\\x_2\\x_3\end{pmatrix} =

314: \begin{pmatrix}\irdeu(x_1+x_3)\\x_2\\\irdeu(x_3-x_1)\end{pmatrix} \quad

315: \text{so} \quad \g^2 \begin{pmatrix}x_1\\x_2\\x_3\end{pmatrix} =

316: \begin{pmatrix}x_3\\x_2\\-x_1\end{pmatrix} \quad\text{and}\quad (\g^2)^4 = id

317: \]

318: $x$ is $h$-symmetric when :

319: \[

320: 	h x(t) = x(t) \iff \left\{

321: 	\begin{aligned}

322: 		x_1(t) &=\irdeu \left(x_1(t+T)+x_3(t+T) \right) \\

323: 		x_2(t) &= x_2(t+T) \\

324: 		x_3(t) &= \irdeu \left(x_3(t+T)-x_1(t+T) \right) \\

325: 	\end{aligned}

326: 	\right.

327: \]

328: There is a strong interaction between $x_3$ and $x_1$ but interestingly $h x(t)

329: = x(t) \implies x_1(t) = x_1(t+8T) \text{ and for } x_1(0) \neq 0, x_1(t)\neq

330: x_1(t+T) \neq $ the same holds for $x_3$ but $x_2(t)=x_2(t+T)$. This shows that

331: two different rhythms have to coexist.

332: \end{example}

333: 

334: An associated definition can be given for a dynamical system. Specifically, we will say that the system $\dot x = f(x,t)$ (or its dynamics $f$) is \emph{$h$-equivariant} if

335: \begin{equation}

336: 	\label{eq:hypo_h_equi}

337: 	f(\gamma x(t),t) = \gamma f(x(t), t+T)

338: \end{equation}

339: 

340: We then obtain for spatio-temporal symmetries a result similar to Theorem~\ref{lem:cont_geq}, describing the transfer of symmetries from system dynamics to system trajectories.

341: 

342: \begin{thm}

343: \label{thm:temporalsym}

344: 	If $f$ is contracting and $h$-equivariant, then after transients the solutions are $p_\gamma T$ periodic and exhibit the spatio-temporal symmetry $h$.

345: \end{thm}

346: 

347: Basically, all the solutions tend to a periodic solution $x_p(t)$ with the symmetry $h$ : $ x_p(t) = \gamma x_p(t+T)$. This result is an extension of the result that a periodic contracting system exhibits a unique solution of same period~\cite{Lohmiller1998}.

348: 

349: \begin{proof}

350: 	If $x(t)$ is a solution, then $\gamma x(t+T)$ is also a solution:

351: 	\[

352: 		\frac{d}{dt}\left( \gamma x(t+T) \right) = \gamma \dot{x}(t+T) =

353: \gamma f(x(t+T),t+T) = f(\gamma x(t+T),t)

354: 	\]

355: 	Thus, since $f$ is contracting, $ x(t) \rightarrow \gamma

356: x(t+T)$ exponentially fast. This in turn shows that the solution tends to a

357: periodic signal exponentially : by recursion

358: 	\[

359: 	x(t) \to \g^{p_\g} x(t+p_{\g}T) = x(t+p_{\g}T)

360: 	\]

361: 	so that $\forall t \in  [0;p_\gamma T]$ $x(t+n p_\gamma T)$ is a Cauchy

362: sequence, and therefore the limiting function $\lim_{n\to \infty} x(t+n p_\gamma

363: T)$ exists, which completes the proof.

364: \end{proof}

365: 

366: \subsection{Spatial symmetries accomodate weaker contraction}

367: 

368: When dealing only with spatial symmetries, we can weaken the

369: contraction condition on $f$ while still transferring the symmetries

370: of $f$ to the system trajectory.

371: 

372: Conisder the linear subspace of $\g$-symmetric states, $\Mg =

373: \{x,\,x=\g x\}$.  Recall first a standard result linking the symmetry

374: of the system to this linear subspace:

375: \begin{lem}

376: 	\label{lem:equi_flow}

377: 	f is $\g$ equivariant $\implies$ $\Mg$ flow invariant

378: \end{lem}

379: 

380: \begin{proof}

381: 	if $x \in \Mg $ and $f$ is $\g$-equivariant : $\dot{x}= f(x)=f(\g x)=\g

382: f(x) = \g \dot{x}$

383: \end{proof}

384: 

385: In this context, we can thus write Theorem \ref{thm:cuong_convergence} as

386: \begin{lem}

387: 	\label{thm:flowcuong}

388: 	If $\Mg$ is flow-invariant by $f$ (or sufficiently, by

389: Lemma~\ref{lem:equi_flow}, if $f$ is $\g$-equivariant) and $f$ contracts toward

390: $\Mg$, then all solutions $x(t)$ converge exponentially to $\g$-symmetric

391: trajectories.

392: \end{lem}

393: 

394: We will denote by $\Vg^T \Vg$ the orthogonal projector on $\Mg^\bot$ .

395: 

396: As these lemma shows, the more generic $\g$ is, the stronger the contraction

397: condition. In the generic lemma \ref{lem:cont_geq} the hypotheses of symmetry

398: and contraction are {\it independent} on the contrary to last lemma

399: \ref{thm:flowcuong} where the contraction condition depends explicitly on $\g$.

400: 

401: This link between symmetry and contraction is not particularly

402: convenient, since later in section \ref{sec:control} we will aim to

403: shape the equivariance of $f$, and thus the symmetries of the output,

404: while preserving sufficient contraction properties. We now show how to

405: avoid this direct dependence.

406: 

407: \subsection{$\Mall$ and $\MG$ spaces}

408: \label{sec:Mall}

409: We show  how to strengthen the contraction condition of lemma

410: \ref{lem:equi_flow} to make it independent of the symmetry condition,

411: or at least allowing to have the theorem hold with a set $\Gamma$ of

412: different symmetries.

413: 

414: First note:

415: \begin{lem}

416: 	\label{lem:Mall2}

417: 	The choice of $V$ to represent the orthogonal projector has no

418: 	effect on the contraction toward $\M^\bot$.

419: \end{lem}

420: 

421: See proof in appendix \ref{proof:lemMall2}. The proof also shows that,

422: by contrast, using a non orthogonal projector $V_2=T V$ with $T$ square

423: invertible is not sufficient in general.

424: 

425: \begin{rem}

426: 	\label{rem:lemMall_inv}

427: 	The above lemma shows that the contraction condition toward a subspace is preserved when applying an orthonormal transformation $M$ to the projector $V$. Moreover one can also trivially change the metric with an orthonormal transformation : \[M \Theta V f V^T \Theta^{-1} M^T <0 \iff \Theta V f V^T \Theta^{-1} <0 \iff \Theta M V f V^T M^T \Theta^{-1} <0\]

428: \end{rem}

429: 

430: Now the main result of this section :

431: \begin{lem}

432: 	\label{lem:Mall}

433: Consider two linear subspaces $\M$ and $\Mp$ , with $\M\subset\Mp$.

434: If $f$ contracts toward $\M$, then $f$ contracts toward $\Mp$.

435: 

436: More precisely, let $V^TV$ (resp. $V_2^TV_2$) be an orthogonal projector onto $\M^\bot$ (resp. $\Mp^\bot$). If $f$ contracts toward $\M$ with constant metric $\Theta$, then taking $U^T$ a partial isometry from $E_{\Mp^\bot}$ to $E_{\M^\bot}$, with kernel $0$ and image $Im(\Theta V V_2^T E_{\Mp})$, $f$ contracts toward $\Mp$ with constant metric $\Theta_2=U\Theta V V_2^T$

437: \end{lem}

438: 

439: See proof and definitions in appendix (\ref{proof:lemMall})

440: \begin{rem}

441: 	\label{rem:lemMall_U}

442: 	One would be tempted to set $U^T = \Theta V V_2^T$ but this is possible only when $\Theta$ itself is an orthonormal transformation. In particular when $\Theta=Id$ we can set $U^T=V V_2^T$ giving us $\Theta_2=Id$

443: \end{rem}

444: 

445: The main consequence of lemma \ref{lem:Mall} is the ability to

446: determine a sufficient contraction condition for a set $\Gamma$ of

447: symmetries. Indeed we just showed that a sufficient contraction

448: condition would be the contraction toward $\MG = \bigcap_{\g \in

449: \Gamma} \Mg $. This condition and the equivariance with respect to a

450: specific $\g$ allow lemma \ref{thm:flowcuong} to be applied to this $\g$.

451: 

452: In the generic linear case, there is no non trivial unifier, since the

453: intersection of all linear subspaces is reduced to $\{0\}$, corresponding to the

454: contraction of the full system.

455: 

456: But when considering the permutations, the common subspace exists :

457: $\Mall=\{\forall i, \forall j,\ x_i=x_j\}$. This subspace of full synchrony will be

458: very handy and may play an important role in neural networks.

459: 

460: Moreover the contraction toward $\Mall$ will be quite easy to prove when using

461: eventual symmetry of the system. To have more insight into the kind of

462: computation, see Section \ref{sec:permutdetails}.

463: 

464: In summary to the contraction and symmetry section, the main theorem we will use can be stated as

465: \begin{thm}

466: \label{thm:sym}

467: Consider a $\gamma$-equivariant or with $\Mg$ flow-invariant system, and assume

468: one

469: of the following contraction properties (sorted by decreasing strength)

470: 	\begin{itemize}

471: 		\item contraction of the system

472: 		\item contraction toward $\MG$ with $\g\in\G$

473: 		\item with $\gamma$ a permutation, contraction toward $\Mall$

474: 	\end{itemize}

475: 	Then all solutions converge exponentially to a  $\gamma$-symmetric

476: trajectory $x(t)$.

477: \end{thm}

478: \begin{proof}

479: 	We only apply Lemma \ref{thm:flowcuong} and \ref{lem:cont_geq}.The only

480: change is the generalization in the fully contracting case by using $\Mg$

481: flow-invariance hypothesis : a contracting system has a unique solution

482: independent of the initial conditions, then taking a trajectory beginning in

483: $\Mg$ will stay in $\Mg$ by flow- invariance thus forcing the unique solution to

484: be in $\Mg$.

485: \end{proof}

486: 

487: \begin{rem}

488: All the symmetries of $f$, complying the theorem, will be transferred to the trajectories.\emph{The system will maximize synchrony }: for example, if $f$ complies the theorem with $\g=(0, 1)$ and $\g=(1, 2)$ then the system will lead to $x_0=x_1=x_2$.

489: \end{rem}

490: 

491: \section{Input continuity}

492: \label{sec:input_continuity}

493: \subsection{Input continuity motivations}

494: 

495: We now exploit these results on global symmetries in a control framework,

496: by introducing control inputs in the system to modulate its dynamics.

497: 

498: We will observe $f$ under the influence of some input $u$:

499: \begin{equation}

500: 	\label{eq:generic_system}

501: 	\dot{x}(t) = g(x,u(t),t) = f(x,t)

502: \end{equation}

503: The input $u$ doesn't represent all the input of the actual system, the rest of

504: the actual input can still be hidden as before inside the function $g$. This

505: choice underlines the fact that we want to study the system response to a

506: decisive part $u$ of the actual input. Then the idea is to control the output

507: with the input to get the property that we want see Section \ref{sec:control}.

508: In an in vivo situation, we can see all the feedback loops from the upper part

509: of the brain to the bottom part of the brain as control inputs $u$, but also the

510: input from the bottom to the top, and every other connections.

511: 

512: The response to $u$ of the system will be some output $s$ defined by the state

513: of the system. Typically we will use $s(t)=x(t)$ or some projection of the state

514: $s(t)=Px(t)$.

515: 

516: With this point of view we introduce {\it input continuity} analysis, in order

517: to describe the properties of a system's output knowing the properties of its

518: input while ensuring stability and robustness.

519: 

520: As a general matter, we want to be able to say two things, first ``if $u$ has

521: this property then $s$ will have that property'' and secondly ``if the input $u$

522: is close to an ideal input $u^\prime$ then $s$ will be close to the ideal

523: output $s^\prime$''. This will be formalized by the notion of

524: ''input continuity'' :

525: \begin{align}

526: 	\forall \epsilon \geq 0,\ \exists \eta \geq 0 \text{ such as }

527: d_1^t(u^\prime , u) \leq \eta \implies d_2^t(s^\prime , s) \leq \varepsilon

528: \label{eq:generic_input_continuity}

529: \end{align}

530: $d_1^t$ (resp. $d_2^t$) is a pseudo distance of the space of the input $u$

531: (resp. the output $s$). This pseudo distances help us to define the notion of

532: being close as traditionally but also the properties corresponding to the chosen

533: pseudo distance : if $d_1^t(u^\prime , u)=0$, then $u$ and $u^\prime$ are in the

534: same class defining  some property see Section \ref{sec:properties_distances}

535: for details with some interesting examples shown in Section

536: \ref{sec:codes_distances}.

537: 

538: \emph{Depending greatly on the distance we use, input continuity will be a

539: modular tool to ensure robustness of specific properties of the output given the

540: properties of the input.}

541: 

542: The modularity of input continuous block is something very generic and powerful,

543: more discussion can be found in section \ref{sec:lego_game}, but from now we

544: will restrict ourself to the study of a powerful input continuity found in

545: contracting systems.

546: 

547: \subsection{Input continuity and contraction}

548: \label{sec:contraction_inputcont}

549: There is no generic way to show input continuity of a

550: system. Depending on the type of system (discrete or continuous) and

551: the distance we use, we would have to examine each case we

552: encounter. But in the case of a contracting system, we can state some

553: powerful generic properties. This will allow us to combine Theorem \ref{thm:sym}

554: and the input continuity without any effort, the contraction being already an

555: hypothesis. The main tool to study input continuity of a contracting system is its robustness :

556: 

557: \subsubsection{Contracting systems}

558: We will compare the perturbed state $\sp$ with perturbed input $\up$

559: and the wanted state $s$ with perfect input $u$ :

560: \begin{align*}

561: 	\dot{s} &= f(s,u,t)\\

562: 	\dot{\sp} &= f(\sp,\up,t) = f(\sp,u,t) + h(\sp,t) \\

563: 	\text{ with } h(t) &= f(\sp,\up,t) - f(\sp,u,t)

564: \end{align*}

565: Then, if $f$ is contracting, we can prove with the generalized form of the

566: robustness seen in ~\cite{Lohmiller1998} that

567: \begin{align}

568: 	\label{robustness_contraction}

569: 	\dot{R} + \lambda R \quad\leq&\quad\|h(t)\| \\

570: 	\label{eq:m_R}

571: 	R(t) \quad\leq &\quad e^{-\lambda (t-t_0)} R(t_0) +

572: \displaystyle\int_{t_0}^{t} e^{-\lambda (t-\tau)} \|h(\tau)\|\, d\tau \\

573: \nonumber

574: \end{align}

575: with $r = \sp - s$, $R(t) = \|r(t)\|$, $\|.\|$ the norm of the space in which

576: $f$ is proved contracting with contraction rate $\lambda$. By convention we will

577: have $R(-\infty) = 0$.

578: \begin{rem}

579: 	If the contraction analysis uses a metric, it is reflected in the norm

580: $\|.\|$, for instance in the case of the use of the 2-norm $\|.\|_2$ and a

581: metric $\Theta$ we will use $\|.\|=\|\Theta\, .\|_2$

582: \end{rem}

583: 

584: We can prove the input continuity considering the space of the input signal and

585: output signal with these two norms :

586: \begin{align}

587: 	\|x\|_{\infty}^{-\infty,t} &= Sup_{\ \tau < t} \{(\|x(\tau\|)\}

588: \label{norm_infiny} \\

589: 	\Na(x) &= \int_{-\infty}^{t}\|x(\tau)\|e^{-\alpha (t-\tau)} \, d\tau \

590: \text{ with } \alpha>0 \label{eq:norm_alpha}

591: \end{align}

592: 

593: \begin{thm}

594: 	If all the signals are bounded and $f$ is

595: contracting and uniformly continuous in time, then we have the input continuity

596: using the uniform norm (\ref{norm_infiny}) in both input and output space.

597: \end{thm}

598: \begin{proof}

599: 	From \ref{robustness_contraction} we have : $\|r\|^{-\infty,t}_{\infty} \leq \frac{1}{\lambda}\|h\|^{-\infty,t}_{\infty}$. In the mean time, contraction gives us space continuity, then using the hypothesis of uniform continuity in time and the Heine's theorem over the compact space of bounded signals, $\exists k \in \mathbb{R},\ \|h\|^{-\infty,t}_{\infty} \leq k \|u^\prime -u\|$.

600: \end{proof}

601: 

602: Note that the boundedness of the input signals is a plausible condition {\it in vivo}.

603: 

604: In our control context, a more flexible and meaningful tool is the

605: norm (\ref{eq:norm_alpha}) with exponentially fast forgetting :

606: \begin{thm}

607: 	\label{thm:continuity_contracting}

608: 	If all the signals are bounded and $f$ is

609: contracting and uniformly continuous in time, then we have the input continuity

610: with $\Na$ in the input space and $\Nb$ in the output space.

611: \end{thm}

612: See proof in appendix \ref{proof:thm_continuity_contracting}. 

613: 

614: \begin{rem}

615: 	When dealing with this kind of norms, Lemma \ref{lem:change_alpha} can be very convenient.

616: \end{rem}

617: 

618: \subsubsection{Systems contracting toward a subspace}

619: \label{sec:input_continuity_toward}

620: With contraction toward linear subspace, we want to ensure the property  ''we

621: are in $\M$`` which can be easily characterized with a generic semi-norm see

622: Section \ref{sec:properties_distances} for more explanations :

623: \begin{equation*}

624: 	\label{eq:normM}

625: 	N_{\mathcal{M}}(x) =  \|V x\|

626: \end{equation*}

627: 

628: We can now state an adapted version of Theorem \ref{thm:continuity_contracting}

629: \begin{thm}

630: 	\label{thm:cont_cont_M}

631: 	If all the signals are bounded, $f$ uniformly (in time) continuous, with Equation \ref{eq:cuong_sysy} contracting and $\mathcal{M}$ flow-invariant, then the system is input continuous with input norm $N_{\alpha,\|.\|}^{-\infty,t}$ and output pseudo norm $N_{\alpha,N_{\mathcal{M}}}^{-\infty,t}$.

632: \end{thm}

633: \begin{rem}

634: 	This obviously works with the uniform norm in the same way.

635: \end{rem}

636: \begin{proof}

637: 	We can apply Theorem (\ref{thm:continuity_contracting}) on the contracting system (\ref{eq:sys_M}) with space norm $\|.\|$, giving us input continuity of this system with norm $N_{\alpha,\|.\|}^{-\infty,t}$ ( resp. $N_{\beta,\|.\|}^{-\infty,t}$ ) as input norm ( resp. output norm ). Using notation of the Theorem, we can link $y$ and $x$, indeed if we set $y=Vx$ we get the system

638: 	\begin{equation}

639: 		\label{eq:sys_M}

640: 		\dot{y}= V \dot{x} = Vf(x) = Vf(V^T y + U^T U x)

641: 	\end{equation}

642: 	which is contracting with respect to $y$, equivalently with the contraction of the system (\ref{eq:cuong_sysy}).Then we have $N_{\mathcal{M}}(x) = \|Vx\| = \|y\|$ so we can directly apply the result to the original system using $N_{\alpha,N_{\mathcal{M}}}^{-\infty,t}$ as output norm.

643: \end{proof}

644: 

645: 

646: \section{Control}

647: \label{sec:control}

648: With the power given by theorem \ref{thm:sym} and the flexibility given by input

649: continuity, we can now get the system to exhibit specific symmetries

650: with the help of a small controlling input. The global property of contraction

651: and input continuity will be required to robustly do transient and multiple

652: changes in the system and the symmetries of the output.

653: 

654: \subsection{Main idea}

655: Rather than looking at symmetrical solutions a system

656: may exhibit, as in the H/K Theorem of Golubitsky et

657: al.~\cite{Golubitsky1999,Buono2001,Golubitsky2000,Josic2006}, we

658: consider what symmetries the system may exhibit when submitted to

659: specific external inputs.

660: 

661: We consider a system of the form \[\dot{x}=f(x,t)=g(x,u(t),t)\]

662: where now $u(t)$ is a ``control input''. We will control the symmetry of the

663: system's output by modifying its input. To do so we will use the theorems

664: \ref{thm:sym} and \ref{thm:temporalsym} on the function $f$. We will need

665: \begin{itemize}

666: 	\item a symmetry condition ($\gamma$-equivariance or flow-invariance)

667: 	\item a contraction condition (contraction or contraction toward a

668: subspace)

669: \end{itemize}

670: 

671: The contraction condition (in any form) will give us input continuity

672: as shown in section \ref{sec:contraction_inputcont}, allowing us to

673: plug the input at any time instead of controlling the system from the

674: beginning, and still be exponentially close to the desired output. The

675: symmetry condition will lead the system to a state expressing the

676: desired symmetry.

677: 

678: The input can have different functions. It can determine the

679: contraction condition as explained in the following

680: \ref{sec:input_selection_of_contraction}, but also, and mainly, change

681: the symmetries of the system, as we now detail.

682: 

683: \subsection{Selection of spatio-temporal symmetries of the system}

684: \label{sec:selection_of_symmetry}

685: We first need to link the symmetries of $f$ and those of $g$ and

686: $u$. $g$ will be said h-equivariant if \[g(\gamma x,\gamma y,t) =

687: \gamma g(x,y,t+T)\]

688: \begin{thm}

689: 	$g$ h-equivariant and $u$ h-symmetric $\implies$ $f$ h-equivariant

690: \end{thm}

691: \begin{proof}

692: 	\begin{align*}

693: 		f(\g x(t),t) &= g(\g x(t),u(t),t) = g(\g x(t),\g u(t+T),t) \quad

694: \text{ by symmetry of }u\\

695: 		\g f(x(t),t+T) &= \g g(x(t),u(t+T),t+T) =g(\g x(t),\g u(t+T),t)

696: \quad \text{by equivariance of} g

697: 	\end{align*}

698: \end{proof}

699: 

700: In the general case, $g$ h-equivariant is not sufficient, moreover,

701: increasing the symmetry thanks to the input is very unlikely, since it

702: would require an intelligent input, quite as complex as the neuron

703: model. This is not our goal since we consider the input to be

704: ''small``, and the neuron model a realistic non linear dynamical

705: system. Thus for practical purposes the theorem is an

706: equivalence. Once the symmetries $\Gamma_g$ of $g$ are determined, we

707: can set an input with the symmetries $\Gamma_u$ to control the final

708: symmetries $\Gamma$ by considering $\Gamma = \Gamma_g \cap

709: \Gamma_u$, i.e., intersecting the symmetries of $g$ with those we set

710: in the input.

711: 

712: \begin{example} Consider the simple dynamics

713: \begin{equation*}

714: 	f(x,t)=g(x,u(t),t) =

715: 	\begin{cases}

716: 		-x_1^3 + u_1(t) + sin(t)\\

717: 		-x_2^3 + u_2(t) + sin(t+\pi/3)\\

718: 		-x_3^3 + u_3(t) + sin(t+2\pi/3)

719: 	\end{cases}

720: \end{equation*}

721: Here $g$ is $h=((1,2,3),\pi/3)$-equivariant, thus also

722: $h^2=((1,3,2),2\pi/3)$-equivariant and $h^3=(id,2\pi)$-equivariant, etc.

723: Taking for example $u$ $h^2$-symmetric, but not $h$-symmetric, we have $f$

724: $h^2$-equivariant but not $h$-equivariant. Note that then the solution will be

725: $6\pi$ periodic instead of $2\pi$ periodic, and also that nothing is needed or

726: proved about the respective phases of the different signals and elements.

727: 

728: This also shows that the creation of symmetries is unlikely. If

729: $g$ was only $h^2$-equivariant, having $u$ $h$-symmetric will not make

730: $f$ $h$-equivariant, but only $h^2$-equivariant.

731: \end{example}

732: 

733: \subsection{Control of the contraction condition of the system}

734: \label{sec:input_selection_of_contraction}

735: Having the system always contracting will probably not be the generic case and

736: the most biologically sound. Rather, we want to ``turn on'' the contraction

737: property at a specific time using the input. It can be represented by :

738: 

739: \begin{equation}

740: \label{eq:activator}

741: 	\dot{x}=f(x,t) - k\chi_{on}x

742: \end{equation}

743: Having $k$ big enough and the activator $\chi_{on}=1$, the system will contract. With some systems like a set of FitzHugh-Nagumo elements, putting such a term only over the potential variables will give contraction toward $\Mall$.

744: 

745: This ``contracting input'' is a transient negative feedback loop which can

746: be turned on and off through the control of the activator. This can't be as

747: simple in neural models, but we propose a quite meaningful and simple

748: ``implementation'' :

749: 

750: We consider the circuit of Figure \ref{fig:frequency_selector_system} but

751: without delay $d$. This circuit seems to be part of the neighborhood of many

752: cortical pyramidal cells in the treatment of extracortical afferent excitations

753: \cite{Buzaski2006}. We suggest a behavior : The pyramidal neuron $x_p$ would

754: have lots of gap junction with its touching inhibitory interneuron $x_i$. The

755: interneuron being way smaller will be driven by the pyramidal one, so that

756: $x_i\simeq x_p$. Next setting a low firing threshold for the interneuron would

757: allow to have it spiking proportionally to its potential. The resulting

758: inhibition of the pyramidal neuron will thus have the desired shape

759: $\simeq-k.x_p$. Input continuity permit to compute the distance between this

760: implementation and the perfect instantaneous negative feedback.

761: 

762: The activator will then be easily implemented by some inhibition of the

763: interneuron : no inhibition means $\chi_{on}=1$, $0$ otherwise.

764: 

765: The first two examples use the activated contraction to control spatio-temporal

766: symmetries (example of section \ref{ex:3ring} or section

767: \ref{ex:unstableinputcont}). Then we show in more details a grid example using

768: the $\Mall$ idea.

769: 

770: 

771: \subsection{Examples}

772: 	Throughout this section we illustrate some of the above possibilities shown through basic examples, mostly using  FitzHugh-Nagumo neural models,

773: 	\begin{align}

774: 	\label{eq:FN}

775: 		&\left\{

776: 		\begin{aligned}

777: 				\dot{v}&=v(\alpha -v)(v-1)-w+I\\

778: 				\dot{w}&=\beta(v-\gamma w)

779: 		\end{aligned}

780: 		\right.\\

781: 		&\alpha=6, \beta=3, \gamma=0.03 \nonumber

782: 	\end{align}

783: with $I$ the synaptic input function.  Although most of the time we refer to

784: our system elements as ``neurons'', one should notice that more generally

785: the theory developed here applies to $f$ representing neural

786: networks, whose equations can be actually very

787: similar to FitzHugh-Nagumo models.

788: 

789: The use of FitzHugh-Nagumo neural models is motivated by its

790: simplicity while still a reasonably descriptive neuron model, and it has

791: the desired properties of contraction toward $\Mall$ when coupled only

792: through the potential variable see \cite{Pham2006}. This property is

793: kept when using the more precise Hodgkin-Huxley model see

794: \cite{Zyto2006}.

795: 

796: \subsubsection{Leading to unstable state : transient synchronization}

797: \label{ex:unstableinputcont}

798: In this example we will use the action of a contracting input making

799: the system contracts toward $\Mall$ from time 75 to 95. This transient

800: contraction results in a transient synchronization, which is often

801: considered as a very important neural processing process

802: \cite{Palva2005,Palva2005a}. Consider the system seen in Figure

803: \ref{fig:syn_perfect_system}. Neuron 1 and 2 are two FitzHugh-Nagumo

804: neurons with an inhibitory symmetrical link between them. As we can

805: see Figure \ref{fig:syn_perfect_results} before we put the contracting

806: input, the mutual inhibitory link leads naturally the system to

807: antiphase. But after synchronizing the two neurons by force with the

808: input, they stay in the unstable state where they are equal see Figure

809: \ref{fig:syn_perfect_nonoise}, this during transient when some level

810: of noise is added see Figure \ref{fig:syn_perfect_noise}.

811: 

812: This example illustrates the idea that with the input we can lead the

813: system to a non 'natural' state, in much more complex networks this

814: could be some basic phenomenon to allow different computations with

815: the same network.

816: 

817: \begin{figure}[!htb]

818: 	\center

819: 	\includegraphics[width=0.4\linewidth]{figs/syn_perfect_fig}

820: 	\caption{System for transient synchronization}

821: 	\label{fig:syn_perfect_system}

822: \end{figure}

823: \begin{align*}

824: 	\dot{x_1} &= f(x_1) -\mu x_2 + e + I_1 \\

825: 	\dot{x_2} &= f(x_2) -\mu x_1 + e + I_2 \\

826: 	I_i &= \lambda \chi_{t\in\lbrack75,95\rbrack}( u(t) - x_i)

827: \end{align*}

828: 

829: \begin{figure}[!htb]

830: 	\center

831: 	\subfloat[without noise]{\includegraphics[width=0.7\linewidth]{figs/syn_perfect_out_x_1_x_2}

832: \label{fig:syn_perfect_nonoise}}\\

833: 	\subfloat[with noise]{\includegraphics[width=0.7\linewidth]{figs/syn_perfect_out_xn_1_xn_2}

834: \label{fig:syn_perfect_noise}}\\

835: 	\subfloat[input $u$ with activator]{\includegraphics[width=0.7\linewidth]{figs/syn_perfect_out_u}}

836: 	\caption{Transient synchronization $\mu=0.1, \lambda=5, noise=5, e=20$ }

837: 	\label{fig:syn_perfect_results}

838: \end{figure}

839: \clearpage

840: \subsubsection{Choose the spatio-temporal symmetry in a 3 ring hysteresis system}

841: \label{ex:3ring}

842: We have here Figure \ref{fig:3ring_system} a ring of 3 FitzHugh-Nagumo

843: neurons, each inhibiting its right neighbor. This system as we can

844: see Figure \ref{fig:3ring_results} has a stable state where none of the neurons

845: spike (here $e$ is not big enough to make them spike because of the

846: overall inhibition) but also another stable state $( (1,2,3) , T/3 )$

847: symmetric, where the neurons are spiking one after the other and each

848: neuron has a period $T$ (the inhibition being in the refractory period

849: of the next one, makes the FitzHugh-Nagumo spike shortly after). To pass

850: from one state to the other we use a contracting input function with

851: an input signal exhibiting the symmetry we want to see, in figure

852: \ref{fig:3ring_results} we lead the system to the rotating wave and

853: then back to silence.  The system

854: \begin{align*}

855: 	\dot{x_1} &= f(x_1) -\mu x_3 + e_1 + I_1 \\

856: 	\dot{x_2} &= f(x_2) -\mu x_1 + e_2 + I_2 \\

857: 	\dot{x_3} &= f(x_2) -\mu x_2 + e_3 + I_2 \\

858: 	I_i &= \lambda\ \chi_{on}( u_i(t) - x_i)

859: \end{align*}

860: \begin{rem}

861: 	The control input to get back to the silent mode is here a long step but

862: can be any spatio-temporal identity signal ( equal for neuron 1, 2 and 3).

863: Inspired from the visual saccades involving bursting, this long step could model

864: a high frequency burst, being here a way to reinitialize our network to a silent

865: state before a new computation cf \cite{Kupper2005}.

866: \end{rem}

867: 

868: \begin{figure}[!htb]

869: 	\includegraphics[width=0.4\linewidth]{figs/3ring_fig}

870: 	\caption{3 ring hysteresis system}

871: 	\label{fig:3ring_system}

872: \end{figure}

873: \begin{figure}[!htb]

874: 	\subfloat{\includegraphics[width=\linewidth]{figs/3ring_out_x_1_x_2_x_3}}

875:  \\

876: 	\subfloat{\includegraphics[width=\linewidth]{figs/3ring_out_u_1_u_2_u_3}}

877: 

878: 	\caption{3 ring hysteresis system, state selection by input}

879: 	\label{fig:3ring_results}

880: \end{figure}

881: \clearpage

882: \subsubsection{Grid and group selection}

883: 

884: This example illustrate the idea of symmetry selection

885: without changing the contraction condition ( see section

886: \ref{sec:selection_of_symmetry}) and illustrating a common issue of 2D

887: segmentation. The concept will be to use a toric grid being

888: contracting toward $\Mall$, then to select thanks to

889: the input the desired (necessarily flow invariant) groups.

890: 

891:  The grid is formed with neurons connected through diffusive

892: connections ( representing gap junctions and other direct contacts

893: between neurons ) to their four closest neighbors. We will set the

894: coupling strength $k$ to be strong enough to ensure contraction toward

895: $\Mall$ ( it exists see balanced coupling in

896: \cite{Pham2006}).Following the control idea of section

897: \ref{sec:selection_of_symmetry}, the system will polysynchronize

898: depending on the flow invariant subspaces found in the input $u$.

899: 

900: Specifically we use a 5x5 grid of identical FN neurons modeled as

901: (\ref{eq:FN}). For each run, the initial conditions are set so that

902: the neurons phase are spread out. We will consider 2 flow

903: invariant patterns chosen among the one we can find in

904: \cite{Antoneli2007}, namely pattern 1 and 2 of figure

905: \ref{fig:grid_patterns}, the coloring represents the wanted flow

906: invariant groups. $input\_0$ ( resp. $input\_1$ ) will represent the

907: input of the white (resp. black) neurons. To separate the two groups

908: of neurons but also showing some interesting interactions (we don't

909: want all the neurons to be synchronized even if the FitzHugh-Nagumo

910: model being a 2 dimensional model goes very easily to full synchrony

911: with this grid connection) we chose (see plots in figure

912: \ref{fig:grid_inputs})

913: \begin{align*}

914: 	input\_0i = 2 \sin(2\pi t/60) +21 \\

915: 	input\_1i = 10 \sin(2\pi t/8) - 20

916: \end{align*}

917: When we will say with noise we add to the input of each neuron a

918: random noise taking a new value between $0$ and $1$ each $0.05$

919: second.

920: 

921: We first apply pattern 1. Without coupling Figure \ref{fig:grid_0150} we observe

922: the natural behavior of the FitzHugh-Nagumo model which 'synchronize' with its

923: input quite easily as can be observed with one of the two groups. Setting

924: $k=0.3$ Figure \ref{fig:grid0.3150} gives the expected behavior, two groups

925: appear exponentially fast.

926: 

927: We then use the input to change the synchronized groups from pattern 1 to pattern

928: 2 at time

929: $t=150$. First as expected the speed of convergence increase with $k$ but also

930: the

931: synchrony among groups, indeed the coupling tends to synchronize

932: groups also.

933: 

934: Adding individual noise Figure \ref{fig:grid_s12_150_noise} obviously

935: prevent synchrony without the coupling, but with the coupling we

936: obtain the desired grouping with some glitches allowed by the input

937: continuity (the difference in the norms are always above a certain

938: mean of the integral of the noises).

939: 

940: \begin{figure}[!htb]

941: 	\center

942: 	\includegraphics[]{figs/grid_pattern1}

943: 	\qquad

944: 	\includegraphics[]{figs/grid_pattern2}

945: 	\caption{Grid pattern used to determine the groups}

946: 	\label{fig:grid_patterns}

947: \end{figure}

948: 

949: \begin{figure}[!htb]

950: 	\center

951: 	\subfloat[Input]{\includegraphics[width=0.8\linewidth]{figs/grid_s12_300}\label{fig:grid_inputs}}\\

952: 	\subfloat[$k=0$]{\includegraphics[width=0.8\linewidth]{figs/grid_k_0_s12_150_noise_0}\label{fig:grid_0150}}\\

953: 	\subfloat[$k=0.3$]{\includegraphics[width=0.8\linewidth]{figs/grid_k_0_3_s12_150_noise_0}\label{fig:grid0.3150}}\\

954: 	\subfloat[$k=0.7$]{\includegraphics[width=0.8\linewidth]{figs/grid_k_0_7_s12_150_noise_0}}

955: 	\caption{Grid pattern 1 followed by pattern 2 at $t=150$}

956: 	\label{fig:grid_s12_150}

957: \end{figure}

958: \begin{figure}[!htb]

959: 	\center

960: 	\subfloat[$k=0$]{\includegraphics[width=\linewidth]{figs/grid_k_0_s12_150_noise_1}}\\

961: 	\subfloat[$k=0.3$]{\includegraphics[width=\linewidth]{figs/grid_k_0_3_s12_150_noise_1}}\\

962: 	\subfloat[$k=0.7$]{\includegraphics[width=\linewidth]{figs/grid_k_0_7_s12_150_noise_1}}

963: 	\caption{Grid pattern 1 followed by pattern 2 at $t=150$ with noise}

964: 	\label{fig:grid_s12_150_noise}

965: \end{figure}

966: \clearpage

967: 

968: \section{Discussion}

969: 

970: Robustness and globalness are interesting properties of the studied

971: methods. Many studies uses approximations about the trajectories,

972: considering that the neurons are close to their limit cycle, for

973: instance~\cite{Kopell2000,Kopell2004,Galan2005}. Since contraction is

974: a global property, nothing is assumed about the location of the state

975: of the system when we plug in a new input, allowing fast input-driven

976: switching between different synchronization patterns. Global

977: exponential convergence to the desired behaviors is obtained, with

978: quantifiable convergence rates.  Globalness also avoids some of the

979: topological difficulties associated to the study of large networks of

980: phase oscilllators. In the input continuity proof, only bounded

981: signals are needed.

982: 

983: The modularity of the tools is a strong property allowing to mix

984: studies of networks done at different scales (neuron, neural mass,

985: neural assemblies and so on).  Indeed, while we use neuron models as

986: our main dynamical system unit, the development can be applied to

987: other dynamical systems networks.

988: 

989: The symmetries used here are quite generic, the extension of spatial

990: symmetries to linear operators and the extension to spatio-temporal

991: symmetries seems important, since it is required to deal with the idea

992: of spatio-temporal pattern coding in the brain, and natural external

993: stimuli.

994: 

995: Two main weaknessness can be pointed out:

996: 

997: First our control over the symmetries of the system doesn't prevent

998: the system to exhibit {\it more} symmetries in the end $-$ mainly

999: ensuring to have two synchronized groups of neurons doesn't prevent to

1000: have in fact total synchrony. To prevent the system to go to more

1001: synchrony, it is important in practice to actively separate the groups

1002: (as we did in the grid example), e.g. through inhibition, or break the

1003: symmetry.

1004: 

1005: Second, the spatio-temporal case is very interesting and quite

1006: unexplored. In this paper we only drew conclusions for fully

1007: contracting systems, a restrictive condition. A relaxed condition

1008: similar to the existence of $\Mall$ in the spatial case would be more

1009: desriable (if perhaps unlikely).

1010: 

1011: Finally, the small circuit of a main neuron and its inhibitory

1012: interneuron is interesting in its own right.  This circuit is proposed

1013: as a plausible implementation of ``contracting inputs'' in section

1014: \ref{sec:input_selection_of_contraction}, but also represents an

1015: frequency selector circuit as seen in example

1016: \ref{ex:frequency_selector}. Its biological relevance may be

1017: further investigated.

1018: 

1019: \section{Appendix}

1020: \subsection{Frequency selector / contraction activator network}

1021: \label{ex:frequency_selector}

1022: The small classical cortical circuit : Figure \ref{fig:frequency_selector_system} seems to be omnipresent on most cortical cells, set to treat the extracortical afferent excitations. \cite{Buzaski2006} We have already seen that this network could be of great use to control the contraction of the system Section \ref{sec:input_selection_of_contraction}. But with different parameters it can be an interesting frequency selector.

1023: 

1024: We will use the fact that we can control the frequency of both neurons with the input (by synchronizing the neuron with the input), conjugated with a fixed delay of inter-inhibition which will be the intrinsic frequency shut down of this circuit :

1025: 

1026: \begin{figure}[!htb]

1027: 	\centering

1028: 	\includegraphics[width=0.2\linewidth]{figs/freqSel}

1029: 	\caption{$d=11$, big enough $\lambda^{\prime}$}

1030: 	\label{fig:frequency_selector_system}

1031: \end{figure}

1032: 

1033: There is a main pyramidal neuron $x_p$ connected to its inhibitory interneuron

1034: $x_i$ with a synaptic delay $d$. We set an input $u$ with a specific frequency. First we use a frequency close to the corresponding delay, see Figure \ref{fig:freqSel_r1} we see $x_i$ which adapt to the input and then since the delay is of a close value, $x_p$ stops to spike. In Figure \ref{fig:freqSel_r2} we kept the delay of Figure \ref{fig:freqSel_r1} but set a further input frequency.

1035: 

1036: The system :

1037: \begin{align*}

1038: 	\dot{x_p} &= f(x_p) + e + \lambda(u(t) - x_p) -w_i x_i(t-d) \\

1039: 	\dot{x_i} &= f(x_i) + e + \lambda^{\prime}(u(t) - x_i)

1040: \end{align*}

1041: 

1042: \begin{figure}[!htb]

1043: \begin{minipage}{0.5\linewidth}

1044: 	\subfloat{\includegraphics[width=0.8\linewidth]{figs/freqSel2_x_p}}\\

1045: 	\subfloat[$d=11$,$\lambda^{\prime}=1.3$, input period $10$]{\label{fig:freqSel_r1}\includegraphics[width=0.8\linewidth]{figs/freqSel2_u_x_i}}

1046: \end{minipage}

1047: \begin{minipage}{0.5\linewidth}

1048: 	\subfloat{\includegraphics[width=0.8\linewidth]{figs/freqSel4_x_p}}\\

1049: 	\subfloat[$d=11$,$\lambda^{\prime}=1.3$, input period $14$]{\label{fig:freqSel_r2}\includegraphics[width=0.8\linewidth]{figs/freqSel4_u_x_i}}

1050: \end{minipage}

1051: \end{figure}

1052: 

1053: 

1054: \clearpage

1055: \subsection{Input continuity precisions}

1056: 

1057: \subsubsection{Lego game}

1058: \label{sec:lego_game}

1059: The input continuity as introduced in Section \ref{sec:input_continuity} and

1060: defined at (\ref{eq:generic_input_continuity}) allow us to play the Lego game :

1061: plug in serial and parallel blocks having input continuity and get a bigger

1062: block with input continuity. To plug in serial, the property of the output of

1063: the first block should of course imply the property needed by the input of the

1064: second block. The parallel block is just a redefinition of the input and input

1065: space using for example as a new distance the $\sup$ of the two original

1066: distances. Feedbacks are of a different kind of plug and we will need some more

1067: refined analysis for example in two steps : block A has two input, one is a

1068: feedback, if we can prove input continuity of A depending only on the first

1069: input, we have some property of the output, then knowing that, we have some

1070: property of the feedback input and we have a new ( and eventually stronger )

1071: input continuity of A using the full property that we now know on the input,

1072: giving the full. This kind of computation is close to the classical idea of

1073: predictive top-down signal which is used to improve the treatment of the

1074: feedforward input signal.

1075: 

1076: \subsubsection{Norms, distances and properties}

1077: \label{sec:properties_distances}

1078: There is a lot of different coding we can think of being used , like phase,

1079: frequency, timing, spatial, etc ( some interesting examples among thousands

1080: \cite{Leutgeb2005,Kupper2005} ) Each defining different ''distances`` between

1081: signals and natural properties we could be tempted to prove on signals. In

1082: general we will have a pseudo-distance in the signal space but a real distance

1083: in the property space, with the equivalence classes $x \in [a] \iff d(x,a)=0$,

1084: representing signals with the same property. To generalize this idea, we can

1085: define a measure $\varphi^t :  \mathbb{R}^n \mapsto \mathbb{R}^m$ of our signal

1086: which describe some properties that is if $\varphi^t(x_1) = \varphi^t(x_2)$ then

1087: $x_1$ and $x_2$ have the same property at time t, with this we have a generic

1088: pseudo-distance defined with the usual norm in the property space~:

1089: \[d^t_{\varphi^t}(x_1,x_2) = \| \varphi^t(x_1) - \varphi^t(x_2) \|^t_m\]. This

1090: construction was used for the input continuity of systems contracting toward a

1091: linear subspace Section \ref{sec:input_continuity_toward}

1092: 

1093: \begin{rem}

1094: 	The interest of using a norm over using a distance : when is available a

1095: norm coming from a dot product, we can define orthogonal projection on

1096: subspaces, giving minimal distance between a real input and the space of desired

1097: input.

1098: \end{rem}

1099: \subsubsection{Classical codes and distances}

1100: \label{sec:codes_distances}

1101: We can define several distances to represent classical coding or properties of

1102: neuronal networks :

1103: \begin{itemize}

1104: 	\item a pseudo-distance for frequency coding :\\

1105: 		Considering the signals defined by their spiking times :

1106: $t^k_{x_1}$ and $t^k_{x_2}$, let $T^t_{x_1}$ be the mean distance between

1107: $t^k_{x_1}$ and $t^{k+1}_{x_1}$ before time $t$ ( representing the mean period

1108: of spiking ) then we could use :

1109: 		\[

1110: 		d^t(x_1,x_2)=|T^t_{x_1}-T^t_{x_2}|

1111: 		\]

1112: 	\item a pseudo-distance for synchrony in a certain subspace defined by a

1113: projector $V$, remark that this is the one we mainly use throughout the paper:

1114: 		\[

1115: 		d^t(x_1,x_2)=|V(x_1)-V(x_2)|

1116: 		\]

1117: \end{itemize}

1118: 

1119: Depending on the signal representation, we will have different classical

1120: distances :

1121: \begin{itemize}

1122: 	\item With signals as a set of spikes, described by the set of spike's

1123: dates : to signal $u$  we associate $\{ \tau^u \}$ so that $u(t)= 1 \text{ if }

1124: t \in \{ \tau^u \} \text{ else } 0$.

1125: 		We can then define :

1126: 		\[

1127: 		d^t(u,v) = \sum_{\tau_k \leq t \text{ in } \{\tau^u\} \cup

1128: \{\tau^v\} } \left( |u(\tau_k) - v(\tau_k)|\right) \text{ with } \tau_k \in

1129: \{\tau^u\}\cup \{\tau^v\}

1130: 		\]

1131: 		This one gives us a very precise distance between discrete

1132: spatiotemporal patterns but is too sensitive.

1133: 		\begin{rem}

1134: 			Something important to notice is the fact that the

1135: distances is defined at a time $t$, and may have access to the history of the

1136: signal. The property can be varying with the time, since the distance is.

1137: 		\end{rem}

1138: 

1139: 	\item We can define sensitivity delay $t_\varepsilon$ also seen as

1140: refractory period, and use a kernel \cite{Gerstner2002} :

1141: 		\[ u(t) = \sum_{\tau^u_k \text{ in } \{\tau^u\}} K(t,\tau^u_k,

1142: \{\tau^u\})\]

1143: 		\begin{itemize}

1144: 			\item  $K(.)= \delta(t - \tau^u_k)$ which gives a

1145: formalization of the above vision,

1146: 			\item $K(.) = \delta(t - t_i^n)$ The spike train itself

1147: $\sum_{t_i^n \in {\cal F}_i} \delta(t - t_i^n)$

1148: 			\item $K(.) = W_{t_i^{n-1}..t_i^n}(t) \,

1149: \frac{t_\epsilon}{t_i^n - t_i^{n-1}}$  The normalized instantaneous frequency

1150: ({\scriptsize $t_i^n$ being predicted after $t_i^{n-1}$})

1151: 			\item $K(.) = \mbox{max}\left(0, \frac{t_\epsilon - |t -

1152: t_i^n|}{t_\epsilon}\right)$  A non-causal measure of the instantaneous spike

1153: density. \includegraphics[height=1cm]{figs/triangles}

1154: 			\item $K(.) = (1-e^\frac{-t_\epsilon}{\tau}) Y(t -

1155: t_i^n) \, e^\frac{-(t - t_i^n)}{\tau}$  A $\tau$ time-constant, low-pass

1156: filtered causal measure of.

1157: 			\item $K(.) = \mbox{ if } i = 0 \mbox { then }

1158: \frac{t_\epsilon}{t_i^0 - t_\bullet} \mbox{ else } 0$  A representation of the

1159: 1st spike as in fast-brain mechanisms \cite{Thorpe2001}  with respect to a time

1160: reference $t_\bullet$.

1161: 		\end{itemize}

1162: 		and then we can define instantaneous distances with

1163: 		\[ d^t(u,v) = | u(t)-v(t)|\]

1164: 		or define a more interesting distance as seen  in the following,

1165: since thanks to the kernels we come back to real signal space.

1166: 	\item With the vision of spike set, we can consider the signals as

1167: binary words (1 if the time is a spiking time, 0 otherwise), and use the usual

1168: binary infinite word distances.

1169: 		This kind of norms can be useful if we are looking at some

1170: binary coded properties, or to do some binary computation \cite{Carnell2005}.

1171: Binary codes or barcodes \cite{Jin2004}.

1172: 	\item Some more statistical distances could be of interest, taking in

1173: account the probability of spiking with respect to the history by some Hebbian

1174: rule, for this we can be inspired by \cite{Aviel2003,Hayon2005,Toyoizumi2005}

1175: 	\item Phase synchrony measure giving also a pseudo distance inspired by

1176: \cite{Pinsky1995}

1177: \end{itemize}

1178: 

1179: Distances in real signal space : $\mathbf{R}^\mathbf{R}$, we can come back to

1180: the usual functional norms, with $T$ the sensitivity window :

1181: \[ \displaystyle N^t_2(u) = \int_{t-T}^{t} u(\tau)^2\, d\tau \qquad \text{ or }

1182: N^t_1(u) = \int_{t-T}^{t} |u(\tau)|\, d\tau \]

1183: 

1184: or the convoluted of these ones, with $\mu \geq 0$

1185: \begin{equation*} N^t_2(u) = \displaystyle\int_{t-T}^{t} u(\tau)^2 \mu(\tau-t)\,

1186: d\tau \qquad \text{ or } N^t_1(u) = \displaystyle\int_{t-T}^{t} |u(\tau)|

1187: \mu(\tau-t) \, d\tau \end{equation*}

1188: And uses the norms as distances if we want with the usual : $d^t(u,v) =

1189: N^t(u-v)$\\

1190: We can list some properties of these norms :

1191: \begin{itemize}

1192: 	\item growing with $T$ so the continuity with $T$ implies the one with

1193: $T^{\prime} \geq T$

1194: 	\item $\mu$ will be very important to describe the system's sensitivity

1195: 	\item if we consider $u$ to be bounded it is sufficient to have $\mu

1196: \sim\frac{1}{\tau^2} $ to allow to have an infinite window $T$

1197: \end{itemize}

1198: The norms are the generic case of the norm defined earlier Equation

1199: \ref{eq:norm_alpha}.

1200: 

1201: 

1202: \subsection{Contraction condition with help of symmetry}

1203: \label{sec:permutdetails}

1204: We have seen that with the contraction condition verified, we can

1205: control the symmetry of the dynamics to control the

1206: solution. Actually, the symmetry of the system can also help us to

1207: simplify the proof of contraction itself.

1208: 

1209: \subsubsection{Circulant functions, symmetries and contraction conditions}

1210: 

1211: \begin{lem}

1212: 	A $\sigma$-equivariance with $\sigma$ a cycle of the size of the space

1213: is something well known: $M$ $\sigma$-equivariant iff $M$ is circulant.\\

1214: 	\label{lem:circulant}

1215: \end{lem}

1216: \begin{proof}

1217: 	$M$ a $m \times m$ matrix is circulant iff $M$ seen as a quadratic form

1218: $M(x,x) = x^T M x$ has the property $M(x, \sigma x) = M(\sigma^{-1}x,x)$ with

1219: $\sigma$ a m-cycle.

1220: 	On the other side we have $\sigma^{-1} = \sigma^T$ and $M(x, \sigma x) =

1221:  x^T M \sigma x = x^T \sigma M x = ( \sigma^{-1}x )^T M x$ by

1222: $\sigma$-equivariance.

1223: \end{proof}

1224: \begin{rem}

1225: 	A matrix $C$ is circulant iff it can be written as :

1226: 	\[

1227: 	C =

1228: 	\begin{pmatrix}

1229: 		c(0) & c(1) & \dots & c(n-1) \\

1230: 		c(n-1) & c(0) & c(1) & \dots \\

1231: 		\ddots & \ddots & \ddots & \ddots \\

1232: 		\ddots & \ddots & \ddots & \ddots

1233: 	\end{pmatrix}

1234: 	\]

1235: 	The eigenvalue of the symmetric part are also simple :

1236: 	\[

1237: 	\mu_i = \sum_{k=0}^{n-1} c(k) cos(2ik\pi/n)

1238: 	\]

1239: \end{rem}

1240: 

1241: 

1242: We will consider a $\sigma$-equivariant (circulant cf \ref{lem:circulant})

1243: system $\fp$ in space $E^\prime$ of size $n^\prime$.We define $\displaystyle

1244: \lambda_j^\prime  = \sum_{k=0}^{n^\prime-1} \dfp{k}{0} cos(2jk\pi/n^\prime)$

1245: \begin{rem}

1246: 	The $\lambda^\prime$ are defined using the first composant of $\fp$ but

1247: thanks to the circulant property, it could be equivalently done using any other

1248: composant

1249: \end{rem}

1250: 

1251: Then we can prove two interesting lemmas :

1252: 

1253: \begin{lem}

1254: 	\label{lem:contcirc}

1255: 	\begin{equation}

1256: 		\label{eq:contcirc}

1257: 		\fp \text{ contracting with identity metric}  \iff \forall j \in

1258: [0..\np-1] ,\,  \lambda_j^\prime < 0

1259: 	\end{equation}

1260: \end{lem}

1261: \begin{lem}

1262: 	\label{lem:vallfrom}

1263: 	\begin{equation}

1264: 		\label{eq:vallfrom}

1265: 		V_\sigma^{\prime} \frac{\partial \fp}{ \partial x}

1266: V_\sigma^{\prime T} < 0 \iff \forall j \in [1..\np-1] ,\,  \lambda_j^\prime < 0

1267: 	\end{equation}

1268: \end{lem}

1269: \begin{rem}

1270: 	We have a condition of size $\np -1$ as it should be.

1271: \end{rem}

1272: \begin{rem}

1273: 	$V_{all}^\prime = V_\sigma^\prime$

1274: \end{rem}

1275: With this two lemmas we have moreover a simple link between the contraction and

1276: the contraction toward the subspace of synchrony.

1277: 

1278: 

1279: \subsubsection{Discrete symmetric system}

1280: 

1281: 

1282: We consider $f$ a $\gamma$-equivariant system :

1283: \begin{equation}

1284: 	\label{eq:hypo_gamma}

1285: 	f\gamma=\gamma f

1286: \end{equation}

1287: 

1288: We will consider that $\gamma$ is a permutation and use notation from  Section

1289: \ref{sec:discrete_operator}. $l_i$ is the size of $\sigma_i $, $E_{I_q}$ correspond to the space invariant by the action of $\g$ which is of dimension $q=n-\sum_i l_i$ since the cycles $\sigma_i$ are chosen non-trivial.

1290: 

1291: Such a $\gamma$-equivariant system will converge exponentially to a solution

1292: $x=\gamma x$ if it has one of the contraction property. We will look at the

1293: strongest one : contraction toward $\Mg =\{x=\gamma x\}$, and precisely at the

1294: sufficient condition $\Vg \frac{\partial f}{ \partial x} \Vg^{T} < 0$ with $\Vg

1295: x = 0 \iff x\in \Mg$.

1296: \begin{thm}

1297: 	\[

1298: 	\Vg \frac{\partial f}{ \partial x} \Vg^{T} < 0 \iff \forall i\in [0..p]

1299: \  \forall j \in \left[ 1 \dots l_i-1 \right], \ \lambda_{j,i} < 0

1300: 	\]

1301: 	with

1302: 	\begin{align*}

1303: 		\lambda_{j,i} &= \sum_{k=0}^{l_i-1} \partial_{k_i} f_{0_i}

1304: cos(2jk\pi/l_i) &

1305: 	\end{align*}

1306: 	$k_i$ being the indice of the $k$th element in the $i$th subspace

1307: (modulo $l_i$ to stay in the $i$th subspace). For example for $i=1$ we have

1308: $k_1= l_1 + k$, $(-2)_1 = l_1 + l_2 -2$ and $f_{0_i}=f_{l_1}$

1309: \end{thm}

1310: \begin{proof}

1311: \begin{rem}

1312: 	\label{rem:poly_multiuni}

1313: 	$x=\gamma x$ is equivalent to the conjunction $\forall i \in [0..p] \

1314: x|_{E_{\sigma_i}}=\sigma_i \left( x|_{E_{\sigma_i}} \right) $. But

1315: $x|_{E_{\sigma_i}}=\sigma_i \left( x|_{E_{\sigma_i}} \right) $ means that $x$ is

1316: in synchrony inside each $E_{\sigma_i}$ : We want to prove a polysynchrony which

1317: can be done by proving the synchrony inside each group.

1318: \end{rem}

1319: We can relate this remark to

1320: \[

1321: \Vg \frac{\partial f}{ \partial x} \Vg^{T} < 0 \iff \forall i \  V_{\sigma_i}

1322: \left. \frac{\partial f}{ \partial x} \right| _{E_{\sigma_i}} {V_{\sigma_i}

1323: }^{T} < 0

1324: \]

1325: 

1326: Which leads us to us the result with Lemma \ref{lem:vallfrom}.

1327: \end{proof}

1328: 

1329: \subsection{Proofs of Theorems, Lemma etc}

1330: 

1331: \subsubsection{Theorem \ref{thm:continuity_contracting}}

1332: \begin{proof}

1333: \label{proof:thm_continuity_contracting}

1334: 	We have the hypothesis:

1335: 	\begin{align*}

1336: 		\forall \tau \geq t_0,\ \|u(\tau)\| \leq M^\prime,\, \|s(\tau)\|

1337: \leq M^\prime\\

1338: 		\NatO(\up - u) \leq \eta^\prime

1339: 	\end{align*}

1340: 	Using the uniform continuity,

1341: 	\begin{align}

1342: 		\forall \tau \geq t_0,\ \|h(\tau)\| \leq M \nonumber\\

1343: 		\Na(h) \leq \eta \label{hypothese1}

1344: 	\end{align}

1345: 	We have \[\Na(r) = \int_{-\infty}^{t}e^{\alpha(\tau-t)}\|r(\tau)\|\,d\tau\]

1346: 	However from the robustness (\ref{eq:m_R})\[ \forall \tau  \ \|r(\tau)\|

1347: \leq \int_{-\infty}^{\tau} e^{\lambda(y-\tau)}\|h(y)\|\,dy\]

1348: 	We can apply Lemma \ref{lem:pre_boundary} with $t_0=-\infty$ and define

1349: $t_1$ the moment of the saturation

1350: 	\begin{equation}

1351: 		e^{\lambda(t_1-t)} = \frac{\eta\lambda}{M}

1352: 		\label{eq:t_1_relation}

1353: 	\end{equation}

1354: 	\begin{align}

1355: 		\Na(r)

1356: 		&\leq \lim_{\,t_0 \to

1357: -\infty}\int_{t_0}^{t_1}e^{\alpha(\tau-t)}\frac{M}{\alpha}\left( 1 -

1358: e^{\lambda(t_0-\tau)} \right)\,d\tau + \int_{t_1}^{t} e^{\alpha(\tau-t)} \eta

1359: e^{\lambda (t-\tau)}\,d\tau \notag \\

1360: 		\intertext{which by calculus using the relation Equation

1361: \ref{eq:t_1_relation}}

1362: 		&\leq \frac{M}{\lambda\alpha}\left( \frac{\eta\lambda}{M}

1363: \right)^{\frac{\alpha}{\lambda}} + \frac{\eta}{\alpha-\lambda}\left(

1364: 1-\left(\frac{\eta\lambda}{M} \right)^{\frac{\alpha-\lambda}{\lambda}}\right)

1365: 		\label{eq:majoration_continuite_infiny}

1366: 	\end{align}

1367: 

1368: 	From (\ref{eq:majoration_continuite_infiny}) we have the continuity,

1369: since all the powers of $\eta$ are positives.

1370: \end{proof}

1371: 

1372: Let's consider some cases :

1373: \begin{itemize}

1374: 	\item if $\alpha \leq \lambda$ We first have to say that we still have a

1375: positive term, since the exponent change also its sign, making $\left(

1376: 1-\left(\frac{\eta\lambda}{M} \right)^{\frac{\alpha-\lambda}{\lambda}}\right)

1377: \leq 0$ then the fact that $M$ is finite is important, otherwise this term will

1378: go to $+ \infty$ and we can have in extreme cases some numerical surprises (even

1379: if we have continuity).

1380: 	\item if $\alpha = \lambda$ then by continuity we get the limit : $

1381: \frac{\eta}{\alpha} + \frac{\eta}{\alpha}ln\left( \frac{M}{\eta\alpha}\right) $

1382: so $M$ should still be bounded to allow us to use this limit..

1383: 	\item if $\alpha \geq \lambda$ then we can get rid of $M$ and perhaps in

1384: first approximation, just keep the main term : $\frac{\eta}{\alpha-\lambda}$

1385: \end{itemize}

1386: 

1387: \begin{rem}

1388: 	If $t_1$ doesn't exist ( i.e. $ \frac{\eta\lambda}{M} \geq 1$ ) it first

1389: means that we did not took $\eta$ small but the calculus gives just the first

1390: term of (\ref{eq:majoration_continuite_infiny}) which is still good.

1391: \end{rem}

1392: \begin{rem}

1393: 	We can also instead of using some case based calculus over the $\lambda$

1394: and $\alpha$ use the lemma \ref{lem:change_alpha} to have a pseudo equivalence

1395: of all of these norms.

1396: \end{rem}

1397: \begin{rem}

1398: 	We should see that in the preceding proof, $\lambda$ is taken as the

1399: contraction rate of the system, but we can use any $\acute{\lambda} \leq

1400: \lambda$ since (\ref{robustness_contraction}) will still be true.

1401: \end{rem}

1402: 

1403: \begin{lem}

1404: 	\label{lem:pre_boundary}

1405: 	a boundary on the norm before $t$:

1406: 	\begin{equation}

1407: 		\label{eq:first_property}

1408: 		\forall t \geq \tau \geq t_0,\

1409: \int_{t_0}^{\tau}e^{\beta(y-\tau)}H(y)\,dy \leq min\left( \eta e^{\beta

1410: (t-\tau)}\ ,\ \frac{M}{\beta}(1 - e^{\beta (t_0-\tau)}) \right)

1411: 	\end{equation}

1412: 	Remark : The inequality is an equality for

1413: 	\[

1414: 	H(y) =

1415: 	\begin{cases}

1416: 		M & \text{ if $y \leq t_1$,}\\

1417: 		0 & \text{ if $t_1 \leq y \leq t$ }

1418: 	\end{cases}

1419: 	\]

1420: 	with $t_1$ the moment of saturation if it exists :

1421: 	\[

1422: 	\frac{M}{\beta}(e^{\beta(t_1 - t)} - e^{\beta(t_0 - t)}) = \eta

1423: 	\]

1424: \end{lem}

1425: \begin{proof}

1426: 	\begin{align*}

1427: 		\forall \tau \geq t_0,

1428: \int_{t_0}^{\tau}e^{\alpha(y-\tau)}H(y)\,dy

1429: 		&= e^{\alpha(t-\tau)} \int_{t_0}^{\tau}e^{\alpha(y-t)}H(y)\,dy\\

1430: 		&\stackrel{\text{\tiny def}}{=} e^{\alpha(t-\tau)} h(\tau)

1431: 	\end{align*}

1432: 	however

1433: 	$h(t_0)=0,\quad h(t)=\eta,\quad \dot{h}(\tau)=e^{\alpha(\tau-t)}H(y)$

1434: 	\begin{alignat*}{5}

1435: 		\text{from (\ref{hypothese1}) } \quad& 0 &\quad& \leq &\quad&

1436: \dot{h}(\tau) &\quad& \leq &\quad& M e^{\alpha(\tau-t)} \\

1437: 		& 0 && \leq && h(\tau) && \leq && \min\left( \eta \ ,\

1438: \int_{t_0}^{\tau}e^{\alpha(y-t)}M\,dy \right) \\

1439: 		&0 && \leq && h(\tau) && \leq && \min\left( \eta \ ,\ \frac{M

1440: e^{-\alpha t}}{\alpha}(e^{\alpha\tau} - e^{\alpha t_0}) \right)

1441: 	\end{alignat*}

1442: \end{proof}

1443: 

1444: 

1445: \begin{lem}

1446: 	\label{lem:change_alpha}

1447: 	The possibility to change the $\alpha$ of the norm keeping a boundary:

1448: 	\begin{eqnarray}

1449: 		\label{eq:second_property}

1450: 		\Nb(d) \leq q(\eta)\\

1451: 		\text{ with } &q(\eta) \leq \eta &\text{ if } \alpha \leq \beta

1452: \nonumber \\

1453: 		\text{ and } &q(\eta) \leq \frac{M}{\beta}\left( \frac{\eta

1454: \alpha}{M} \right)^{\frac{\beta}{\alpha}} &\text{ if } \beta \leq \alpha

1455: \nonumber

1456: 	\end{eqnarray}

1457: 	Remark : it is also true that we keep the boundary with $t_0 \neq

1458: -\infty$ but the result is less interesting.

1459: \end{lem}

1460: \begin{proof}

1461: 	With the notation of lemma \ref{lem:pre_boundary} :

1462: 	\begin{align*}

1463: 		\Nb(d)

1464: 		&= \int_{t_0}^{t}e^{\beta(\tau-t)}D(\tau)\,d\tau \\

1465: 		&= e^{(\alpha-\beta)t}

1466: \int_{t_0}^{t}\dot{h}(\tau)e^{(\beta-\alpha)\tau}\,d\tau \\*

1467: 		\intertext{integrating by parts and taking the limit

1468: $t_0=-\infty$ when possible :}

1469: 		&= h(t) + (\alpha -\beta)

1470: \int_{t_0}^{t}h(\tau)e^{(\beta-\alpha)(\tau-t)}\,d\tau\\*

1471: 		\intertext{for $\alpha \geq \lambda$ and using the property of

1472: $h(\tau)$ seen in lemma \ref{lem:pre_boundary} }

1473: 		&\leq \eta + (\alpha - \beta) \left[ \int_{t_0}^{t_1}\frac{M

1474: e^{-\alpha t}}{\alpha}(e^{\alpha\tau} - e^{\alpha

1475: t_0})e^{(\beta-\alpha)(\tau-t)} + \int_{t_1}^{t}\eta

1476: e^{(\beta-\alpha)(\tau-t)}\right]\\

1477: 		\intertext{which gives us, with $t_0=-\infty$ ( which works

1478: without any new hypothesis)}

1479: 		&\leq  \eta + (\alpha - \beta) \left[ \frac{M

1480: e^{\beta(t_1-t)}}{\alpha\beta} + \frac{\eta}{\beta - \alpha}(1 -

1481: e^{(\beta-\alpha)(t_1-t)}) \right]\\

1482: 		\intertext{ and using the relation \eqref{eq:t_1_relation} }

1483: 		&\leq \frac{M}{\beta}\left( \frac{\eta\alpha}{M}

1484: \right)^{\frac{\beta}{\alpha}}

1485: 	\end{align*}

1486: \end{proof}

1487: 

1488: 

1489: \subsection{Proof of Lemma \ref{lem:vallfrom}}

1490: 

1491: 

1492: 

1493: \begin{proof}

1494: 

1495: 	A natural projector to the subspace of synchrony is $W$:

1496: 	\begin{align*}

1497: 		\begin{aligned}

1498: 			W =  I - \sigma =

1499: 			\begin{pmatrix}

1500: 				1 & -1 & 0 & \dots & 0 \\

1501: 				0 & \ddots & \ddots & \ddots & \vdots \\

1502: 				\vdots & \ddots & \ddots & \ddots & 0 \\

1503: 				0 & \ddots & \ddots & \ddots & -1\\

1504: 				-1 & 0 & \dots & 0 & 1

1505: 			\end{pmatrix}\\

1506: 		\end{aligned}

1507: 	\end{align*}

1508: 

1509: 	To use our natural projector, we first have to remark that it

1510: 	is not really a projector to the orthogonal space. Such a projector

1511: 	can be obtained by removing the redundancy inside $W$, because

1512: 	it is a system of dimension $n$ but represent an hyperspace

1513: 	of dimension $n-1$.

1514: 

1515: 

1516: 	Then this noticed, instead of looking for the condition $\Vp

1517: \frac{\partial f}{ \partial x} \Vpt < 0$ we will look for the equivalent

1518: condition $W \frac{\partial f}{ \partial x} W^T $ strictly negative except for

1519: one null eigenvalue.

1520: 

1521: 

1522: 	\begin{align*}

1523: 		A &= W \frac{\partial \fp}{ \partial x} W  \\

1524: 		&= \left( \dfp{s}{r} - \dfp{s+1}{r} - (\dfp{s}{r+1} -

1525: \dfp{s+1}{r+1})  \right)_{(r,s)\in [0..\np-1]^2} \\

1526: 		&= \left( 2 \dfp{s}{r} - \dfp{s-1}{r} - \dfp{s+1}{r}

1527: \right)_{(r,s)\in [0..\np-1]^2} \text{ using that $\partial \fp$ is circulant}

1528: 	\end{align*}

1529: 

1530: 	\begin{rem}

1531: 		All indices are modulo $\np$.

1532: 	\end{rem}

1533: 	\begin{rem}

1534: 		$W$ is circulant.

1535: 	\end{rem}

1536: 

1537: 	Since $A$ is circulant (product of circulant matrix), it is defined by

1538: 	\[

1539: 	a(k) = A_{0,k} = 2 \dfp{k}{0} - \dfp{k-1}{0} - \dfp{k+1}{0}

1540: 	\]

1541: 	We are interested in $C$ its symmetric part. It is also a circulant

1542: matrix, so defined by $c(k) \ k\in[0..l_i-1]$ :

1543: 	\[

1544: 	c(k) = \frac{a(k)+a(-k)}{2} = \frac{1}{2}\left(2 \dfp{k}{0}  -

1545: \dfp{k-1}{0} - \dfp{k+1}{0} + 2 \dfp{-k}{0} - \dfp{-(k-1)}{0} - \dfp{-(k+1)}{0}

1546: \right)

1547: 	\]

1548: 	One of the interest of circulant matrix is that we know their eigenvectors

1549: 	and associated eigenvalues : taking

1550: 	$\wp_j = e^{\mathbf{i} \frac{j 2 \pi}{\np}}$ one of the $\np$th root of 1

1551: 	( ${\wp_j}^{\np}=1$), we construct the eigenvector

1552: 	$\displaystyle v_j=( 1, \wp_j ,{\wp_j}^2, \dots, {\wp_j}^{\np-1})$

1553: 	associated to the eigenvalue

1554: 	$\displaystyle \mu_j^\prime= \sum_{k=0}^{\np-1} c(k){\wp_j}^k$\\

1555: 	\begin{rem}

1556: 		Since we took $C$ symmetric ($c(k) = c(-k )$), the eigenvalues

1557: $\mu_j^\prime$ are all real as one could remark regrouping $ c(k){\wp_j}^k +

1558: c(-k){\wp_j}^{-k} = c(k)cos(2jk\pi/\np)$.

1559: 	\end{rem}

1560: 

1561: 	With some simple regrouping  :

1562: 

1563: 	\begin{align*}

1564: 		\mu_j^\prime &= \sum_{k=0}^{\np-1} c(k){\wp_j}^k = 2 (1 -

1565: ({\wp_j} + {\wp_j}^{-1})/2 ) \lambda_j^\prime = 2 (1 - cos(2j\pi/l_0))

1566: \lambda_j^\prime \\

1567: 		\text{with } \lambda_j^\prime &= \sum_{k=0}^{\np-1} \partial_{k}

1568: f_0 ({\wp_j}^k + {\wp_j}^{-k})/2 = \sum_{k=0}^{\np-1} \partial_{k} f_0

1569: cos(2jk\pi/\np)

1570: 	\end{align*}

1571: 

1572: 	And :

1573: 	\begin{align*}

1574: 		&\mu_0^\prime = 0\\

1575: 		\forall j \in [1..\np-1],\qquad & sign(\lambda_j^\prime) = sign

1576: (\mu_j^\prime)

1577: 	\end{align*}

1578: 

1579: 	Since $W$ is of dimension $\np$, but represent a $\np-1$ dimensional

1580: space, corresponding to $\mu_0^\prime$. Then by virtue of Theorem 1 of

1581: \cite{Pham2006} the contraction to the subspace of synchrony $E^\prime_{all}$ is

1582: ensured with $\forall j \in [1..\np-1] ,\, \mu_j^\prime < 0$ or equivalently

1583: $\forall j \in [1..\np-1] ,\, \lambda_j^\prime < 0$

1584: \end{proof}

1585: 

1586: 

1587: \subsection{Contraction toward $\Mall$ using symmetry}

1588: \subsubsection{Proof of Lemma \ref{lem:Mall}}

1589: Let's recall the definition of a partial isometry : it is an isometry from the orthogonal of its kernel to its image.

1590: \begin{proof}

1591: \label{proof:lemMall}

1592: We take $VV^T = I_{E_{\M^\bot}}$, $V_2V_2^T = I_{E_{\Mp^\bot}}$, and $\forall x \in E_{\M^\bot},\ x^T\Theta V f V^T \Theta^{-1}x<0$.

1593: 

1594: Note first that $V_2V^T\Theta^T\Theta V V_2^T > {\bf 0}$. Indeed, $\Theta$ is invertible and $\forall y \in E_{\Mp^\bot},\ VV_2^Ty = 0$ implies $\ V_2^Ty = 0$ (since $\Mp^\bot \subset \M^\bot$), which in turn implies $\ y = 0$ (by definition).

1595: 

1596: Next note that \[ \exists \Theta_2 \in GL(E_{\Mp^\bot}),\ \forall y \in E_{\Mp},\ \exists x \in E_{\M},\ V^T \Theta^{-1}x = V_2^T \Theta_2^{-1} y \text{ and } V^T \Theta^Tx = V_2^T \Theta_2^T y\] Indeed, since $VV^T=I_{E_{\M^\bot}}$ this is equivalent to :

1597: 	\begin{align*}

1598: 		\exists \Theta_2 \in GL_m&,\ \forall y \in E_{\Mp^\bot},\ &\Theta V V_2^T\Theta_2^{-1} y &= (\Theta^T)^{-1} V V_2^T \Theta_2^Ty \\

1599: 		\exists \Theta_2 \in GL_m&,\ &V_2V^T\Theta^T\Theta V V_2^T  &= \Theta_2^T\Theta_2

1600: 	\end{align*}

1601: where $m$ is the dimension if the subspace $E_{\Mp^\bot}$, $\Theta_2 $ exists because $V_2 V^T \Theta^T \Theta V V_2^T > {\bf 0}$.

1602: 

1603: Finally, by unitary freedom of square roots for symmetric positive operators there exists a partial isometry $U$ such that $\Theta_2 = U \Theta V V_2^T$. Thus, \[\Theta_2 V_2 f V_2^T \Theta_2^{-1} = U \Theta V V_2^T V_2 f V_2^T (U \Theta V V_2^T)^{-1} = U \Theta V f V^T \Theta^{-1} U^T<0 \] where the last expression is negative definite from the hypothesis.

1604: 

1605: As is $\Theta_2$, $U$ is defined up to isometries. We can thus take any $U$ having the sufficient following properties. $U^T$ is basically the partial isometry embedding $E_{\Mp}$ in $Im(\Theta V V_2^T E_{\Mp})$ which is of the same dimension but in a bigger space : $U^TU$ projects $E$ onto $Im(\Theta V V_2^T E_{\Mp})$ and $UU^T=I_{E_{\Mp}}$. \end{proof}

1606: 

1607: \subsubsection{Proof of Lemma \ref{lem:Mall2}}

1608: \begin{proof}

1609: \label{proof:lemMall2}

1610: 	Using a similar construction as in proof \ref{proof:lemMall}, with $m$

1611: the dimension of the subspace $E_{\M^\bot}$, we need

1612: 	\begin{align*}

1613: 		\exists \Theta_2 \in GL_m,\quad \forall y \in E_{\M^\bot},

1614: \Theta V V_2^T\Theta_2^{-1} y &= (\Theta^T)^{-1} V V_2^T \Theta_2^Ty

1615: 	\end{align*}

1616: 	which is equivalent to :

1617: 	\begin{align*}

1618: 		\exists \Theta_2 \in GL_m,\quad (V_2^T)^{-1}V^T\Theta^T\Theta V

1619: V_2^T  &= \Theta_2^T\Theta_2 \\

1620: 		(T^T)^{-1}\Theta^T\Theta T^T  &= \Theta_2^T\Theta_2

1621: 	\end{align*}

1622: 	things works with $T$ unitary, but if not unitary there are few chances

1623: for i to work.

1624: \end{proof}

1625: 

1626: \bibliographystyle{unsrt}

1627: 

1628: \bibliography{biblio}

1629: 

1630: \end{document}

1631: 

1632: