1: % modified from journal submission to use standard article style, and include all sections and bibliography in this file

2: \documentclass{article}

3: \usepackage{epsfig}

4: 

5: \begin{document}

6: 

7: \long\def\comment#1{}

8: \newcommand{\etc}{{\em etc}}

9: \newcommand{\eg}{{\em e.g.}}

10: \newcommand{\ie}{{\em i.e.}}

11: \newcommand{\ea}{{\em et al.}}

12: \newcommand{\noi}{\noindent}

13: 

14: \newenvironment{itm}{\begin{itemize}\setlength{\itemsep}{-0.02in plus 0in}}{\end{itemize}}

15: 

16: 

17: \title{Mathematical Analysis of Multi-Agent Systems\thanks{The research reported here was supported

18: in part by the Defense Advanced Research Projects Agency (DARPA) under contract

19: number F30602-00-2-0573, in part by the National Science Foundation under Grant

20: No. 0074790, and by the ISI/ISD Research Fund Award.

21: The views and conclusions contained herein are those of the authors

22: and should not be interpreted as necessarily representing the official

23: policies or endorsements, either expressed or implied, of any of the

24: above organizations or any person connected with them.

25: }}

26: \author{Kristina Lerman~~~~Aram Galstyan\\USC Information Sciences Institute \and Tad

27: Hogg\\Hewlett-Packard Labs}

28: 

29: \comment{\author{Kristina \surname{Lerman}$^1$

30: \email{lerman@isi.edu}}

31: \author{Aram \surname{Galstyan}$^1$ \email{galstyan@isi.edu}}

32: \author{Tad \surname{Hogg}$^2$ \email{tad.hogg@hp.com}}

33: \runningauthor{Kristina Lerman, Aram Galstyan and Tad Hogg}

34: \runningtitle{Mathematical Analysis of Multi-Agent Systems}

35: \institute{1. USC Information Sciences Institute\\

36: 2. Hewlett-Packard Laboratories} }

37: \date{}

38: \maketitle

39: 

40: \begin{abstract}

41: We review existing approaches to mathematical modeling and

42: analysis of multi-agent systems in which complex collective

43: behavior arises out of local interactions between many simple

44: agents. Though the behavior of an individual agent can be

45: considered to be stochastic and unpredictable, the collective

46: behavior of such systems can have a simple probabilistic

47: description. We show that a class of mathematical models that

48: describe the dynamics of collective behavior of multi-agent

49: systems can be written down from the details of the individual

50: agent controller. The models are valid for Markov or memoryless

51: agents, in which each agents future state depends only on its

52: present state and not any of the past states. We illustrate the

53: approach by analyzing in detail applications from the robotics

54: domain: collaboration and foraging in groups of robots.

55: 

56: \vspace{1cm} \noindent {\bf Keywords:} multi-agent systems,

57: robotics, mathematical analysis, stochastic systems

58: \end{abstract}

59: 

60: 

61: 

62: %****************************************************

63: 

64: \newpage

65: 

66: %\input{intro}

67: \section{Introduction}

68: \label{sec-Intro}

69: 

70: Distributed systems composed of large numbers of relatively simple

71: autonomous agents are receiving increasing amount of attention in

72: the Artificial Intelligence (AI), robotics and networking

73: communities. Unlike complex deliberative agents, the subject of

74: much of AI research of the past two decades, simple agents have

75: no, or limited, capacity to reason about data, plan action or

76: negotiate with other agents. Although individual agents are far

77: less powerful than traditional deliberative agents, distributed

78: multi-agent systems based on such simple agents offer several

79: advantages over traditional approaches: specifically robustness,

80: flexibility, and scalability. Simple agents are less likely to

81: fail than more complex ones. If they do fail, they can be pulled

82: out entirely or replaced without significantly impacting the

83: overall performance of the system. They are, therefore, tolerant

84: of agent error and failure. They are also highly scalable –--

85: increasing the number of agents or task size does not require

86: changes in the

87: %greatly affect the complexity of the

88: agent control programs nor compromise the performance of the

89: system. In systems using deliberative agents, on the other hand,

90: the high communications and computational costs required to

91: coordinate group behavior limit the size of the system to at most

92: a few dozen agents. Larger versions of such systems require

93: division into subgroups with limited and simplified interactions

94: between the groups~\cite{simon96}. In many cases, these

95: interacting subgroups can in turn be viewed abstractly as agents

96: following relatively simple protocols, as, for example, in

97: market-based approaches to multi-agent

98: systems~\cite{clearwater96}.

99: %Yet another advantage is that the

100: %simplicity of agent's interactions often makes such systems

101: %amenable to quantitative mathematical analysis.

102: 

103: There is no central controller directing agents' behavior, rather,

104: these multi-agent systems are {\em self-organizing}, meaning that

105: constructive {\em collective} ( macro\-scopic) behavior emerges

106: from {\em individual} (micro\-scopic) decisions agents make. In

107: most cases, these decisions are based on purely local information

108: (that comes from other agents as well as the environment).

109: Self-organization is ubiquitous in nature --- bacteria colonies,

110: amoebas and social insects such as ants, bees, wasps, termites,

111: among others --- display interesting manifestations of this

112: phenomenon. Indeed, in many of these systems while the individual

113: and its behavior appear simple to an outside observer, the

114: collective behavior of the colony can often be quite complex. The

115: apparent success of these organisms has inspired computer

116: scientists and engineers to design algorithms and distributed

117: problem-solving systems modeled after them ({\em e.g.}, swarm

118: intelligence~\cite{Beni88,BDT99,SchoonderwoerdEtAl97} and

119: biologically-inspired

120: systems~\cite{Melhuish98,GolMat00,IMB2001,bojinov02}). Moreover,

121: current developments in micromechanical systems

122: (MEMS)~\cite{berlin97} and proposals for coordinated behavior

123: among microscopic robots~\cite{abelson99,freitas99} will require

124: agent control programs capable of scaling to extremely large

125: numbers of agents. Such agents will encounter various microscopic

126: environments, some with counterintuitive

127: properties~\cite{purcell77}, making it difficult to design

128: appropriate deliberative control programs. Furthermore, at least

129: in their initial development, such machines are likely to face

130: severe power, computational and communication limitations and

131: hence require a focus on collective behavior from computationally

132: simple agents.

133: 

134: 

135: \comment{ Two paradigms dominate the design of multi-agent

136: systems. The first, what we will call the traditional paradigm, is

137: based on deliberative agents, while the second, the swarm

138: paradigm, is based on simple agents and distributed control. In

139: the past two decades, researchers in the Artificial Intelligence

140: and related communities have, for the most part, operated within

141: the traditional paradigm. They focused on making the individual

142: agents, be they software agents or robots, smarter and more

143: complex by giving them the ability to reason, negotiate and plan

144: action. In these deliberative systems complex tasks can be done

145: either individually or collectively. If collective action is

146: required to complete some task, a central controller is sometimes

147: used to coordinate group behavior. The controller keeps track of

148: the capabilities and state of each agent, decides which agents are

149: best suited for a specific task, assigns them to the agents and

150: coordinates communication between them (see, {\em e.g.}, Electric

151: Elves \cite{Elves}, RETSINA \cite{RETSINA}). Deliberative agents

152: are also capable of collective action in the absence of central

153: control; however, in these cases agents require knowledge about

154: capabilities and states of other agents with which they may

155: interact. For instance, a multi-agent system may break into a

156: number of coalitions containing several agents, with each

157: coalition being able to accomplish some task more effectively than

158: a single agent can. One approach to coalition formation is for

159: agents to compute the optimal coalition structure

160: \cite{SandholmShehory:99} and form coalitions based on this

161: calculation.  Acquiring the knowledge necessary to coordinate

162: collective behavior may be expensive and impractical, especially

163: for systems containing many agents.

164: 

165: Swarm Intelligence~\cite{Beni88,BenWan89,BDT99} represents an

166: alternative approach to the design of multi-agent systems. Swarms

167: are composed of many simple agents. There is no central controller

168: directing the behavior of the swarm, rather, these systems are

169: {\em self-organizing}, meaning that constructive collective ({\em

170: macro\-scopic}) behavior emerges from local ({\em micro\-scopic})

171: interactions among agents and between agents and the environment.

172: Self-organization is ubiquitous in nature --- bacteria colonies,

173: amoebas and social insects such as ants, bees, wasps, termites,

174: among others --- are all examples of this phenomenon. Indeed, in

175: many of these systems while the individual and its behavior appear

176: simple to an outside observer, the collective behavior of the

177: colony can often be quite complex. One of the more fascinating

178: examples of self-organization is provided by {\em Dictyostelium

179: discoideum}, a species of soil amoebas. Under normal conditions of

180: abundant food supply these single-celled organisms are solitary

181: creatures. However, when food becomes scarce, the amoebas emit

182: chemical signals which cause the colony to aggregate, creating a

183: multicelled organism \cite{devreotes89dictyostelium,Loomis99} that

184: is sensitive to light and heat and can travel far more efficiently

185: than its component cells~\cite{NYTimes}. Next, the collections of

186: cells within the super-organism differentiate into two distinct

187: types --- stalks and spores --- which allows the colony to

188: reproduce by releasing spores which will lie dormant, waiting for

189: more favorable conditions. There are multiple examples of complex

190: collective behavior among social insects as well: trail formation

191: in ants, hive building by bees and mound construction by termites

192: are just few of the examples. The apparent success of these

193: organisms has inspired computer scientists and engineers to design

194: algorithms and distributed problem-solving systems modeled after

195: them \cite{BDT99,GolMat00,IMB2001,SchoonderwoerdEtAl97}.

196: 

197: 

198: Swarms offer several advantages over traditional systems based on

199: deliberative agents: specifically robustness, flexibility,

200: scalability, and suitability for analysis. Simple agents are less

201: likely to fail than more complex ones. If they do fail, they can

202: be pulled out entirely or replaced without significantly impacting

203: the overall performance of the system. They are, therefore,

204: tolerant of agent error and failure. Swarm-based systems are also

205: highly scalable –-- increasing the number of agents or task size

206: does not greatly affect system's performance. In systems using

207: deliberative agents, the high communications and computational

208: costs required to coordinate agent behavior limit the size of the

209: system to at most a few dozen agents. Finally, the simplicity of

210: agent's interactions with other agents make swarms amenable to

211: quantitative mathematical analysis. }

212: 

213: The main difficulty in designing multi-agent systems (MAS) with

214: desirable self-organized behavior is understanding the effect

215: individual agent characteristics have on the collective behavior

216: of the system. In the past, few analysis tools have been available

217: to researchers, and it is precisely the lack of such tools that

218: has been a chief impediment to the wider deployment of

219: biologically-inspired MAS. Another impediment has been the

220: difficultly of building hardware required for large-scale

221: experiments. Researchers had a choice of experiments with

222: relatively few agents or simulation for studying behavior of a

223: MAS. Experiments with real agents, {\em e.g.}, robots, allow them

224: to observe MAS under real conditions; however, experiments are

225: very costly and time consuming, and systematically varying

226: individual agent parameters to study their effect on the group

227: behavior is often impractical. Simulations, such as sensor-based

228: simulations of robots~\cite{StagePlayer,Michel98},  attempt to

229: realistically model the environment, the robots' imperfect sensing

230: of and interactions with it. Though simulations are much faster

231: and less costly than experiments, they suffer from many of the

232: same limitations, namely, they are tedious and the results are not

233: generalizable. Exhaustive scan of the entire parameter space is

234: often required to reach any conclusion. Moreover, simulations do

235: not scale well with the system size --- unless computation is

236: performed in parallel, the greater the number of agents, the

237: longer it takes to obtain results.

238: 

239: 

240: Mathematical modeling and analysis offer an alternative to the

241: time-consuming and costly experiments and simulations. Using

242: mathematical analysis we can study dynamics of multi-agent

243: systems, predict long term behavior of even very large systems,

244: gain insight into system design: {\em e.g.}, what parameters

245: determine group behavior and how individual agent characteristics

246: affect the MAS. Additionally, mathematical analysis may be used to

247: select parameters that optimize group performance, prevent

248: instabilities, \etc. Conversely, such analytical tools can also

249: provide design guidelines for agent programs. Specifically, these

250: tools rely on various simplifying approximations to the agent

251: behaviors. By deliberately designing agents to closely match these

252: approximations, the resulting collective behavior will correspond

253: to the analytic predictions. In this case, the tools will give a

254: good indication of how to optimize the design to achieve desired

255: behaviors. As one example, in market-based

256: systems~\cite{clearwater96}, designing agents to satisfy the

257: assumptions of purely competitive markets allows a simple analysis

258: of resulting behaviors in terms of market equilibria. Of course,

259: restricting the agent design choices to achieve a close

260: correspondence with analytic tools may limit the performance of

261: the system, but this may be a useful tradeoff to achieve a simpler

262: understanding of overall system behavior. Thus an important

263: question for applying mathematical analysis for multi-agent

264: systems is identifying situations in which simple analytic tools

265: give a useful approximation to the collective behavior.

266: 

267: Mathematical modeling and analysis of large-scale collective

268: behaviors is being increasingly  used outside of the physical

269: sciences where it has had much success. It has been applied to

270: ecology~\cite{ecology}, epidemiology~\cite{epidemiology}, social

271: dynamics~\cite{Helbing}, artificial

272: intelligence~\cite{hogg87PhysRep}, and behavior of

273: markets~\cite{hirshleifer78}, to name just a few disciplines.

274: 

275: In this paper we survey existing work on mathematical modeling and

276: analysis of artificial multi-agent systems. We also describe a

277: methodology for creating analytic models of collective behavior of

278: a multi-agent system. This type of analysis is valid for systems

279: composed of agents that obey the Markov property: where each

280: agent's future state depends only on its present state. Many of

281: the currently implemented multi-agent systems, specifically,

282: reactive and behavior-based robotics, satisfy this property.

283: \comment{Our approach is based on viewing these systems as

284: stochastic systems. We derive a class of mathematical models that

285: describe the collective dynamics of a multi-agent system. The

286: resulting models are often quite simple and may be easily written

287: down by analyzing the behavior of a single agent. Our approach is

288: general and applicable to many kinds of agent-based systems, such

289: as software agents~\cite{HubermanHogg88,Lerman00a},

290: robots~\cite{Lerman01a,Lerman02a} and sensors.} We illustrate the

291: approaches on robotics problems, comparing theoretical predictions

292: with experimental and simulations results whenever possible.

293: 

294: %\input{math}

295: \section{Mathematical Models}

296: \label{sec-Math}

297: 

298: A mathematical model is an idealized representation of a process.

299: Constructing a mathematical model proceeds incrementally. To be

300: useful, the model must explicitly include the salient details of

301: the process it describes so its predictions reasonably match the

302: actual behaviors of interest. On the other hand, the model should

303: also be as simple as possible, ideally to allow analytic treatment

304: and identification of qualitatively important relationships

305: between individual and system behaviors. The precise choice of

306: model involves a tradeoff between accuracy in describing reality

307: and ease of use in providing explanations of the behavior. In our

308: analysis we will strive to construct the simplest mathematical

309: model that captures all of the most important details of the

310: multi-agent system we are trying to describe.

311: 

312: 

313: %\subsection{Microscopic {\em vs} Macroscopic Models}

314: Mathematical models can generally be broken into two classes:

315: $mic\-ro\-sco\-pic$ and $macro\-sco\-pic$. Microscopic

316: descriptions treat the agent as the fundamental unit of the model.

317: There are several variations of the microscopic approach, as

318: described in the following section. Macroscopic models, on the

319: other hand, directly describe the collective behavior of a group

320: of agents. Such models are the focus of the present paper.

321: 

322: \subsection{Microscopic Models}

323: Microscopic models treat the individual agent as the fundamental

324: unit of the model. These models describe the agent's interactions

325: with other agents and the environment. Solving or simulating a

326: system composed of many such agents gives researchers an

327: understanding of the global behavior of the system.

328: 

329: \subsubsection{Equations of Motion Approach}

330: A common method used by physicists to study a system consisting of

331: multiple entities consists of writing down and solving equations

332: of motion for each entity. This approach has been adapted by some

333: to describe agent-based pattern forming systems~\cite{Bonabeau97},

334: including behaviors exhibited by colonies of biological organisms,

335: such as slime mold~\cite{Loomis99} and social

336: insects~\cite{Deneubourg92}. Of particular relevance to the agents

337: community is the work of Schweitzer and

338: coworkers~\cite{Schweitzer+al97,HelbingSchweitzer} on the active

339: walker model of trail formation by ants and people. Active walkers

340: are randomly walking agents that can influence the environment

341: (\eg, by depositing pheromone), in addition to being influenced by

342: it. Schweitzer \ea\ proposed a microscopic model of the

343: interaction of the ground potential (created by pheromone or

344: pedestrians' footprints) with the equations of motion of active

345: walkers.

346: 

347: For large systems, solving equations with many degrees of freedom

348: is often impractical. In some cases, it may be possible to derive

349: a macroscopic model with fewer degrees of freedom from the

350: microscopic model. Helbing, Schweitzer and coworkers did this in a

351: later work~\cite{HelbingSchweitzer}, where they derived a model

352: that describes the behavior of subpopulations of active walkers.

353: Although these models of trail formation may be faulted for not

354: being biologically realistic, they reproduce trail-forming

355: behavior of real ants, such as the ability to discover and link

356: distributed food sources without {\em a priori} knowledge of their

357: location. Such models may be especially useful to study

358: pheromone-based trail formation and navigation in

359: robots~\cite{VauStoSukMat00,VauStoSukMat00b}.

360: 

361: The main disadvantages of microscopic models are their poor

362: scaling properties and that it is not always easy or obvious how

363: to write down the equations of motion of each agent. Even if a

364: model can be written down, in most cases it will not be

365: analytically tractable, and the solution will have to be simulated

366: on a computer. By resorting to simulation, one looses much of the

367: power of mathematical analysis.

368: 

369: 

370: 

371: \subsubsection{Microscopic Simulations}

372: Microscopic simulations, such as molecular

373: dynamics~\cite{RobDuncan}, cellular

374: automata~\cite{abelson99,Wolfram} and particle hopping

375: models~\cite{CSS00}, are a popular tool for studying dynamics of

376: large multi-agent systems. In these simulations, agents change

377: state stochastically or depending on the state of their neighbors.

378: The popular Game of Life is an example of cellular automata

379: simulation. Another example of the microscopic approach is the

380: probabilistic model developed by Martinoli and

381: coworkers~\cite{Martinoli99,MarIjsGam99,IMB2001} to study

382: collective behavior of a group of robots. Rather than compute the

383: exact trajectories and sensory information of individual robots,

384: Martinoli {\em et al.} model each robot's interactions with other

385: robots and the environment as a series of stochastic events, with

386: probabilities determined by simple geometric considerations.

387: Running several series of stochastic events in parallel, one for

388: each robot, allowed them to study the group behavior of the

389: multi-robot system.

390: 

391: 

392: \subsection{Macroscopic Models}

393: A macroscopic description offers several advantages over the

394: microscopic approach. It is more computationally efficient,

395: because it uses many fewer variables. A macroscopic model can

396: often be solved analytically, yielding important insights into the

397: behavior of quantities of interest. The macroscopic descriptions

398: also tend to be more universal, meaning the same mathematical

399: description can be applied to other systems governed by the same

400: abstract principles. At the heart of this argument is the concept

401: of separation of scales, which holds that the details of

402: microscopic interactions (among agents) are only relevant for

403: computing the values of the parameters of the macroscopic model.

404: This idea has been used by physicists to construct a single model

405: that describes the behavior of seemingly disparate systems, {\em

406: e.g.}, pattern formation in convecting fluids and chemical

407: reaction-diffusion systems~\cite{Walgraef97}. This principle of

408: systems consisting of nearly decomposable parts applies broadly

409: not only to physical systems but also to naturally evolved

410: systems, as found in biology and economics, and designed

411: technological artifacts~\cite{courtois85,simon61,simon96}. From

412: the perspective of large-scale agent systems, this decomposition

413: often arises from processing, sensory and communication

414: limitations of the individual agents. In effect, these limits mean

415: agents can only pay attention to a relatively small number of

416: variables in the full system~\cite{hogg87PhysRep}, and will

417: generally communicate concise summaries of their states to other

418: agents. Of course, the two description levels are related, and it

419: may be possible in some cases to derive the parameters of the

420: macroscopic model from microscopic theory.

421: 

422: Macroscopic models are very popular and have been successfully

423: applied to a wide variety of problems in physics, chemistry,

424: biology and the social sciences. In most of these applications,

425: the microscopic behavior of individual entity (a Brownian particle

426: in a volume of gas or an individual residing in US) is quite

427: complex, often stochastic and unpredictable, and certainly

428: analytically intractable. Rather than account for the inherent

429: variability of individuals, scientists model the behavior of some

430: {\em average} quantity that represents the system they are

431: studying (volume of gas or population of US). Such macroscopic

432: descriptions often have a very simple form and are analytically

433: tractable. They are sometimes called phenomenological models,

434: because they are not derived from microscopic theories. It is

435: important to remember that such models do not reproduce the

436: results of a single experiment --- rather, the behavior of some

437: observable averaged over many experiments or observations. Such a

438: probabilistic approach is the basis for statistical physics.

439: 

440: Usually, the relative size of fluctuations in statistical systems

441: decreases with the number of components. In these cases, the

442: system is almost always found near its average behavior and so the

443: average is a good description for most individual experiments. It

444: is this observation that allows the convenient study of average

445: properties to describe behaviors actually seen in most

446: experiments. In some cases, fluctuations can become large, \eg,

447: near phase transitions in physical systems. Such behaviors are

448: also seen in computational systems, particularly those involved

449: with combinatorial search~\cite{hogg96d}, where long-tailed

450: distributions have typical behavior far from that of the average.

451: Thus when using macroscopic models to determine behavior of

452: averages, it is important to keep in mind the possibility that

453: actual system behaviors could be far from average. Fortunately, in

454: the context of multi-agent systems, such large fluctuations will

455: require an unexpectedly large statistical correlation in agent

456: behaviors, which is unlikely in situations in which the agents are

457: fairly independent and each act on only a few aspects of the

458: overall system state (\eg, based on local sensory information).

459: 

460: 

461: \subsubsection{Finite Difference Equations}

462: A macroscopic model can be frequently written down as a finite

463: difference equation describing the change in the value of a

464: dynamic variable over some time interval $\Delta t$. For example,

465: in a model of population dynamics of US,

466: \begin{eqnarray*}

467:   N(t+\Delta t)& =& N(t)+\Delta t R(t)N(t)\\

468:   R(t) & = & \frac {N(t+\Delta t) - N(t)} {\Delta t N(t)},

469: \end{eqnarray*}

470: \noindent where $N(t)$ is the (time-dependent) US population,

471: $\Delta t$ is a decade used by the Census Bureau, and $R(t)$ it

472: the rate of change of population due to births, deaths,

473: immigration and emigration. In general, $R(t)$ will also depend on

474: the choice of $\Delta t$. The modeler finds an appropriate $R$ to

475: describe population growth of US, and solves the equations to

476: project population growth into the future.

477: 

478: This description provides a finite-difference equation for the

479: behavior of $N$ at integer multiples of $\Delta t$. This has been

480: used to model a robotic

481: system~\cite{MartinoliISER02,MartinoliEaston02}, and is

482: particularly appropriate for \emph{synchronous} systems, \ie,

483: where all agents make decisions at the same time (such as parallel

484: update cellular automata).

485: 

486: In the continuous limit ($\Delta t \rightarrow 0$ or large $N$),

487: the difference equation become a differential equation, known as

488: the rate equation. For the example above, the rate equation is

489: %\begin{equation*}

490:  $ \frac {dN(t)}{dt}=R(t)N(t)$.

491: %\end{equation*}

492: which is also applicable to \emph{asynchronous} systems. In many

493: cases, the behavior of this differential equation matches that of

494: the difference equation~\cite{bender78}. However, this is not

495: always the case: synchronous and asynchonous systems can have very

496: different collective behaviors~\cite{huberman93a}. Large scale

497: agent systems interacting with an environment, \eg, robots, will

498: often need to respond to environmental signals that arrive at

499: unpredictable times. Such systems are likely to be better viewed

500: as asynchronous, for which the differential equation approach is

501: most suited.

502: 

503: 

504: 

505: \subsubsection{Rate Equations}

506: An alternate way to derive the rate equation is to start with the

507: master equation for a stochastic system and macroscopically

508: average it to get the rate equation for the dynamics of average

509: quantities. Section~\ref{sec:Math2} presents a tutorial on this

510: approach. However, in order to create a model of a multi-agent

511: system, one does not need to start with the master equation ---

512: one can easily write down the rate equations by examining the

513: details of individual agent controller.

514: 

515: The rate equations are deterministic. In stochastic systems,

516: however, rate equations describe the dynamics of average

517: quantities. How closely the average quantities track the behavior

518: of the actual dynamic variables depends on the magnitude of

519: fluctuations. Usually the larger the system, the smaller are the

520: (relative) fluctuations. In a small system, the experiment may be

521: repeated many times to average out the effect of fluctuations.

522: Pacala {\em et al.}\cite{Pacala} showed that in models of task

523: allocation in ants, the exact stochastic and the average

524: deterministic models {\em quantitatively} agree in systems

525: containing as few as ten ants. The agreement increases as the size

526: of the system grows. Martinoli and Easton~\cite{MartinoliISER02}

527: have shown quantitative agreement with simulations in a system of

528: 16--24 robots.

529: 

530: The rate equation has been used to model dynamic processes in a

531: wide variety of systems. The following is a short list of

532: applications: in chemistry, it has been used to study chemical

533: reactions~\cite{Gardiner}; in physics, the growth of semiconductor

534: surfaces \cite{BarabasiStanley95} among others; in ecology to

535: study dynamics of populations~\cite{Cushing}, including

536: predator-prey systems \cite{Haberman}; in biology to model the

537: behavior of social insects \cite{Pacala,Theraulaz98}. The rate

538: equation has also found application in the social sciences

539: \cite{Helbing} and in AI. Huberman, Hogg and

540: coworkers~\cite{HubermanHogg88,Kephart90} mathematically studied

541: collective behavior of a system of agents (they called a

542: computational ecology), where each agent chooses between two

543: alternative strategies. They start with underlying probability

544: distributions and derive rate equations for the average numbers of

545: agents using each strategy. In fact, the same equations can be

546: written down by examining the macroscopic state diagram of the

547: agents. Yet another application of the approach presented here is

548: coalition formation in electronic marketplaces~\cite{Lerman00a}.

549: 

550: In the robotics domain, Sugawara and

551: coworkers~\cite{Sugawara97,SugSanYosAbe98} developed simple

552: analytical models of cooperative foraging in groups of

553: communicating  and non-co\-mmu\-ni\-cating robots. \comment{We

554: discuss their models in Section~\ref{sec:foraging-sugawara}.

555: Although these models are similar to ours, they are overly

556: simplified and fail to take crucial interactions among robots into

557: account.} Kazadi \ea~\cite{Kazadi02} study the general properties

558: of multi-robot aggregation using phenomenological macroscopic

559: models. Agassounon and Martinoli~\cite{Agassounon02} present a

560: model of aggregation in which the number of robots taking part in

561: the clustering task is based on the division of labor mechanism in

562: ants. Lerman \emph{et al.} have analyzed

563: collaborative~\cite{Lerman01a} and foraging~\cite{Lerman02a}

564: behavior in groups of robots. Results from these works will be

565: used to illustrate the modeling methodology described in this

566: paper. The focus of the present paper is to show that there is a

567: principled way to construct a macroscopic analytic model of

568: collective dynamics of a MAS, and, more importantly, a practical

569: ``recipe'' for creating such a model from the details of the

570: microscopic agent controller.

571: 

572: 

573: \section{Macroscopic Analytic Models}

574: \label{sec:Math}

575: \subsection{MAS as Stochastic Systems}

576: The behavior of individual agents in a multi-agent system has many

577: complex influences, even in a controlled laboratory setting.

578: Agents are influenced by external forces, many of which may not be

579: anticipated. For robots, external forces include friction, which

580: may vary with the type of surface the robot is moving on, battery

581: power, sound or light signals, {\em etc}. Even if all the forces

582: are known in advance, the agents are still subject to random

583: events: fluctuations in the environment, as well as noise in the

584: robot's sensors and actuators. Each agent will interact with other

585: agents that are influenced by these and other events. In most

586: cases it is difficult to predict the agents' exact trajectories

587: and thus know which agents will come in contact with one another.

588: Finally, the agent designer can take advantage of the

589: unpredictability and incorporate it directly into the agent's

590: behavior. For example, the simplest effective policy for obstacle

591: avoidance in a robot is for it to turn a random angle and move

592: forward. In summary, the behavior of simple agents in a

593: complicated environment is so complex, the MAS is best described

594: probabilistically, as a stochastic system.

595: 

596: 

597: Before we present a methodology for mathematical a\-na\-ly\-sis of

598: sto\-chas\-tic sys\-tems, we need to define some terms. {\em

599: State} labels a set of related agent behaviors required to

600: accomplish a task. For example, when a robot is engaged in a

601: foraging task, its goal is to collect objects, such as pucks,

602: scattered around the arena and bring them to a home base. The

603: foraging task can be thought of as consisting of the following

604: high-level behavioral requirements~\cite{Arkin} or states

605: %% insert a picture here from Arkin

606: \begin{description}

607: \item[Homing] --- return the puck to a home base after it is picked up (includes collision avoidance)

608: \item[Pickup] --- if a puck is detected, close gripper

609: \item[Searching] --- wander around the arena in search of pucks (includes collision avoidance)

610: \end{description}

611: Each of these high level states may consist of a single action or

612: behavior, or a set of behaviors. For example, when a robot is in

613: the {\em Sear\-ching} state, it is wandering around the arena,

614: detecting objects and avoiding obstacles. In the course of

615: accomplishing the task, the robot will transition from the {\em

616: Searching} to {\em Pickup} and finally to {\em Homing} states. We

617: define these states to be mutually exclusive, \ie, so that each

618: agent in a multi-agent system is in exactly one of a finite number

619: of states during a sufficiently short time interval. Note that

620: there can be one-to-one correspondence between agent

621: actions/behaviors and states. However, in order to keep the

622: mathematical model compact and tractable, it is useful to {\em

623: coarse-grain} the system by choosing a smaller number of states,

624: each incorporating a set of agent actions or behaviors. Such

625: coarse-graining is particularly relevant when we are only

626: interested in behaviors described at this coarser level of

627: abstraction.

628: 

629: In general, the full description of an agent in its environment

630: could involve an arbitrary amount of detail, \eg, its exact

631: location in the environment. While such continuous states could be

632: included in the formalism we present here, for simplicity we take

633: the possible states of interest to be a finite set. Even when

634: continuous values, such as location, are relevant to particular

635: applications, it may be sufficient to treat these values with just

636: a few coarse-grained regions.

637: 

638: We associate a unit vector $\hat{q}_k$ with each state $k =

639: 1,2,\ldots,L$. The configuration of the system is defined by the

640: occupation vector

641: \begin{eqnarray}

642: \label{eqn-occupation} {\vec n} = \sum_{k=1}^{L}{n_k {\hat q_k}}

643: \end{eqnarray}

644: where $n_k$ is the number of agents in state $k$. The probability

645: distribution $P({\vec n},t)$ is the probability the system is in

646: configuration ${\vec n}$ at time $t$.

647: 

648: \subsection{The Stochastic Master Equation: A Tutorial}

649: \label{sec:Math2} For systems that obey the Markov property, the

650: future is determined only by the present and not by the past.

651: Clearly, agents that plan or use memory of past actions to make

652: decisions, will not meet this criterion directly. While it is

653: always possible to include the contents of an agent's memory as

654: part of its state to maintain the Markov property, this can result

655: in a vast expansion in the number of states to consider.

656: 

657: Fortunately, many MAS studied by various researchers, specifically

658: those based on reactive and behavior-based robots and many types

659: of software agents and sensors, do satisfy the Markov property

660: with a fairly modest number of states. We restate the Markov

661: property in the following way: the configuration of a system at

662: time $t + \Delta t$ depends only on the configuration of the

663: system at time $t$. In terms of coarse-grained states, we require

664: this property to apply to the system described at this level of

665: abstraction, at least to sufficient approximation.

666: 

667: The Markov property allows us to rewrite the marginal probability

668: density $P({\vec n}, t + \Delta t)$ in terms of conditional

669: probabilities for transition from ${\vec{n}}'$ to ${\vec n}$:

670: \begin{eqnarray*}

671: P({\vec n}, t + \Delta t) = \sum_{{\vec{n}}'}{P({\vec n}, t +

672: \Delta t |{{\vec{n}}'},t)P({{\vec {n}}'}, t)}.

673: \end{eqnarray*}

674: Using the fact that

675: \begin{eqnarray*}

676: \sum_{{\vec{n}}'}{P({{\vec {n}}'}, t + \Delta t|{\vec n},t)} = 1,

677: \end{eqnarray*}

678: allows us to write the change in probability density as

679: \begin{eqnarray}

680: \label{eqn-probability} P({\vec n}, t + \Delta t) - P({\vec n}, t)

681: &=& \sum_{{{\vec{n}}}'}{P({\vec n}, t + \Delta t |{\vec

682: {n}}',t)P({\vec {n}}', t)} \nonumber \\

683: & & - \sum_{\vec{n}'}{P({\vec {n}}', t + \Delta t |{\vec

684: n},t)P({\vec n}, t)}.

685: \end{eqnarray}

686: 

687: \noindent In the continuum limit, as ${\Delta t} \rightarrow 0$,

688: Eq.~\ref{eqn-probability} becomes

689: \begin{eqnarray}

690: \label{eqn-master} {{\partial P({\vec n}, t)}\over {\partial t}} =

691: \sum_{\vec{n}'}{W({\vec n}|{\vec{n}'};t)P({\vec{n}}', t)} -

692: \sum_{\vec{n}'}{W({\vec {n}'}|{\vec n};t)P({\vec n}, t)}\,,

693: \end{eqnarray}

694: with transition rates defined as

695: \begin{eqnarray}

696: \label{eqn-transition} W({\vec n}|{\vec {n}'};t) = {\lim_{\Delta t

697: \rightarrow 0}} {{P({\vec n}, t + \Delta t |{\vec {n}'},t)} \over

698: {\Delta t}},

699: \end{eqnarray}

700: provided this limit exists, \eg, changes are not synchronized to a

701: global clock which would make $P({\vec n}, t + \Delta t |{\vec

702: {n}'},t)$ equal to zero for all ${\vec n} \neq {\vec {n}'}$ when

703: $\Delta t$ is sufficiently small.

704: 

705: Equation \ref{eqn-master} says that the configuration of the

706: system is changed by transitions to and from states. It is known

707: as the Master Equation and is used widely to study dynamics of

708: stochastic systems in physics and chemistry~\cite{Gardiner},

709: traffic flow~\cite{Mahnke97,Mahnke99} and

710: sociodynamics~\cite{Helbing}, among others. The Master Equation

711: also applies to semi-Markov processes in which the future

712: configuration depends not only on the present configuration, but

713: also on the time the system has spent in this configuration. As we

714: will see in a later section,  transition rates in these systems

715: are time dependent, while in pure Markov systems, they are

716: time-independent.

717: 

718: \subsection{The Rate Equation}

719: 

720: The Master equation (Eq.~\ref{eqn-master}) fully determines the

721: evolution of a stochastic system. Once the probability

722: distribution $P({\vec {n}}, t)$ is found, one can calculate the

723: characteristics of the system, such as the average and the

724: variance of the occupation numbers. The Master equation is almost

725: always too complex to be analytically tractable. Fortunately we

726: can vastly simplify the problem by working with the average

727: occupation number, $\langle \vec{n} \rangle$ (the Rate Equation).

728: To derive an equation for $\langle \vec{n} \rangle$, we multiply

729: Eq.~\ref{eqn-master} by $ \vec{n}$ and take the sum over all

730: configurations:

731: \begin{eqnarray}

732: \label{eqn-rate-1}

733:  {\partial \over {\partial t}}\langle \vec{n} \rangle

734: & \equiv & {\partial \over {\partial t}}\sum_{\vec{n}}{\vec{n}

735: P({\vec{n}},t)} \nonumber \\

736: & = & \sum_{\vec{n}} \sum_{\vec{n}'}\vec{n}{W({\vec n}|{\vec

737: {n}'};t)}P(\vec{n}',t)

738:  -\sum_{\vec{n}}\sum_{\vec{n}'}\vec{n}{W({\vec n}'|{\vec

739: {n}};t)}P(\vec{n},t) \nonumber \\

740: & = & \sum_{\vec{n}} \sum_{\vec{n}'}(\vec{n}'-\vec{n}){W({\vec

741: n}'|{\vec{n}};t)}P(\vec{n},t) \nonumber \\

742: & =& \biggl < \sum_{\vec{n}'}(\vec{n}'-\vec{n}){W({\vec

743: n}'|{\vec{n}};t)} \biggl

744: >

745: \end{eqnarray}

746: where $\langle ... \rangle$ stands for averaging over the

747: distribution function $P(\vec n,t)$.  The time--evolution of a

748: particular occupation number is obtained from the vector equation

749: Eq.~\ref{eqn-rate-1} as

750: 

751: \begin{equation}

752: \label{eqn-rate-2} {\partial \over {\partial t}}\langle n_k

753: \rangle=\biggl < \sum_{\vec{n}'}(n_{k}'-n_k){W({\vec

754: n}'|{\vec{n}};t)} \biggr >

755: \end{equation}

756: 

757: Let us assume for simplicity that only individual transitions

758: between states are allowed,

759:  {\em i.e.}, $W({\vec n}'|{\vec{n}};t)\neq 0$ only if ${\vec n}'-{\vec

760:  n}=\hat{q}_i-\hat{q}_j$, $i\neq j$,

761:  and let $w_{ij}$ be the transition rate  from

762: state $j$ to state $i$. This is appropriate for systems with

763: asynchronous updates for the agents since in that case it is very

764: unlikely that two agents would make a change at the same time.

765: Note that in general $w_{ij}$ may be a function of the occupation

766: vector $\vec{n}$, $w_{ij}=w_{ij}(\vec{n})$. Define a matrix ${\bf

767: D}$ with off-diagonal elements $w_{ij}$ and with diagonal elements

768: $D_{ii}=-\sum_kw_{ki}$. Then we can rewrite Eq.~\ref{eqn-rate-1}

769: in a matrix form as

770: \begin{equation}

771: {\partial \over {\partial t}}\langle \vec{n} \rangle=\langle {\bf

772: D}(\vec n ) \cdot \vec n \rangle \approx {\bf D}(\langle \vec n

773: \rangle)\cdot \langle \vec n \rangle

774: \end{equation}

775: where we have used so the called mean-field approximation $\langle

776: F(\vec{n})\rangle \approx F(\langle \vec n \rangle)$. The

777: mean-field approximation is often used in statistical phy\-sics

778: and is well justified for unimodal and sharp distribution

779: functions~\cite{opper01}.

780: %In the absence of interaction (direct or indirect) between agents the MF

781: %approximation is exact since the transition rates $w_{ij}$ are

782: %independent of the occupation vector $\vec n$. In this simple case

783: The average occupation numbers obey the following system of

784: coupled linear equations

785: 

786: \begin{equation}

787: \label{eqn-rate-3} {\partial \over {\partial t}}\langle n_k

788: \rangle= \sum_{j} w_{jk}(\langle \vec n \rangle) \langle

789: n_{j}\rangle - \langle n_k \rangle \sum_{j}w_{kj}(\langle \vec n

790: \rangle)

791: \end{equation}

792: 

793: The above equation is known as the Rate Equation. It has the

794: following interpretation: occupation number $n_k$ will increase in

795: time (first term in Eq.~\ref{eqn-rate-3}) due to transitions from

796: other states to state $k$, and it will decrease in time due to the

797: transitions from the state $k$ to other states (second term).

798: 

799: 

800: \subsubsection {Transition rates}

801: 

802: Finding an appropriate mathematical form for the transition rates

803: is the main challenge in applying the rate equations to real

804: systems. Usually, the transition is triggered when an agent

805: encounters some stimulus

806: --- be it another agent in a particular state, an object, its

807: location, {\em etc}. For simplicity, we will assume that agents

808: and triggers are uniformly distributed in space (though we will

809: consider systems where agents interact in space, it does not

810: necessarily have to be physical space, but a network, the Web,

811: {\em etc}). The assumption of spatial uniformity may be reasonable

812: for agents that randomly explore space ({\em e.g.}, searching

813: behavior in robots tends to smooth out any inhomogeneities in the

814: robots' initial distribution); however, it fails for systems that

815: are strongly localized, for instance, where all the objects to be

816: collected by robots are located in the center of the arena. In

817: these anomalous cases, the transition rates will have a more

818: complicated form and in some cases it may not be possible to

819: express them analytically altogether. If the transition rates

820: cannot be calculated from first principles, it may be expedient to

821: leave them as parameters in the model and estimate them by fitting

822: the model to data.

823: 

824: %dilute limit

825: 

826: 

827: \subsubsection{Rate Equation and Multi-Agent Systems}

828: The rate equation is a useful tool for mathematical analysis of

829: macroscopic, or collective, dynamics of many agent-based systems.

830: To facilitate the analysis, we begin by drawing the macroscopic

831: state diagram of the system. The state diagram can be constructed

832: from the details of the individual agent's behavior, as will be

833: illustrated in the applications. Not every microscopic, or

834: individual agent, behavior need become a macroscopic state. In

835: order to keep the model tractable, it is often useful to

836: coarse-grain the system by considering several related behaviors

837: as a single state. As as example considered in detail below, we

838: may take the searching state of robots to consist of the actions

839: {\em wander in the arena, detect objects} and {\em avoid

840: obstacles}. When necessary, the searching state can be split into

841: three states, one for each behavior; however, we are often

842: interested in the $minimal$ model that captures the important

843: behavior of the system. Coarse-graining presents a way to

844: construct such a minimal model. In addition to states, we must

845: also specify transitions between states. These will be represented

846: as arrows leading from one state to another.

847: 

848: Each state in the macroscopic state diagram corresponds to a

849: dynamic variable in the mathematical model --- the average number

850: of agents in that state --- and it is coupled to other variables

851: via transitions between states. The mathematical model will

852: consist of a series of coupled rate equations, one for each state,

853: which describes how the number of agents in that state changes in

854: time. This quantity may increase due to $incoming$ transitions

855: from other states, and it may decrease due to $outgoing$

856: transitions to other states. Every transition will be accounted

857: for by a term in each equation, with transition rates specified by

858: the details of the interactions between agents.

859: 

860: In the next sections we will illustrate the details of the

861: approach by applying it to study problems in the robotics domain.

862: Our examples include foraging (Section~\ref{sec:foraging}) and

863: collaboration (Section~\ref{sec:sp}) in groups of robots.

864: 

865: %\input{foraging}

866: \newcommand{\ti}{{\tilde t}}

867: \section{Foraging in a Group of Robots}

868: \label{sec:foraging}

869: 

870: 

871: \comment{ Robot collection and foraging is one of the oldest and

872: most studied problems in robotics. In this task a single robot or

873: a group of robots has to collect objects scattered around the

874: arena and to assemble them either in some random location

875: (collection task) or a pre-specified ``home'' location (foraging

876: task). These tasks have been studied under a wide variety of

877: conditions and architectures, both experimentally and in

878: simulation. Below is a partial list of the previous work on

879: collection that can be categorized in the following way:

880: \begin{itm}

881: \item Task type

882:     \begin{itm}

883:     \item collection\cite{Beckers94,MarIjsGam99}

884:     \item foraging\cite{Mataric92,GolMat00,ArkBalNit}

885:     \end{itm}

886: \item System type

887:     \begin{itm}

888:     \item single robot

889:     \item group of robots (homogeneous\cite{GolMat00} and heterogeneous\cite{GolMat00,ParkerPhD})

890:     \end{itm}

891: \item Controller type

892:     \begin{itm}

893:     \item reactive

894:     \item behavior-based\cite{Mataric92,GolMat00}

895:     \item hybrid\cite{ArkBalNit}

896:     \end{itm}

897: \item Communication

898:     \begin{itm}

899:     \item no communication\cite{GolMat00}

900:     \item direct\cite{ArkBalNit,Sugawara97}

901:     \item stigmergetic (through modification of the environment) \cite{HolMel00,VauStoSukMat00}

902:     \end{itm}

903: \end{itm}

904: 

905:  The broad appeal of

906: this problem is explained both by ubiquity of collection in

907: general and foraging in particular in nature --- as seen in the

908: food gathering behavior of many insects --- as well as its

909: relevance to many military and industrial applications, such as

910: de-mining, mapping and toxic waste clean-up. Foraging has been a

911: testbed for the design of physical robots and their controllers,

912: as well as a framework for exploring many issues in the design and

913: implementation of multi-robot teams.}

914: 

915: Robot collection and foraging are two of the oldest and most

916: studied problems in robotics. In these tasks a single robot or a

917: group of robots has to collect objects scattered around the arena

918: and to assemble them either in some random location (collection

919: task~\cite{Beckers94,MarIjsGam99}) or a pre-specified ``home''

920: location (foraging task~\cite{Mataric92,GolMat00,ArkBalNit}).

921: These tasks have been studied under a wide variety of conditions

922: and architectures, both experimentally and in simulation: in

923: homogeneous  and heterogeneous~\cite{GolMat00} systems, using

924: behavior-based~\cite{Mataric92,GolMat00} and hybrid

925: control~\cite{ArkBalNit}, no communication~\cite{GolMat00}, direct

926: communication~\cite{ArkBalNit,Sugawara97}, as well as indirect

927: communication through the environment~\cite{HolMel00}. The broad

928: appeal of this problem is explained both by ubiquity of collection

929: in general and foraging in particular in nature --- as seen in the

930: food gathering behavior of many insects --- as well as its

931: relevance to many military and industrial applications, such as

932: de-mining, mapping and toxic waste clean-up. Foraging has been a

933: testbed for the design of physical robots and their controllers,

934: as well as a framework for exploring many issues in the design and

935: implementation of multi-robot teams.

936: 

937: 

938: 

939: In this section, we focus on analysis of foraging in a homogeneous

940: non-communi\-ca\-ting multi-robot systems using behavior-based

941: control, the type of systems studied by Mataric and collaborators

942: \cite{Mataric92,GolMat00}. In Sec.~\ref{sec:sugawara} we will

943: present Sugawara {\em et al.}'s model of foraging in communicating

944: robots. Figure~\ref{fig:foraging_robots} is a snapshot of a

945: typical experiment with four robots. The robots' task is to

946: collect small pucks randomly scattered around the arena. The arena

947: itself is divided into a search region and a small ``home'', or

948: goal, region where the collected pucks are deposited. The

949: ``boundary'' and ``buffer'' regions are part of the home region

950: and are made necessary by limitations in the robots' sensing

951: capabilities, as described below. Each robot has an identical set

952: of behaviors governed by the same controller. The behaviors that

953: arise in the collection task are \cite{GolMat00}:

954: \begin{description}

955:   \item[Avoiding] obstacles, including other robots and boundaries.

956: This behavior is critical to the safety of the robot.

957:   \item[Searching] for pucks: robot moves forward and at random intervals

958:   turns left or right through a random arc.  If the robot enters the Boundary region, it

959: returns to the search region. This prevents the robot from

960: collecting pucks that have already been delivered.

961:   \item[Detecting] a puck.

962:   \item[Grabbing] a puck.

963:   \item[Homing] if carrying a puck, move towards the home location.

964:   \item[Creeping] activated by entering Buffer region. The robot will start using the

965:   close-range detectors at this point to avoid the boundaries.

966:   \item[Home] robot drops the puck. This activates the exiting behavior.

967:   \item[Exiting] robot exits the home region and resumes search.

968: \end{description}

969: \begin{figure}

970:   \epsfxsize = 2.5in

971:   \center{\epsffile{foraging_robots.eps}}

972:   \caption{Diagram of the foraging arena (courtesy of D.~Goldberg).}\label{fig:foraging_robots}

973: \end{figure}

974: 

975: 

976: 

977: 

978: %\subsection{Interference}

979: Interference, caused by competition for space between spatially

980: extended robots, has long been recognized as a critical issue in

981: multi-robot systems \cite{FonMat96,Sugawara97}. When two robots

982: find themselves within sensing distance of one another, they will

983: execute obstacle avoiding maneuvers in order to reduce the risk of

984: a potentially damaging collision. The robot stops, makes a random

985: angle turn and moves forward. This behavior takes time to execute;

986: therefore, avoidance increases the time it takes the robot to find

987: pucks and deliver them home. Clearly, a single robot working alone

988: will not experience interference from other robots. However, if a

989: single robot fails, as is likely in a dynamic, hostile

990: environment, the collection task will not be completed. A group of

991: robots, on the other hand, is robust to an individual's failure.

992: Indeed, many robots may fail but the performance of the group may

993: be only moderately affected. Many robots working in parallel may

994: also speed up the collection task. Of course, the larger the

995: group, the greater the degree of interference --- in the extreme

996: case of a crowded arena, robots will spend all their time avoiding

997: other robots and will not bring any pucks home.

998: 

999: Several approaches to minimize interference have been explored,

1000: including communication \cite{Parker98} and cooperative strategies

1001: such as trail formation \cite{VauStoSukMat00b} and bucket brigade

1002: \cite{FonMat96,OstSukMat01}. In some cases, the effectiveness of

1003: the strategy to minimize interference will also depend on the

1004: group size \cite{OstSukMat01}. Therefore, it is important to

1005: quantitatively understand interference between robots and how it

1006: relates to the group and task sizes before choosing alternatives

1007: to the default strategy. For some tasks and a given controller,

1008: there may exist an optimal group size that maximizes the

1009: performance of the system \cite{ArkBalNit,FonMat96,OstSukMat01}.

1010: Beyond this size the adverse effects of interference become more

1011: important than the benefits of increased robustness and

1012: parallelism, and it may become beneficial to choose an alternate

1013: foraging strategy. We will study interference mathematically and

1014: attempt to answer these questions.

1015: 

1016: %Previous work

1017: Nitz {\em et al.}\cite{ArkBalNit} briefly addressed the question

1018: of what is an appropriate number of robots for a foraging task in

1019: a given environment. By simulating foraging in groups of up to

1020: five communicating robots, they observed an increase in

1021: performance when adding one to three robots as compared to a

1022: single worker. However, the performance seemed to level out and

1023: even degrade with further additions. Performance of

1024: non-communicating robots seemed to improve as the group size grew,

1025: at least up to the group size of five. No simulations for larger

1026: group sizes were carried out.

1027: %Previous mathematical analysis

1028: \comment{Sugawara and coworkers~\cite{Sugawara97,SugSanYosAbe98}

1029: carried out a quantitative study of foraging in groups of

1030: communicating and non-communicating robots. They  developed a

1031: simple rate equation-based model of foraging and analyzed it under

1032: different conditions, that included non-communicating robots.

1033: However, their treatment of interference is different from ours,

1034: as will be explained below. }

1035: 

1036: 

1037: 

1038: \subsection{Rate Equation Model of Foraging}

1039: As mentioned above, interference is the result of competition

1040: between two or more robots for the same resource, be it physical

1041: space, the puck both are trying to pick up, energy, communications

1042: channel, \etc. In the foraging task, competition for physical

1043: space, and the resulting avoidance of collisions with other

1044: robots, is the most common source of interference.  In this

1045: section we examine the foraging scenario where robots are required

1046: to collect pucks and bring them to a specified ``home'' location.

1047: 

1048: 

1049: 

1050: At a macroscopic level, during some short time interval, every

1051: robot can be considered to be in the searching, homing or avoiding

1052: states, as shown in Fig.~\ref{fig:homing_diagram}. We assume that

1053: actions like detecting and grabbing a puck take place on a

1054: sufficiently short time scale that they can be incorporated into

1055: the search state. Likewise, creeping, \etc, can be incorporated

1056: into the homing state.\footnote{If we find that the given

1057: descriptive level does not adequately capture the behavior of a

1058: real or simulated system, we can consider more states in the

1059: model. For now, we are interested in the minimal model that

1060: reproduces salient features of the foraging system.} Initially the

1061: robots are in the search state. When a robot finds a puck, it

1062: picks it up and moves to the ``home'' region. Execution of the

1063: homing behavior requires a period of time $\tau_h$. At the end of

1064: this period, the robot deposits the puck at home and resumes

1065: search for more pucks. While the robot is homing, it will

1066: encounter and try to avoid other robots.

1067: 

1068: \begin{figure}[htpb]

1069:   \epsfxsize=1.8in

1070:   \center{

1071:   \epsffile{forage_diagram2b.eps}}

1072: %  \vskip 0.1in

1073:   \caption{State diagram of a multi-robot foraging system with homing.}\label{fig:homing_diagram}

1074: \end{figure}

1075: 

1076: Each state in the diagram corresponds to a dynamic variable. Let

1077: $N_s(t)$, $N_h(t)$, $N_{av}^s(t)$, $N_{av}^h(t)$ be the number of

1078: searching, homing, avoiding while searching and avoiding while

1079: homing robots at time $t$, with the total number of robots,

1080: $N_0=N_s(t)+ N_h(t)+ N_{av}^s(t)+ N_{av}^h(t)$, a constant. We

1081: model the environment by letting $M(t)$ be the number of

1082: undelivered pucks at time $t$.

1083: %% Todo

1084: Also, let $\alpha_r$ be the rate of detecting another robot and

1085: $\alpha_p$ the rate of detecting a puck. These parameters connect

1086: the model to the experiment, and they are related to the size of

1087: the robot and the puck, robot's detection radius and the speed of

1088: the robot. It was shown experimentally~\cite{GolMat00} that

1089: interference is most pronounced near the home region, because the

1090: density of robots is, on average, greater there. Therefore, we

1091: expect the rate of encountering other robots to be greater near

1092: the home region and introduce $\alpha_r^{\prime}$, the rate of

1093: detecting another robot while homing. The following equations

1094: describe the time evolution of the dynamic variables\footnote{For

1095: simplicity, we do not include wall avoidance in the equations, but

1096: do take it into account when fitting model to the data.}:

1097: \begin{eqnarray}

1098: \label{eq:forage-searching3} \frac{d{N_s(t)}}{dt}&=&

1099: -\alpha_pN_s(t)[ M(t) -N_h(t)-N_{av}^h(t) ] \nonumber \\

1100: & & - \alpha_r N_s(t)[N_s(t)+N_0] \nonumber\\

1101: & & + \frac{1}{\tau_h} N_h(t)+\frac{1}{\tau} N_{av}^s(t), \\

1102: \label{eq:forage-homing3} \frac{d{N_h(t)}}{dt}&=&\alpha_pN_s(t)[

1103: M(t) -N_h(t)-N_{av}^h(t) ]  \nonumber

1104: \\

1105: & & -\alpha_r^{\prime}N_h(t)[N_h(t)+N_0]\nonumber \\

1106: & & + \frac{1}{\tau} N_{av}^h(t) -\frac{1}{\tau_h} N_h(t),

1107: \\

1108: \label{eq:forage-avoiding-while-homing3}

1109: \frac{d{N_{av}^h(t)}}{dt}&=&\alpha_r^{\prime}N_h(t)[N_h(t)+N_0]-\frac{1}{\tau}

1110: N_{av}^h(t), \\

1111: \label{eq:forage-pucks3} \frac{d{M}(t)}{d t} & = &

1112: -\frac{1}{\tau_h}N_h(t).

1113: \end{eqnarray}

1114: 

1115: The first term in Eq.~\ref{eq:forage-searching3} accounts for a

1116: decrease in the number of searching robots when robots find pucks

1117: and start homing. The second term says that the number of

1118: searching robots decreases when two searching robots detect each

1119: other and commence avoiding maneuvers or when a searching robot

1120: detects another robot in any of the remaining states. The number

1121: of available pucks is just the number of pucks in the arena less

1122: the pucks held by the homing robots. The last two terms in the

1123: equation require more explanation. We assume that  it takes on

1124: average $\tau_h$ time for a robot to reach home after grabbing a

1125: puck. Then the average number of robots that deliver pucks during

1126: a short time interval $dt$ and return to the searching state can

1127: be approximated as $dt N_h/\tau_h$. Likewise, after a period of

1128: time $\tau$, $dt N_{av}^s/\tau$ robots leave the avoiding state

1129: and resume searching. We assume that interference will increase

1130: the homing time for each robot: if $\tau^{0}_h$ is the average

1131: homing time in the absence of collisions with other robots, then

1132: the effective homing time can be estimated as

1133: %\begin{equation}\label{eq:homing_time}

1134:  $ \tau_h = {\tau_h}^0[1 + \alpha_r^{\prime} \tau N_0]\,

1135: $.%\end{equation}

1136: 

1137: The remaining equations have similar interpretations. We can take

1138: advantage of the conservation of the total number of robots to

1139: compute $N_{av}^h(t)$. These equations are solved numerically

1140: under the conditions that initially, at $t=0$, there are $M_0$

1141: pucks and $N_0$ searching robots.

1142: 

1143: Figure \ref{fig:fhom1} shows the time evolution of the fraction of

1144: searching robots and pucks for $M_0=20$, $N_0=5$, $\tau= 3\ s$,

1145: $\tau_{h}^{0}=16\ s$. The number of searching robots (solid line)

1146: first quickly decreases as robots find pucks and carry them home,

1147: but then it increases and saturates at some steady state value as

1148: the number of undelivered pucks approaches zero (dashed line). The

1149: fraction of the searching robots in the steady state depends on

1150: the avoiding time parameter, which determines the fraction of

1151: robots in the avoiding state.

1152: 

1153: \begin{figure}[htbp]

1154: \center{

1155: %  \begin{tabular}{cc}

1156:   \epsfxsize = 2.4in

1157:   \epsffile{fhom1.eps}

1158: %  \epsfxsize = 2.4in

1159: %  \epsffile{fhom2.eps}\\

1160: %  \end{tabular}

1161: %  \vskip 0.1in

1162:   \caption{ Time evolution of the fraction of searching robots (solid line) and undelivered pucks (dashed line) for $\tau = 3\ s$, $\alpha_p=0.015$, $\alpha_r=0.04$, and $\alpha_r^{\prime}=0.08$.

1163:    %(b) Time evolution of the fraction of uncollected pucks for different robot

1164: %  group sizes: (i) $N_0=5$ (dashed), (ii) $N_0=15$ (solid), and (iii) $N_0=20$

1165: %(dotted). In all cases $M_0=20$.

1166: } \label{fig:fhom1}

1167:   }

1168: \end{figure}

1169: 

1170: 

1171: % simulations

1172: To validate results of the model, we ran foraging simulations

1173: using Player/Stage simulator~\cite{StagePlayer} for groups of one

1174: to ten robots and twenty pucks randomly scattered around the

1175: arena. Details of the simulations are presented in

1176: \cite{Lerman02a}.

1177: 

1178: 

1179: Figure \ref{fig:homing-time}(a) shows the total time required to

1180: complete the task for two different interference strengths, as

1181: measured by the avoiding time parameter $\tau$. The solid line is

1182: the result of the model's prediction for $\tau = 3\ s$, and the

1183: dotted line for $\tau = 1.5\ s$\footnote{Although in the

1184: simulations we specified the avoidance time to be 1\ s, multiple

1185: collisions caused the average avoiding time per collision to be

1186: slightly higher.}, and $\tau_h^0 = 16\ s$, $\alpha_p=0.015$,

1187: $\alpha_r = 0.04$, and $\alpha_r^{\prime}=0.08$. \comment{We have

1188: also taken into account the effect of wall avoidance, it's

1189: strength given by $\alpha_w = 0.04$.} The simulations data shows

1190: that the average avoiding time per collision increases as the

1191: group size grows. This is due to multiple avoidance moves per each

1192: attempt to avoid collision, caused by an increase in the local

1193: density of robots. Therefore, in the model, we take the avoiding

1194: time parameter $\tau$ as a linearly increasing function of $N_0$,

1195: with the initial value of $\tau^0=3\ s$ (or $1.5\ s$). The

1196: agreement between the model and simulations is good. \comment{Note

1197: that there are minor differences between the values of the

1198: parameters that best fit the data and the values we compute from

1199: the simulations: we took $\tau_h^0 = 16.00\ s$ (the data shows the

1200: average homing time for a single robot is $14.95 \pm 0.98$), and

1201: $\alpha_p = 0.015$. Aside from these slight differences in the

1202: parameter values, the agreement between model and simulations is

1203: good. The differences are probably caused by the fact that our

1204: model does not take into account the time robots spend in the

1205: reverse homing behavior, during which they cannot search for

1206: pucks.}

1207: 

1208: The final plot (Fig.~\ref{fig:homing-time}(b)) shows that

1209: interference causes deterioration in relative performance. The per

1210: robot efficiency is a monotonically decreasing function of the

1211: group size. Thus, adding one new robot to the group decreases the

1212: performance of every robot, though if the initial group size was

1213: less than the optimal size, adding a robot will increase the

1214: overall efficiency of the group.

1215: 

1216: \begin{figure}[htbp]

1217: \center{

1218:   \begin{tabular}{l}

1219:   \epsfxsize = 2.4in

1220:   \epsffile{homing.eps}\\

1221:   \epsfxsize = 2.6in

1222:   \epsffile{fhom5.eps}

1223:   \end{tabular}

1224:   \caption{The time it takes the group to collect and deliver pucks home for two interference strengths, $\tau = 3\ s$, and $\tau = 1.5 s$ and $\tau_h^0=16\ s$, $\alpha_p=0.015$, $\alpha_r = 0.04$, $\alpha_r^{\prime}=0.08$.

1225:  (b) Efficiency per robot

1226:   } \label{fig:homing-time}

1227:   }

1228: \end{figure}

1229: 

1230: 

1231: 

1232: \subsection{Foraging in Communicating Robots}

1233: \label{sec:sugawara} Sugawara and

1234: coworkers~\cite{Sugawara97,SugSanYosAbe98,Sugawara+al99} carried

1235: out quantitative studies of foraging in groups of communicating

1236: robots in different environments. In their system when a robot

1237: finds a puck, it broadcasts a signal for a duration of time $x$.

1238: If other robots detect the signal, they turn and move towards it.

1239: After the interaction period, the broadcasting robot turns off the

1240: signal, and makes a transition to the homing state. The

1241: macroscopic state diagram for this system is shown in

1242: Fig.~\ref{fig:sugawara} and is composed of the following states:

1243: searching (S), broadcasting (B), homing (H), moving to the signal

1244: source (M), and avoiding (A), with the corresponding dynamic

1245: variables representing the average number of robots in each state.

1246: 

1247: \begin{figure}

1248:   \epsfxsize 2.4in

1249:   \center{

1250:   \epsffile{sugawara-diagram.eps}}

1251:   \caption{State diagram of foraging robots from Sugawara {\em et al.}~\protect\cite{Sugawara97}}\label{fig:sugawara}

1252: \end{figure}

1253: 

1254: \begin{eqnarray}\label{eq:sugawara}

1255:   \frac {dN_h} {dt} & = & \frac{1}{(x+1)}N_b - \frac{1}{\tau} N_h \\

1256:   \frac {dN_b} {dt} & = & - \frac{1}{(x+1)}N_b + \alpha N_s + \frac {\gamma} {a+N_a}N_a \\

1257:   \frac {dN_s} {dt} & = & -\alpha N_s + nN_m + \frac{1}{\tau}N_h - a l(x) N_s N_b\\

1258:   \frac {dN_a} {dt} & = & \frac{v}{d}N_m -\frac {\gamma} {a+N_a}N_a\\

1259:   \frac {dN_m} {dt} & = & -\frac{v}{d}N_m -bN_m +a l(x) N_s N_b

1260: \end{eqnarray}

1261: \noindent where $\alpha$ is the probability to find a puck, $b$

1262: probability to lose signal source, $\tau$ time to return home, $x$

1263: interaction duration, $a$ is the probability of catching the

1264: broadcast signal, $l(x)$ turn angle, $d$ average distance between

1265: interacting robots, $v$ velocity of the robot, and $\gamma$

1266: probability to find a puck by following other robots.

1267: 

1268: Interference between robots due to collision avoidance is assumed

1269: to be negligible except during crowding near a broadcasting robot;

1270: therefore, avoiding terms appear only in the equations describing

1271: broadcasting and moving robots. The strength of interference is

1272: phenomenologically described by a simple function that is

1273: inversely proportional to the density of robots.

1274: 

1275: Sugawara \emph{et al.} found that for non-communicating robots

1276: ($x=0$), the time to complete the task was proportional to the

1277: inverse of the number of robots --- $T \approx N^{-1}$. This is

1278: true when robots work independently of one another. Interaction

1279: through signal broadcast improves the efficiency of group behavior

1280: --- $T \approx N^{\beta}$, $-1 < \beta < -2$  --- for most

1281: durations $x$ of interaction. This result is independent of

1282: whether the puck distribution was homogeneous or localized

1283: (specified by parameter $\gamma$) . These findings were confirmed

1284: by simulations and experiments with physical robots.

1285: 

1286: At first glance, the results of Sugawara \emph{et al.} appear to

1287: contradict those presented in Fig.~\ref{fig:homing-time}(a), which

1288: shows that performance time $T$ decreases with group size $N$ up

1289: to some critical value and then starts to increase. \comment{In a

1290: larger study of foraging in robots using the approach described

1291: here~\cite{Lerman02a}, we showed that the time to complete a

1292: collection task (foraging without homing) decreases monotonically

1293: with $N$ as $T \approx N^{-0.7}$, even when avoiding behavior is

1294: taken into account.} In fact, the results of both works support

1295: the same basic conclusion that avoiding needs to be included in

1296: the model when crowding conditions occur (the exact density

1297: threshold remains to be determined). We believe the apparent

1298: discrepancy in results can be explained by the difference in

1299: systems being modeled. In Sugawara {\em et al.} work, ``home'' is

1300: located at the center of the arena. Avoiding is not taken into

1301: account in the no-interaction model because there is less crowding

1302: near the home region than near a broadcasting robot. In the system

1303: we studied, on the other hand, ``home'' is located at the edge of

1304: the arena, and crowding is more pronounced. Both studies construct

1305: a minimal model required to explain the observed behavior of the

1306: system. The differences in the models and conclusions can be

1307: traced back to the differences in the systems being modeled and

1308: their behavior.

1309: 

1310: %\input{stick-pulling2}

1311: \section{Collaboration in Robots}

1312: \label{sec:sp} Collaboration can significantly increase the

1313: performance of a multi-agent system. In some systems collaboration

1314: is an explicit requirement, because a single agent cannot

1315: successfully complete the task on its own. Such ``strictly

1316: collaborative''~\cite{Martinoli99} systems are common in insect

1317: and human societies, {\em e.g.}, in transport of an object too

1318: heavy or awkward to be lifted by a single ant, flying the space

1319: shuttle, playing a soccer match, {\em etc}. Collaboration in a

1320: group of robots has been studied by several

1321: groups~\cite{MatNilSim95,KubBon00,MarMon95,IMB2001,KitTamStoVel98}.

1322: We will focus on one group of experiments initiated by Martinoli

1323: and collaborators~\cite{MarMon95} and studied by Ijspeert et

1324: al.~\cite{IMB2001} that take a swarm approach to collaboration. In

1325: this system collaboration in a homogeneous group of simple

1326: reactive agents was achieved entirely through local interactions,

1327: {\em i.e.}, without explicit communication or coordination among

1328: the robots. Because they take a purely swarm approach, their

1329: system is a compelling and effective model of how collaboration

1330: may arise in natural systems, such as insect societies.

1331: 

1332: \subsection{Stick-pulling Experiments in Groups of Robots}

1333: The stick-pulling experiments were carried out by Ijspeert {\em et

1334: al.} to investigate the dynamics of collaboration among locally

1335: interacting simple reactive robots. Figure~\ref{fig:experiment} is

1336: a snapshot of the physical set-up of the experiments. The robots'

1337: task was to locate sticks scattered around the arena and pull them

1338: out of their holes. A single robot cannot complete the task (pull

1339: the stick out) on its own --- a collaboration between two robots

1340: is necessary for the task to be successfully completed. In a more

1341: general case, a collaboration between an arbitrary number of

1342: robots may be required to successfully complete the tasks (because

1343: sticks may be of varying length). The collaboration occurs in the

1344: following way: one robot finds a stick, and waits for a second

1345: robot to find it, lifting the stick partially out of its hole.

1346: When a second robot finds it, it will grip the stick and pull it

1347: out of the ground, successfully completing the task. (In the

1348: general case, a group of some size has to accumulate at the site

1349: of the stick before the required number of robots necessary to

1350: complete the task is present.)

1351: 

1352: \begin{figure}[hptb]

1353: \epsfxsize = 4.0in \center{\epsffile{arena.eps}} %sprobots.eps}}

1354: %\centerline{\includegraphics[width=4pc]{arena.eps}}

1355: \caption{Physical set-up of the stick-pulling experiment showing

1356: six Khepera robots (courtesy of A.~Martinoli).}

1357: \label{fig:experiment}

1358: \end{figure}

1359: 

1360: The actions of each robot are governed by a simple controller,

1361: outlined in Figure~\ref{fig:controller}. The robot's default

1362: behavior is to wander around the arena looking for sticks and

1363: avoiding obstacles, which could be other robots or walls. When a

1364: robot finds a stick that is not being held by another robot, it

1365: grips it, lifts it half way out of the ground and waits for a

1366: period of time specified by the {\em gripping time parameter}. If

1367: no other robot comes to its aid during the waiting period, the

1368: robot releases the stick and resumes the search for other sticks.

1369: If another robot encounters a robot holding a stick, a successful

1370: collaboration will take place during which the second robot will

1371: grip the stick, pulling it out of the ground completely, while the

1372: first robot releases the stick and resumes the search. After the

1373: task is completed, the second robot also releases the stick and

1374: returns to the search mode, and the experimenter replaces the

1375: stick in its hole.

1376: 

1377: Ijspeert {\em et al.} studied the dynamics of collaboration in

1378: stick-pulling robots on three different levels: by conducting

1379: experiments with physical robots; with a sensor-based simulator of

1380: robots; and using a probabilistic microscopic model. The physical

1381: experiments were performed with groups of two to six Khepera

1382: robots in an arena containing four sticks. Because experiments

1383: with physical robots are very time consuming, Webots, the

1384: sensor-based simulator of Khepera robots~\cite{Michel98}, was used

1385: to systematically explore the parameters affecting the dynamics of

1386: collaboration. Webots simulator attempts to faithfully replicate

1387: the physical experiment by reproducing the robots' (noisy) sensory

1388: input and the (noisy) response of the on-board actuators in order

1389: to compute the trajectory and interactions of each robot in the

1390: arena. The probabilistic microscopic model, on the other hand,

1391: does not attempt to compute the trajectories of individual robots.

1392: Rather, the robot's actions

1393: --- encountering a stick, a wall, another robot, a robot gripping

1394: a stick, or wandering around the arena --- are represented as a

1395: series of stochastic events, with probabilities based on simple

1396: geometric considerations. For example, the probability of a robot

1397: encountering a stick is equal to the product of the number of

1398: ungripped sticks, and the detection area of the stick normalized

1399: by the arena area. Probabilities of other interactions can be

1400: similarly calculated. The microscopic simulation consists of

1401: running several processes in parallel, each representing a single

1402: robot, while keeping track of the global state of the environment,

1403: such as the number of gripped and ungripped sticks. According to

1404: Ijspeert {\em et al.} the acceleration factor for Webots and real

1405: robots can vary between one and two orders of magnitude for the

1406: experiments presented here. Because the probabilistic model does

1407: not require calculations of the details of the robots'

1408: trajectories, it is $300$ times faster than Webots for these

1409: experiments.

1410: 

1411: \begin{figure}[hptb]

1412: \epsfxsize = 3.0in \center{\epsffile{spagent.eps}}

1413: \caption{Flowchart of the robots' controller (from Ijspeert et

1414: al~\protect\cite{IMB2001}). } \label{fig:controller}

1415: \end{figure}

1416: 

1417: \subsubsection{Experimental Results}

1418: \label{sec:martinoliresults} Ijspeert {\em et al.} systematically

1419: studied the collaboration rate (the number of sticks successfully

1420: pulled out of the ground in a given time interval), and its

1421: dependence on the group size and the gripping time parameter. They

1422: found very good qualitative and quantitative agreement between the

1423: three different levels of experiments, as shown in

1424: Figure~\ref{fig:martinoliresults}. Their main observation was

1425: that, depending on the ratio of robots to sticks (or workers to

1426: the amount of work), there appear to be two different regimes in

1427: the collaboration dynamics. When there are fewer robots than

1428: sticks, the collaboration rate decreases to zero as the value of

1429: the gripping time parameter grows. In the extreme case, when the

1430: robot grabs a stick and waits indefinitely for another robot to

1431: come and help it, the collaboration rate is zero, because after

1432: some period of time each robot ends up holding a stick, and no

1433: robots are available to help. When there are more robots than

1434: sticks, the collaboration rate remains finite even in the limit

1435: the gripping time parameter becomes infinite, because there will

1436: always be robots available to help pull the sticks out. Another

1437: finding of Ijspeert {\em et al.} was that when there are fewer

1438: robots than sticks, there is an optimal value of the gripping time

1439: parameter which maximizes the collaboration rate. In the other

1440: regime, the collaboration rate appears to be independent of the

1441: gripping time parameter above a specific value, so the optimal

1442: strategy is for the robot to grip a stick and hold it

1443: indefinitely.

1444: 

1445: \begin{figure}[hptb]

1446: \epsfxsize = 3.0in \center{\epsffile{spresults.eps}}

1447: \caption{Collaboration rate vs. the gripping time parameter for

1448: groups of two to six robots and four sticks (from Ijspeert {\em et

1449: al}). Heavy symbols represent experimental results, symbols

1450: connected by lines are the results of sensor-based simulations,

1451: while the smooth heavy lines are the results of the probabilistic

1452: microscopic model.} \label{fig:martinoliresults}

1453: \end{figure}

1454: 

1455: In the following section we present a macroscopic mathematical

1456: model of the stick-pulling experiments in a homogeneous

1457: multi-robot system. Such a model is useful for the following

1458: reasons. First, the model is independent of the system size, {\em

1459: i.e.} the number of robots; therefore, solutions for a system of

1460: $5,000$ robots take just as long to obtain as solutions for $5$

1461: robots, whereas for a microscopic description the time required

1462: for simulation scales at least linearly with the number of robots.

1463: Second, our approach allows us to directly estimate certain

1464: important parameter values, ({\em e.g.}, those for which the

1465: performance is optimal) without having to resort to the time

1466: consuming simulations or experiments. It also enables us to study

1467: the stability properties of the system, and see whether solutions

1468: are robust under external perturbation or noise. These

1469: capabilities are important for the design and control of large

1470: multi-agent systems.

1471: 

1472: \subsection{The Rate Equation Model of Collaboration}

1473: In order to construct a mathematical model of collective behavior

1474: in stick-pulling experiments, it is helpful to draw the

1475: macroscopic state diagram of the system. On a macroscopic level,

1476: during a sufficiently short time interval, each robot will be in

1477: one of two states: $searching$ or $gripping$. Using the flowchart

1478: of the robots' controller (Fig.~\ref{fig:controller}) as

1479: reference, we include in the search state the set of behaviors

1480: associated with the looking for sticks mode, such as wandering

1481: around the arena (``look for sticks'' action), detecting objects

1482: and avoiding obstacles; while the gripping state is composed of

1483: decisions and an action inside the dotted box. We assume that

1484: actions ``success'' (pull the stick out completely) and

1485: ``release'' (release the stick) take place on a short enough time

1486: scale that they can be incorporated into the search state. Of

1487: course, there can be a discrete state corresponding to every

1488: action depicted in Fig.~\ref{fig:controller}, but this would

1489: complicate the mathematical analysis without adding much to the

1490: descriptive power of the model. While the robot is in the obstacle

1491: avoidance mode, it cannot detect and try to grip objects;

1492: therefore, avoidance serves to decrease the number of robots that

1493: are searching and capable of gripping sticks. We studied the

1494: effect of avoidance in \cite{Lerman01a} and found that it does not

1495: qualitatively change the results of the simpler model that does

1496: not include avoidance; therefore, we will leave it out for

1497: clarity.

1498: 

1499: 

1500: In addition to states, we must also specify all possible

1501: transitions between states. When it finds a stick, the robot makes

1502: a transition from the search state to the gripping state. After

1503: both a successful collaboration and when it times out

1504: (unsuccessful collaboration) the robot releases the stick and

1505: makes a transition into the searching state, as shown in

1506: Fig.~\ref{fig:macro}. These arrows correspond to the arrow

1507: entering and the two arrows leaving the dotted box in

1508: Fig.~\ref{fig:controller}. We will use the macroscopic state

1509: diagram as the basis for writing down the rate equations that

1510: describe the dynamics of the stick-pulling experiments. Note that

1511: the experimental system was a semi-Markov system, because the

1512: robot's transition from gripping to the searching state depended

1513: not only on its present state (gripping) but also on how long the

1514: robot has been in the gripping state.

1515: %%This property of the system

1516: %%can be captured by time-dependent transition rates.

1517: 

1518: 

1519: \begin{figure}[hptb]

1520: \epsfxsize = 2.0in \center{\epsffile{spmacro.eps}}

1521: \caption{Macroscopic state diagram of the multi-robot system. The

1522: arrow marked 's' corresponds to the transition from the gripping

1523: to the searching state after a successful collaboration, while the

1524: arrow marked 'u' corresponds to the transition after an

1525: unsuccessful collaboration, {\em i.e.}, when the robots releases

1526: the stick without a successful collaboration taking place. }

1527: \label{fig:macro}

1528: \end{figure}

1529: 

1530: We can simplify analysis by considering a modified version of the

1531: robot controller presented in Fig.~\ref{fig:controller}, where

1532: instead of waiting a specified period of time, each robot releases

1533: the stick with some probability per unit time. Note that this

1534: simplification restores the Markovian property of the system. We

1535: also construct and analyze a more complex model

1536: (Sec.~\ref{sec:gripping}) that explicitly includes the gripping

1537: time parameter. We show that the system based on the simplified

1538: controller produces qualitatively the same macroscopic behavior as

1539: the original system, which is modeled by the second and more

1540: complex model. Both systems are described by the same macroscopic

1541: state diagram and differ only in the details of the transition

1542: rate between the gripping and searching states.

1543: 

1544: %\subsection{The Dynamical Model}

1545: Each box in Fig.~\ref{fig:macro} corresponds to a macroscopic

1546: state and therefore requires a dynamic variable to describe it.

1547: Thus, the variables of our model are $N_s(t)$ and $N_g(t)$, the

1548: number of robots in the searching and gripping states

1549: respectively. Also, let  $M(t)$ be the number of uncollected

1550: sticks  at time $t$. The latter variable does not represent a

1551: macroscopic state, rather it tracks the state of the environment.

1552: The mathematical model of the stick-pulling experiments consists

1553: of a series of coupled rate equations, each describing how the

1554: dynamic variables evolve in time:

1555: \begin{eqnarray}

1556: \label{eq:ode1} \frac{dN_s}{dt}& =& -\alpha N_s(t)\biggl (

1557: M(t)-N_g(t)\biggr ) + \tilde{\alpha} N_s(t)N_g(t) +\gamma N_g(t) \,,\\

1558: \label{eq:odeN}

1559: N_0 & = & N_s + N_g  \,,\\

1560: \label{eq:odeM} \frac{dM}{dt}& = & -\tilde{\alpha}N_s(t)N_g(t) +

1561: \mu(t) \,,

1562: \end{eqnarray}

1563: where $\alpha$, $\tilde{\alpha}$ are the rates at which a

1564: searching robot encounters a stick and a gripping robot

1565: respectively, $\gamma$ is the rate at which robots release sticks

1566: and $\mu(t)$ is the rate at which new tasks are added. The

1567: parameters $\alpha$, $\tilde{\alpha}$, and $\gamma$ connect the

1568: model to the experiment: $\alpha$ and $\tilde{\alpha}$ are related

1569: to the size of the object, the robot's detection radius, or

1570: footprint, and the speed at which it explores the arena.

1571: 

1572: 

1573: The three terms in Eq.~\ref{eq:ode1} correspond to the three

1574: arrows between the states in Fig.~\ref{fig:macro}. The first term

1575: accounts for the decrease in the number of searching robots

1576: because some robots find and grip sticks; the second term

1577: describes the successful collaborations between two robots, and

1578: the third term accounts for the failed collaborations (\ie, when a

1579: robot releases a stick without a second robot present), both of

1580: which lead to an increase the number of searching robots. We do

1581: not need a separate equation for $N_g$, since this quantity may be

1582: calculated from the conservation of robots condition,

1583: Eq.~\ref{eq:odeN}. The last equation states that the number of

1584: sticks, $M(t)$, decreases in time at the rate of successful

1585: collaborations, Eq.~\ref{eq:odeM}. The equations are subject to

1586: the initial conditions that at $t=0$ the number of searching

1587: robots in $N_0$ and the number of sticks is $M_0$.

1588: 

1589: To proceed further, let us introduce

1590: \begin{eqnarray}

1591: \label{eq:sp-defs}

1592: n(t)&=&N_s(t)/N_0,\\

1593: m(t)&=&M(t)/M_0, \\

1594: \beta&=&N_0/M_0, \\

1595: R_G&=&\tilde{\alpha}/\alpha,\\

1596: \label{eq:sp-defsN} \tilde{\beta}&=&R_G \beta

1597: \end{eqnarray}

1598: and dimensionless time $t \rightarrow \alpha M_0 t$. Here $n(t)$

1599: is the fraction of robots in the search state and $m(t)$ is the

1600: fraction of uncollected sticks at time $t$. Due to the

1601: conservation of number of  robots, the fraction of robots in the

1602: gripping state is simply $1 - n(t)$. Equations

1603: \ref{eq:ode1}--\ref{eq:odeM} can be rewritten in dimensionless

1604: form as:

1605: \begin{eqnarray}

1606: \label{eq:ode2} \frac{dn}{dt}& = & -n(t)[m(t)+\beta

1607: n(t)-\beta]+\tilde{\beta}n(t)[1-n(t)] + \gamma [1-n(t)] \\

1608: \label{eq:ode3} \frac{dm}{dt}& = & -\beta

1609: \tilde{\beta}n(t)[1-n(t)] +\mu^{\prime}

1610: \end{eqnarray}

1611: Equations \ref{eq:ode2}--\ref{eq:ode3} together with initial

1612: conditions $n(0)=1$, $m(0)=1$ determine the dynamical evolution of

1613: the system. Note that only two parameters, $\beta$ and $\gamma$,

1614: appear in the equations and, thus, determine the behavior of

1615: solutions. The third parameter $\tilde{\beta}=R_G \beta$ is fixed

1616: experimentally and is not independent. Note that we do not need to

1617: specify $\alpha$ and $\tilde{\alpha}$ --- they enter the model

1618: only through $R_G$ (throughout this paper we will use

1619: $R_G=0.35$).\footnote{The parameter $\alpha$ can be easily

1620: calculated from experimental values quoted in

1621: \protect\cite{IMB2001}. As a robot travels through the arena, it

1622: sweeps out some area during time $dt$ and will detect objects that

1623: fall in that area. This detection area is $V_R W_R dt$, where

1624: $V_R=8.0\ cm/s$ is robot's speed, and $W_R=14.0\ cm$ is robot's

1625: detection width. If the arena radius is $R=40.0\ cm$, a robot will

1626: detect sticks at the rate $\alpha = V_R W_R /\pi R^2  = 0.02\

1627: s^{-1}$. According to \protect\cite{IMB2001}, a robot's

1628: probability to grab a stick already being held by another robot is

1629: $35\%$ of the probability of grabbing a free stick. Therefore,

1630: $R_G = \tilde{\alpha}/\alpha = 0.35$. $R_G$ is an experimental

1631: value obtained  with systematic experiments with two real robots,

1632: one holding the stick and the other one approaching the stick from

1633: different angles. }

1634: 

1635: 

1636: 

1637: %\subsubsection{Results}

1638: We assume that new sticks are added to the system at the rate that

1639: the robots pull them out; therefore, the number of sticks does not

1640: change with time ($m(t)=m(0)=1$). This  situation may be realized

1641: experimentally by replacing the sticks in their holes after they

1642: are pulled out by robots.  A steady-state solution, if it exists,

1643: describes the long term time-independent behavior of the system.

1644: To find it, we set the left hand side of Eq.~\ref{eq:ode2} to

1645: zero:

1646: \begin{equation}

1647: \label{eq:steady-state} -n[1+\beta n-\beta]+\tilde{\beta}n[1-n] +

1648: \gamma [1-n]=0.

1649: \end{equation}

1650: \noindent This quadratic equation can be solved to obtain steady

1651: state values of $n(\beta,\gamma)$.

1652: 

1653: Figure \ref{fig:sp_steady_state}(a) shows the dependence of the

1654: fraction of searching robots in the steady state on the parameters

1655: $\beta$ and $\gamma$. The x-axis has units of time, although the

1656: scale is different than in Fig.~\ref{fig:martinoliresults}. Note,

1657: that for small enough $\beta$'s $n(\gamma)\rightarrow 0$ as

1658: $1/\gamma \rightarrow \infty$. The intuitive reason for this is

1659: the same one given in Section~\ref{sec:martinoliresults}: when

1660: there are fewer robots than sticks, and each robot holds the stick

1661: indefinitely (vanishing release probability), after a while every

1662: robot is holding a stick, and no

1663: robots are searching. For larger values of $\beta$, %> 1/(1+R_G),

1664: however, $n(\gamma)\rightarrow const \neq 0$ as $1/\gamma

1665: \rightarrow \infty$. Figure~\ref{fig:sp_steady_state}(b) shows how

1666: a typical solution $n(t)$ relaxes to its steady state value.

1667: 

1668: \begin{figure}[tp]

1669: \begin{tabular}{cc}

1670: \epsfxsize = 2.1in {\epsffile{sp-ss2.eps}} & \epsfxsize =

1671: 2.1in {\epsffile{sp-relaxation.eps}}\\

1672: (a)&(b)\\

1673: \end{tabular}

1674: %\vskip 0.1in

1675: \caption{ (a) Steady state solution vs inverse stick release rate

1676: $1/\gamma$. (b) Typical relaxation  to the steady state of the

1677: fraction of searching robots for $\gamma=0.2$, $\beta=0.5$.}

1678: \label{fig:sp_steady_state}

1679: \end{figure}

1680: 

1681: \begin{figure}[thbp]

1682: \epsfxsize = 3.2in \center{\epsffile{sp-coll2.eps}} \vskip 0.1in

1683: \caption{ Collaboration rate per robot vs inverse stick release

1684: rate $1/\gamma$ for $\beta=0.5$, $\beta=1.0$, $\beta=1.5$. These

1685: values of $\beta$ correspond, respectively, to two, four, and six

1686: robots in the experiments with four sticks ({\em cf.}

1687: Fig.~\protect{\ref{fig:martinoliresults}}).}

1688: \label{fig:sp_collaboration}

1689: \end{figure}

1690: 

1691: The collaboration rate is the rate at which robots successfully

1692: pull sticks out of their holes. The steady-state collaboration

1693: rate $R(\gamma;\beta)$ is given by the following equation:

1694: \begin{equation}

1695: \label{eq:collaboration} R(\gamma,\beta)=\beta \tilde{\beta}

1696: n(\gamma,\beta)[1-n(\gamma,\beta)]\,,

1697: \end{equation}

1698: \noindent where $n(\gamma,\beta)$ is the steady-state number of

1699: searching robots for a particular value of $\gamma$ and $\beta$,

1700: and $(1 - n(\gamma, \beta))$ is the steady-state number of

1701: gripping robots.  Figure \ref{fig:sp_collaboration} depicts the

1702: collaboration rate as a function of $1/\gamma$. For $\beta >

1703: \beta_c$ the collaboration rate increases monotonically with

1704: $1/\gamma$. However, for

1705:  $\beta < \beta_c$ there is an optimal stick release rate which

1706: maximizes the collaboration rate. The optimal value of $\gamma$

1707: which maximizes the collaboration rate can be computed from the

1708: condition $dR(\gamma, \beta)/d\gamma=\beta \tilde{\beta}

1709: d(n-n^2)/d \gamma=0$, with $n$ given by roots of

1710: Eq.~\ref{eq:steady-state}. Another way to compute the optimal

1711: release rate is by noting that for a given value of $\beta$ below

1712: some critical value, the collaboration rate is greatest when half

1713: of the robots are gripping and the other half are searching.

1714: Substituting $n=1/2$ into Eq.~\ref{eq:steady-state}, leads to

1715: $\gamma_{opt}=1-(\beta+\tilde{\beta})/2$. $\gamma_{opt}$ vanishes

1716: as $\beta$ exceeds its critical value, $\beta_c=2/(1+R_G)$;

1717: therefore, for $\beta >\beta_c$, $n>1/2$, and no optimal release

1718: rate exists.

1719: 

1720: 

1721: The three curves in Fig.~\ref{fig:sp_collaboration} are

1722: qualitatively similar to those in Fig.~\ref{fig:martinoliresults}

1723: for $2$ robots ($\beta = 0.5$), $4$ robots ($\beta = 1.0$) and $6$

1724: robots ($\beta = 1.5$). Even the grossly simplified model

1725: reproduces the following conclusions of Ijspeert {\em et al.}: the

1726: different dynamical regimes depending on the value of the ratio of

1727: robots to sticks ($\beta$) and the optimal gripping time parameter

1728: for $\beta < \beta_c$.

1729: 

1730: In the next section, we construct a model of the stick pulling

1731: experiments that explicitly includes the gripping time parameter

1732: $\tau$. The collective behavior predicted by the more accurate

1733: model is both qualitatively and quantitatively similar to that

1734: predicted by the simplified model.

1735: 

1736: 

1737: 

1738: 

1739: \subsection{Model with Gripping Time Parameter}

1740: \label{sec:gripping} A more accurate mathematical description of

1741: the stick pulling experiments should explicitly includes the

1742: gripping time parameter $\tau$. Note that the system is now a

1743: semi-Markov system, because transitions from gripping to the

1744: searching state depend not only on the present state (gripping)

1745: but also on how long the robot has been in the gripping state,

1746: {\em i.e.}, whether or not it has timed out. This property of the

1747: system will be captured by time-dependent transition rates. The

1748: system is described by the same macroscopic state diagram,

1749: Fig.~\ref{fig:macro}, and the same set of equations,

1750: Eq.~\ref{eq:ode1}--\ref{eq:odeM}, with only the last term in

1751: Eq.~\ref{eq:ode1} different. We write the new equation below:

1752: \begin{eqnarray}

1753: \label{eq:ode1a}

1754: \frac{dN_s}{dt}& =& -\alpha N_s(t)\biggl ( M(t)-N_g(t)\biggr ) + \tilde{\alpha} N_s(t)N_g(t) \nonumber \\

1755: & & +\alpha N_s(t-\tau)\biggl ( M(t-\tau)-N_g(t-\tau)\biggr )

1756: \Gamma(t;\tau).

1757: \end{eqnarray}

1758: All the parameters have the same meaning as before.

1759: $\Gamma(t;\tau)$, the fraction of failed collaborations at time

1760: $t$, is the probability no robot came ``to help'' during the time

1761: interval $[t-\tau,t]$. This is a time-dependent parameter, and it

1762: describes unsuccessful transitions from the gripping state in this

1763: semi-Markov system. To calculate $\Gamma(t;\tau)$ let us divide

1764: the time interval $[t-\tau,t]$ into $K$ small intervals of length

1765: $\delta t=\tau/K$. The probability that no robot comes to help

1766: during the time interval $[t-\tau,t-\tau+\delta t]$ is simply

1767: $1-\tilde{\alpha} N_s(t-\tau)\delta t$. Hence, the probability for

1768: a failed collaboration is

1769: \begin{eqnarray}

1770: \Gamma(t;\tau)& =& \prod_{i=1}^{K}[1-\tilde{\alpha}\delta t N_s(t-\tau+i\delta t )] \Theta (t-\tau) \nonumber \\

1771: \label{fc1} & \equiv & \exp\biggl [\sum_{i=1}^{K}

1772: \ln[1-\tilde{\alpha}\delta t N_s(t-\tau+i \delta t)]\biggr ]

1773: \Theta (t-\tau)

1774: \end{eqnarray}

1775: The step function $\Theta(t-\tau)$ ensures that $\Gamma (t; \tau)$

1776: is zero for $t<\tau$. Finally, expanding the logarithm in

1777: Eq.~(\ref{fc1}) and taking the limit $\delta t \rightarrow 0$ we

1778: obtain

1779: \begin{equation}

1780: \label{fc2}

1781: \Gamma(t;\tau)=\exp[-\tilde{\alpha}\int_{t-\tau}^{t}dt^{\prime}N_s(t^{\prime})]

1782: \Theta(t - \tau)

1783: \end{equation}

1784: 

1785: 

1786: We rewrite the model in dimensionless form using variable

1787: transformations given by Eqs.~\ref{eq:sp-defs}--\ref{eq:sp-defsN}

1788: and dimensionless gripping time parameter $\tau \rightarrow \alpha

1789: M_0 \tau$:

1790: \begin{eqnarray}

1791: \label{eq:ode2a} \frac{dn}{dt}& = & -n(t)[m(t)+\beta

1792: n(t)-\beta]+\tilde{\beta}n(t)[1-n(t)]\nonumber \\

1793: & & + n(t-\tau)[m(t-\tau)+\beta n(t-\tau)-\beta]\times \gamma(t;\tau) \\

1794: \label{eq:ode3a}

1795: \frac{dm}{dt}& = & -\beta \tilde{\beta}n(t)[1-n(t)] +\mu^{\prime}\\

1796: \label{eq:gamma} \gamma(t;\tau)& = &

1797: \exp[-\tilde{\beta}\int_{t-\tau}^{t}dt^{\prime}n(t^{\prime})]

1798: \end{eqnarray}

1799: 

1800: Equations \ref{eq:ode2a}--\ref{eq:gamma} are solved subject to

1801: initial conditions $n(0)=1$ and $m(0)=1$ to determine the dynamic

1802: evolution of the system. Parameters $\beta$ and $\tau$ alone

1803: appear in the equations and thus, determine the behavior of the

1804: system.

1805: 

1806: %\subsubsection{Results}

1807: Equation \ref{eq:ode2a} has a non-trivial steady-state solutions

1808: which satisfy the following transcendental equation:

1809: \begin{equation}

1810: \label{trans}

1811: -1+(\beta+\tilde{\beta})(1-n)+(1-\beta(1-n))e^{-\tilde{\beta}\tau

1812: n}=0

1813: \end{equation}

1814: Figure \ref{fig:sp_static_steady_state} shows the dependence of

1815: the steady state fraction of searching robots on the gripping time

1816: $\tau$ for different values of $\beta$. Note, that for small

1817: enough $\beta$'s $n(\tau)\rightarrow 0$ as $\tau \rightarrow

1818: \infty$, {\em i.e.}, when there are fewer robots than sticks, and

1819: each robot holds the stick indefinitely, after a while every robot

1820: is holding a stick, and no robots are searching. For $\beta >

1821: 1/(1+R_G)$, however, $n(\tau)\rightarrow const \neq 0$ as $\tau

1822: \rightarrow \infty$. The inset in

1823: Fig.~\ref{fig:sp_static_steady_state} shows how a typical

1824: solution, $n(t)$, relaxes to its steady state value. The

1825: oscillations are characteristic of time-delay differential

1826: equations, and their period is determined by $\tau$.

1827: 

1828: \begin{figure}[tp]

1829: \epsfxsize = 3.2in \center{\epsffile{spfig1.eps}} \vskip 0.1in

1830: \caption{ Steady state solution vs (dimensionless) gripping time

1831: parameter $\tau$: for $\beta=0.5$ (short dash), $1$ (long dash),

1832: $1.5$ (solid line). Inset shows a typical relaxation to the steady

1833: state for $\tau=5$, $\beta=0.5$.}

1834: \label{fig:sp_static_steady_state}

1835: \end{figure}

1836: 

1837: \begin{figure}[bp]

1838: \epsfxsize = 3.2in \center{\epsffile{coll-tau.eps}} \vskip 0.1in

1839: \caption{ Collaboration rate per robot vs (dimensionless) gripping

1840: time parameter $\tau$ for $\beta=0.5$ (short dash), $\beta=1$

1841: (long dash), $\beta=1.5$ (solid line). These values of $\beta$

1842: correspond, respectively, to two, four, and six robots in the

1843: experiments with four sticks ({\em cf.}

1844: Fig.~\protect{\ref{fig:martinoliresults}}).}

1845: \label{fig:sp_static_collaboration}

1846: \end{figure}

1847: 

1848: The steady--state collaboration rate $R(\tau;\beta)$, the rate at

1849: which robots pull sticks out of their holes, is given by:

1850: %\begin{equation}

1851: %\label{eq:collaboration}

1852: $R(\tau,\beta)=\beta \tilde{\beta} n(\tau,\beta)[1-n(\tau,\beta)]$

1853: %\,,

1854: %\end{equation} \noindent

1855: where $n(\tau,\beta)$ is the number of searching robots in the

1856: steady-state for a particular value of $\tau$ and $\beta$ (and $(1

1857: - n(\tau, \beta))$ is the number of gripping robots in the

1858: steady-state). Figure \ref{fig:sp_static_collaboration} depicts

1859: collaboration rate as a function of $\tau$. For $\beta > \beta_c$

1860: the collaboration rate increases monotonically with $\tau$.

1861: However, for $\beta < \beta_c$ there is an optimal gripping time,

1862: $\tau=\tau_{opt}$, which maximizes the collaboration rate. We use

1863: the same arguments as before to understand this behavior: maximum

1864: collaboration rate for a given $\beta$ is achieved for

1865: $n(\tau,\beta)=1/2$. For $\beta > \beta_c$, however, the solution

1866: of  Eq.~\ref{trans} is always greater than $1/2$, so an optimal

1867: solution does not exist. For $\beta < \beta_c$, simple analysis

1868: gives

1869: \begin{equation}

1870: \tau_{opt}=\frac{2}{\tilde{\beta}}\ln\frac{1-

1871: \beta/2}{1-1/2(\beta+\tilde{\beta})}, \ \ \beta <

1872: \beta_c=\frac{2}{1+R_G}

1873: \end{equation}

1874: \noindent This dependence on $R_G$ was quantitatively confirmed

1875: through embodied and microscopic

1876: simulations~\cite{MartinoliEaston02}. Figure \ref{fig:optimal}

1877: compares the optimal gripping time parameter and inverse of the

1878: optimal release rate predicted by the two models. For $\beta < 1$

1879: the two models give $quantitatively$ similar results, in addition

1880: to predicting the same $\beta_c$. This example illustrates our

1881: claim that in many cases a minimal model is sufficient to explain

1882: and predict interesting system properties. Ref. \cite{Lerman01a}

1883: presents more details of the analysis of the collaboration task,

1884: including the effects of interference.

1885: 

1886: 

1887: \begin{figure}[tp]

1888: \epsfxsize = 2.0in \center{\epsffile{optimals.eps}} \vskip 0.1in

1889: \caption{ Optimal gripping time parameter and inverse of the

1890: optimal release rate vs $\beta$ in the two models}

1891: \label{fig:optimal}

1892: \end{figure}

1893: 

1894: 

1895: 

1896: 

1897: 

1898: \subsection{Difference Equation Model of Collaboration}

1899: Martinoli and Easton~\cite{MartinoliISER02,MartinoliEaston02}

1900: consider a more fine-grained macroscopic model than the one

1901: described above that takes into account more of the individual

1902: robot behaviors shown in Fig.~\ref{fig:controller}. Their model

1903: consists of coupled finite difference equations, one for each

1904: state: searching ($N_s$), avoiding ($N_a$), interference ($N_i$),

1905: success dance ($N_d$), and gripping ($N_g$). The equation for how

1906: the number for searching robots changes in time is presented

1907: below; equations for other variables are similar.

1908: \begin{eqnarray*}\label{eqn:collaboration-difference}

1909:   N_S(k+1) & = & N_s(k) - (\alpha_w+\alpha_r)N_s(k)

1910:   -\tilde{\alpha}N_g(k)N_s(k) \\

1911:   & - & \alpha (M_0-N_g(k)N_s(k)  +  \alpha_w N_s(k-T_a) \\

1912:   & + & \alpha_r N_s(k-T_{ia}) + \tilde{\alpha}N_g(k-T_{ca})N_s(k-T_{ca})\\

1913:   & + & \tilde{\alpha}N_g(k-T_{cda})N_s(k-T_{cda})\\

1914:   & + & \alpha(M_0-N_g(k-T_{cda}))N_s(k-T_{cga})\Gamma(k,T_{ga}).

1915: \end{eqnarray*}

1916: Here, $\alpha_w$ and $\alpha_r$ are the rates at which robots

1917: encounter a wall or another robot. The current time step is $k$,

1918: $k=0, 1, 2, \ldots$, and $T_{xyz}=T_x+T_y+T_z$ are the number of

1919: time steps required to complete actions such as avoidance ($T_a$),

1920: success dance ($T_d$) or stick centering ($T_c$).

1921: $\Gamma(k,T_{ga})=\Pi_{j=k-T_{ga}}^{k}{[1-\tilde{\alpha}N_s(j)}\Theta(k-T_{ga})$

1922: is the discrete time analog of Eq.~\ref{fc2}.

1923: 

1924: \begin{figure}[tp]

1925: \epsfxsize = 3.2in \center{\epsffile{web_micro_macro_80cm.eps}}

1926: \vskip 0.1in \caption{ Collaboration rate as a function of the

1927: gripping time parameter for several robot group sizes. Results are

1928: for embodied simulations (dotted lines), the microscopic model

1929: (dashed lines), and the macroscopic model (solid lines) in the 16

1930: stick 80 cm radius arena. (Figure courtesy of A.~Martinoli) }

1931: \label{fig:collaboration-difference}

1932: \end{figure}

1933: 

1934: Figure \ref{fig:collaboration-difference} shows collaboration rate

1935: as a function of the gripping time parameter for different robot

1936: group sizes. Collaboration rate is computed from the number of

1937: extracted sticks per unit time, as before:

1938: $C(k)=\tilde{\alpha}N_s(k-T_{cd})N_g(k-T_{cd})$. We can see that

1939: for groups as small as 8 robots, the results of the macroscopic

1940: model quantitatively agree with embodied and microscopic

1941: simulations.

1942: 

1943: 

1944: %\input{conclusion}

1945: \comment{

1946: \section{Related Work}

1947: \label{sec:related}

1948: 

1949: With the exceptions noted below, there has been very little prior

1950: work on mathematical analysis of multi-agent systems. The closest

1951: in spirit to our paper is the work by Huberman, Hogg and coworkers

1952: on computational ecologies~\cite{HubermanHogg88,Kephart90}. These

1953: authors mathematically studied collective behavior in a system of

1954: agents, each choosing between two alternative strategies. They

1955: derived a rate equation for the average number of agents using

1956: each strategy from the underlying probability distributions. Our

1957: approach is consistent with theirs --- in fact, we can easily

1958: write down the same rate equations from the macroscopic state

1959: diagram of the system, without having to derive them from the

1960: underlying probability distributions. Computational ecologies can,

1961: therefore, be considered an application of the methodology

1962: described in this paper. Yet another application of the approach

1963: presented here is the author's work on coalition formation in

1964: electronic marketplaces \cite{Lerman00a}.

1965: 

1966: In the robotics domain, Sugawara and

1967: coworkers~\cite{Sugawara97,SugSanYosAbe98} developed simple

1968: state-based analytical models of cooperative foraging in groups of

1969: communicating  and non-co\-mmu\-ni\-cating robots and studied them

1970: quantitatively. Although these models are similar to ours, they

1971: are overly simplified and fail to take crucial interactions among

1972: robots into account. In separate papers, we have analyzed

1973: collaborative~\cite{Lerman01a} and foraging~\cite{Lerman02a}

1974: behavior in groups of robots. The focus of that work is on

1975: realistic models and the comparison of the models' predictions to

1976: experimental and simulations results. For example, in

1977: ~\cite{Lerman01a}, we considered the same model of collaborative

1978: stick-pulling presented here, but studied it under the same

1979: conditions as the experiments. In \cite{Lerman02a}, we found that

1980: we had to include avoiding-while-searching and wall-avoiding

1981: states in the model in order to obtain good quantitative agreement

1982: between the model and results of sensor-based simulations. The

1983: focus of this paper, on the other hand, is to show that there is a

1984: principled way to construct a macroscopic model of collective

1985: dynamics of a MAS, and, more importantly, a practical ``recipe''

1986: for creating such a model from the details of the microscopic

1987: controller.

1988: 

1989: }

1990: 

1991: 

1992: \section{Limitations of this approach}

1993: 

1994: The rate equation approach we presented is valid for Markov and

1995: semi-Markov systems, in which the agent's future state depends

1996: only on its present state and, for semi-Markov systems, on how

1997: much time it has spent in that state, and not on any of the past

1998: states. While many systems, including reactive and behavior-based

1999: robotics and some software agent systems, clearly obey the Markov

2000: property, other systems composed of agents with memory, learning

2001: or deliberative capabilities do not, and therefore, cannot be

2002: described by the simple models presented here. However, the rate

2003: equations are useful for studying systems of simple agents.

2004: Moreover, with some additional complexity, this approach can be

2005: extended to cover a broader range of agent behaviors. For

2006: instance, in simple learning scenarios, agents could adjust their

2007: behaviors based on the number of times they enter certain states.

2008: For instance, to estimate the density of objects or other robots

2009: in their environment based on the number of encounters. Thus the

2010: transition rates would depend on the agent histories. Although the

2011: rate equations wouldn't model the learning process per se, it

2012: could include the effect on system behavior of improved

2013: discrimination ability. Such a model could indicate the likely

2014: tradeoff between exploring the environment to improve parameter

2015: estimates and exploit those estimates for the task at hand.

2016: 

2017: Another potential limitation of the approach is that it is best as

2018: a description of large systems. The benefit of working with large

2019: systems is that, usually, one may safely ignore fluctuations.

2020: Although it is possible to use the Master Equation to derive the

2021: equation for the fluctuations about the average quantities, in

2022: practice it is too algebraically messy. Fortunately, there exists

2023: empirical evidence \cite{Pacala,MartinoliEaston02} that

2024: approximate average models can provide a good quantitative

2025: description of systems as small as a dozen agents. Large

2026: collections of very simple agents are also seen to be reasonably

2027: described by relations among a few aggregate

2028: variables~\cite{bohringer97}.

2029: 

2030: Developing a suitable stochastic model for a multi-agent system

2031: requires an understanding of the environment and agent actions

2032: sufficient to determine the appropriate state description and

2033: resulting transition rates. More generally, this issue can be

2034: viewed as identifying an appropriate statistical ensemble, i.e.,

2035: set of states with associated probabilities for their

2036: occurrence~\cite{hogg87PhysRep}. Even oversimplified models can be

2037: useful in giving qualitative understanding of the likely design

2038: tradeoffs prior to a more detailed evaluation via simulation or

2039: experiments. Nevertheless, it can be difficult to determine the

2040: transition rates between states, especially if they involve

2041: correlated activities among several agents. Along these lines is

2042: the question of to what extent a detailed model of the agent

2043: behaviors must be included. Complex agent programs could give rise

2044: to complex transition rates that make the analysis described here

2045: impractical. However, even in those cases, there may be some

2046: aggregate behaviors that can be approached by suitable

2047: coarse-graining. An extreme example is economics, which predicts

2048: various aggregate human behaviors without requiring detailed

2049: psychological models.

2050: 

2051: Beyond these general limitations of the stochastic approach

2052: described here, we introduced several simplifying assumptions.

2053: While these are not strictly necessary for the validity of the

2054: overall approach, they are important for producing analytically

2055: simple models. In particular we suppose the interesting system

2056: behavior is governed by averages and the transition rates are

2057: spatially uniform so it is not necessary to include position as

2058: part of the state. We also extensively use the mean-field

2059: approximation. In cases where this is not sufficient, more

2060: accurate approximations of our statistical models are

2061: possible~\cite{opper01}, but are more difficult to evaluate.

2062: 

2063: When evaluating the suitability of stochastic models, in addition

2064: to these technical limitations, it is important to note the kinds

2065: of results the models can deliver. In particular, they address

2066: properties of the distribution of outcomes, \eg, average and

2067: variance, over a set of repeated experiments. This is often

2068: appropriate for evaluating how a multi-agent system will likely

2069: perform for a class of problems. However, if one is interested in

2070: worst-case bounds, the behavior in exceptional situations or

2071: extreme values of the distribution (\eg, time required to find the

2072: first or last object in a search scenario), these stochastic

2073: techniques are unlikely to provide much insight unless the

2074: distributions happen to be limited to a narrow range. Thus the

2075: usefulness of these models depends not only on the complexity of

2076: the agent behaviors but also on the nature of the collective

2077: properties of interest.

2078: 

2079: 

2080: \section{Conclusion}

2081: \label{sec-Conclusion} We have presented an overview of

2082: mathematical approaches for modeling and analysis of multi-agent

2083: systems, focusing on macroscopic analytic models. Moreover, we

2084: have described a general methodology for mathematical analysis of

2085: such systems. Our analysis applies to a class of systems known as

2086: stochastic Markov systems. They are stochastic, because the

2087: behavior of each agent is inherently probabilistic in nature and

2088: unpredictable, and they are Markovian, because the state of an

2089: agent at a future time depends only on the present state (and

2090: perhaps on much time the agent has spent in this state) and not on

2091: any past state. Though each agent is unpredictable, the

2092: probabilistic description of the collective behavior is

2093: surprisingly simple. Our mathematical approach is based on the

2094: stochastic Master Equation, and on the Rate Equations derived from

2095: it, that describe how the average macroscopic, or collective,

2096: system properties change in time. In order to create a

2097: mathematical model, one needs to account for every relevant state

2098: of the multi-agent system as well as for transitions between

2099: states. For each state there is a dynamical variable in the

2100: mathematical model and a rate equation that describes how the

2101: variable changes in time.

2102: 

2103: We illustrated the mathematical formalism by applying it to study

2104: collective behavior of robotic systems. Even the simplest type of

2105: dimensional analysis of the equations yields important insights

2106: into the system, such as what are the important parameters that

2107: determine the behavior of the system.

2108: 

2109: 

2110: 

2111: In the applications, we focused on the simplest mathematical

2112: models ({\em i.e.,} those using the smallest possible number of

2113: states) that capture the salient features of each system. These

2114: simple models provide a good description of the $qualitative$

2115: behavior of the system, but in order to also obtain good

2116: $quantitative$ agreement with experiment or simulations, it is

2117: often necessary to include more states in the model. The approach

2118: presented here can be easily extended to describe heterogeneous

2119: agent systems. As a simple example, consider two, possibly

2120: interacting, populations of foraging robots, each described by

2121: different physical parameters. The model of a heterogeneous

2122: foraging system will consist of two sets of coupled differential

2123: equations, one for each sub-population, possibly with couplings

2124: between them to represent interactions between the two

2125: populations. It is trivial to extend the analysis to more than two

2126: populations.

2127: 

2128: The models also provide design guidelines: choosing the agent

2129: behaviors to closely match the simplifying assumptions of this

2130: approach allows evaluating the resulting collective behavior via

2131: the rate equations.

2132: 

2133: \section*{Acknowledgements}

2134: The authors thank Dani Goldberg, Bernardo Huberman, Jeff Kephart,

2135: Alcherio Martinoli, Maja Matari{\'c}, Richard Ross and Onn Shehory

2136: for useful discussions.

2137: 

2138: % ---- Bibliography ----

2139: %

2140: %\bibliographystyle{plain}

2141: %\bibliography{agents,robots,physics,lerman,tad}

2142: \begin{thebibliography}{10}

2143: 

2144: \bibitem{abelson99}

2145: Harold Abelson et~al.

2146: \newblock Amorphous computing.

2147: \newblock {\em Communications of the ACM}, 43:74--82, May 2000.

2148: 

2149: \bibitem{Agassounon02}

2150: William Agassounon and Alcherio Martinoli.

2151: \newblock A macroscopic model of an aggregation experiment using embodied

2152:   agents in groups of time-varying sizes.

2153: \newblock In {\em Proc. of the IEEE Conf. on System, man and Cybernetics

2154:   SMC-02, October 2002, Hammamet, Tunisia. To appear.} 2002.

2155: 

2156: \bibitem{Arkin}

2157: Ronald~C. Arkin.

2158: \newblock {\em Behavior-Based Robotics}.

2159: \newblock The MIT Press, Cambridge, MA, 1999.

2160: 

2161: \bibitem{BarabasiStanley95}

2162: A.-L. Barabasi and H.E. Stanley.

2163: \newblock {\em Fractal Concepts in Surface Growth}.

2164: \newblock Cambridge University Press, 1995.

2165: 

2166: \bibitem{Beckers94}

2167: R.~Beckers, O.~E. Holland, and J.~L. Deneubourg.

2168: \newblock From local actions to global tasks: Stigmergy and collective

2169:   robotics.

2170: \newblock In Rodney~A. Brooks and Pattie Maes, editors, {\em Proceedings of the

2171:   4th International Workshop on the Synthesis and Simulation of Living Systems

2172:   \(Artificial Life {IV}\)}, pages 181--189, Cambridge, MA, USA, July 1994. MIT

2173:   Press.

2174: 

2175: \bibitem{bender78}

2176: Carl~M. Bender and Steven~A. Orszag.

2177: \newblock {\em Advanced Mathematical Methods for Scientists and Engineers}.

2178: \newblock McGraw Hill, NY, 1978.

2179: 

2180: \bibitem{Beni88}

2181: G.~Beni.

2182: \newblock The concept of cellular robotics.

2183: \newblock In {\em Proc. of the 1988 IEEE Int. Symp. on Intelligent Control},

2184:   pages 57--62, Los Alamitos, CA, 1988. IEEE Computer Society Press.

2185: 

2186: \bibitem{berlin97}

2187: Andrew~A. Berlin and Kaigham~J. Gabriel.

2188: \newblock Distributed {MEMS}: New challenges for computation.

2189: \newblock {\em Computational Science and Engineering}, 4(1):12--16,

2190:   January-March 1997.

2191: 

2192: \bibitem{bohringer97}

2193: Karl~F. Bohringer et~al.

2194: \newblock Computational methods for design and control of {MEMS}

2195:   micromanipulator arrays.

2196: \newblock {\em Computational Science and Engineering}, 4(1):17--29,

2197:   January-March 1997.

2198: 

2199: \bibitem{bojinov02}

2200: Hristo Bojinov, Arancha Casal, and Tad Hogg.

2201: \newblock Multiagent control of modular self-reconfigurable robots.

2202: \newblock {\em Artificial Intelligence}, 142:99--120, 2002.

2203: \newblock Preprint available at {Los Alamos} archive cs.RO/0006030.

2204: 

2205: \bibitem{Bonabeau97}

2206: Eric Bonabeau.

2207: \newblock From classical models of morphogenesis to agent-based models of

2208:   pattern formation.

2209: \newblock {\em Artifical Life}, 3:191--211, 1997.

2210: 

2211: \bibitem{BDT99}

2212: Eric Bonabeau, Marco Dorigo, and Guy Theraulaz.

2213: \newblock {\em Swarm Intelligence: From Natural to Artificial Systems}.

2214: \newblock Oxford University Press, New York, 1999.

2215: 

2216: \bibitem{CSS00}

2217: D.~Chowdhury, L.~Santen, and A.~Schadschneider.

2218: \newblock Statistical physics of vehicular traffic and some related systems.

2219: \newblock {\em Physics Reports}, 329:199, 2000.

2220: 

2221: \bibitem{clearwater96}

2222: Scott~H. Clearwater, editor.

2223: \newblock {\em Market-Based Control: A Paradigm for Distributed Resource

2224:   Allocation}.

2225: \newblock World Scientific, Singapore, 1996.

2226: 

2227: \bibitem{courtois85}

2228: P.~J. Courtois.

2229: \newblock On time and space decomposition of complex structures.

2230: \newblock {\em Communications of the ACM}, 28(6):590--603, June 1985.

2231: 

2232: \bibitem{Cushing}

2233: J.~M. Cushing.

2234: \newblock {\em An Introduction to Structured Population Dynamics}, volume~71 of

2235:   {\em CBMS-NSF Regional Conference Series in Applied Mathematics}.

2236: \newblock Society for Applied and Industrial Mathematics, Philadelphia, 1998.

2237: 

2238: \bibitem{Deneubourg92}

2239: Jean-Luis Deneubourg, Guy Theraulaz, and R.~Beckers.

2240: \newblock Swarm-made architectures. in: Toward a practice of autonomous

2241:   systems.

2242: \newblock In F.J. Varela and P.~Bourgine, editors, {\em Proceedings of The

2243:   First European Conference on Artificial Life}, pages 123--133, Cambridge, MA,

2244:   1992. MIT Press/Bradford Books.

2245: 

2246: \bibitem{epidemiology}

2247: O.~Diekmann and J.~A.~P. Heesterbeek.

2248: \newblock {\em Mathematical Epidemiology of Infectious Diseases : Model

2249:   Building, Analysis and Interpretation}.

2250: \newblock John Wiley \& Sons, New York, mathematical and computational biology

2251:   edition, April 2000.

2252: 

2253: \bibitem{RobDuncan}

2254: R.~Dun\-can et~al.

2255: \newblock Statistical paradigms for robotic swarm modeling.

2256: \newblock http://www.mhpcc.edu/research/ab98/98ab41.html, 1998.

2257: 

2258: \bibitem{FonMat96}

2259: Miguel~Schneider Fontan and Maja~J Matari{\'c}.

2260: \newblock A study of territoriality: The role of critical mass in adaptive task

2261:   division.

2262: \newblock In P.~Maes, M.~J. Matari{\'c}, J.~A. Meyer, J.~Pollack, and

2263:   S.~Wilson, editors, {\em From Animals to Animats 4: Proceedings of the 4th

2264:   International Conference on Simulation of Adaptive Behavior}, pages 553--561,

2265:   Cambridge, MA, 1996. MIT Press.

2266: 

2267: \bibitem{freitas99}

2268: Robert~A. {Freitas Jr.}

2269: \newblock {\em Nanomedicine}, volume~1.

2270: \newblock Landes Bioscience, Georgetown, TX, 1999.

2271: \newblock Available at www.nanomedicine.com.

2272: 

2273: \bibitem{Gardiner}

2274: C.~W. Garnier.

2275: \newblock {\em Handbook of Stochastic Methods}.

2276: \newblock Springer, New York, NY, 1983.

2277: 

2278: \bibitem{StagePlayer}

2279: Brian~P. Gerkey, Richard~T. Vaughan, Kasper Støy, Andrew Howard,

2280: Gaurav~S.

2281:   Sukhatme, and Maja~J Matari{\'c}.

2282: \newblock Most valuable player: A robot device server for distributed control.

2283: \newblock In {\em Proc. of the {IEEE/RSJ} International Conference on

2284:   Intelligent Robots and Systems (IROS 2001), Wailea, Hawaii, October 29 -

2285:   November 3, 2001}. 2001.

2286: \newblock http://www-robotics.usc.edu/player/.

2287: 

2288: \bibitem{GolMat00}

2289: Dani Goldberg and Maja~J Matari{\'c}.

2290: \newblock Robust behavior-based control for distributed multi-robot collection

2291:   tasks.

2292: \newblock Technical Report IRIS-00-387, USC Institute for Robotics and

2293:   Intelligent Systems, 2000.

2294: 

2295: \bibitem{ecology}

2296: W.~S.~C. Gurney and R.~M. Nisbet.

2297: \newblock {\em Ecological Dynamics}.

2298: \newblock Oxford University Press, New York, NY, 1998.

2299: 

2300: \bibitem{Haberman}

2301: Richard Haberman.

2302: \newblock {\em Mathematical Models: Mechanical Vibrations, Population Dynamics,

2303:   and Traffic Flow}.

2304: \newblock Society of Industrial and Applied Mathematics ({SIAM}), Philadelphia,

2305:   PA, 1998.

2306: 

2307: \bibitem{Helbing}

2308: Dirk Helbing.

2309: \newblock {\em Quantitative Sociodynamics: Stochastic Methods and Models of

2310:   Social Interaction Processes}, volume~31 of {\em THEORY AND DECISION LIBRARY

2311:   B: Mathematical and Statistical Methods}.

2312: \newblock Kluwer Academic, Dordrecht, 1995.

2313: 

2314: \bibitem{HelbingSchweitzer}

2315: Dirk Helbing, Frank Schweitzer, Joachim Keltsch, and Peter Molnar.

2316: \newblock Active walker model for the formation of human and animal trail

2317:   systems.

2318: \newblock {\em Physical Review}, E 56(3):2527--2539, 1997.

2319: 

2320: \bibitem{hirshleifer78}

2321: J.~Hirshleifer.

2322: \newblock Competition, cooperation, and conflict in economics and biology.

2323: \newblock {\em The American Economic Review}, 68(2):238--243, May 1978.

2324: 

2325: \bibitem{hogg87PhysRep}

2326: T.~Hogg and B.~A. Huberman.

2327: \newblock Artificial intelligence and large scale computation: A physics

2328:   perspective.

2329: \newblock {\em Physics Reports}, 156:227--310, 1987.

2330: 

2331: \bibitem{hogg96d}

2332: Tad Hogg, Bernardo~A. Huberman, and Colin~P. Williams, editors.

2333: \newblock {\em Frontiers in Problem Solving: Phase Transitions and Complexity},

2334:   volume~81, Amsterdam, 1996. Elsevier.

2335: \newblock Special issue of {\it Artificial Intelligence}.

2336: 

2337: \bibitem{HolMel00}

2338: Owen Holland and Chris Melhuish.

2339: \newblock Stigmergy, self-organization and sorting in collective robotics.

2340: \newblock {\em Artificial Life}, 5(2), 2000.

2341: 

2342: \bibitem{huberman93a}

2343: Bernardo~A. Huberman and Natalie~S. Glance.

2344: \newblock Evolutionary games and computer simulations.

2345: \newblock {\em Proceedings of the National Academy of Science USA},

2346:   90:7716--7718, August 1993.

2347: 

2348: \bibitem{HubermanHogg88}

2349: Bernardo~A. Huberman and Tad Hogg.

2350: \newblock {The behavior of computational ecologies}.

2351: \newblock In B.~A. Huberman, editor, {\em {The Ecology of Computation}}, pages

2352:   77--115, Amsterdam, 1988. Elsevier (North-Holland).

2353: 

2354: \bibitem{IMB2001}

2355: A.~J. Ijspeert, A.~Martinoli, A.~Billard, and L.~M. Gambardella.

2356: \newblock Collaboration through the exploitation of local interactions in

2357:   autonomous collective robotics: The stick pulling experiment.

2358: \newblock {\em Autonomous Robots}, 11(2):149--171, 2001.

2359: 

2360: \bibitem{Kazadi02}

2361: Sanza Kazadi, A.~Abdul-Khaliq, and Ron Goodman.

2362: \newblock On the convergence of puck clustering systems.

2363: \newblock {\em Robotics and Autonomous Systems}, 38(2):93--117, 2002.

2364: 

2365: \bibitem{Kephart90}

2366: Jeffrey~O. Kephart, Tad Hogg, and Bernardo~A. Huberman.

2367: \newblock Collective behavior of predictive agents.

2368: \newblock {\em Physica}, D 42:48--65, 1990.

2369: 

2370: \bibitem{KitTamStoVel98}

2371: H.~Kitano, M.~Tambe, P.~Stone, and M.~Veloso.

2372: \newblock The {RoboCup} synthetic agent challenge 97.

2373: \newblock {\em Lecture Notes in Computer Science}, 1395:62--??, 1998.

2374: 

2375: \bibitem{KubBon00}

2376: C.~R. Kube and E.~Bonabeau.

2377: \newblock Cooperative transport by ants and robots.

2378: \newblock {\em Robotics and Autonomous Systems}, 30(1--2):85--101, 2000.

2379: 

2380: \bibitem{Lerman02a}

2381: Kristina Lerman and Aram Galstyan.

2382: \newblock Mathematical model of foraging in a group of robots: Effect of

2383:   interference.

2384: \newblock {\em Autonomous Robots}, 13(2):127--141, 2002.

2385: 

2386: \bibitem{Lerman01a}

2387: Kristina Lerman, Aram Galstyan, Alcherio Martinoli, and Auke

2388: Ijspeert.

2389: \newblock A macroscopic analytical model of collaboration in distributed

2390:   robotic systems.

2391: \newblock {\em Artificial Life Journal}, 7(4):375--393, 2001.

2392: 

2393: \bibitem{Lerman00a}

2394: Kristina Lerman and Onn Shehory.

2395: \newblock {Coalition Formation for Large-Scale Electronic Markets}.

2396: \newblock In {\em Proceedings of the International Conference on Multi-Agent

2397:   Systems ({ICMAS}'2000), Boston, MA, 2000.}, pages 167--174, 2000.

2398: 

2399: \bibitem{Mahnke99}

2400: R.~Mahnke and J.~Kaupu\^zs.

2401: \newblock Stochastic theory of freeway traffic.

2402: \newblock {\em Physical Review}, E59(1):117--125, 1999.

2403: 

2404: \bibitem{Mahnke97}

2405: R.~Mahnke and N.~Pieret.

2406: \newblock Stochastic master-equation approach to aggregation in freeway

2407:   traffic.

2408: \newblock {\em Physical Review}, E56(3):2666--2671, 1997.

2409: 

2410: \bibitem{Martinoli99}

2411: A.~Martinoli.

2412: \newblock {\em Swarm Intelligence in Autonomous Collective Robotics: From Tools

2413:   to the Analysis and Synthesis of Distributed Control Strategies}.

2414: \newblock PhD thesis, PhD Thesis No 2069, EPFL, 1999.

2415: 

2416: \bibitem{MarIjsGam99}

2417: A.~Martinoli, A.~J. Ijspeert, and L.~M. Gambardella.

2418: \newblock A probabilistic model for understanding and comparing collective

2419:   aggregation mechanisms.

2420: \newblock In Dario Floreano, Jean-Daniel Nicoud, and Francesco Mondada,

2421:   editors, {\em Proceedings of the 5th European Conference on Advances in

2422:   Artificial Life ({ECAL}-99)}, volume 1674 of {\em LNAI}, pages 575--584,

2423:   Berlin, September~13--17 1999. Springer.

2424: 

2425: \bibitem{MarMon95}

2426: A.~Martinoli and F.~Mondada.

2427: \newblock Collective and cooperative group behaviors: Biologically inspired

2428:   experiments in robotics.

2429: \newblock In O.~Khatib and J.~K. Salisbur, editors, {\em Proc. of the Fourth

2430:   Int. Symp. on Experimental Robotics ISER-95}. Springer Verlag, June-July

2431:   1995.

2432: 

2433: \bibitem{MartinoliISER02}

2434: Alcherio Martinoli and Kjerstin Easton.

2435: \newblock Modeling swarm robotic systems.

2436: \newblock In B.~Siciliano and P.~Dario, editors, {\em Proc. of the Eight Int.

2437:   Symp. on Experimental Robotics ISER-02, Sant'Angelo d'Ischia, Italy},

2438:   Springer Tracts in Advanced Robotics 5, pages 297--306, New York, NY, july

2439:   2003. Springer Verlag.

2440: 

2441: \bibitem{MartinoliEaston02}

2442: Alcherio Martinoli and Kjerstin Easton.

2443: \newblock Optimization of swarm robotic systems via macroscopic models.

2444: \newblock In A.~C. Schultz, L.~E. Parker, and F.~E. Schneider, editors, {\em

2445:   Proc. of the Second Int. Workshop on Multi-Robots Systems, March, 2003,

2446:   Washington, DC}, pages 181--192. Kluwer Academic Publishers, 2003.

2447: 

2448: \bibitem{Mataric92}

2449: M.~Matari{\'c}.

2450: \newblock Minimizing complexity in controlling a mobile robot population.

2451: \newblock In {\em Proceedings of the 1992 IEEE International Conference on Robo

2452:   tics and Automation}, pages 830--835, Nice, France, 1992.

2453: 

2454: \bibitem{MatNilSim95}

2455: M.~J. Matari{\'c}, M.~Nilsson, and K.~Simsarian.

2456: \newblock Cooperative multi-robot box pushing.

2457: \newblock In {\em Proceedings of the 1995 IEEE/RSJ International Conference on

2458:   Intelligent Robots}, 1995.

2459: 

2460: \bibitem{Melhuish98}

2461: C.~Melhuish, O.~Holland, and S.~Hoddell.

2462: \newblock Collective sorting and segregation in robots with minimal sensing.

2463: \newblock In R.~Pfeifer, B.~Blumberg, J.-A. Meyer, and S.W. Wilson, editors,

2464:   {\em {From Animals to Animats, Proceedings of the Fifth International

2465:   Conference of The Society for Adaptive Behavior (SAB98)}}, pages 465--470.

2466:   MIT Press, 1998.

2467: 

2468: \bibitem{Michel98}

2469: O.~Michel.

2470: \newblock Webots: Symbiosis between virtual and real mobile robots.

2471: \newblock In J.-C. Heudin, editor, {\em {Proc. of the First Int. Conf. on

2472:   Virtual Worlds, Paris, France,}}, pages 254--263. Springer Verlag, 1998.

2473: \newblock See also http://www.cyberbotics.com/webots/.

2474: 

2475: \bibitem{ArkBalNit}

2476: E.~Nitz, R.~C. Arkin, and T.~Balch.

2477: \newblock Communication of behavioral state in multi-agent retrieval tasks.

2478: \newblock In Lisa {Werner, Robert; O'Conner}, editor, {\em Proceedings of the

2479:   1993 {IEEE} International Conference on Robotics and Automation: Volume 3},

2480:   pages 588--594, Atlanta, GE, May 1993. IEEE Computer Society Press.

2481: 

2482: \bibitem{opper01}

2483: Manfred Opper and David Saad, editors.

2484: \newblock {\em Advanced Mean Field Methods: Theory and Practice}.

2485: \newblock MIT Press, Cambridge, MA, 2001.

2486: 

2487: \bibitem{OstSukMat01}

2488: Esben~H. {\O}stergaard, Gaurav~S. Sukhatme, and Maja~J.

2489: Matari{\'c}.

2490: \newblock Emergent bucket brigading - a simple mechanism for improving

2491:   performance in multi-robot constrained-space foraging tasks.

2492: \newblock In {\em Proceedings of the 5th International Conference on Autonomous

2493:   Agents ({AGENTS}-01)}, 2001.

2494: 

2495: \bibitem{Pacala}

2496: Stephen~W. Pacala, Deborah~M. Gordon, and H.~C.~J. Godfray.

2497: \newblock Effects of social group size on information transfer and task

2498:   allocation.

2499: \newblock {\em Evolutionary Ecology}, 10:127--165, 1996.

2500: 

2501: \bibitem{Parker98}

2502: Lynne Parker.

2503: \newblock Alliance: An architecture for fault-tolerant multi-robot cooperation.

2504: \newblock {\em IEEE Transactions on Robotics and Automation}, 14(2):220--240,

2505:   1998.

2506: 

2507: \bibitem{purcell77}

2508: E.~M. Purcell.

2509: \newblock Life at low reynolds number.

2510: \newblock {\em American Journal of Physics}, 45:3--11, 1977.

2511: 

2512: \bibitem{Loomis99}

2513: W.-J. Rappel, A.~Nicol, A.~Sarkissian, H.~Levine, and W.~F.

2514: Loomis.

2515: \newblock Self-organized vortex state in two-dimensional dictyostelium

2516:   dynamics.

2517: \newblock {\em Physical Review Letters}, 83:1247--1250, 1999.

2518: 

2519: \bibitem{SchoonderwoerdEtAl97}

2520: Ruud Schoonderwoerd, Owen Holland, and Janet Bruten.

2521: \newblock Ant-like agents for load balancing in telecommunications networks.

2522: \newblock In W.~Lewis Johnson and Barbara Hayes-Roth, editors, {\em Proceedings

2523:   of the 1st International Conference on Autonomous Agents}, pages 209--216,

2524:   New York, February~5--8 1997. ACM Press.

2525: 

2526: \bibitem{Schweitzer+al97}

2527: Frank Schweitzer, Kenneth Lao, and Fereydoon Family.

2528: \newblock Active random walkers simulate trunk trail formation by ants.

2529: \newblock {\em BioSystems}, 41:163--166, 1997.

2530: 

2531: \bibitem{simon96}

2532: Herbert~A. Simon.

2533: \newblock {\em The Sciences of the Artificial}.

2534: \newblock MIT Press, Cambridge, MA, 3rd edition, 1996.

2535: 

2536: \bibitem{simon61}

2537: Herbert~A. Simon and Albert Ando.

2538: \newblock Aggregation of variables in dynamic systems.

2539: \newblock {\em Econometrica}, 29:111--138, 1961.

2540: 

2541: \bibitem{Sugawara97}

2542: Ken Sugawara and Masaki Sano.

2543: \newblock Cooperative acceleration of task performance: Foraging behavior of

2544:   interacting multi-robots system.

2545: \newblock {\em Physica}, D100:343--354, 1997.

2546: 

2547: \bibitem{SugSanYosAbe98}

2548: Ken Sugawara, Masaki Sano, Ikuo Yoshihara, and K.~Abe.

2549: \newblock Cooperative behavior of interacting robots.

2550: \newblock {\em Artificial Life and Robotics}, 2:62--67, 1998.

2551: 

2552: \bibitem{Sugawara+al99}

2553: Ken Sugawara, Masaki Sano, Ikuo Yoshihara, K.~Abe, and

2554: T.~Watanabe.

2555: \newblock Foraging behavior of multi-robot system and emergence of swarm

2556:   intelligence.

2557: \newblock In {\em Proc. of IEEE Int. Conf. on Systems, Man, and Cybernetics

2558:   (SMC-99)}, pages 257--262, 1999.

2559: 

2560: \bibitem{Theraulaz98}

2561: Guy Theraulaz, Eric Bonabeau, and Jean-Luis Deneubourg.

2562: \newblock Response threshold reinforcement and division of labour in insect

2563:   societies.

2564: \newblock {\em Proc. of Royal Society of London}, Serie B:327--332, 1998.

2565: 

2566: \bibitem{VauStoSukMat00b}

2567: Richard~T. Vaughan, Kasper St{\o}y, Gaurav~S. Sukhatme, and

2568: Maja~J.

2569:   Matari{\'c}.

2570: \newblock Blazing a trail: Insect-inspired resource transportation by a robot

2571:   team.

2572: \newblock In {\em Proceedings of the 5th International Symposium on Distributed

2573:   Autonomous Robotic Systems (DARS), Knoxville, TN}, 2000.

2574: 

2575: \bibitem{VauStoSukMat00}

2576: Richard~T. Vaughan, Kasper St{\o}y, Gaurav~S. Sukhatme, and

2577: Maja~J.

2578:   Matari{\'c}.

2579: \newblock Whistling in the dark: cooperative trail following in uncertain

2580:   localization space.

2581: \newblock In Carlos Sierra, Gini Maria, and Jeffrey~S. Rosenschein, editors,

2582:   {\em Proceedings of the 4th International Conference on Autonomous Agents

2583:   ({AGENTS}-00)}, pages 187--194, NY, June ~3--7 2000. ACM Press.

2584: 

2585: \bibitem{Walgraef97}

2586: Daniel Walgraef.

2587: \newblock {\em Spatio-Temporal Pattern Formation and with Examples from Physics

2588:   and Chemistry and and Materials Science}.

2589: \newblock Springer, New York, NY, 1997.

2590: 

2591: \bibitem{Wolfram}

2592: Stephen Wolfram.

2593: \newblock {\em Cellular Automata and Complexity}.

2594: \newblock Addison-Wesley, Reading, Mass., 1994.

2595: 

2596: \end{thebibliography}

2597: 

2598: 

2599: 

2600: \end{document}

2601: