1: \newcommand{\ea}{{\em et al.}}

2: 

3: \section{Related Work}

4: \label{sec:prior}

5: 

6: % modeling of collective behavior

7: Mathematical modeling and analysis of the collective behavior of

8: MRS is a relatively new field with approaches and methodologies

9: borrowed from other fields, including mathematics, physics, and

10: biology. Recently, a number of researchers attempted to

11: mathematically analyze  multi-robot systems by using

12: phenomenological models of the type present here. Sugawara

13: \ea~\cite{Sugawara97,SugSanYosAbe98} developed a simple model of

14: cooperative foraging in groups of communicating  and

15: non-co\-mmu\-ni\-cating robots. Kazadi \ea~\cite{Kazadi02} studied

16: the general properties of multi-robot aggregation using

17: phenomenological macroscopic models. Agassounon and

18: Martinoli~\cite{Agassounon02} presented a model of aggregation in

19: which the number of robots taking part in the clustering task is

20: based on the division of labor mechanism in ants. These models are

21: \emph{ad-hoc} and domain specific, and the authors give no

22: explanation as to how to apply such models to other domain. In

23: earlier works we have developed a general framework for creating

24: phenomenological models of collective behavior in groups of

25: robots~\cite{Lerman02nasa,Lerman04sab}. We applied this framework

26: to study collaborative stick-pulling in a group of reactive

27: robots~\cite{Lerman01} and foraging in robots~\cite{Lerman02}.

28: 

29: Most of the approaches listed above are implicitly or explicitly

30: based on stochastic processes theory. Another example of the

31: stochastic approach is the probabilistic microscopic model

32: developed by Martinoli and

33: coworkers~\cite{Martinoli99,MarIjsGam99,IMB2001} to study

34: collective behavior of a group of robots. Rather than compute the

35: exact trajectories and sensory information of individual robots,

36: Martinoli {\em et al.} model each robot's interactions with other

37: robots and the environment as a series of stochastic events, with

38: probabilities determined by simple geometric considerations.

39: Running several series of stochastic events in parallel, one for

40: each robot, allowed them to study the group behavior of the

41: multi-robot system.

42: 

43: 

44: \comment{ Application-level studies of adaptation and learning in

45: multi-robot systems have recently been carried out

46: ~\cite{Kaelbling90,Mataric97,Martinoli02learn,Jones03icra,Dahl03icra}.

47: The RoboCup robot soccer domain provided a fruitful framework for

48: introducing learning in the context of multi-agent and multi-robot

49: systems. Several authors examined the use of reinforcement

50: learning to learn basic soccer skills, coordination

51: techniques~\cite{Riedmiller01} and game strategies~\cite{Stone01}.

52: Li {\em et al.}~\cite{Martinoli02learn} introduced learning into

53: collaborative stick pulling robots and showed in simulation that

54: learning does improve system performance by allowing robots to

55: specialize. No analysis of the collective behavior or performance

56: of the system have been attempted in any of these studies. }

57: 

58: 

59: So far very little work has been done on mathematical analysis of

60: multi-robot systems in dynamic environments. We have recently

61: extended~\cite{Lerman03aamas} the stochastic processes framework

62: developed in earlier work to robots that change their behavior

63: based on history of local observations of the (possibly changing)

64: environment~\cite{Lerman03iros}. In the current paper we develop

65: these ideas further, and present the exact stochastic model of the

66: system, in addition to the phenomenological model.

67: 

68: 

69: %%%%%%

70: Closest to ours is the work of Huberman and

71: Hogg~\cite{HubermanHogg88}, who mathematically studied collective

72: behavior of a system of adaptive agents using game dynamics as a

73: mechanism for adaptation.  In game dynamical systems, winning

74: strategies are rewarded, and agents use the best performing

75: strategies to decide their next move. Although their adaptation

76: mechanism is different from our dynamic task allocation mechanism,

77: their analytic approach is similar to ours, in that it is based on

78: the theory of stochastic processes. Others have mathematically

79: studied collective behavior of systems composed of large numbers

80: of concurrent learners~\cite{Wolpert99,Sato03}. These are

81: microscopic models, which only allow one to study collective

82: behavior of relatively small systems of a few robots. We are

83: interested in macroscopic approaches that enable us to directly

84: study collective behavior in large systems. Our work differs from

85: earlier ones in another important way:  we systematically compare

86: theoretical predictions of mathematical models with results of

87: experiments carried out in a sensor-based simulator.

88: