1: \documentstyle [12pt]{article}
2: %\documentstyle [aaai]{article}
3: %\documentstyle [ijcai01]{article}
4: %\parskip\medskipamount
5: \newcommand {\ignore} [1] {}
6: \newtheorem{theorem}{Theorem}[section]
7: \newtheorem{lemma}[theorem]{Lemma}
8: \newtheorem{corollary}[theorem]{Corollary}
9: \newtheorem{definition}{Definition}[section]
10: \newtheorem{proposition}[theorem]{Proposition}
11: \newtheorem{claim}[theorem]{Claim}
12: \newtheorem{observation}[theorem]{Observation}
13: \newenvironment{proof}{\noindent{\bf Proof:\/}}{\hfill $\Box$\vskip 0.1in}
14: \newenvironment{proofsp}{\noindent{\bf Proof}}{\hfill $\Box$\vskip 0.1in}
15: \def \AA{{\cal A}}
16: \def \GG{{\cal G}}
17: \def \HH{{\cal H}}
18: \def \TT{{\cal T}}
19: \def \PP{{\cal P}}
20: %\input{epsf}
21:
22: \begin{document}
23:
24: \title{Rational Competitive Analysis}
25:
26: \author{Moshe Tennenholtz
27: \thanks{Permanent address: Faculty of Industrial Engineering and Management,
28: Technion--Israel Institute of Technology, Haifa 32000, Israel}\\
29: Computer Science Department\\
30: Stanford University \\
31: Stanford, CA 94305}
32: \date{}
33: \maketitle
34:
35: \begin{abstract}
36: Much work in computer science has adopted competitive analysis as a
37: tool for decision making under uncertainty. In this work we extend
38: competitive analysis to the context of multi-agent systems. Unlike
39: classical competitive analysis where the behavior of an agent's environment
40: is taken to be arbitrary, we consider the case where an agent's environment
41: consists of other agents. These agents will usually obey some (minimal)
42: rationality constraints. This leads to the definition of {\em rational
43: competitive analysis}. We introduce the concept of rational competitive
44: analysis, and initiate the study of competitive analysis for
45: multi-agent systems. We also discuss the application of rational
46: competitive analysis
47: to the context of bidding games, as well as to the classical one-way
48: trading problem.
49: \end{abstract}
50:
51:
52: \section{Introduction}
53:
54: Competitive analysis is a central tool for the design and analysis
55: of algorithms and protocols for decision making under uncertainty
56: \cite{BorElYa}.
57: % Competitive analysis is an approach to decision making
58: %in face of uncertainty.
59: It is a well studied and widely applicable approach that fits the framework
60: of qualitative decision-making in AI (see e.g.
61: \cite{DT97,BraTenjour}).
62: %,tan-pearl:1994a,doyle-wellman:1994a}).
63: %BTJACM99,Lehmann96,
64: The competitive analysis approach attempts to minimize the ratio
65: between the payoff an agent obtains and the payoff
66: he could have obtained had
67: he known the behavior of the environment. For example, consider the
68: following trading problem (see \cite{BorElYa}, Chapter 14).
69: An agent who holds \$100 may wish to exchange them for British pounds. At each point in time, e.g. every minute in between 8AM and 4PM,
70: an exchange ratio
71: of dollars
72: and pounds is announced. This ratio changes dynamically, and in
73: an unpredicted
74: manner.
75: The agent would need to choose the time in which it will trade his
76: \$100 for pounds.
77: Notice that if
78: the agent would have known the sequence of exchange rates, $e$,
79: then he could have chosen a strategy $o(e)$ that maximizes his
80: payoff. If the agent chooses a strategy $s$, then we
81: can compute the ratio of the payoffs obtained by $o(e)$ and $s$.
82: We can do similarly for every sequence of exchange rates $e'$.
83: Based on this, we can compute
84: the highest (i.e. worst) ratio, over all possible sequences, that might
85: be obtained when we compare optimal strategies to $s$. This ratio is
86: denoted by $R(s)$. According to the competitive analysis approach,
87: the agent will apply the {\em competitive ratio} decision criterion:
88: he will choose
89: a strategy $s$ for which $R(s)$ is
90: minimal. This decision criterion may be quite helpful when we
91: lack probabilistic assumptions about the environment. For
92: example, assume that the minimal value of $R(\cdot)$, which
93: is obtained by some
94: strategy $s$, is 2. Then, by selecting $s$,
95: the agent guarantees himself a payoff which is at
96: least half of the optimal payoff that he could have obtained had
97: he known the actual environment behavior.
98:
99: The competitive ratio has also an additive variant (also termed
100: minimax regret \cite{Milnor}), where we replace the term "ratio"
101: by the term "difference" in the definition of $R$. So, in our
102: example, if a strategy $s$ that minimizes $R(\cdot)$ obtains a
103: competitive difference/regret of 20,
104: then this implies that by performing $s$ the agent
105: gets a payoff (e.g. worth in British pounds) which is at most 20
106: lower than what he could have obtained had he known the behavior of the
107: environment. In the sequel, we will use the additive version of
108: the competitive ratio decision criterion.
109:
110: Competitive analysis
111: % based on the competitive ratio decision
112: % rule,
113: has been applied to a variety of classical problems
114: in computer science,
115: such as the k-server problem \cite{KoutsoupiasPapadimitriou} and
116: paging \cite{FiatKarpLuby}, as well as to
117: more general algorithmic
118: problems \cite{BorodinLinialSaks}.
119: In all of these studies the environment that the agent acts in is
120: non-strategic, and therefore does not assume to follow any "rational"
121: behavior. In this paper we extend the concept of competitive
122: analysis to the context of multi-agent systems. In a multi-agent system
123: the
124: environment in which an agent takes his decision consists of
125: other "rational" agents. Following
126: previous work on competitive analysis, our
127: approach is non-Bayesian and normative;
128: we would like to find a decision rule for
129: the agent that will rely as little as possible on assumptions about
130: the behavior of his environment. Therefore, we adopt the requirement
131: that agent $A$ should
132: rule out a behavior $b_1$ of agent $B$ only if $b_1$ is dominated
133: by another behavior, $b_2$, of that agent.
134: Dominated behaviors are purely irrational in any decision
135: making model. The agent will choose his behavior according to the
136: competitive ratio decision rule. However, he should consider only
137: rational behaviors of the other agents; a behavior of an
138: agent will be considered
139: irrational if and only if it is dominated by another behavior of it.
140:
141:
142: In section 2 we describe bidding games, a family of games that
143: will serve us for the illustration of the basic concepts developed
144: in this paper. Bidding games are representatives of k-price
145: auctions, a central class of economic mechanisms \cite{kpricegeb}.
146: In section 3 we present a competitive analysis of bidding games.
147: In section 4 we introduce rational competitive analysis, a new
148: tool for normative decision making, that generalizes competitive
149: analysis to the context of rational environments, and apply it
150: to bidding games.
151: In section 5 we consider repeated (multi-stage) games. We present several
152: results on the relationships
153: between (rational) competitive analysis
154: of repeated games and the competitive analysis
155: of the particular (one-shot) games they consist of.
156: Then, in section 6, we discuss and study variants and modifications
157: of the one-way trading problem, using rational competitive
158: analysis. In particular, we
159: study the multi-agent one-way trading problem.
160:
161: \section{Bidding games}
162:
163: We start by recalling the general definition of a
164: (strategic form) game.
165:
166: \begin{definition}
167: A {\em game} is a tuple $G=\langle
168: N=\{1,2,\ldots,n\},S=\{S_i\}_{i=1}^n, \{U_i\}_{i=1}^n \rangle$,
169: where $N$ is a set of $n \geq 2$ players, $S_i$ is the set of
170: strategies available to player $i$, and $U_i: \Pi_{j=1}^n S_j
171: \rightarrow R$ is the utility function of player $i$.
172: \end{definition}
173:
174: In a game, each player selects a strategy from a set of available strategies.
175: The tuple of strategies selected, one by each player, determines
176: the payoff of each of the agents%
177: \footnote{Here and elsewhere we use the terms player and agent
178: interchangeably.} (as prescribed by the utility functions).
179:
180: In a bidding game, a center attempts to obtain a service from a set of
181: potential suppliers. Each such supplier has a certain cost associated
182: with that service. This cost is taken to be an integer between $K-T$
183: and $K$, where $K$ and $T$ are w.l.o.g integers, $K>T > 0$. Each agent
184: will offer his service and ask for a payment in the range in between
185: $K-T$ and $K$. We will associate the request for payment of
186: $K-i$ with the
187: integer $i$, where $0 \leq i \leq T$. The center will choose
188: as the service provider the supplier
189: with the lowest asking price. There are various ways
190: for determining the payment to that agent; in particular, the agent can be
191: paid his asking price, the second lowest asking price,
192: or the third lowest
193: asking price. We assume that the costs for providing the service by
194: each of the agents are common knowledge among them, although
195: the center might not be familiar with these costs.
196: Although this is quite natural
197: for the above procurement problem, other assumptions can be treated
198: similarly.
199:
200: Our definition of bidding games will capture
201: the above, by considering
202: a fully isomorphic situation, namely: the
203: auctioning of a good.
204: The good is held by the center.
205: Each agent has a valuation (i.e. maximal willingness to pay) for the good.
206: Each agent needs to decide on his bid. The center will
207: allocate the good to the agent with the highest bid (rather than
208: to the agent with the lowest asking price,
209: as in the isomorphic procurement problem).
210:
211:
212: Formally, we have:
213:
214:
215: \begin{definition}
216: Given a set of $n$ players, and an integer $T >> 1$,
217: a bidding game is determined by
218: the tuple
219: B=$(x_1,\ldots,x_n,k)$ where $x_i={l_i \over T}$ for some integer
220: $0 \leq l_i \leq T$, and $1 \leq k \leq n$ is an integer.
221: Player $i$'s strategy in $B$ is a decision about $b_i\in [0,T]$.
222: Given a strategy profile $b=(b_1,b_2,\ldots,b_n)$ denote by $b_{[i]}$
223: the $i$-th order statistic of this tuple.
224: Let $M(b)$ be the number of elements of $b$ that equals $b_{[1]}$.
225: Then, $U_i(b)= {1 \over M(b)} (x_i - {b_{[k]} \over T})$ if
226: $b_i=b_{[1]}$, and $U_i(b)=0$ otherwise.
227: \end{definition}
228:
229:
230:
231: In the above formalism, $x_i$ is the valuation of agent $i$
232: (that is normalized to the interval [0,1]), while $b_i$ denotes
233: the bid made by agent $i$.
234: The
235: payment made by the winner
236: is determined by the parameter $k$. If $k=1$ we get the
237: standard high-bid wins (or first-price) auction; if $k=2$ then we
238: get the famous Vickrey (second-price) auction, while if $k=3$ we
239: get the case of third-price auctions.%
240: \footnote{Third-price auctions have been shown to have
241: appealing properties in the context of Internet Auctions
242: \cite{kpricegeb}.}
243:
244:
245: For ease of presentation we will assume that $ 2 \leq l_i < T$
246: for every $1 \leq i \leq n$, that $i \neq j$ implies $l_i \neq
247: l_j$, and that $T \geq n$.
248:
249: \section{Competitive Analysis}
250:
251: In a game, agent $i$ is facing an environment that consists of
252: the other agents. The actions to be selected by these agents are
253: not under the control of $i$. Following the literature on
254: competitive analysis, the competitive ratio decision rule
255: may be used in order
256: to choose an action for that agent.
257:
258: \noindent{\bf Definition 3.1}
259: %\begin{definition}
260: %
261: {\em
262: Given a game $G$, and a strategy profile $s=(s_1,s_2,\ldots,s_n)
263: \in \Pi_{j=1}^n S_j$, the regret of player $i$ is given by
264: $Reg_i(s_i,s_{-i})=max_{t \in S_i}
265: U_i(s_1,\ldots,s_{i-1},t,s_{i+1},\ldots,s_n) - U_i(s)$. A
266: strategy $s \in S_i$ is a competitive strategy for agent
267: $i$ if $s \in argmin_{t
268: \in S_i} max_{q \in S_{-i}} Reg_i(t,q)$, where $S_{-i}$ denotes
269: the possible strategy profiles of players in $N \setminus \{i\}$.}
270: %\end{definition}
271:
272: Given the above definition we are interested in applying
273: competitive analysis to bidding games. We now present three
274: claims about competitive analysis of bidding games.
275: These claims are associated with the competitive analysis of
276: 1st,2nd, and 3rd-price auctions, respectively.
277:
278: \begin{claim} Given the bidding game
279: B=$(x_1,\ldots,x_n,1)$, a competitive strategy for agent $i$ yields
280: a regret value of $\alpha \over T$, where $\alpha$
281: equals the upper integer value of ${{l_i-1} \over {2}}$.
282:
283: \end{claim}
284:
285: \noindent{Basic idea behind proof:} Agent $i$ can lose by
286: submitting a bid that is higher than his valuation. On the other
287: hand, by submitting a bid that is below ${l_i}-1$ agent $i$ might
288: lose, since agent $j \neq i$ might submit $l_i-1$ as a winning bid.
289: Since agents may submit the bid 0,
290: agent $i$ will minimize his regret by submitting a bid
291: that equals (the upper integer value of) half of the difference
292: between $l_i-1$ and 0.
293:
294: \begin{claim} Given~the~bidding game B=$(x_1,\ldots,x_n,2)$ a competitive strategy
295: for agent $i$ yields a zero
296: regret.
297:
298: \end{claim}
299:
300: \noindent{Basic idea behind proof:} Here the optimal strategy for
301: an agent, regardless of what the others do, is to send his actual
302: valuation as his bid; this is a well known property of the Vickrey
303: auction \cite{Wolfstetter}. As a result we get a regret of 0.
304:
305: \begin{claim} Given the bidding game
306: B=$(x_1,\ldots,x_n,3)$, and assume w.l.o.g that $x_1 > x_2 > \cdots > x_n$,
307: then agent $j$'s competitive strategy is to send the bid
308: ${min(2l_j,T)}$.
309:
310:
311: \end{claim}
312:
313: \noindent{Basic idea behind proof:} Given that agents may submit
314: the bid 0, agent $j$ might reach a regret of ${{l_j} \over T}$ if he is not
315: the winner. Submitting however a bid that is higher than $2l_j$
316: may also lead to a regret of ${{l_j} \over T}$, given that the agents may
317: submit $2l_j$ as their bids. Combining these
318: observations, we
319: get that submitting the bid ${{min(2l_j,T)}}$ is the
320: competitive strategy.
321:
322:
323:
324: \section{Rational Competitive Analysis}
325:
326: Although competitive analysis is a most powerful concept from a
327: non-Bayesian
328: normative perspective, it may be quite restrictive when we consider
329: decision-making in multi-agent systems. Following the spirit of competitive
330: analysis for normative decision making,
331: we refrain from using probabilistic assumptions and
332: game-theoretic equilibrium analysis.%
333: \footnote{The debate about whether competitive ratio and non-Bayesian
334: decision making are expressive or useful for normative or descriptive
335: objectives is beyond the scope of this paper; see \cite{BTJACM99} for sound and
336: complete
337: axiomatization of the competitive ratio decision criterion.}
338: However, one can
339: still improve on the use of competitive analysis by considering
340: minimal rationality requirements.
341:
342:
343: %\begin{definition}
344: \noindent{\bf Definition 4.1}
345: {\em
346: Given a game $G=\langle N=\{1,2,\ldots,n\},\{S_i\}_{i=1}^n,\{U_i\}_{i=1}^n
347: \rangle$, we say that a strategy $s_i \in S_i$ weakly dominates a strategy
348: $s_i^{'} \in S_i$ if $U_i(s_i,t) \geq
349: U_i(s_i^{'},t)$ for every strategy profile $t$ of the players in
350: $N \setminus \{i\}$, and there exists such strategy profile $t'$ for which
351: $U_i(s_i,t') >
352: U_i(s_i^{'},t')$. A strategy $s \in S_i$ will be called rational is there
353: is no other strategy $\bar{s} \in S_i$ that weakly dominates it.
354: Given a game $G$, the set of rational strategies for player $i$ will
355: be denoted by $Rat(S_i)$.}
356: %\end{definition}
357:
358: In any reasonable model agents will choose only from the set of
359: non-dominated strategies. Our idea is therefore to combine the
360: powerful idea of competitive analysis and this minimal
361: requirement of rationality, in order to re-introduce competitive
362: analysis into the framework of multi-agent systems.
363:
364: %\begin{definition}
365: \noindent{\bf Definition 4.2}
366: {\em
367: A strategy $s \in S_i$ is a rationally competitive strategy if $s
368: \in argmin_{t \in S_i} max_{q \in Rat(S_{-i})} Reg_i(t,q)$, where
369: $Rat(S_{-i})$ denotes the possible rational strategy profiles of
370: players in $N \setminus \{i\}$, i.e. each player $j \in N
371: \setminus \{i \}$ chooses its strategy from $Rat(S_j)$.}
372:
373: %\end{definition}
374:
375: Basically, a rationally competitive strategy applies the
376: competitive ratio decision criterion, while taking into account only
377: rational activities of the environment. As the following
378: claims illustrate, rational competitive analysis introduces
379: an improved approach to
380: normative decision making.
381:
382: \begin{claim}
383: Given the bidding game
384: B=$(x_1,\ldots,x_n,1)$, a rational competitive strategy for
385: agent $i$ yields a regret of $\alpha \over T$, where $\alpha$
386: equals the upper integer
387: value of ${min(l_i,max_{j; j\neq i} l_j)-2} \over {2}$.
388: \end{claim}
389:
390: {\noindent Basic idea behind proof:} We observe that any strategy
391: that tells agent $j$ to submit a bid which is greater than or equals to
392: his valuation is dominated by the strategy of submitting his
393: valuation minus 1. Given our assumptions about the possible valuations,
394: all other strategies, excluding the strategy of
395: submitting the bid 0, are not dominated. As a result, from the
396: perspective of agent $i$, if his valuation is the highest one, he
397: will minimize his regret if he will make a bid that is half of the
398: distance between
399: $max_{j; j\neq i} l_j - 1$ and 1. If agent $i$'s valuation is not the
400: highest one then he will minimize his regret (again, taking into
401: account the assumptions on possible valuations) if he will make a bid
402: in between $l_i-1$ and 1.
403:
404: Notice that rational competitive analysis allows us to improve upon the
405: type of reasoning carried out in claim 3.1. Technically,
406: in the case of a bidding game with $k=1$, rationality
407: implies that we need to take the minimum between $l_i$ and the highest other
408: agents' valuation in our analysis, rather than consider $l_i$ only.
409:
410:
411:
412: \begin{claim}
413: Given the bidding game B=$(x_1,\ldots,x_n,2)$
414: a rational competitive strategy for agent $i$ yields a zero
415: regret.
416:
417: \end{claim}
418:
419:
420: As we can see, unlike the major effect of the rationality
421: assumption in the case of a first-price auction, there is no change
422: in the analysis in the case of a second-price auction. In the case of a
423: third-price auction, we see again the effect of the revised
424: notion:
425:
426: \begin{claim}
427: Given the bidding game B=$(x_1,\ldots,x_n,3)$, and assume
428: w.l.o.g that $x_1 > x_2 > \cdots > x_n$, then a rational
429: competitive strategy for agent $j \ (j=1,2)$, is to submit the
430: bid $min(2l_j-l_{[3]},T)$, where $l_{[3]}$ corresponds to the
431: 3rd highest $x_k$; a zero-regret rational competitive strategy
432: for agent $i, 3 \leq i \leq n$, is to submit $l_i$.
433:
434: \end{claim}
435:
436: \noindent{Basic idea behind proof:} First, observe that any
437: strategy where the agent submits a bid that is below that agent's
438: valuation is dominated by the strategy that tells him to submit
439: his actual valuation as his bid. As a result, for agents $3,4,\ldots,n$
440: there is a 0 regret in
441: submitting their actual valuations as their bids.
442: Let us assume that agent $i$ (where $i$ is 1 or 2)
443: submits a bid, then it can lose ${{l_i-l_{[3]}} \over T}$ if it turns out not
444: be the highest bidder (since agent $j$ submits a higher bid).
445: On the other hand, by submitting the bid $b_i > l_i$ a loss
446: of ${{b_i-l_i} \over T}$ may be caused, since (from the perspective of agent $i$)
447: two other agents may submit the bid $b_i$.
448: This implies that the bid
449: $min(2l_i-l_{[3]},T)$ will minimize this agent's regret.
450:
451:
452: As we can see, in the case of $k=3$ as well, rational competitive analysis
453: for bidding games leads to an improved normative approach to decision making.
454: In particular, the competitive strategy of Claim 3.3 specifies a too high bid,
455: and is not a rationally competitive strategy; as a result, it fails to
456: serve in a multi-agent context.
457:
458: \section{Rational competitive analysis in repeated games: folk theorems}
459:
460: We first recall the notion of finitely repeated games \cite{FudTir}.
461:
462: \begin{definition}
463: Given an integer $l >0$ and a game $G=\langle
464: N=\{1,2,\ldots,n\},S=\{S_i\}_{i=1}^n, \{U_i\}_{i=1}^n \rangle$,
465: a {\em repeated game} $RG=(G,l)$ with respect to $G$ is a game where
466: $G$ is repeatedly played $l$ times. $RG$ consists of the
467: following strategies and utility functions:
468: a strategy of agent $i$ in $RG$ determines the strategy of $G$ to be taken
469: by $i$ in the $k$-th iteration of $G$, as a function of the history of
470: strategies of $G$
471: selected by the others in iterations $1,2,\ldots,k-1$.
472: Given a tuple of strategies of $RG$, one for each agent, the payoff
473: for agent $i$ is the sum of its payoffs along the $l$ iterations.
474: A sub-game of a repeated game $RG$ is a repeated game that starts from
475: iteration $ 1 \leq q \leq l$ of $RG$ and consists of $l-q+1$ iterations.
476: A (rationally) competitive strategy in $RG$ is a strategy
477: that is a (rationally) competitive strategy at each of the sub-games
478: of $RG$.%
479: \footnote{This definition is in the spirit of sub-game perfect equilibrium
480: in game theory.}
481: \end{definition}
482:
483:
484:
485: Repeated games have been of much interest in the game-theory literature,
486: due to the fact they enable to study agents' actions as a function
487: of past events and other agents' actions.
488: The study of repeated games is central to the
489: understanding of
490: basic issues in coordination and cooperation (e.g. \cite{Axelrod}), as well
491: as for the study of learning in games (e.g. \cite{FudLevlearn}).
492:
493:
494: One of the central challenges for the study of repeated games is to
495: establish general theorems (titled folk-theorems)
496: that explain/recommend behavior in
497: these (repeated) games by means of solution concepts for
498: the games they consist of. In our case, it would be of interest
499: to understand what will be a rationally competitive strategy in a
500: repeated game, and try to relate it to the competitive analysis
501: of the simple one-shot game that takes place at each iteration.
502:
503: We now present a general result about competitive analysis in
504: repeated games. For ease of presentation we will assume that $G$
505: is a two-player game, where all payoffs are distinct. We will also
506: assume w.l.o.g that all payoffs are non-negative. Given a
507: repeated game $(G,l)$, let us denote the highest payoff for agent
508: $i$ in $G$ by $h_i(G)$, and the second highest payoff of agent
509: $i$ in $G$ by $sh_i(G)$.
510:
511:
512: \begin{theorem}
513:
514: Given a repeated game $(G,l)$ and assume that for each agent $i$
515: $h_i(G) \geq 2 \cdot sh_i(G)$, then
516: a rationally competitive strategy for agent $i$
517: in the game $(G,l)$ is obtained
518: by performing the competitive strategy of it in $G$ on iterations
519: $1,2,\ldots,l-1$ and performing the rational competitive strategy of
520: it in $G$ on the
521: last iteration.
522: \end{theorem}
523:
524: \noindent{Basic idea behind proof:}
525: %Consider the decision-making process
526: %by agent $j$, and let us denote the other agent by $i$.
527: From the
528: perspective of agent $i$, assuming we are at stage $k < l$, the
529: selection of any strategy $s$ of $G$ by $j$ can be complemented to
530: a non-dominated strategy of $j$; this non-dominated strategy will tell
531: $j$ to choose the strategy associated with $h_j(G)$ in stages
532: $k+1,\ldots,l$. The reason
533: that the resulting strategy is not dominated is that $j$
534: considers the strategy where $i$ will also choose in stages
535: $k+1,\ldots,l$ the strategy (of his) in $G$ that corresponds to $h_j(G)$,
536: and does it only if in stage $k$ agent $j$ chooses $s$; in addition,
537: according to this strategy $i$ will choose the strategy that corresponds
538: to $sh_j(G)$ is stage $k$.
539: This
540: implies that agent $i$ should consider at stages $1,2,\ldots,l-1$
541: all possible strategies of agent $j$ in $G$. In the last stage
542: agent $i$ is no longer subject to the above considerations and
543: will choose the rationally competitive strategy of $G$.
544:
545:
546: The above theorem shows a strong connection between competitive analysis
547: in repeated games and competitive analysis in simple single-shot games.
548: As it turns out, this connection can be
549: further generalized to a much richer context:
550:
551: \begin{definition} Let $\bar{G}=(G_1,G_2,\ldots,G_m)$
552: be a sequence of games
553: where $N$ is the set of players in each of the games in the sequence,
554: and game $G_i$ is played in iteration $i$.
555: The strategy of agent $t$ in $\bar{G}$ determines its strategy in
556: $G_i$, $1\leq i \leq m$,
557: as a function of the strategies of $G_j$, $1 \leq j < i$, selected
558: by the other
559: agents in previous iterations.
560: Given a tuple of strategies of $\bar{G}$, one for each agent, the
561: payoff of agent $i$ is taken as the sum of its payoffs in the $m$
562: iterations.
563:
564: \end{definition}
565:
566:
567:
568: \begin{theorem}
569:
570: Given a sequence of games $\bar{G}=(G_1,G_2,\ldots,G_m)$ where
571: $N$ is the set of players in each of the games in the sequence,
572: and game $G_j$ is played in iteration $j$,
573: and assume that
574: $h_i(G_k) \geq 2 \cdot sh_i(G_l)$ for every $1 \leq k,l \leq m$,
575: and for every agent $i$, then
576: a rationally competitive strategy for agent $i$
577: in the game $\bar{G}$ is obtained
578: by performing the competitive strategy of $G_j$ in iterations
579: $1,2,\ldots,l-1$ and performing the rational competitive strategy of
580: $G_m$ in the
581: last iteration.
582: \end{theorem}
583:
584: The above theorem can be generalized into a situation where $n$
585: games from among the set of games $\{G_1,G_2,\ldots,G_m\}$ are
586: executed in some random order (with possible repetitions).
587: Formally, this can be captured by the following definition and
588: theorem:
589: %\begin{definition}
590:
591: \noindent{\bf Definition 5.3}
592: {\em
593: Given a set of games $G=\{G_1,G_2,\ldots,G_m\}$, a random game
594: with respect to $G$, $\bar{G}$, is a sequence of $n$ games
595: $(g_1,g_2,\ldots,g_n)$, where $g_i \in G \ (1 \leq i \leq n)$
596: and $N$ is the set of players in each of the games in the
597: sequence. The game to be played in iteration $i$, $g_i$, is
598: randomly selected from the set $G$ independently of previous
599: selections made. The strategy of agent $t$ in $\bar{G}$ determines
600: its strategy in $g_i$, $1\leq i \leq n$, as a function of the
601: strategies of $g_j$, $1 \leq j < i$, selected by the other agents in
602: previous iterations. Given a tuple of strategies of $\bar{G}$,
603: one for each agent, the payoff of agent $t$ is taken as the sum
604: of its payoffs in the $n$ iterations.
605: A sub-game of a random game $\bar{G}$ with respect to $G$, is
606: a random game with respect to $G$ that starts from iteration $1 \leq j \leq n$
607: and consists of $n-j+1$ iterations as above.
608: A (rationally) competitive strategy in a random game is required to be
609: a (rationally) competitive strategy at each sub-game of it.
610: }
611: %\end{definition}
612:
613: \begin{theorem}
614:
615: Given a random game $\bar{G}$ with respect to
616: $G=\{G_1,G_2,\ldots,G_m\}$, and assume that
617: $h_i(G_k) \geq 2 \cdot sh_i(G_l)$ for every $1 \leq k,l \leq m$,
618: and for every agent $i$, then
619: a rationally competitive strategy for agent $i$ in the game
620: $\bar{G}$ is obtained
621: by performing the competitive strategy of game $g_i$ in
622: iterations
623: $1,2,\ldots,n-1$ and performing the rational competitive strategy
624: of the game $g_n$ on the last iteration in the sequence.
625: \end{theorem}
626:
627: \section{One-way trading in multi-agent systems}
628:
629: In the previous section we have discussed competitive analysis
630: for multi-agent systems in the framework of general repeated games
631: and random games. In this section we look at a particular variant
632: of repeated games that extends a well known and fundamental framework
633: for competitive analysis -- the one-way trading (see citations
634: in chapter 14 of \cite{BorElYa}).
635:
636: One way to present the structure of one-way trading is as
637: follows. An agent $a$ seeks buying $X$ units of a good or of a
638: service. A supplier $A$ wishes to supply these units of good to
639: $a$. The agents act in an environment that determines the actual
640: payment for a unit of good in a non-deterministic way. For example,
641: the payments might be specified in dollars, but since agent $A$ is
642: a British company the actual payoffs it will obtain for providing
643: the good will depend on
644: the exchange ratio of the dollar and the
645: British pound. Formally, the environment
646: announces at each point in time, $1,2,\ldots,t$, the payoff that
647: will be obtained by agent $A$ for supplying a unit of good. The
648: announcements are selected in an unpredicted non-deterministic
649: manner from the interval $[m,M]$, where $M > m
650: > 0$. For example, when $K$ is announced at point $i$,
651: agent $A$ can supply the $X$ units of good and obtain
652: a payoff of $X \cdot K$. Our assumption is that agent $A$ will
653: obtain a zero payoff by not providing the units of good.
654: The
655: decision problem that agent $A$ faces is as follows: at each point
656: he needs to decide whether he would like to supply the
657: units of good in the current rate. We assume that when agent $A$
658: is willing to provide the service then he will provide and be paid
659: for the whole quantity of goods requested by agent $a$ (this property
660: is termed one-way search).
661:
662: The competitive analysis approach tells agent $A$ in the above
663: scenario to minimize his regret value. As it turns out, the
664: competitive strategy in this case will tell the agent to accept
665: the offer (i.e supply the units of good) when the payoff
666: reaches ${{M-m} \over 2}$ in stage $j\leq t-1$, and to accept
667: the offer on stage $t$ otherwise.
668: % (if we consider the
669: %multiplicative competitive ratio the term ${{M-m} \over 2}$
670: %should be replaced by $\sqrt{M \over m}$ \cite{}).
671:
672: One-way trading is a typical setting for the use of competitive
673: analysis. We now extend it to the case of several agents, where
674: more than one agent may wish to supply the units of good
675: requested by $a$. We will first develop the multi-agent
676: framework without
677: considering the rationality assumption, and then will extend it to
678: the case of rational competitive analysis.
679:
680: \subsection{Multi-agent one-way trading}
681:
682: For ease of exposition we consider the case of trading two
683: agents (i.e. two suppliers who can provide the units of good
684: requested by $a$): $A_1$ and $A_2$. The payment offers for the
685: two agents are taken to be independent. For example, agent 1 may be a
686: British company and agent 2 may be a Japanese company, and
687: therefore the actual payment offers for them (from their
688: perspective) will reflect the exchange ratio between the dollar
689: and the British pound, and the exchange ratio between the dollar and
690: the Japanese Yen, respectively. Formally, we have:
691:
692: \begin{definition}
693: Let $M_1,M_2,m_1,m_2,t,X=2K$ be positive
694: integers, where $M_1 > m_1$ and $M_2
695: > m_2$, $t \geq 3$, and $X$ is even. A multi-agent one way trading $T=\langle
696: N=\{1,2\},X,t,m_1,M_1,m_2,M_2 \rangle$ is a random game with the
697: following players, strategies, and payoffs:
698:
699: \begin{enumerate}
700: \item
701: The players are 1 and 2.
702: \item There are $t$ iterations. Each iteration $i$ is associated with
703: a pair of numbers $(a_1,a_2)$ where $m_1 \leq a_1 \leq M_1$ and
704: $m_2 \leq a_2 \leq M_2$. At each iteration each agent can "take"
705: or "pass". However, if an agent takes in iteration $j$ then both
706: agents can only "pass" in all iterations $j \geq i$.
707: \item The payoff of each agent in iteration $i$ is 0 if it passes;
708: if an agent performs "take" in iteration $i$ then its payoff will
709: be $a_i X$ if the other agent passes and $a_i K$ if the other
710: agent takes.
711: \end{enumerate}
712: \end{definition}
713:
714: Intuitively, "take" means a decision of accepting the offer,
715: while "pass" means rejecting it (at the given point). If both
716: agents agree to "take" then each one of them will supply half of
717: the units (and the payoff will be splitted among the agents).
718: We
719: now show what is the structure of the competitive strategy in a
720: multi-agent one-way trading setting.
721:
722: \begin{theorem}
723: Given a multi-agent one way trading $T=\langle
724: N=\{1,2\},X,t,m_1,M_1,m_2,M_2 \rangle$, a competitive strategy for agent $i$
725: is as follows:
726:
727: \begin{enumerate}
728: \item For iterations $1 \leq j \leq t-1$, take iff $a_i \geq
729: {{2M_i+m_i} \over 4}$
730: \item If you arrive at iteration $t$ then take.
731: \end{enumerate}
732: \end{theorem}
733:
734: \noindent{Basic idea behind proof:}
735: Consider iteration $j$, $1 \leq j \leq t-1$, and consider the announcement
736: $a_i=Y$. Then, by taking in round $j$, agent $i$ might suffer a regret of
737: $2M_iK-2YK$ (notice that there is a regret when an agent takes only
738: if the other does not take at that iteration).
739: By not taking in stage $j$ agent $i$ might suffer a regret
740: of $2YK-m_iK$ (which is in fact $max(KY,2YK-m_iK)$).
741: In order to minimize the regret agent $i$ will therefore have
742: to take whenever $Y$ satisfies that $2M_iK-2YK=2YK-m_iK$, i.e. when
743: $a_i={{2M_i+m_i}\over 4}$. The fact that the regret is minimized in iteration
744: $t$ by taking rather than passing is immediate.
745:
746:
747:
748: \subsection{Rational competitive analysis for multi-agent one way
749: trading}
750:
751: We now show the result
752: of applying rational competitive analysis to the context of
753: multi-agent one way trading:
754:
755:
756: \begin{theorem}
757: Given a multi-agent one way trading $T=\langle
758: N=\{1,2\},X,t,m_1,M_1,m_2,M_2 \rangle$, a rational
759: competitive strategy for agent $i$
760: is as follows:
761:
762: \begin{enumerate}
763: \item For iterations $1 \leq j \leq t-1$, if the other agent, $k$, is
764: announced that $a_k=M_k$, then take.
765:
766: \item For iterations $1 \leq j \leq t-2$, if (1) does not hold then
767: take iff $a_i \geq
768: {{2M_i+m_i} \over 4}$
769:
770: \item If (1) does not hold, then in iteration $t-1$
771: take iff $a_i \geq
772: {{M_i+m_i} \over 4}$.
773: \item If you arrive at iteration $t$ then take.
774: \end{enumerate}
775: \end{theorem}
776:
777: \noindent{Basic idea behind proof:}
778: Notice that if the other agent, $k$, is announced that $a_k=M_k$ then taking
779: dominates any other strategy of it. In no other cases we can say that taking
780: or passing in iterations $1,2,\ldots,t-1$ is dominated. Also, in stage $t$
781: passing is dominated by taking.
782: As a result we will get that agent $i$ will minimize its regret by taking
783: when $a_k=M_k$ or when it arrived in the last iteration.
784: Assume that $a_i=Y$ in iteration $t-1$, then
785: the maximal regret we get by taking is $M_iK-2KY$, and by passing
786: the maximal regret in this case is $2KY-m_iK$ (which is in fact
787: $max(KY,2KY-m_iK)$). This implies that
788: the regret is minimized
789: when $M_iK-2YK=2KY-m_iK$ ), i.e.
790: when $Y = {{M_i+m_i} \over 4}$.
791: The other case that refers to iterations $1,2,\ldots,t-2$ will be treated as
792: in the case of (standard) competitive analysis.
793:
794:
795: %It is worth to notice that the above analysis made use of the fact we require
796: %from a rationally competitive strategy to be such in any sub-game.
797:
798: \section{Conclusion}
799:
800: Competitive analysis
801: is a major tool in computer science, which has been used in a variety
802: of contexts.
803: In this paper we have introduced rational competitive
804: analysis. Rational competitive analysis generalizes competitive
805: analysis to the context of multi-agent systems.
806: Moreover, we have shown its use in the context of bidding games
807: and one-way trading, two problems of considerable importance,
808: as well as in the context of general repeated games.
809: Our approach adopts the non-Bayesian normative
810: approach adopted in previous work, but modifies it to incorporate
811: minimal rationality requirements. Such requirements are essential in
812: multi-agent domains.
813: Many of the previous studies in the context of competitive analysis
814: can be naturally extended to multi-agent
815: domains, and then rational competitive analysis can serve as a fundamental
816: tool for the study of these extensions.
817: We see the study of such extensions as a most attractive
818: research topic, and hope
819: that others will join us in addressing this challenge.
820:
821: \begin{thebibliography}{10}
822:
823: \bibitem{Axelrod}
824: R.~Axelrod.
825: \newblock {\em The Evolution of Cooperation}.
826: \newblock New York: Basic Books, 1984.
827:
828: \bibitem{BorodinLinialSaks}
829: A.~Borodin, N.~Linial, and M.~Saks.
830: \newblock An optimal online algorithm for metrical task systems.
831: \newblock {\em Journal of the ACM}, 39:745--763, 1992.
832:
833: \bibitem{BorElYa}
834: Allan Borodin and Ran El-Yaniv.
835: \newblock {\em On-Line Computation and Competitive Analysis}.
836: \newblock Cambridge University Press, 1998.
837:
838: \bibitem{BraTenjour}
839: R.~Brafman and M.~Tennenholtz.
840: \newblock {Modeling Agents as Qualitative Decision-Makers}.
841: \newblock {\em Artificial Intelligence}, 94, 1997.
842:
843: \bibitem{BTJACM99}
844: R.~I. Brafman and M.~Tennenholtz.
845: \newblock An axiomatic treatment of three qualitative decision criteria.
846: \newblock {\em Journal of the ACM}, 47(3), March 2000.
847:
848: \bibitem{DT97}
849: J.~Doyle and R.~Thomason.
850: \newblock Qualitative preferences in deliberation and practical reasoning.
851: \newblock Working notes of the AAAI spring symposium, 1997.
852:
853: \bibitem{FiatKarpLuby}
854: A.~Fiat, R.m. Karp, M.~Luby, L.A. McGeoch, D.D. Sleator, and N.E. Young.
855: \newblock On competitive algorithms paging algorithms.
856: \newblock {\em Journal of Algorithms}, 12:685--699, 1991.
857:
858: \bibitem{FudLevlearn}
859: D.~Fudenberg and D.~Levine.
860: \newblock {\em The theory of learning in games}.
861: \newblock MIT Press, 1998.
862:
863: \bibitem{FudTir}
864: D.~Fudenberg and J.~Tirole.
865: \newblock {\em Game Theory}.
866: \newblock MIT Press, 1991.
867:
868: \bibitem{KoutsoupiasPapadimitriou}
869: E.~Koutsoupias and C.~Papadimitriou.
870: \newblock On the k-server conjecture.
871: \newblock {\em Journal of the ACM}, 42(5):971--983, 1995.
872:
873: \bibitem{Milnor}
874: J.~Milnor.
875: \newblock {Games Against Nature}.
876: \newblock In R.~M. Thrall, C.H. Coombs, and R.L. Davis, editors, {\em Decision
877: Processes}. John Wiley \& Sons, 1954.
878:
879: \bibitem{kpricegeb}
880: D.~Monderer and M.~Tennenholtz.
881: \newblock K-price auctions.
882: \newblock {\em Games and Economic Behavior}, 31:220--244, 2000.
883:
884: \bibitem{Wolfstetter}
885: E.~Wolfstetter.
886: \newblock Auctions: An introduction.
887: \newblock {\em Journal of Economic Surveys}, 10(4):367--420, 1996.
888:
889: \end{thebibliography}
890:
891: %\bibliographystyle{aaai}
892: %\bibliographystyle{named}
893:
894: \end{document}
895:
896: Rationalizable Competitive Strategies}
897:
898: The results presented in the previous section show how beneficial
899: rational competitive analysis can be. These aree of course most positive
900: news, but as we show now one should still be careful with the interpretation
901: of these results. In the definition of rational competitive analysis we assume that agents will not use any action may suggest that given even
902: further restrictions on what rational behaviors
903: \begin{enumerate}
904: \item Split the previous two sections into separate discussions of
905: competitive strategies and rationally competitive strategies.
906: \item Present rationalizable competitine strategies:
907: we would like to minize regret when the other players may use
908: an strategy that cannot be removed by a several steps in which dominant
909: strategies are removed.
910: In a third-price auction the first removal implies that agents will
911: submit at least their valuations; in a second stage this implies
912: that agents 2+ will be indifferent between submitting their valuations
913: and bidding 0; in a third stage this will imply that agent 1 will be indifferent between submitting its valuation and submitting 1; the next stage
914: will tell us however that agents may consider the submission of arbitrary
915: valuations...
916: \end{enumerate}
917:
918: \section{Folk-Theorems}
919:
920: We will look at games like prisoners dilemma, and general repeated
921: games, and talk about competitive strategies there.
922: