Epistemic Models

Game Theory is a fascinating subject, since it is concerned with the reasoning of human beings about other human beings. More precisely, it studies the behavior of people who reach decisions, knowing that the final outcome will not only depend on their own decision, but also on the decisions reached by other people. In order to evaluate the consequences of his own decisions, the decision maker (also called player) must therefore form a subjective belief about the behavior of the others. Once such a belief has been formed, the player is expected to choose the decision (also called strategy) that is optimal for him,  given his belief about the opponents' behavior. This is the characteristic of a rational player.

One of the central questions in game theory is: Which beliefs about the opponents' behavior may be viewed as reasonable? This is crucial, since strategies that are only optimal against unreasonable beliefs seem unreasonable themselves.  At the same time, this question is not that easy to answer since, intuitively, a belief about the behavior of an opponent is "reasonable" if one expects the opponent to choose strategies that are optimal for him against a "reasonable" belief: a circular argument.

In order to analyze the problem of which beliefs may be viewed as reasonable and which not, it is extremely helpful to model the way in which players may reason about the possible choices, beliefs, and ways of reasoning of their opponents.  Epistemic models do exactly this: They provide a formal language that allows us to make statements such as "player 1 believes that player 2 will choose strategy x" or "player 1 believes that player 2 believes that player 1 will choose strategy x", etcetera. In other words, epistemic models provide a language that is able to formalize how players reason about each other. In my opinion, the development of such epistemic models has been one of the most valuable achievements in modern game theory.

One of the advantages of an epistemic model is that it allows us to develop a formal theory of rationality. For instance, one can formalize statements such as ''player 1 chooses rationally", "player 1 believes that player 2 chooses rationally", or "player 1 believes that all other players believe that all players choose rationally", and so on. One can even formalize the following sequence of statements: player 1 chooses rationally, player 1 believes that every player chooses rationally, player 1 believes that every player believes that every player chooses rationally, player 1 believes that every player believes that every player believes that every player chooses rationally, ad infinitum. This is called common belief in rationality, and it form the basis for most rationality concepts in game theory.

In my research, I am especially interested in epistemic models and rationality concepts for dynamic games. The crucial difference between dynamic games and static games is the problem of belief revision. In a dynamic game, a player may reach several decisions over time, and before each decision the player may completely or partially observe the decisions that have been reached by the others so far. It is therefore possible that player 1 observes that player 2 has reached decision x or y, whereas he believed until that moment that player 2 would actually choose decision z. In this situation, player 1 must revise his belief about player 2's strategy choice, and also perhaps his belief about player 2's belief. The crucial question here is: How should player 1 revise his belief? This is exactly the question that I have been investigating during the last few years, and that I will probably investigate for some years to come. More about this question can be read on the page about my research.  

On the Interpretation of Nash Equilibrium

Many courses in Game Theory start with the discussion of Nash equilibrium.  One reason is that it has played a fundamental role in the development and application of Game Theory. Another reason, which is probably as important, is that the mathematical definition of Nash equilibrium is easy: A Nash equilibrium is a profile of probability distributions over pure strategies, one for each player, such that every probability distribution is a best response against the other distributions. Hence, if there are two players, a Nash equilibrium is simply a pair of probability distributions that are mutual best responses.

The interpretation of Nash equilibrium is, however, far from easy, at least in my opinion.  A possible interpretation that is often used is to view the probability distribution over player i's pure strategies as player i's mixed strategy. That is, player i randomizes by choosing each of his pure strategies with a certain objective probability, for instance by tossing a coin or dice. A Nash equilibrium would then be a profile of mixed strategies, with each mixed strategy being optimal given the mixed strategies of the opponents.

I see at least two problems with this interpretation. First of all, I do not believe that people in practice randomize when reaching serious decisions. In fact, I am not aware of a correct practical example in which such randomization occurs. Also from a theoretical point of view, there is no reason to expect an expected utility maximizer to randomize between pure strategies: Such randomization would only be optimal if each of the pure strategies involved would be optimal by itself. But if this is so, then why not play one of these optimal pure strategies (with probability 1)? I have read and heard some misleading examples, which seem to suggest that randomization occurs in practice, but, when looking at them carefully, do not. Take, for instance, the example of the tennis player who must choose where to serve: in the left corner or in the right corner. Some people claim that this tennis player uses a mixed strategy, by saying that he purposely varies between left and right.  However, each time the tennis player serves he has already made up his mind, and will either send the ball to the right with probability 1, or send the ball to the left with probability 1. So, in fact he is using a pure strategy in which he varies between left and right over time, but no randomization occurs. Another misleading example I have read is about an employer who decides to randomly monitor  an employee, without informing the employee about when he will be monitored. The claim that is being made is that the employer is using a mixed strategy here. However, the employer actually chooses the points in time at which he wants to monitor the employee, and he chooses such a monitoring plan with probability 1. Hence, if the set of pure strategies for the employer is defined correctly, one must conclude that he chooses a pure strategy, and not a mixed strategy.

Another problem I see with the aforementioned interpretation of Nash equilibrium lies in the phrase "...each mixed strategy being optimal given the mixed strategies of the opponents". This seems to suggest that a player, when choosing a strategy, knows what the opponents are planning to do. More than once, students have asked me how a player can know what the others will do. I think these students are completely right: there is no reason to assume that a player can correctly forecast the behavior of others. In fact, when thinking about it, it seems completely irrelevant for a player whether or not he can correctly guess the real behavior of his opponents. What matters for player i's strategy choice is his subjective belief about the opponents' behavior. On the basis of this belief, he decides which strategy to choose. Whether or not the opponents act in accordance with these beliefs is not relevant anymore, since it will then be too late for the player to change his decision.      

This leads me to a second interpretation of Nash equilibrium that is often used, namely one in which the probability distribution over player i's pure strategies is interpreted as the opponents' probabilistic belief about player i's pure strategy choice. In a two player game, a Nash equilibrium would then consist of player 1's probabilistic belief about player 2's pure strategy choice and player 2's belief about player 1's pure strategy choice, such that each player 2 strategy that is assigned positive probability by player 1's belief must be optimal for player 2, given his belief about player 1's choice, and vice versa.

An advantage, in my opinion, of this interpretation, is that it no longer assumes that players randomize. Probabilities arise because they reflect a player's uncertainty about the opponent's behavior, and this seems perfectly reasonable. However, I still see a problem with this interpretation. I have problems, namely, with the phrase  "...each player 2 strategy that is assigned positive probability by player 1's belief must be optimal for player 2, given his belief about about player 1's choice". This seems to suggest that player 1 is aware of the true belief that player 2 has about player 1's choice. Again, I see no reason why player 1 would be able to correctly guess player 2's belief. Moreover, for player 1's decision problem it seems completely irrelevant whether or not he is eventually right about player 2's belief. What matters for player 1 is his belief about player 2's belief. It is on the basis of this second order belief that he forms his belief about player 2's choice, and, in turn, this belief about player 2's choice leads him to his own strategy choice.

This reasoning leads me to a possible interpretation of Nash equilibrium that I, personally, find compelling. For simplicity, take a two player game, and consider a probability distribution p over player 1's pure strategies, and a probability distribution q over player 2's pure strategies.  A possible interpretation of p and q is that they both reflect player 1's personal view of the game. More precisely, p is player 1's belief about player 2's choice, q is player 1's belief about player 2's belief about player 1's choice, p is player 1's belief about player 2's belief about player 1's belief about player 2's choice, and so on. Hence, one views the game completely from player 1's perspective, and p and q are both objects that personally belong to player 1.  Within this setting, saying that p and q are a Nash equilibrium amounts to saying that player 1's belief about player 2's choice is justified by player 1's belief about player 2's belief about player 1's choice. Moreover, player 1's belief about player 2's belief about player 1's choice is justified by player 1's belief about player 2's belief about player 1's belief about player 2's choice, and so on. As you can see, this interpretation no longer assumes that player 1 correctly forecasts player 2's strategy choice or beliefs. Within this interpretation, a Nash equilibrium (p,q) is an object that completely belongs to the mind of a single player.  It is therefore possible that different players hold different Nash equilibria, while participating in the same game.

On the Interpretation of Strategies in Dynamic Games

Consider the following dynamic game between player 1 and player 2: In round 1, player 1 may exit or stay. If he exists, the game is over. If he stays, round 2 will start where player 2 may choose between exit or stay. If he exits, the game is over. If he stays, round 3 will be reached where player 1 may choose between outcome a and outcome b. By definition, (exit,a) would be a behavioral strategy for player 1, prescribing the choice exit in round 1, and the choice a in round 3. 

In order to judge whether this object (exit,a) is intuitively meaningful or not, one should be explicit about the interpretation of its ingredients exit and a: Do they represent actual choices by player 1, or beliefs about player 1's choices?

If both exit and a are to represent actual choices by player 1, then the pair (exit,a) does not make intuitive sense as a choice  plan for player 1. Namely, if he implements this plan correctly, he would exit in the first round, so the third round would never be reached. Hence, to say that player 1 would choose a in round 3 is completely redundant, since one may expect that a serious decision maker will be careful enough to correctly implement his choice plans.  In fact, if player 1 would be in a position to choose a, this would only possible if he is not executing the choice plan (exit,a). As a choice plan for player 1, it would therefore be enough to say that he chooses exit at the first round, and that's it.

This problem of interpreting behavioral strategies as actual choice plans often leads to confusion. Take, for instance, the concept of subgame perfect equilibrium, which is defined by means of behavioral strategies. Suppose that the behavioral strategies (exit,a) for player 1 and stay for player 2 constitute a subgame perfect equilibrium in the game described above. One cannot justify player 2's choice of stay by interpreting (exit,a) as a choice plan for player 1. Namely, if player 2 is called upon to move, he knows that player 1 has chosen stay in the first round, and hence cannot be implementing (exit,a). The justification for player 2's choice of stay comes from the fact that player 2, in round 2, believes that if he would choose stay, then player 1 will choose a. Hence, the ingredient a in (exit,a) is best interpreted as player 2's conditional belief in round 2 about player 1's choice in round 3.

One could go one step further by interpreting the whole behavioral strategy (exit,a) as representing player 2's conditional beliefs about player 1's behavior. In this case, player 2 would initially believe that player 1 chooses the choice plan exit, but upon observing that player 1 has stayed in the first round, his revised belief is that player 1 chooses the choice plan (stay,a). Similarly, player 2's behavioral strategy can be interpreted as player 1's belief about player 2's choice plan. Within this interpretation, there is no need to define problematic choice plans like (exit,a) for player 1, in which choices are defined at situations that should not occur if this choice plan would be executed correctly.