Surprised By The Gambler’s And Hot Hand Fallacies? A Truth In The Law Of Small Numbers
Bocconi University – Department of Decision Sciences; IGIER – Innocenzo Gasparini Institute for Economic Research
Universidad de Alicante – Departamento de Fundamentos del Análisis Económico
September 15, 2015
IGIER Working Paper #552
Abstract:
We find a subtle but substantial bias in a standard measure of the conditional dependence of present outcomes on streaks of past outcomes in sequential data. The mechanism is a form of selection bias, which leads the empirical probability (i.e. relative frequency) to underestimate the true probability of a given outcome, when conditioning on prior outcomes of the same kind. The biased measure has been used prominently in the literature that investigates incorrect beliefs in sequential decision making — most notably the Gambler’s Fallacy and the Hot Hand Fallacy. Upon correcting for the bias, the conclusions of some prominent studies in the literature are reversed. The bias also provides a structural explanation of why the belief in the law of small numbers persists, as repeated experience with finite sequences can only reinforce these beliefs, on average.
Surprised By The Gambler’s And Hot Hand Fallacies? A Truth In The Law Of Small Numbers – Introduction
Jack takes a coin from his pocket and decides that he will flip it 4 times in a row, writing down the outcome of each flip on a scrap of paper. After he is done flipping, he will look at the flips that immediately followed an outcome of heads, and compute the relative frequency of heads on those flips. Because the coin is fair, Jack of course expects this empirical probability of heads to be equal to the true probability of flipping a heads: 0.5. Shockingly, Jack is wrong. If he were to sample one million fair coins and flip each coin 4 times, observing the conditional relative frequency for each coin, on average the relative frequency would be approximately 0.4.
We demonstrate that in a finite sequence generated by i.i.d. Bernoulli trials with probability of success p, the relative frequency of success on those trials that immediately follow a streak of one, or more, consecutive successes is expected to be strictly less than p, i.e. the empirical probability of success on such trials is a biased estimator of the true conditional probability of success. While, in general, the bias does decrease as the sequence gets longer, for a range of sequence (and streak) lengths often used in empirical work it remains substantial, and increases in streak length.
This result has considerable implications for the study of decision making in any environment that involves sequential data. For one, it provides a structural explanation for the persistence of one of the most well-documented, and robust, systematic errors in beliefs regarding sequential data|that people have an alternation bias (also known as negative recency bias; see Bar-Hillel and Wagenaar [1991]; Nickerson [2002]; Oskarsson, Boven, McClelland, and Hastie [2009]) |by which they believe, for example, that when observing multiple flips of a fair coin, an outcome of heads is more likely to be followed by a tails than by another heads, as well as the closely related gambler’s fallacy (see Bar-Hillel and Wagenaar (1991); Rabin (2002)), in which this alternation bias increases with the length of the streak of heads. Accordingly, the result is consistent with the types of subjective inference that have been conjectured in behavioral models of the law of small numbers, as in Rabin (2002); Rabin and Vayanos (2010). Further, the result shows that data in the hot hand fallacy literature (see Gilovich, Vallone, and Tversky [1985] and Miller and Sanjurjo [2014, 2015]) has been systematically misinterpreted by researchers; for those trials that immediately follow a streak of successes, observing that the relative frequency of success is equal to the overall base rate of success, is in fact evidence in favor of the hot hand, rather than evidence against it. Tying these two implications together, it becomes clear why the inability of the gambler to detect the fallacy of his belief in alternation has an exact parallel with the researcher’s inability to detect his mistake when concluding that experts’ belief in the hot hand is a fallacy.
In addition, the result may have implications for evaluation and compensation systems. That a coin is expected to exhibit an alternation “bias” in finite sequences implies that the outcome of a flip can be successfully “predicted” in finite sequences at a rate better than that of chance (if one is free to choose when to predict). As a stylized example, suppose that each day a stock index goes either up or down, according to a random walk in which the probability of going up is, say, 0.6. A financial analyst who can predict the next day’s performance on the days she chooses to, and
whose predictions are evaluated in terms of how her success rate on predictions in a given month compares to that of chance, can expect to outperform this benchmark by using any of a number of different decision rules. For instance, she can simply predict “up” immediately following down days, or increase her expected relative performance even further by predicting “up” only immediately following longer streaks of consecutive down days.1
See full PDF below.
