Month: January 2013

Reading Notes: “The Flaw of Averages” (Savage, 2009)

I just finished reading The Flaw of Averages – Why We Underestimate Risk in the Face of Uncertainty (2009) by Sam Savage, a professor at Stanford. The author states on his website that “[s]imply stated, the Flaw of Averages implies that”:

Plans based on average conditions are wrong on average.

The book deals with uncertain numbers (e.g. how many sales will product X have per month in the next year?), and more specifically, the erroneous forecasting of uncertain numbers due to incorrect use of averages. Savage distinguishes two forms of the Flaw of Averages:

  • Weak Form of the Flaw of Averages: using a single number (or regression line) in forecasting future values of an uncertain number, instead of taking into account the distribution of possible outcomes;
  • Strong Form of the Flaw of Averages: also screwing up the average itself. From page 83: “Consider a drunk staggering down the middle of a busy highway and assume that his average position is the centerline. Then the state of the drunk at his average position is alive, but on average he’s dead.”

Pages 130–132 list the Seven Deadly Sins of Averaging, which were first published in the article Probability Management in ORMS Today in 2006. In fact, the list has grown beyond seven since then. But Savage states on page 130:

I plan to go on calling them the Seven Deadly Sins regardless of how long the list becomes. Be sure to check in at to see where it stands today.

Both the 2009 edition of his book and today’s version of the website list twelve sins. Both lists reference scenarios explained elsewhere in the book. Therefore, I will quote sins 1 to 7 from the self-contained ORMS Today article; I will quote sins 8 to 12 from the book, and/or refer within []’s to online resources of my choice.

  • The Family with 1 1/2 Children: Often the “average” scenario, like the “average” family with 1 1/2 children, is non-existent. For example, a bank may have two main groups of young customers — students with an average income of $10,000 and young professionals with an average income of $70,000. Would it make sense for the bank to design products or services for customers with the average income of $40,000?
  • Why Everything is Behind Schedule: Imagine a software project that requires 10 separate subroutines to be developed in parallel. The time to complete each subroutine is uncertain and independent, but known to average three months, with a 50 percent chance of being over or under. It is tempting to estimate the average completion time of the entire project as three months. But for the project to come at three months or less, each of the 10 subroutines must be completed at or below its average duration. The chance of this is the same as flipping 10 sequential heads with a fair coin, or less than one in a thousand!
  • The Egg Basket: Consider putting 10 eggs all in the same basket, versus one by one in separate baskets. If there is a 10-percent chance of dropping any particular basket, then either strategy results in an average of nine unbroken eggs. However, the first strategy has a 10-percent chance of losing all the eggs, while with the second, there is only one chance in 10 billion of losing all the eggs.
  • The Risk of Ranking: It is common when choosing a portfolio of capital investment projects to rank them from best to worst, then start at the top of the list and go down until the budget has been exhausted. This flies in the face of modern portfolio theory, which is based on the interdependence of investments. According to the ranking rule, fire insurance is a ridiculous investment because on average it loses money. But insurance doesn’t look so bad if you have a house in your portfolio to go along with it.
  • Ignoring Restrictions: Consider a capital investment in infrastructure sufficient to provide capacity equal to the “average” of uncertain future demand. It is common to assume that the profit associated with average demand is the average profit. This is generally false. If actual demand is less than average, clearly profit will drop. But if demand is greater than average, the sales are restricted by capacity. Thus, there is a downside without an associated upside, and the average profit is less than the profit associated with the average demand.
  • Ignoring Optionality: Consider a petroleum property with known marginal production costs and an uncertain future oil price. It is common to value such a property based on the “average” oil price. If oil price is above average, the property is worth a good deal more. But if the price drops below the marginal cost of production, the owners have the option to halt production. Thus, there is an upside without an associated downside, and the average value is greater than the value associated with the average oil price. (…)
  • The Double Whammy: Consider a perishable inventory of goods with uncertain demand, in which the quantity stocked is the “average” demand. If demand exactly equals its average, then there are no costs associated with managing the inventory. However, if demand is less than average then there will be spoilage costs, and if demand is greater than average there will be lost sales costs. So the cost associate with average demand is zero, but average cost is positive.
  • The Flaw of Extremes: In bottom-up budgeting, reporting the 90th percentile of cash needs leads to ever thicker layers of unnecessary cash as the figures are rolled up to higher levels. Even more harmful things result from focusing on above- or below-average results, such as test scores or health-related statistics. (…) [From p138: T]he flaw of extremes results from focusing on abnormal outcomes such as 90th percentiles, worse than average cancer rates, or above average test scores. Combining or comparing such extreme outcomes can yield misleading results. (…) The smaller the sample size, the greater the variability of the average of that sample.
  • Simpson’s Paradox: [see Simpson’s Paradox (Wikipedia) and Chapter 18 online supplement]
  • The Scholtes Revenue Fallacy: [From p146: T]he Scholtes Revenue Fallacy occurs when revenue is the result of multiplying two uncertain numbers, such as (…) price and quantity. If the two uncertain numbers are inversely (negatively) interrelated, the average revenue is less than the revenue associated with the average uncertainties. If the two uncertain numbers are directly (positively) interrelated, the average revenue is greater than the revenue associated with the average uncertainties.
  • Taking credit for chance occurrences: We all like to take credit for our hard work, but some successes may be due to dumb luck. (…) [This is about null hypothesis (statistical) testing. See  Statistical hypothesis testing (Wikipedia) and Chapter 20 online supplement]
  • Believing there are only eleven deadly sins: The twelfth of the Seven Deadly Sins is being lulled into a sense of complacency, thinking you now know all of the insidious effects of averages.

Sam Savage did a great job: The Flaw of Averages is written in an amusing and down-to-earth style, and is a worthy read. If you don’t like mathematics, rest assured: no mathematical background or skill are required to enjoy it.

Further reading on statistics:



The Affirmation of Humanism: A Statement of Principles (Paul Kurtz)

UPDATE 2014-12-09: some readers may also want to check out the Reality of Morality project by The Brights Network (@TheBrightsNet), that provides an overview of scientific perspectives on human morality. And the accompanying list of 97 scientific studies that substantiate claims of morality’s natural origins.

In salute to skepticism and secular humanism — while respecting others worldviews —, I care to share The Affirmation of Humanism: A Statement of Principles, as published in 1997 by Paul Kurtz (1925-2012). Kurtz was a philosophy professor, a prominent American skeptic and secular humanist, and one of the greatest voices of reason in the last four decades. We owe the existence of the Council for Secular Humanism (known of the Free Inquiry magazine) and the Committee for Skeptical Inquiry (known of Skeptical Inquirer magazine) to many great minds, and Kurtz was one of the most prevalent among them. If you enjoy science & reason, consider subscribing to Free Inquiry and/or Skeptical Inquirer.

  • We are committed to the application of reason and science to the understanding of the universe and to the solving of human problems.
  • We deplore efforts to denigrate human intelligence, to seek to explain the world in supernatural terms, and to look outside nature for salvation.
  • We believe that scientific discovery and technology can attribute to the betterment of human life.
  • We believe in an open and pluralistic society and that democracy is the best guarantee of protecting human rights from authoritarian elites and repressive majorities.
  • We are committed to the principle of the separation of church and state.
  • We cultivate the arts of negotiation and compromise as a means of resolving differences and achieve mutual understanding.
  • We are concerned with securing justice and fairness in society and with eliminating discrimination and intolerance.
  • We believe in supporting the disadvantaged and the handicapped so that they will be able to help themselves. We attempt to transcend divisive parochial loyalties based on race, religion, gender, nationality, creed, class, sexual orientation , or ethnicity, and strive to work together for the common good of humanity.
  • We want to protect and enhance the earth to preserve it for future generations, and to avoid inflicting needless suffering on other species.
  • We believe in enjoying life here and now and in developing our creative talents to their fullest.
  • We believe in the cultivation of moral excellence.
  • We respect the right to privacy. Mature adults should be allowed to fulfill their aspiration, to express their sexual preferences, to exercise reproductive freedom, to have access to comprehensive and informed health-care, and to die with dignity.
  • We believe in the common moral decencies: altruism, integrity, honesty, truthfulness, responsibility. Humanist ethics is amenable to critical, rational guidance. There are normative standards that we discover together. Moral principals are tested by their consequences.
  • We are deeply concerned with the moral education of our children. We want to nourish reason and compassion. We are engaged by the arts no less than by sciences.
  • We are citizens of the universe and are excited by discoveries still to be made in the cosmos.
  • We are skeptical of untested claims to knowledge, and are open to novel ideas and seek new departures in our thinking.
  • We affirm humanism as a realistic alternative to theologies of despair and ideologies of violence and a source of rich personal significance and genuine satisfaction in the service to others.
  • We believe in optimism rather than pessimism, hope rather than despair, learning in the place of dogma, truth instead of ignorance, joy rather than guilt or sin, tolerance in the place of fear, love instead of hatred, compassion over selfishness, beauty instead of ugliness, and reason rather than blind faith or irrationality.
  • We believe in the fullest realization of the best and noblest that we are capable of as human beings.