The Perils of Not Knowing that You Don’t Know

Suppose you needed to get from New York to Washington for a personal emergency. An airline told you that the projected time of departure for your one-hour flight was 2pm this afternoon. Of course, there is some uncertainty about this. The unstated promise of the airline is that this uncertainty will be kept within familiar bounds, with delays up to an hour not unusual, and delays of several hours possible with unexpected weather or mechanical failure. But suppose the airline secretly knew that due to a large set of unreliable links in the chain like a flight attendants’ union that might strike, a large risk of major engine failures throughout its fleet, and problems with the mechanics’ union that might prevent repairs, that the possible delays were not in HOURS, but in DAYS. You would be angry at the airline for not telling you this, and once you knew how much they didn’t know, you would quickly drop the airline and take the train instead.

The moral of the story is that knowing how much uncertainty there is about a projection – that is, knowing how much the projector DOESN’T KNOW – is often more important than the projection. An “estimated departure time based on the best available information” is meaningless to an airline customer if the uncertainty is about DAYS rather than HOURS.

Not knowing that you don’t know is at the root of many recent disasters, starting with the crisis itself. Holders of opaque derivatives apparently didn’t know how leveraged and exposed they were to shocks like falling housing prices, nor did they know how uncertain the housing prices were. Outside of economics, not knowing what we didn’t know is one of many causes for bad outcomes in Iraq and Afghanistan.

Not knowing that you don't know was our concern about the World Bank's global poverty projections. In his response on this blog and on the New York Times blog Economix, Ravallion misses our point. It is the degree of uncertainty of his poverty projection that is unacceptable. He said: “Faced with all these perceived “impossibilities,” Easterly and Freschi would apparently prefer to wait and see rather than take action when it is needed, based on the information available at the time.” Ravallion called our stance “analytic paralysis in the face of uncertainty.” He portrays us as unwilling to live with ANY uncertainty, which is ridiculous. Economics is filled with uncertainty.

But when the uncertainty is so large as it is with the poverty projections (for all the MAJOR reasons we pointed out, which Ravallion does not address*), then the implication is not “paralysis” but choosing actions that take into account the uncertainty. For example, you DON’T want to have a centralized bureaucracy like the World Bank allocate global poverty relief based on such wild uncertainty. It would be better to support local coping mechanisms (public or private) that flexibly respond at the local level using local knowledge about crisis effects like poverty, hunger, and dropping out of school.

An article in the Wall Street Journal illustrated well how uncertain theoretical economic predictions can be used in the very un-theoretical realm of influencing how scarce aid resources are directed, and to whom:

A joint development committee of the World Bank and the International Monetary Fund estimates the lingering financial crisis could drive an additional 90 million people around the world into "extreme poverty." To combat the staggering statistic, the World Bank is aggressively lobbying for its first capital increase in two decades.

Not only would it be the wrong response to channel poverty relief through the centralized bureaucracy, but confusion is here piled on uncertainty. The capital increase is NOT for lending to the poor countries, but to the middle-income ones, as was made clear in a number of other news stories which explained that the money would be shared between the IBRD, the branch of the Bank which lends to middle-income countries on near-commercial terms, and the IFC, which lends to companies.

This only strengthens our original argument that the poverty projection was a political exercise. We respect Ravallion’s academic work, but this exercise seems to be in a different category. Not knowing what you don’t know is indeed dangerous.

*The closest Ravallion comes is citing his paper that tests the assumption of distribution neutrality of growth ON AVERAGE, which still misses our point about variance around the average. His claim that what people thought would happen before the crisis is a good benchmark for what would have happened without the crisis simply begs the question of the accuracy of country growth forecasts--not responding to our concern that they are radically unreliable (with or without crises). Since his point is invalid, it remains true that his projection of the effect of the crisis on growth is based on no meaningful evidence.