The Civil War in Development Economics

Few people outside academia realize how badly Randomized Evaluation has polarized academic development economists for and against. My little debate with Sachs seems like gentle whispers by comparison. Want to understand what’s got some so upset and others true believers? A conference volume has just come out from Brookings. At first glance, this is your typical sleepy conference volume, currently ranked on Amazon at #201,635.

But attendees at that conference realized that it was a major showdown between the two sides, and now the volume lays out in plain view the case for the prosecution and the case for the defense of Randomized Evaluation.

OK, self-promotion confession, I am one of the editors of the volume, and was one of the organizers of the conference (both with Jessica Cohen). But the stars of the volume are the speakers and commentators: Nava Ashraf (Harvard Business School), Abhijit Banerjee (MIT), Nancy Birdsall (Center for Global Development), Anne Case (Princeton University), Alaka Halla (Innovations for Poverty Action), Ricardo Hausman (Harvard University), Simon Johnson (MIT), Peter Klenow (Stanford University), Michael Kremer (Harvard), Ross Levine (Brown University), Sendhil Mullainathan (Harvard), Ben Olken (MIT), Lant Pritchett (Harvard), Martin Ravallion (World Bank), Dani Rodrik (Harvard), Paul Romer (Stanford University), and David Weil (Brown). Angus Deaton also gave a major luncheon talk at the conference, which was already committed for publication elsewhere. A previous blog discussed his paper.

Here’s an imagined dialogue between the two sides on Randomized Evaluation (RE) based on this book:

FOR: Amazing RE power lets us identify causal effect of project treatment on the treated.

AGAINST: Congrats on finding the effect on a few hundred people under particular circumstances, too bad it doesn’t apply anywhere else.

FOR: No problem, we can replicate RE to make sure effect applies elsewhere.

AGAINST: Like that’s going to happen. Since when is there any academic incentive to replicate already published results? And how do you ever know when you have enough replications of the right kind? You can’t EVER make a generic “X works” statement for any development intervention X. Why don’t you try some theory about why things work?

FOR: We are now moving in the direction of using RE to test theory about why people behave the way they do.

AGAINST: I think we might be converging on that one. But your advertising has not yet got the message, like the JPAL ad on “best buys on the Millennium Development Goals.”

FOR: Well, at least it’s better than your crappy macro regressions that never resolve what causes what, and where even the correlations are suspect because of data mining.

AGAINST: OK, you drew some blood with that one. But you are not so holy on data mining either, because you can pick and choose after the research is finished whatever sub-samples give you results, and there is also publication bias that shows positive results but not zero results.

FOR: OK we admit we shouldn’t do that, and we should enter all REs into a registry including those with no results.

AGAINST: Good luck with that. By the way, even if do you show something “works,” is that enough to get it adopted by politicians and implemented by bureaucrats?

FOR: But voters will want to support politicians who do things that work based on rigorous evidence.

AGAINST: Now you seem naïve about voters as well as politicians. Please be clear: do RE-guided economists know something the local people do not know, or do they have different values on what is good for them? What about tacit knowledge that cannot be tested by RE? Why has RE hardly ever been used for policymaking in developed countries?

FOR: You can take as many potshots as you want, at the end we are producing solid evidence that convinces many people involved in aid.

AGAINST: Well, at least we agree on the on the much larger question of what is not respectable evidence, namely, most of what is currently relied on in development policy discussions. Compared to the evidence-free majority, what unites us is larger than what divides us.

Shamans and Development Experts

Robert Wright writes in his fascinating book, The Evolution of God:

There is in the world today a great and mysterious force that shapes the fortunes of millions of people…There are people who claim to have special insights into this force…Most of them have often been wrong about the future, and many of them have been wrong most of the time. In fact, it’s not clear their advice is worth anything at all…Nonetheless, {they} have a profitable line of work... Why? Because whenever people sense the presence of a puzzling and momentous force, they want to believe there is a way to comprehend it. If you can convince them that you’re the key to comprehension, you can reach great stature.

 Wright is not  talking about religion, but about stockbrokers and the stock market. Wright correctly points out that much research demonstrates that stockbrokers do NO better than you and me at beating the stock market. Yet we pay stockbrokers.

Wright then discusses shamans in hunter-gatherer societies, which meet a similar demand – somebody who claims to be able to influence the vast uncertainties of hunting failures, weather, and disease, which makes hunter-gatherers feel better. To Wright, stockbrokers meet the same religious need as shamans.

 With both shamans and stockbrokers, it helps a lot that we humans are really bad at distinguishing pure randomness from real accomplishment (as this blog is fond of pointing out).

 There is always one shaman/stockbroker who is performing above average due to sheer luck. We refuse to accept that it’s pure luck and we are drawn to this shaman/stockbroker. Then he (it’s usually a he) reverts to average. Again, we refuse to believe this is random, we say that he has now lost his touch and we switch to following the new above-average champion. So our general faith in the power of shamans/stockbrokers remains intact.

Some of you have guessed where I am going with this. Development experts are a lot like stockbrokers. To get specific, development experts offer the hope that their expertise will raise a country’s growth rate, despite the lack of ANY clear empirical evidence that they know how to raise a country’s growth rate.

 Few can accept the current consensus of the empirical growth and development literature -- that we know something about what works in the long run for development (institutions, human capital, etc.) but virtually nothing about how to raise growth in the short to medium run. As Abhijit Banerjee puts it:

it is not clear that the best way to get growth is to do growth policy of any form. Perhaps making growth happen is ultimately beyond our control. Maybe all that happens is that something goes right for once (privatized agriculture raises incomes in rural China) and then that sparks growth somewhere else in the economy, and so on. Perhaps we will never learn where it will start or what will make it continue. (In new book edited by Jessica Cohen and I, “What works in development: thinking big and thinking small”)

 Yet the demand for development experts is as intense as ever. For example, the World Bank spent $4 million on an expert-laden Growth Commission to raise growth, issuing a report in 2008, with this finding:

It is hard to know how the economy will respond to a policy, and the right answer in the present moment may not apply in the future.

Despite the possibly limited utility of this breakthrough,  the World Bank Growth Commission is now being emulated elsewhere.  The UK Department for International Development just spent £37 million on assembling academic experts into a new International Growth Centre (IGC) that will promote “sustainable growth in developing countries by providing demand-led policy advice based on frontier research.”

Relentless demand for growth experts despite lack of evidence for growth expertise may have a faith-based component, just like the demand for shamans and stockbrokers.

 Maybe I shouldn’t spend so much time mocking the “growth experts” who claim to know something they don’t (especially the day after being named one of the Top 100 Global Thinkers for being a “growth expert”).  Maybe I should just accept that people are going to keep listening to shamans on growth and development, and maybe shamans could even play a useful role. How to make sure that those shamans at the very least, do no harm, and maybe even recommend sensible things?

 First, select the shamans on academic merit. Second, such quality academics should stick to their guns that nobody should waste time on experts directing growth from the top based on expert knowledge that does not exist. Such a top-down expert approach would actually do harm in unintentionally strengthening authoritarian and excessively statist approaches. The development shamans could still perform inspirational religious rituals as long as they don't actually affect anything.  Third, there are still plenty of useful things that academics can say about more specialized areas, such as health, financial regulation, macroeconomic management, education, clean water, and agriculture.

Indeed, the DFID International Growth Centre assembled a jaw-dropping list of stellar academic economists, fulfilling requirement (1). On (2) and (3), what seems to be going on at IGC is “bait and switch.” A few at the top spuriously promise the ability to raise growth, while the quality academics just get on with applying their specialized knowledge to specialized problems. At least there are good economists saying good things at the bottom.

Let’s hope that the power is shifting from the development shamans, who should be reassuring but powerless, to useful economists with useful knowledge.

Friday Round Up

Monkeys Do Markets In a recent experiment, a team of scientists trained a vervet monkey to open a container of apples, a task no other monkey in her group could do. She was well-compensated for this service by the other monkeys, who began to spend a lot of time grooming her (apparently, grooming is the monkey unit of exchange). Then, the scientists trained another monkey in the group to get the apples, and the “price” for the service (ie the amount of grooming the apple-providing monkeys received) went down. NPR Correspondent Alex Bloomberg explained:

[W]hen there was a monkey monopoly on the skill, the monkeys paid one price. But when it became a duopoly, the price fell to an equilibrium point, about half of what it had been. And this all happened despite the fact that we're talking about monkeys here. Monkeys can't do math.

What’s the point, other than research studies are really bizarre? Acquiring a sought-after new job skill leads to a higher income, even among monkeys. And, monkey markets can still set prices, even though the market participants can’t add, sign contracts, or talk. And, perhaps, complex markets can be the product of an unintentional, spontaneous order:  Out of the chaos of many monkeys running around hitting one another on the heads, pulling nits off each other’s fur, following only the simple rules of monkey hierarchies and monkey appetites…a functioning market emerges.

The Most Remote Place in the World is Three Weeks from Anywhere

Along the lines of our recent post, Africa Desperately Needs Trade Links: A Pictorial Essay, check out this feature from the New Scientist.

Bad Bosses Suck (Worse than War?)

I can think of lots of reasons why a local aid worker in Iraq might forego a secure paycheck and quit their job. Long lines and indignities at the security checkpoints to get in and out of the Green Zone every day. The dangers inherent to working with foreigners, like the threat of kidnapping or injury to themselves or their families.

But a paper based on conversations with local and international aid staff working in Iraq found that staff attrition and high turnover was more commonly caused by plain old bad bosses and poor treatment of staff. That’s not to say that poor management and dangerous environments aren’t linked in some causal way.  The paper pointed out that difficulties of aid worker life in hostile environments, like the lack of frequent contact with beneficiaries, problems building trust, and disparities in the amount of risk assumed by Iraqis vs international staff, magnify the effects of bad management.

I’m sure these “lessons learned” are old news to anyone who’s done aid work amidst hostilities.  But they are worth noting this week as observers of the attack on the UN guesthouse in Kabul asked whether there will soon be a Green Zone in Afghanistan, and in light of last month’s decision to bump up the amount of non-military aid the US gives to Pakistan, which may (or then again, may not, depending on how the aid is distributed) give aid workers a larger footprint there.

China in Africa

Finally, a couple notable books out to shed light on the little-understood subject of China’s  aid to Africa: The Dragon's Gift: The Real Story of China in Africa by Deborah Brautigam, and China into Africa: Trade, Aid, and Influence, a collection of essays published by Brookings.

Set a Big Goal. Give All to Meet It. This is Stupid.

The first two sentences come out of thousands of commencement addresses, not to mention inspirational foreign aid addresses. But they’re bad advice. Social entrepreneurs in foreign aid might learn from private sector entrepreneurs, who don’t stick to fixed goals.

A University of Illinois graduate moved to Silicon Valley with a great goal (perhaps inspired by the Illini commencement address) – develop security software for hot-selling handheld devices like the Palm Pilot. He assumed that enterprises were soon going to be using Palms as primary means of communication and sharing documents, and would need security to protect business secrets. “Any minute now, there’ll be millions of people begging for security on their handheld devices,” he thought. He was wrong – he never found a demand for handheld security software.

He could have kept trying to make his original idea work. Entrepreneurs that do stick to fixed goals are very good at least at one thing – wasting investors’ money. An idea for an online grocery startup, Webvan, managed to go through $1 billion before finally pulling the plug.

Illinois Man was different. He shifted to Plan B. Sell his cryptography software. Still no takers. We can skip over Plans C, D, and E, which all failed.

Plan F was a system for securely transferring cash from one Palm Pilot to another. He put a demo on the Internet so people could see how great it would be for Palm Pilots. People liked the web demo and started using it for real transactions, while the demand from Palm users still failed to materialize. eBay users started asking if they could put the web demo in their ads for people to pay them. There was no demand for the product, only for the web demo.

Illinois Man finally realized what might succeed. He forgot about Palm Pilots. Plan G was a system for making secure online payments for sites like eBay. His Plan G company was called PayPal, and his name was Max Levchin. eBay eventually bought PayPal for $1.5 billion. The story is from a new book by John Mullins and Randy Komisar, Getting to Plan B.

This principle translates to social entrepreneurship. Mohammed Yunus was trained as an irrigation economist. If he had stuck to fixed goals they would have involved irrigation. We would never of heard of him or of microcredit (at least until someone else without fixed goals came along).

Why is entrepreneurship so hard? It is always very uncertain what you can do well that the customers will want. Finding that sweet spot is a process of trial and error and gathering feedback. Research cited by Mullins and Komisar shows it takes 58 new product ideas to develop one successful product.

An even better book on this theme of searching for what works rather than sticking to the pre-conceived plan is Bill Duggan’s Strategic Intuition. Duggan uses a nice military analogy. Why was Napoleon such a successful general? Before Napoleon, military strategy was about how to take a fixed position from the enemy. Napoleon realized that the point of a war was to defeat the other army. He said, forget the targets, just keep the army moving relative to the other army until you are in the most advantageous position, then attack.

Of course, this advice about flexible goals might conflict with previous advice about specialization. Yes, you want to get the gains from specialization, but be flexible about what your specialization will produce for the customers. So you might be a health specialist, but which among many interventions that you could do will pay off? James Grant, a former director of UNICEF, knew he wanted to save children’s lives, but he was open to any way to do it. He and his staff stumbled on oral rehydration therapy, which has since saved millions of babies from dying from diarrheal diseases at a cost of about 12 cents per dose.

The economists’ version of this mentality is to keep doing and redoing your cost/benefit analysis. The rational next step is one with the highest ratio of benefits to costs. Many goals are set that ignore cost/benefit analysis altogether (Millennium Development Goals, for example). Other goals might be set with some vague notion of predicted costs and benefits, but these predictions are usually wrong. You have to be willing to adjust as benefits and costs turn out to be different that expected, and often shift to another activity altogether – Plans B,C,D,…Z.

Obviously the official aid world of MDG Plans and Poverty Reduction Strategy Papers does not have a clue about the entrepreneurship needed to solve problems.

Official aid doesn’t know how to operate in a world where most Problems are solved by entrepreneurs who originally intended to solve a DIFFERENT Problem. So it’s up to you private and social entrepreneurs.

In which I don’t care about genocides that kill only .01 percent of the population

My WSJ review on Tracy Kidder’s book on the Burundian genocide survivor generated this comment from a reader (abbreviated here, the full version is posted as a comment on the blog): Mr. Easterly,

You point out that "only" 0.01% of Africans have been killed by war and genocide... each year... for the past four decades. This is only slightly higher than the percentage of Europeans who died in the Holocaust each year between 1940 and 1945, meaning that Africa has merely suffered something like a 40-year Holocaust.

In fact, 0.01% is significantly lower than the percentage of Americans killed each year in the second world war (0.08% or so, on average), a minor conflict barely mentioned in writings of the time. During the Vietnam conflict we were losing only about 0.002% of our population each year for about 16 years and people would barely shut up about it.

Thus I propose that we adopt 0.01% of the population as the Easterly Threshold, requiring that any discussion of a conflict failing to achieve this level of decimation include a disclaimer that most of the population has not, in fact, yet died. Where populations are suitably difficult for us Americans to distinguish from one another, this percentage will be calculated on an arbitrarily continental or sub-continental basis. This immediately puts the whole history of the 20th century in a much rosier light: using the Easterly Threshold, a group like the Khmer Rouge barely clears the hurdle, massacring just 0.012% of Asia's population in a year.

Regards,

Jonathan Custer

Lakeland, Florida (soon Birmingham, England)

Dear Mr. Custer,

Congrats on your tour de force demolishing my argument that nobody should care about genocides that kill only 0.01 percent of the population or less.

You force me to admit that if a genocidal soldier killed one of my own loved ones, I myself would get only moderate comfort from the statistic that this corresponded to an American death rate of only 0.000000333 % (1 out of 300 million).

Your argument is so skillful, let’s not get pedantic that my article never made the “only” argument; it actually said that the .01 percent statistic is also “of no comfort to Africans today who are victims of still much too frequent horrors; bless anyone who can stop the horrors or help the victims.”

I was foolishly hoping the .01 percent number might induce the casual reader to re-examine his belief that the typical African family consists of a wife-beating alcoholic male and starving refugee females raped by child soldiers, soon after massacred by the janjaweed just before they would have died of AIDS anyway.

On correcting stereotypes, consider the Willie Horton ad of the presidential election of 1988 of the George Bush, Sr. vs. Michael Dukakis. A political group allied with Bush ran an ad featuring a scary picture of Willie Horton (see also the video), a black man in prison for murder whom Dukakis granted a weekend furlough. He then raped a woman while on furlough. The ad is partially credited with winning the election for Bush.

I would argue that white voters over-reacted in their fears of black crime. The propensity of black males to commit crimes is lower than the general public thinks, and other whites, not blacks, commit most crimes against whites. According to your interpretation, my attempt to correct a stereotype means I don’t care about the victims of Willie Horton. So this is a good opportunity to clarify I am not, in fact, in favor of rape and murder. I'm not that keen on genocide either.

Actually, I can do two things at once: (1) argue against exaggerated stereotypes and (2) care about the victims of crimes regardless of whether they fit stereotypical patterns. But thanks for your argument forcing me to clarify this.

Satirically Yours,

Bill Easterly

Burundi-based aid worker pushes back further on Burundi stereotype

Dear Professor Easterly, A former colleague from Wellesley forwarded me your WSJ review of Tracy Kidder's Strength in What Remains.

I live in rural Burundi, and wanted to thank you for challenging the apparent depiction of this beautiful and complex nation as "a place of unrelieved poverty, violence, disease and human degradation." Burundi is certainly very poor, and I am working with the landless Batwa, by far the poorest of the poor. But I am more struck by affirmations of human dignity than I am by human degradation, and I live quite safely here as a single woman. Public health in Burundi is what one might expect in a nation still emerging from a long civil war, but even in healthcare, there are bright spots; e.g., a well-functioning national tuberculosis plan. As it happens, I was at the Village Healthworks clinic in Kigutu on Thursday for the funeral of Déo's father (I know one of his brothers), and was impressed with its facilities and programs.

I was struck by your question about whether anyone would ever write a book about the Ghanaian economist who uses his well-earned success in the west to contribute to a peaceful democracy at home. Will anyone ever write a book about Déo's classmate Jeanne-Odette Niyongere, who completed advanced training in France and is now a gynecologist on the Faculty of Medicine at the University of Burundi and married to an economist in the Ministry of Finance, with whom she is raising four children in a comfortable home in Bujumbura? Or about my Mutwa colleague Béatrice Munezero, who during the civil war founded a school for Batwa and other children at risk that has now reached the 8th Grade?

You refer to Déo's escape from the Hutu slaughter of Tutsis during the genocidal year of 1994. This gives the impression that Burundi's situation was much the same as Rwanda's at that time. Burundi and Rwanda are aptly described as "faux jumeaux" -- fraternal rather than identical twins in a strict translation of the phrase, but also siblings whose comparison tends to play one or the other false. Burundian Hutus did rise up and slaughter Tutsis in late 2003 after the assassination of the first democratically elected president, Melchior Ndadaye. Ndadaye, who was also the first Hutu president, was apparently tortured to death by members of the Tutsi-dominated army. This was the same army that in 1993 killed 300,000 Hutus in response to a suspected plan to topple the Tutsi-controlled government. Once it secured Bujumbura in late 1993, the army moved into Burundi's interior, where the killings of Tutsis had started, and began the indiscrimate slaughter of Hutus. One of my colleagues described late 1993 to me as follows: "When President Ndadaye was murdered, Hutus began killing Tutsis with clubs and machetes. A few weeks later, the army arrived with machine guns and grenades and began hunting men like beasts."

All this is to say that the situation in Burundi was and is very complex, and that Burundians know this and are free to talk about it. You may also be aware of this. I am grateful to Tracy Kidder for having written a book to put Burundi on the map for westerners, and to you for having so thoughtfully reviewed it. In the forest of books on Rwanda (of which there seems to be at least one for each of the thousand hills that usually figure in their titles), it's good to hear of a Burundian tree.

Sincerely,

Jodi Mikalachki

Education and Community Development Worker

MCC Burundi