Dear UK Government, why won't you let me retire as Official Sachs Critic?

UPDATE 3: FEB 8 4:50PM: Twitter War reveals that Millennium Village Blog accused Clemens and Demombynes of hard hearts towards suffering (search the blog for "suffering"). UPDATE 2: FEB 8 4:30PM: concluding coverage by @PSIHealthyLives @viewfromthecave of the Great Twitter War prompted by this post between @aidwatch and @earthinstitute, with collateral attacks on @m_clem, ending in a non-acceptance of debate by head of @earthinstitute.

UPDATE Feb 8 11:30am: sent comment with link to this post to DFID Independent Commission for Aid Impact. Spokesperson promptly responded, rejected my comment for consideration on technical grounds, but did warmly invite me to complete the anonymous online mass Survey Monkey. It probably doesn't mean much, unless the Independent Commission already learned the brilliant strategy of bureaucratizing the critics? I do feel a wee bit sorry for one of your Commisioners, the great John Githongo, who presumably did not risk his life so he could be reading anonymous results from Survey Monkey.

Nobody is more tired of the interminable Sachs-Easterly debate than one guy named Easterly...alas, I seem to be stuck in a kind of Critic Trap, in which the ideas criticized keep reappearing unchanged, requiring equally unchanged criticisms, keeping me in chronic peril of taking myself way too seriously.

So it was with great weariness I heard the news that the British aid agency DFID (otherwise probably the best bilateral aid agency) is close to financing a brand new Millennium Village in northern Ghana, near Bolgatanga. I had hoped for something better from the new UK government, which had seemed like an improvement over the Blair and Brown ("We know the answers, just double aid") team .

As it happened, I passed by the proposed MVP site last summer. The proposed villages are right on the main road in one of the most NGO-intensive places anywhere (see the sign below, in which NGOs apparently own the region).

The usual critique that selection bias of the Millennium Villages makes evaluation  impossible may be somewhat relevant given the political realities that (1) the current government chose the villages for the MVP, (2) the incumbents have frequently promised to do more for the North, (3) the MVP came along and may be a high visibility way to keep that promise, and (4) ergo, the government will likely do everything possible to make the project succeed, showing nothing about scalability for thousands of villages elsewhere. In short, this new MV may be about as informative as my feeding my own children is informative on whether child nutrition programs work.

And how good is the track record of the MVP taking evaluation seriously? Michael Clemens and Gabriel Demombynes posted the following on the World Bank Africa blog last Friday:

In a June 2010 report called Harvests of Development, the Project claimed that the impacts of the project included expanded cell phone ownership.  For example, the MVP claimed that increases in cell phone ownership at the Ghana site were caused by the project, in this extract from page 91 of the MVP report:

This claim has little basis, because cell phone ownership has been expanding at about the same rate all around the MVP site in areas untouched by the project. ....

But on Tuesday, months after multiple discussions we’ve had with MVP leaders on our research, a post on the MVP’s blog restated the claim that the increase in mobile phone ownership at the intervention sites was caused by the Project...

{The Clemens and Demombynes paper does the same cell phone analysis with the same results in the MV of Sauri, Kenya.}

They were responding to a blog post on the MVP web site on February 2, 2011 as follows:

Sauri looks back on five years of success

Infrastructure: ... The proportion of households owning a mobile phone has increased four-fold....

In short, independent observers made an irrefutable argument that a claim was invalid, the MVP heard the argument, seemed to accept it, and then repeated the previous claim unchanged.

Or in other words, if nobody is listening to any evaluations anyway, if I am bored and I am boring everyone else, why should I want to be Official Sachs Critic any longer?

Messrs. Clemens and Demombynes, you may want to check out a new job opening...

Fake it till you make it

A new report slams the UK’s aid agency with accusations of spending money on “Fake Aid.” Produced by the London-based International Policy Network (IPN), “Fake Aid” casts a critical eye on the agency’s communications programming, finding that “increasing amounts of DfID funds are channeled through non-governmental organizations (NGOs) to fund lobbying activities, marketing and the promotion of political ideology, often within the UK.” The report uses DfID documents to show that the aid agency uses its communications budget to promote the human-rights based approach to development (which we have debated a number of times on this blog), intentionally crowding out other approaches. The budget for DfID’s communications programs has more than tripled since 2000, from £40 million in 2000 to £140 million in 2008.

Much of this money is spent through a group of NGOs—including Oxfam UK, Save the Children UK, ActionAid and Christian Aid—which receive unrestricted DfID funds thanks to long-standing relationships with the aid agency, rather than a competitive bidding process. According to a UK government audit document, open bidding was introduced after most of these organizations were already selected, and currently remains closed to new applicants.

DfID also lacks adequate information to judge whether the funds given to these organizations are appropriately spent, since reporting practices have been characterized by self-reported, inconsistent data, and a lack of external, independent evaluations, again according to a UK government audit and DfID annual reviews.

For example, over a five-year period, DfID faulted the Catholic Agency for Overseas Relief (CAFOD) for failing to provide “an overview of the progress and impact of the programme as a whole” and criticized them for measuring inputs rather than results, but still renewed their funding in 2008 for £13.8 million.

One organization funded by DfID persuaded the government of Gambia to ban all-inclusive package holidays, an outcome which seems to contradict the UK’s policy on trade and development to promote the private sector as an important driver of poverty reduction.

The effectiveness of another organization, both founded and entirely funded by DfID to “provide a forum for BME [black and ethnic minority] voluntary and community sector organizations and communities on issues relating to international development,” was challenged by an independent audit. The audit found that there was no working email address for almost half the group’s members, and that “there is a lack of clarity over the purpose of the organization.”

Despite these examples of poor practice, we would suggest a few points of disagreement with the conclusions in “Fake Aid”:

First, it is not unreasonable for DfID to spend money educating UK tax payers about what DfID does, or about international development issues, as the report implies. IPN calls this “propaganda” but doesn’t convincingly explain how “propaganda” is different from “making people aware of concepts that IPN happens to disagree with.”

Second, while £140 million is a sizable sum (enough for 230 million malaria treatments says IPN) it is still less than 3 percent of DfID’s total budget. In an industry with no consistent financial practices, it’s hard to say what is normal, but it would come as no surprise to find that most nonprofits, and most aid agencies, spend more than 3 percent on communications and publicity.

“Fake Aid” concludes: “As the total amount spent on these programs reaches the £1 billion mark, it is reasonable to ask whether they have improved the lives of people in poor countries.” It is a fair question. These programs may be an appropriate use of DfID funds, but the burden falls to DfID to prove it.

USAID: Don’t Ask, Don’t Tell

USAID says on its web site: “The effective functioning of our constitutional democracy depends upon the participation in public life of a citizenry that is well informed.” USAID also has signed onto an inspirational document signed by all aid agencies around the world promising full disclosure of information. Anything else would be hypocrisy when the aid donors are constantly preaching to poor country governments that they should be “transparent.” Which is why we were wondering: why have we still not gotten an answer to questions on USAID reporting on aid tying we first asked USAID five weeks ago? This was in response to our blog on USAID aid tying information on February 24th. We had been having trouble getting ANY response at all from the USAID press office ever since, despite sending no less than nine polite emails (and one slightly less polite email) and equally numerous voicemail messages their way. Our hopes flickered briefly when the USAID press office left us a voice mail on March 13, promising to get back to us right away. Since then, not a word, more emails from us, and … silence.

Bending over backwards, we thought we’d give them another shot, looking for data on USAID assistance broken down by country and by sector (which we easily got in five minutes with one phone call to the British aid agency DFID – see below).

USAID has a general inquiry line, which is supposed to connect you to a “team of knowledgeable information specialists [who] can point you to the sources that have the information that you need.” Thank goodness, a live person answered the phone. But from there, things went downhill. No one whom we were able to speak with in three hours of phone calls and research on the USAID website seemed to really understand our question, and only after many tries did we reach someone who could make a credible guess about which office to direct our questions to.

At one point we were referred to the Congressional Budget Justification, an 873-page document created to request funds from Congress. At another point, we were told that we would need to gather the data from each country desk separately. This would be difficult, since the data provided on each country webpage does not always seem to be uniform or comparable, and most often just refers you back to the mammoth CBJ. USAID’s random aid numbers are so confusing and so scattered that the USAID staff themselves apparently can’t make sense of them, judging by their inability to answer simple questions.

Is it just intrinsically impossible for aid agencies to be responsive to questions and data requests? A few weeks ago, we happened to be looking for data on how much the UK aid agency DFID spends on the type of aid known as budget support, across several African countries. After a few minutes of trying to manipulate some unwieldy OECD data sets, we clicked over to the DFID website. They also had a public inquiry line listed right on the ‘Contact Us’ page. When we dialed the UK number, a live person answered the phone. This person clearly understood the question, and transferred the call directly to another knowledgeable, live person who also understood exactly what we were looking for. She told us where the data was located on the DFID website, and then actually guided us to it in exactly five mouse clicks. We had clear, user-friendly data and a precise answer to our question in less than five minutes.

(We were tough on DFID on their budget support practices, but we praise them to the skies for opening themselves up to public scrutiny so we could discuss the issue at all.)

Why does this matter? Well USAID was right that “a citizenry that is well informed” is one of the only hopes to hold public agencies accountable, and thus improve the likelihood that USAID dollars actually reach some real poor people.

President Obama has inspired great hope by promising improved US government transparency, but maybe USAID ignores the President as much as they ignore the citizens.

Update: USAID has since emailed us back about our sectoral data request! Unfortunately the analyst at the USAID Knowledge Services Center who responded did not point us to any better data sources than those we had already found in our fruitless three-hour quest the day before. On the bright side, though, the email does vastly improve USAID’s track record on answering emails.