Best and Worst of Official Aid 2011- new release

By Claudia Williamson, Post-Doctoral Fellow, Development Research Institute Rhetoric on “aid effectiveness” keeps escalating, is there anything to show for it?

The past (almost) two years, Bill and I have been collecting data, combing through that data, and refining the numbers to ‘grade’ aid agencies and assess overall trends in aid practices. We waited until our paper passed peer review to release our findings. Rhetoric versus Reality: The Best and Worst of Aid Agency Practices has now been accepted for publication in a special issue of World Development. {{1}}

Our work updated Easterly and Pfutze’s 2008 study, Where Does the Money Go: Best and Worst Practices in Foreign Aid, on five dimensions of agency ‘best practices’: aid transparency, minimal overhead costs, aid specialization, delivery to more effective channels, and selectivity of recipient countries based on poverty and good government.  Based on these measures, we calculate an overall agency score using original data and 2008 OECD data. These scores only reflect the above practices; they are NOT a measure of whether the agency’s aid is effective at achieving good results.

There is slight improvement in transparency and more donors are moving away from ineffective channels. But transparency is still at unacceptably low levels. For example, two agencies (MOFA Japan and France’s DgCiD) fail to report any aid data at all.

The most conspicuous failures in both trends and levels are in specialization and selectivity. Luxembourg is as unspecialized as the US with a 70^th of the aid flow. Many such unspecialized small donors likely have most of their aid eaten up by fixed costs before the funds reach any beneficiaries. At the same time, allocation to corrupt countries is increasing, not decreasing. Aid to corrupt autocrats is not explained by emphasis on the least developed countries; donors such as the US, Sweden, and Norway do poorly on both income selectivity and autocracy/corruption selectivity.

The best bilateral agency is UK’s Department for International Development (DFID).

DFID is one of ten agencies that fully reports aid flows to OECD, and it lists number of staff, administrative costs, salaries and benefits and its ODA budget on its website. DFID also has relatively low administrative costs and salaries and benefits relative to aid disbursements (2.6% and 1.6% respectively). DFID relies on more effective channels of aid disbursements, not tying any of its aid and dispersing relatively little food aid (1.3%) (pages 53-54).

Japan, New Zealand, and Germany also do well, rounding out the top five best agencies.  The United States ranks below average mainly because of poor performance on selectivity and choosing to allocate aid through ineffective channels. As we write in the paper, “the foreign policy needs of the US superpower and the lobbies for particular aid channels seem to dominate the politics of American aid” (page 54).

Another theme that emerged is that the Scandinavian countries’ reputation of altruism based on aid volume does NOT translate to good practices; they have below average scores on specialization and transparency and are mediocre in the overall ranking.

Lastly, the UN agencies on average are worse than the other multilateral agencies and the bilateral agencies, and the differences are statistically significant. Above all, they are worse on overhead and transparency. On overhead, they have an average ratio of 46 percent of administrative costs to ODA. UNDP reports no data on its operating costs or ODA, now even worse than its minimal reporting in 2008.

The two goals of the paper were to test if: 1) donors’ rhetoric matches reality; and 2) they are making any improvements in doing so. Our answer is no on both accounts.

Postscript: Fortunately, we are now part of a larger community running independent checks on aid. For other recent aid quality exercises, see Birdsall and Kharas, 2010; Knack, Rogers and Eubank, 2010; and Ghosh and Kharas, 2011.

[[1]]The dataset for the paper can be downloaded here[[1]]

World Vision Super Bowl Shirts: the Final Chapter

Remember back in February when World Vision’s proud announcement that they were sending abroad 100,000 Super Bowl champion T-shirts emblazoned with the name of the losing team, as they have for the last 15 years, provoked aid blogger ire? We’ve been following the controversy—and occasionally piling on joining in—and here’s the latest. In an email to Aid Watch, World Vision disclosed that total transport and administrative cost per T-shirt was 58 cents, which is uncomfortably high relative to low market values (a quick spot check  produces estimates ranging from 20 cents to $1.20 for a T-shirt) in Africa's saturated second-hand clothing markets.

World Vision also sent us documents from two districts in central in Uganda* that received donated clothing, although NOT specifically the loser Super Bowl T-shirts that started this whole controversy. We learned that the donated clothing was used as part of World Vision’s health programs which aim to “improve access to better health services, safe water and sanitation.” Specifically, World Vision said:

Provision of clothing was done for women and children in extremely poor conditions to protect them from weather and to raise their self-esteem. Providing clothing also served to increase trust among the beneficiaries and encourage them to participate in other health services, including voluntary counseling and testing for HIV.

This led us to focus on the health and HIV/AIDS sections as we sifted through the documents for answers to two questions which arose in the debate.

First question we asked: Can World Vision show that they rigorously assess the need for gifts-in-kind in the communities where they work?

World Vision answered: Needs assessments are carried out by national offices, and the rigor of these assessments varies from office to office.

What the documents showed: World Vision sent us one program design document from the final phase of a 12-year, multi-sector program that ended in 2010, and one needs assessment from a neighboring region (WV couldn’t find the needs assessment for the 12-year project).

The needs assessment identified the most important problems faced by the community, and made recommendations how WV should deal with them. It did not discuss at any point the clothing needs of villagers, or how clothing donations might alleviate any of the problems mentioned in the 67-page report.

The program design documents, intended to “point out gaps that still exist in the community as expressed by the people,” made only one mention of gifts-in-kind. “Gifts in kind will be planned for on annual basis and this is meant to supplement the project fund in achieving project planned activities.”

The main report did not mention a need for clothing. However, I did learn that the region described is among those most heartbreakingly affected by HIV/AIDS, with high numbers of orphans and child-headed households, and after some digging I found an HIV/AIDS sub-report embedded within the main report that did mention clothing:

Most of these [orphaned children] lack care and support in terms of emotional coping, physical requirements like food, shelter, clothing, and limited access to basic social services like education and health.

Another embedded sub-report (actually a proposal for outside funding to support HIV/AIDS orphans in the area) was more specific:

Special needs will be identified for each of the selected families and the project will organize to procure and provide the essential needs for the children and guardians. These will include beddings, bicycles, clothing, cooking pans, washing basins and water tanks.

Our conclusion on the first question: No.

Second question we asked: Can WV point to any evidence that the 15-year distribution of Super Bowl T-shirts, or, more broadly, any distribution of clothing, has "facilitate[d] good, sustainable development"?

World Vision answered: No, “because the Superbowl clothing isn’t a program. It’s a donation. We evaluate the results of our programs…many of the programs where we use GIK have been enormously successful in facilitating good, sustainable development. Our evidence for that would be individual program evaluations from a variety of national offices.”

What the documents showed: WV sent us one annual report and program evaluations for each phase of the same 12-year project discussed above. After hours of reading, a picture emerged of a community decimated by the HIV/AIDS epidemic and valiantly struggling to provide support for the large populations of vulnerable children made orphans or adopted into already over-stretched extended families.

An annual report from 2006 gave the only specific accounting of the type of gifts-in-kind distributed:

GIK was received and distributed to children and these included 115 pairs of canvas shoes, 50 pairs of baby shoes, 900 T- shirts, 225 Gin trousers, 500 pairs of socks, 125 dolls and 200 blankets. This benefited 1615 children in the community.

In a report from the first phase of the project, evaluators noted that some villagers were able to sell eggs from a poultry project to buy clothes (this shows that clothing is available for purchase in the community, and probably not at prohibitive prices for most people). Clothing was also mentioned as an obstacle to achieving the program’s “Christian Witness” objective: the poor don’t attend church because “they lack good cloth to put on and feel not worth attending.”

Regarding World Vision’s ability to show success in facilitating sustainable development through their programming in general, the 2006 evaluation said “tracking changes…attributable to World Vision support” is “quite difficult” because over the course of the 12-year project priorities and goals shifted, and because early baseline measurements don’t match up with later evaluations.

Nonetheless, the final report attributed many positive health outcomes to project activities. For example, reduced malaria incidence; improved sanitation practices; and reduced prevalence of HIV/AIDS.

We don’t see any basis for attribution of these outcomes to World Vision, since the program was not designed in such a way to make such attribution possible. The resources provided by World Vision—clinics built, medicines supplied, HIV awareness courses given—are characterized as improving health outcomes, but also as very thinly spread over a large area with acute health needs.

As to sustaining project gains as WV funding ends, WV reported that local organizations have been trained in skills like proposal writing, resource mobilization and networking so that they can take over WV services. Villagers in the final survey said they learned “vocational, business management, leaderships, improved farming, HIV/AIDS care, positive parenting, and sanitation management skills,” all of which would provide a “pillar to further development in this area.”

Our conclusion on the second question: While we appreciate WV’s transparency in sharing these documents with Aid Watch, we have to conclude that the answer is no. There is no real evidence in these hundreds of pages of reports that the clothing donations are more than a minor afterthought to World’s Vision’s health programming (although gifts-in-kind are a major source of World Vision’s revenue). Given the aforementioned costs required to ship donations from the US abroad there is no development-related reason to continue this outdated, dependency-creating practice.

*World Vision asked us not to publish the names of the regions, or any other identifying information about the projects.

--

Related posts:

In Zambia, Pittsburgh won the Super Bowl: Why is World Vision perpetuating discredited T-shirt aid?

World Vision responds to blogger questions

State Department accountable through glossy photos

by Chris Coyne, F.A. Harper Professor of Economics at the Mercatus Center at George Mason University

Investors in the Kwality Kites Corporation gather to listen to the CEO’s ‘year in review’ presentation.

“In 2010,” begins the CEO, “we coordinated plans to deliver kites while supporting sustainable operations”

An investor raises her hand: “Can you tell us what you mean by ‘coordinated plans’ and ‘sustainable operations’ and what they have to do with the bottom line?”

“Of course” the CEO replies. “Here is a glossy picture of a smiling child flying one of our kites.”

“That is indeed a high quality photograph with a fine sheen” the investor responds, “but I am wondering how this relates to my investment.”

“In 2010,” the CEO continues, “we improved our understanding of our dynamic business environment, established and staffed various units, assisted in a professionalization program, and assessed our efforts.”

Another investor interjects, “What is the cost of these achievements? What is the value added? What was the outcome of these assessments?”

“These assessments” the CEO replies “are now being used to inform and guide our future efforts to address various issues.”

The investors protest almost in unison: “can you provide us with ANY specifics? We are all concerned about the profitability of the company.”

“Can you really measure the value of a child smiling?” replies the CEO.

--

I admit this is an absurd parody. That is, unless you’re the Office of the Coordinator for Reconstruction and Stabilization (S/CRS), charged with implementing aid and other operations of the U.S. State Department in war zones like Iraq and Afghanistan. S/CRS recently released its “Conflict Prevention and Stabilization Operations: 2010 Year in Review” report.

The report provides no clear benchmarks for assessment let alone a basic discussion of the annual budget or expenditures. There is no discussion of why activities were chosen, how much was spent, or even the slightest effort to discuss the value added.

The report does provide a few actual numbers. We are told that the S/CRS oversees $442 million associated with “1207 programs” intended to respond to “destabilizing events,” In 2010, $90 million was approved for these projects. The ‘assessment’ concludes that “existing projects will take several years to complete” and that “previously funded programs continue to have an impact.” We learn that the Civilian Response Corps was deployed to 28 posts in 2010.

With no actual effort to assess the reported ‘highlights,’ we must rely on the pictures provided to distract the reader from the lack of content. This includes a half page picture of a young boy flying a kite in Afghanistan and numerous employee profiles including pictures with locals. Also included is a before and after picture of a dirt road which has since been paved. Still no discussion of cost or value added.

There is a major difference between my parody and the work of the S/CRS. If my story were true, the investors would lose their money. But ineffective efforts of the S/CRS will cost resources and human lives. If the S/CRS is unable to provide rudimentary reporting and transparency, how confident can we be that they can stabilize and reconstruct entire societies?  Answering this question is too important to be distracted by smiling children…even if they are flying kites.

Why development history matters for the Millennium Villages Project

by Ed Carr, Associate Professor in the Department of Geography at the University of South Carolina A growing volume of critical writing on the Millennium Villages project (MVP) includes blog posts, journalistic pieces, scholarly works, and, recently, one partial social impact study. Nearly all point to project outcomes that could have been avoided had the project seriously engaged with the long history of field-based experiences in development.

Here, I will focus on just one example: Because the MVP did not critically evaluate the effect of its own assumptions about what works in development, a conflict between project goals and the needs of the villagers has emerged in at least one site.

The MVP is part of Millennium Promise, an effort to make progress towards the Millennium Development Goals (MDGs).  As a result, the MVP framed its interventions around the MDGs.  For example, in 2005 the MVP website described community participation in this MDG-centric manner:

An open dialogue [between MDG-trained teams and villagers] will cover topics such as local problems as related to the MDGs, constraints and opportunities for achieving the MDGs at their village level, initial discussions on possible solutions and approaches for achieving the MDGs, and general impressions/consensus on being included as a Millennium Villages Project site.

The project’s founders have stated that the MVP was built on the “core truth” that there are “known packages of effective and generally low-cost interventions” that can address poverty.  A review of the MVP described it as a pilot project seeking to “provide successful evidence of how to achieve the Millennium Development Goals”. The project’s focus on finding “successful evidence” for the efficacy of these packages of interventions suggests that the project has an interest in validating the importance of the problems identified in the MDGs and justifying the interventions of the MVP.

This creates a conflict of interest for the field staff of the MVP: What if the evidence does not show success? And what to do when the local community’s concerns do not align with either these solutions or the MDGs?

Those familiar with the history of development work know that such conflicts of interest are chronic. Take the classic by Robert Chambers:  Whose Reality Counts. He describes what happened when he examined a consultant's glowing report on a World Bank irrigation scheme and found evidence that the conclusions were wrong:

My points were more or less accepted, but then the matter was consigned to an indeterminate limbo.  Nothing was done.  Far from being rejected or modified, the consultant’s conclusions were published unchanged, and without reference to the criticisms....The consultants knew that the World Bank, which had commissioned the study, was keen to justify the new approach.  They knew what result was wanted.  Supported by the consultants’ unchanged report, the new approach was implemented on a large scale.  So, even if bad news is reported, it may be avoided, rejected or finessed out of sight. (p.82)

Another disconnect appeared in a UNDP/OECD evaluation of a project in Mali: “it has to be asked how the largely positive findings of the evaluations can be reconciled with the poor development outcomes (1985-1995) and the unfavorable views of local people.” (1999)

Similarly, a classic work by James Ferguson (1994) recounts a World Bank project to teach better farming techniques in a mountainous region of Lesotho, out of touch with local people who had long ago learned to abandon the poor soils of that region and work as migrants in South African mines.

There are the same significant pressures on the MVP field staff to press participants to conform to project assumptions and expectations, and to reject or finesse evidence and feedback that does not. Those designing and implementing the MVP should have addressed possible conflicts between their goals and those of the communities. They did not. As a result, I was not surprised to see this quote from a woman living in a Rwandan Millennium Village, from a recent study:

The MV has to meet with local community to learn more about what people really want because sometimes the MV brings things that the community doesn’t need or want.

This and several other issues with the MVP were easy to see from the outset (see here and here). But to recognize them required a familiarity with the history of development and a self-awareness that the Millennium Village Project itself has never shown.

--

Ed Carr is an associate professor in the Department of Geography at the University of South Carolina.  His book Delivering Development: Globalization’s Shoreline and the Road to a Sustainable Future was released by Palgrave Macmillan on February 1, 2011.  He blogs at Open the Echo Chamber.

Read all Aid Watch posts on the Millennium Villages project here.

Eternal sunshine of the useless charts

After all the blogging we’ve done on how hard it is to find complete and accurate information (as opposed to “success stories”) on USAID’s website, I think we’d be remiss not to mention a new US government site launched just before the holidays. The Foreign Assistance Dashboard is the first version of a site that will someday allow users to create charts and tables showing where and how well US aid funds have been spent.

On the plus side, it looks good, makes pretty charts, and it’s easy to use. In future iterations, US officials have said that it will publish data in an internationally comparable format. (This is important so that recipient countries, which receive aid from so many different donors, can get a full picture of aid inflows.)

The page on Pakistan tells us, for example, that $3 billion has been requested from Congress for Pakistan in 2011; $1.6 billion of that is for “Peace and Security,” and most of that is specifically for “Stabilization Operations and Security Sector Reform.”

On the minus side, it’s missing most of the information that actually matters to anyone tracking where the money goes and measuring its impact. The country information pages are incomplete because they exclude funds allocated to regional offices rather than country offices. And, as you can see from the below chart, the site has only data from USAID and State, and only shows appropriated amounts, not what has actually been spent.

While I admire the guts it took to publish such an aspirational matrix, I fear the day may still be far away when we will see a nice row of Xs in that last performance data column. Still, a recent editorial from transparency guru Owen Barder reminds why that is a goal worth pushing for:

The shift to a global information standard for aid sounds a rather dull and technocratic change, but a common standard for sharing information unlocks a world of possibility. It will enable the information from multiple aid agencies to be easily used by governments, parliaments and citizens in donor and developing nations.

It democratises aid, removing the monopoly of information and power from governments and aid professionals. It inspires innovation and informs learning. It reduces bureaucracy. It also makes it possible for communities to collaborate, for citizens to hold governments to account and for the beneficiaries of aid to speak for themselves.

A rare glimpse at censorship in action in real time on the Net

I was really pleased recently to get a link to a blog, which from the link description strongly agreed with me on my controversial Lennon vs. Bono piece in the Washington Post, also featured on Aid Watch. I mean really pleased -- my roster of supporters just doubled! I dropped the neighbor’s baby that I was holding and rushed over to my computer to click on the link, waiting with growing excitement as the link slowly loaded, to give me...to give me...finally...and...finally...and...again...finally......

Well, at least, I got an unintended rare glimpse of the world of blog censorship LIVE and in REAL TIME, self-imposed and otherwise. As the to-remain-anonymous anonymous witness-protection-protectee told me in an anonymously wistful anonymous email:

“He's too strong.  He's just too strong.”

Can we get the World Bank to say the D- word?

UPDATE 10/12 1PM: we have a winner! (see end of post) UPDATE: No winnners yet, see end of post.

Following last Friday's post on the New Yorker profile of Justin Lin, I had this email exchange with the World Bank media officer David Theis, who kindly responded promptly to my inquiries.

Original Inquiry Fri, Oct 8, 2010 at 11:30 AM:

David Theis Media Chief, World Bank

Dear Mr. Theis, As I am sure you are aware, the current New Yorker has a profile of {World Bank Chief Economist} Justin Lin, especially his advocacy of an authoritarian development model. Does this reflect World Bank policy? In other words, is it official World Bank policy to endorse the authoritarian approach to development? If not, does the World Bank endorse instead a democratic approach to development? or does it simply take no position? Many thanks, Bill Easterly

Reply Fri, Oct 8, 2010 at 12:29 PM:

Bill, No, we are not advocating an authoritarian development model. In fact, Bob Zoellick's recent speech at Georgetown University (http://go.worldbank.org/5VEUBEBHY0) is entitled "Democratizing Development Economics." Many thanks. David

My follow-up question Fri, Oct 8, 2010 at 1:15 PM:

David, thanks so much for being so responsive. If you don't mind, a follow-up question. Mr. Zoellick's speech you mention is using "democratizing" in a different context. He does not bring up the issue of democratic vs. authoritarian regimes in developing countries. So when you say that you are "not advocating an authoritarian development model," I am unclear whether you are saying you are against this model, or whether you are neutral. Could you please clarify? Many thanks, Bill

His reply Sat, Oct 9, 2010 at 9:55 AM:

Bill, I believe "we are not advocating an authoritarian development model" is quite clear. Thanks. David

Bonus Reader exercise: find the word Democracy on any official World Bank website, or in any speech by Mr. Zoellick, or in any other official report authored by the World Bank.  The winner will receive two free tickets to the launch of the World Bank's new Policy Research Report: "D#m#cr#cy: Not Advocating Its Savage Repression."

Footnote: I also corresponded with David Theis on another question in the same series of letters that remains a bit unclear , and will be featured in a future post.

UPDATE: 3:30 pm no convincing winners yet as far as the World Bank offering an official embrace of Democratic Values, as opposed to isolated reports by individual authors and a few stray Zoellick remarks. You have got to do better, guys!! Or is it impossible?

UPDATE 10/12 1PM: We have a winner...

...except in reverse. Since nobody was able to provide a compelling example of the World Bank affirming democratic values, Aid Watch decided to give the prize to David Ellerman for his piquant comment that he was once forced to substitute the word "participation" for "democracy" in a major World Bank speech. This allows Aid Watch to selflessly quote its own previous posts deriding participation as a meaningless buzzword , which goes all the way back to colonial times and was therefore not seen as inconsistent with even Imperial Autocracy.

TransparencyGate: the end of the road

by Till Bruckner, PhD candidate at the University of Bristol and former Transparency International Georgia aid monitoring coordinator. Sixteen months after I first filed a Freedom of Information Act request with USAID for the budgets of American-financed NGO projects in Georgia, I have reached the end of the road. Rejecting my appeal, USAID has confirmed that it continues to regard NGO project budgets as “privileged or confidential” information, and will not release budgets without contractors’ permission.

The opacity of USAID’s subcontracting makes it impossible for researchers to get access to comprehensive and comparable data that could inform debates about the effectiveness of delivering aid through NGOs. For example, the issue of aid fragmentation within NGOs could only be raised because Oxfam GB voluntarily provided a researcher with a list of all its projects abroad.

USAID is on very thin ice when it tries to push developing country institutions to become more accountable. The next time USAID lectures an African official on the importance of transparency in public procurement, I hope she will pull out a list of blacked-out budgets and argue that her ministry is following American best practice when it treats all financial details of its subcontracting arrangements as “privileged or confidential.”

Financial opacity also remains the default position for most NGOs. CARE and Counterpart instructed USAID to release more information in response to this FOIA, and they deserve credit taking for this step. However, USAID’s latest information release suggests that no other NGO has given the green light for such information sharing.

The recent public statements by NGOs and other aid actors reveal wildly divergent understandings of what accountability should mean in practice. As InterAction points out, “the issue at hand is what constitutes relevant information, and to whom specific information should be disclosed.”

What information is relevant? Scott Gilmore argued that we should be interested in accountability for outcomes rather than for expenditures, and many commentators on this blog have questioned the desirability or utility of public access to NGOs’ salary figures or NICRA rates, and raised concerns about privacy, security and competitive disadvantages.

I continue to believe that project proposals, including uncensored budgets, are essential components of a meaningful rendering of account. Proposals spell out what an NGO plans to achieve, when, where, why and how, and at what cost. If we don’t even know what a project sets out to do, and with what resources, how can we hold it to account for its success or failure?

Equally, there is disagreement on who qualifies as a legitimate stakeholder. CNFA and Mercy Corps have both emphasized that they feel themselves obliged to render account to institutional donors and beneficiaries, but not necessarily to third parties. This line of argument glosses over the sad reality that NGOs do not reveal project budgets to their beneficiaries either. Also, as charities enjoy tax-exempt status and spend public money, we are all donors, like it or not. And we all care about the beneficiaries, so we are all “aid watchers”.

If project budgets are not particularly relevant, and scrutiny by ordinary citizens does not bolster accountability, why do international NGOs regularly make their local sub-grantees post project budgets in public places for all to see? As far as I know, no Northern NGO has worried that such excessive transparency may compromise the privacy, security or competitiveness of community-based NGOs in the South.

This FOIA journey has shown one thing above all: NGOs (save Oxfam GB) simply do not want outsiders to see their project budgets, full stop. Not a single NGO has used this forum to announce its willingness to give beneficiaries or other stakeholders access to its project proposals and budgets in the future, even though every country director has these documents on his hard drive and could attach them to an email within two minutes.

Project budgets are shown only to those stakeholders who have the power to force NGOs to open their books: donors, headquarters, and audit institutions. The poor and powerless have to be content with whatever information NGOs choose to provide.

Can NGOs be accountable without showing outsiders where the money goes? The Humanitarian Accountability Project thinks so. “Public disclosure of financial information is not a requirement for HAP membership,” HAP recently confirmed. InterAction concurs, stating that it “purposefully does not define in our standards specific mandates for disclosure.” InterAction also highlights “the request of some donors to keep their financial support private.”

Transparency International, drawing parallels to the oil and gas industry, strongly disagrees: “Competitive advantage or even privacy, are not acceptable exceptions. Only personal physical security suffices.” Aidinfo observes that “the burden of proof is shifting to those who would keep information secret.”

Donor-abetted secrecy jars with President Obama’s call at last week’s MDG summit: “Let’s resolve to put an end to hollow promises that are not kept.  Let’s commit to the same transparency that we expect of others.”

InterAction's statement on NGO accountability

Editor’s note: Aid Watch asked InterAction for a contribution to the debate originally sparked by Till Bruckner’s post The accidental NGO and USAID transparency test. See below for a list of all related posts. Statement from Barbara J. Wallace, InterAction’s Vice President of Membership and Standards, on NGO Accountability

Washington, DC (September 27, 2010)—InterAction appreciates the active discussion about NGO accountability and transparency, and has been monitoring the debate. The variety of opinions and information is valuable. This is not a new conversation for us. Our community of U.S.-based international NGOs has been discussing these issues for more than 20 years and they are the basis for development of our Private Voluntary Organization (PVO) Standards. These standards state that each member organization ‘shall be committed to full, honest and accurate disclosure of relevant information concerning its goals, programs, finances and governance.’ The issue at hand is what constitutes relevant information, and to whom specific information should be disclosed.

We deplore the nature of the debate taking place on your blog.  Highly inflammatory accusations have been made against NGOs—now proven to be untrue—using faulty research and questionable methodology. Whistleblowers bring disturbing information to light, but also bear responsibility for sharing accurate information based on fact. Instead of igniting a constructive conversation with a different take on accountability and transparency, this blog chose to smear the NGO community using conclusions from incomplete information that fit the author’s premise rather than engage in constructive discussion about what constitutes sufficient transparency, generating a lot of attention, but little constructive change.

InterAction purposefully does not define in our standards specific mandates for disclosure, nor do we dictate to any organization the details of their organizational management. InterAction’s membership is wide and diverse. Our largest members are worldwide and have total budgets over $1 billion. Our smallest has an annual budget under $60,000, works in one country and has no paid staff. Our members have varying operational models, methodologies and organizational capacities. This makes it almost impossible to compare the raw data from one organization with the raw data of another. The following caution in evaluating comparative detailed financial information may be useful as an example of this pitfall.

The ratio of indirect cost to direct cost or total cost varies and depends on many factors. It will be difficult if not impossible to get a definitive or measurable indicator for cost reasonableness since each organization has a different accounting/allocation methodology. That is to say there are numerous differences in both workforce and accounting classifications as to direct or indirect costs, as well as other variables such as the extent to which subcontractors are used, the structure of an organization, the expanding and declining business base for individual organizations, and the differing accounting methodology of one organization verses that of another. For example: one company may have a large labor overhead ratio to direct labor because it includes vacation and sick leave along with other types of overhead costs directly related to labor, while another organization will have a lower ratio because they direct charge vacation and sick leave. Neither practice is preferred over the other and both are equally acceptable. They are merely different. (www.usaid.gov/business/regulations/BestPractices/pdf)

In our working groups, annual Forum and CEO Retreats ongoing discussions about the responsibility of accountability and definition and value of transparency in a newly technologically advanced world are common themes.  In addition to the request of some donors to keep their financial support private (and the NGOs legal responsibility to honor that request),  organizational capacity, security concerns and other circumstances all have to be taken into consideration when determining whether or not specific information should be made public in part or in its entirety. All NGOs are not the same. That said,InterAction member organizations believe that informed citizens provide important insight, and they value the participation of informed local citizens in project design, implementation and evaluation and have used such insights to improve practices and methodology.

As we become an increasingly technologically advanced and interconnected society, NGOs will need to keep pace with new requests for information, and to balance questions of security, law, individual privacy rights, and proprietary methodology with this ever increasing demand.  InterAction and member organizations will continue to discuss NGO accountability, transparency and the impact of technology on our standards within its communities of practice, and will continue to wrestle with the increasing demands for public information and the reduction of unrestricted funding for operating expenses, which include this kind of public information management.

The discussion about what constitutes accountability under what circumstances and to whom continues. The most important focus of that discussion is what difference the NGO makes to the population it serves.

Related posts:

The accidental NGO and USAID transparency test Till Bruckner Responds to Critics on Meaningful Transparency NGO Response: CNFA Reaffirms Commitment to Transparency World Vision responds on transparency USAID and NGO transparency: When in doubt, hide the data Response from Mercy Corps on Transparency NGO Transparency: Counterpart International to release budget Transparency International clarifies the debate, deplores attacks on Till Bruckner Statement from CARE on Bruckner FOIA Request Return to TransparencyGate: Humanitarian Accountability Partnership weighs in

Return to TransparencyGate: Humanitarian Accountability Partnership weighs in

Editor’s note: Aid Watch asked HAP for a contribution to the debate originally sparked by Till Bruckner’s post The accidental NGO and USAID transparency test. See below for a list of all related posts. The HAP (Humanitarian Accountability Partnership) Secretariat is encouraged that issues of NGO accountability are being discussed in fora such as this, and in particular that the debate is now going beyond the sector.

While public disclosure of financial information is not a requirement for HAP membership,  members and other aid organisations that apply for certification with the 2007 HAP Standard in Humanitarian Accountability and Quality Management are expected to make their humanitarian plans and progress reports available to key stakeholders. Humanitarian plans include the overall goals and objectives, the timeframe and linked financial summary. Progress reports include progress against the plan, including against the financial summary, and are communicated at time intervals suitable to the different audiences.

As part of the HAP certification process, the auditor notes commitments made by the organisation, including in relation to financial integrity and reviews the management system that supports the implementation of commitments. Specifically linked to the current discussion, the auditor also verifies that the organisation has in place an information policy that includes narrow criteria for non-disclosure, and reviews how the organisation shares financial summaries and progress reports.

Financial transparency is an important aspect of an organisation’s public accountability, recognised as such by various initiatives, including HAP. However, financial transparency is not a sufficient indicator of accountability, in particular of accountability to the group that matters most, the people whom an aid organisation aims to assist. Financial details alone do not reveal much about the quality of the services provided to communities: do the projects deliver good quality services that meet the communities’ needs? Are the interventions appropriate in the specific context? Is the population that the organisation is aiming to assist included in the design and implementation of programmes? Are communities satisfied with programme delivery? Are they able to provide feedback and affect the way in which organisations that affect their lives work? Are they able to raise complaints and receive a response when programming is poor?  And last but not least: are organisations continuously improving the way in which they operate to best serve the needs of those they aim to assist?

The fact that an agency may not make its financial information publicly available at all times does not necessarily mean that it is not transparent or that it lacks accountability. There may be significant risks associated with sharing such information in certain contexts, in which case the organisation should provide a justification for non-disclosure.

Related posts:

The accidental NGO and USAID transparency test Till Bruckner Responds to Critics on Meaningful Transparency NGO Response: CNFA Reaffirms Commitment to Transparency World Vision responds on transparency USAID and NGO transparency: When in doubt, hide the data Response from Mercy Corps on Transparency NGO Transparency: Counterpart International to release budget Transparency International clarifies the debate, deplores attacks on Till Bruckner Statement from CARE on Bruckner FOIA Request

Heated debate with John McArthur on MDGs and accountability

In 2000, nearly every country in the world made a promise to achieve a set of eight goals, including poverty reduction, women’s empowerment and universal primary education by 2015. How far have we gotten? Host Michel Martin speaks with two opposing voices about the progress made this far: John McArthur, CEO of Millennium Promise, and William Easterly, professor of economics at New York University.

Listen to the interview on NPR's Tell Me More. Once in the media player, the segment is called "UN Convenes to Assess Global Progress"- it's 12 minutes long.

Maybe this is why accountability for Millennium Development Goals did not work out that well

Spot the made-up world hunger numbers

Today's exercise, dear readers, is to spot made up numbers in major news stories. Leading newspapers today report on FAO's new world hunger numbers (see FT and NYT). The FAO reports that the number of hungry people fell from 1.02 billion in 2008 to 925 million in 2009. That's very good news, unless it didn't happen.

Inquiring minds want to know:

(1) how did the FAO come up with a number for 2009, when the World Development Indicators (WDI) of the World Bank are only reporting malnutrition numbers up through 2008?

(2) how did the the FAO even come up with a number for 2008, when the current WDI reports malnutriton indicators (either height for age or weight for age for children under 5 ) for only 4 countries?

(3) the FAO says that two-thirds of the hungry are concentrated in seven countries: (in order of number, with WDI latest year of data between 2005 and 2008 reported in parentheses): China (none), India (2006), the Democratic Republic of Congo (2007), Bangladesh (2007), Indonesia (2007), Ethiopia (2005), and Pakistan (none). So how did they arrive at numbers for these countries for both 2008 and 2009?

(4)  is there any possibility that political pressure surrounding the hunger Millennium Development Goal (MDG) led to the creation of numbers based on the alternative methodology known as "wild guesses"? If so please explain why we cannot talk about the tragedy of world hunger without making up numbers?

(5) a good test of how serious is the UN about the hunger MDG is that it would have devoted a lot of effort to improving the data on hunger. Does the UN pass this test?

Perhaps the FAO has very good answers for these questions.  I am asking you to do some of the work here, dear readers, and investigate the mystery of the missing hunger indicators underlying non-missing precise world hunger data. I will write a follow up post as you and others respond.

Statement from CARE on Bruckner FOIA request

AidWatch received the following statement from CARE regarding Till Bruckner's AidWatch post on USAID and NGO transparency:

Statement from CARE (Aug. 30, 2010):

Contrary to what Till Bruckner suggested in a recent blog, CARE did not withhold information in response to his FOIA request to USAID regarding certain projects in the Republic of Georgia. Our records indicate that CARE never received the request from USAID to review CARE’s budget information before USAID provided it to Mr. Bruckner. USAID’s email request to CARE went to two inoperative emails; one was for a former employee and one went to a current employee, but the email address was incorrect.  As a result, the CARE document that USAID sent to Mr. Bruckner was redacted without CARE’s knowledge.

We have since reviewed the document and will ask USAID to produce it in full without any redactions, including our indirect cost rate, which was the primary information that had been withheld.

Transparency International clarifies the debate, deplores attacks on Till Bruckner

Editor's note: Transparency International Georgia submitted this contribution to the debate originally sparked by Till Bruckner's post The accidental NGO and USAID transparency test. We at TI Georgia have closely followed this debate about whether and to what extent USAID and its NGO contractors should make their budgets public. Till Bruckner began his quest for answers while he was working with us in 2008-09, although his pursuit of the NGO budgets via FOIA requests to USAID was not conducted under the auspices of TI Georgia.

Mercy Corps’ response to the debate begins by stating, “it is unfortunate that the discussion has devolved into insinuations about NGO motives rather than an open discussion of what constitutes meaningful accountability in aid work.” Yet nowhere in this debate has TI Georgia witnessed an attack against the motives of aid workers. Transparency can open up a discussion on the global aid system and how to better address problems that lack of openness can lead to in ANY organisation:  waste, inefficiencies, redundancies and, sometimes, fraud and corruption. As Scott Gilmore points out in his post, the impact is what matters.

The discussion on the blog has been engaging, open, honest and productive. But we do not agree with attacks against individuals who speak up against problems that they see. To call someone a “self-appointed watchdog” misses the point. Others would call Till a whistleblower – exactly the practice we should encourage and protect if we are serious about delivering on our development promises, protecting aid funds and transforming the aid system.

Mercy Corps proceeds to make an ad hominem attack against Bruckner by drawing attention to problems in an assignment he did on behalf of TI Georgia in 2009. Mercy Corps’ criticisms of the unpublished report are fair. That is why TI Georgia chose not to publish the report. But we fail to see how problems in drafting that report are relevant to the question of whether NGOs should publish their budgets or not.

Further, claims that raw budget data are not useful to measure NGO effectiveness are misguided. The question of aid effectiveness is tied up inherently with having access to ALL the information behind aid programs, allowing for comparison and analysis. While releasing project budget documents may not be a catch-all indicator for transparency of an NGO or donor, it is certainly a meaningful one.

Even if conditions are precarious, such as in humanitarian assistance work, there should be a clear NGO or donor policy on transparency, and organisations need to be open about which information cannot enter the public domain – and why. TI, in its handbook, recommends that financial information should only remain secret if its publication endangers staff or beneficiaries.

Mercy Corps argues:

NGOs have different cost structures and different methodologies, and budget documents reveal little about which are most effective.  Certain types of projects – such as technical assistance or gender-based violence prevention – tend by their nature to be heavy on labor costs and light on capital items, while food distribution or micro-lending tend to be lighter on labor costs and heavier on capital requirements.

There are two separate arguments above. The second is about differences between types of assistance projects. No one advocating for aid transparency has implied that one kind of cost structure is inappropriate. Let us see the numbers, assess them and discuss our concerns with you.

The first is an argument we hear from NGOs over and over again: that publishing their budgets will erode their competitiveness. This argument has not gotten the attention it deserves in this exchange. The most sensitive information in those budgets, even before salaries, is the Negotiated Indirect Cost Rate Agreements (NICRAs) that every NGO competing for USAID funds has, as Counterpart’s post highlights. NICRAs arrive in sealed envelopes and are carefully guarded secrets within the industry, differing widely in structure from one organization to another. Perhaps someone can explain why USAID contracting uses this system – presumably if all NICRAs were the same (or if they were all public), NGOs would be slightly more willing to disclose their budgets

In its current state, USAID’s system rewards secrecy and discourages public accountability. We applaud Bruckner for his efforts to raise these serious questions and we look forward to the viewpoints of more NGOs, USAID, and others on the topic.

Related posts:

The accidental NGO and USAID transparency test Till Bruckner Responds to Critics on Meaningful Transparency NGO Response: CNFA Reaffirms Commitment to Transparency World Vision responds on transparency USAID and NGO transparency: When in doubt, hide the data Response from Mercy Corps on Transparency NGO Transparency: Counterpart International to release budget

NGO Transparency: Counterpart International to release budget

Editor’s note: Aid Watch received the following statement from Counterpart International in response to a request for comment on Till Bruckner’s post The accidental NGO and USAID transparency test. We have checked our records regarding Mr. Bruckner's FOIA request to USAID for information about our Georgia program budget. Our server logs indicate that USAID's attempt in June to contact us about this FOIA request was unsuccessful because the message was sent to two former Counterpart employees, whose e-mail addresses were no longer active.  As a result, the Counterpart document that Mr. Bruckner received was redacted by USAID without Counterpart's input or knowledge.

With respect to Mr. Bruckner's specific FOIA request, we asked for and have now received from USAID a copy of its letter asking for our comments and recommended redactions. We have now responded, asking that only our NICRA rate - which is proprietary and competitive information - be redacted.  USAID has informed us that they will provide Mr. Bruckner with a revised copy of the budget reflecting this change.

--

Related posts:

The accidental NGO and USAID transparency test Till Bruckner Responds to Critics on Meaningful Transparency NGO Response: CNFA Reaffirms Commitment to Transparency World Vision responds on transparency USAID and NGO transparency: When in doubt, hide the data Response from Mercy Corps on Transparency

Our internal foreign aid program

The US Recovery Act (aka “stimulus package”) has put out this great map of where the money is being spent by Congressional District.

As I looked at where the money is being spent in the part of the country pictured (the part I know best), there did not seem to be a lot of rhyme or reason between Congressional Districts as far as population or need. Is it random? Could it be (shock!!!) that where the money is spent depends on the party, power, and skill of the Congressperson from that district?

The other interesting thing about the graph is the summation of total spending $218.737 billion and the creation of 749,142 jobs. Did it occur to them that somebody might divide the first number by the second and come up with the number per job (slightly less than $300K). Did anything think of just paying workers $300K directly, and letting them stay home and read poetry?

Of course, I am outside my area of expertise here. Perhaps the main point that I can make is that it is really GREAT that the US government was so transparent as to put these maps up (and many more – go to the web site!), so that citizens (hopefully including more intelligent commentators than me) can give feedback. When will foreign aid spending have maps like this? I hear rumors the day may be coming…

Response from Mercy Corps on Transparency

Editor's note: Aid Watch received the following from Mercy Corps in response to a request for comment on Till Bruckner's post The accidental NGO and USAID transparency test. We have reproduced the Mercy Corps response here in full: Till Bruckner’s posts on NGO accountability raise interesting questions – and it is unfortunate that the discussion has devolved into insinuations about NGO motives rather than an open discussion of what constitutes meaningful accountability in aid work.  While Mercy Corps appreciates being referred to as among the “most transparent NGOs,” we would argue that the willingness to release project budget documents to a third party is not a good indicator of an NGO’s overall level of accountability to its beneficiaries and stakeholders.  By applying such a narrow focus, Bruckner unfortunately turns what could be a rich and textured discussion of aid methodology into a less germane question of responsiveness to a self-appointed watchdog.

The rest of the story on Georgia First off, a bit more background on the Transparency International research project that Bruckner referenced in the original post.  In May 2009, Bruckner contacted Mercy Corps through our main website, informing us in vague terms that a report on NGO transparency would be forthcoming.  He requested a contact email where he could send the report for our review.  Our Georgia Country Director sent Bruckner an email three days later requesting that any inquiries be directed to him; Bruckner never followed up.  We heard nothing more until the following month, when we and the other agencies received an email from a different TI official, informing us that their project had developed a ranking of NGO transparency “based on the content provided on their websites”.  TI noted that it would “be extremely happy to improve the score of your organization and re-write the relevant sections of the report if you email us the complete proposals with budgets for all your ongoing projects as public documents (we will hide individual salary information before re-posting on our website).”  However, the agencies were given just three working days to review the report and turn any information over to TI-Georgia.

Mercy Corps and the other agencies contacted had serious concerns about TI’s approach.  We felt that the web-centric methodology of the study was flawed; that the format of the resulting report misrepresented how TI had engaged with our agencies (in fact, TI had not engaged us at all at that point); and felt, as agencies in the midst of a major ongoing post-conflict aid effort, that a three-day turnaround on the demand for proposal and budget data was inadequate.  The NGOs sent a letter of reply to TI expressing these concerns, and suggesting that TI take this opportunity to engage with us in a dialog on how to move forward with aid transparency in Georgia.  TI-Georgia eventually did come to meet with the NGOs and listen to our concerns.  As a result of that meeting TI agreed to redo the survey at a later date; however there was no further contact from TI after that meeting.  Bruckner’s posts make no reference to the NGOs’ substantive concerns, nor to the subsequent dialog with TI over these issues.  Here are the original emails, as well as the NGOs’ letter of response to TI, so that readers can draw their own conclusions about whether the issue has been presented fairly.

That process aside, there are important questions at stake here of how, and to whom, aid agencies should be accountable:

Does budget-sharing constitute accountability? Raw budget data, and the willingness to share it, is a poor proxy for real, comprehensive aid accountability – especially when divorced from the context of the project itself.  NGOs have different cost structures and different methodologies, and budget documents reveal little about which are most effective.  Certain types of projects – such as technical assistance or gender-based violence prevention – tend by their nature to be heavy on labor costs and light on capital items, while food distribution or micro-lending tend to be lighter on labor costs and heavier on capital requirements.  Posting the raw budgets of a handful of agencies reveals little about whether those agencies are effective, whether they are accountable to their beneficiaries, or whether their cost structures are reasonable. Furthermore, there can be legitimate reasons for withholding budget data.  To take one example, even Transparency International’s own handbook on Preventing Corruption in Humanitarian Operations acknowledges that “the highly volatile environments in which aid is often delivered means it’s important to recognise that public information about the value of programme resources and their transport may sometimes jeopardise staff and beneficiary security, particularly in conflict contexts.”

How does accountability make aid more effective? The ultimate purpose of accountability and transparency is to provide more effective aid.  As the SPHERE manual’s guidance note on Communication and Transparency states: “the sharing of information and knowledge among all those involved is fundamental to achieving a better understanding of the problem and to providing coordinated assistance.”  This is best achieved when the beneficiary population is involved in the design, implementation, and monitoring of that activity.  This integration of stakeholders into core parts of the project management process ensures that divergence between community needs and NGO deliverables is kept to a minimum.

What constitutes accountability to our constituents? NGO accountability focuses on our beneficiaries, our donors, and the governments of the countries where we work.  We orient our accountability systems to provide appropriate information, in appropriate formats, to these constituencies.  To prevent fraud and ensure good financial stewardship we institute rigorous financial and administrative policies (for an example, check out Mercy Corps’ Office-in-a-Box) and subject ourselves to extensive compliance requirements and a barrage of external audits.  Given the extensive burden that these requirements impose on already-stretched staff, it is often true that NGOs are less responsive to the demands of third-party watchdogs.  At a program level, NGO accountability means providing community stakeholders with means of evaluating a project’s goals, the appropriateness of its implementation strategy, and ultimately its effectiveness in meeting those goals.  Publicizing budget data may be a part of the approach depending on the context – but it is not a substitute for a project methodology that puts a core focus on community priorities and community participation.

--

Related posts:

The accidental NGO and USAID transparency test Till Bruckner Responds to Critics on Meaningful Transparency NGO Response: CNFA Reaffirms Commitment to Transparency World Vision responds on transparency USAID and NGO transparency: When in doubt, hide the data