Dissidents and Philanthropists

VALUABLE UPDATE! 12:22 PM 8/2/2012 GiveWell gives its side of the conversation (see end of this post) Had a conversation with Holden Karnofsky and Stephanie Wykstra of the philanthrophy firm GiveWell, along with an anonymous philanthropy adviser.   I enjoyed the spirited give and take.  At the end, had the feeling "well, nice chat, but I failed to be convincing." Stephanie took notes of my side of the conversation; here's the full transcript on our site. Wish I had taken notes on their side! Below are some extracts from my side:

... the “what works in aid” debate is phrasing the question wrong. You really want to know what works for whom, which will then lead to the question at the heart of economics and politics: who gets to decide what happens? This isn’t answered by randomized controlled trials (RCTs) that show that an intervention improves some quantitative measure of well-being. Markets and democracy are better feedback mechanisms than RCTs, and they provide resolution on “who gets to decide?” Seeing what people buy and asking them what they want gives better indicators of what works for them than quantitative indicators...

As an example: recently the World Bank funded a project in Uganda. The project ended up burning down farmers’ homes and crops and driving the farmers off the land. A lot of quantitative indicators like GDP would have shown up as improved as a result of the project, but there were many people whose rights were grossly violated in the process

But researchers don’t want their job to be more difficult than it is. If you ask for not only a RCT but also a guarantee that it’s not concealing unacceptable harm, you’re making it harder, and RCTs are already expensive and hard to begin with. It’s inconvenient for the researchers to acknowledge these problems.

Development happens when people have the opportunity to choose what they want, choose whether or not to give consent for an intervention that affects them, protest if they don’t like what’s being done to them and have a mechanism to exit if they don’t like what’s being done.

Have a system in place to ensure that you’re actually making people better off rather than harming them. Others would be better than I am on how to do this in practice, but just to start the discussion... it could mean offering {beneficiaries} a menu of options that they can choose from, and learning from their responses. More broadly, promoting rights of poor people might have indirect positive consequences that are a lot larger than the benefits of individual interventions.

...a lot of things that people think will benefit poor people (such as improved cookstoves to reduce indoor smoke, deworming drugs, bed nets and water purification tablets) {are things} that poor people are unwilling to buy for even a few pennies. The philanthropy community’s answer to this is “we have to give them away for free because otherwise the take-up rates will drop.” The philosophy behind this is that poor people are irrational. That could be the right answer, but I think that we should do more research on the topic. Another explanation is that the people do know what they’re doing and that they rationally do not want what aid givers are offering. This is a message that people in the aid world are not getting

Funding is biased toward a technocratic approach. Aid agencies do not want to deal with additional complexities like asking the people who they work with for consent or giving them choice. They already have hard jobs. They don’t want to hear about research that makes their job harder.

Dissidents.... say things that people don’t want to hear. Angus Deaton, Lant Pritchett, Ross Levine, and Andrei Shleifer are examples. Dissidents are a positive feature of a system that makes it more robust. A consensus model is prone to groupthink. Even if we dissidents were wrong, it would still be important that people like us challenge the mainstream consensus to make them rethink what they’re doing. Cass Sunstein wrote a book about this (Why Societies Need Dissent.) There are probably many more dissidents that we haven’t heard of. There are a lot of dissident aid workers who can’t speak publically without losing their jobs, and so keep quiet or write anonymous blogs.

UPDATE 12:27PM 8/2/2012: GiveWell has posted its side of the conversation on its blog. Here are some extracts:

• We don’t believe in a “first, do no harm” rule for aid. ... we believe that it isn’t practical to eliminate all risks of doing harm...
• people simply undervalue things like insecticide-treated nets. Brett Keller observes that irrationality about one’s health is common in the developed world. In the developing world, there are substantial additional obstacles to properly valuing medical interventions such as lack of the education and access necessary to even review the evidence.
• We believe that empowering locals to choose their own aid is much harder in practice than it may sound– and that the best way to achieve the underlying goal may well be to deliver proven health interventions. We’ve argued this point previously.
• Bottom line: Prof. Easterly ... see{s} himself as a “dissident”; his role is to challenge the way things are done without recommending a particular course of action. We see ourselves as advisors to donors, helping them to give as well as possible today. So while we share many of Prof. Easterly’s concerns – and would be highly open to new approaches to addressing these concerns – we’re also in the mindset of moving forward based on the best evidence and arguments available at the moment.

Harry Potter and the Soccket of Fire

Joseph Campbell in his classic The Hero with a Thousand Faces described a "monomyth" common to many classic tales, which has since been applied to classics of our time like Lord of the Rings, Star Wars, and Harry Potter:

A hero ventures forth from the world of common day into a region of supernatural wonder: fabulous forces are there encountered and a decisive victory is won: the hero comes back from this mysterious adventure with the power to bestow boons on his fellow man.

According to my extensive research (1 Wikipedia article), these boons are things the hero brings back that "may be used to improve the world."

These hero stories are also very common in development and philanthropy, the latest example being the Soccket, a boon in the form of a soccer ball that generates electricity for poor children, discovered by some very admirable innovators.

Now I'm not going to do what you expect and get all crotchety at this point and say this is all useless nonsense. The great thing about the Hero Stories is that they sometimes really do come true, and in fact our market system is great for generating real hero stories.  Our most recent real-life hero story might be entitled "Steve Jobs and the Tablet of iPad".

The difference of course is that Steve Jobs only became a hero after the iPad was accepted by consumers. The problem in development philanthropy is that we get excited about the hero story before we even know whether the "boon" works for the intended consumers. (As Seth Gitter points out, there is no information on the Soccket web site on whether it is good bang for the buck or really any evidence on how poor kids reacted to the Soccket)

So my constructive advice for the Soccket creators is please give us a little more evidence on how well the Soccket works for those poor consumers and a bit less of rich people testifying how excited they are about this story.

And the lesson for the day is that the real-life hero narratives in development are more likely to come from private entrepreneurs than from philanthropy.

UPDATE 2:45pm EDT RESPONSE FROM SOCCKET CO-INVENTOR (I EXTRACTED FROM LONGER EMAIL)

Hi Professor,

As co-founder of Uncharted Play and a co-inventor of the SOCCKET, I want to thank you for your interest in, and perhaps more importantly, your critique of our movement. Constructive comments like yours help us to confirm that we are on the right track to creating the maximum positive social impact in communities around the world.

… I am no stranger to the need for objective monitoring and evaluation of development efforts, and I have made sure to incorporate rigorous data collection into the DNA of Uncharted Play as an organization. Further, we design all of our products for, and in collaboration with, our end users in developing contexts. For example, children's feedback from Mexico, South Africa, El Salvador, Nigeria, and Brazil has been critical as we continue to iterate our technical designs for the SOCCKET.

…kids have found the product to be truly magical. The response has been universally positive, and variations on the same scene unfold each time we first present the ball. First, it's pure joy – and that is before the kids even know there is anything different about the ball.  When we actually say that the ball is special, that it can harness energy and power a lamp or a phone, there is always a collective yell of excitement.  Then, when we plug in a lamp to demonstrate,the kids’ eyes just pop out of their heads, and you can see the wheels beginning to turn.  There’s a moment of silent amazement, and then, right away, kids start brainstorming their own ideas. …I certainly think that type of creative inspiration qualifies as a "boon" for our users even if there is no MDG that adequately captures it or tried-and-true metric for recording it.

In this vein, the point you made in your post about the need for testimonials from users is well taken; we will be adding user comments from our field studies to our website….

{added responses to Seth Gitter}:

Uncharted Play is a social enterprise, not an NGO, so we are held accountable by our investors and kept afloat by revenue.  As such, it would be all too easy to ignore impact evaluation altogether so that we can dedicate more of our financial and human resources to sales, marketing, etc.. However, as I mentioned above, we are truly focused on collaborating with communities to implement meaningful, catalytic programs, and - rather than resting on our laurels or focusing strictly on profit - we are taking aggressive action to track outcomes, both quantitatively and qualitatively.

We are a new company (just over 1 year old), and we do not have the resources to support a randomized controlled trial, especially since funding for this is often restricted to 501(c)3 organizations (which, as I mentioned above, we are not). Thus, we can only speak to improvements from pre- to post-intervention as tracked by our partners on the ground.

By replacing pollutant kerosene lamps and providing extra hours of light after the sun goes down, clean energy solutions like the SOCCKET can immediately and dramatically improve the lives of billions around the globe. By tracking suppressed demand for kerosene, we are gathering very persuasive evidence about the SOCCKET’s direct impact on households.

Further, Uncharted Play is focused on FUN and letting kids be kids. If it were just about producing as much energy as efficiently as possible, we would be distributing a hand-crank. The big difference is that, unlike a hand-crank, a soccer ball is fun. We are working to distribute a product that emphasizes the joy in life, not another object that simply reminds users of what they lack. The whole point of SOCCKET is that it's supposed to be fun. We aim to remain firmly in the territory of the whimsical without degenerating into mere frivolity.

Our studies are not publicly available at this time.  If you have any suggestions about where we could get funding to help us launch an open online platform, please let my team know - we'd love to do it!

Thanks again for your interest...

Best – Julia

Julia C. Silverman, Co-founder & Chief Social Officer, Uncharted Play, Inc.

ARTICLE: Inception Statistics Revisited

Professor Easterly and Laura Freschi wrote a letter to the Lancet Medical Journal criticizing the methodology in a 193-country study on stillbirths, in which researchers could obtain actual data on stillbirths from only 33 countries and twice modeled the stillbirth estimates for the other countries:

Many will argue that modelled numbers (or in this case, twicemodelled numbers) are better than no numbers at all. To this we ask, better for what, and for whom? We question the wisdom of creating policy based on fi gures with such a tenuous basis in reality. Could the irresponsible lowering of standards on data possibly refl ect an advocacy agenda rather than a scientific agenda, or is it just a coincidence that Save the Children is featured among the authors of the new data?

The correspondence includes a response from the authors of the original study, who argue that "improving data quality and quantity is a high priority but in the meantime modelling is indispensable."

They previously discussed the issue in an April 2011 post on the Aid Watch blog.

Read More: >National, regional, and worldwide estimates of stillbirth rates in 2009 with trends since 1995: a systematic analysis (The Lancet, April 14, 2011) >Inception Statistics (Aid Watch blog, April 18, 2011) >Correspondence published in the Lancet, with the authors' reply (September 3, 2011)