UPDATE: Just received notice of drastic punishment for this post: invited to join HDR 2011 Advisory Panel I’ve complained previously about how design of the UN Millennium Development Goals make sub-Saharan Africa look worse than it really is. Now I realize that UNDP’s new Human Development Report (HDR) does the same thing. Not alleging any conspiracy here, it seems unintentional, but is then not caught because … well we all know Africa is supposed to look terrible.[1]

My HDR education comes from Martin Ravallion, who has a great new paper on the new methodology of the Human Development Index (HDI). (Martin does not mention the Africa angle, but provides the necessary insights described below).

Of course, UNDP has an impregnable position: while their results get huge publicity, the methodology behind the results is interesting to approximately 3 people. As an avid promoter of hopeless causes, here goes…

The biggest change in method was that the new HDI is a geometric average rather than a normal (additive) average. Geometric average means you multiply the separate indices (each ranging between 0 and 1) for income, life expectancy, and education together and then take the cube root (I know your pulse starts to race here…)

Now, students, please notice the following: if one of these indices is zero, then the new HDI will be zero, regardless of how great the other indices are. The same mostly applies if one of the indices is close to zero. The new HDI has a “you’re only as strong as your weakest link” property, and in practice the weakest link turns out to be very low income (and guess which region has very low income).

So, as Martin noted, the new HDI relative to the old HDI penalizes countries with very low income compared to decent numbers on life expectancy and education. One reason I think this is unintentional is that these are exactly the cases that the HDR used to celebrate! The biggest losers here are Zimbabwe, Liberia, DR Congo, Burundi, Madagascar, Malawi, Niger, and Togo.

Martin makes the “decent life expectancy doesn’t help you if you have low income” point in a different way: the new HDI has vastly different numbers for the value of life between poor and rich countries. Martin had previously made this criticism of the old HDI in a paper published in 1997, which Aid Watch covered in a previous post. The HDR addressed this criticism by making the problem much worse. Previously we were all whining about differences in the value of life of 70 times between rich and poor – now it’s a differential of 17,000 to one. Sorry, Zimbabweans, UNDP thinks your lives are worth 50 cents.

But wait, Africa has another GREAT chance to perform well -- the HDR also gives mucho publicity to the “top movers” in HDI over 1970-2010, ranked in order of percentage increase. My old MDG paper mentioned above said Africa would look better on percentage increases in health or education indicators. Indeed, Ethiopia, Burkina Faso, Niger, Mali, and Burundia all had more than a 300% increase in educational enrollments (using the UNDP’s own data) from 1970 to 2010.

So naturally, among the champion improvers are … Oman and Nepal … and no sub-Saharan African countries in the top 10. What happened?

In yet another twist, the HDR ranked the top improvers measured as deviations from the average growth in HDI of others at similar initial HDI in 1970. Since almost all of the bottom ranks of the HDI are sub-Saharan African (exacerbated by the above “weakest link” methodology), Africa will only do well if it does better on average than – Africa.

If we forget the deviations thing, and just rank by growth in the HDI from 1970 to 2010, then sub-Saharan Africa would get 6 out of the top 10 improvers.

If you have read this far, you get a medal. So what’s the lesson of all this mumbo-jumbo about methodology? Maybe you could make a case for the new methodology, but at the very least it’s clear that obscure choices of method make a big difference in who you celebrate – and who you make look bad. And way too often, Africa winds up in the latter category.

Postscript: we want to thank UNDP for generously making all their data and methodology available to us even though they knew we were critical, because they also generously gave reactions to a preliminary draft based on an earlier dataset we downloaded from the HDR website. They did not change our minds and the new dataset confirmed our earlier results. But we give them great credit for constructive engagement. The paper that describes their methodology is here.

[1] This post uses the words “Africa” and “sub-Saharan Africa” interchangeably, following common development-speak. North Africa is in a very different situation from that described here.

From Aid to Equality

The First Law of Development Stats: Whatever our Bizarre Methodology, We make Africa look Worse