A couple of days ago, the World Bank shocked the world by announcing it would discontinue its Doing Business report, which measures the business climate in each country. The statement about the matter said:
After data irregularities on Doing Business 2018 and 2020 were reported internally in June 2020, World Bank management paused the next Doing Business report and initiated a series of reviews and audits of the report and its methodology. In addition, because the internal reports raised ethical matters, including the conduct of former Board officials as well as current and/or former Bank staff, management reported the allegations to the Bank’s appropriate internal accountability mechanisms.
After reviewing all the information available to date on Doing Business, including the findings of past reviews, audits, and the report the Bank released today on behalf of the Board of Executive Directors, World Bank Group management has taken the decision to discontinue the Doing Business report.
“Data irregularities” (plural!)? “Conduct of Board officials, staff, and management”? Seems like something really juicy and sordid happened.
How to do Doing Business
The Doing Business Report is an annual report published by the World Bank since 2004. In it, World Bank researchers, plus independent investigators, collect data on the ease of opening, operating, and shutting down a business in 190 countries - mostly by asking local officials a series of questions. The topics broached are:
Starting a business - how difficult, costly, and/or time consuming it is to start a business in a country.
Dealing with construction permits - for the construction industry, how easy it is to obtain professional advise, labor, or licenses; plus submitting documents or being subject to inspections.
Getting electricity - how easy it is to secure a permanent supply of electricity
Registering property - how easy the process for acquiring, registering, and/or selling property as a company is, plus quality of resolution of land disputes.
Getting credit - what are the legal rights of borrowers and lenders, plus what kind of information lenders require/have/use about borrowers.
Protecting minority investors - how well outside investors (minority investors) are protected against mismanagement of companies by managers.
Paying taxes - how high tax rates are, how many taxes a company pays, and how it easy and/or time-consuming it is to file taxes and/or receive refunds.
Trading across borders - how time-consuming and costly it is to sell goods to either a regional trading partner or an overseas destination, in terms of both tax burdens and paperwork required.
Enforcing contracts - how easy it is to resolve a commercial dispute, and what is the quality, efficient, and cost-effectiveness of the judicial process.
Resolving insolvency - how well processes of business insolvency are resolved.
Each country gets a score and a ranking from the World Bank, and then all countries are ranked based on their score on each category and their total score. The highest ranked country is New Zealand, for example, and the top spots are all populated by high income nations or newly modernizing countries, like Thailand (21), Mauritius (13), Malaysia (12), the Baltics (mid teens all of them), and Georgia (7).
The World Bank generally also suggests reforms and improvements for all countries in these areas. Some recommendations are pretty common sense, such as making forming a business easier, streamlining bureaucratic procedures, improving infrastructure and public utilities, or making legal oversight better. Others are… less so, particularly the tax section (more on this later).
Doing Business does badly
There are two very obvious issues, right out of the gate. The first is that a ton of those things are very hard to measure, and basically value judgments. The second is that they might not be equally important, and might pose different burdens to people in different countries: an American entrepreneur might be concerned with construction permits, a Ghanian one with securing electricity, a Russian one with the legal system, and an Argentinian one with exorbitant tax rates.
But the Doing Business report has been under fire for the past three or so years. In 2018, the Center for Global Develpment published a piece on a very peculiar fact: Chile’s ranking seemed to, without any change in policies, rise when the conservative Sebastián Piñera won elections, and drop when the center-left Michelle Bachelet took office. And just comparing the indicators and the number of countries, Chile did decline a bit during Bachelet’s terms and improve a bit during Piñera’s - but not as much as the Bank reported.
In fact, Paul Romer (at the time the Chief Economist of the World Bank) started off this whole drama by telling a reporter at the Wall Street Journal that he suspected the World Bank was manipulating the report. Romer later retracted his comments and resigned, although not many other countries showed large disparities - basically just Iran, Paraguay, and Burundi.
Then there’s India. Speaking at Davos in (also) 2018, Narendra Modi gave India three items of praise: world’s largest democracy, fastest growing major economy, and leader in economic reform. But, if you actually look at how India would have performed without changes in the number of countries included and without changes in methodology, then the “Modi bounce” disappears altogether.
The World Bank did defend itself from both sets of allegations. About the Romer-Chile Affair, it said that the changes weren’t politically motivated, the indicator is just not particularly reliable and is being misinterpreted by everyone… even if the World Bank presents it in a way that makes that precise misreading more likely. And about India, at least the changes served the purpose of making the report better, and India did improve a little with a constant sample and methodology. But the “we couldn’t be saying that people need to read our ranking as a ranking even if we promote it that way” defense is… less than credible.
Anyhow, this whole thing probably points to three big problems. The first is that the Doing Business report doesn’t seem particularly good at predicting the business environment of a country: not only the numbers of countries that don’t change anything move around too much, but actually doing the things the report wants you to do doesn’t seem to actually predict whether or not your ranking improves or declines. Plus, the actual utility of the report, like many other “single number for complicated topic” rankings is… dubious.
There’s a pretty big problem on the whole premise. The Doing Business report takes a very small-L libertarian position, only looking at the cost government regulations and actions have but not at whether or not they provide any benefits. Granted, making starting a business extremely expensive and time consuming is not good, but the whole “lower taxes are always good” isn’t exactly an uncontroversial thing to just proclaim - particularly just aggregating various tax rates like the World Bank does.
The dirty doings of Doing Business
Now let’s get to the juicy part. An external review of the World Bank’s conduct was carried out, and it found… not good things: data manipulation, bullying, political pressures on staff members, corruption, etc.
In 2018 (not a particularly good year, it seems) the World Bank was trying to negotiate with China over money. Coincidentally, World Bank authorities started asking around how China was “coming up” on the rankings, and when it appeared it would fall short, a group of senior staffers met and discussed how to boost China’s ranking.
Current IMF managing director, and then World Bank CEO Kristalina Georgieva apparently was elbows deep in it - she became directly involved in the “improving China’s score” project, instructing staff members on how to improve the country’s numbers and chastising China-related personnel on the matter. At one point, a senior figure sent the following, highly entertaining, “low ranking caporegime explains himself to the Don in a mob movie” email to Georgieva:
… I suggest we go back to Plan A, which is to present the actual Doing Business numbers, and have an extensive explanation about how China has made substantial progress (…) but that, through no fault of their own, the countries in the ranking neighbourhood did exceptionally well this year.
Simeon Djankov, Kristalina Georgieva’s right hand man (and, per Paul Romer, her “consigliere and fixer”) wasn’t pleased, and proposed another solution to that same manager in private. Georgieva told that person to “wrap it up” and thanked him for “doing his bit for multilateralism”. And to continue the mobster antics, Georgieva showed up to someone’s house in person, after hours, to collect a physical copy of the report, saying that it was a “very unusual year” and thanking the Doing Business team for “resolving the problem” with China’s ranking.
In 2020, a similar story happened with Saudi Arabia - where Djankov intervened directly to muddle the data so he could ensure Saudi Arabia topped the “Top Improvers” list instead of Jordan. This change at least to some degree motivated by Saudi Arabia paying for World Bank consulting services to improve its Doing Business performance. Lastly, and even more paradoxically, Djankov ordered the team to freeze Azerbaijan’s score for no particular reason other than his personal dislike of the country more than any evidence against the Azeri government’s conduct.
Plus, apparently the 2021 report was mostly completed before this scandal broke out, and current World Bank president David Malpass (appointed by Donald Trump) tried to pull a reverse Georgieva and change the methdology to decrease China’s score, so the report was an all around circus.
Conclusion
Many sources, most notably The Economist, have called on Georgieva to resign as Managing Director of the IMF. Paul Romer in particular seems vindicated, though, tweeting:
The best lesson here is that single-number indicators for complex economic relationships are probably not good. There is plenty of evidence that countries pay a lot of attention to how they’re ranked, taking mesures that don’t improve much on the actual thing being measured but does measure their rankings. This also often comes down to outright data manipulation: in 2016, for example, Argentina was expelled from the PISA education rankings due to manipulating the submitted data to look better.
In the end, boiling down a really complex item to a simple number composed of easy-to-game indicators may not be a very good way to handle international economics. Discontinuing Doing Business is probably the right call.