Despite the considerable time, money, and effort that aid agencies, international organizations, and NGOs expend producing analysis and advice to inform or influence policymakers in developing countries, there is a remarkable lack of understanding about which of these instruments are most and least effective at spurring and sustaining reforms – and why.
In an attempt to answer these questions, AidData gathered firsthand experiences and insights from 6,750 policymakers and practitioners in 126 low-and middle-income countries. Through their analysis, they identify overall trends, attributes of influential assessments, and both intended and unintended effects of such assessments. These are summarized in their Marketplace of Ideas for Policy Change report.
Here’s an excerpt from the conclusion:
“The Minister’s adviser would probably tell her that, of the wide variety of assessments at her disposal, the policy analysis and advice contained in the assessments of large, global international organizations have proven to be particularly valuable to other reformers. He might also warn the Minister not to rely on the analysis and advice provided by assessment suppliers who lack an in-country presence. These suppliers may not be able to provide the contextual insights needed for the effective design of specific reform features. Additionally, after explaining that external performance assessments are generally associated with more successful reform efforts when they source their data from the governments they assess, the adviser would probably suggest that the Minister pay particularly close attention to assessments that draw upon the data that the government is already producing.
The Minister and her adviser would eventually confront the issue of whether to use country-specific performance assessments or cross-country benchmarking assessments. The adviser would likely explain that each type of assessment has its own advantages and disadvantages. Country-specific assessments contain in-depth diagnostic and advisory content, which may resonate with local stakeholders because they are more attuned to local needs and realities. These assessments tend to be more popular with policymakers across the developing world; however, they can also leave host government officials with a false sense of confidence in the analysis and advice they provide, thus bypassing critical internal processes of introspection, deliberation, and iterative problem solving.
Cross-country benchmarking assessments, on the other hand, are less attractive at first blush because they lack nuance and context specificity. However, when decision-makers confront complex issues of corruption, informality, and institutional dysfunction, they can provide greater policy flexibility and maneuverability than country-specific assessments, enabling the government to experiment with different reform strategies and iteratively adapt in pursuit of better, de facto outcomes.
Noting the Minister’s interest in Greek mythology, the adviser might remind her of the many perils Odysseus faced on his journey home from Troy and counsel the Minister to avoid steering her country into the “capability trap” that has ensnared other well-intentioned, would-be reformers. He would encourage her to focus on simple, solvable problems before undertaking more complex reforms. To underscore his point, he might even slip a copy of a short article by Lant Pritchett, Michael Woolcock, and Matt Andrews into the Minister’s read file, which describes the capability trap as a dynamic that enables [policymakers] to document instances of apparent reform and thus assure a continued flow of development resources to their country or sector, despite the fact that the reforms themselves may be generating few actual improvements in performance.”
Read the full report for more.