Reinhart-Rogoff economic study got it wrong, and so did the political reaction

REUTERS/Albert Gea
Protesters holding a banner reading "Let's stop this anti-social budget/Together it's possible" as they march against government austerity measures in Barcelona on Sunday.

I used to make MPR’s Gary Eichten crazy by answering questions on economics with: “Well, on the one hand…but on the other hand…”

Politicians hate this, too. They want unambiguous cures, remedies and solutions — and they want them now.  

For a time, it seemed some politicians had exactly what they wanted when it came to dealing with the national debt. But the past two weeks have turned into the revenge of ambiguity. Let me explain.

In 2010, Harvard economists Carmen Reinhart and Kenneth Rogoff concluded that  “median growth rates for countries with public debt over roughly 90 percent of GDP are about one percent lower than otherwise; average (mean) growth rates are several percent lower.” They set out their case in five pages of tables, graphs, prose, footnotes and citations to other work.

Recently, Thomas Herndon, a University of Massachusetts graduate student, tried to replicate the Reinhart-Rogoff study, but he couldn’t do so. Once he got the original data from the authors, he found they made a spreadsheet error which, when corrected, changed their conclusion. Now, it seems that there is no statistically significant difference between countries that have debt-to-GDP ratios below and above 90 percent.

How academics work

This case is an example of how academics do their work. We publish studies; others try to slice them, dice them, replicate them, extend them, and do all sorts of mean, nasty, ugly things to them. If our conclusions withstand this scrutiny, we have confidence in them and build on them; if not, we throw them out and start over again.

I constantly urge my students do this in their own research projects. Go find a study, try to replicate and extend it to another time or place, and see what happens.  You might learn something (and so could the rest of us.)

Unfortunately, soon after the Reinhart-Rogoff study was released, it became the topic of overwrought sound bites used by (mostly conservative) pundits and politicians: Debt-to-GDP ratio of 90 percent is a tipping point! Countries go past that point at their own peril! A grand bargain of tax increases and spending cuts had to be enacted right away to avoid a drastic slowdown in U.S. growth!

Thomas Herndon
U. of Massachusetts at AmherstThomas Herndon

Herndon’s work then morphed into a Replication poke in the eye: Ha, Reinhart and Rogoff were wrong! We don’t need to worry about debt! Stop harping on the debt and start stimulating the economy! Even Stephen Colbert got into the act.  

Why should you care about any of this? Because it carries an important lesson about the relationship among policies, politicians and academics.

Academics always live with a deep sense of what we know and what we don’t know about our subjects. We’re always waiting for someone to show that we’re wrong, that we made a mistake, that we missed a source.

Nobel laureate Robert Solow put it well when he urged professors to embrace this tension, saying: “Don’t omit qualifications. Never claim more than you actually believe or can justify.”  

Professors are reluctant to make thunderous pronouncements about economic policy; they have to face the music of other professors scrutinizing their work.

Policy entrepreneurs

This is why politicians prefer to get their support from a different kind of professional intellectual: policy entrepreneurs. Policy entrepreneurs are like professors in that they have advanced degrees, but instead of working in academia and subjecting their work to the scrutiny of their peers, they work for organizations where they can promote their views without being challenged.  

Paul Krugman, who coined the term in 1994, noted that a policy entrepreneur suffers “from none of the professor’s inhibitions. They offer unambiguous diagnoses, even where the professors are uncertain; they offer easy answers, even where the professors doubt that any easy answer can be found.”

Politicians prefer policy entrepreneurs for a simple reason: Politicians are looking for intellectual support for positions they’ve already taken. So they’ll go out and find a think tank or policy shop that has the kind of policy entrepreneur they like. They’re not interested in starting with a position and having it challenged.

Reinhart and Rogoff’s study started as academic work full of qualifications and subtlety. Policy entrepreneurs got a hold of it and turned it into a single number that held the gospel truth.

So here’s the lesson that that you, dear reader, should keep in mind when reading about economic policy: Beware of policy entrepreneurs bearing clear answers.  They’re probably wrong.  

Learn to listen for the uncertainty, ambiguity and discomfort that academic studies deliver, and repeat after me: “On the one hand… on the other hand…”

Comments (7)

  1. Submitted by chuck holtman on 05/01/2013 - 01:18 pm.

    Your point is sound in the abstract

    But the facts of the Reinhart-Rogoff matter may not support it.

    Once consequence of the continuing concentration of private wealth is the corruption apace of those institutions that move policy and discourse, public and private. The fact that an economist is ensconced in an academic institution (even, or especially, an elite one) doesn’t mean that economist is not a “policy entrepreneur.” There is evidence that both Carmen Reinhart and Kenneth Rogoff have close ties to Pete Peterson, the billionaire austerity pimp, and the Peterson Institute, as well as the American Enterprise Institute. I haven’t done the conclusive investigative analysis, but your logical proposition – that because these economists are at Harvard, they must only be interested in the search for truth – doesn’t hold.

    With due respect, you also weaken your piece by forcing yet another tiresome false symmetry. No thoughtful opponent of austerity is saying we don’t need to worry about the debt. We’re saying that stimulus is a higher priority in the immediate term, and debt can be managed quite readily over the longer term (at least could be, if certain simple and fair remedies were not taboo). And we’re saying, perhaps, that in the real world austerity has been shown so conclusively to be injurious to the general public welfare, that those who continue to advance it can only be motivated by the intent to advance the selfish interests of the fortunate few on the backs of the less well off.

  2. Submitted by Lora Jones on 05/01/2013 - 01:21 pm.

    Contrary to this article, it appears the Reinhart Rogoff study

    Started OUT as a policy entrepreneurism. Both have strong ties to and positions with the Peterson Institute — the right wing “think” tank formed by the man who’s primary mission for the past twenty years has been to privatize social security. What better way to move that agenda forward than to try to convince the more reality-based, less academia-averse democrats that the safety net must be shredded than to cook the books with a Harvard Study! The sad part is that too many of them, including apparently, our President, took the bait.

    I just hope that now that this “study” has been proven to be both deceptive and untrue, and the tangible and damaging effects of austerity take hold in Europe, President Obama will openly repudiate chained CPI.

  3. Submitted by jody rooney on 05/01/2013 - 01:54 pm.

    A Thank you to Professor Johnston

    I think the take away is peer review and an understanding of the subtlety of research and the limits as to what one piece of research can tell you.

    The key is that unlike private research like the American Enterprise Institute’s academic work has to be replicable and as this points out the raw data has to be available to other professionals. Any researcher worth there salt will ask for the raw data or data source. When it isn’t immediately forth coming it raises suspicions. In my mind someone is likely cooking the books.

    While the one handed economist is often sought by various interest groups very few will not have an “on the other hand”. Unfortunately that leads to a distrust of all economists. While economics is a social science it is also has a quantitative branch which is recognized as part of the STEM (Science, Technology, Engineering and Math)group.

    As far as policy goes the results should be evaluated over time and reassessed periodically if we want it to be effective in changing markets and political wills.

    • Submitted by Vj Laxmanan on 05/01/2013 - 08:12 pm.

      Economics and STEM

      @ Jody Rooney While economics is a social science it is also has a quantitative branch which is recognized as part of the STEM (Science, Technology, Engineering and Math )group. …

      Yes, very true. Unfortunately, the overuse of statistical analysis, indeed the exclusive reliance on statistics as opposed to “mathematical model building”, has turned much of the quantitative MATH part into a nightmare, IMHO.

      The “hard” sciences use simple mathematical models to describe complex phenomena. The models are then tested against empirical evidence. And they are refined. That is how Planck developed his famous radiation law. The derivation of the Planck law, a modified version of what is know as Wien’s law (which was an empirical law fitted to the data) was in response to newer experimental observations that showed deviations from Wien’s law.

      Unfortunately, I do not see this type of model building approach in economic or financial data analyses. Therein lies a big part of the problem. I have published several articles on this topic which could be accessed via a Google search using my name here.

  4. Submitted by Vj Laxmanan on 05/01/2013 - 07:13 pm.

    The Debt-GDP Relationship

    I have just uploaded the following document describing my research on the Debt-GDP relations, prompted mainly by the discovery of the Reinhart-Rogoff errors and the ensuing debates.A brief summary is given below.

    A remarkably simple and linear relation between the GDP x and the Debt y is revealed if we analyze the empirical observations on various modern economies of the 21st century. A brief survey is provided here which supports the law y = hx +c where h, the slope of the graph, is the rate of increase of the Debt y as the GDP x increases. The nonzero c means that the Debt/GDP ratio y/x = h + (c/x) will tend to its maximum value h, the slope of the GDP-Debt graph, as the GDP x increases. The nonzero intercept c, which has been called the “work function” (see earlier articles on the Debt-GDP problem by the present author), is analogous to the work function conceived by Einstein to explain photoelectricity, or the work function of a professional baseball player. The simple linear law that has been deduced here seems to have escaped the attention of all professional economists to date.

  5. Submitted by Larry Copes on 05/02/2013 - 09:55 am.

    Somebody had to do it

    Wasn’t it Harry Truman who said “Give me a one-handed economist!” I think it’s wise to be skeptical of anyone who claims to have an absolute answer to anything. Even 8 + 5 does not equal 13 if you’re on a 12-hour clock.

  6. Submitted by Paul Udstrand on 05/02/2013 - 10:27 am.

    Failed peer review

    Actually, the fact that a basic spread sheet error evaded the peer review tells us something about the rigor of this discipline, as does the fact that it took so long to detect. The difference between the policy entrepreneurs and academics is this field may more illusory than some people would like us to believe.

    If anything this incident can just be added to the growing mountain of evidence that the field of economics has drifted into downward spiral of ideologically driven mediocrity and irrelevance.

    VJ is correct, economist seem to spend much of their time developing and celebrating formulas that have little of any relationship to real economies. These formulas may be the product of prodigious arithmetic, but they are untestable and devoid of any supporting evidence. The result is an economic discourse that struggles for coherence.

Leave a Reply