Academic slugfest & replication chains: getting replicated hurts

Screen Shot 2013-03-08 at 6.28.24 PMGetting replicated gives you citations, but it hurts. Often, authors respond with a paper to defend their earlier work. They often claim the replication was: fundamentally flawed, contains statistical and reporting errors, is of trivial nature, or less realistic and of limited utility. Such replication chains are not just entertaining academic slugfest, but they are useful because they provide discussions about data and methods.

Replication chain on international trade and democracies

Screen Shot 2013-03-13 at 4.25.12 PM
Step 1 – Original article: Free to Trade: Democracies, Autocracies, and International Trade, Edward D. Mansfield, Helen V. Milner and B. Peter Rosendorff, The American Political Science Review, Vol. 94, No. 2, Jun., 2000.

Abstract: Relatively little research has focused on whether countries’ political institutions affect their international trade relations. We address this issue by analyzing the relationship between regime type and trade policy. In a formal model of commercial policy, we establish that the ratification responsibility of the legislature in democratic states leads pairs of democracies to set trade barriers at a lower level than mixed country-pairs (composed of an autocracy and a democracy). We test this hypothesis by analyzing the effects of regime type on trade during the period from 1960 to 1990. The results of this analysis accord with our argument: Democratic pairs have had much more open trade relations than mixed pairs.

 

Screen Shot 2013-03-13 at 4.35.58 PMStep 2 – Replication: Political Regimes and International Trade: The Democratic Difference Revisited, by Xinyuan Dai, The American Political Science Review, Vol. 96, No. 1, Mar., 2002.

“I revisit these important questions by highlighting some problematic aspect of the analysis by Mansfield, Milner, and Rosendorff (2000). Contrary to their central conclusion, I find that whether the aggregate trade barriers are lower for a democratic pair than those for a mixed pair depends on the preferences of the decision makers involved. Thus, although domestic political institutions are important, they alone are insufficient to predict a higher level of cooperation among democracies.”

 

Screen Shot 2013-03-13 at 4.36.09 PMStep 3 – Answer from original author: Replication, Realism, and Robustness: Analyzing Political Regimes and International Trade, by Edward D. Mansfield, Helen V. Milner and B. Peter Rosendorff, The American Political Science Review, Vol. 96, No. 1, Mar., 2002.

“Xinyuan Dai (2002) incorrectly asserts that our conclusion depends on the preferences of the decision makers who formulate trade policy. We show that Dai fails to accurately replicate our model, and hence erroneously claims that the new equilibria she deduces are consistent with it. In addition, we demonstrate that in altering one of our assumptions, Dai offers a model that is less realistic as well as inconsistent with the substantive literature on international bargaining. Finally, we question the robustness of her approach. Due to these problems of replication, realism, and robustness, we conclude that Dai’s model is of limited utility.”

 

More replication chains

… are here.

Advertisements
Tagged , , , , ,

2 thoughts on “Academic slugfest & replication chains: getting replicated hurts

  1. […] original author assessed a replication of his work stating that it is “less realistic”, “inconsistent with […]

    Like

  2. […] and then the original author sometimes writes a third article defending his/her findings (see here). Recently, a failed replication made it into the news, and suddenly there’s a […]

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: