How to get a Masters degree for a replication

The best sign of acknowledgement of research is, for a young researcher, a graduate degree. The Norwegian University of Science and Technology shows that it understands the value of replication by awarding an MA to researcher Beate Øien, who replicated a paper in quantitative human rights for her thesis.

There might be more out there – but this is the first MA thesis I came across in my research that is dedicated to replicating a paper. Beate Øien from the Norwegian University of Science and Technology, Department of Sociology and Political Science, wrote her thesis “Revisiting Foreign Direct Investment and Collective Labor Rights: Replicating “the positive case” of economic globalization” based on a paper by Layna Mosley and Saika Uno called “Racing to the Bottom or Climbing to the Top? Economic Globalization and Collective Labor Rights” (Comparative Political Studies 2007 40: 923-948).

Mosley/Uno’s (2007) paper examines the impact of economic globalization on workers’ rights in developing countries through a statistical analysis of 90 developing nations from 1986 to 2002. Under supervision by political scientist Jonathon W. Moses (who discusses replication with me in a separate interview) Beate Øien set out to “investigate the robustness of their initial finding” by doing “strict replication”. She finds that Mosley/Uno’s (2007) conclusion is confirmed in the replication. However, “the external validity of their findings is (…) dependent upon which years are included in the sample.”

Øien’s MA thesis is well structured. First, she makes very clear why the approach of replication is taken. She then discusses theory and hypothesis, research design, and results of the replication.

Why replication?

Øien refers to Lamal (1991), Popper (2002 [1963]), King (1995) and Hendrick (1991). “Replications are necessary as objectivity and self-correction distinguish the scientific method from other approaches to knowledge (Lamal 1991). The demarcation principle regarding scientific knowledge is falsification according to Karl Popper.”

According to King (1995: 445), the most productive method of building on existing research is to replicate an existing finding; to follow the same path as a previous researcher and improve on methodology or data in some way. Replications provide information about the validity and reliability of the original study. If the results are in agreement, replications provide for external validity. If the results are in disagreement, however, doubt is cast on the internal validity of the original study. The results of a replication provide more information on the disconfirmation or confirmation of a theory (Lamal 1991). A norm against replication would lead to a fragmented and dispersed literature with a lack of continuity within the discipline (Hendrick 1991: 42). (Øien 2011: 3-4)

Goals of replication

Øien’s three goals of replication are:

1. “Validation or confirmation of conclusions rules out the possibility that the results are due to chance.”
2. “[R]eplications are designed to investigate the extent to which the conclusion of the original study is still valid when moderate changes in the covariables and conditions are introduced.
3. “[T]he most obvious goal is simply to gather more data to try and achieve a more precise conclusion (Bayarri and Mayoral 2002: 207).”

How is replication an ‘original’ contribution?

Some general advice on how to replicate and at the same time to contribute something original (=publishable) are given by Gary King (2006. Publication, Publication. PS: Political Science and Politics 39: 119–125.) [pdf; updates here].

• find a recent article in a peer-reviewed journal; by a well-known author, highly cited
• after replicating the article, find useful, additional, or even contradictory information not discussed in the article
• make one improvement and show how that changes our knowledge about the original paper’s research question (multiple imputation of missing data, selection bias, omitted variable bias, the model specification, differential item functioning, the functional form, adding control variables or better measures, extending the time series and conducting out-of-sample tests, applying a better statistical model)

Øien goes beyond the original paper by checking the robustness of the original model during different time periods and through a sensitivity analysis of variables, and runs several additional estimation techniques (OLS with lagged dependent variable and Newey-West standard error).

The findings of Øien’s replication

Øien is able to replicate most of the findings of Mosley/Uno (2007). She also refines the results further by adding other estimations and building year-subsets. While Mosley/Uno conclude that FDI increases labor rights, Øien finds that:

• FDI stock needs to reach a certain level of penetration to have a positive (possibly indirect) effect on worker rights
• When taking years into consideration, the short term or direct effects of FDI flows seem rather negatively related to labor rights (this can also be seen when replacing FDI accumulated stock with FDI short-term flow)
• “[E]ven though FDI can be considered the positive case of economic globalization, this impact is very small and not present before countries have been receiving large levels of FDI. Workers in developing countries, have to wait a long time before the positive indirect effects of FDI provide them with the possibility to engage more in bargaining with their employers.” (Øien 2011: 47-48)

Øien’s master thesis is an excellent example of how replication can lead to a masters degree. She pretty much followed all textbook rules of replication and original additions. If anyone out there knows of more such examples that I could collect here, let me know!

Links

• My interview with the professor who supervised the replication thesis.

MA thesis based on replication:
Øien, Beate, 2011, “Revisiting Foreign Direct Investment and Collective Labor Rights: Replicating “the positive case” of economic globalization”, MA Thesis, Norwegian University of Science and Technology, Department of Sociology and Political Science [Info] [pdf]

Replicated paper:
Layna Mosley and Saika Uno, “Racing to the Bottom or Climbing to the Top? Economic Globalization and Collective Labor Rights” Comparative Political Studies 2007 40: 923-948. [pdf] [replication data]

How to replicate and publish:
King, Gary 2006. Publication, Publication. PS: Political Science and Politics 39: 119–125. [pdf; updates here]

Some of Øien’s Literature:
Lamal,  P.A.  (1991)  “On  the  Importance  of  Replication”.  In  James  W.  Neuliep  (ed),   Replication Research in the Social Sciences. California: Sage Publications, Inc., pp. 31-35.

King,  Gary  (1995)  ”Replication,  Replication”.  PS: Political Science & Politics 28 (3): 444- 452.

Popper, Karl (2002 [1963]) Conjectures and Refutations. The Growth of Scientific Knowledge. London and New York: Routledge Classics

Hendrick,  Clyde  (1991)  “Replications,  Strict  Replications,  and  Conceptual  Replications:  Are   They  Important?”.  In  James  W.  Neulip  (ed),  Replication Research in Social Sciences. California: Sage Publications, Inc., Pp. 41-49.