Many academic journals hesitate to publish replications

While most political scientists would agree that replications and cross-checks advance knowledge, many academic journals hesitate to publish replications. John Ishiyama, University of North Texas, discusses how replication studies can get published to create more incentives to do replications.

Ishiyama_JohnIn the recent symposium on reproducibility in PS: Political Science & Politics (Vol 47, Issue 1), John Ishiyama, University of North Texas, discusses possible ways in which replication studies can be published.

So far, with many journals emphasizing original research articles, there are “little obvious rewards” to conduct replications in political science.

Ishiyama defends decisions of journals, such as the American Political Science Review (APSR), to focus on original work and reject replications.

[A]s editors of the APSR we are also reluctant to publish such studies in the Review, because this would open up a “cheap” way for authors to have their work published in the APSR, and every Tom, Dick, and Harriet (pardon the expression) could potentially seek to replicate some study, just to get published in the Review.

 

How to publish replication studies

APSR
At the same time, Ishiyama stresses that some form of venue for the publication of replication studies is “necessary” to “raise the degree of scientific rigor in the field.”

His suggestions:

  1. Occasional “Forums” in all journals, including APSR, which potentially allow for the incorporation of replication studies
  2.  

  3. A special section in a journal which is dedicated to the production of replication reports
  4.  

  5. Publication of replication studies in journals that are “not a general research journal“, such as PS, and Perspectives on Politics, International Studies Perspectives, and the International Studies Review
  6.  

  7. An electronic “blog like” venue for the publication of replication studies, something like the Monkey Cage
  8.  

  9. Publishing replication studies by major journals in an online supplement directly linked to the original articles; journals should then highlight those articles that have been replicated multiple times
  10.  

  11. Creating a new journal for replications

 

The American Political Science Association must act

apsaIshiyama stresses that the American Political Science Association should take up this issue (and it already has), to create incentives for scholars to engage in replication studies.

[T]he APSA should consider potential alternative venues for the publication of replication studies (or perhaps “forums” or debates) of pieces that appear in APSA journals. Now it is not exactly clear how this should be done, if it could be done online, if it requires an editorial team, what the relationship would be with the existing APSA journals, and how would this be related to Cambridge University Press, but if we are tomove forward as a discipline, we must have some venue available for the publication (or at least making public) such studies.

 

Read the article by Ishiyama

Screen Shot 2014-01-31 at 8.53.50 AMJohn Ishiyama (2014). Replication, Research Transparency, and Journal Publications: Individualism, Community Models, and the Future of Replication Studies. PS: Political Science & Politics, 47, pp 78-83. doi:10.1017/S1049096513001765.

Abstract: Recently, the importance of research transparency via replication studies has been greatly discussed in most of the social sciences, political science included. Indeed, as Gherghina and Katsanidou (2013) and Freese (2007) note, to some extent, the discussion has been prompted by the tremendous changes in publishing in the past decade or so. With the enormous expansion in data availability and instant publication made possible by the Internet, there now are many opportunities to verify the findings presented in the discipline’s major journals. “Replication, replication” has not only become the mantra for political science, but for economics, psychology, and quantitative sociology as well. These developments opened a debate on how to best “guard the high standards or research practice and allow for the maximum use of current knowledge for the further development of science” (Gherghina and Katsanidou 2013, 1; for similar sentiments see King 1995).

 

Other articles from the PS Symposium

Arthur Lupia and Colin Elman write about New ethics guidelines for data Access & research transparency.

Thomas M. Carsey discusses his experiences of assigning replications to students – a thought-provoking and excellent read!

Tagged , , , , ,

2 thoughts on “Many academic journals hesitate to publish replications

  1. […] and robustness checks, are an important step in the scientific process. Yet most journals still hesitate to publish replications, with rare and notable […]

    Like

  2. […] 1) published a symposium to discuss the state of reproducibility in the field. The topics included assigning replications to students; journal publication of failed replications; and new ethics guidelines on data sharing […]

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 1,794 other followers

%d bloggers like this: