More and more scholars integrate replication into their teaching. In a previous post, I wrote about challenges during the Cambridge Replication Workshop, where students replicated a published paper in eight weeks. For anyone planning to design a similar course, here are thoughts on a good target audience, and why the course should use R instead of STATA or SPSS.
The Cambridge Replication Workshop 2013 was a pilot project at the Social Sciences Research Methods Centre. During eight weeks, PhD students replicated a paper in their field. They had weekly 2-hour sessions with a short lecture that guides them through the replication steps, and a practical session with TAs who helped them with problem solving (see syllabus, handouts and assignments here). I collected feedback from my teaching assistants (Aiora Zabala, Chris Bentz and Vaishali Mahalingam to improve the workshop in the future – and to give ideas to other teachers using replication in the classroom.
Whom to target
When setting up the Cambridge course, we designed it for graduate students in the social sciences. I was not sure if MPhil students, doing a one or two-year degree, would have the time for such a project, or if we should let only doctoral students attend. We left it open in the course description, and in the end only PhD students applied. One of the TAs said that they felt that “PhDs will be motivated by the possibility to get something for their thesis out of the course, and they may have more time to dedicate to it than taught master students.” If the TAs had to make a choice, they could see such a course working for final year undergraduates, MPhil’s/MA’s, and 1st year PhD students though if it can be adjusted to their program.
Which software is best for replication?
I had decided to teach the course in R, because the Social Sciences Methods Centre at Cambridge, which covers stats teaching, is gradually switching most of its courses to R. I also decided that only R, and not several software programs, should be used for replication, so that students (and TAs) could check each others’ work easily. This means that a prerequisite for the replication course was knowledge of R. So could the project have worked with a different software? This is what the TAs say after having worked with the students intensively:
- “I think it does matter that we use R instead of SPSS. In R we have to understand the underlying code and the statistics better instead of just clicking buttons to give us graphs and p-values. And this helps to be more clear about the analyses applied.”
- “I think it is very instrumental to teach it using the same software for all, so that they can exchange their replications and learn from each other – and R may be one of the best suited for that, due the myriad of methods to be used in replication.”
- “I would definitely prefer that this workshop is done in R due to the nature of the software (open-source, flexible etc).”
- “Cross-software workshops are difficult since students exchange scripts/code at some point to check each others’ work.”
Issues with R knowledge
Having said that, we did encounter problems with R knowledge (this could happen with other software as well): It is hard to assess if students really do have basic knowledge of R, and to what extent. The occasional student didn’t have the expected basics, but by the time this was discovered, the course was already mid-way. We don’t yet know how to deal with this. For the academic year 2013/14, we will ask students to provide a confirmation note by their supervisor/advisor that they have the required knowledge for the course.
Read also the previous post about pitfalls and challenges during the replication projects.