Registered Reports#
What are Registered Reports?#
Registered Reports are an article type divided into two published parts- a study protocol and a research article. The study protocol is reviewed prior to the data being collected and the research being carried out.
Fig. 98 Image from Centre for Open Science- https://www.cos.io/initiatives/registered-reports#
For the first stage of registered reports, researchers detail their proposed methods, hypotheses and analyses in a published study protocol. This is then peer reviewed, allowing authors to refine their methodology from reviewer feedback if needed. The protocol is then provisionally accepted, meaning it will be published by the journal if the authors conduct the experiment in accordance with their approved protocol. The authors then carry out their registered methodology for their study.
Once the study is complete, authors write and publish a research article, which includes results and interpretations. This is also peer reviewed.
Registered reports are a form of pre-registration, which you can read more about here.
Around 300 journals publish the registered report format. You can find a full list here.
Why write a Registered Report?#
Registered reports began in psychology, but this format has now spread and can be used in any field.
The idea behind the registered report format is to encourage best practices by putting the importance of the methodology and research question first. By pre-registering your planned method and analyses, this eliminates questionable research practices such as selective reporting of results, including low statistical power, p-hacking and HARKing. It also allows you to get useful feedback and improve on your study design.
A huge benefit of publishing a registered report is avoiding publication bias. It can be difficult to publish negative or ‘less novel’ results in journals. The registered report format means that when you have your protocol approved, your research article is provisionally accepted for publication and will be published regardless of whether you get negative results or not.
This frees up researchers to be rewarded for simply doing good and transparent science without having to sell an impactful story about their research.
Registered reports are also a useful format for researchers who want to carry out replication studies and other novel, resource-intensive projects that may otherwise be too risky to attempt where successful publication is contingent on the results. More info can be found in this Royal Society blog on registered reports.
How do you write a Registered Report?#
You will write your registered report in two parts.
For a stage 1 study protocol, you should write about background of your research question, motivations, hypotheses and your planned methods. This can include your experimental procedures, analysis pipeline, and statistical analysis. You may also want to include any pilot data or experiments that you’ve undertaken so far.
After you’ve carried out your investigations, you should prepare your stage 2 research article. This should include the introduction and methods from your original stage 1 article with the addition of your results and discussion. You should include the outcome of all the analyses you outlined in your stage 1 protocol. If you undertook any further analyses that you didn’t outline in your protocol then you should report these too with clear justification and methodological details.
How do you review a Registered Report?#
You can find general guidance on how to peer review a paper in our peer review chapter here.
For registered reports specifically, they are reviewed twice. This happens once at stage 1 (study protocol) and again at stage 2 (research article). Journals try and ensure the reviewers are the same for both stages, but if this is not possible then they will ensure the new reviewer is aware of the stage 1 protocol when they review.
It may be useful to focus on the following for review stage 1 and stage 2 papers- adapted from F1000Research’s guidance:
Stage 1 Study Protocol:
Is the rationale for, and objectives of, the study clearly described?
Is the study design appropriate for the research question (including statistical power analysis, where appropriate)?
Have the authors pre-specified sufficient outcome-neutral tests for ensuring that the results obtained can test the stated hypotheses, including positive controls and quality checks?
Stage 2 Research Article:
Are the data able to test the authors’ proposed hypotheses by satisfying the approved outcome-neutral conditions (such as quality checks, positive controls)?
Are the introduction, rationale and stated hypotheses the same as the approved Stage 1 submission?
Did the authors adhere precisely to the registered experimental procedures? If not, has an explanation been provided regarding any change?
Are any unregistered post hoc analyses added by the authors justified, methodologically sound and informative?
Are sufficient details of the methods and analysis (such as statistical) provided to allow replication by others?
Personal Case Study on Registered Reports#
Folco Panizza gives his thoughts on publishing a Registered Report. Interviewed by Marta Mangiarulo.
Why did you choose to publish a Registered Report?
I was inspired to write a registered report by a talk given by Chris Chambers at my university. I realised that the projects for my doctoral thesis would have benefited from a clear design upfront. I was the first one to try this at my institution, so I also took it as a career challenge to improve my scientific practices and perhaps convince others that the undertaking was not so difficult.
How did you get started?
I did some searching in the literature and tried to find some examples of previous registered reports. There weren’t many examples, but I found it useful to look into the presentation notes that Prof. Chambers generously made available for advice.
What impact do you think your Registered Report made/is making to your research/project/community?
After my initial submission, I was invited to share my experience as part of a series of talks at my university. I think my talks helped make registered reports and standard pre-registration less exotic than they initially seemed, although I am not aware of any specific research that adopted either procedure.
What tools/software did you use most in your Registered Report work?
My original registered report was based on the recommendations from the journal I originally submitted it to (Royal Society Open Science). After that, I mostly relied upon the Open Science Framework’s standard pre-registration format: it has improved quite a bit since I first used it, and I think it nudges researchers into providing more details than other pre-registration forms such as the ones from aspredicted.org
Do you have any top tips for other people that might be interested in preregistering their research?
Steal from others’ preregistrations. It really helps to jump-start stalled registrations. And – general advice – don’t underestimate the importance of literature research to improve your design, especially for methods-heavy registrations. For example, I happened to find an improved operationalisation of a dependent variable right after I had pre-registered my study. I do not consider it a big problem to amend the pre-registration, but it is neater if you don’t have to!
Do you have any tips on things to avoid?
Do not write your pre-registration right before running your experiment. Writing things down is useful because it forces you to find the blind spots in your reasoning. I would suggest that you start writing as soon as you have the idea.
Is there anything else that you want to add to the interview that has not been covered in the questions above?
Preregistrations are not meant for all research questions out there. Let’s not forget about the importance of exploratory questions and how they can expand our pre-registered hypotheses. Keeping the focus of a pre-registration restricted to our main hypotheses does not mean that we cannot then explore all the unexpected ramifications of the study results.