Amaral 2021 Nature

Amaral OB, Neves K (2021) Reproducibility: expect less of the scientific paper. Nature 597:329-31. https://doi.org/10.1038/d41586-021-02486-7

» PMID: 34526702 Open Access

Amaral OB, Neves K (2021) Nature

Abstract: Make science more reliable by placing the burden of replicability on the community, not on individual laboratories.

• Bioblast editor: Gnaiger E

Selected quotations

• The Reproducibility Project: Cancer Biology has so far managed to replicate the main findings in only 5 of 17 highly cited articles3, and a replication of 21 social-sciences articles in Science and Nature had a success rate of between 57 and 67% (ref. 4).

• We do think that the effort will pay off in terms of reproducibility. But if every paper in discovery science is to adopt this mindset, a typical high-profile article might easily take an entire decade of work, as well as a huge budget.

• There are typically three main expectations for a top-notch article in laboratory science: first, report original findings from exploratory research; second, confirm that they represent robust phenomena through further experiments using different methods; and, finally, suggest theoretical mechanisms to explain the results. However, these represent different aspects of the scientific process and do not have to be achieved all at once5.

• Studies have shown that neither statistical power6 nor quality of reporting of individual experiments7 improve as journal impact increases.

• Articles byindividual research groups should thus be regarded as preliminary by default.

• Basic exploratory science would be helped if editorial policies reduced requests for new experiments and refrained from demanding evidence of clinical potential.

• There needs to be training, funding and rewards for researchers to focus on managing collaborations, participating in large experiments and synthesizing data — especially because this involves sacrificing academic freedom to some extent.

• Requiring researchers to register how they will blind their study is more flexible than enforcing how they do it, but still serves to eliminate bias.

• Moving the burden of reproducibility from individual researchers to organized communities can ultimately raise the bar of what is considered scientific fact, and could also have a salutary effect on the public communication of science.

Cited by

Gnaiger E (2021) Beyond counting papers – a mission and vision for scientific publication. Bioenerg Commun 2021.5. https://doi:10.26124/BEC:2021-0005 - Living Communication

Labels:

BEC2021.5f, MitoFit2022QC