Cookies help us deliver our services. By using our services, you agree to our use of cookies. More information

Stodden 2020 Proc Natl Acad Sci U S A

From Bioblast
Publications in the MiPMap
Stodden Victoria, Seiler Jennifer, Ma Zhaokun (2020) An empirical analysis of journal policy effectiveness for computational reproducibility. Proc Natl Acad Sci U S A 115:2584-9.

» PNAS Open Access

Stodden Victoria, Seiler Jennifer, Ma Zhaokun (2020) Proc Natl Acad Sci U S A

Abstract: A key component of scientific communication is sufficient information for other researchers in the field to reproduce published findings. For computational and data-enabled research, this has often been interpreted to mean making available the raw data from which results were generated, the computer code that generated the findings, and any additional information needed such as workflows and input parameters. Many journals are revising author guidelines to include data and code availability. This work evaluates the effectiveness of journal policy that requires the data and code necessary for reproducibility be made available postpublication by the authors upon request. We assess the effectiveness of such a policy by (i) requesting data and code from authors and (ii) attempting replication of the published findings. We chose a random sample of 204 scientific papers published in the journal Science after the implementation of their policy in February 2011. We found that we were able to obtain artifacts from 44 % of our sample and were able to reproduce the findings for 26 %. We find this policy—author remission of data and code postpublication upon request—an improvement over no policy, but currently insufficient for reproducibility.

Bioblast editor: Gnaiger E

For analysis of computational data, noncomputational articles (e.g., theoretical results, experimental results) were eliminated.


Labels: MiParea: Instruments;methods 






Publication efficiency, X-mass Carol