Where do the numbers published in scientific articles come from and how often are they correct? Recently we discovered that it can be a lot more difficult to answer this question than one might hope. Our attempt to reproduce values reported in 35 articles published in the journal Cognition revealed analysis pipelines peppered with errors. I outline some elements of a reproducible workflow that may help to mitigate these problems in future research.
Many efforts are underway to promote data sharing in psychology, however it is currently unclear if the in-principle benefits of data availability are being realized in practice. In a recent study, we found that a mandatory open data policy introduced at the journal Cognition led to a substantial increase in available data, but a considerable portion of this data was not reusable. For data to be reusable, it needs to be clearly structured and well-documented. Open data alone will not be enough to achieve the benefits envisioned by proponents of data sharing.
An extensive and ongoing attrition of the modern scholarly record is impeding a number of important research activities that support verification, discovery, and evidence synthesis. Recently, we launched the Data Ark initiative – an attempt to retrieve, preserve, and liberate important scientific data. However, most of our data requests were not successful. How can we ensure the longevity and accessibility of important research artifacts?