Do social science studies published in Nature and Science replicate?
Nature Human Behaviour
August 28, 2018
The results of efforts to replicate 21 experimental social science studies published in Nature and Science between 2010 and 2015 are reported in Nature Human Behaviour this week. The primary replication method used in the study found that one out of the four Nature papers and seven of the seventeen Science papers evaluated did not replicate.
Understanding the extent to which scientific findings can be trusted is of fundamental importance. Recent attempts to replicate published research from across various fields and journals found that a substantial number of studies do not replicate. To ensure replication rates are not being underestimated, however, higher-powered replication studies with larger sample sizes have been called for.
Colin Camerer, Brian Nosek and colleagues conducted high-powered replications of the 21 experimental social science studies - using sample sizes around five times larger than the original sample sizes. They find that 62% of the replications show an effect in the same direction as the original studies. This effect varies between 57% and 67%, depending on how replicability is determined. The authors suggest that the original studies are likely to have contained both false positives and inflated effect sizes. Based on a survey of 400 social scientists, the authors also report a strong correlation between the research community’s expectations of each study’s replicability and the reality.
In an associated collection of eight Correspondences, non-replicating study authors discuss why their results did not replicate, providing their views into potential sources of irreproducibility. In an accompanying News & Views, Malcolm Macleod argues that replication studies are necessary for establishing a solid foundation for the social sciences, and can generate important new research into the parameters that could lead to more robust scientific conclusions.
doi:10.1038/s41562-018-0411-7 | Original articledoi:10.1038/s41562-018-0403-7 | Original article
doi:10.1038/s41562-018-0424-2 | Original article
doi: 10.1038/s41562-018-0404-6 | Original article
doi:10.1038/s41562-018-0409-1 | Original article
doi:10.1038/s41562-018-0426-0 | Original article
doi:10.1038/s41562-018-0408-2 | Original article
doi: 10.1038/s41562-018-0399-z | Original article
doi:10.1038/s41562-018-0405-5
Research highlights
-
May 27
Archaeology: First Pompeiian human genome sequencedScientific Reports
-
May 27
Ecology: Local environment and population demographics, not genetics, influence bottlenose dolphin ‘names’Scientific Reports
-
May 27
COVID-19: Small study suggests maternal psychological distress during the pandemic may be associated with altered fetal brain developmentCommunications Medicine
-
May 27
Sustainability: Converting vehicle waste plastics into grapheneCommunications Engineering
-
May 27
Robotics: Growing human tissue grafts using humanoid robotsCommunications Engineering
-
May 26
Archaeology: Uncovering ancient Amazonian architectureNature