While the replication crisis can be a commonly mentioned issue, the background and causes of it may not be clear to all. Originating in the early 2010s, the term replication crisis refers to the fact that findings of many types of scientific research are often difficult or even impossible to replicate due to one or multiple issues raising concerns about the credibility of the findings. Along with this, the accuracy of existing scientific findings as well the matter of following correct procedure may be questioned.
What is the replication/reproducibility crisis?
For more than a decade, scientists have been trying to emphasize the importance of replicability and thecredibility of scientific findings. While these conversations have been taking place, the vulnerability of scientific research to the replicability crisis has not improved significantly. With a lack of work that is able to be replicated, the industry may run into problems with reliable resources becoming harder to find. While there are multiple factors affecting the replicability of scientific research, some argue that many of these can be resolved or improved to ensure a standardized process and enhanced ability to replicate a piece of existing research.
Main factors contributing to the crisis
While there are differing opinions about the causes of the replication crisis, it is commonly attributed to these factors:
Failure to comply with proper practices
Research and scientific practices should be scrupulous and have high attention to detail. In many cases, peer-reviewed research ends up being cited in other pieces of work and the methodology is replicated or adapted to build onto the research so it’s important for this work to be accurate. While some works can be easy to replicate, in many cases, research findings are shown to be unreplicable. It has been suggested that many published studies are also likely to report inflated effects, due to small, underpowered studies with insufficient sample sizes and the 5% statistical significance that is generally accepted in research. This may cause a significant amount of false-positive results, further contributing to replicability issues.
Rush to publish
Even though publishing a research paper can be a long and difficult process, if there are no flaws and it complies with the proper practices, in a perfect scenario the research and methodology should not be difficult to replicate. However, the use of statistical tricks or other methods to rush the process could result in the inability to replicate them.
Lack of transparency
The diverse nature and lack of detailed records of scientific research can result in a reduction of transparency - including data sets, and also methodologies such as instruments used, the conditions of the experiment and the exact set up. This could be an issue as replicating a piece of research accurately would require the scientist to recreate it using the same resources and procedures. If all or at least most journals developed accessibility policies in place for disclosing data, this might improve the replicability process and ensure simpler sharing of scientific research.
Issues with the peer-review system
Over time, it has become increasingly difficult for journals to find competent reviewers for scientific research. With little to no incentives for reviewers and the increasing number of research papers being published, it is becoming more common for research to only be reviewed by one or two individuals. Further contributing to this, researchers that had their work published and peer-reviewed by this small sample, may assume that their work is correct and not check over the findings once completed, therefore causing non-replicable work to be distributed.
Public interest
Some topics may seem more interesting to the general public than others which can cause a trade-off situation in the academic review system. Interesting or appealing topics are likely to get more attention and be shared widely via multiple channels, but this popularity does not necessarily guarantee the best results from the research itself. The trade-off between quality and popularity might be causing the review system to apply more lax reproducibility standards to research that is considered to be more ‘likable’ or engaging.
Final notes
While relying on thoroughly tested measures and widely accessible scientific practices as well as ensuring larger sample sizes can potentially help improve the quality of research and help fix the replication crisis, this matter should be taken seriously by both individuals as well as research institutions and scientific journals. Working transparently and coming together is something each researcher could do to help improve the current situation and avoid the reduction of reliability in scientific research.
