Internet Resources

Understanding the complexities of retractions

Recommended resources

Amy Riegelman is social sciences librarian, email: aspringe@umn.edu, and Caitlin Bakker is biomedical/research services liaison at the University of Minnesota, cjbakker@umn.edu

Reasons for retracted publications range from honest errors made by authors or publishers to research misconduct (e.g., falsified data, fraudulent peer review). A retraction represents a status change of a publication in the scholarly literature. Other examples of status changes include correction or erratum. A retraction could be initiated by many parties, including authors, institutions, or journal editors. The U.S. National Library of Medicine annually reports on the number of retracted publications indexed within PubMed. While the overall rate of retractions is still very small, retractions have increased considerably in the last decade from 97 retracted articles in 2006 to 664 in 2016.1

As librarians help users navigate research platforms and maintain awareness of publication status changes, it is important to understand both the publishing and discovery landscape. Guidelines exist to help publishers and platforms identify retractions, but a recent study found inconsistent representations of retractions across various platforms.2 Another consideration is when scholars export citations or full-text articles out of various discovery platforms to personal libraries (e.g., Mendeley, DropBox).

Philip Davis studied retracted articles residing in personal libraries and nonpublisher websites. Among the findings, Mendeley libraries contained many retracted articles, and Davis concluded that this decentralized access without automated status updates “may come with the cost of promoting incorrect, invalid, or untrustworthy science.”3

Beyond honest error or research misconduct that result in retracted publications, there is other evidence of scientists engaging in questionable research practices. A systematic review and meta-analysis by Danielle Fanelli studied rates of scientists admitting to research misconduct, as well as assessing research misconduct of colleagues.4 Fanelli found that 1.97% admitted to research misconduct in the form of fabricating, falsifying, or modifying data, while 33.7% of respondents admitted to other questionable research practices.5 Regarding practices of colleagues, 72% responded that colleagues engaged in questionable research practices.6

The resources included in this article were selected to provide context and help librarians and information specialists understand and identify retractions within scholarly communication. Readers may gain awareness of publication guidelines and best practices that apply to both publisher-hosted scholarly output, as well as downstream access to all electronic versions. Lastly the authors selected resources reflecting access and transparency trends that may be affecting retraction rates.

Understanding retractions

  • “Amending Published Articles: Time to Rethink Retractions and Corrections?” There is concern that the stigma of retraction prevents researchers from publicly acknowledging errors in previous publications. This preprint argues that destigmatizing post-publication correction would encourage greater transparency. The authors argue that rather than issuing retractions, journals should issue amendments that would be marked as “insubstantial” for minor errors, such as typographical errors, “substantial” for more significant changes that nevertheless do not change the overall meaning of the article, such as a correction to a figure or a change of authorship, or “wholesale/complete,” to denote research that “as a whole is considered unreliable in its current form.” Access: http://www.biorxiv.org/content/early/2017/03/24/118356.
  • “Keeping the Pool Clean: Prevention and Management of Misconduct Related Retractions Conference Proceedings.” A grant-funded retractions-themed conference occurred at Colorado State University in 2016, and the proceedings and details are open access and available online. The grant was awarded from the Office of Research Integrity (ORI) within the U.S. Department of Health and Human Services. One of the presenters represented the ORI and spoke about the role of ORI to “[p]rotect federal funds, correct the literature.” If interested in a general understanding of retractions as well as implications for the citation lifecycle, view Scientific Publishing’s Wild West presented by Adam Marcus, a cofounder of Retraction Watch. Access: https://dspace.library.colostate.edu/handle/10217/173563.
  • “Report of the Subcommittee on Replicability in Science Advisory Committee to the National Science Foundation Directorate for Social, Behavioral, and Economic Sciences.” Retractions and other status changes occur for many reasons, but one reason is the failure to replicate findings. This report from the Subcommittee on Replicability in Science Advisory Committee to the National Science Foundation Directorate for Social, Behavioral, and Economic Sciences provides helpful context as well as definitions for replicability, reproducibility, and generalizability. As summarized in this report, failure to replicate or reproduce could be due to a range of activities or omissions ranging from malicious fabrication to human or statistical tool-based errors. This report concludes with five suggestions to help prevent publication of studies that report illusory results. One of these reads: “Identify questionable research practices that cause illusory findings to make their way into the published literature.” Access: https://www.nsf.gov/sbe/AC_Materials/SBE_Robust_and_Reliable_Research_Report.pdf.
  • “Science Isn’t Broken: It’s Just a Hell of a Lot Harder Than We Give it Credit For.” This article by science writer Christie Aschwanden summarizes the issues regarding the “reproducibility crisis” and emphasizes that failure to replicate should not be seen as a threat. This article explains that science is self-correcting and that retractions shouldn’t be “viewed as a stain on the scientific enterprise.” The author layers on the role of media coverage of science in glossing over nuance and hyping studies that may contain illusory results. Access: https://fivethirtyeight.com/features/science-isnt-broken/.
  • “The Characteristics of Medical Retraction Notices.” While research on the role of librarians in educating patrons about retractions is limited in the modern environment, this 1992 article describes the activities librarians regularly performed in a print environment, including tagging pages of retraction articles and affixing a citation to retraction notices, displaying featured retractions, and communicating via library newsletters featuring articles “on the dangers of citing works that may have been retracted.” Access: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC225694/.

    Journal guidelines and best practices

  • Guidelines for Transparency and Openness Promotion (TOP) in Journal Policies and Practices. Sponsored by the Center for Open Science, the TOP Committee developed guidelines to address journal policies and procedures around transparency. These guidelines encourage sharing, replication, and clear, complete reporting to facilitate verification. The guidelines are comprised of eight standards, each of which is organized into three levels to provide sufficient flexibility for different disciplines and journals. Text from the TOP Guidelines includes language about requiring access to data, code, research materials, and other documentation in a trusted repository. Authors are responsible for these conditions, and “[f]ailure to do so may lead to an editorial expression of concern or retraction of the article.” Access: https://osf.io/ud578/.
  • Recommendations on Publishing and Editorial Issues from International Committee of Medical Journal Editors (ICMJE). ICMJE is a small, international group of editors of general medical journals, including the Journal of the American Medical Association, the New England Journal of Medicine, The Lancet, and PLOS Medicine. ICMJE produces recommendations that can be broadly adopted by nonmember journals. Recommendations are best practices intended “to help authors, editors, and others involved in peer review and biomedical publishing create and distribute accurate, clear, reproducible, unbiased medical journal articles.” These recommendations include guidance on when and how to issue corrections and retractions, including how to address issues of version control. Access: http://www.icmje.org/recommendations/browse/publishing-and-editorial-issues/.
  • Retraction Guidelines from the Committee on Publication Ethics (COPE). COPE is a group of more than 10,000 members worldwide, including editors of academic journals and others interested and involved in publication ethics. They provide guidance on publication ethics with a particular focus on how to address cases of research and publication misconduct. This includes retraction guidelines that outline when a retraction should be issued, as opposed to a correction or an expression of concern and what a notice of retraction should include. COPE also provides flowcharts on what to do if various ethical concerns are raised during peer review or following publication. Access: https://publicationethics.org/resources/guidelines.
  • COPE
  • “Retraction Policies of Top Scientific Journals Ranked by Impact Factor.” In this open access article, David B. Resnik, Elizabeth Wager, and Grace E. Kissling sought and analyzed retraction policies from journals. Seventy-four percent of 200 science journal editors responded. Sixty-five percent of these respondents had retraction policies. This study also coded for the source of the retraction policy (e.g., COPE), as well as procedures or linking retractions within electronic databases. Access: http://dx.doi.org/10.3163/1536-5050.103.3.006.

    Identifying retractions

  • CrossMark. CrossMark is a multi-publisher initiative from CrossRef that will display the status of a scholarly article if one selects the CrossMark branded button on certain publisher platforms. For participating publishers, the CrossMark button gives users the current status of a scholarly article. Upon selecting the CrossMark icon, a user could see confirmation that an article is indeed up-to-date or a status update (e.g., retraction, correction). Participation is only open to CrossRef members and requires that publishers deposit metadata and add code to their DOI. Publishers are still responsible for maintaining the content (submitting updates), and they must include the CrossMark button on all of the formats offered (e.g., HTML, PDF). Since this service is for publishers, this eliminates inclusion in other nonpublisher discovery platforms. Access: www.crossref.org/crossmark/.
  • Check for updates
  • Open Retractions. As a web interface and API, Open Retractions is intended to trace retraction notices for any article’s DOI. Unfortunately, as of November 2017, this tool failed on several attempts with known retracted articles, including the DOI for Wakefield (and others) Ileal-Lymphoid-Nodular Hyperplasia, Non-Specific Colitis, and Pervasive Developmental Disorder in Children. The Github README for this tool explains more about this tool, including that the data is being pulled by PubMed API using bionode-ncbi and CrossRef API using crossref-retractions. While this tool does not appear to be functioning as originally intended, it may lead to alternative tools or improved operations for retraction awareness and confirmation. Access: http://openretractions.com/.
  • Retraction Watch. Launched in August 2010, Retraction Watch is a blog interface venue for information seekers to stay current on self-correction and scientific misconduct. Retraction Watch also has a social media presence on Twitter, @RetractionWatch, where followers are able to monitor the latest retractions. The Center for Scientific Integrity functions as the parent organization for Retraction Watch, and their stated mission is to promote integrity and transparency within publishing. Access: www.retractionwatch.com.
  • Retraction Watch
  • The Office of Research Integrity (ORI) Case Summaries. The Office of Research Integrity within the U.S. Department of Health and Human Services provides case summaries for instances where administrative actions were imposed in response to research misconduct findings. The purpose of the case summaries are to provide information for each unique research misconduct allegation. ORI is responsible for overseeing and directing research integrity of the Public Health Service (PHS). Among other things, ORI reviews and monitors misconduct investigations when biomedical and behavioral research research was funded by PHS research training grants. The summaries on this page are time sensitive. Only current cases are represented, and expired cases have been removed. One 2017 case summary example points to an National Institutes of Health (NIH) grant awardee engaging in research misconduct, including falsifying and/or fabricating data in 11 figures in a PLOS One published article. As explained in the case summary, NIH recommended that the article be retracted. Access: https://ori.hhs.gov/case_summary.
  • The Retraction Watch Retraction Database (Beta). In late 2016, Retraction Watch revealed a database of retractions, The Retraction Watch Retraction Database (Beta). With this tool, users are able to search various fields (e.g., author, journal, DOI) to discover if something has been retracted. According to a December 2016 Retraction Watch blog post, the database is still in beta, and content continues to be added. As of November 2017, it was reported to contain more than 15,000 known retractions. The parent company for Retraction Watch is The Center For Scientific Integrity, a nonprofit. Access: http://retractiondatabase.org/.

    More eyeballs effect: Access and transparency

  • EQUATOR Network. The EQUATOR (Enhancing the QUality And Transparency Of health Research) Network has developed more than 370 reporting guidelines or tools that provide guidance to authors on the key components that should be included in a research paper to give a clear picture of a study’s methods and results. These guidelines are usually presented as checklists that include minimum criteria based on the study design. The EQUATOR Network’s reporting guidelines are included in ICMJE’s “Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals.” However, it should be noted that reporting guidelines are applicable in a wide variety of fields, including the social sciences. Access: http://www.equator-network.org.
  • PubMed Commons. PubMed Commons is a tool provided by NLM’s National Center for Biotechnology Information. PubMed Commons is another example of postpublication peer review, where the community of scholars can comment on articles, but the PMC Guidelines, state that comments should not include: “Allegations of misconduct on the part of authors, reviewers, editors and publishers.” Access: https://www.ncbi.nlm.nih.gov/pubmedcommons/.
  • PubPeer. PubPeer is an online journal club and a nonprofit foundation with a stated goal to “improve the quality of scientific research by enabling innovative approaches for community interaction.” Commenters could choose to identify themselves in comments or comment anonymously, and such comments could draw attention to errors (e.g., manipulated figures) and expose research misconduct. PubPeer also has a browser plugin available that allows for linking to PubPeer comments when users are on PubMed and journal websites. Access: www.pubpeer.com.
  • “Why Growing Retractions Are (Mostly) a Good Sign.” Since 1980, the proportion of journals retracting publications increased approximately 25 times faster than journals issuing corrections. In this PLOS Medicine article, Daniele Fanelli argues that this indicates that journals are better equipped to detect misconduct and more willing to retract publications than they may have previously been. However, cases of misconduct are not increasing. Fanelli concludes that “[t]he recent rise in retractions . . . is most plausibly the effect of growing scientific integrity, rather than growing scientific misconduct.” Access: http://dx.plos.org/10.1371/journal.pmed.1001563.


  1. U.S. National Library of Medicine, “Key MEDLINE Indicators Archive: FY2005-2009,” 2013, https://www.nlm.nih.gov/bsd/bsd_key_archive3.html; U.S. National Library of Medicine, “Key MEDLINE Indicators,” 2016, https://www.nlm.nih.gov/bsd/bsd_key.html.
  2. Caitlin Bakker and Amy Riegelman, “The Complexities of Retractions: Data Integrity, Discovery, and Tools,” Research Data Access and Preservation Summit, 2017, Seattle, Washington, https://dx.doi.org/10.17605/osf.io/7cxak.
  3. Philip M. Davis, “The Persistence of Error: A Study of Retracted Articles on the Internet and in Personal Libraries,” Journal of the Medical Library Association, doi:10.3163/1536-5050.
  4. Daniele Fanelli, “How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data,” PLoS ONE 4 (5): e5738. doi:10.1371/journal.pone.0005738.
  5. Ibid.
  6. Ibid.
Copyright Amy Riegelman, Caitlin Bakker

Article Views (Last 12 Months)

No data available

Contact ACRL for article usage statistics from 2010-April 2017.

Article Views (By Year/Month)

January: 26
February: 20
March: 24
April: 26
May: 12
June: 17
July: 20
August: 20
September: 11
January: 22
February: 32
March: 29
April: 28
May: 42
June: 22
July: 28
August: 27
September: 28
October: 32
November: 34
December: 23
January: 48
February: 33
March: 35
April: 31
May: 25
June: 20
July: 38
August: 23
September: 26
October: 29
November: 31
December: 16
January: 64
February: 61
March: 54
April: 58
May: 46
June: 32
July: 132
August: 27
September: 36
October: 39
November: 32
December: 36
January: 58
February: 64
March: 38
April: 33
May: 59
June: 62
July: 43
August: 76
September: 49
October: 43
November: 51
December: 41
January: 1758
February: 118
March: 166
April: 159
May: 129
June: 148
July: 98
August: 66
September: 58
October: 78
November: 74
December: 53