Citation searching and bibliometric measures: Resources for ranking and tracking

Robin Kear; Danielle Colbert-Lewis


A professor asks you, “Who is citing my articles? How many times have I been cited? What is my h-index score?”

A student asks you, “How do I know if this article is important?”

Another professor asks you, “Which journal should I publish in for my tenure?”

A different student asks you, “What are the best journals in the field of Anthropology?”

The avenues to evaluate citation tracking and journal ranking have greatly increased in the past six years. Bibliometrics, the applicon of mathematical and statistical analysis to books, journals, and other publications, 1 allows us to choose journal collections, assist with applications for research funding, evaluate journal status, and find significant contributors in a subject area. Most importantly, the expertise librarians have in this area can be used to help our faculty prove their scholarly contribution and achieve success in their tenure process.

Any aggregator of citations could create its own bibliometric measures if they are willing to invest the time and expense. Elsevier and Thomson Reuters are the major players in creating citation and journal measures. Each vendor primarily uses its own unique data, journals, publications, authority files, indexes, and subject categories. Both companies have given their data to research labs to create new metrics that are freely available online from the labs and are also included in the company’s products for subscribers. One metric, the h-index, is vendor neutral but there is no overarching tool that exists across vendors. The bibliometrics field is an exciting one to watch and continues to produce beautiful data visualization projects.

Citation tools and measures

  • Citation Map. Web of Knowledge created this tool that depicts backwards and forwards citation of an article using a map format. The backwards feature represents the citations in the current selected document, and the forward feature represents the documents that have cited the current selected document. This tool gives dynamic representation of the impact that a document has on a field, a topic area, or trend. Access: http://wokinfo.com.
  • Citation Reports. This tool by Web of Knowledge, gives a graphical presentation and snapshot of a set of articles. Upon conducting a search (by topic, author, or publication name) and selecting a group of documents, users will find a link to Web of Knowledge to Create Citation Reports. The report includes graphs for Published Items in Each Year and Citations in Each Year. The Results Found section of the report details the Sum of times cited, Sum of times cited without self-citations, Citing articles, Average citations per item, and the h-index. Citation Reports serves as a helpful tool as it helps users to understand various trends of a topic, how an author has published over time, and the most cited articles related to that search topic. Access: http://wokinfo.com.
  • Citation Tracker. This Scopus product gives detailed information on finding, checking, and tracking citations. Upon conducting a search and selecting a group of documents in Scopus, Citation Tracker, displayed under the View Citation Overview page, gives users an overview of how many times a selected document has been cited upon its initial selection, and the number of documents that cited the selected document since 1996 and the h-index. This citation overview information assists researchers by helping them evaluate the currency and impact of research overtime. Access: http://www.info.sciverse.com/scopus.
  • h-graph. Scopus’s h-graph visually depicts the impact of an author’s research or set of articles. By selecting h-graph, users have a choice of three different graphs to view or select: the h-index, articles published, and citations. Users will also see these graphs under the author’s name or by conducting a document search. At the end of the search, the user then selects an author’s hyperlinked name under the research section via a view h-graph link. Access: http://www.info.sciverse.com/scopus.
  • h-index. J. E. Hirsch created the hindex in his 2005 paper titled “An index to quantify an individual’s scientific research output.” In this foundational paper, Hirsch tries to answer the question, “[H]ow does one quantify the cumulative impact and relevance of an individual’s scientific research output?” In the abstract, Hirsch proposes that the index h, defined as the number of papers with citation number ≥h, serves as a useful index to characterize the scientific output of a researcher.The h-index measures the broad impact of a researcher’s work. When conducting research for the aforementioned article, the data focused primarily on physicists, however the data has proven applicability to other areas of sciences and more recently to humanities. Web of Knowledge and Scopus both include the h-index in searches for an author. In Web of Knowledge, a user finds the h-index under the Citation Reports section at the beginning of a search. In Scopus, the user will find the h-index presented by conducting a search and selecting an author’s hyperlinked name under the Research section. Access: Web of Knowledge at http://www.wokinfo.com and Scopus at http://www.info.sciverse.com/scopus.

Subscription databases for citation searching

  • Academic Search Premier. This database by EBSCOhost has a few tools that allow researchers to search for citation effectively. These tools include Find Similar Results, Cited References, and Citation Matcher. The Find Similar Results feature uses Smart- Text Searching to find other documents that share a relation to the original article. By conducting a Cited Reference search, the user receives a list of articles that cite the original article. Located under the More option feature in Academic Search Premier (top menu bar), the Citation Matcher feature allows researchers to search for specific works. Access: http://www.ebscohost.com/academic/academic-search-premier.
  • Google Scholar. Google Scholar has the Cited By feature, which allows researchers to see how many times a document has been cited. At the conclusion of a search, the results show the number of times that both articles and documents have been cited. This feature helps researchers get a preliminary idea of the articles and research that make an impact in a field of interest. Google has a new tool called Google Scholar Citations that will create a public profile for users that includes their citation metrics as calculated by Google. This feature is currently in testing with a small group of users. Keep in mind that Google Scholar includes citations from an array of sources in its cited by calculations, including Microsoft PowerPoint and Word documents, and gives everything an equal rank. Access: http://scholar.google.com and for more information on the new Scholar Citations see http://scholar.google.com/intl/en/scholar/citations.html.
  • JSTOR. JSTOR, by ITHAKA, has two search options: the Citation Locator and Advanced Search. These tools serve as good options for researchers that know of a journal’s accessibility in JSTOR and have citation information available. Citation Locator takes you directly to the article. Once the user selects a document from a search, he or she has the option to use the Google Scholar feature Items Citing regarding the citation of the search item. By selecting this feature, a search will be run in Google Scholar for articles citing the original target article. Users have another feature in JSTOR at their disposal called Track Citations. By selecting this feature, users receive an e-mail when another article in JSTOR cites the original target article. Access: http://about.jstor.org.
  • PsycINFO. Produced by the American Psychological Association, PsycINFO has the Find Citation feature as one of the search options on the main search page. This tools works well when the researcher has detailed citation information accessible to enter into the search. Access: http://www.apa.org/psycinfo.
  • Scopus. This Elsevier database contains the Citation Tracker in the View Citation Overview area and the number of times a document has been cited within that overview, and also the h-index. Another feature of Scopus called Related Documents shows documents that have shared references. Following a user’s search and selection of a document, the Related Documents feature allows the researcher to find other documents that have shared references, author, and/or keywords. Access: http://www.info.sciverse.com/scopus.
  • Web of Knowledge. This database by Thomson Reuters offers search tools like Cited Reference Search in order to directly access specific documents. Moreover, once a user chooses a search in conducted and document, the Times Cited tool helps users to see how many other documents have cited the originally selected document. Users may also view the Times Cited on the search page and the h-index. All of these features benefit researchers by helping them evaluate the significance of a document to a topic, a field of study, or a trend. Access: http://wokinfo.com.

Journal tools and measures

  • Article Influence Score. This score came out of the same project by Carl Bergstrom’s team at University of Washington as Eigenfactor and uses Thomson Reuters JCR citation data. It calculates the relative importance of a journal on a per-article basis. It is the journal’s Eigenfactor Score divided by the fraction of articles published by the journal. That fraction is normalized so that the sum total of articles from all journals is 1. A score greater than 1.00 indicates that each article in the journal has above-average influence. A score less than 1.00 indicates that each article in the journal has below-average influence. Access: More information, including visualization of the data, and scores for certain years are available freely through the Eigenfactor Web site at http://www.eigenfactor.org and/or through the Journal Citation Reports subscription database available from Thomson Reuters at http://thomsonreuters.com/products_services/science/science_products/a-z/journal_citation_reports/.
  • Eigenfactor Score. This score came out of the Metrics Eigenfactor Project, a bibliometric research project conducted by Carl Bergstrom and his laboratory at the University of Washington. The Eigenfactor Score uses Thomson Reuters JCR citation data and measures the number of times articles from the journal published in the past five years have been cited in the JCR year. Similar to the Impact Factor, the Eigenfactor Score is essentially a ratio of the number of citations to total number of articles. However, unlike the Impact Factor, the Eigenfactor Score counts the citations to journals in both the sciences and social sciences, eliminates self-citations, and weights each reference according to a stochastic measure of the amount of time researchers spend reading the journal. Eigenfactor scores are scaled so that the sum of the Eigenfactor scores of all journals listed in Thomson’s JCR is 100. The top 1,000 journals, as ranked by Eigenfactor score, all have scores above 0.01. Access: More information, including visualization of the data, and scores for certain years are available freely through the Eigenfactor Web site at http://www.eigenfactor.org and/or scores are included as part of the Journal Citation Reports subscription database available by subscription from Thomson Reuters at http://thomsonreuters.com/products_services/science/science_products/a-z/journal_citation_reports/.

  • Impact Factor. The Impact Factor is certainly the longest and most well-known journal measure. Eugene Garfield first had the idea for the measure in 1955, and it was published as part of the Science Citation Index in 1961. The journal Impact Factor is the average number of times articles from the journal published in the past two years have been cited in the Journal Citation Reports (JCR) year using Thomson Reuter’s citation data. An Impact Factor of 1.0 means that, on average, the articles published one or two years ago have been cited one time. Citing articles may be from the same journal; however, most citing articles are from different journals. Access: Through the subscription database Journal Citation Reports, available from Thomson Reuters at http://thomsonreuters.com/products_services/science/science_products/a-z/journal_citation_reports.
  • Journal Analyzer Tool. In Scopus, this tool provides a view of journal performance using Elsevier citation data. Using citations from nearly 18,000 titles from 5,000 international publishers, the Journal Analyzer has scores from 1999 forward. The journal analyzer charts the SJR score, SNIP score, total number of citations received by the journal, total number of documents of the journal, and the percentage of documents that are not cited from that year. Access: http://www.info.sciverse.com/scopus.
  • SJR (SCImago Journal Rank) Score. This score was created by the SCImago Lab and Professor Felix de Moya-Anegon at the University of Granada in Spain using Elsevier citation data from Scopus. SJR starts with a basic per-peer-reviewed document measure that is the number of citations received by a journal divided by the number of citations in that journal for the three previous years. SJR then weights the citations with a prestige metric that normalizes across the citation behavior of different journals. Access: http://www.scimagojr.com/index.php and http://www.journalmetrics.com and/or through the Scopus subscription database available from Elsevier at http://www.info.sciverse.com/scopus.
  • SNIP (Source Normalized Impact per Paper). SNIP was developed by Henk Moed at the CWTS (Centre for Science and Technology Studies) of Leiden University in the Netherlands using Elsevier citation data from Scopus. SNIP also starts with a basic per peer-reviewed document measure that is the number of citations received by a journal divided by the number of citations in that journal for the three previous years. SNIP then considers the citations in the context in which they are made (the journal), including a citation’s potential (or how likely it is to be cited) that normalizes across disciplines. Access: http://www.journalindicators.com and http://www.journalmetrics.com or through the Scopus subscription database at http://www.info.sciverse.com/scopus.

Subscription databases for journal measures

  • InCites. This Thomson Reuters database is a citation-based research evaluation tool of people, programs, and peers. Bibliometrics included in the analysis are total papers, total citations, citation impact (cites per paper), percent cited and uncited papers, collaboration indicators, expected citation count, expected citation rates for category, mean/median citation, h-index score, citation frequency distribution, source article rankings, and citation articles rankings. Access: http://thomsonreuters.com/products_services/science/science_products/a-z/incites/.

  • Journal Citation Reports. This Thomson Reuters database can be accessed separately or through the ISI Web of Knowledge platform. The database is the only way to access the Impact Factor metrics, and it also includes the Eigenfactor and Article Influence scores, five-year Impact Factor, and the immediacy index. The Sciences Edition covers more than 6,500 journals; the Social Sciences Edition covers more than 1,900, both from 1997 forward. Access: Pricing and sales contact information at http://thomsonreuters.com/products_services/science/science_products/a-z/journal_citation_reports.
  • Scopus. Scopus, from Elsevier, is a database that includes coverage of scientific, technical, medical, and social science journals and conference proceedings. The bibliometrics included are h-index score, hgraph, citation tracker, journal analyzer, and SNIP and SJR journal scores. Access: http://www.info.sciverse.com/scopus.
  • Web of Knowledge. From Thomson Reuters, Web of Knowledge is a platform that includes the sciences, social sciences, arts and humanities journals, and conference proceedings. The platform can include Web of Science and Journal Citation Reports, depending on the institution’s subscriptions. Citation metrics included are h-index score, citation reports, citation mapping, citing articles, and sums and averages of citations. Access: http://wokinfo.com.

Further viewing

  • Web of Knowledge Training You- Tube Channel. Watch Thomson Reuter’s WoK (Web of Knowledge) channel for short videos on citation counts, cited reference searching, and citation reports features available through the subscription database. Access: http://www.youtube.com/user/WoKtraining.
  • ScopusTV YouTube Channel. Watch Elsevier’s ScopusTV channel for short videos on the h-index, SNIP, and SJR scores available through the subscription database. Access: http://www.youtube.com/user/ScopusTV.

Bibliography
Bergstrom, C.. ( 2007 ). “Eigenfactor: Measuring the value and prestige of scholarly journals.”. College and Research Libraries News 68 (5), 314-316 –. Retrieved 7/14/2011 from http://www.ala.org/ala/mgrps/divs/acrl/publications/crlnews/2007/may/eigenfactor.cfm
Garfield, E.. ( 2006 ). “The history and meaning of the journal impact factor,”. JAMA: The Journal of the American Medical Association 295(1), 90-93 –. doi: [CrossRef] .
Gonzalez-Pereira, B. Guerrero-Bote, V. Moya-Anegon, F.. ( 2009 ). “The SJR indicator: A new indicator of journals’ scientific prestige,”. retrieved 7/14/2011 from http://arxiv.org/abs/0912.4141.
Hirsch, J.E.. 2005 . An index to quantify an individual’s scientific research output. . Proceedings of the National Academy of Sciences of the United States of America, 102 (46), 16569-16572 –. doi: [CrossRef] .
Moed, H.F.. ( 2009 ). Measuring contextual citation impact of scientific journals. , retrieved 7/14/2010 from http://arxiv.org/abs/0911.2632.

Notes
1. Oxford University Press. ( 2011 , June.). Bibliometrics, n. Retrieved 7/23/11, from OED Online, www.oed.com/view/Entry/241665.
2. Please visit the bibliometrics LibGuide they created together at pitt.libguides.com/bibliometrics.
Copyright © 2011 Robin Kear and Danielle Colbert-Lewis

Article Views (Last 12 Months)

No data available

Contact ACRL for article usage statistics from 2010-April 2017.

Article Views (By Year/Month)

2019
January: 36
February: 35
March: 67
April: 50
May: 41
June: 54
July: 29
2018
January: 34
February: 52
March: 64
April: 93
May: 117
June: 71
July: 21
August: 29
September: 37
October: 95
November: 56
December: 30
2017
April: 1
May: 66
June: 9
July: 25
August: 20
September: 19
October: 55
November: 37
December: 30