Improving subject guides with existing citation analyses data: Water resources at Oregon State University

Andrea A. Wirth; Margaret Mellinger

Librarians use citation analyses regularly, typically with the goal of informing collection development decisions or evaluating existing collections. Having completed a thorough local citation study of water resources, graduate student theses, and dissertations at Oregon State University (OSU), we were left with the thought that we could use this data beyond the original collections-centered purpose. Specifically, we felt that further analysis of the nearly 1,800 citations could help us create a subject guide for water resources students— one that did a good job of embracing the varied array of resources available with the target community in mind.

The citation study

Library evaluations are part of the information gathered when new academic programs are proposed at OSU. In 2003, librarians evaluated the OSU Libraries collection to determine whether it was adequate to support a Water Resources Graduate Program (WRGP) and where gaps in coverage existed. We conducted a citation study in 2009–20101 to see if the 2003 library evaluation predicted which journals students used most frequently in the five years after the program began.

We used a core list of water resources journal titles (compiled in 2003 from faculty publications), the Journal Citation Reports (JCR) Water Resources category, and interlibrary loans (ILL) on water-related titles. We then captured in a spreadsheet every citation (1,794) from each water resources thesis and dissertation written between August 2004 and July 2009. Each citation was initially assigned to one of four very broad categories: book, Web site, other, and journal. After much data cleanup, we next compared the journal citations to the core list that was developed as a result of the 2003 library evaluation.

Our retrospective analysis found, in brief, that faculty publications predicted future student use better than JCR or ILL data. We also found that journals appearing on more than one of the 2003 data source lists (faculty publishing, JCR, ILL) were more likely to be used by the students. Finally, we identified journals that students cited heavily, but were not included in the 2003 core list. These additional titles clearly demonstrated the highly interdisciplinary nature of the WRGP, and the value of conducting a citation analysis in the early years of a new program.

Supporting the WRGP today

OSU’s WRGP is a collaboration of several departments and comprises three tracks: engineering, science, and policy and management (PM).2 Organizationally, it is located in the Graduate School. Because students in the program take courses from multiple departments, we, as the water resources librarians, lack consistent opportunities to work with water resources students in class settings. A primary vehicle we have for delivering water-focused reference and instruction to graduate level water researchers is a robust library guide, which also serves to direct students to us for further in-depth help.

With comprehensive citation analysis data on hand, we built a subject guide that provides a clear local connection to the featured content. The citation data also helped us work through the “information scatter”3 of this interdisciplinary program as we created the guide.

Developing the subject guide

For this secondary use of our data, we were interested in all citation types equally. We further evaluated the citations previously labeled simply as Web site and other, with an eye to the origin of the resource rather than the format.

The categories we used for analyzing the data for our subject guide were: journal, book, government document, “non-governmental” resources that did not fit the book or journal category, which included a large number of resources from nongovernmental organizations as well as documents from corporations and educational institutions. These categories were also examined in the context of the three water resources program tracks.

We posed several questions to determine how to use the citation analysis data in building a subject guide:

  • Are the citation patterns between tracks different enough to warrant multiple guides?
  • If there is one water resources guide, should it be organized by format of material or theme based on track?
  • Which resources should be highlighted from each citation category, and how should they be emphasized?
  • Did the citation analysis add value to the end product?
  • Is improved subject guide development a practical use of existing of citation data?

Structure of the guide

We use Library à la Carte, an open source content management system, to create subject guides, course assignment pages, and tutorials. Subject guides are structured using a limited number of tabs and highlight various resource types (books, journals, etc.). Modules within each tab organize content such as library databases, the online catalog, Web sites, and images. Subject guides can be quickly made and customized, but we still needed to consider the best structure for the water resources content.

One of the first questions we addressed was whether subject guides were needed for each track. Few disciplines span social science, science, and engineering in the way the WRGP does. It is uncommon, except with the format-based guides (e.g., maps), to create a single guide for an audience with such diverse disciplinary research interests.

Using the citation data as a starting point, we found that students in the three tracks used different categories of resources in varying degrees, but use in the three tracks was not so varied that the guides would be markedly different. Rather we became concerned that creating different guides would be confusing for students and decrease our ability to maintain the guides efficiently.

Not surprisingly, students in all tracks used research articles more frequently than any other type of resource. However, articles were cited about a third less often in PM than in Engineering and Science. Books, government documents, and nongovernmental publications and Web sites were cited more often in PM theses.

The United States Geological Survey (USGS) was the most cited government agency, but USGS publications were used primarily by engineering and science students (see Table 1).

Table 1: Average number of citations per thesis, by category and WRGP track.

Content selected for the guide

There is more written on the organization of subject guides, particularly in light of the development of content management tools such as Library à la Carte and LibGuides, than there is practical guidance for content selection. Articles about content selection are often critiques of content quality or (usually poor) maintenance of links to resources.4

Addressing the types of content to include on subject guides, Ron Gilmour recommends collection policies for guides to ensure they stay focused on local needs.5 Data from a citation analysis is one way of adding to subject librarian knowledge of local needs and builds on general reference and instruction techniques used to create subject guides.6

In the case of the water resources guide, we combined our existing knowledge of resources with the insights we gleaned from the citation analysis. Subject guides at OSU are often organized by resource format. This has a practical purpose in that broadly divided types of resources are discovered through the use of different research tools. The results of the citation analysis, particularly for journals, government documents, and organization Web sites, helped us choose and highlight relevant content for the different sections (see Figure 1).

Figure 1. Subject guide.

Although our decision to structure the guide based in part on resource type (journals and books) may go against some recent advice on the structure of Web guides, it follows others concerning creating “avenues of awareness”7 for the students. For example, including journals cited by peers may help current students expand their knowledge of journals available to them. Additionally, we developed a tab that focuses on primarily organizational Web sites and data, which was heavily influenced by the citations.

Changes to guide as a result of citation analysis

How did this secondary use of citation data help us create a better guide? Overall, we found that a combination of librarian expertise and detailed knowledge of what graduate students are citing made a more complete subject guide. Specific elements were added to the guide based on this analysis:

  • a section dedicated to journals cited ten or more times in the theses and dissertations,
  • organizations that support water resources research and/or provide access to water related data,
  • confidence in emphasizing USGS resources since citations to the USGS far outnumbered any other government sources, and
  • a more critical review of databases in an effort to include those that help users discover the varied content.

In most instances, we focused on the resources that were cited frequently, but there were some exceptions. Some infrequently cited sources were added to the guide because of their potential usefulness to students (e.g., Food and Agriculture Organization Web site). On the other hand, we left off some frequently cited sources, such as advocacy group Web sites and those cited by only one student, albeit frequently.


The guide8 could not have been built solely on sources cited by theses. For example, research databases are standard component of subject guides, and analysis of the citations does not directly address database use. However, building on the knowledge gained about journal and government publication citations helped identify the most logical databases to incorporate.

While we do not recommend conducting citation analysis for the sole purpose of enhancing subject guides, we did find it a fruitful way to re-use existing data.

Others who have conducted a local citation study for collection assessment could profitably re-purpose the data to create more robust and relevant subject guides.

1. Wirth, AA.. Mellinger, M. , “Five Years Later: Predicting Student Use of Journals in a New Water Resources Graduate Program,”. Issues in Science & Technology Librarianship 64, Winter 2011 ,
2. OSU’s Water Resources Graduate Program,
3. Dilevko, J. Dali, K. , “Improving Collection Development and Reference Services for Interdisciplinary Fields through Analysis of Citation Patterns: An Example Using Tourism Studies,”. College & Research Libraries 65, no. 3 ( 2004 ): 216-41 –.
4. Lyons, C. Kirkwood, H. , “Business Library Web Sites Revisited: An Updated Review of the Organization and Content of Academic Business Library Web Sites,”. Journal of Business & Finance Librarianship 14, no. 4 ( 2009 ): 333-47 –.Jackson, R. Pellack, LJ.. , “Internet Subject Guides in Academic Libraries,”. Reference & User Services Quarterly 43, no. 4 ( 2004 ): 319-27 –.
5. Gilmour, R. , “Old Wine in New Skins: Thoughts on Academic Library Web Guides,”. College & Research Libraries News 71, no. 7 ( 2010 ): 350-73 –.
6. Kuruppu, PU.. Moore, DC.. , “Information Use by Phd Students in Agriculture and Biology: A Dissertation Citation Analysis,”. portal: Libraries & the Academy 8, no. 4 ( 2008 ): 387-405 –.
7. Kapoun, J. , “Re-Thinking the Library Pathfinder,”. College & Undergraduate Libraries 2, no. 1 ( 1995 ): 93-105 –.
8. OSU Water Resources guide,
Copyright © 2012 Andrea A. Wirth and Margaret Mellinger

Article Views (2018)

No data available

Contact ACRL for article usage statistics from 2010-April 2017.