ACRL College & Researc Libraries News

College & Research Libraries News

INTERNET RESOURCES: Resources for library assessment: Tools for a new era

by Kathleen Bauer

Librarians are increasingly turning to the Internet and electronic resources to provide patrons with new services. As electronic resources and services become an increas- ingly large segment of all library resources and services, librarians must find ways to ad- equately report their usage.

Finding adequate and accurate measures becomes more important as libraries experience a decline in traditional services such as questions at the reference desk. For many libraries, electronic resources represent an area of growth, and so reflecting their use in library statistics is key to justifying budget requests and demonstrating the importance of the library.

There are many ways of measuring library usage: typically librarians count books circulated, volumes shelved, money spent on acquisitions, gate counts, and questions answered at the reference desk. While these measures are widely accepted, they do not reflect electronic services. To accurately reflect this usage, libraries must at a minimum be able to count and report Web hits, database sessions and searches, and e-mail contacts. Counting these things will force librarians to look at some new tools for assessment.

In this climate, where electronic and print resources and services co-exist, libraries seeking to perform careful analysis of usage and funding will need to use both traditional and new tools. Indeed, this new climate may represent an opportunity for librarians to review all the data that they gather and all of the tools they use to analyze that data. Some of the tools librarians may find helpful are available via the Internet, including software packages for studying Web usage and for performing statistical analyses, sites that provide data for use in benchmarking with comparable libraries, and sites that can help with studying demographics in their community.

Web analysis software

Measurement of Web usage depends on studying server log files, which are text files that record each time a file on its server is requested. This line at a minimum includes the date and time, IP address of the requester, size in bytes of the file requested, and a result status of every hit on a Web site.

Log files were actually designed to help network administrators gauge traffic on a site, not to judge how people use a site, and so they are not a particularly elegant tool for studying how someone uses a site. However, they are the only tools available, without using cookies or registration and logins, which can measure usage of files on a Web site. Log files are usually incredibly long text files that would be impossible to interpret by hand. Luckily software is available to help parse these files.

About the auther

Kathleen Bauer is librarian at the Yale School of Medicine, e-mail: kathleen.bauer@yale.edu

• Analog. Analog is a popular free software tool designed by Stephen Turner from the University of Cambridge. This is very basic software that will do simple analysis of log files. Nothing fancy, but it will tell you the number of hits on your Web site and the most requested files. Access: http://www.analog.cx/.

• WebTrends. This software is offered in different packages at different prices depending on the complexity of your Web site and the desired level of sophistication of analysis. WebTrends lets the user customize detailed reports through a menu-driven system. WebTrends offers more graphics than the free software alternatives. WebTrends reports search strings and search engines used to find your site. Access: http://www.webtrends.com/.

Demographics

To accurately plan for what new ser- vices are necessary and desired by pa- trons, librarians need to understand demographics in their communities. Demographic information can help libraries to better understand the population they serve and to look for trends that may affect them in the future.

• U.S. Census Bureau Access Tools. The U.S. Census Bureau gathers detailed data on the population through the diennial census and other surveys. Tools available at this site include State and County Quickfacts, where commonly asked for county-level informa- tion can be accessed through a series of pulldown boxes. Data available include per- centage of the population by race, gender, education level, home ownership, and median household in- come listed with the corresponding state-level information.

More detailed census tract level information is available from Censtats.¹ Access. http://www. census.gov/main/www/access. html.

Economic benchmarks

Benchmarking data can help a library design reports that show where they stand in relation to other libraries or in relation to more general indicators of the economy.

• Consumer Price Index. Economic data indicate whether a library’s funding is keeping pace with inflation. An important inflation indicator is the Consumer Price Index, produced by the Bureau of Labor Statistics in the Commerce Department of the U.S. government. This index measures the cost of an imaginary basket of market goods, and reports the increase or decrease in that price. Data is available on a monthly and an annual basis. Access: http://stats.bls.gov/ cpihome.htm.

Library benchmarks

• Association of Research Libraries (ARL) Statistics and Measurement Program. The Association of Research Libraries has been measuring various aspects of library services since the 1970s and some of this data is available on their Web site. A drawback to this data is that ARL represents an elite group of 120 large academic and research libraries, so the data is not a good reference point for all libraries. It does however represent a significant repository of information about trends in some libraries. Data available include the “ARL Annual Salary Survey” with selected data available from 1995-99. Data collected from 1973 to 1994 are available only in print. Data include average and median salaries by race, gender, position, experience, and geographic region.

Also available from ARL is the “Library Materials Budget Survey” compiled by Collection Development Officers of the Large Research Libraries Discussion Group. This data comes from an even smaller group of 40 libraries in ARL, although, starting in 1998, the budget survey was sent to all ARL libraries. Detailed spreadsheets are available dating back to 1987. The survey began to ask about expenditures for electronic resources in the 1993-94 fiscal year, but this information is still sketchy. For example, no members reported electronic expenditures for 1998-99.

A third source of data from ARL is the “E&G Report,” or library expenses as a percentage of educational and general expenditures. Data available starting in 1982 show steep declines in the percentage of university dollars spent on libraries. Access: http://www. arl.org/stats/.

• National Center for Education Statistics (NCES) Library Statistics Program.NCES began collecting library statistics in 1989. Information is available from a range of library types, including academic, school and public libraries. The NCES’s Academic Library Survey is a complement to the Integrated Postsecondary Education Data System (IPEDS). Data available for academic libraries include total salaries, gate count, expenditures, expenditures per full-time equivalent student, expenditures per category (e.g., computer hardware and software, serials subscriptions, etc.), and reference transactions. Access: http ://nces. ed. gov/ surveys.

• State Library Resources Arranged by State.This is a compilation of data available for libraries in the 50 individual states by Joe Ryan. Each state compiles data independently and this is not a coordinated effort to collect data for comparison between states. Access: http://web.syr.edu/~jryan/infopro/statsl.html.

• U.S. National Commission on Libraries and Information Science (NCOS) Statistics and Surveys.This site contains the results of a series of surveys beginning in 1995 discussing public libraries and Internet connectivity. Access: http://www.nclis.gov/ statsurv/ statsurv. html.

Measuring usage of electronic resources

Unfortunately there are no accepted standards for gathering and reporting data about electronic resources, and there are no nationally available data for benchmarking. Some groups that follow are attempting to create standards. However none of these groups offer an easy blueprint for others to adopt. They serve as a starting point in thinking about measuring usage of electronic resources in your library.

• Developing National Library Network Statistics & Performance Measures.Not much at this site seems to be working, but “Proposed Data Elements/Statistics” can help give you a brief idea of some things you could use to measure usage of electronic resources in your library. These proposed measures came from a project run by John Carlo Bertot, an associate professor at the School of Information Science and Policy of the State University of New York at Albany. Access: http://www.albany.edu/~imlsstat/ propstats.html.

• Equinox Library Performance Measurement and Quality Management System.Equinox is a European project funded by Telematics for Libraries. The Equinox project proposes a list of possible library performance indicators and seeks to encourage libraries to adopt these indicators. The purpose is to provide standard data that would allow comparison between libraries. However, these have not been made into a widely agreed to standard. Access: http://equinox. dcu.ie/.

• Guidelines for Statistical Measures of Usage of Web-Based Indexed, Abstracted, and Full Text Resources.Adopted in 1998 by the International Coalition of Library Consortia (ICOLC), these guidelines represent the data the ICOLC wish database vendors to include in statistical reports. This represents another method in attacking the problem of the lack of standards. Instead of asking libraries to adopt the standards, these guidelines are aimed at database vendors and electronic full-text publishers. The guidelines request that vendors supply to libraries data on a monthly basis and give counts of the numbers of searches and sessions in each database. Access: http://www.library.yale.edu/ consortia/webstats.html.

Statistical software and other statistics resources

Carrying out assessment activities will at some point require the use of statistical manipulation of data. Most spreadsheet software products include some statistical capabilities. Microsoft Excel, for example, can perform many statistical calculations, such as measurements of central tendency (e.g., average), standard deviation, t-tests, and others. More sophisticated analysis may require more powerful computing abilities. The two companies listed below offer statistical software packages of varying power and price.

• SAS. Software from the SAS Institute. Access: http://www.sas.com/.

• Rice Virtual Lab in Statistics. This site offers some nice reviews of statistical concepts. It includes HyperStat, an online textbook, and simulations that demonstrate how some statistics equations work. Access: http://www.ruf.rice.edu/~lane/rvls.html.

Notes

1. At the time this review was written, data from the 2000 census were not available.