ACRL College & Research Libraries News

College & Research Libraries News

Research Forum: Consumer research for college libraries: Doing the job in-house

By James V. Di Giambattista

College Librarian and Director of Learning Resources Hawaii Loa College

The Learning Resources Center at Hawaii Loa College entered its fifth year of operation in August 1985. In addition to a new facility the Center’s academic library incorporated a number of technological innovations that were new to the College. These included DIALOG and ORBIT search capabilities, a computer-based circulation, catalog, and acquisitions system, and increased purchasing of microforms. The Learning Resources Committee of the faculty and the Director of Learning Resources concluded that this set of innovations, and the services based on them, were now familiar to the College and should be given a formative evaluation.

The Committee and the Director found themselves involved in a program of consumer research designed to examine the library needs of various user groups, their preferences for service, information needs, and adequacy of the current program. It began the research with a survey directed at the needs of College faculty and the curriculum.

For the purposes of research, the responses of all faculty members were given equal weight. The Committee did not distinguish between part-time and regularly appointed instructors. The overwhelming majority of teachers at Hawaii Loa (77%) hold regular, full-time appointments. The decision making processes among a small faculty of fifty-one are democratic enough that concerns voiced by any faculty member are considered widely.

As a small library, just over 50,000 volumes, the premier aim of the unit is curriculum support.

Staff members work to meet the constant demand for resources dealing with current course topics. Fifty thousand relatively recent acquisitions go far in this cause. The areas of user needs included in the survey were reference services, both computerbased and traditional desk services; reserve and circulation services; and collection devlopment and evaluation. Respondents were asked to evaluate the book and periodical collections in their area of specialization and to anticipate any need for changes in subject area direction and coverage.

Bibliographic and library skills instruction were examined from four perspectives: the format of instruction, responsibility for assuring an opportunity for students to acquire skills, planning linkages between BI and the subject area curriculum and evaluating the use of library skills across the curriculum. The Committee decided that it was valuable for the library to have a profile of the beliefs of the instructional faculty about how bibliographic instruction was best accomplished. Variables considered in developing this sequence of survey questions included whether the BI should be delivered in individualized, small group, or class size format? Should the job be done only by a librarian, in cooperation with the instructor, or did the instructor prefer to do the whole job? Should the BI program be planned as part of a course, exist outside the course curriculum, or be integrated into the course assignments completely? Finally, our survey allowed us to assess the use of library research projects assigned by various instructors and gain a sampling of what they felt were the most successful projects.

Since the installation of a computer-based acquisitions, circulation, and catalog system in 1981, many features of technical support services had changed. Although the catalog is not online, it was important to gather some idea of the reaction to this possibility. Ordering and arrival notification processes had undergone the most change. Notification was faster, but less personal. The staff felt the need to evaluate the effects of library technology on our users.

Finally, the consumer survey process was the only impartial way to sample priorities for services. Faculty respondents were asked to prioritize their need for reference and instructional services, collection development needs in the short and long term, and to rank the importance of major categories of expenditure. This included reference and research services, collection development, and technical services.

Directing and carrying out a library consumer survey

Survey development and administration is a planned process that requires the participation of the community of respondents and a degree of popular support. In cooperation with a sponsoring committee the library must establish the need for information and the appropriate means of obtaining it, whether a mail-out or interview type survey. The committee membership should reach well beyond the library staff into the library’s constituencies if it hopes to examine user needs thoroughly and impartially. The committee specifies the goals of the research and develops a list of the categories of information needed. The committee then appoints no more than two members to work with the director in drafting the instrument.

Once the draft survey instrument is complete, it is tested to see if it elicits the kind of information sought. Responses from a small group representative of the sample is adequate basis for making any revisions in the instrument. The survey administrator, probably a member of the library faculty, then pulls a controlled sample of the population to be surveyed. This may include all undergraduate students, all faculty members or any sub-group. It is important that the sample, or list of persons to be queried, represents the whole population and is not skewed toward gaining more responses from any one group.

Consumer surveys can be mailed to respondents, or respondents can be interviewed in person or by telephone. In a small institution where the library has consistent and frequent contact with respondents, a mail-out survey can generate significant response. This is particularly true if the survey administrator is able to follow-up each non-response with a call or personal note.

On larger campuses and with surveys of the student body, a mail-out questionnaire is likely to generate responses only from interested library users. The opinions of non-users or infrequent users would go unnoticed. This is the best justification for conducting interview-type surveys either by telephone or in person.

Telephone surveys are a useful technique for respondents with stable addresses and telephone numbers. A survey of faculty could be conducted easily by telephone, while doing the same with a student population, particularly off-campus students, is problematic. Random sampling of students in personal interviews is more productive. In either case, interviewers must be trained to apply the survey instrument and to produce consistent and complete surveys during the interview. Their ability to record accurately responses to openended items with speed is particularly important.

In most commercial survey research, telephone or personal interviews yield results that are preferable to a mail-out type survey. Mail-out surveys to most respondents are kept simple, asking only direct, multiple-choice questions. In general, they do not yield detailed results. From our experience, a detailed mail-out survey among college faculty is a reliable instrument. The level of education and sense of responsibility among faculty may be the distinguishing factor.

Compiling survey results

Once data collection is finished, only those mailout surveys that have complete or reasonably complete response are counted. Only those interviewtype surveys which are fully complete are utilized, because the only reason for incompleteness is interviewer error. Samples of over one hundred completed surveys should be keypunched and the computer used to produce tables of item analysis and cross-tabulation. For responses of less than 100 usable surveys, it is practical to hand-tabulate results.

Responses to open-ended questions must be encoded to yield uniform statistical results. Each open-ended response is placed in one category among five to ten possible choices and keypunching and item analysis proceed from the number of responses that fit into any one category. Finally, after analyses of response to each item in the questionnaire, tables are developed comparing responses to related items. Cross tabulations are developed to compare the answers of various segments of respondents to the same questions to examine the consistency of response and explain deviation.

In our survey, a significant number of faculty members felt that students lacked a sufficient knowledge of library skills to complete their assignments, yet faculty showed little interest in teamteaching library skills with a librarian or planning course assignments in conjunction with a member of the library faculty. This cross-tabulation demonstrates a perceived need for library skills among students and a failure of the course planning mechanism to meet that need.

Report writing

Reporting research findings in a way that validates the techniques used to gather information and makes those findings understandable is the heart of survey research. Any written report of the project should state clearly the goals of the research and why the survey was initiated. It should describe the sample population and review the method used to select the sample. Any unusual problems the researcher encountered in the course of the survey that might affect the direction of the results is discussed openly. The actual statistical findings are reviewed in detail. Responses to important survey items are displayed in tables in the text of the report along with cross-tabular data that further interprets the response to individual survey items. The significance of survey results is discussed in terms of the survey objectives. As the statistics are revealed, the results build their case.

Executive summary is perhaps the most important and neglected area of report writing. Executive summary distills the significance of the research for decision makers and interested persons beyond the research committee. If the entire report is the record of the survey, the executive summary discusses the significance of the findings for decisions that may follow. It is not simply a set of recommendations, but a succinct two to four page review of the results and an interpretation of their meaning. Executive summary is most useful in keeping decision makers and those people affected by the results aware of verifiable trends in user needs and preference. Within a college or university administration, it is often the basis for recommending or supporting action and change.

Focus groups

Structured interviews and discussions, often called “focus groups,” can be useful in developing a survey instrument or in refining the results of a previously administered survey. One problem with conducting consumer research in-house is that the persons designing the instrument tend to seek information in the channels where they expect to find it. There is no outside consultant to represent the fresh and sometimes untutored perspective of the library user. Focus groups can serve this purpose.

Participants selected for a focus group discussion should be representative of the sample and willing to talk about their needs and perceptions of library services. The setting and atmosphere of the discussion should be relaxed to encourage discussion. Most focus group sponsors serve food and beverages as refreshment and a stimulant for conversation.

Discussion is conducted according to a set of predetermined questions and the leader is responsible for guiding the progression of talk to meet the needs of the researchers. Participants must be allowed the freedom to express opinions fully, while dialogue among group members is kept to a minimum unless it is useful in furthering the discussion. An observer should be present for reliability, and to develop some record of response important to survey development or interpretation of results.

Focus group results can and should be included in the research report and the results detailed as a further refinement in method.

In-house research vs. contracted services

As many commercial and industrial companies are now discovering, it is possible and feasible to conduct meaningful consumer research as an inhouse effort. Given the often peculiar functioning of academic institutions, and the very specialized concerns of academic libraries, research there may best be conducted in-house.

The practical considerations of in-house research lend themselves well to college and university libraries. The research process, particularly a mail-out survey, occurs over a duration of time. It requires prompt and methodic follow-up, but will not monopolize any member of the staff completely until the statistics are developed and the report written. The publics of most academic institutions are easily identified and tracked. Pulling a sample should not be more difficult than a trip to the registrar’s office for a computer printout. Within most institutions, certainly small colleges, there is the sense of a need to respond when a question is asked in fairness. Finally, research results are given greater weight when they are developed by academics for use within the academy. Consumer research conducted in-house means that individual student or faculty member’s responses really count and are understood in their context because the researchers are not outsiders.

Reasons for research

At Hawaii Loa College research was initiated to assess the full program of information services, including library, audiovisual, and academic computing services. Five years ago the College built a new facility that physically integrated all information service units and placed them under one administration. The Center was to develop a common mission and identity among all units, along with common service objectives. One goal of the survey was to assess if that mission had been accomplished.

The survey results were used to set priorities for budgeting across all information service units, and within each unit’s budget. In the Library, faculty felt that collection development was the overwhelming priority and that reference and research services were adequate to meet student needs. There was less concern for library instruction across the curriculum than had been anticipated.

In a developing institution like Hawaii Loa College, these considerations were crucial to budgeting a $500,000 grant the Learning Resources Center received from a private foundation. Faculty members, responding to a number of items on the Library’s holdings, felt that the critical mass of materials in the general circulating collection needed to support their area of the curriculum was not yet available and needed to be given first priority above all other needs in any area of the LRC.