College & Research Libraries News

Evaluating the library BI program

By Gail Z. Eckwright

Students’ comments shed some light on a program ’s effectiveness

Evaluating a bibliographic instruction (BI) program is a difficult task. How do you determine whether or not the program is a suc- cess? At the University of Idaho Library we de- cided to subject our BI program to some scru- tiny. We concluded that if we expected students to listen to us, then perhaps they deserved to be heard as well. Consequently, we devised an evaluation form which provided us with some interesting insights into our BI program and into the students’ perspectives on the pro- gram as well.

Out with the old

University of Idaho librarians have long been giving formal “tours” and instruction to any and all comers. However, in the 1989 fall semester we embarked upon a new program of bibliographic instruction aimed directly at the 975 students enrolled in the English 104 classes. We overhauled the library tour format that had been in use for more than ten years.

We replaced the tours with two 30-minute bibliographic instruction sessions. The new BI is conducted in a remote area of the library, with chairs for all of the students. An LCD (liquid crystal display) panel and an overhead projector are used to demonstrate the computerized portions of the instruction. During the first 30-minute session LaserCat is taught to the students using the LCD panel and projector. The second 30-minute session is devoted to teaching periodical indexes using overhead transparencies made with PageMaker software. At the end of each instructional session the students are given a written assignment that can be completed in 10-15 minutes. The assignment is then returned to the librarian who gave the instruction and ultimately returned by the librarian to the English 104 instructor.

This method of BI does require the full cooperation of the English Department and all of its instructional assistants (IAs) and faculty. The director of writing who oversees the IAs has given this program her whole-hearted support, so participation has been at the 100% level!

Not all of the instructors (and therefore, not all students) are as enthusiastic as the director of writing, however. During the 1989/90 academic year we used a brief evaluation form to get some idea of the students’ reactions to the BI program.

Evaluating the new

Completion of the evaluation forms was optional. Some instructors encouraged their students to return the forms to the librarians; other instructors did not give the forms to the students, and hence the evaluation process was halted dead in its tracks.

We received 778 completed evaluation forms for the year. This figure represents about a 41% return rate.

We had 12 librarians participating in the program. The instructional format was consistent from librarian to librarian. Differences in teaching methods, styles, and techniques certainly account for some of the comments made by the students. Also, every class had students who were disenchanted with the very idea of library instruction, and their remarks and ratings usually were predictably low. Generally speaking, the instruction was considered a success if a majority of those students returning the surveys gave a 5 or better on the evaluation scale where 1 was low, 7 was high, and 4 was in the middle.

Gail Z. Eckwright is humanities librarian at the University of Idaho, Moscow

The confidence quotient

Question 1 on the evaluation form had three parts. The first part, which could be referred to as the “confidence quotient” asked: After completing the library training session and related assignments, how confident do you feel about your ability to identify and locate desired periodicals and books at the U of I Library?

Of the students responding to this question, 137 (17.6%) gave a 4 ( neither “very confident” nor “not very confident” mark). In this same category, 80 students (10.2%) gave a 3 or below mark, indicating that after the library sessions they were not very confident about using the library’s catalog and indexes. The good news was that 561 students (72.1%) gave marks of 5,6, or 7, indicating that they felt confident and very confident about using the library’s catalog and indexes. Also, our success rate improved from the fall semester to the spring semester. In the. fall, 14% of the students gave a 3 or below in the confidence quotient; in the spring only 6% gave those marks. The confident and very confident students increased from 66% in the fall to 80% in the spring. The indication here is that perhaps our instruction improved as we became more familiar with our “new and improved” teaching methods.

We were told that Vandal football is #1, that we needed haircuts, should provide doughnuts and coffee, and to find chairs that didn’t “mark us for life. ”

Other parts to question 1 asked “How effective were the sessions? (a) LaserCat and (b) Periodicals.” The responses to these questions showed a great deal of variance from student to student. One librarian might receive a 2 from one student. Another student in the same class might give a 7. These wide-ranging responses made these survey questions more difficult to quantify. Perhaps at best the responses here can be attributed to learning theories and modalities; a teaching method that appeals to one student may not appeal to another. A librarian who examined these questions could determine in general how well a session went and perhaps consider making some changes if a majority of the evaluations were at the 4 or lower range on the rating scale.

Some comments (thoughtful and otherwise)

The last three questions on the evaluation form asked for narrative comments. These questions provided for some thoughtful, as well as thoughtless, answers. The questions were: What were the most valuable parts of the sessions? What can we do to improve the sessions? Any other comments?

It is difficult, it seems, for any humans (especially college freshmen) to ignore an invitation to make anonymous comments after being required to sit through instruction. These students were no exception. We were told that Vandal football is #1, that we needed haircuts, should provide doughnuts and coffee and popcorn, and to find chairs that didn’t “mark us for life” (also known as waffle-bottom chairs). Fortunately, many of the comments were more helpful and enabled us to improve our instruction.

For example, some students indicated that a tour after the sit-down instruction would help them understand where and how materials were located in the library. Some librarians then incorporated a modified tour into the instruction, and found that it did help the students. Some students picked up minor tidbits about the library, such as information about government documents or interlibrary loans, which they found useful.

The following excerpts from the students’ comments indicate that some of them appreciated the sessions, but they saw some negative aspects about them, too:

“Make them longer. Make students do more examples.”

“I see no need for sessions. If you can read, the computer is easy.”

“I hope if I can’t find what I’m looking for someone will be there to help.”

“I did not learn anything from the sessions. All that was talked about I’d already learned on my own with just a little time spent in the library.”

“Extend the sessions to a couple of days each.”

“Try to make it exciting—it was tooo boring.”

“Make it a little more challenging—it was very easy.”

“Library work is just not the most exciting crap I’ve done.”

“I felt assignments were good, because they gave you a little on hands experience.”

“Don’t insult are [sic] intelligance [sic] so much. We are not stupid, shorten the LaserCat section. If a person can get onto the computer they are easy to use. I figured them out last year in about five minutes.”

More feedback

Other comments were somewhat more encouraging:

“I understood everything until I went to do it.”

“Most people cannot learn to look things [up] just by watching a demonstration. Good effort but I think it is up to the students to go in and get experience with the system after the tour and sessions. Experience= knowledge.”

[The most valuable part was] “Learning how to find stuff on the LaserCats. Before they were real confusing.”

“It’s very helpful. Maybe make more interesting so people don’t blow it off. The sessions are very helpful—but students will only get out of it what they want to—I didn’t really pay attention the second day, but that’s my problem.”

“When we went to actually use the equipment we weren’t stressed because we didn’t have a big paper due. We could ask questions without feeling embarrassed or impatient.”

“I think it’s great that you show students how the library is available for them!”

“If I had no idea how to use the library/ LaserCat it would have been a good thing for me but I have been using the LaserCat since it was put into the library because I found it much easier to use than the card catalog.”

“I had trouble running the LaserCat before, but now it is a snap.”

General comments

Then there were the general comments regarding the facilities, the teaching methods, etc.:

“Can we have a more comfortable room?”

“Go slower.”

“Don’t hold the meeting in the dugeon [sic].”

“Make them more to the pt. Just tell how to do things—who cares about the rest of the stuff.”

“Thank you, it did help.”

“Go back to card catalogs.”

“I think you needed to have more helpers when we performed the tasks [assignments].”

“Allow people who already know how to use the library the chance to test out.”

“Offer it earlier [in the semester] and to all U of I students.”

“Walk us through the library.”

“Don’t treat students like grade schoolers.”

On an up note

And, of course, the best part of the comments were those more positive and encouraging remarks, such as:

“This will help in further research.”

“LaserCat presentation on the overhead was effective.”

“Now I know the magazines are on all floors!”

“I didn’t know about the indexes for the periodicals.”

“I’m glad someone is kind enough to give us some idea of what the library is really for.”

“I never realized how big the library really was until then & how much information they had.”

“I am glad we took time as an English class to learn about the library otherwise I would really be lost.”

“Helpful librarians."

“I learned a lot, thank you! Before my confidence level would have been a 1.” [Confidence level=5]

“It’s a nice library, the sessions did help.”

Is it worth it?

The many varied comments indicate that the students’ reactions cannot easily be summarized.

The evaluations did show more favorable comments than criticisms, which was gratifying to us as librarians and teachers. Given the varied response to the library instruction, is it worth our time to continue with the BI program? Absolutely. If nothing else comes of it, students meet with one or two librarians. They learn our faces and perhaps our names, so that when they return to the library for help, something or someone is familiar to them. The library becomes a little less impersonal and a little more approachable. And as the comments indicate, the students do learn from the BI sessions.

One final individual evaluation form further points to why we should continue. To the question “What can we do to improve the sessions?” the student responded:

“Omit them entirely. I didn’t go to the sessions—I just am assuming that they are a waste of time as most sessions in libraries usually are. I, personally, always feel lost in a library.”

I have no doubt that the English 104 sessions are a necessary component in our library BI program. At the UI Library we will continue to work at improving both the program and the presentation so that students no longer need “feel lost.” ■

Copyright © American Library Association

Article Views (Last 12 Months)

No data available

Contact ACRL for article usage statistics from 2010-April 2017.

Article Views (By Year/Month)

January: 5
February: 3
March: 3
April: 7
May: 7
June: 1
January: 4
February: 5
March: 0
April: 3
May: 4
June: 2
July: 4
August: 0
September: 7
October: 5
November: 3
December: 7
January: 0
February: 0
March: 0
April: 0
May: 0
June: 0
July: 0
August: 14
September: 3
October: 6
November: 3
December: 4