LibQUAL coding cohorts: The benefits of involving library employees in LibQUAL+ analysis

Allyson Washburn


Since 2001, the Harold B. Lee Library (HBLL) at Brigham Young University (BYU) has participated with other libraries, both within and outside of the United States, in the Association of Research Libraries’ (ARL) survey of service quality, LibQUAL+. Analysis of received data is provided by LibQUAL+ team, however, the open-ended comments are not interpreted. While the statistical analysis makes important data available to inform our improvement efforts, the comments section often supplies even more useful information.

Previously, our library statistician coded survey comments into basic categories. This longitudinal view of the data helps the library understand where progress occurs and where problems persist. Library administrators use the data in their deliberations, and findings are reported in a meeting of the library as a whole.

While planning for the fall 2011 survey, HBLL’s assessment unit decided to implement a project to code comments involving library employees from all departments. The coding project undertaken in 2012 provided another perspective on the data and proved to be beneficial, both personally and collectively, to library employees. Working in teams to read and assess the comments allowed them to see and discuss patrons’ concerns. Ultimately, the project had a positive influence on the library as a whole and is still affecting our services today.

Library employee involvement

The impetus for the LibQUAL+ coding project was a poster session at the 2010 Library Assessment Conference. Daniel O’Mahony reported on a project he led at Brown University using library employees (principally subject librarians and public services librarians) to code their LibQUAL+ comments.1 Our assessment unit was interested in learning whether Brown’s process would be replicable. We modeled our project on Brown’s, with modifications based on local circumstances. This idea of using library employees to code LibQUAL+ comments is not unique to BYU or Brown University, however few libraries have used a large group of library employees (more than one-to-four individuals) to do the coding.

The library opted to use LibQUAL+ LITE in 2011, and the sample size of respondents increased by 50 percent. More surveys were completed and about one-third more comments were received than in previous years. With 942 comments to code, enlisting the help of library employees (as O’Mahoney did) seemed to be a practical way to discover themes and issues that were present.

The primary goal of the project was to disseminate the results widely throughout the library so that all employees would gain a better understanding of student perceptions of library services and resources. A secondary goal was to take advantage of library employees’ creative and innovative ideas for solutions to identified areas of weakness. Finally, this methodology spread the work of coding throughout the library.

The team coding process

During the administration of the survey, we began discussing the logistics of coding the comments and the requisite training needed for a successful project. Brown University shared their taxonomy with us, and as we read all 942 comments, it became apparent that with minor editing or substitutions most of Brown’s taxonomy terms were relevant for use in coding our comments.

Prior to parsing out the survey according to the self-identified majors of respondents, BYU colleges had to be harmonized with the LibQUAL+ disciplines. Between 75 and 150 comments were then loaded into Excel spreadsheets by discipline. Separate spreadsheets were also constructed containing all comments that mentioned online content or the Web page. Although comments were separated by the respondents major, the comments reviewed by each group reflected the respondents overall library experiences and not just aspects of their subject area. Spreadsheets were arranged with the four LibQUAL+ dimensions (Library as Place, Affect of Service, Information Control, and General) across the top of the sheet with the codes from the taxonomy in columns under each dimension (see image 1).


Image 1. Excel spreadsheet used for coding.

Coding the comments was a three-step process. First, after reading a comment on the spreadsheet, coders decided the LibQUAL+ category that best fit the comment. Second, they consulted the appropriate section of the taxonomy, looked through the codes, read the explanation, and examined some keywords that might appear in a comment with that code (see image 2).


Image 2. Taxonomy used in the LibQUAL+ coding project.

Before going on to step three, any differences of opinion about what the assigned code should be were discussed and resolved. Third, when team members agreed on the code, they then decided whether the comment was positive, negative, or neutral. Returning to the spreadsheet, the coders entered a P, N, or I in the appropriate code column next to the comment. Because we used an Excel spreadsheet, we had to use a different letter (I) for neutral in order to distinguish between the negative and neutral comments.

Teams of library faculty and staff coders were recruited via the library’s e-mail electronic list and personal visits to department chairs and individuals. Eleven teams of three members each and one team of two members were organized. The coders were placed on teams based on their discipline and, ultimately, were assigned a sheet of comments from respondents in that discipline (with some exceptions). Volunteers were trained during a meeting where they were introduced to their group, learned about the taxonomy, and practiced coding comments. Time was allotted for each team to practice coding ten comments from a previous LibQUAL+ survey. Coders were instructed that some comments contained information that could and should be coded in more than one of the dimensions. Assessment unit members circulated among the teams to answer questions and clarify issues during the training.

At the conclusion of the coding exercise, each team was asked to report their codes for one comment on the spreadsheet. This debriefing was used to help assess and clarify coding misunderstandings before teams worked individually on the 2011 data.

Teams received their spreadsheets the day following the training so they could begin coding while the process was still clear in their minds. Each team determined times and places for their work sessions and had a deadline by which their coding needed to be completed. Coders were asked to resolve all differences of opinion that arose using the taxonomy provided. Discussion was encouraged to help team members come to a consensus, but if that was impossible, the majority ruled.

Focus group assessment

When all groups finished coding, we organized focus groups in order to assess the coders’ experience and to elicit changes and improvements for future LibQUAL+ coding projects. We wanted data about the training, the process, the team approach, and the individual experience. We also wanted suggestions for disseminating the information to the library as a whole. Finally, we hoped to capture suggestions that did not surface in previous discussions for improving the process for the project.

Thirty of the 35 coders were able to attend a focus group. Questions were constructed to guide the discussion and four focus groups were held. For the most part, participants were engaged in the discussion and shared their ideas freely. Recordings were transcribed and reviewed by assessment personnel.

Spreadsheet and taxonomy issues were discussed. However, of particular interest to us were the following themes:

Coders liked working in teams. Nearly three-quarters (73%) of participants who commented on working in teams found it a positive experience. They felt it helped them to see perspectives different from their own.

One participant described her experience: “People would point out different things. You’d get different perspectives and viewpoints between the three of us. We always ended up agreeing, but being able to discuss it through made it a lot better.”

The coders also mentioned that being part of a team helped them meet the deadline for completion because they felt accountable to their team to participate and get the coding done. Working in a team was also helpful for newer employees who had limited or no knowledge of certain areas or services in the library. (These were mainly coders from the library’s IT department who had been employed at the library less than a year.)

One coder said, “It helped me as a new employee . . . [to] better understand different pieces of the library and how they fit together in some of the user perspectives.”

Coders also liked working in teams that included members from different parts of the library. They became interested in what other departments were doing. Only one individual found it a negative experience and another saw a disadvantage in the time required for a group to code the comments versus doing it individually. The remaining four had neutral feelings about it.

Participants felt that they benefitted personally from coding the comments. A little over half (54%) of the individuals who commented on their experience said that they felt that they personally benefitted from the coding experience. Many of these individuals cited specific examples where their perspective changed. A library IT employee commented that “It seems that everyone hates the Web site,” however, he indicated that through the coding process he gathered ideas of how the site could be made less confusing.

Several coders remarked that the library had most things the survey respondents thought we were lacking, highlighting the need to discover better methods of informing patrons about the services the library offers. Other coders found that the comments had more helpful information than they had originally thought. Several mentioned that participating in the coding helped them be more effective in helping students. The following illustrates a particularly enlightening example for one participant in a group of coders:

We were going through our [comments]one day, and one of the students said, ‘I really dislike that I have to go find the books on the shelves’. . . . We were all like, ‘Woah, what do you want? Robots to go get them?’ And . . . we were laughing at it. The very next day, I had a student come up and I was talking to her and…I said, ‘Do you know how to find the call numbers,’ and she said ‘No, my… high school librarian, always got the books for me.’ And then it clicked. . . . That’s what that other student’s comment was. They probably don’t know how to find the books.

The individuals who did not indicate a sense of personal benefit frequently noted their frustration when working on the project because they wanted to help or inform patrons who are unaware of services that are already being offered.

Most said they would participate in a coding project again. Of those who answered this question, 79 percent responded in the affirmative. These individuals commented that it was beneficial to go through the comments and personally see how interactions with library users could be improved. Several coders stated that it was an “eye-opener.” Others liked the aspect of getting to know library employees from different parts of the library and are now engaged in networking and talking about library issues with them.

One coder indicated that it was “really energizing” and all coders enjoyed seeing the positive comments as well. Two coders who said they would not participate again felt that trained students could do the coding or said they would only do it if they felt obligated. One mentioned, “I’ll do it to be a team player, but I would rather not.” Two were neutral about future participation.

Conclusion

The focus group comments helped us to see where we could improve the process the next time around. Conversations with the library’s Administrative Council helped us determine the methods of sharing the information library-wide. Plans were made to disseminate the findings as widely as possible throughout the library and provide discussion venues for all employees. The comments in their entirety were posted on the library’s internal wiki, along with comments arranged by the discipline of the survey participant.

Additionally, a session during the fall library retreat was devoted to working with the comments in groups. The group discussion reaped even more ideas from library personnel in regards to addressing the concerns of library patrons. Even though we have done a lot of work with the comments, the process is not complete. We will hold a discussion in a library-wide meeting to prioritize problems needing solutions. Thus far the project has served as a unique learning experience for the library and has proved to be of great benefit to all library employees involved.


Note
1. O’Mahony, D. , “Sharing the Wealth: A Process for Engaging a Large Group in Coding LibQUAL+® Survey Comments. ,” poster presented at the 2010 Library Assessment Conference, Baltimore, Maryland, October 24–27, 2010.
Copyright © 2013 Allyson Washburn

Article Views (2018)

No data available

Contact ACRL for article usage statistics from 2010-April 2017.