Brevity, complexity, availability: Extrapolating patron preferences from video analytics

Sarah Hartman-Caverly


Video tutorials are a useful medium for demonstrating skills in academic scholarship. Web videos expand librarians’ scope of impact temporally and geographically, delivering our expertise at our patrons’ points of need. Developing effective, appealing, and accessible video tutorials requires a considerable investment in time, skills, and equipment. Analysis of a video tutorial pilot by Library Services at Delaware County Community College (DCCC) suggests that patrons value tutorials that are brief, demonstrate complex processes, and are available for anytime, anywhere access.

As the librarian liaison to the Allied Health, Emergency Services and Nursing division at DCCC, I worked with division faculty to produce nine brief video tutorials that address the requirements of a capstone research project undertaken by all nursing students in their final semester of the associate degree program.

The tutorials demonstrate database searching, source evaluation, and APA reference list and citation formatting for select reference types. They are intended to supplement a required one-hour information literacy seminar, optional research and citation workshops, and a research guide developed for the assignment. Videos are uploaded to YouTube and embedded in the research guide, linked from the learning management system, and promoted during the seminar, workshops, and reference transactions. Data from YouTube Analytics provide some insight into students’ use of these tutorials during an eight-week period from late January through mid-March 2014, when they were working on the research assignment.

Despite efforts to restrict tutorial length to fewer than six minutes,1 the average view duration and percentage of viewed reports revealed that students do not watch complete tutorials from beginning to end. For the nine tutorials in the pilot, the average video length is three minutes and thirty-eight seconds, and view duration was two minutes and twenty-eight seconds, resulting in an average of one minute and ten seconds of unwatched video time. Furthermore, recordings of two hour-long workshops were made available in response to faculty and student demand. The average length of the edited workshop recordings is thirty-eight minutes and thirty seconds, but the average view duration for these videos was only three minutes and twenty seconds.

A closer look at the average percentage viewed report identified one tutorial for which the average percentage viewed was 125 percent. This certainly stuck out as an outlier in the data; the trend is between 40 and 70 percent viewed per video, with an average of 63 percent. The three-minute tutorial was produced in the same style as the other tutorials and includes the same introductory and concluding information. What differentiates this video from the others is that it outlines the complex and, at times, counterintuitive process of producing a formatted citation for a given article from a proprietary database. A possible explanation for the percentage viewed exceeding 100 percent is that students rolled back the tutorial footage to view each step in time with the process of actually generating a citation from the database. The tutorial demonstrated a process, which was both sufficiently complex and significant to the success of the assignment, that students not only completed the video, but also replayed portions of it in order to master the skill.

The stacked area chart is a feature of YouTube Analytics that graphs relative video popularity over time. As each of the nine tutorials demonstrates a discrete technique, it is reasonable to expect the popularity of each video to peak where most students are engaged in the particular stage of the research process documented in each tutorial. Put simply, tutorials about database searching should be most popular at the beginning of the pilot, tutorials about source evaluation and database-generated citations should be most popular toward the middle of the pilot, and tutorials about in-text and references list citations should be most popular toward the end of the pilot. This distribution is apparent in the data for the first six weeks of the pilot, followed by a week during which the tutorials were essentially not used. The final week of the pilot saw renewed interest in tutorials addressing all stages of the research process, from database searching to citation formatting.

This data is informative for tutorial evaluation as well as service planning. While it will not surprise public services librarians, the stacked area chart beautifully illustrates the proverbial “calm before the storm”2—the period of time when most students are putting the finishing touches on a long-term research assignment, but their procrastinating counterparts are just springing into action. Cross-referencing the dates of the late surge in video use confirms that the uptick began on the Friday and peaked on the Monday before the assignment’s Wednesday deadline. Put another way, students who waited until the last possible minute to begin work on their research assignments still had access to my expertise, irrespective of the fact that I was not available to provide in-person instruction the weekend prior to the deadline.

This data challenged me to consider whether these video tutorials actually enable procrastination and reinforce behaviors that undermine the deliberate, recursive research process in which we try to instruct students. While I’ve dismissed these students as procrastinators, that’s not an entirely fair characterization of a cohort in which many students work full-time, are primary caregivers, and attend classes and clinical rotations on evenings and weekends. I would prefer to think that the tutorials increased access to library instruction for nontraditional students whose work and life schedules conflict with my teaching and reference shifts. Whoever this data represents, one thing is clear: my knowledge was available to them even while my person was not.

Interpreting video analytics both confirms some assumptions about student behavior and suggests ways in which limited resources can be allocated to maximize impact. Encouraging student engagement is challenging enough in-person; competition for student attention online is a losing battle. Video tutorials should be segmented such that they are shorter than six minutes, with fewer than five seconds of introductory branding.3 Ideally, tutorial length should be capped at three minutes to ensure maximum engagement.4 Important or novel content should be front-loaded in the video; a summary of learning objectives and generic branding can be included at the end of the tutorial.

While all tutorials received some use, the only video that students viewed in its entirety (and then some) was one documenting a relatively complex process for generating and editing a formatted citation for a database article. This suggests that students value the ability to pause and replay video content for mastery of complex topics. Librarians should not shy away from using the brief video tutorial format to convey advanced research skills and concepts, such as identifying and using metadata to evaluate search results or generating and formatting complex source citations.

Trends in tutorial video views can also inform strategic allocation of academic library resources with respect to end-of-semester deadlines. Plotting reference statistics for DCCC’s spring 2014 semester, during which the tutorial pilot ran, also suggests a “calm before the storm” pattern of transactions during the last week of class and exam week. For that two-week period, daily reference statistics decrease from 96 transactions on the penultimate Monday of the teaching semester to a low of 12 transactions on Saturday. This is followed by a surge to 102 transactions on the last day of classes (when many research projects are due), and decreased demand for reference services during finals week.

If these trends are consistent from semester to semester, the final week of class presents an opportunity for librarians to intervene and disrupt the cycle of procrastination. Targeted outreach can encourage students to avail themselves of reference services as classes wind down, rather than waiting until the last minute, when demand for library services spikes again. Many academic libraries extend hours during reading and exam periods, providing quiet study space and sometimes offering reference services around the clock.

Further investigation into end-of-semester service use might warrant the decoupling of access to quiet study facilities and access to reference services, and the allocation of extended services during the last week of classes instead of or in addition to finals.

Examining video tutorial usage data can lead to improvements in in-person, as well as virtual library, services. The data demonstrate that students rarely persist through entire video tutorials, so content should be front-loaded and presented as succinctly as possible. Content does not need to be repeated within the video for reinforcement, as usage data show that students will pause and replay segments as needed to achieve a learning objective. Tutorial usage during this pilot revealed a “calm before the storm,” as video views lagged two weeks before the assignment due date, only to surge again days before the deadline. This pattern prompted a cursory investigation into reference statistics for the same semester, which reveal a similar usage pattern.

This trend warrants further analysis to determine whether targeted interventions in the period leading up to campus-wide assignment and exam deadlines can effectively preempt procrastination and improve the quality of student work.


Notes
1. Guo, PJ.. Kim, J. Rubin, R. , “How Video Production Affects Student Engagement: An Empirical Study of MOOC Videos. ” (conference paper, ACM Learning @ Scale Conference, Atlanta, Georgia, March 2014), https://groups.csail.mit.edu/uid/other-pubs/las2014-pguo-engagement.pdf.
2. Hartman-Caverly, S. , YouTube Stacked Area Chart, TIF file, accessed April 13, 2015, http://bit.ly/1FCe842.
3. “Captivate Your Audience. ,” YouTube, accessed April 13, 2015, www.youtube.com/yt/playbook/captivate-your-audience.html.
4.

Guo, Kim, and Rubin, “How Video Production Affects Student Engagement.

Copyright © 2015 Sarah Hartman-Caverly

Article Views (2017)

No data available

Contact ACRL for article usage statistics from 2010-April 2017.