ACRL TechConnect

From cradle to grave

The life cycle of a digital learning object

Valerie Beech is associate librarian, email: valerie.beech@marquette.edu, and Eric Kowalik is instructional designer, email: eric.kowalik@marquette.edu, at the Marquette University Raynor Memorial Libraries

Most librarians have probably experienced finding that a website they liked has disappeared, perhaps a video on YouTube, a tutorial, or even just an informative webpage. Sometimes the URL has simply changed, and the item can be found again. Other times the item has truly been retired. Without trying to track down the original creator or hosting body, we may never know exactly what happened nor why. Since we also place links to some of these items on our library webpages, disappearing websites create broken links or “link rot.”1 Librarians are also creators of some of these disappearing websites.

This article attempts to advocate for a better practice when retiring learning objects and webpages in the library setting. It is intended primarily as a case study by providing an example of how to shepherd a learning object through the end of its life cycle. Nothing can “live forever,” so it is wise to plan for the eventual demise of these tools, if only as a courtesy to others.

Finding literature about the “end of life” of software or information systems takes effort. Quick searches in LISTA will bring up hundreds of articles about the software tools available or used to create an item, the creation process, reception, maintenance, and assessment of the item created. However, finding discussion about the process for withdrawing or de-accessioning a digital object or creation is more difficult. The vocabulary related to “end of life” often has more than one meaning or is used in more than one context. In addition to the phrases end of life and life cycle, we found these terms in use: de-accessioning, decommissioning, retiring, sunsetting, exiting, discontinuance, and closedown. Researchers in engineering and business seem to have published the most in this area.

In the engineering and business literature, there are discussions about modeling the life cycle of software,2 the difficulties in dropping or retiring a software system,3 and the trade- offs to be considered.4 In engineering, the IEEE (Institute of Electrical and Electronics Engineers) even created a standard to address the software product lifecycle, though they later adopted the guidelines of the ISO (International Organization for Standardization).5 Within the library literature, there is a recent article advocating for a review process in libraries for the evaluation, acquisition, and retention/replacement of software or systems.6

Businesses and organizations are usually considering these questions at the level of the enterprise and considering profitability and strategy. The questions and their answers may not be as weighty or significant for a simple tutorial or an individual webpage. However, there are still societal costs if this form of knowledge simply disappears.7 As libraries build more digital tools, they should also consider plans for the future of their work. Marquette University Libraries faced such a situation when a learning object created for First Year English (FYE) was no longer being used locally. Before discussing the process used to determine and execute the removal of the learning object, here is a brief description of the tutorial’s origin.

Figure 1: Screenshot of the tutorial.

Figure 1: Screenshot of the tutorial.

Tutorial’s background

The development of this learning object (Figure 1) came about in 2012 as follows: there was a staff change in the library leadership for FYE, and a desire for a library-related ice-breaker activity with which to begin FYE workshops. The idea for the tutorial came from the librarians’ long-standing experience of working at the reference desk with students who had difficulties identifying the journal title in a citation. Several librarians worked with the library’s instructional designer on prototyping the learning object. After several iterations, the tutorial was incorporated into the lesson plan for the FYE library workshops in fall 2012.

According to Google Analytics, during September 2012, when the majority of FYE courses had their library orientation workshops, the tutorial had 2,419 unique pageviews with an average time on page of 3 minutes and 9 seconds. The activity was well received by librarians and faculty, leading to discussions of making the project open source and promoting it to the larger library community.

After consulting with the department head and Marquette’s Office of Research and Sponsored Programs, the decision was made to release the code for the tutorial under version 3 of the GNU General Public License. One can read more about the licensing options in Lindberg.8

The project code was loaded to the Marquette Libraries’ GitHub site, an online code repository. The project was then promoted to the larger library community through sharing the project among colleagues at other institutions, via social media, listservs, and presentations at academic library conferences.

A 2014 poster presentation at the Michigan Library Association Annual Conference resulted in interest from a librarian at Grand Valley State University. Sharing the project code with Grand Valley State University resulted in them creating a version in Articulate Storyline, a rapid e-learning development tool, and adding new features. This led to a collaboration between the Marquette Libraries and the Grand Valley State Libraries in developing an activity for Chicago Style.9 The promotion efforts resulted in a high usage rate of the original HTML tutorial by those outside of Marquette University. A review of the Google Analytics for the tutorial from September 2012 until June 2018 revealed 38,439 pageviews from the United States and 19 other countries. Six countries had views of 300 or more: Puerto Rico (1,087), Canada (823), Peru (757), China (724), Indonesia (395), and Ecuador (329).

Retirement decision

Despite the popularity of the HTML activity, one disadvantage was that there was no way to track student interaction with the activity. Being available on the open web, we could pull usage statistics, i.e., how many people visited the activity and how long they spent on it. Reviewing the IP address allowed us to know whether the tutorial was being used by someone at Marquette, but there was no way to tell for what course it was being used. A new version of the tutorial, which can be integrated into the FYE course sites in the Learning Management System (LMS) and provides student usage data, made the original HTML tutorial obsolete by the start of the fall 2014 semester. Even though the Marquette library no longer used the HTML tutorial for instruction, the positive reaction from librarians at other institutions and the Google Analytics indicated that the tutorial was still being used. As a result, the team decided to keep the HTML tutorial online.

But in 2017, Marquette University adopted an electronic accessibility policy requiring that all websites meet the Web Content Accessibility Guidelines document, version 2.0 standards by September 1, 2020. The new policy required us to assess the HTML tutorial for accessibility. Due to its reliance on a mouse for the drag-and-drop effect, the HTML tutorial was deemed not in compliance with Marquette’s new electronic accessibility policy. We deliberated about whether to invest the resources to bring the tutorial into compliance, but due to the following reasons, it was determined the HTML tutorial should be sunset:

  • Lack of use by Marquette library instruction program. Though the tutorial was still being used, it was not by people at Marquette. Therefore, it did not make sense to divert resources and staff time to update something not used locally.
  • Cost of resources and time to update.After a complete technical review of the code used for this project, it become apparent that the investment to update the code would be substantial.
  • A better version existed. Since the shift in the FYE program, the tutorial had been updated to a format that worked with an LMS and was accessible.

After the tutorial’s retirement was decided upon, we developed a retirement plan adapted from the IEEE document on the Software Development Lifecycle.10 The outline included these steps:

  • Initiate: Obtain authorization from local stakeholders.
  • Plan: Determine archiving options, contact other units involved, and develop a retirement timeline.
  • Execute: Test the archive (code repository), remove the project from the website, and publicize the retirement of the project on the website and code repository.
  • Close: Remove retirement notices on the website.

Our process included the following specific steps:

  • In March 2018, a message was added to the landing page of the tutorial stating that the tutorial would be removed on June 29, 2018. The message included links to the GitHub repository in case others might want access to the code and to host it at their own institution. A link was also included to the new version of the tutorial that could be loaded into an LMS.
  • At the end of March 2018, the GitHub repository for the project was archived, making it read-only. While others could continue to copy the code, they could no longer update the repository. A message was added to the GitHub repository stating the reasons for its retirement and links to the new version of the tutorial (Figure 2).
  • On June 29, 2018, the tutorial was removed from the Marquette Libraries’ website. The landing page was replaced with a brief message explaining why the tutorial had been removed and offering links to the GitHub repository for the original version of the tutorial and the new version.
  • On July 1, 2020, the aforementioned page will be removed from the Marquette Libraries’ website and a permanent redirect for that URL will direct visitors to the GitHub project repository.
Figure 2: Screenshot of notice on the GitHub repository webpage

Figure 2: Screenshot of notice on the GitHub repository webpage (https://github.com/MarquetteRML/MLA-citation-drag-and-drop).

Lessons learned

Just as the software development community is now focusing on digital asset management,11 so should 21st-century libraries, whether that be for research data, digital versions of print materials, or online learning tools. As has already been mentioned, libraries are great at documenting the creation, implementation, and assessment of these digital creations. But we need to do more, and write more, about handling the closedown process. That will ensure that current users of online products are not thrown into a digital abyss when an item is removed from service. It also ensures that old software does die and doesn’t just fade away.


  1. Leighton Walter Kille, “The Growing Problem of Internet ‘Link Rot’ and Best Practices for Media and Online Publishers,” Journalist’s Resource (website), updated October 9, 2015, https://journalistsresource.org/studies/society/internet/website-linking
  2. Vaclav T. Rajlich and Keith H. Bennett, “A Staged Model for the Software Life Cycle,” Computer 33, no. 7 (2000): 66-71; Brent Furneaux and Michael Wade, “The End of the Information System Life: A Model of IS Discontinuance,” DATABASE 41, no. 2 (May 2010): 45–69.
  3. Andreas Schroeder and Benoit A. Aubert, “Barriers to IT Exit,” paper presented at the Thirty Sixth International Conference on Information Systems, Fort Worth, Texas, October 2015, https://aisel.aisnet.org/icis2015
  4. Meghan Squire, “The Lives and Deaths of Open Source Code Forges,” OpenSym ’17: Proceedings of the 13th International Symposium on Open Collaboration, Galway, Ireland, August 23–25, 2017, https://dl.acm.org/citation.cfm?doid=3125433.3125468.
  5. IEEE Standard for Developing a Software Project Life Cycle Process, IEEE Std 1074-2006 (New York, NY: Institute of Electrical and Electronics Engineers, Inc., approved 30 March 2006); IEEE Guide–Adoption of ISO/IEC TR 24774:2010–Systems and Software Engineering–Life Cycle Management–Guidelines for Process Description, IEEE Std 24774-2012 (New York, NY: IEEE Computer Society, 27 April 2012).
  6. Laura Costello, “Applying a Software Development Product Cycle to Library Technology Adoption and Development,” Journal of Library Administration 58 (2018): 334–45.
  7. Jean-François Abramatic, Roberto Di Cosmo, and Stefano Zacchiroli, “Building the Universal Archive of Source Code,” Communications of the ACM 61, no. 10 (October 2018): 29–31.
  8. Van Lindberg, Intellectual Property and Open Source (Sebastopol, California: O’Reilly Media, 2008) chapters 8–10 and 12, https://learning.oreilly.com/library/view
  9. Betsy Williams, Rita Kohrman, Justin Melick, Valerie Beech, and Eric Kowalik, “One Tutorial, Two Universities: How Technology Can Be Adapted to Meet the Needs of Multiple Libraries,” paper presented at the Proceedings of the Forty-Fourth Annual LOEX Library Instruction Conference, Pittsburgh, Pennsylvania, May 6, 2016, https://works.bepress.com/eric_kowalik/13/ .
  10. “Figure D.2—System retirement SPLCM,” IEEE Standard for Developing a Software Project Life Cycle Process, IEEE Std 1074-2006 (New York, NY: Institute of Electrical and Electronics Engineers, Inc., approved 30 March 2006), 97.
  11. Elizabeth Ferguson Keathley, Digital Asset Management: Content Architectures, Project Management, and Creating Order out of Media Chaos (Berkeley, California: Apress, 2014); Brent Furneaux and Michael R. Wade, “An Exploration of Organizational Level Information Systems Discontinuance Intentions,” MIS Quarterly 35, no. 3 (September 2011): 573–98.
Copyright Valerie Beech, Eric Kowalik

Article Views (Last 12 Months)

No data available

Contact ACRL for article usage statistics from 2010-April 2017.

Article Views (By Year/Month)

January: 23
February: 22
March: 18
April: 16
May: 12
June: 0
January: 0
February: 0
March: 0
April: 0
May: 0
June: 0
July: 0
August: 0
September: 0
October: 0
November: 996
December: 50