Grace Liu, Amy Pajewski, and Rachel M. McMullin

Evaluating Online Sources

Introducing a 4-Step Strategy

Grace Liu is the business librarian and associate professor at West Chester University (WCU)’s FHG Library, email: yliu@wcupa.edu. Amy Pajewski is student success librarian and associate professor at the WCU FHG Library, email: apajewski@wcupa.edu. Rachel M. McMullin is the humanities librarian and professor at the WCU FHG Library, email: rmcmullin@wcupa.edu.

© 2024 Grace Liu, Amy Pajewski, and Rachel M. McMullin

In 2021, Grace Liu published an article titled “Moving Up the Ladder of Source Assessment: Expanding the CRAAP Test with Critical Thinking and Metacognition” in C&RL News.1 This article was a result of collaboration with West Chester University management faculty aimed to enhance source evaluation practices by incorporating critical thinking and metacognition. The article introduced a 4-step source assessment strategy and documented our initial efforts to enrich source evaluation practices, aligning with the concepts of affective learning and metacognition outlined in ACRL’s Framework for Information Literacy for Higher Education.2

Building on this work, over the past two years, the three authors of this article came together and further refined the source assessment strategy initially developed for the management class, adapting it to serve a wider audience of students from any discipline. We intended to create a learning module that can supplement one-shot library instruction and go into more depth in teaching source evaluation. With the support of a grant from the Committee for Excellence in Learning and Teaching at our institution, we developed a comprehensive learning module that adapts the initial 4-step strategy to focus on evaluating online sources, including videos, quizzes, exercises, infographic guides, and feedback surveys.3

This article aims to introduce the new 4-step strategy for evaluating online sources to the academic librarian community. Our primary goal is to provide support to fellow librarians in their information literacy instruction, especially when confronted with the difficulties of teaching complex information evaluation skills within limited one-shot library instruction sessions. The approaches outlined in this strategy can be used in one-shot classes to initiate source evaluation discussions with students. Additionally, the learning module can be used outside of the classroom to extend students’ learning experiences. By sharing this strategy, we aspire to foster meaningful discussions and encourage the exchange of best practices among academic librarians regarding effective source evaluation strategies.

4-Step Strategy for Evaluating Online Sources

The 4-step strategy is designed to align with the natural process when encountering an article. It guides students through the series of evaluative steps including: (1) forming initial impressions by checking the six quality indicators of the online sources; (2) assessing quality using the CRAAP test, delving deeper into considerations of currency, relevance, authority, accuracy, and purpose; (3) applying critical thinking and lateral reading skills to evaluate the content from various perspectives and sources; and (4) reflecting on their own biases and assumptions throughout the evaluation. The level of analysis goes from simple to more substantial as one moves through the steps. It builds upon the familiar territory students may have encountered in K–12 education, but also pushes them to expand their analytical and reflective skills beyond that level.

Here are the 4-step strategies for evaluating online sources:

Step 1. Check and Identify: Get the First Impression

• URL: Is the source recognizable or reputable? What is the domain name of the website (.gov, .edu, .org, .com, etc.)?
• Date: When was the article written or updated? What time period is covered by the content?
• Author: Is there an author? Is it a person or an organization?
• About page: Does the website have an “About Us” page that describes who they are?
• Links/citations to sources: Are there links or citations to other sources? Does it link to an internal or an external source?
• Layout: Does it provide easy navigation? Does it contain too many distractions?

Step 2. Investigate and Discover: Gather More Evidence with the CRAAP Test

• Currency: Is the article out of date for my topic? Can I find more current information?
• Relevance: Does the content relate to my topic or answer my question? Is the source appropriate for research purposes?
• Authority: Does the author(s) have education or experience that makes them an expert on this topic?
• Accuracy: Is the information accurate? Where does it come from? Can I verify it with a source? Is the article under peer review or editorial review?
• Purpose: What is the purpose of the website? What potential biases does it have?

Step 3. Read and Think: Use Lateral Reading and Critical Thinking Skills

• Consult diverse sources: Have I looked at a variety of sources? Have I compared different perspectives?
• Personal stories versus broader research: Is the source based on anecdotes or research? Is the research method valid and reliable? Who funds the research and what are their views and interest?
• Logical reasoning versus fallacies: Are the arguments convincing? Does the evidence support the conclusion? Does it contain overgeneralizations?
• Track evidence: Have I followed upstream and downstream sources (backward and forward citations) to gather all evidence? Can I reconcile the differences and form my own opinions?

Step 4. Reflect and Practice: Apply Metacognitive Skills and Reflective Practice

• Pierce the filter bubbles: Am I surrounded by sources with the same views? Have I searched the other side of the story with opposite or neutral search terms?
• Examine our own biases: Have I brought my own biases into source evaluation? Have I weighed the reasons from both sides? Do I favor this source because it affirms my belief?
• Climb down the ladder of inference: Have I added personal or cultural meaning to understand the content? Are my assumptions or prior knowledge questionable? Have I come to the conclusion too quickly?
• Practice what we learn: Have I applied the source evaluation strategies in real life? Am I aware of my own biases and cognitive limitations? Have I kept an open mind when forming my beliefs? Have I realized that a simple fact can be disruptive and suspend my judgment until I see the big picture?

The Strategy in Practice

After developing the strategy, we integrated it into the university’s learning management system (D2L), which not only made it more accessible but also scalable, allowing it to reach a larger number of students. During the fall semester of 2022, we introduced the learning module to the whole campus. Hoping to reach as many students as possible, we crafted a promotion plan that included multiple components. First, we promoted the learning module through our university-wide first-year writing and first-year experience programs as well as a campus-wide email to all faculty. Second, the development of our module happened to coincide with the debut of a co-curricular transcript program offered by our Office of Student Affairs, and we worked with them to include the 4-step strategy in that program. Finally, we marketed the module directly to students. We collaborated with our library communications technician to design marketing materials, including flyers distributed in the library and advertisements for the library homepage and campus digital signage.

As of February 1, 2024, a total of 1,785 students have completed the module and received a certificate of completion. We also embedded a survey in the module to gather feedback from the participants. As of February 1, 2024, we had also received responses from 1,696 students. According to the survey results, 77.6% of the respondents indicated that they acquired new knowledge that would help them in evaluating online sources. Additionally, 78.8% reported feeling more confident about their ability to evaluate online sources after completing the module. An encouraging 83.4% of the participants expressed their intention to apply the knowledge and skills they gained from the learning module in the future. These outcomes indicate that the learning module has been effective in enhancing students’ understanding, confidence, and intention to use the strategies taught in evaluating online sources.

Accessing the Learning Materials

The infographic guide to the 4-step strategy is included in this article as figure 1. All the learning materials are also made available via ACRL Framework Sandbox, including links to the videos, a high-resolution PDF version of the guide, video scripts, and exercises with answer keys.4 The instructional videos are presented on a dedicated Google site.5 All the materials provided are distributed under a CC BY Creative Commons License for sharing. So, you can adapt these materials to make your own learning modules to fit the specific needs of the instruction program on your campus.

Conclusion

Although we recognize that our approach may not fully address the complexities of source evaluation, we are dedicated to continuous improvement. We highly appreciate feedback from librarians, as it will greatly contribute to enhancing and refining this learning module. We have provided a link for librarians to share their feedback: https://forms.gle/HCQqUBe1zQbMkcb18. We strongly believe that by working collaboratively and consistently refining the approach, we can bring significant benefits to both our librarian community and the learners we serve.

Notes

1. Grace Liu, “Moving Up the Ladder of Source Assessment: Expanding the CRAAP test with Critical Thinking and Metacognition,” C&RL News 82, no. 2 (2021), https://doi.org/10.5860/crln.82.2.75.
2. ACRL, “Framework for Information Literacy for Higher Education,” 2016, https://www.ala.org/acrl/standards/ilframework.
3. More details on the development of the learning module are available at Grace Liu and Amy Pajewski, “Leveraging Campus Partnerships: Developing and Promoting a Learning Module to Scaffold Students’ Ability to Evaluate Online Sources,” LOEX Conference Proceedings, 2023, https://commons.emich.edu/loexconf/.
4. Grace Liu, “4-Step Strategy for Evaluating Online Sources [A Learning Module],” https://sandbox.acrl.org/library-collection/4-step-strategy-evaluating-online-sources-learning-module.
5. Grace Liu, “4-Step Strategy for Evaluating Online Sources,” https://sites.google.com/view/4-step-evaluating-sources.