Critical appraisal

OPTO31252 R19–0553

Library for Educators

6 min readFeb 3, 2022

Supporting materials

Slides/materials: slides and handouts
Online session materials: mentimeter

Practicalities

Group: 30–50
Length: 50 minutes
Room: a smaller seminar style room is best
Discipline: any, the examples are relevant to optometry but these could be tailored
Level: UG year 3

Learning outcomes

After engaging with this support, you will be able to:

Identify an appropriate tool to use for finding information for your specific purpose
Identify and use key databases in your discipline
Identify and use relevant specialist information common in your discipline
Understand the conventions for citation styles and bibliographies required for your assignment
Identify and implement an appropriate information management technique
Develop strategies for assessing the appropriateness of sources to use in your assignments
Evaluate the strengths of online user-generated content as sources of information

Suggested online resources

Session content

This session is the first of two which will support this particular cohort students with their final project, which is an assessed literature review. It covers critical appraisal and analysis, and is complemented by a later session covering referencing.

At the start students will be given some context, explaining the purpose of the session and an outline of what will be covered:

Choosing relevant search results
Assessing the validity of your sources
Analysing the results
Incorporating critical analysis

Introducing critical appraisal (Slide 3)

Ask the room who is aware of the concept of critical appraisal? See if there are any volunteers who would like to offer a definition before flipping to slide 3.

Then summarise the concept using the quote on slide 3, taken from the MLE online resource about critical appraisal:

Activity: assessing relevance

Ask students to work in groups and consider how they would evaluate the first excerpt on their handout, which has been taken from a relevant article in the field of optometry.

Student should make a note of the elements/pieces of information which would help them decide whether the article was relevant and a credible source of information?

Ask for feedback from the group after 5 minutes has passed.

Where should I look?

Use slide 6 to highlight to students how they can use different sections of a study report to “critical appraise” the study:

Marking criteria (slide 7)

Relate the skills explored in the last activity directly back to the marking criteria for their literature review assignment:

The content of your report will be examined for….the relevance of information to the topic (i.e. have you been selective in the information that you have included?).

The hierarchy of evidence (slide 8)

Talk students through the different types of scientific study shown in the pyramid, highlighting that the pyramid reflects a generally accepted view of the relative value of different types of scientific study. Randomised control studies, at the top of the pyramid, are considered to be the most methodologically robust type of research. At the bottom of the pyramid are case reports and case series, a type of study which only looks at a small number of patients at a time, which are seen to have less significance. Note that, when assessing the validity and significance of a scientific/medical study it is important to consider the following:

The sample size (how many people were involved in the study)
The level of control researchers had over participants exposure to a particular treatment
Whether a control group was used

Refer students to the My Learning Essentials online resource about critical appraisal, embedded in their Blackboard space, which gives further information about the different types of scientific study you might come across, and the relative strengths and weaknesses of each one.

Activity (slides 10–11)

In their groups, students consider the extract from the methods section of the example article (the second excerpt on the handout) and answer the following questions:

What type of study do you think it is? (RCT, Cohort Study, Case control study, Cross sectional survey, case report/series)
Discuss the strengths/weaknesses of the methodology used to carry out the research.

Analysing the results (Slides 12–15)

After you have deciding if something is relevant to your topic, and the methodology used it valid, the next step is to evaluate the results. Always aim to look at the results of a piece of research before reading the author’s discussion. The results are the raw evidence and the discussion is the authors interpretation of that evidence. You may have a different view of what the results say and their significance than that of the author.

Slide 14

Use the example newspaper article on slide 14 (which erroneously reported that there were only 100 cod left in the North Sea) to highlight why it is important to go to the original source of evidence and critically evaluate the information you use in assignments.

The Telegraph ran the story about looking at data from the International Council for the Exploration of the Sea (ICES). It then asked researchers from the British government’s Centre for Environment, Fisheries and Aquaculture Science (Cefas) for help with the numbers. According to Cefas, however, the journalists “misunderstood the data”. The Telegraph chose to class an adult cod as aged over 13. But that’s not merely an adult cod. It’s an ancient cod. Equivalent to being over 100 years old.

Highlight the following points:

Always go to the original source of the data and make your own interpretation
When evaluating a source of information consider why it has been put together and what affect that has on how the information is presented (in the case of the Telegraph article the purpose was to inform a lay audience/entertain whereas the original research would have

Ways to be critical (slides 16–17)

Build on the learning from the previous activity by explaining to students that “being critical” does not just mean commenting on the strengths/weaknesses of a particular study. In the context of a literature review you are being asked to comment on the research topic as a whole, that means making connections across different studies, rather than just looking at each one in isolation. In order to do this students might want to create a mind map/take notes on themes which emerge during the searching process.

You can show critical awareness by doing all of the following:

Highlight strengths/weakness
Point out trends or themes
Highlight similarities and differences
Compare/contrast
Inconsistencies
Clinical significance
Statistical significance

Next steps and further help (Slides 18–23)
Highlight support which is available via drop ins and provide guidance on useful next steps following the presentation. Demonstrate the location of the My Learning Essentials online resources within Blackboard using slide 20, and highlight the most relevant resources e.g. “Being critical”

Internal ID: R19–0553