Integrating the best available health evidence into decision-making processes is an important part of evidence-based practice. However, before applying clinical research findings to patients, health care providers need to determine whether or not the research findings are trustworthy and valid.
It is important to remember that a published study is not necessarily trustworthy or sound. This is evident now more than ever with the ongoing race to develop a COVID-19 vaccine. Questionable scientific research reporting on potential COVID-19 vaccines and therapies is everywhere. Clinical research needs to be evaluated before being relied upon to guide clinical decisions—this is where critical appraisal comes in.
What is critical appraisal?
Critical appraisal is “the process of assessing and interpreting evidence by systematically considering its validity, results, and relevance”.
To critically appraise clinical research is to look at its methodological quality, trustworthiness and applicability to a patient’s specific situation. Critical appraisal helps us ask two essential questions when consuming information:
- Is the information believable? (Internal Validity)
- Does the information apply to my situation? (External Validity)
When should critical appraisal be conducted?
As a key part of evidence-based practice, critical appraisal should be conducted often! More specifically, research should always be critically appraised before being applied to a patient in a clinical scenario.
For example, let’s say a recently published study in your discipline shows a new therapy to be effective and safe for patients, and you are considering the new therapy for a patient of your own. Critically appraising the study will ensure that the findings are valid and the therapy is appropriate for your patient.
Beyond clinical scenarios, critical appraisal is also a key part of evidence synthesis. For example, as part of the systematic review process, articles included in the review are critically appraised for validity.
Using critical appraisal checklists
Critical appraisal checklists (or worksheets) are developed by various authoritative groups and are a step-by-step guide through the appraisal process. Any type of clinical study can be critically appraised, and each study type has its own checklist. There are many critical appraisal checklists out there, including the following sets:
- Joanna Briggs Institute (JBI): Critical Appraisal Tools
- Centre for Evidence-Based Medicine (CEBM): Critical Appraisal Tools
- Critical Appraisal Skills Programme (CASP): Appraisal Checklists
- Scottish Intercollegiate Guidelines Network (SIGN): Critical Appraisal Notes and Checklists
As an example, the Randomized Controlled Trial Checklist from CASP includes questions about study methodology such as:
- Did the trial address a clearly focused issue?
- Was the assignment of patients to treatments randomized?
- Were all of the patients who entered the trial properly accounted for at its conclusion?
Critical appraisal can be conducted without the use of a formal checklist, but using an established set of appraisal criteria may be helpful if you are new to critical appraisal or want to ensure that you are being consistent in the way that you appraise clinical research. For more info on critical appraisal, email us at AskLibrary@nshealth.ca or book a one-on-one online consult with a librarian.
Reference:
The Cochrane Collaboration. (n.d.). Glossary. Retrieved from https://community.cochrane.org/glossary
Kristy Hancock
Librarian Educator
Sometimes the information you need can’t be found by searching a traditional database like PubMed. ‘Grey literature’ is the term used for documents produced outside of the traditional commercial publishing industry. It includes things like clinical practice guidelines, government reports, dissertations, unpublished clinical trials, policies, statistics, and conference papers or posters.
Finding grey literature takes time and can be a daunting task. Here are three tips for finding the grey literature you need.
1. Ask yourself, “Who is likely to produce the information I need?” This will help you determine where to search. For example, if you’re looking for policies on the appropriate use of restraints, check the websites of health districts and authorities, and large hospitals.
If academic institutions are likely to have done research on your topic, try an institutional repository that includes literature produced by its members. Visit the Canadian Association of Research Libraries or OpenDOAR for lists of repositories.
If you’re not sure what body might produce the information you want, check out the New York Academy of Medicine’s Grey Literature Publishers List or CADTH’s (Canadian Agency for Drugs and Technologies in Health) Grey Matters: a practical tool for searching health-related grey literature to narrow it down.
2. Use Google, but use it effectively. Google can lead you to a wealth of grey literature. Bring relevant results to the top and avoid wading through excess by giving Google very specific directions. Use the fields in Google’s Advanced Search function to limit where it searches (e.g. a specific organization’s website, or sites ending with a certain domain like .gov or .edu) or the format of documents included in the results (e.g. PDF or PowerPoint).
You can achieve the same results by using Google ‘commands’ in front of the words you type in Google’s search box. Say you’re looking for a report on global immunization rates. You could use the search string “immunization rates” filetype:pdf site:https://www.who.int/ to tell Google to search the World Health Organization’s website (command = site:) for PDF documents (command = filetype:) on the topic.
3. Document your searches. You may have to search multiple places to find what you’re looking for. Using a checklist and tracking sheet will help you to keep track of where you’ve looked and what success you’ve had. It will also serve as a record of your search should you or someone else need to replicate it in the future. For each resource you search, make note of:
Visit our Grey Literature subject guide for more tips, including starting places for searching for different types of grey literature and detailed guidance on using Google commands. As always, reach out to Library Services if you have questions.
Lana MacEachern
Library Technician
Aberdeen Regional Hospital, Northern Zone