In our last installment of the research series, we discussed the importance of finding a reputable database for sourcing research and developed critical skills for performing searches that yield quality studies. Today we will be digging into how to engage with a research article once you’ve found it.
When you’re first beginning, reading a research paper can feel like a struggle as the language is describing unfamiliar ideas and the order is not necessarily linear–the best way to read it is not from beginning to end! You’ll also find there are vastly different calibers of journals publishing these articles, so the art of reading a research paper actually begins with critically appraising its source.
Before taking the time to read anything in a research article, go through the following journal evaluation steps to see if it’s worth your time.
Critical Appraisal of Journals
- Does the journal seem legitimate?
The first thing to consider is whether the journal is peer-reviewed or not. When an article is peer-reviewed, it doesn’t just reflect the point of view of the authors–it also reflects the challenges placed on the authors by the reviewers, minimizing bias and maximizing integrity. Checks and balances!
To figure out whether a journal is peer-reviewed, search its submission guidelines by Googling the name of the journal and the words “submission guidelines.” The review process will vary from journal to journal. The British Medical Journal’s (BMJ) review process is demonstrated as an example.
https://www.bmj.com/about-bmj/publishing-model
Unfortunately, some journal publishers lack the solid and ethical model outlined by the BMJ. Sometimes this mismanagement is purposeful and sometimes not, but study authors and the public generally always suffer as a result. These journals make their money by charging fees to authors but not delivering on agreements. They often promise authors inclusion in top-notch research databases, then fail to keep that promise. They pledge a robust peer-review process, then don’t deliver on either. These journals are known as predatory journals. Because of their lack of scientific and ethical integrity, they don’t make good information sources.
Another way to check the legitimacy of a journal is through its rank or impact factor. These are measures of the frequency that a journal is cited in other relevant published sources. While they can be a bit of a popularity contest, highly ranked journals are almost never predatory and are virtually always legitimate.
SCImago Journal & Country Rank is a database that allows you to look up the rank of various scientific journals.
- What is the publication date?
Old studies can be interesting, but don’t reflect the most current understanding of a topic. Try to find studies published in the last 5 years or less. Sometimes, though, you’ll need to rely on older studies because they were monumental in establishing a scientific discovery that is now accepted as fact and is no longer being actively researched.
Critical Appraisal of Authors
Once you’ve determined the journal itself is of reputable quality, the next step is to evaluate the authors of the study.
- What are the authors’ levels of expertise?
A quick online search of the authors to look for a short bio will probably give you all of the information you need. Often, this is found at their institution of employment. Searching bios of the first and last authors are generally sufficient to verify expertise on the research team.
- What are the authors’ biases?
Having biases is not bad. We all have them, whether we’re conscious of them, or not. It’s when we allow those biases to influence our treatment of or approach to a topic– in a way that is inconsistent with the objective truth– that they become problematic.
The most important factor is to identify whether the author’s bias has interfered with their search for truth. Additionally, be aware of your own bias as you choose what studies to inform yourself with. Look for studies that challenge your ideas as well as ones that support them.
- What are author affiliations?
Often, authors are working at a hospital or university but, sometimes, they are employees of or have an interest in a company that could benefit from a certain study finding. Other times, the researchers are technically unbiased, but the study was funded by a company or industry organization. The Acknowledgements section (generally either at the beginning or end of the paper) may reveal these conflicts of interest that can be clues to hidden bias.
Conflicts of interest aren’t a guarantee that the study’s results are inaccurate. It may very well be a beautifully designed study with reliable and objective results–but there’s a higher chance that it isn’t when compared to a study with no conflicts of interest. Author/study affiliations are simply red flags to shift your bias detective skills into a higher gear. Ideally, the study would be repeated and the results confirmed by other unbiased, unaffiliated researchers.
In the next part of our series, we will explore the sections of an article and their primary purpose.
Meredith Kinsel-Ziter, NTP, BCHN
Meredith Kinsel-Ziter, NTP, BCHN, is one of the Nutritional Therapy Association’s Lead Instructors and a Curriculum Contributor.
