Now that you are more informed on how to critically appraise a journal and the authors of an article, let’s explore the sections of an article and their primary purpose.
Abstract
The Abstract is our first glimpse of a research paper. It summarizes key points from each section of the study, including some larger context, study design, results, and conclusions. After you’ve verified that the journal and publication date are reliable and current, use the abstract to determine if reading the entire article is worth your time.
Though you may be able to suss out obvious bias in the brief description of the study design found here, that task is much easier when reading the full text. If you can’t access the full text, tread lightly with the info in the abstract and how you apply it.
Introduction
With the Introduction section, we’re finally into the full text of the research paper. Consider how much critical appraisal we’ve done already and we’ve only just arrived at the “meat” of the article!
The introduction is where the study’s authors describe what is already established on the topic to set the context and background for their research question. They will also clearly define their hypothesis (research question) and explain why it was important to ask that question. It’s a great idea to move to this section after reading the abstract.
Methods
In this section, researchers describe what they did (both in study design and with the statistical analysis) and who it was done on. The study’s methods should also be described clearly enough that the study could be undertaken by another group of researchers interested in verifying the outcome. Often, a reader will skip Methods after reading the Introduction and move directly to the Discussion section instead. If, after reading the Discussion, your interest is still piqued, head back to Methods section and give it a skim.
Some helpful questions to ask when assessing the methods are:
- What was the study’s sample size?
A study’s sample size is simply the number of units (people, animals, etc.) that were measured. This number is generally expressed using the variable “n.” For example, a study that measured macronutrient ratios in 387 people with Type II Diabetes has an n of 387 (or, n=387).
The number of people in a sample size can make a huge difference in the real-life application of the results of the study. With more people, you get more data. An adequately large sample size allows scientists to get closer to the ‘true’ average within the whole population.
- What was the makeup of the sample population?
This question aims to help you uncover who the people were that were counted in the study’s sample size. This is often based on various “exposures” or “outcomes”, depending on the study design.
The makeup of the sample population also prompts you to think about who was in the control group, if one was used. This group’s exposures and characteristics should be as carefully matched to the sample population as possible, minus the exposure of interest. Many specific issues may arise when either the sample group or control group were not well chosen, but they all boil down to the study’s results not being applicable to “real life.”
- What was the duration of the study?
Often, to save money in the execution of a research study and to increase the chances of participant retention and compliance, prospective studies are done over relatively short time frames. There are many studies out there that draw an unfair conclusion based on results garnered from a biologically insufficient period of time.
- What was the methodological quality of the study?
This includes the appropriateness of the protocol, the quality of the treatment, data gathering methods, the specific data points that the researchers chose to measure, and so on. Here’s where we can really get out our magnifying glasses as nutritional therapy detectives!
Issues you may run into include–but aren’t limited to–use of poor quality foods, macronutrients, or micronutrients and extrapolating the results to all versions of that food or nutrient; not considering interactions between nutrients; misrepresentation of the actual methods used; using a non-inert substance as a placebo; inappropriate extrapolation of findings on a specific substance (vegetable oil, for instance) to a large family of nutrients (ex. all polyunsaturated fatty acids).
Results
The Results section answers the question, “What did the researchers find?”. Did a change occur or not, due to the experimental intervention? Was that change significant? As with Methods, this section is often skipped over and returned to after reading the Abstract, Introduction, Conclusion, and Discussion.
Results should ideally be communicated without interpretation or discussion (interpretation is generally saved for the Discussion section). This section should read in a logical progression based on either the design of the study OR the importance of the findings. Generally, pay close attention to whether or not the author’s findings confirmed their hypothesis! Authors will often utilize tables, graphs, and charts to summarize information. Look over images and tables as you’re scanning through the article, even if you don’t read this section in detail! They’ll give you a lot of information without having to wade through statistical jargon.
The two main concepts that the Results section of a paper is aimed at displaying are:
- Are the results statistically significant?
- Are the results clinically significant?
When it comes to the statistical significance of a study’s findings, what scientists are really asking is, “If my null hypothesis findings are true, how likely is it that my results would have occurred simply by chance?”.
Now, clinical significance. This is a big one for clinicians. Sometimes the results of a study–even if shown to be statistically significant–may not make a big difference in clinical practice. Because researchers love having measurable metrics for things, even clinical significance is determined by specific mathematical equations, represented in the form of a statistic.
Discussion
The Discussion section is one of the final sections of a research paper. It serves to highlight the findings of the study and put them into context. This section should answer the question, “What do the results mean?”. It’s often read immediately following the Introduction section.
Discussions will generally include why the authors feel their findings are important and unique in the context of broader scientific knowledge. The findings of the study are usually discussed from most to least important, with comparisons of findings to similar studies. They will attempt to interpret their results and note possible clinical applications. They should also review the strengths and weaknesses of the study.
Critical Appraisal of Discussion
- How did this study’s results compare to results from other similar studies?
If the results are substantially different than what has been found in other research projects, make sure the authors thoroughly explain these discrepancies. Do your own detective work by looking up related studies to see if you can identify differences in the methods or analysis that could explain them.
You may notice important differences in the design of the studies (ex., morning vs. evening, blinded vs. non-blinded, differing control variables), the participants (ex., women vs. men, large cohort vs. small cohort), or the methods used to collect and measure data (ex., blood tests vs muscle biopsies).
- Have potential biases influenced the author’s interpretation of the results?
If you were impressed (either positively or negatively) by the information you gleaned from the Discussions section, pay particular attention to how the information in that section compared to the information in the Results section. Does the language accurately represent the statistical calculations of the Results section? Notice qualifying language like “suggests” or “possibly” as opposed to definitive, concrete language. Is the use of this language warranted? Are they interpreting correlation as causation?
- What were the limitations of the study?
An important part of the Discussion section is that the authors discuss the various factors that could be impacting the study’s results. This has huge impacts on a study’s clinical importance. Make sure you’re considering the weaknesses and limitations of the study as you’re reading through the other sections. To truly understand this, you’ll likely need to include the Methods section in your reading.
Conclusion
The final part of a research paper is the Conclusion. Think of it as a summary of the Introduction and Discussions sections. Purdue’s online writing lab instructs that the Conclusion should restate the topic and its importance, restate the hypothesis, briefly touch on the findings in the broader context of the subject and field, and possibly provide a call to action/future research directions (https://owl.purdue.edu/owl/general_writing/common_writing_assignments/argument_papers/conclusions.html).
Critical Appraisal of the Conclusion
- Are the conclusions supported by the data?
As with the Discussions section, be on the lookout that the authors draw appropriate conclusions based on their findings. Once again, the precise findings presented in the Results section should match the importance placed on those findings in the Conclusion. They should also be clinically contextualized appropriately to the study’s design. For instance, the results of animal studies should not be directly extrapolated to clinical meaningfulness in humans. Observational studies, such as a case-control study, should not be said to ‘prove’, but rather ‘suggest’. Remember, observational studies show correlation, not causation.
In the discussion of future research directions, the authors should represent their findings objectively within the larger context of research on the subject and current clinical standards. When their results are important, they shouldn’t bury them… and when they’re not that significant, they shouldn’t overstate the value. They should characterize their place accurately and suggest next steps for better understanding or treatment of the topic that can be applied clinically.
References
This is the last section of a research paper. It lists each source the authors have cited in their paper, in alphabetical order of the first author’s last name, if the paper uses APA citation styling. If you spent time reading through references to educate yourself on the topic of the study as you were moving through the Introduction section, you may have already spent some time here! In many instances, you’ll never read this section. If you do, though, it’s generally last.
Critical Appraisal of References
- Are the author’s citations reliable?
If the findings of a paper leave an impression on you (either good or bad), checking the in-text citations to verify they’re being represented accurately is a wonderful idea. It’s honestly unsettling how often authors cite data taken completely out of context or misinterpreted. In addition, notice whether they are frequently citing their own work. By itself, this isn’t a red flag. But, if they are citing themselves without using other unassociated research to back their claims, this is a flag and could represent confirmation bias.
You’ve now done some thorough research on how to read a research paper! Challenge yourself to find an article on a nutrition topic of interest to you and go through these steps. Asks questions, make observations, explore each section and spend some time determining of the article is of merit before you embrace its content as truth.
Meredith Kinsel-Ziter, NTP, BCHN
Meredith Kinsel-Ziter, NTP, BCHN, is one of the Nutritional Therapy Association’s Lead Instructors and a Curriculum Contributor.
