How to Read an Article for Journal Club

I. BASICS

A. Your “first pass” at reading the article can help you answer a few basic questions, which include

1. How was the study designed?

2. How well was the study executed?

3. What were the results and how might they affect your practice?

B. Once you have a general understanding of the manuscript, a more detailed systematic evaluation (as below) can be performed during your second pass

II. LEVELS OF EVIDENCE

A. Level 1 evidence: Randomized control trial or a meta-analysis of Randomized controlled trials (RCTs)

B. Level 2 evidence: Prospective cohort study

C. Level 3 evidence: Retrospective cohort study

D. Level 4 evidence: Case series

E. Level 5 evidence: Case report or expert opinion

III. STUDY DESIGN

A. Randomized control study: Two groups are created through a random process. The two groups may receive two different treatments or one group may receive a treatment and one a placebo. Patients are followed for an outcome of interest.

1. An RCT is blinded if the subjects do not know to which group they have been assigned.

2. An RCT is double-blinded if neither the subjects nor the investigators know the groupings.

3. An important point is that secondary analysis of RCT data is NOT level 1 evidence.

4. Once patients are analyzed outside of the group to which they were randomized, the study becomes a prospective cohort study.

B. Observational studies can be divided into three types, including cohort studies, cross-sectional studies, and case–control studies.

1. Cohort study: Two groups of patients are identified and followed over time

a. Prospective cohort study

i. Follows two groups of patients for a distinct time period to observe for outcome(s) of interest.

ii. An intervention may be provided.

iii. Patients are identified prior to the intervention.

b. Retrospective cohort study

i. Identifies groups of patients who have already had an intervention and completed a treatment course.

ii. Groups are stratified by a variable of interest (e.g., an intervention) and examined.

iii. Many “chart-review” studies that compare two groups are retrospective cohort studies.

______________
*Denotes common in-service examination topics

2. Cross-sectional study

a. A large group is examined at a single point in time to look at prevalence of risk factors and outcomes.

b. Typically provides data on the entire population, as opposed to a highly selected group (e.g., in a case–control study).

3. Case–control study

a. Patients with an outcome of interest (“cases”) are identified.

b. A similar group of patients (“controls”) without the outcome are also identified.

c. The frequencies of risk factors in each group are then identified and examined.

d. Particularly useful for rare disorders.

C. Crossover study

1. The same patient receives different interventions at the same time (e.g., different peels on the left and right sides of the face) or at different times (a patient’s great toe wound is treated with one dressing regimen last year and a different dressing regimen now).

2. The effect of interventions is compared.

3. Essentially eliminates confounding because patients are compared to themselves and not to other patients.

D. Case series

1. Discusses a single group of patients, all of whom received the same treatment.

2. No comparisons between groups can be made because there is only one group.

E. Case report: The care of a single patient is discussed

F. Systematic review

1. Identifies all published literature relevant to a specific clinical question using highly standardized (and repeatable) search criteria.

2. Studies are evaluated using predefined standards for research quality.

3. All studies that meet inclusion criteria are considered and an evidence-based standard is generated.

4. The goal is to provide an exhaustive, evidence-based review of a specific clinical question.

5. The transparent, highly standardized approach seeks to minimize bias.

G. Meta-analysis

1. A subtype of systematic review that pools data from similar studies to augment sample size.

2. Meta-analysis helps to overcome problems with reduced statistical power that result from sample-size constraints, particularly for rare events.

3. The goal is to provide a true measure of effect size based on pooled data.

IV. BIAS

A. Any tendency that prevents unprejudiced consideration of a question.

B. Bias occurs when a systematic error is introduced into a research paradigm that selects or encourages one outcome or answer over another.

C. Bias can occur at any stage of a research project.

D. Pretrial bias

1. Flawed study design.

2. Errors in patient recruitment, such as selection bias or channeling bias

a. Selection bias: The criteria used to place patients into a cohort are inherently different.

b. Channeling bias: Patient prognostic factors or degree of illness dictates the cohort into which they are placed.

E. Bias during the trial

1. Interviewer bias: A systematic difference exists in how information is solicited, recorded, or interpreted.

2. Recall bias: The outcome of treatment (good or bad) affects the subject’s recollection of events before or during the treatment process.

3. Performance bias: Due to the lack of standardization in how the intervention is performed at different sites or by different surgeons.

F. Bias after a trial: Citation bias: Researchers or study sponsors may be unwilling to publish results that show no effect or a harmful effect of an intervention. Thus, more manuscripts demonstrate positive results than negative results.

V. CONFOUNDING

A. Occurs when an observed association is attributed to three factors

1. The exposure or intervention.

2. The outcome of interest.

3. A third variable that is associated with both the exposure/intervention and the outcome of interest.

B. Inappropriate or incomplete analysis will demonstrate an association that is incorrect. Example: A study shows that advanced cancer stage is associated with wound healing problems but did not consider receipt of radiation in their analysis. Patients with more advanced cancer are more likely to get postoperative radiation therapy. Thus, cancer stage is confounded by radiation. The true association is that patients who receive radiation are more likely to have wound healing problems.

C. Researchers can control for known confounders by pretrial study design and matching on key risk factors.

1. Regression techniques can control for known confounders during the analysis.

2. Unknown confounders can only be controlled by randomization.

VI. CRITICAL APPRAISAL OF A MANUSCRIPT

A. Should be done in a step-by-step fashion to address several key questions, as below

B. These can easily be answered by filling out a companion worksheet as you read the article

1. Why did the authors choose to perform this study?

2. What is the author’s hypothesis or hypotheses?

3. What was the study design, and what type of evidence does this study represent?

4. What were the key independent variables? What was the dependent or outcome variable?

5. How does the study’s design minimize or eliminate bias? What potential sources of bias are present and how might these affect the author’s conclusions?

6. What types of statistical analyses were performed?

7. Was a sample size calculation performed prior to the study? If so, was the study adequately powered?

8. Was the appropriate control utilized?

9. Were the appropriate experiments utilized and the methods adequately described?

10. Were the appropriate controls used?

11. Were the appropriate assays performed?

12. Does quality of the figures match the supposition of the conclusions?

13. Were the appropriate references cited?

14. What were the main findings or results?

15. What conclusions do the authors make, and are these conclusions supported by their data?

16. Does the discussion adequately discuss the clinical potential of the study?

17. How will this article change your practice?

PEARLS

1. Review of journal club articles should be performed using a systematic approach.

2. A companion worksheet is helpful to summarize your thoughts prior to journal club.

QUESTIONS YOU WILL BE ASKED

1. What factors may introduce bias into a study?

Bias is a multifactorial process that can occur in the planning, execution, analysis, or publication phases of research. See above.

2. What are the differences between observational studies and randomized controlled trials?

Observational studies follow patients without intervening. Randomized controlled trials perform a specific intervention and then follow patients to evaluate outcomes.

3. What are the Levels of Evidence?

a. Level 1 evidence: Randomized control trial or a meta-analysis of RCTs.

b. Level 2 evidence: Prospective cohort study.

c. Level 3 evidence: Retrospective cohort study.

d. Level 4 evidence: Case series.

e. Level 5 evidence: Case report or expert opinion.