Receiver Operating Characteristic (ROC) Curves and Meta-Analyses

Study Notes

ROC curves are used to find the optimal cut-off point in a continuously distributed measurement that best predicts whether a condition is present or absent.
ROC curves compare the sensitivity and specificity for all possible cut-offs.
To construct a ROC curve, sensitivities and specificities for different values of a continuous test measure are tabulated and then plotted.
The true positive rate (sensitivity) is plotted on the y-axis, and the false positive rate (1-specificity) is plotted on the x-axis.
Each point on the ROC curve represents a sensitivity/specificity pair corresponding to a particular possible cut-off point.
A perfect test has a ROC curve that passes through the upper left corner (100% sensitivity, 100% specificity).
A diagnostic test can have two cut-offs: one to rule out the disease and another to rule in disease.

The area under the ROC curve (AUC) is a global measure of the ability of a test to discriminate between disease presence and absence.
An AUC of 0.5 represents a test with no discriminative ability, while an AUC of 1.0 represents a test with perfect discrimination.
The AUC can be used to compare the accuracy of two or more tests.

Three different cut-off points can be used for a diagnostic test: a general optimal test, a screening test, and a diagnostic test.
The cut-off point for a screening test is chosen to maximize sensitivity, while the cut-off point for a diagnostic test is chosen to maximize specificity.
The optimal cut-off point, or Youden Index, is the point on the curve that is closest to the top of the left-hand y-axis.

A systematic review is the synthesis of multiple studies addressing a similar research question.
A meta-analysis is a statistical method to aggregate the results of studies to determine the strength of a particular association.

A forest plot is a graphical representation of the results of multiple studies asking the same question.
The main results of a meta-analysis are presented graphically as a forest plot.
The effect in each study can be summarized in different ways, such as relative risk (RR), odds ratios (OR), or risk differences (RD).
The pooled estimate or summary estimate is the effect in each study summarized in a single statistic.

Reading a meta-analysis can be broken down into 4 basic steps: what is the summary measure, what does the forest plot show, what does the pooled effect mean, and was it valid to combine studies?
The axes of a forest plot show the effect size (e.g., RR, OR, RD) on the x-axis and the study identifier on the y-axis.
The horizontal line and whether it crosses the "line of null effect" is important to note for each study.
The line of null effect represents the null hypothesis, and if the horizontal line crosses it, the study result is not statistically significant.

Heterogeneity refers to the variation in the effects between studies.
Publication bias occurs when studies with negative or null findings are less likely to be published.
A funnel plot can be used to assess publication bias.

Fixed-effect model: considers the variability between studies as exclusively due to random variation.
Random-effects model: assumes a different underlying effect for each study and takes this into consideration as an additional source of variation.
The choice of model depends on the research question and the characteristics of the studies being combined.