Diagnostic Testing Principles: Sensitivity & Specificity

Week 3 Learning Objectives

• Outline the main principles of diagnostic testing and how these can be applied in different contexts
• Explain and compute sensitivity and specificity
• Justify choices about sensitivity and specificity in different situations
• Explain, compute, and interpret predictive values
• Explain, compute, and interpret Likelihood Ratios
• Describe the method for establishing ideal cut-off values in diagnostic testing
• Explain and assess the use of parallel and series testing

Ruling In and Ruling Out: Key Concepts

• Dichotomous outcome: an outcome with two possible results
• Non-dichotomous (continuous) measurement: a measurement that doesn't have set groups
• 2 x 2 table: can be used to analyze diagnostic tests

Making a Diagnosis

• Key decision support tool
• Legal act of veterinary science
• Methods of diagnosing infectious disease includes:
  • Isolation of agent
  • Identification of agent's genes (molecular epidemiology)
  • Clinical signs
  • Pathognomonic (characteristic) changes
  • Biochemical changes
  • Demonstration of an immune response: detection of antigens and antibodies (serological epidemiology)
  • Clinical history
  • Pathognomonic changes
  • Demonstration of an immune response: detection of antibodies

Reference Intervals

• A range of values within which healthy animals mostly fall
• Established with at least 120 reference individuals, using the nonparametric ranking method
• Usually calculated to encompass 95% of observations from healthy animals
• Averages can be shown as mean +/- 2 standard deviations
• As a result, 5% of healthy individuals will have observed values outside reference limits.

Categorical Data

• Data of set groups can be split into:
  • Dichotomous (YES/NO - eg male or female)
  • Nominal (categories with no order - e.g. breed)
  • Ordinal (categories with some order - e.g. body condition score)

Nominal Data

• Data of un-set groupings can be split into:
  • Discrete (only particular integer values - e.g. litter size)
  • Continuous (all values theoretically possible)

Tests Based On Continuous Data

• Tests that rely on a binary response necessitate a cut-off point. Rapid antigen tests exemplify this
• Examples of use of tests are Giardia, Parvo, Heartworm, Lepto and Pancreatic health
• Differing discriminatory potentials of a biological marker are possible
  • No discrimination possible
  • Imperfect discrimination
  • Perfect discrimination

ELISA Tests

• ELISA tests use a cut off to evaluate if the test is positive or negative

Cut Off Decision Making

• Gauassian distribution method: 95% of values within 2 SD of mean are considered normal
• Predictive Value method
• Receiver Operator Characteristic Curve (ROC): Map the test's ability to correctly identify positive and negative cases and select the best compromise cut off

Diagnostic Testing Truth Layout

• Testing can result in:
  • True Positive [a]: animal has the condition and tests positive for it
  • False Positive [b]: animal does not have the condition but tests positive for it.
  • False Negative [c]: animal has the condition but tests negative for it
  • True Negative [d]: animal does not have the condition and tests negative for it.

Errors

• False Positive: The test result for an individual is positive, but the disease/ condition is not present
• False Negative: The test result for an individual is negative but the disease/condition is present.

Gold Standard Testing

• Gold standard testing is:
  • The best available or benchmark diagnostics
  • Often used for comparison purposes
  • Often not 100% accurate
  • Sometimes terminal

Testing The Tests

• Comparing with gold standard tests is possible using:
  • Sensitivity (Se):
    • Proportion of animals with the disease that test positive
    • Ability of the test to correctly identify diseased animals
    • Indicates of how many false negative animals can be expected
    • Se = a / a+c
  • Specificity (Sp):
    • Proportion of animals without the disease that test negative
    • Ability of the test to correctly identify non-diseased animals
    • Indicates how many false positives animals can be expected
    • Sp= d / b+d

Prevalence

• Before moving forward, it is important to consider prevalence
  • True prevalence:
    • The prevalence of disease in a herd
  • Apparent prevalence:
    • The proportion of animals in the population that test positive
    • True prevalence = apparent prevalence if we have 100% sensitivity
    • True prevalence doesn't = apparent prevalence

Positive and Negative Predictive Value

• It is important to consider:
  • Given that an animal has tested positive to a particular condition, what is the probability of the animal really having the condition?
  • Positive Predictive Value (PPV) answers that question
  • Given that an animal has tested negative, what is the probability that the animal is really free from the condition?
  • Negative Predictive Value (NPV) answers that question
  • PPV = a/a+b
  • NPV = d/c+d

Likelihood Ratios

• These can helps describe how likely it is that the test result could have been produced by a diseased rather than a non-diseased animal, independent of prevalence
  • Positive LR:
    • A measure of how much more likely an animal is to test positive if they have the disease, when compared with an animal without the disease
    • Ratio of proportion of affected animals that test positive TO the proportion of healthy animals that test positive
    • = a/a+c / b/b+d
    • Usually greater than 1
    • LR+ = Se/ 1-Sp
  • Negative LR:
    • A measure of how much more likely an animal is to test negative if they are disease free when compare with a diseased animal
    • Ratio of proportion of affected animals that test negative TO the proportion of healthy animals that test negative
    • c/a+c / d/b+d
    • Usually smaller than 1
    • LR- = 1-Se /Sp

Diagnostic Test 2x2 Table Definitions

• Accuracy can be estimated as TP + TN / Total
• PPV can be estimated as TP / TP + FP
• NPV can be estimated as TN / TN + FN
• DSe (%) and DSp (%) where: DSe = TP / (TP + FN), Dsp = TN / (TN + FP)
• Prevalence can be thought of as true prevalence (diseased / total) and apparent prevalence (test positive / total)

Individual Versus Herd Testing

• Up until now, we have been considering individual level testing, but what happens when we test at a herd level?
• SeH: Probability that a herd test positive result, if prevalence is above a specified threshold
• SpH: Probability that am uninfected herd will test negative
• Depends on:
  • Sp and Se at individual level
  • Prevalence
  • Number of animals tested
  • Number of reactors designated to give pos or neg result

How Do We Choose Which Test To Use?

• We need to consider the following:
  • Objectives of testing and the important of ruling in/out the presence of disease
  • Consequences of false positive and negatives
  • Feasibility
  • Costs, technical skills, duration

Selection Criteria For Individual Tests

• For demonstrating freedom in a defined population:
  • Demonstration of freedom
  • Exporter-centric view
  • Potential for False Positive implications
  • Aim is to Minimize positive result being false (i.e. max PPV)
  • Use test with highest DSp
• Certifying freedom for transboundary movements:
  • Certifying freedom for transboundary movements
  • Importer-centric view
  • Potential for False Negative implications
  • Aim is to Minimize negative result being false (i.e. max NPV)
  • Test with highest DSe

Multiple / Different Tests

• Mutiple (different) tests can be used in animals
  • Series testing:
    • Only animals that test positive to both tests are considered positive
  • Parallel testing:
    • Animals that test positive to at least one test are considered positive
• These can be achieved when
  • Tests are different, independent (different biological markers)
  • Same disease, same animal same time

Improving Decision Making

• Series testing:
  • Tests must all be positive
  • Increases Sp and PPV
  • Decreases Se and NPV
  • Animal asked to 'prove' animal has the condition
• Parallel testing:
  • Useful when there is a penalty for false negatives (missing diseased)
  • At least one test must be positive
  • Increases Se and NPV
  • Decreases Sp and PPV
  • Animal is being asked to 'prove' it is healthy

Characteristics of Multiple Test Strategies:

• Consider:
  • Effect of strategy
  • Greatest predictive values
  • Purpose
  • Application and setting
  • Comments

Why Are We Testing?

• There are two main testing strategies:
  • Screening versus diagnosis
  • Tests for screening
    • Healthy animals
    • High Se and High NPV (minimize false negatives)
    • Quick, low cost - Capable of testing large numbers
  • Tests for Diagnosis
    • Sick individuals, need to rule disease in or out
    • High Sp, high PPV
    • Cost not as important - small numbers tested

Control And Eradication Programs

• Control achieves reduction of morbidity and mortality from disease, but is ongoing / treating / preventing over time, and uses a broad range of measures to address occurrences of disease
• Eradication achieves Extinction of an infectious agent / reduction of disease prevalence below level at which transmission can occur but is Time Limited, and more regional

Eradication Implementation

• Early stages of disease control/eradication program is characterised by:
  • High numbers of infected animals
  • Lower test Specificity
• Later Stages of disease contro/eradication programs are characteristed by:
  • Falling rates of infected animals
  • Where testing specificity is the more important factor