Category: Statistics

We want accurate tests, don’t we?  By that, don’t we mean that we want precise test results?  Well, not exactly.  Before we leave the subject of laboratory statistics, there are a few more words we need to learn.

Accuracy is the ability of a test to aim for the target.  Visually, accuracy looks like this:

Even though none of the shots hits the bullseye, we can tell where the shooter is aiming.

Precision is the ability of a test to get the same answer repeatedly, illustrated like this:

Even though none of the shots hits the bullseye, all of the shots go to the same spot.

Putting these terms together, we can describe different aspects test performance.  Poor accuracy and poor precision look like this:

The shots are not centered on the bullseye, and they do not hit the same spot.  Together, favorable accuracy and precision can be illustrated like this:

All the shots hit the bullseye, repeatedly.  This is what an accurate and precise test looks like, too.  Accuracy and precision are both desirable but different aspects of test systems.

But are accuracy, precision, sensitivity and specificity enough to interpret tests?  Consider the case where these aspects of a test system are optimized.  In other words, the test is highly accurate, precise, sensitive and specific.  But what if the condition being tested for is not prevalent in the tested population.  Prevalence is the percentage of people in a given population who have the condition of a positive test.  Consider a test is 99.99% specific.  That means that for every 10,000 results, there is only 1 false positive.  But let’s say that the condition exists in the population at a rate of only 1 per 10,000 individuals.  In other words, out of 10,000 results, there is only one true positive result.  But we’ve already said that out of 10,000 tests, there will be one false positive result.  Therefore, if we get a positive result, there a 50% chance it’s the false positive, not the true positive.  That is what is meant by the term positive predictive value.   

Both positive and negative predictive values can be measured.  These are expressed mathematically as PPV = TP / (TP + FP) and NPV = TN / (TN + FN), respectively.  It’s okay to skip the math; just remember that predictive values are dependent on the prevalence of the condition; sensitivity and specificity are not.

Predictive value is important for test interpretation but should not be used for test selection.  Instead, sensitivity, specificity, accuracy and precision should be used to guide appropriate test selection.  These are the terms by which test systems are judged, and the key aspects of test performance analyzed by the FDA before tests are approved for use in the United States.

We want tests that are highly sensitive and highly specific for the condition being tested, but that is not always possible.  Often, we must sacrifice one for the other.  Simply stated, negative results can be trusted when there is high sensitivity, and positive results can be trusted when there is high specificity.  So, we have to ask: is it better not to miss negatives or positives?  

There is not usually a neat separation between healthy patients and patients with disease.  Instead, patient populations exist in overlapping distributions, which can be illustrated as follows:

The vertical blue line represents the cutoff between positive and negative test results.  In this illustration, the cutoff is placed in a compromise position between the two populations, creating a group of false negatives (FN) and false positives (FP).  

If a test is highly sensitive, the cutoff is shifted to the left, eliminating false negative results, but increasing the number of false positive results.

If a test is highly specific, the cutoff is shifted to the right, eliminating false positive results, but increasing the number of false negatives.  As we have discussed previously, this is the situation with antigen tests for SARS-CoV-2. 

When screening large populations for disease, it is important not to miss possible positives, so we choose a test that highly sensitive.  We do not want any false negatives.  False positives can be sorted out later; this is just a screen after all.  On the other hand, it is important that confirmatory tests have high specificity.  When we are confirming disease in a population selected by a screen, we want to eliminate false positives.

If the goal of testing for SARS-CoV-2 is to avoid false negative results, favor sensitivity over specificity.  But this trade-off is not necessary with all test systems.  PCR tests increase sensitivity by amplification and increase specificity with detection probes unique to the virus.  The result is a separation between populations, increasing specificity and sensitivity at the same time:

Are sensitivity and specificity the only considerations when evaluating a test?  No, it is more complicated, but I am sure you guessed that.  We will talk about other measures of test systems and the results they produce next time.