Evaluating new diagnostic tests in the field – are we doing it right?

Recent systematic reviews of diagnostic studies have reported frequent methodological shortcomings. These include failure to report participants’ age and gender, state of disease, and study eligibility criteria, as well as use of inappropriate study design, and lack of appropriate blinding. Studies with adequate design should clearly define the reference standard for the study including definitions of disease and non-disease states. The reference standard should be well-accepted and reproducible, have cut-points that are defined before-hand, and should not incorporate the results of the test being evaluated. The study should assess the tests in patients who have not yet received treatment unless it is certain that treatment does not affect test results. Those performoing the test should be blinded to the clinical diagnoses and while those making the clinical diagnoses should be blinded to test results. The analysis should use a 2x2 table that can evaluate sensitivity/specificity, predictive value and likelihood ratio.

Evaluation of a new diagnostic tool should follow a standardized approach with four phases, similar to the evaluation for testing new medications to provide the evidence needed for FDA licensing. Phase one studies should provide information on technical aspects of the test, including its reproducibility, as well as the expected range of values when the test is performed in normal healthy subjects. Participants should be healthy volunteers although gender, age and ethnic origin should be considered when selecting these volunteers. Phase two studies are of two types: Phase 2a is a case control study in which the test is performed in patients with disease and healthy controls. Phase 2B is a cohort study; consecutive patients who are being investigated for the disease or condition which the new test is intended to diagnose are enrolled. Case control studies are essentially proof of concept studies, as they will over-estimate test accuracy - results in practice are usually worse than in case-control studies due to spectrum bias. The cohort design should be prospective and the population enrolled should be suspected of having the disease and should be a consecutive series, or randomly selected patients. No exclusions, due to a failed test or unclear diagnosis, should be permitted. Phase three studies are randomized trials. Patients suspected of having the disease, are randomized to receive the test, or not. Unfortunately these studies pose several ethical issues, and are not well accepted by provider and patients - often resulting in a highly selected study sample. Phase four studies should provide information from a societal perspective—cost effectiveness, disease modeling, and data base studies.