Introduction
Prenatal screening for fetal trisomy has been available since the 1970s when risk assessment was based solely on maternal age [1]. Subsequent advances included measurement of maternal serum alpha-fetoprotein and additional biochemical and ultrasonographic markers but experienced a significant transformation in the last decade with the molecular genetic testing of cell-free DNA in maternal plasma. Screening for trisomy 21 using cell-free DNA, also known as noninvasive prenatal testing or noninvasive prenatal screening, offers higher sensitivity and a nearly 100 times lower false positive rate than standard biochemical and ultrasonographic screening [2]. Despite the decades of experience with prenatal screening, there is concern that the high performance of this newer screening test, with sensitivities of well over 90% for trisomy 21, 18, and 13 and specificities of greater than 99.9% [3], contributes to a misperception that cell-free DNA testing is diagnostic [4, 5]. This has led to calls for healthcare providers to understand and communicate positive predictive value (PPV) [6-8].
PPV is a clinically relevant statistical measure that indicates how likely individuals that screen positive are to be affected by the condition assessed. For a given test in a given population, the PPV is the proportion of all positive test results that represent true positives. Unlike sensitivity and specificity, PPV is not an intrinsic characteristic of a test. It is a function of the prevalence of the condition in the population studied. Lower disease prevalence results in lower PPV because the proportion of true positives decreases relative to the false positives. Therefore, PPV is effectively a reflection of the pretest probability of the condition as much as it is the performance of the test itself. A PPV established from a clinical study is only applicable in another setting if the population or individual tested has a comparable pretest risk to the study population. This creates challenges for counseling based on PPV because each specific patient will have their own unique clinical factors that are different from those of study populations.
Online PPV calculators attempt to account for selected patient-specific risk factors using modeling [9, 10]. An assigned patient-specific prior risk, most commonly based on maternal age, is input along with test sensitivity and specificity. The output is often referred to as a patient-specific PPV; however, it has been suggested this number is more appropriately referred to as an “estimation of posttest risk,” because PPV is actually a population statistic [11].
There have been recommendations that laboratories report PPV in addition to sensitivity and specificity [7, 8, 12]. The purpose of this study was to use real-world data in a clinical laboratory setting to explore different approaches to communication around PPV.
Materials and Methods
The data for this study were derived from samples submitted to the Ariosa Diagnostics Inc. Clinical Laboratory in San Jose, CA, USA, between April 2015 and September 2016 for the Harmony® prenatal test. The Harmony test is a targeted cell-free DNA test that uses DANSR (Digital Analysis of Selected Regions) assays and a fetal-fraction optimized algorithm to determine the probability of trisomy 21, trisomy 18, and trisomy 13 [13-15].
As part of a Health Insurance Portability and Accountability Act of 1996-compliant quality monitoring program, a laboratory registry coordinator routinely contacts ordering providers by telephone to obtain the results of diagnostic testing or birth outcome after test results indicating a high probability for trisomy. This is done during the pregnancy and again after the expected delivery date if prenatal diagnosis was not performed. Only centers in the United States that have agreed to participate are contacted. Information is provided on a voluntary basis. Concordance between test results and fetal chromosomes is determined from verbal or written report of prenatal diagnostic testing (i.e., traditional chromosome analysis, microarray, or other genetic testing on samples obtained from amniocentesis, chorionic villous sampling, or products of conception) or neonatal examination.
For high-probability results for trisomy 21, 18, and 13, maternal age, gestational age, indication, and concordance were extracted from the laboratory database. All information was de-identified prior to analysis. The study was reviewed and determined to be exempt from institutional review board oversight by the Western Institutional Review Board.
Concordant results were considered true positives (TP) and discordant results as false positives (FP). Mosaic results were considered neither concordant nor discordant and were excluded from the analysis. Observed PPV was calculated as TP/(TP + FP). The study population was divided into 4 maternal age-groups: under 30 years, 30–34 years, 35–39 years, and over 39 years of age.
Modeled PPV was calculated using an online tool [10]. Input variables for the calculator include chromosome condition, maternal age, sensitivity, and specificity. Calculations were performed for each patient in the study. For each maternal age-group, the PPV values from all the patients in that group were averaged to determine the expected PPV. The tool’s default values for test sensitivity and specificity were used. At ages where the calculator did not generate a PPV due to insufficient data to determine prevalence, the closest maternal age with data was used.
Risk status was classified as high risk or low risk based on the International Classification of Diseases (ICD) codes provided to the laboratory (details provided in supplementary material; see www.karger.com/doi/10.1159/000512501 for all online suppl material). Two revisions were used during the study period so revision 9 (ICD-9) codes were converted to revision 10 (ICD-10) codes for analysis.
Results
Ordering providers were contacted for 2,143 test results with an increased probability for fetal trisomy 21, 18, or 13 (Table 1). Clinical information was obtained for 1,411 cases (65.8%). Of these, diagnostic information from a prenatal procedure or newborn evaluation was available for 715 and other information, such as abnormal ultrasound findings or pregnancy outcome (e.g., fetal demise and stillbirth), many of which were suggestive but not diagnostic for aneuploidy, was available for 696. 732 cases had no outcome information because the provider did not respond, declined to provide information, or no longer had the patient under their care.
The proportion of test results with diagnostic information was consistent across the population when grouped into 4 subgroups by maternal age. These proportions were 29.9, 34.9, 34.7, and 32.8% for the <30, 30–34, 35–39, and >39 years subgroups, respectively.
Only cases with diagnostic information were considered for the analysis. These included 504 trisomy 21, 131 trisomy 18, and 80 trisomy 13 results. After the exclusion of 1 case of mosaic trisomy 21, the final cohort consisted of 714 cases.
In the final cohort, median maternal age was 36.0 years and median gestational age was 12.7 weeks. Table 2 shows these characteristics with the patients grouped in 4 maternal age categories.
In the final cohort, the observed PPV for all 3 trisomies combined was 90.5%. Individually, the PPVs were highest for trisomy 21 (98.2%), followed by trisomy 18 (90.1%) and trisomy 13 (42.5%). PPV was generally higher in the older maternal age-groups. The values for the 3 trisomies combined were 79.2, 86.8, 94.1, and 96.0% in the <30, 30–34, 35–39, and >39 years subgroups, respectively.). Figure 1 shows the observed PPV for each trisomy in the 4 age-groups. The ranges were 96.3–99.5% for trisomy 21, 77.4–97.0% for trisomy 18, and 30.4–80.0% for trisomy 13.
Fig. 1.
The bar chart depicts the number of concordant and discordant results observed in the clinical laboratory population with the demarcation between the stacked bars representing observed PPV. The line graph plots the PPV predicted by the PPV calculator. PPV, positive predictive value.
When PPV was calculated with the online tool, the values generated for each group were lower than those observed in the clinical population. The modeled ranges were 53–95% for trisomy 21, 16–78% for trisomy 18, and 6–61% for trisomy 13 (shown in Fig. 1).
Of the 271 women under 35 years of age, ICD codes were available for 260 cases. 177 of these samples were classified as low-risk indications and 83 as high-risk indications. The PPV observed for all 3 trisomies combined was 90.4% in the high-risk group compared to 79.7% in the low-risk group; this is significantly higher at p < 0.05 (z = 2.1459, p = 0.01578; 2-proportion Z-test).
Discussion
This study used the concordance between laboratory test results and clinical outcome to investigate the PPV of a cell-free DNA test for trisomy 21, 18, and 13 in a clinical laboratory population and subsets of this population. PPV increased with increasing maternal age. Across all age-groups, the PPV was highest for trisomy 21, followed by trisomy 18 and then trisomy 13. These patterns are consistent with the pattern predicted by using the PPV calculator. The higher prevalence of trisomy associated with maternal age and the higher prevalence of trisomy 21 compared to trisomy 18 and 13 would be expected to result in a higher PPV.
The PPV for each trisomy in the subsets of this study population was consistently higher than the PPV predicted by the PPV calculator. One likely explanation is that the clinical study population includes pregnancies with fetal ultrasound findings or other high-risk screening results in which women chose a second level screen using cfDNA before diagnostic testing. This would lead to a higher prevalence of fetal trisomy than would be predicted by maternal age alone and therefore a higher overall PPV across all age-groups. Another possible explanation is the default values for specificity that are used by the calculator. Specificity is a strong influencer of PPV; at the same prevalence, a higher specificity will lead to a higher PPV. The specificity values (99.91% for trisomy 21, 99.87% for trisomy 18, and 99.87% for trisomy 13) used by the calculator are derived from a meta-analysis that includes data for multiple testing platforms [3]; however, the specificity observed in clinical validation studies for the specific cell-free DNA test used in this study is higher: 99.96, 99.98, and 99.98% for trisomy 21, trisomy 18, and trisomy 13, respectively [16].
Other laboratories performing cell-free DNA testing have reported clinical experience using retrospectively obtained pregnancy outcome information in similarly large series [17-19]. The observations in the current study are consistent with these previous reports, showing PPV to be higher for trisomy 21 than trisomies 18 and 13. The specific PPV values generated from these studies vary greatly and are not directly comparable because they use different referral populations and different procedures for data collection. One study also examined PPV in different subgroups of the clinical population. DiNonno et al. [19] compared PPV in women <35 to women ≥35 years and described the PPV in younger women as only modestly lower. In that report, the combined PPV for trisomy 21, 18, and 13 was 87.0% in the <35 age-group and 93.5% in the ≥35 age-group. The corresponding values in the current study were 83.4 and 94.8%. Although PPV values cannot be compared between the studies (again, because of their different populations), the higher than expected PPVs in the younger age-groups are likely due, at least in part, to the presence of higher-risk pregnancies in both populations. Whereas DiNonno et al. did not consider other clinical risk factors in their analysis, this study further subdivided the <35 years age-group by diagnosis code and demonstrated a significantly higher PPV in the higher-risk subgroup.
The primary limitation of this study is that it relies on retrospective follow-up and thus does not have complete outcome information. Other laboratories studying clinical experience are similarly challenged and report similar proportions of cases with diagnostic outcome obtained, generally between 30 and 40% [17-19]. In each of these studies, the calculation of PPV is based on the concordance observed in cases with outcome information and assumes that the ratio of concordant and discordant cases reported to the laboratory accurately reflects the entire cohort. Here, only cases with diagnostic outcome information (from prenatal diagnosis or neonatal evaluation) were considered in the PPV calculations. It is possible that concordant cases were overrepresented (and PPV overestimated) because patients with affected pregnancies pursued diagnostic testing after other high-risk screening tests or abnormal ultrasound findings whereas patients with unaffected pregnancies declined diagnostic testing after a normal fetal ultrasound. Conversely, it is possible that concordant cases were underrepresented because diagnostic testing was not performed or not reported due to other findings consistent with fetal aneuploidy. It is notable that other pregnancy outcome information was received for an additional 32% of cases. In many instances, these were reports of abnormal ultrasound findings, fetal loss, or stillbirth, all of which are highly suggestive of a chromosome abnormality and would have increased the PPV had they been included (data not shown).
This study has looked at PPV from 2 different perspectives: (1) observed from clinical laboratory follow-up information and (2) modeled with the use of a PPV calculator. Clear differences in the values obtained were found, raising questions about how PPV should be communicated to patients by clinical care providers and laboratories. Reporting a PPV from a population study, whether it be from a large prospective study such as the NEXT study [2] or a clinical experience study such as this one, may not be applicable to any given patient. The use of a PPV calculator can provide more patient-specific information but is highly dependent on the input values. Using the default values (specifically prevalence based on maternal age) and not considering other components of risk may lead to an underestimation of PPV. The PPV calculator used in this study allows for manual input of estimated prevalence and test sensitivity and specificity; however, this requires clinicians to account for the many clinical factors that affect a priori risk and have accurate information about specificity for any testing platforms used. It is less useful in the hands of a laboratory because the diagnosis codes provided with samples are only very general, leaving maternal age as the only available patient-specific parameter.
Ultimately, PPV is an important statistical calculation that may provide a foundation for discussion and setting expectations regarding the significance of screening results. However, because it is a population statistic rather than a tool for the interpretation of individual patient results, specific PPV values should be approached with caution and an understanding of their derivation. Investigation into healthcare providers’ current understanding of PPV and its use in patient counseling for cell-free DNA screening would be valuable to guide future educational efforts.
Acknowledgements
The authors would like to thank Angela Dunhour and Yunwei Wang for their roles in the acquisition and processing of data. We are also grateful to the patients and their providers for being willing to provide the outcome information that contributes to laboratory quality monitoring and makes this study possible.
Statement of Ethics
This research (RSS-NIPT-496) was conducted ethically in accordance with the World Medical Association Declaration of Helsinki. The study has been granted an exemption from requiring ethics approval by the Western Institutional Review Board (WIRB 1-1176803-1). Informed consent was not sought for this study as it was limited to the use of de-identified, existing data.
Conflict of Interest Statement
All authors are employees of Roche Sequencing Solutions Inc.
Funding Sources
Roche Sequencing Solutions Inc. funded this work.
Author Contributions
K.W. contributed to the design of the work, conducted the analysis and interpretation of data, and drafted the manuscript. A.B. contributed to the acquisition of data and revision of the work for important intellectual content. M.S. contributed to the conception and design of the work and revised the work for important intellectual content. All authors gave final approval of the version for publication and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Get Permission
PubMed ID
