Development and Internal Validation of a Practical Model to Identify Observe Patients of the European Society of Cardiology 0/1-h Algorithm at Low Risk of a Coronary Diagnosis

Background: Patients with suspected non-ST-elevation acute coronary syndrome (NSTE-ACS) assigned to the “observe” zone of the European Society of Cardiology (ESC) 0/1-h algorithm form a heterogeneous group known to have an unfavourable prognosis. We aim to elucidate the clinical characteristics and management of these patients and generate a model that is predictive of a coronary diagnosis at index visit to the emergency department (ED). Methods: A retrospective observational cohort study, including adult patients presenting to the ED with suspected NSTE-ACS assigned to the “observe” zone of the ESC 0/1-h algorithm. Multivariable logistic regression analysis was performed for the prediction of a coronary diagnosis. Internal validation was performed using bootstrap resampling. Results: A total of 750 patients were included; mean age 66 ± 13 years, 35% women, 50% with prior history of coronary artery disease (CAD). In 372 (50%) patients a diagnosis was established within 30 days of index presentation, of whom 169 (45%) patients had a coronary-related event. Multivariable logistic regression analysis generated a 12-point risk score incorporating 5 variables for the prediction of such event, including type of angina, chest pain occurring during inspiration, prior history of CAD, ST-segment deviation on electrocardiogram, and estimated glomerular filtration rate <60. The final model had an optimism-corrected c-statistic of 0.78 (95% confidence interval [CI]: 0.74–0.82). A score <6 ruled out a coronary event in 276 (37%) patients, with a sensitivity and negative predictive value of 90% (95% CI: 84–94) and 94% (91–96), respectively. Conclusion: A score <6 identifies patients at low risk of a coronary diagnosis and can guide clinical decision-making in choosing the appropriate diagnostic test.


Introduction
The introduction of high-sensitivity cardiac troponins (hs-cTn) has led to accelerated diagnostic protocols for the assessment of suspected non-ST-elevation acute coronary syndrome (NSTE-ACS) [1][2][3]. In the recent European Society of Cardiology (ESC) guideline on the management of patients with a suspected ACS without persistent ST-segment elevation, the use of a 0/1-h algorithm in This  centres with hs-cTn assays is recommended [4]. The ESC 0/1-h algorithm has two separate thresholds for "ruleout" and "rule-in" to optimize diagnostic accuracy and efficacy on both sides of the spectrum.
However, a substantial number of patients do not qualify for either category after repeated hs-cTn testing and are assigned to the "observe" zone. These patients are known to have an unfavourable prognosis, with 1-year survival rates comparable to patients assigned to the "rule-in" category [5,6]. Management of these patients remains challenging, and hospital admission and additional (non)invasive testing are often necessary. A more individual approach is recommended where management decisions should be based on the degree of clinical suspicion of NSTE-ACS on a case by case basis [4]. The aim of this study was to elucidate the clinical characteristics and management of patients assigned to the "observe" zone and to generate a model that is predictive of a coronary diagnosis.

Methods
This retrospective observational cohort study included 750 consecutive adult patients with suspected NSTE-ACS and acute chest pain as principal symptom. Patients presented to the emergency department (ED) at the Erasmus MC, University Medical Centre, Rotterdam, an academic tertiary referral hospital, between February 1, 2012, and January 31, 2020. Patients were included if they were aged 18 years and older and assigned to the "observe" zone of the ESC 0/1-h algorithm based on repeated hs-cTn measurements. The start date of February 1, 2012, was chosen as this corresponded with the introduction of the fifth-generation Elecsys hs-cTnT (hs-TnT) assay (Roche Diagnostics) in our centre. Although the 0/1-h algorithm was first recommended in the 2015 ESC guideline on the management of patients with a suspected ACS without persistent ST-segment elevation, we have also included patients that presented at the ED prior to this recommendation, with repeated hs-cTn measurements which would assign them to the "observe" zone based on current guidelines. Data regarding clinical characteristics, symptoms and signs, electrocardiogram (ECG) data, laboratory results, downstream testing, patient management, and diagnosis were obtained from electronic patient records. This study was conducted according to the principles of the Declaration of Helsinki and was approved by the local institutional review board.

Diagnostic Procedure
Patients were categorized into three diagnostic groups if the diagnosis was established by treating physicians within 30 days of presentation to the ED: (1) coronary (including NSTE-ACS, chronic coronary syndromes, spontaneous coronary artery dissection, and coronary vasospasm); (2) cardiac, non-coronary (including nonischemic cardiomyopathies, valvular heart disease, brady-and tachyarrhythmias, and [peri] myocarditis); (3) non-cardiac (containing all relevant non-cardiac diagnosis such as pulmonary embolism, anaemia, infections, malignancies, and malignant hypertension).

Study Endpoint
The primary outcome was a coronary diagnosis at index visit to the ED, established by the treating physician, based on all available clinical data.

Statistical Analysis
Continuous data are presented as mean ± SD or median (interquartile ranges), and categorical data are presented as proportions (percentages). We used logistic regression analysis for the prediction of a coronary diagnosis. Initially, univariable analysis was performed to evaluate the ability of chosen variables to predict a coronary diagnosis. We examined 29 potential predictors, including sex, age, cardiovascular risk factors, medication, symptoms, ECG findings, and laboratory results. Variables with a p < 0.10 entered the multivariate stage, and a multivariable logistic regression model was constructed to predict a coronary diagnosis, using the stepwise backward selection method, with a value of p = 0.05 as a model-entry criterion. Variables were checked for multicollinearity using correlations and variance inflation factors to avoid redundancy in the prediction model. Dichotomization of the continuous variable estimated glomerular filtration rate (eGFR) was based on the clinically relevant threshold for kidney disease. The relative magnitude of the model regression coefficients from statistically significant variables in the final multivariable model was used to calculate an individual patient's risk score for the prediction of a coronary diagnosis. Each point assigned to covariates in the final multivariable model was rounded to the nearest integer to simplify the calculation of the final score in clinical practice. The model discrimination abilities were evaluated by the c-statistic of the final multivariate model. For further internal validation, 1,000 bootstrap samples were obtained, and an optimism-corrected cstatistic was calculated [7]. The optimal cut-off value for the risk score was chosen after visual inspection of the diagnostic accuracy parameters at various cut-offs, with the aim to optimize the efficacy (defined as the number of patients ruled out) of the score while maintaining high sensitivity and negative predictive value (NPV) for the prediction of a coronary diagnosis. Receiver-operating characteristic curve analysis of the computed risk score was compared with a validated risk score, namely the History, ECG, Age, Risk factors, and Troponin (HEART) score [8,9].  Patients with ECGs suggestive of an ST-elevation myocardial infarction were excluded from this analysis. ACE, angiotensin-convertingenzyme; ARB, angiotensin II receptor blocker; CAD, coronary artery disease; CCB, calcium channel blocker; DOAC, direct oral anticoagulant; ECG, electrocardiogram; eGFR, estimated glomerular filtration rate; ESC, European Society of Cardiology; GRACE, The Global Registry of Acute Coronary Events; HEART, history; EKG, age, risk factors, and troponin; PAD, peripheral arterial disease; TIA, transient ischemic stroke; TIMI, the thrombolysis in myocardial infarction; VKA, vitamin K antagonist. *Categorized according to the Diamond and Forrester classification [10]. # ST-segment deviation was defined as ≥1 mm ST-segment deviation in one or more leads [11].
Survival data of all patients were acquired using in-hospital medical records and municipal civil registry. Cumulative survival of up to 1-year follow-up was calculated and stratified by diagnostic categories, i.e., (1) coronary; (2) cardiac, non-coronary, and (3) non-cardiac, with corresponding 95% confidence intervals and plotted using Kaplan-Meier curves. The log-rank test was used to examine differences between diagnostic categories in the Kaplan-Meier analyses, correcting for multiple comparisons using the Bonferroni method.
All statistical analyses were performed using SPSS version 24.0 (IBM, Armonk, NY, USA) and R version 4.0.2 (R Project for statistical computing, Vienna, Austria). All tests were two-tailed, and a p value <0.05 was considered statistically significant.

Results
Between February 1, 2012, and January 31, 2020, 4,370 consecutive adult patients presented with suspected NSTE-ACS and acute chest pain, of whom 2,928 patients were triaged towards "rule-out" and 692 patients as "rule-in" by the ESC 0/1-h algorithm. In total, 750 suspected NSTE-ACS patients were assigned to the "observe zone" of the ESC 0/1-h algorithm and comprised the study population ( Fig. 1). Baseline characteristics of included patients are shown in Table 1. The mean age was 66 ± 13 years and the proportion of women was 35%. Half of the patients had a prior history of coronary artery disease (CAD). In 32% of the patients first blood draw was performed within 3 h of chest pain onset. The median hs-TnT value in the study cohort at baseline was 18 (14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25) and the second median hs-TnT value was also 18 (14-25). Patients with a coronary diagnosis were more often male with pre-existing CAD and more often had ischemic ECG abnormalities as compared to the rest of the cohort (Table 1).

Resource Use in the Observe Group
Non-invasive ischemia testing was performed in 54 (7%) patients within 30 days of index presentation and revealed signs of myocardial ischemia or infarction in 31 patients ( Table 2). Invasive coronary angiography (ICA) was performed in 151 (20%) patients, of whom 92 (12%) patients underwent percutaneous coronary intervention and 3 (0.4%) patients underwent coronary artery bypass grafting within 30 days of index presentation. 263 (35%) patients were admitted to the hospital after evaluation at the ED and 411 (55%) patients were sent home with an appointment at the outpatient clinic (Table 2).

Final Diagnosis
In 372 (50%) patients, a clinically relevant diagnosis related to index visit was established, based on all available clinical data (Fig. 2). The remaining 378 (50%) patients were discharged from the ED after exclusion of acute significant pathologies and had no obvious diagnosis after 30 days that explained their chest pain. Among the patients with an established diagnosis, the most common diagnosis was coronary disease, followed by "cardiac, non-coronary" diseases and finally non-cardiac diseases.

Prediction of a Coronary Diagnosis
Univariable regression analysis of the relationship with a coronary diagnosis identified an association with female sex, hypertension, prior history of CAD, aspirin use, statin use, P2Y12 inhibitor use, calcium channel blocker use (a)typical angina, chest pain occurring during inspiration, heart rate, ST-segment deviation, and pathological Q waves on ECG and eGFR <60 (Table 3) Points assigned to the five covariates in the final multivariable model are depicted in Table 4. From the final multivariable model, a 5-item risk score was generated with a score from 0 to 12. Prior history, ECG, kidney function, and type of chest pain (PEKT) score is composed of prior history of CAD (2 points), acute chest pain: atypical angina (2 points) or typical angina (4 points), chest pain occurring during inspiration (−3 points), STsegment deviation (2 points), and eGFR <60 (−1 point), with all patients receiving an additional 4 points regardless of their symptoms at presentation. The median score in the study cohort was 6 [5][6][7][8]. Sensitivity, specificity, positive predictive value, and NPV according to the PEKT score risk score are presented in Table 5. Bootstrap internal validation of the PEKT score revealed an optimismcorrected c-statistic of 0.78 (0.74-0.82). A PEKT score lower than 6 identified 276 (37%) patients at low risk of having a coronary diagnosis, with a sensitivity and NPV of 89.9 (84.4-94.0) and 93.8 (90.6-96.0), respectively. In comparison, the HEART score had a c-statistic of 0.69 (0.64-0.74) (Fig. 3).

Discussion
In this retrospective observational study, we investigated the clinical characteristics and management of suspected NSTE-ACS patients assigned to the "observe" zone of the ESC 0/1-h algorithm and we developed and internally validated the PEKT score, a simple 5-item tool to identify patients at low risk of a coronary diagnosis. We hereby report several important findings.
First, "observe" patients were generally elderly men with a prior history of CAD. Second, a clinically relevant diagnosis related to index visit at the ED was established in only 50%, with CAD being the most commonly established diagnosis. Third, cumulative 1-year survival in "observe" patients was 92% and patients with a coronary diagnosis appeared to have a relatively better survival outcome as compared to patients in other diagnostic categories.
Previous data from the Advantageous Predictors of Acute Coronary Syndromes Evaluation group portrayed Patients with ECGs suggestive of an ST-elevation myocardial infarction were excluded from this analysis. ACE, angiotensin-converting-enzyme; AF, atrial fibrillation or atrial flutter; ARB, angiotensin II receptor blocker; CAD, coronary artery disease; CCB, calcium channel blocker; DOAC, direct oral anticoagulant; ECG, electrocardiogram; eGFR, estimated glomerular filtration rate; ESC, European Society of Cardiology; OR, odds ratio; PAD, peripheral arterial disease; TIA, transient ischemic stroke; VKA, vitamin K antagonist. * Categorized according to the Diamond and Forrester classification [10]. # ST-segment deviation was defined as ≥1 mm ST-segment deviation in one or more leads [11]. similar clinical characteristics of "observe" patients as found in our study cohort, with the population generally being older men and with known pre-existing CAD [6,12]. The proportion of men in our study was even higher in patients with a coronary-related diagnosis. Interestingly, in half of the patients, NSTE-ACS or a life-threatening condition was excluded, however, an underlying cause for their symptoms was not found within 30 days of presentation at the ED. Patients without a diagnosis still have an unfavourable prognosis beyond 30 days of presentation, representing a heterogeneous group of patients with complex (non-)cardiac causes of troponin leakage. Pa-tients with a coronary diagnosis seem to have a better outcome when compared to other diagnostic categories. A possible explanation is that patients with a coronary diagnosis had a more readily treatable and simple event as compared to other diagnostic categories. In addition, we cannot rule out that patients without a coronary diagnosis may have had undiagnosed CAD, which untreated may have contributed to the survival differences.

Resource Use
Our data shows that the treating physician had a preference for ICA as compared to other imaging mo- In the final score, all patients receive an additional 4 points regardless of their symptoms at presentation. Patients with ECGs suggestive of an ST-elevation myocardial infarction were excluded from this analysis. CAD, coronary artery disease; CCB, calcium channel blocker; eGFR, estimated glomerular filtration rate; OR, odds ratio. * Categorized according to the Diamond and Forrester classification [10]. # ST-segment deviation was defined as ≥1 mm ST-segment deviation in one or more leads [11]. dalities, such as coronary computed tomography angiography (CCTA), in patients assigned to the "observe zone." The reason may be a combination of available expertise built over the years of using ICA and a lack of evidence for the usefulness of CCTA in the "observe zone." Previous trials in low-risk patients with suspected NSTE-ACS have shown the efficacy of CCTA; however, this still remains to be seen for patients assigned to the observe zone [13,14]. Currently, we are in the final phase of a prospective, double-blind, observational, multicentre study (COURSE trial) that aims to determine the usefulness of CCTA in patients with lowrange positive troponins [15].

PEKT Score
Patients in the "observe" group with pre-existing CAD have a more than 2-fold increase in the incidence of a coronary diagnosis when presenting to the ED with acute chest pain. Considering that in our study a large proportion of "observe" patients have pre-existing CAD, this is an important risk factor to take into consideration in further patient management decisions at the ED.
Acute chest pain plays a central role in our novel risk score, with typical angina having the greatest weight in predicting a coronary diagnosis among all 5 predictors. However, patients with chest pain which specifically worsened during inspiration have a clearly lower likeli-hood of having a coronary diagnosis. This highlights the importance of proper history taking in patients with chest pain complaints and its downstream effect on the decision to perform an ICA.
Patients in the "observe" group presenting with impaired renal function defined as eGFR <60 were less likely to have a coronary diagnosis as compared to patients with normal renal function. This phenomenon is partially ascribed to non-ischemic elevation of troponin levels in patients with kidney disease, as opposed to myocardial necrosis leading to a rise of troponin levels in patients with myocardial infarction [16,17]. In patients with renal dysfunction, diagnostic protocols such as the ESC 0/1-h algorithm, also have a decreased specificity for type 1 myocardial infarction [18][19][20], i.e., even if patients with renal dysfunction develop myocardial infarction, it is less likely to have a coronary cause.
The PEKT score performed well and was able to discriminate between patients that need further coronary investigation and those that do not. A PEKT score lower than 6 identifies patients at low risk of coronary diagnosis with a relatively high NPV. In these patients treating physicians may consider an alternative diagnosis. It is important to note that our risk score warrants prospective external validation prior to use in daily clinical practice. The HEART score, which has been developed and validated in patients with suspected NSTE-ACS, was a relatively Fig. 3. ROC curves of risk scores for the prediction of a coronary diagnosis. AUC, area under the curve; HEART score, history, ECG, age, risk factors, and troponin score; PEKT score, prior history, ECG, kidney function, and type of chest pain score; ROC, receiver-operating characteristic.
Cardiology 2022;147:251-260 DOI: 10.1159/000523718 good predictor of a coronary diagnosis in our cohort but did not perform as well.
Our scoring system is simple and composed of easily obtainable clinical characteristics. The PEKT score may provide clinicians at the ED an opportunity to simplify their decision-making and may help reduce variability in patient management for the "observe" group.

Limitations
The present study has several limitations that should be acknowledged. First, its single-centre, retrospective design has some inherent limitations, such as selection bias. Thus, the study sample may not be a true representation of "observe" patients, which limits generalizability. Second, the final diagnosis was established by treating physicians as opposed to a central adjudication process of final diagnoses. Lack of final diagnosis adjudication in turn may have affected the developed risk score, considering that a potential change in final diagnosis after adjudication would alter the outcome of our diagnosis-based risk score.

Conclusion
Patients assigned to the "observe" group were in general elderly men with known pre-existing CAD. A substantial proportion of patients do not have an established diagnosis after 30 days and patients with a coronary diagnosis seem to have a better outcome when compared to other diagnostic categories. A PEKT score <6 identifies patients at low risk of a coronary diagnosis and may obviate the need for further coronary investigation in up to 40% of patients in our cohort.

Statement of Ethics
This study was conducted according to the principles of the Declaration of Helsinki, approved by the Medical Research Ethics Committee of Erasmus Medical Center in Rotterdam, The Netherlands (registration number MEC-2020-0157) and a waiver of informed consent was obtained.