Prediction of ADRs and Estimation of Polypharmacy in Older Patient’s Population: Retrospective Study in Russian Gerontology Center
Ali Al-Ragawi 1, 2 , Sergey Zyryanov 1, 3, * , Elena Ushkalova 1 , Olga Butranova 1 , Anton Pereverzev 4, ‡
- Department of General and Clinical Pharmacology, People’s Friendship University of Russia (PFUR), Moscow, Russia
- Department of Pharmacology, Taiz University, Taiz, Yemen
- Multifield City Clinical Hospital № 24, Moscow, Russia
- Pirogov National Research Medical University, Moscow, Russia
‡ Current Affiliation: Clinical and Research Centre of Gerontology, Moscow, Russia
* Correspondence: Sergey Zyryanov
Academic Editor: Michael Fossel
Received: October 10, 2018 | Accepted: February 25, 2019 | Published: March 05, 2019
OBM Geriatrics 2019, Volume 3, Issue 1 doi:10.21926/obm.geriatr.1901038
Recommended citation: Al-Ragawi A, Zyryanov S, Ushkalova E, Butranova O, Pereverzev A. Prediction of ADRs and Estimation of Polypharmacy in Older Patient’s Population: Retrospective Study in Russian Gerontology Center. OBM Geriatrics 2019; 3(1): 038; doi:10.21926/obm.geriatr.1901038
© 2019 by the authors. This is an open access article distributed under the conditions of the Creative Commons by Attribution License, which permits unrestricted use, distribution, and reproduction in any medium or format, provided the original work is correctly cited.
Abstract
Background: Adverse drug reactions (ADRs) are common in the elderly and often have serious clinical and economic consequences. Prediction of ADRs is a valuable tool to improve pharmacotherapy and outcomes in this category of patients. Objective: Prediction of ADRs in patients ≥ 65 years old, based on GerontoNet risk score with emphasis on polypharmacy. Methods: Pharmacoepidemiological retrospective study of medical records of patients admitted to the Russian Gerontology Clinical Research Center (RGCRC) between 1 June and 30 December 2017 (n = 201). The GerontoNet ADR risk score was calculated for all patients. We used χ2 tests and t-test to compare categorical variables and analyses of variance (ANOVA) to compare continuous variables. Receiver operator characteristic (ROC) curves were constructed to test the ability of the GerontoNet risk score to predict ADRs. Polypharmacy parameters before admission and during hospital stay were analyzed. Results: The study included 201 medical records of elderly patients. The mean age (± SD) was 76.1 ± 6.2 years, 74.6% of patients were female and the median number of drugs per patient was 8 (range 6–11). Before hospital admission, exposure to ADRs was reported in 46 patients (22.9%). GerontoNet risk score before admission (mean ± SD) was 2.54 ± 1.4; 111 patients (55.2%) were with 0-2 points and 90 (44.8%) with ≥3 points. GerontoNet risk score during hospital stay was 4.79 ± 2.1. Score was higher in patients with polypharmacy (≥5 drugs), compared with those without. The area under the curve (ROC) was 0.887 (95% CI: 0.84-0.94), the cut-off point of 4 indicated a very good sensitivity and good specificity of results. Before admission, polypharmacy was seen in 36.3% of patients whereas during hospital stay - in 91%. Polypharmacy exposure increased during hospital stay (P ˂ 0.001); consequently, GerontoNet risk score was increased. Variables contributing to ADR risk score include renal failure, polypharmacy, age ≥75 years and the presence of comorbidities. During hospital stay patients with a high GerontoNet risk score (≥4) were predicted to be exposed to ADRs at least four times more frequently than patients with low GerontoNet risk score (0-3) (OR: 15.08; 95%CI 7.3–31.15; OR: 3.103; 95% CI, 0.984–9.785). Conclusions: High prevalence of multi-morbidity among the elderly is one of the leading factors of polypharmacy for this population. Adverse reactions in older people can be assessed using GerontoNet, although the GerontoNet ADR risk score improperly classified 21.9% of patients as high-risk group. Review of appropriateness of medication use is essential, as polypharmacy enhance risk of adverse effects in the elderly population.
Keywords
Adverse drug reaction (ADR); elderly; GerontoNet ADR risk score; polypharmacy
1. Introduction
The elderly population has a marked trend to be increased worldwide; it is supposed to grow from about 9% (2018) up to 16% and more in mid-century [1]. The same population aging trends are seen in the Russian Federation - an increase in the population of elderly people in absolute number and proportion [2].
Increase in proportion of people over 65 means increase in a number of patients living with illness and disability. There is an increase of total morbidity rate in the elderly: in the Russian Federation in 2013 total number of morbidity cases was 208,228.9 per 100,000 of people, that is 1.3 times higher than the rate in general population of a country (161,061.8 people per 100 thousand people). According to the data of clinical studies, about 80% of elderly patients has at least one chronic disease and 55% to 98% patients are affected by multi-morbidity (co-existence of a number of chronic diseases in the same individual) [3]. Data from population-based longitudinal study (3 years) revealed that 33.6% of participants with no chronic disease and 66.4% of those with at least one disease at baseline developed multi-morbidity: the incidence rate was 12.6 per 100 person-years (95% CI: 9.2–16.7) and 32.9 per 100 person-years (95% CI: 28.1–38.3), respectively [4]. A cross-sectional study of 1.7 million patients registered in the primary care in Scotland showed the level of multi-morbidity at 81.5% for individuals over 85 [5].
The combination of acute and chronic pathology in elderly category of patients results in a strong tendency for polypharmacy. Scottish study revealed great increase of population over 65 with polypharmacy, with 16.4% receiving 10 or more medications [6]. The data from the systematic review performed by Storms H. et al (2017) revealed next prevalence of inappropriate medication use in the elderly: 18.5% to 82.6% (median 46.5%) when Beers criteria in general (‘B1991,’ ‘B1997,’ ‘B2003’ and ‘B2012’) were used; 21.3% to 63.0% (median 35.1%) when inappropriate medication use was assessed only with the complete criteria list of Beers 2003 (‘B2003,’ both criteria independent of disease ‘ID’ and criteria considering disease ‘CD’); 63.0% to 82.6% in studies used 2012 update of the Beers criteria (‘B2012’); 23.7% to 79.8% (median 61.1%) ) for studies used STOPP and from 30.5% up to 74.0% (median 48.6%) for studies used START [7]. The prevalence of multi-morbidity, polypharmacy, and inappropriate medication use in the elderly results in a dramatic increase of ADR risk, also worsened by pharmacokinetic and pharmacodynamic age-related changes. Investigations show that up to 60% of nursing home residents experience ADRs [8]. 10% median ADR-related hospital admission rate for patients ≥65 was revealed in the systematic review performed by Kongkaew C. et al (2008) [9] and 11% rate in the systematic review of Alhawassi TM et al (2014) [10]. Polypharmacy is one of the leading risk factors for ADR: there is a 13% increased risk of ADRs in the case of two drugs administration, 58% in the case of five drugs and 82% in case of seven or more drugs [11]. Beijer H.J. et al (2002) demonstrated that about 88% of the ADRs related to hospitalizations are preventable in the elderly, compared with 24% in the younger patients [12]. Thus, the tool to predict ADR risk in the elderly is highly desirable for real clinical practice, though many attempts were made with uncertain results.
The GerontoNet ADR risk score has been developed based on the analytic work of the Italian Group of Pharmacoepidemiology in the Elderly (GIFA) (5936 older inpatients), a validation study that included 483 patients (mean (+SD) age was 80.3 + 7.6 years) with the area under the receiver operator characteristic curve 0.70 (95% confidence interval, 0.63-0.78) [13]. A recent validation of this tool was performed in the study of Mirko Petrovic et al (2017) based on the data from the prospective CRIteria to assess appropriate Medication use among Elderly complex patients (CRIME) cohort [14]. The GerontoNet ADR risk score ranges from 0 to 10 points and includes six variables, with each scoring based on the strength of its association with ADRs: presence of 4 or more comorbid conditions (1 point), renal failure (1 point), heart failure (1 point), liver disease (1 point), number of drugs (1 point when between 5 and 7; 4 points when 8 or more) and a history of ADRs (2 points).
Objective of the study: prediction of ADRs in patients ≥ 65 years old, based on GerontoNet risk score with emphasis on polypharmacy.
2. Materials and Methods
2.1 Study Design
Retrospective Pharmacoepidemiologic study. Study was approved by the Ethic Committee of Medical Institute of People’s Friendship University of Russia, (Protocol No. 9 from 17/05/2018).
2.2 Study Setting and Population
The records of patients ≥65 years old discharged between 1st of June and 30th of December 2016 from Russian Gerontology Clinical Research Center (RGCRC) in Moscow were studied. Inclusion criteria: ≥65 years old, administration of at least 1 drug. The final study included 201 records. Analyzed data: demographics, reasons for admission, comorbidities, diagnosis at admission and at discharge, baseline laboratory data, medications taken before admission and during hospital stay, and medications prescribed at discharge. Data were collected on a standardized data collection form (Microsoft Excel® 2016). All patient’s records were subdivided into 3 age groups (65–74, 75–84 and ≥ 85 year). Estimated creatinine clearance rate (eCCr) was evaluated using Cockcroft-Gault formula. The state law on personal data protection was fulfilled. We defined polypharmacy as use of ≥ 5 drugs, and hyperpolypharmacy as use of 8 or more drugs, based on the results of systematic review made by Masnoon N. et al (2017) [15]. The GerontoNet ADR risk score was calculated for all patients. Receiver operator characteristic (ROC) curve was constructed to test the ability of the GerontoNet risk score to predict ADRs.
2.3 Statistical Analysis
Patients data were stored in Excel 2016 (Microsoft®, USA) and exported to SPSS ver. 22 (IBM®, USA) for statistical analysis. Descriptive statistics were generated to describe the characteristics of the study population. We used χ2 tests and t tests to compare categorical variables and analyses of variance (ANOVA) to compare continuous variables. Receiver operator characteristic (ROC) curves were constructed, and the area under the curve (AUC) was calculated to determine the predictive ability of the GerontoNet ADR risk score. Sensitivity and specificity were calculated for a cut-off point of GerontoNet score before admission and during hospital stay.
3. Results
3.1 Patient Characteristics
We studied medical records of 201 patients. 56.7% of patients were admitted in the cardiac ward, 32.4% in the therapeutics and 10.9% in neurology. The patients mean age (+ SD) was 76.1 ± 6.3 years (min – 65, max - 94); 74.6% patients were female, 85.1% of patients had at least four chronic comorbidities: hypertension, ischaemic heart disease, cerebrovascular diseases and atherosclerosis of cerebral vessels. In 143 patients (71.1%), eGFR was ≤60 ml/min/1.73 m2; 13 patients (6.5%) had heart failure class III-IV (based on New York Heart Association (NYHA) classification); 7 patients (3.5%) had a diagnosis of hepatic failure; 146 patients (72.6%) had ischemic heart disease (IHD); 96 patients (47.8%) had heart failure; 123 patients (61.2%) had arterial hypertension and 44 patients (21.9%) had cardiac arrhythmias.
The mean (± SD) number of medications per patient during hospital stay was 8.5±3.2 (min – 6, max - 11). The most commonly used medications were Bisoprolol, Amlodipine, Atorvastatin, Losartan, Lisinopril, Enalapril, Indapamide, Piracetam, Benzodiazepines, Acetylsalicylic acid, Furosemide and laxatives. Polypharmacy at admission was observed in 66 patients (32.8%), which resulted in the increase of GerontoNet risk score by 4 points. Hyperpolypharmacy was seen in 7 patients (3.5%), which resulted in the increase of GerontoNet risk score by 1point.
ADRs were observed in 46 patients (22.9%) before admission. Characteristics of the study population are shown in the Table 1.
Table 1 Characteristics of the study population.
GerontoNet risk score (Mean± SE) was 2.54±1.4 before admission and 4.79±2.1 during hospital stay. Before admission the number of drugs and previous ADRs were the strongest predictors of GerontoNet ADR score ≥4 (P ˂0.001 and P ˂0.001 respectively), and presence of four or more comorbidities was considered as a predictor of ADR score ≥4 with p=0.031. The same was observed for the hospital stay period, with the number of drugs and previous ADRs being the strongest predictors, P ˂0.001. Before admission, 111 (55.2%) of patients were with 0-2 points and 90 (44.8%) patients were with ≥3 points of GerontoNet risk score. The prevalence of polypharmacy was high during hospital stay where 30.9%, patients used 5-7 medications, and 60.1% used ≥8 medications (Figure 1).
Figure 1 Polypharmacy before admission and during hospital stay.
3.2 GerontoNet Risk Score and Determination of Cut-Off Point
The mean (95% CL) GerontoNet ADR risk score before admission was 2.54 (2.35-2.74) points, range (0-7). It was higher in patients with polypharmacy compared with those without (29.8% and 15%, respectively).
ROC analysis was used to evaluate the diagnostic accuracy of the GerontoNet ADR risk score (AUC of 0.887 (95% CL: 0.84-0.94)) as a tool to predict drug-related ADRs. GerontoNet ADR risk score showed a very good sensitivity (82%) and good specificity (80%) in population with score ≥ 3, so it promised a proper identification of patients who are at high risk for ADRs (Figure 2). The median (IQR) GerontoNet risk score in patients with ADRs was 3.91 (3.66-4.16), and in patient without ADRs was 2.14(1.93-2.34), P = 0.000. GerontoNet risk score increased with polypharmacy exposure and comorbidity presence (at least 4 diseases), but there was no association between increased GerontoNet risk score and previous ADRs (P = 0.513).
The mean GerontoNet ADR risk score during hospital stay (95% CI) was 4.79 (4.51-5.08), range (0-9). It was higher in patients with polypharmacy compared with those without (66.2% and 24.9%, respectively). Gender differences were detected: mean (CI 95%) GerontoNet score in women was greater than in men (2.67 (2.45-2.90) vs 2.16 (1.76-2.56), P = 0.023). The number of patients with polypharmacy and hyperpolypharmacy was the highest in the 75-84 and ≥ 85-year age groups (29% and 18.4%, respectively). Polypharmacy was not associated with heart failure (P= 0.448) or liver disease (P= 0.716).
Figure 2 Receiver Operator Curve of GerontoNet ADR risk score and ADRs (i) on admission and (ii) during hospital stay. Receiver operator characteristic (ROC) curve was constructed to test the ability of the GerontoNet risk score to predict ADRs.
3.3 Adverse Drug Reactions
Forty-six patients (22.9%) experienced ADR before hospital stay; of them, 82.6% had renal failure, 89.1% were female, 93.5% had polypharmacy during hospital stay period, and 80.4% had at least four comorbidities. Patients with ADRs were compared with patients without ADRs. Median age (77 versus 76 years, P = 0.723) was of no difference and GerontoNet risk score was strongly associated with ADRs (P = 0.000). Patients with previous ADRs (n=38) had renal impairment (26.6%); P = 0.051) more typically. Number of daily prescribed drugs during hospital stay per patient was of no difference (8.31± 3.21versus 9.22 ± 3.01, P = 0.089). There was a difference between these subgroups in terms of gender P = 0.010, and no differences in terms of burden of comorbidity, heart failure, polypharmacy before admission and liver disease (Table 2).
We compared patients with and without polypharmacy during hospitalization. Subjects with polypharmacy were older; had higher GerontoNet ADRs risk score before admission and during hospital stay; and had higher number of used medications. Conversely, there were no statistically significant differences between these two groups in gender or previous ADRs exposure. Subjects with polypharmacy had chronic heart failure and liver disease more frequently, although it was statistically insignificant (P > 0.381, P > 0.122, respectively). The rate of renal failure was higher in patients with polypharmacy compared with patients without polypharmacy (66.9% vs 4.5%, P ≤ 0.038). Relations between polypharmacy and predictors of ADRs are shown in the Table 3.
Effects of age and gender on GerontoNet ADRS risk score and parameter of polypharmacy before admission were tested with a 2-way ANOVA analysis. Table 4 shows that advanced age and gender had no significant effect.
Table 2 Predictors of ADRs before admission. Comparison of descriptive characteristics in patients with and without ADRs.
Table 3 Relations between polypharmacy and predictors of ADRs.
Three age groups were compared by the parameters of GerontoNet risk score, polypharmacy before admission and the number of drugs used. There was no significant difference between these age groups. However, patients ≥ 85 had the highest GerontoNet score (2.95 (2.26-3.64)), more likely to have had polypharmacy before admission (0.7 (0.268-1.132)), and had used more drugs (9.10 (7.97-10.23)).
There was no significant difference between age subgroups in gender. Mean GerontoNet risk score in women was greater than in men (CL 95%: 2.67 (2.45-2.90) and 2.16 (1.76-2.56), respectively).
Table 4 Effect of age on GerontoNet Risk Score and predictors of ADR for men and women.
4. Discussion
Our study revealed polypharmacy increase with advanced age, (from 33.9% at 65 years old to 47.5% at 75 years old). Publicized studies also report high polypharmacy prevalence among the elderly population [15,16]. The most common reason for simultaneous prescription of several drugs in this category of patients is the presence of comorbidities. In our study, elderly patients were highly exposed to polypharmacy during hospital stay. There was no significant effect of gender or previous ADRs on parameters of GerontoNet ADR risk score and polypharmacy in our study; this finding is consistent with the results of an Australian study [16].
Notably, we identified an association between ADRs and gender. Additionally, patients with comorbidities and polypharmacy before admission were more likely to be exposed to ADRs independent of other factors. For patients with ADRs during hospital stay, the results of our study were similar with published results of other authors [16]. Unfortunately, there is no ultimate definition of polypharmacy in the medical society. To operationalize this definition, researchers choose various cut-off points.
Our results revealed GerontoNet score ≥4 (a cut-off point to predict ADRs) in 67.2% of patients. The use of GerontoNet Risk Score in our study resulted in improper classification of 21.9 % of patients as high-risk group. It may be the result of hyperpolypharmacy (>7 drugs) in the studied population, which gave an increase of GerontoNet risk score by 4 points.
In our study, GerontoNet score showed good sensitivity and specificity, so it can be used as a tool to identify patients at high risk for ADRs. According to our data, GerontoNet risk score during hospital stay was greater than that before admission and hence, its predictable value is higher for hospital stay period. Potentially, inappropriate prescribing is a common problem for elderly patients in hospitals. The use of a tool to predict ADRs can reduce polypharmacy, improve outcomes in hospitalized elderly patients and decrease therapy costs [17,18]. G. Onder et al. [13] revealed that serious ADRs in the elderly occur in 64% of the cases. In the presence of eight or more GerontoNet risk score points, ADRs develop in 21.7%, and the number of simultaneously administered drugs is the strongest risk factor for the development of the ADRs. GerontoNet risk score in a validation study of 483 patients results in correct prediction of ADRs in 71% of cases (95% CI 0.68–0.73) [13]. In the prospective study of O’Connor et al., the use of GerontoNet risk score incorrectly marked 38% of patients as low risk ADR group [19]. ADR prediction tools should be ideally tested on representative sample of a target population to evaluate clinical impact before widespread implementation.
Possible explanations why our results are different from those in the original GerontoNet ADR risk score study may be related to the interpretation of the variables within the score. In addition, calculation of the risk score in individuals has the potential to vary across different countries.
5. Conclusion
The results of our study suggest that ADRs in the elderly can be predicted with the use of GerontoNet Risk Score, although it improperly classified 21.9 % of patients as high-risk group. The prevalence of a high number of prescriptions (>7 drugs) in study population elevated GerontoNet risk score by 4 points may contribute to improper prediction rate.
Author Contributions
Sergey Zyryanov, who is the senior author, provided the intellectual input and approved the protocols to be followed in the study. Ali Al-Ragawi, who is the presenting author, is responsible for preparing the manuscript and analysing data. Olga Butranova and Elena Ushkalova both contributed in the critical revision of the manuscript. Anton Pereverzev contributed in conception design and data collection.
Funding
No sources of funding were used to assist in the preparation of this article.
Competing Interests
The authors have declared that no competing interests exist.
References
- https://www.prb.org/2018-world-population-data-sheet-with-focus-on-changing-age-structures/
- http://ac.gov.ru/files/publication/a/8485.pdf.
- Marengoni A, Winblad B, Karp A, Fratiglioni L. Prevalence of chronic diseases and multimorbidity among the elderly population in Sweden. Am J Public Health. 2008; 98: 1198-1200. [CrossRef]
- Melis R, Marengoni A, Angleman S, Fratiglioni L. Incidence and predictors of multimorbidity in the elderly: a population-based longitudinal study. PLoS One. 2014; 9: e103120.
- Barnett K, Mercer SW, Norbury M, Watt G, Wyke S, Guthrie B. Epidemiology of multimorbidity and implications for health care,research, and medical education: A cross sectional study. Lancet. 2012; 380: 37-43. [CrossRef]
- Guthrie B, Makubate B. The rising tide of polypharmacy and potentially serious drug interactions 1995–2010: Repeated cross sectional analysis of prescribing in one region. Prim Health Care Dev. 2012; 13: 45 2E.2, doi: 10.1186/s12916-015-0322-7 [CrossRef]
- Storms H, Marquet K, Aertgeerts B, Claes N. Prevalence of inappropriate medication use in residential long-term care facilities for the elderly: A systematic review. Eur J Gen Pract. 2017; 23: 69-77. [CrossRef]
- Dilles T, Vander Stichele R, Van Bortel L, Elseviers M. The development and test of an intervention to improve ADR screening in nursing homes. J Am Med Dir Assoc. 2013; 14: 379.e1-379.e6. [CrossRef]
- Kongkaew C, Noyce PR, Ashcroft DM. Hospital admissions associated with adverse drug reactions: a systematic review of prospective observational studies. Ann Pharmacother. 2008; 42: 1017-1025. [CrossRef]
- Alhawassi TM, Krass I, Bajorek BV, Pont LG. A systematic review of the prevalence and risk factors for adverse drug reactions in the elderly in the acute care setting. Clin Interv Aging. 2014; 9: 2079-2086.
- Prybys K, Melville K, Hanna J, Gee A, Chyka P. Polypharmacy in the elderly: Clinical challenges in emergency practice: Part 1: Overview, etiology, and drug interactions. Emerg Med Rep. 2002; 23: 145-153.
- Beijer HJ, de Blaey CJ. Hospitalisations caused by adverse drug reactions (ADR): A meta-analysis of observational studies. Pharm World Sci. 2002; 24: 46-54. [CrossRef]
- Onder G, Petrovic M, Tangiisuran B, Meinardi MC, Markito-Notenboom WP, Somers A, et al. Development andvalidation of a score to assess risk of adverse drug reactionsamong in-hospital patients 65 years or older. Arch Intern Med. 2010; 170: 1142-1148. [CrossRef]
- Petrovic M, Tangiisuran B, Rajkumar C, van der Cammen T, Onder G. Predicting the risk of adverse drug reactions in older inpatients: External validation of the GerontoNet ADR risk score using the CRIME Cohort. Drugs Aging. 2017; 34: 135-142. [CrossRef]
- Masnoon N, Shakib S, Kalisch-Ellett L, Caughey GE. What is polypharmacy? A systematic review of definitions. BMC Geriatr. 2017; 17: 230. [CrossRef]
- Haider SI, Ansari Z, Vaughan L, Matters H, Emerson E. Prevalence and factors associated with polypharmacy in Victorian adults with intellectual disability. Res Dev Disabil. 2014; 35: 3071-3080. [CrossRef]
- Best O, Gnjidic D, Hilmer SN, Naganathan V, McLachlan AJ. Investigating polypharmacy and drug burden index in hospitalised older people. Inter Med Jour. 2013; 43: 912-918. [CrossRef]
- Frazier SC. Health outcomes and polypharmacy in elderly individuals: an integrated literature review. J Gerontol Nurs. 2005; 31: 4-11. [CrossRef]
- Payne R, Avery A, Duerden M, Saunders C, Simpson C, Abel G. Prevalence of polypharmacy in a Scottish primary care population. Eur J Clin Pharmacol. 2014; 70: 575-581. [CrossRef]
- Poudel A, Hubbard RE, Nissen L, Mitchell C. Frailty: A key indicator to minimize inappropriate medication in older people. QJM. 2013; 106: 969-975. [CrossRef]
- O’Connor M, Gallagher P, Byrne S, O’Mahony D. Adverse drug reactions in older patients during hospitalisation: Are they predictable? Age Ageing. 2012; 41: 771-776. [CrossRef]
- Lavan AH, Gallagher P. Predicting risk of adverse drug reactions in older adults. Ther Adv Drug Saf. 2016; 7: 11-22. [CrossRef]
- Walckiers D, Van der Heyden J, Tafforeau J. Factors associated with excessive polypharmacy in older people. Arch Public Health. 2015; 73: 50. [CrossRef]
- Maher R, Hanlon J, Hajjar E. Clinical consequences of polypharmacy in elderly. Expert Opin Drug Saf. 2014; 13: 57-65. [CrossRef]