Predicting Suicide Risk among South African University Students: Optimal Cutoff Score for the Center for Epidemiological Studies Depression Scale
Department of Psychology, University of the Western Cape, Robert Sobukwe Road, Cape Town, South Africa
† These authors contributed equally to this work.
Academic Editor: Luigi De Gennaro
Received: April 28, 2023 | Accepted: June 28, 2023 | Published: July 03, 2023
OBM Neurobiology 2023, Volume 7, Issue 3, doi:10.21926/obm.neurobiol.2303173
Recommended citation: Pretorius T, Padmanabhanunni A. Predicting Suicide Risk among South African University Students: Optimal Cutoff Score for the Center for Epidemiological Studies Depression Scale. OBM Neurobiology 2023; 7(3): 173; doi:10.21926/obm.neurobiol.2303173.
© 2023 by the authors. This is an open access article distributed under the conditions of the Creative Commons by Attribution License, which permits unrestricted use, distribution, and reproduction in any medium or format, provided the original work is correctly cited.
The COVID-19 disease outbreak was a significant psychological stressor. The unpredictable nature of the pandemic, high mortality rates from infection and the implementation of measures globally to curb its spread contributed to significant distress and adverse mental health outcomes, particularly among vulnerable population groups . COVID-19 prevention measures including national lockdowns, restrictions on movement, and closures of non-essential services, schools and higher educational institutions contributed to job and food insecurity and unemployment . These stressors can contribute to significant psychological distress. Those with pre-existing psychiatric disorders were reported to experience increased symptomology. In contrast, others were at heightened risk for developing mental health disorders such as depression, anxiety and post-traumatic stress disorder (PTSD) . Individuals who contracted COVID-19 and their families faced social stigma and isolation, which can aggravate stress . Existing research has confirmed that stress-related psychiatric disorders such as depression, substance use and PTSD are associated with an increased risk of suicidal behavior . There have also been multiple reported [1,5] incidents of COVID-19-related stressors precipitating suicidal behavior and completed suicide. In a meta-analysis investigating COVID-19-related suicide in 54 studies that included data from over 300 000 people, Dubé and Colleagues  found elevated rates of suicidal ideation, suicidal behavior and self-harm compared to pre-pandemic studies.
University students have been identified as a particularly vulnerable group for adverse mental health outcomes owing to the distinctive implications of the COVID-19 prevention measures on the educational sector. The closing of institutions of higher learning, migration to online learning environments and lack of access to academic and peer supports has been reported to have markedly elevated stress levels among this population group and contributed to psychological distress . Furthermore, even prior to the pandemic, university students were identified as being at increased risk for adverse mental health outcomes and suicidality . This has been ascribed to the many challenges associated with the transition from adolescence to early adulthood, including diminished parental guidance and support, negotiating a new academic environment and associated pressures, and increased opportunities for risky behavior (e.g., alcohol and drug use) . A cross-sectional study undertaken among Ghanaian students  reported prevalence rates for suicidal ideation (15.2%), attempted suicide (6.3%), death wishes (24.3%) and suicidal intention (6.8%). Psychological distress was identified as a salient predictor of suicidality. Using data from a national health survey for higher education in Norway, Sivertsen and Colleagues  reported prevalence rates for suicidal thoughts (21.0%, and 7.2% within the past year) and suicide attempts (4.2%). A population-based prevalence survey in Scotland  demonstrated that 11% of adults reported a suicide attempt at some stage in their lives and 16% indicated that they had engaged in non-suicidal self-injury.
Although various factors can enhance the risk of suicide, one of the strongest predictors of suicidality is the presence of depressive symptomology . Furthermore, the severity and variability of depressive symptoms have been reported as the only salient indicator of suicide attempts [12,13] among vulnerable individuals. Hence, identifying symptom characteristics that are risk factors for suicidality can facilitate a more accurate risk assessment and allow for the targeting of interventions.
The Center for Epidemiologic Studies Depression Scale (CES-D) is one of the most frequently used self-report instruments for depression [13,14]. The CES-D is a 20-item instrument developed to screen for depression, to assess for the presence of depressive symptoms and to detect vulnerability of having a depressive disorder . It assesses four components namely depressed affect, positive affect, somatic complaints, and interpersonal relations . Total scores can range from 0 to 60. Radloff  proposed that people who score 16 and above on the CES-D are likely to be clinically depressed but other studies [14,16] have suggested that the cut-point needs to be above 20 if the instrument is to be an effective diagnostic tool. Using a meta-analytic approach involving a review of 28 studies using the CES-D, Vilagut and Colleagues  proposed an optimal cutoff score of 20 with a specificity of 0.78, sensitivity of 0.83, and diagnostic odds ratio of 16.64.
The CES-D has demonstrated good reliability and validity for assessing depression in various population groups including university students , older adults  as well as in children and adolescents . It has also been reported to predict the risk of suicide. For example, a Chinese study of suicide attempters  found that CES-D scores were significantly higher in suicide attempters and that depression was associated with attempted suicide. A study involving a community sample of adults  reported that higher scores on the CES-D predicted greater suicidal thoughts and suicidal behavior. Lu and Colleagues  found that depressed affect, as measured by the CES-D, was a significant predictor of suicidal behavior among a cohort of adolescents in the United States. These authors  also reported that two CES-D items namely “I felt sad” and “I felt lonely” independently predicted suicidal ideation. In a study of community-dwelling adults, St John and Colleagues  found that a single item of the CES-D (i.e., “I felt depressed”) predicted suicide and this association persisted even after controlling for age, gender and educational status.
The current study aims to extend this research by using receiver operating characteristic (ROC) analysis to determine the optimal cutoff score on the CES-D when screening for suicide risk in university students during COVID-19 in South Africa. The South African government implemented one of the strictest lockdowns following the disease outbreak in 2020. Universities were required to migrate to online learning and teaching and students were compelled to leave campus residences and return home . This placed considerable pressure on students particularly those who did not have access to digital technology or resided in underprivileged settings with minimal resources. Uncertainty about their academic future, feelings of hopelessness, and concerns about their family's well-being may have aggravated student distress . Although the lockdown eased in 2021 and 2022, Universities continued to use hybrid modes of engagement owing to concerns about contagion.
ROC provides the most comprehensive description of diagnostic accuracy. Diagnostic precision was measured and reported for decades based on the number of correct diagnostic decisions . This percentage-correct measure is disadvantageous as it does not reveal the relative frequencies of false-positive and false-negative errors. These errors have different diagnostic and clinical consequences . ROC has the benefit of being able to estimate and report all of the combinations of specificity and sensitivity that a diagnostic scale can provide . Sensitivity refers to the fraction of individuals with the condition (e.g., clinical depression) in question and who are correctly diagnosed as positive. In contrast, specificity refers to the fraction of individuals without the condition who are correctly diagnosed as negative . ROC indices account for false-positive and false-negative diagnoses implicitly, and a change in the condition's prevalence does not affect the numerical values. ROC reports all of the combinations of sensitivity and specificity that a diagnostic test can provide .
2. Materials and Methods
2.1 Participants and Procedure
A cross-sectional research design was adopted in this study. Participants (N = 337) were randomly sampled (5% margin of error, 95% confidence interval) young adults enrolled in undergraduate studies at a Western Cape, South Africa university. We used Google Forms to create an electronic version of the questionnaires used in the study. The electronic link together with the description of the study, as well as an invitation to participate were sent to 1500 randomly selected students. The response rate therefore was 22.5%. The study was conducted during the first wave of COVID-19 in 2020. Most of the sample were women (77.2%) and resided in an urban area (75.4%). The mean age of the sample was 21.95 years (SD = 4.68).
Ethical approval for this study was obtained from the Humanities and Social Sciences Research Committee of the university (Ethics reference number: HS20/5/1). The survey was completed anonymously, and all the participants provided informed consent prior to accessing the survey. The participants were also provided with the contact details of the South African Depression and Anxiety Group and the Centre for Student Support Services at the university if they experienced any psychological distress after completing the questionnaire.
Participants completed the CES-D and the Beck Hopelessness Scale (BHS) . This study was part of a bigger study focusing on the mental health of students; therefore, only limited demographic information was collected.
The CES-D is a 20-item measure of depressive symptoms that is scored on a four-point scale ranging from “rarely or none of the time” (0) to “most or all of the time” (3). An example item of the CES-D is “I had trouble keeping my mind on what I was doing." Radloff  provided evidence of reliability across different population groups (Cronbach’s alpha = 0.84 to 0.90), and of validity e.g., the correlation between scores on the CES-D and clinical ratings of depression. Pretorius  examined the applicability of the CES-D in a sample of South African students. They reported a reliability coefficient of 0.90 and concerning validity, found that scores on the CES-D were significantly related to a measure of life change. The CES-D was also used with a sample of schoolteachers in South Africa and satisfactory reliability coefficients were reported in that study (alpha = 0.92, omega = 0.93) .
The BHS is a 20-item measure of hopelessness and pessimism about the future. Participants respond to the 20 items on a dichotomous scale of true/false. An example of an item of the BHS is “the future seems vague and uncertain to me." Beck reported a reliability coefficient (K-R) of 0.93 for the BHS when used in a sample of hospitalized patients who had attempted suicide . In addition, correlations between the BHS and clinical ratings of hopelessness and other self-report measures of hopelessness provided evidence for the scale's validity. In South Africa, the BHS has been used with young adults  and schoolteachers , and the reliability coefficients in these two samples were 0.88 and 0.89, respectively.
2.3 Data Analysis
IBM SPSS for Windows version 28 was used for the data analysis. This included descriptive statistics (means and standard deviations), reliabilities (alpha and omega) and correlation between depression and hopelessness (Pearson r). ROC analysis was used to determine the optimal cutoff point on the CES-D for identifying suicide risk. The literature has generally identified a cutoff score of 9 on the BHS as predictive of suicide risk. A meta-analytic review identified 9 as a standard cutoff point in several studies . In the ROC analysis the area under the curve (AUC) provides an estimate of the diagnostic efficiency of the CES-D. Generally, the following guidelines can be used to evaluate AUC: <0.70 poor, 0.70-0.79 fair; 0.80-0.89 good, and 0.90-1.00 excellent . The ROC analysis also provides sensitivity and specificity indices for every cutoff score. Sensitivity, in this instance, would refer to the probability of correctly identifying those who are at a high risk of suicide (true positives), while not classifying those at risk as not being at risk (false negatives). Specificity, on the other hand, is the probability of correctly identifying all those who are not at risk of suicide (true negatives), while not classifying those that are not at risk as being at risk (false positives) . In this study, we decided to optimize sensitivity rather than specificity based on Youngstrom’s  view that high sensitivity is more important than specificity with screening tests to avoid missing cases at risk. For diagnostic tests specificity would be more important. Before the ROC analysis of the whole sample, we compared the AUC of men and women to determine if a single AUC can represent the whole sample. SPSS also indicates overall model quality when groups are being compared. If the overall model quality is above 0.5 it indicates good predictive ability. We also used Cohen’s Kappa to measure accuracy, obtained using an Excel worksheet . In general, a Kappa above 0.2 can be regarded as fair. Lastly, we used Excel to obtain positive and negative predictive values for every cutoff score. Positive predictive values (PPV) refer to the probability that someone who is identified as at risk of suicide is actually at risk, while negative predictive values (NPV) refer to the probability that someone who is identified as not at risk is not at risk.
The mean score for the CES-D was 27.5 (SD = 13.36, range = 0-57) and for the BHS it was 4.7 (SD = 4.4, range = 0-20). The two instruments demonstrated satisfactory reliability (CES-D = 0.92; BHS = 0.88). There was a substantial positive association between depression and hopelessness (r = 0.56, p < 0.001), indicating that high levels of depression are associated with high levels of hopelessness.
The ROC curve for comparing men and women is shown in Figure 1.
Figure 1 ROC curves for men and women. The blue curve is men; the green curve is women.
The AUC for men was 0.80 (SE = 0.06, 95% CI: 0.68, 0.91) and for women was 0.81 (SE = 0.04, 95% CI: 0.74, 0.88) reflecting a good diagnostic efficiency for the CES-D for both men and women. There was no statistical difference between AUC’s of men and women (z = −0.28, p = 0.78). The overall model quality for men and women is shown in Figure 2.
Figure 2 Overall model quality for men and women (men: gender = 1; women: gender = 2).
Figure 2 indicates that the diagnostic efficiency of the CES-D for women was marginally better than for men. However, the model quality was above 0.5 for both.
The ROC curve for the whole sample is presented in Figure 3.
Figure 3 ROC curve predicting suicide risk based on the CES-D.
The AUC for the total sample when the CES-D was used to screen for suicide risk was 0.81 (SE = 0.03, 95% CI: 0.75, 0.87), which in terms of the guidelines reflects a good diagnostic efficiency for the CES-D in terms of screening for suicide risk. The results of the ROC analysis, together with the indices of accuracy (Cohen’s Kappa) and the predictive values (PPV and NPV) are reported in Table 1.
Table 1 reflects that a cutoff score of 30 with a sensitivity of 89.30 and a specificity of 62.60 and an acceptable Kappa of 0.304 represents an acceptable cutoff point. Thus, a cutoff score of 30 would have an 89.3% probability of correctly identifying true positives and false negatives, and a 62.6% probability of correctly identifying true negatives and false positives. The contingency table in BOX 1 reflects the number of true and false positives and negatives based on a cutoff of 30.
Box 1 Contingency table based on CES-D cutoff score of 30.
The COVID-19 pandemic significantly impacted the mental health of university students . Existing studies documented elevated levels of psychological distress arising from pandemic-related stressors including the closing of universities, the transition to remote digital learning and disruptions in access to academic and peer support networks . Students also had to contend with the economic uncertainty produced by the pandemic and issues related to job and food security . The current study was conducted at a historically disadvantaged institution (HDI) in South Africa. HDIs were created during the apartheid era as part of segregationist policies of differential education for different race groups and were significantly under-resourced . Although much has changed since the transition to democracy in the country, students at HDIs are predominantly from previously marginalized groups and reside in disadvantaged community contexts characterized by poverty, unemployment, gang violence and increased rates of substance use . These contextual factors can amplify pandemic-related stress and precipitate negative psychological outcomes [34,35]. Elevated levels of depressive symptomology among university students have been consistently identified as a significant mental health consequence of the pandemic and its prevention measures [8,9,36]. Depression confers a significantly greater risk for suicidal ideation and suicide attempts and for completed suicide [11,21]. For this reason, measures of depression must be able to accurately diagnose symptomology and thereby identify risk. The CES-D is a frequently used measure of depression severity and has been utilized in both research settings and by clinicians to screen for depressive symptomology . The instrument has demonstrated sound psychometric properties and has been extensively used among different populations [14,18]. However, the classic cutoff score proposed by the developer in 1977  has yielded many false positives . There is also limited evidence to support the original cutoff score of 16 and outdated cutoff scores can enhance the risk of false positives and false negatives, which can significantly limit the instrument's utility as a screening tool [37,38].
The current study aimed to extend research on the diagnostic accuracy of the CES-D when screening for suicide risk by using ROC analysis to determine the optimal cutoff score on the instrument among a sample of South African university students. The study found that a cutoff score of 30 with a sensitivity of 89.30 and a specificity of 62.60 represented an acceptable cutoff point. This finding contrasts with existing studies that have used ROC analysis. For example, Campo-Arias and Colleagues  assessed the psychometric properties of the CES-D among Columbian adults using ROC analysis. They reported that a score of 20 represented this cut-off point, with a sensitivity of 0.96, specificity of 0.73. A study  investigating the utility of the CES-D as a depression screening tool among low-income women attending primary care clinics reported a cutoff score 16, which yielded a sensitivity of 0.95 and specificity of 0.70. Quiñones and Colleagues  reported that higher cut points for the CES-D performed better in correctly identifying true negatives and true positives for major depressive disorder among veterans. These authors  reported a cutoff of 18 with a sensitivity of 92% specificity and 72% among a sample of United States veterans. A study  aimed at determining the optimal cutoff on the CES-D for depression in a community corrections sample reported an optimal cutoff of 23 with a sensitivity of 78.2% and specificity of 79.8%. These differences in optimal cutoff points reinforce arguments [37,40] that the cutoff for measures of mental health outcomes must be determined for specific populations rather than relying on a universal cutoff point for screening a particular population. This is due to the possibility that measurement scales assess different constructs among diverse population groups. While previous studies have investigated cutoff points for clinical depression, this study differs from previous research as it focuses on the utility of CES-D cutoff points in screening for suicide risk. Although the CES-D has demonstrated excellent discrimination between individuals who experience depression and are at greater risk of suicidality, screening scales are essentially rough measures of psychiatric disorders. Therefore, further clinical evaluation and assessment is needed to confirm the accuracy of scores.
The present study is limited by a cross-sectional design, which impacts the generalizability of our findings and conclusions about causality. Although we conjecture that increased psychological distress may be due to COVID-19-related factors, other variables could probably have contributed to this. The study was also undertaken in one geographic area and among a student sample from one institution. The gender distribution of participants was not proportional and the sample consisted predominantly of women but women are generally over-represented in the higher education sector. Existing studies  suggest that women are more vulnerable to depression, which can impact the study findings. Future research using a longitudinal design and a more diverse sample is needed to confirm our results. In addition, the study used a self-report method, and measurement and social bias are possible, which can impact the results. Using a triangulation design involving clinician-administered standardized interviews and self-report measures may yield greater accuracy in diagnosis. Despite these limitations, the study has important implications. It highlights the potential need for cutoff points on measures of mental health disorders to be population specific to enhance the accuracy of diagnostic decision-making, the early identification of risk and the timely implementation of interventions.
In conclusion, the CES-D was demonstrated to be a valid screening tool for use among a student population at a historically disadvantaged South African university. A cutoff score of 30 was the most fitting for predicting suicide risk. It is recommended that when the instrument is used in clinical settings and among student populations, those with scores of 30 and above need to be followed up with a clinician-administered interview to confirm risk and facilitate timely intervention. Overall, the CES-D has the potential to identify those at risk of suicide and can provide an important first step in intervention efforts. It can also potentially be used as an outcome measure to assess whether interventions that seek to reduce the risk of suicide are effective.
Conceptualization and methodology: Anita Padmanabhanunni & Tyrone Pretorius; Analysis and data curation: Tyrone Pretorius. All authors discussed and interpreted obtained results. All authors contributed to the writing and editing of the manuscript over several iterations.
The authors have declared that no competing interests exist.
- Caballero-Domínguez CC, Jiménez-Villamizar MP, Campo-Arias A. Suicide risk during the lockdown due to coronavirus disease (COVID-19) in Colombia. Death Stud. 2022; 46: 885-890. [CrossRef]
- Padmanabhanunni A, Pretorius TB. Promoting well-being in the face of a pandemic: The role of sense of coherence and ego-resilience in the relationship between psychological distress and life satisfaction. S Afr J Psychol. 2023; 53: 124-133. [CrossRef]
- Buneviciene I, Bunevicius R, Bagdonas S, Bunevicius A. The impact of pre-existing conditions and perceived health status on mental health during the COVID-19 pandemic. J Public Health. 2022; 44: e88-e95. [CrossRef]
- Dubé JP, Smith MM, Sherry SB, Hewitt PL, Stewart SH. Suicide behaviors during the COVID-19 pandemic: A meta-analysis of 54 studies. Psychiatry Res. 2021; 301: 113998. [CrossRef]
- Rider EA, Ansari E, Varrin PH, Sparrow J. Mental health and wellbeing of children and adolescents during the COVID-19 pandemic. BMJ. 2021; 374: n1730. [CrossRef]
- Padmanabhanunni A, Pretorius TB. When coping resources fail: The health-sustaining and moderating role of fortitude in the relationship between COVID-19-related worries and psychological distress. African Saf Promot. 2020; 18: 28-47.
- Padmanabhanunni A, Wiid C. From fear to fortitude: Differential vulnerability to PTSD among South African university students. Traumatology. 2022; 28: 167-174. [CrossRef]
- Sivertsen B, Hysing M, Knapstad M, Harvey AG, Reneflot A, Lønning KJ, et al. Suicide attempts and non-suicidal self-harm among university students: Prevalence study. BJPsych Open. 2019; 5: e26. [CrossRef]
- Owusu-Ansah FE, Addae AA, Peasah BO, Oppong Asante K, Osafo J. Suicide among university students: Prevalence, risks and protective factors. Health Psychol Behav Med. 2020; 8: 220-233. [CrossRef]
- O'Connor RC, Wetherall K, Cleare S, Eschle S, Drummond J, Ferguson E, et al. Suicide attempts and non-suicidal self-harm: National prevalence study of young adults. BJPsych Open. 2018; 4: 142-148. [CrossRef]
- Efstathiou V, Stefanou MI, Siafakas N, Makris M, Tsivgoulis G, Zoumpourlis V, et al. Suicidality and COVID-19: Suicidal ideation, suicidal behaviors and completed suicides amidst the COVID-19 pandemic (Review). Exp Ther Med. 2022; 23. doi: 10.3892/etm.2021.11030. [CrossRef]
- Melhem NM, Porta G, Oquendo MA, Zelazny J, Keilp JG, Iyengar S, et al. Severity and variability of depression symptoms predicting suicide attempt in high-risk individuals. JAMA Psychiatry. 2019; 76: 603-613. [CrossRef]
- Lu W, Lindsey MA, Irsheid S, Von Eugene N. Psychometric properties of the CES-D among black adolescents in public housing. J Soc Soc Work Res. 2017; 8: 595-619. [CrossRef]
- Jiang L, Wang Y, Zhang Y, Li R, Wu H, Li C, et al. The reliability and validity of the Center for Epidemiologic Studies Depression Scale (CES-D) for chinese university students. Front Psychiatry. 2019; 10: 315. [CrossRef]
- Radloff LS. The CES-D scale: A self-report depression scale for research in the general population. Appl Psychol Meas. 1977; 1: 385-401. [CrossRef]
- Tran TD, Kaligis F, Wiguna T, Willenberg L, Nguyen HTM, Luchters S, et al. Screening for depressive and anxiety disorders among adolescents in Indonesia: Formal validation of the centre for epidemiologic studies depression scale-revised and the Kessler psychological distress scale. J Affect Disord. 2019; 246: 189-194. [CrossRef]
- Vilagut G, Forero CG, Barbaglia G, Alonso J. Screening for depression in the general population with the Center for Epidemiologic Studies Depression (CES-D): A systematic review with meta-analysis. PLoS One. 2016; 11: e0155431. [CrossRef]
- Karim J, Weisz R, Bibi Z, ur Rehman S. Validation of the eight-item Center for Epidemiologic Studies Depression Scale (CES-D) among older adults. Curr Psychol. 2015; 34: 681-692. [CrossRef]
- Stockings E, Degenhardt L, Lee YY, Mihalopoulos C, Liu A, Hobbs M, et al. Symptom screening scales for detecting major depressive disorder in children and adolescents: A systematic review and meta-analysis of reliability, validity and diagnostic utility. J Affect Disord. 2015; 174: 447-463. [CrossRef]
- Yang L, Jia CX, Qin P. Reliability and validity of the Center for Epidemiologic Studies Depression Scale (CES-D) among suicide attempters and comparison residents in rural China. BMC Psychiatry. 2015; 15: 76. [CrossRef]
- Cheung YB, Liu KY, Yip PSF. Performance of the CES-D and its short forms in screening suicidality and hopelessness in the community. Suicide Life Threat Behav. 2007; 37: 79-88. [CrossRef]
- St John PD, Montgomery P. Does a single-item measure of depression predict mortality? Can Fam Physician. 2009; 55: e1-e5.
- Gardner IA, Greiner M. Receiver-operating characteristic curves and likelihood ratios: Improvements over traditional methods for the evaluation and application of veterinary clinical pathology tests. Vet Clin Pathol. 2006; 35: 8-17. [CrossRef]
- Metz CE. Receiver operating characteristic analysis: A tool for the quantitative evaluation of observer performance and imaging systems. J Am Coll Radiol. 2006; 3: 413-422. [CrossRef]
- Beck AT, Weissman A, Lester D, Trexler L. The measurement of pessimism: The hopelessness scale. J Consult Clin Psychol. 1974; 42: 861-865. [CrossRef]
- Pretorius TB. Cross-cultural application of the center for epidemiological studies depression scale: A study of black South African students. Psychol Rep. 1991; 69: 1179-1185. [CrossRef]
- Jackson K, Noordien Z, Padmanabhanunni A, Pretorius TB. The mediating role of teacher identification in the relationship between psychological distress and teacher satisfaction during COVID-19. Inquiry. 2022; 59: 00469580221110520. [CrossRef]
- Padmanabhanunni A, Pretorius TB, Stiegler N, Bouchard JP. A serial model of the interrelationship between perceived vulnerability to disease, fear of COVID-19, and psychological distress among teachers in South Africa. Ann Med Psychol (Paris). 2022; 180: 23-28. [CrossRef]
- McMillan D, Gilbody S, Beresford E, Neilly LIZ. Can we predict suicide and non-fatal self-harm with the Beck Hopelessness Scale? A meta-analysis. Psychol Med. 2007; 37: 769-778. [CrossRef]
- Algorta GP, Dodd AL, Stringaris A, Youngstrom EA. Diagnostic efficiency of the SDQ for parents to identify ADHD in the UK: A ROC analysis. Eur Child Adolesc Psychiatry. 2016; 25: 949-957. [CrossRef]
- Trevethan R. Sensitivity, specificity, and predictive values: Foundations, pliabilities, and pitfalls in research and practice. Front Public Health. 2017; 5: 307. [CrossRef]
- Youngstrom EA. A primer on receiver operating characteristic analysis and diagnostic efficiency statistics for pediatric psychology: We are ready to ROC. J Pediatr Psychol. 2014; 39: 204-221. [CrossRef]
- Padmanabhanunni A, Pretorius TB, Isaacs SA. We are not islands: The role of social support in the relationship between perceived stress during the COVID-19 pandemic and psychological distress. Int J Environ Res Public Health. 2023; 20: 3179. [CrossRef]
- Padmanabhanunni A. Trauma nation: Exposure to traumatic events among South African university students. J Community Psychol. 2020; 48: 1853-1862. [CrossRef]
- Kantamneni N. The impact of the COVID-19 pandemic on marginalized populations in the United States: A research agenda. J Vocat Behav. 2020; 119: 103439. [CrossRef]
- Rudenstine S, McNeal K, Schulder T, Ettman CK, Hernandez M, Gvozdieva K, et al. Depression and anxiety during the COVID‐19 pandemic in an urban, low-income public university sample. J Trauma Stress. 2021; 34: 12-22. [CrossRef]
- Henry SK, Grant MM, Cropsey KL. Determining the optimal clinical cutoff on the CES-D for depression in a community corrections sample. J Affect Disord. 2018; 234: 270-275. [CrossRef]
- Campo-Arias A, Díaz-Martínez LA, Rueda-Jaimes GE, Cadena-Afanador LdP, Hernández NL. Psychometric properties of the CES-D scale among Colombian adults from the general population. Rev Colomb Psiquiatr. 2007; 36: 664-674.
- Thomas JL, Jones GN, Scarinci IC, Mehan DJ, Brantley PJ. The utility of the Ces-D as a depression screening measure among low-income women attending primary care clinics. Int J Psychiatry Med. 2001; 31: 25-40. [CrossRef]
- Quiñones AR, Thielke SM, Clark ME, Phillips KM, Elnitsky C, Andresen EM. Validity of center for epidemiologic studies depression (CES-D) scale in a sample of Iraq and Afghanistan veterans. SAGE Open Med. 2016; 4: 2050312116643906. [CrossRef]
- Kuehner CP. Why is depression more common among women than among men? Lancet Psychiatry. 2017; 4: 146-158. [CrossRef]