Assessment of Loneliness During the Pandemic: Comparing Various Short Forms of the UCLA Loneliness Scale in South Africa Using Classical Test Theory and Mokken Analysis

Tyrone Pretorius ^†,*, Anita Padmanabhanunni ^†

Department of Psychology, University of the Western Cape, Robert Sobukwe Road, Cape Town, South Africa

† These authors contributed equally to this work.

* Correspondence: Tyrone Pretorius

Academic Editors: Ami Rokach and David Berman

Special Issue: The Many Shades of Loneliness

Received: May 01, 2022 | Accepted: August 09, 2022 | Published: August 17, 2022

OBM Neurobiology 2022, Volume 6, Issue 3, doi:10.21926/obm.neurobiol.2203132

Recommended citation: Pretorius T, Padmanabhanunni A. Assessment of Loneliness During the Pandemic: Comparing Various Short Forms of the UCLA Loneliness Scale in South Africa Using Classical Test Theory and Mokken Analysis. OBM Neurobiology 2022;6(3):22; doi:10.21926/obm.neurobiol.2203132.

© 2022 by the authors. This is an open access article distributed under the conditions of the Creative Commons by Attribution License, which permits unrestricted use, distribution, and reproduction in any medium or format, provided the original work is correctly cited.

1. Introduction

During the COVID-19 pandemic, governments around the world implemented stringent lockdown measures making social isolation and remote work mandatory. Although these regulations were necessary to prevent the spread of the virus, they significantly deteriorated mental health conditions, characterized by increased rates of depression, anxiety, and loneliness. Loneliness was the most prevalent mental health issue associated with COVID-19 [1], which led the COVID-19 pandemic to be known as the “loneliness pandemic” [2]. Loneliness is an uncomfortable emotional state arising from the perception that the social needs of an individual are not being fulfilled [3]. Some studies have empirically shown that the prevalence of loneliness has increased in the general population. For example, a study in the United Kingdom [4] reported a loneliness prevalence rate of 27% in 2020, while another study [5] found that 28.63% and 7.20% of the participants reported feeling lonely sometimes and often respectively, in 2020. A study based on six Middle Eastern countries conducted in 2021 reported that 40.6% of the respondents expressed a moderately high degree of loneliness [6]. Conversely, one study found no significant changes in loneliness during various phases of the pandemic [7]. Some studies have investigated the trajectory of loneliness during the pandemic [8,9]. Others have reported an increase in the prevalence of loneliness in the general population at the onset of the COVID-19 outbreak, with especially high rates among people under quarantine [10]. A study also found a decrease in the level of loneliness by the first year of the pandemic [11]. This might be due to several factors, including a decrease in compliance with social distancing and quarantine measures and an increase in the use of technology to connect with others [11,12]. Some studies focused on identifying sub-groups of the population that might be more susceptible to loneliness, including adolescents [8], the elderly [13], and university students [14], as well as the correlates of loneliness. Changes in the romantic status during the pandemic, loss of a significant other due to infection, young individuals, living alone, and pre-existing mental health problems are associated with the risk of loneliness [12].

The Evolutionary Theory of Loneliness [15] provides a framework for understanding the increase in the perception of loneliness during the pandemic. It states that people have an innate need for social connections as it provides protection and helps people in managing adversities. In the absence of social connections, people feel lonely. When people feel lonely, they seek social contact; however, during the pandemic, a social connection was considered to be a potential threat due to the risk of infection. Additionally, the need to follow government guidelines regarding social distancing and quarantine prevented people from contacting others. Keller et al. [16] suggested that this negative feedback loop where social contact is considered to be dangerous and the resultant social withdrawal can produce chronic feelings of loneliness, characterized by a feeling of isolation and lack of meaningful connections [16].

The importance of conducting studies on loneliness has been highlighted by pre-pandemic studies, where loneliness was found to be associated with adverse mental health outcomes, including depression and anxiety, and high mortality [11]. Similar results were found during the pandemic. Specifically, loneliness was found to be associated with significant psychological distress, lower engagement in COVID-19-related prevention measures, and an increase in maladaptive coping mechanisms, including substance use [12,17]. Thus, further studies on loneliness and its effect on mental and physical health are necessary for developing COVID-19 public health interventions.

To determine the effects of pandemic-related loneliness, its correlates, and the protective factors that might mitigate the problems, a detailed investigation using robust instruments with good psychometric properties is necessary for research and clinical purposes. Additionally, the metric equivalence of instruments across different contexts needs to be established. This might help to determine valid inter-group differences.

The most widely used measure of loneliness is the University of California, Los Angeles Loneliness Scale (UCLA-20). This scale has three versions; the original version was published in 1978 [18], in which all items were negatively phrased; a revised version was published in 1980 and contained 10 positive and 10 negative items [19]; finally, a third version was published in 1996 in which the response format was modified, and the scale consisted of 11 negative items and nine positive items [20]. Authors have generally provided satisfactory reliability and validity indices for the different versions of the UCLA-20 [18,19,20]. Similar satisfactory reliability and validity indices have been reported in different countries, including Iran [21], Poland [22], Japan [23], Spain [24], and Denmark [25]. However, although the psychometric properties were satisfactory, the results regarding the dimensionality and factor structure of the UCLA-20 were inconsistent. Russel considered the UCLA-20 to be unidimensional and showed through factor analysis that the UCLA-20 assesses a global bipolar measure of loneliness with two uncorrelated method factors, in which the negative items loaded on one factor and the positive items loaded on another factor [20]. Other studies have found mixed results. One-factor [25], two-factor [26], three-factor [27,28], and bifactor solutions [29,30] have been reported.

One disadvantage of the UCLA-20 is its relative length. The scale is seldom used independently as researchers are often interested in the correlates of loneliness. When used along with other instruments, the length of the UCLA-20 can lead to a questionnaire burden on the respondents, as well as erratic responses, where the respondents respond without paying close attention to the questions. To overcome this problem, several shorter versions of the UCLA-20 have been developed, including a three-item version [31], a four-item version [19], a five-item version [32], a six-item version [33], a seven-item version [34], an eight-item version [35], a 10-item version [20], an 11-item version [36], and a 16-item version [37].

Das et al. [38] reviewed the loneliness scales used in pandemics and found that the UCLA-3, UCLA-8, and UCLA-10 were the three short versions of the UCLA-20 most frequently used, specifically in 56% of the studies they reviewed. In this study, we investigated the psychometric properties of three short versions of the UCLA Loneliness Scale, including two versions previously used, i.e., the 10-item version [20] and the eight-item version [35], as well as a five-item version developed in this study. We used the classical test theory and item response theory (Mokken analysis) in two different groups of samples, one including teachers and the other including students. Assessing measurement equivalence across different population groups is important [39,40]. Similar coherence or structure of the psychometric properties of data from multiple groups provides strong evidence of metric equivalence. This is the first study from South Africa to use the Mokken analysis to investigate the psychometric properties of the UCLA loneliness scale.

2. Materials and Methods

2.1 Participants

The participants consisted of a random sample of students from a university in Cape Town, South Africa (N = 355) and a convenience sample of teachers (N = 337) from different parts of South Africa. Most participants in the student sample were women (77.2%) and resided in an urban area (75.4%); the mean age of the student sample group was 21.95 years (SD = 4.7). The participants in the teacher sample group were mainly based in the Western Cape province of South Africa (82.3%) and resided in urban areas (61.7%). Most teachers were women (76.6%), and the mean age of the group was 41.89 years (SD = 12.42).

Both studies were approved by the Institutional Review Board of the University of the Western Cape (Ethics reference number: students, HS20/5/1, 1 June 2020; teachers: HS21/3/8, 14 May 2021). The studies were conducted following the guidelines of the Declaration of Helsinki. Participation was voluntary, and the participants provided informed consent on the landing page of the link to the survey.

2.2 Instruments

The instruments used to conduct this study included the UCLA Loneliness Scale (UCLA-20] [20], the trait scale of the State-Trait Anxiety Inventory [STAI-T] [41], the Beck Hopelessness Scale [BHS] [42], the Center for Epidemiological Studies Depression Scale [CES-D] [43], and the Satisfaction with Life Scale [SWLS] [44]. These instruments were used to assess the concurrent validity of the UCLA short-form scales. Additionally, participants completed a brief demographic survey.

The UCLA-20 is commonly used to measure loneliness [45]. It consists of 20 items with a four-point scale ranging from 1 (Never) to 4 (Often). The estimates of internal consistency range from 0.88–0.93, as found in various studies [10,11,12,13,14,21,22,23,24,25]. Two South African studies reported acceptable reliability (0.81 and 0.92) for the UCLA-20 when used with student samples [46,47]. The structure of the UCLA-20 was also confirmed using confirmatory factor analysis (CFA) with ancillary bifactor indices in a South African study [30].

The STAI-T is a 20-item measure of trait anxiety with a four-point scale ranging from “Almost Never” (1) to “Almost Always” (4). An example item of the STAI-T is, “I feel that difficulties are piling up so that I cannot overcome them”. A recent meta-analysis showed that the STAI has been cited in more than 17,000 papers [48]. The STAI-T generally demonstrates satisfactory reliability ranging from 0.87–0.93 [49]. The scale was also used in South Africa [50] and showed an internal consistency estimate of 0.90.

The BHS is a widely used 20-item measure of hopelessness. All 20 items are responded to on a dichotomous scale of “True” or “False”. An example item of the BHS is, “I cannot imagine what my life would be like in 10 years”. The BHS generally shows adequate reliability, with estimates ranging from 0.89–0.98 [51,52,53]. The BHS was used in two studies on young adults in South Africa, and the reliability estimates of 0.88 and 0.82 were reported in those studies [54,55].

The CES-D is a 20-item measure of depressive symptoms and uses a four-point scale ranging from “Most or all of the time” (0) to “Rarely or none of the time” (3). An example item of the CES-D is, “I could not get going”. The CES-D has been translated into several languages, including Farsi [56], Chinese [57], German [58], and multiple Nigerian languages [59], and the reliability estimates are generally above 0.75. The CES-D was used in two studies in South Africa, with reliability estimates of 0.90 [60] and 0.92 [61].

The SWLS is a five-item measure of life satisfaction. It is scored on a seven-point scale ranging from 1 (Strongly disagree) to 7 (Strongly agree). An example item of the SWLS is, “If I could live my life over, I would change almost nothing”. The reliability and validity of the SWLS are satisfactory [62,63,64], with reliability coefficients ranging from 0.86–0.91. The SWLS has also shown satisfactory reliability in a South African study [40].

2.3 Procedure

To conduct both studies, we used Google Forms to develop a web-based version of the instruments. The student sample was randomly selected (95% confidence interval, 5% margin of error) through the Registrar’s office. For this purpose, the Registrar’s office used an algorithm on the email addresses of students that generated a random number for each email address. The link was emailed to the selected students, and reminders were sent twice per month for four months. The teacher participants were recruited by convenience sampling. An electronic link was posted on several Facebook sites for the teachers.

2.4 Data Analysis

The classical test theory analysis is the traditional way in which the psychometric properties of the UCLA Loneliness Scale are examined. IBM SPSS Statistics (version 28) was used for the classical test theory analysis, which included inter-item correlations, item-total correlations, exploratory factor analysis (EFA), descriptive statistics, and reliabilities. Inter-item correlations should range from 0.15 to 0.85 [65]; an inter-item correlation higher than 0.85 indicates redundancy of items. Item-total correlations indicate the extent to which each item contributes to the measurement of the latent construct and should be greater than 0.50 [66]. IBM SPSS Amos version 27 (IBM Corp.) was used to perform a CFA. The indices that were used to examine model fit in the CFA [67] included chi-squared (χ², good fit: p > 0.05), the goodness-of-fit index (GFI, good fit: >0.95), the Tucker-Lewis index (TLI, good fit: >0.90), the comparative fit index (CFI, good fit: >0.90), and the root-mean-square error of approximation (RMSEA, good fit <0.08). To support the CFA, ancillary bifactor indices were calculated using the Bifactor Indices Calculator [68], which included OmegaH, the explained common variance (ECV), and the percentage of uncontaminated correlations (PUC). The ECV describes the percentage of variance in the items accounted for by the total scale (also called the general factor) and the subscales (also called the specific factors). OmegaH indicates the proportion of variance in total scores that is explained by the total scale, and PUC describes the percentage of correlations between items that can be explained by the total scale [69,70]. While there are specific criteria for these indices, Reise et al. [71] suggested that they should be considered together. When PUC values are below 0.80, ECV is above 0.60, and OmegaH is above 0.70, the scale can be considered unidimensional.

Since the four-point response format of the UCLA loneliness scale is a weak ordinal scale, Mokken analysis, which is a non-parametric item-response theory (IRT) approach, was also used to examine the psychometric properties of the loneliness scale. IRT approaches are less concerned with total scores and focus more on the responses to individual items [72]. Mokken analysis was conducted using the “Mokken” package [73] in R [74]. Mokken analysis is a non-parametric alternative to the parametric item response theory, such as Rasch analysis. Mokken analysis provides a scalability coefficient for each item (H_i), as well as for the total scale (H). The coefficient H_i indicates the contribution of every item to the measurement of the latent trait. Mokken [75] suggested that an H_i coefficient greater than 0.30 indicates that the item contributes to the measurement of the scale. The H coefficient for the entire scale provides a measure of the homogeneity of items and indicates the strength of the scale. Sijtsma and van der Ark [76] suggested that H <0.40 indicates a weak scale, H ≥0.40 and <0.50 indicates a moderate scale, and H ≥0.50 indicates a strong scale. The Mokken analysis also provides an automated item selection procedure (AISP), which indicates whether a scale is unidimensional or multidimensional. To determine whether items can distinguish between high and low scorers on the latent trait (monotonicity), the Mokken analysis provides a Crit value for each item. Finally, another Crit value is used to identify items that respondents with the same standing on the latent trait might have responded to differently (invariant item ordering or IIO). For both monotonicity and IIO, Crit values >80 indicate serious violations [76].

Other types of validity of the investigated short-form scales include convergent, discriminant, and concurrent validity. The average variance extracted (AVE) and composite reliability (CR) were used to assess convergent validity. The AVE reflects the level of variance explained by the latent construct, as opposed to the variance attributable to the measurement error; CR is considered to be a less biased measure of reliability than Cronbach’s alpha. In general, AVE above 0.50, CR above 0.70, and AVE less than CR indicate convergent validity [77]. Discriminant validity was assessed by comparing the variance in the items accounted for by the latent construct to the variance the construct shares with other constructs. Discriminant validity occurs if the latent construct accounts for greater variance in the items that contribute to its measurement (AVE) than the variance shared with related constructs (maximum shared variance or MSV; average shared variance or ASV), such as anxiety and depression [78]. Finally, concurrent validity was assessed based on correlations with other indices of psychological well-being, including anxiety, hopelessness, depression, and life satisfaction.

3. Results

The reliabilities of the five scales used in the study were as follows: UCLA-20: α = 0.92 for teachers and students; STAI-T: α = 0.91 for teachers and α = 0.90 for students; BHS: α = 0.89 for teachers and α = 0.88 for students; CES-D: α = 0.92 for teachers and students; SWLS: α = 0.90 for teachers and α = 0.89 for students. The inter-item correlations for the 10-item UCLA for the sample comprising teachers are reported in Table 1.

Table 1 The descriptive statistics and inter-item correlations for the UCLA-10 for the samples comprising teachers and students.

The descriptive statistics and inter-item correlations for the UCLA-8 are reported in Table 2.

Table 2 The descriptive statistics and inter-item correlations for the UCLA-8 for the samples comprising teachers and students.

All inter-item correlations for the UCLA-10 were within the acceptable range of 0.15–0.85 [52] in the sample comprising teachers. In the sample comprising students, all inter-item correlations were within the acceptable range except for the intercorrelation between items 9 and 10. The intercorrelation between the items “There are people I can talk to” (item 9) and “People I can turn to” (item 10) was 0.87, which indicated redundancy between these items.

All inter-item correlations for the UCLA-8 were within an acceptable range for the sample comprising teachers except for the intercorrelation between items 4, 6, and 7. The intercorrelations between item 6 (“I can find companionship”) and both item 4 (“I feel left out”) and item 7 (“I feel isolated from others”) were below 0.15. In the sample comprising students, the only intercorrelation below 0.15 was between items 6 and 7. These findings suggested that items 6 and 7 might not contribute to the homogeneity of the items.

The UCLA-10 and UCLA-8 were subjected to EFA (principal components with varimax rotation) for the samples comprising teachers and students. The results of the EFA for the sample comprising teachers are reported in Table 3. The results were similar for the sample comprising students, and thus, only the results of the sample comprising teachers are reported here.

Table 3 Factor analysis of the short forms of UCLA for the sample consisting of teachers.

The two factors explained 59.27% and 58.64% of the variance in the eight-item version of the scale for students and teachers, respectively. In the case of the 10-item version, the two factors explained 66.16% and 64.99% of the variance for students and teachers, respectively. All positive items loaded on one factor and all negative items loaded on another factor. To determine whether these are substantive factors that account for a considerable proportion of variance, a CFA with ancillary bifactor analysis was conducted. The UCLA-8 bifactor model is shown in Figure 1A, and the UCLA-10 bifactor model is shown in Figure 1B.

Figure 1 The bifactor models of UCLA-10 and UCLA-8. The regression weights were standardized. Rectangles indicate measured variables, and ellipses indicate latent variables.

The model fit indices showed that the two models fit the data very well. For both UCLA-8 and UCLA-10, the fit indices were GFI = 0.98, TLI = 0.99, CFI = 0.99, and RMSEA = 0.03. For UCLA-8, χ² = 16.84 and p > 0.05, and for UCLA-10, χ² = 31.40 and p > 0.05. Despite the well-fit indices, the pattern of loadings was problematic in both models. There were negative pattern loadings and factor loadings below the generally acceptable level of 0.70 [65]. In the UCLA-8, the loadings on all subscales were non-significant, but all were significant on the total scale (p < 0.01). However, items 3 (“I am an outgoing person”), 6 (“I can find companionship”), and 7 (“I am unhappy about being withdrawn”) had loadings less than 0.50 on the total scale. Along with these problematic loadings, the ancillary bifactor indices did not support the interpretation of these two versions of the UCLA-20 as multidimensional. The ECV of the total scale of UCLA-8 and UCLA-10 were 0.71 and 0.66, respectively, for the sample comprising teachers. The OmegaH was 0.75 for both versions, and the PUC was 0.43 and 0.56 for UCLA-8 and UCLA-10, respectively. The sample comprising students had similar indices. The finding that PUC was below 0.80, ECV was above 0.60, and OmegaH was above 0.70 indicated that there was some multidimensionality but not enough to rule out the interpretation that the scale was essentially unidimensional [71].

For the UCLA-10, the AISP found that all 10 items loaded on one scale in the samples comprising teachers and students, which indicated that the scale was a unidimensional measure of loneliness. However, for the UCLA-8, AISP found that the items loaded on two scales in the samples comprising teachers and students; hence, the eight-item version was not unidimensional. The Mokken and classical test theory indices for the items of the short-form scales are reported in Table 4.

Table 4 The Mokken and classical test theory indices at the item level for the short-form scales.

The UCLA-10 had acceptable indices at the item level in both samples, as shown in Table 4. All item-total correlations were above 0.50, and all H_i coefficients were greater than 0.30. These item-total correlations and H_i coefficients suggested that all items of the UCLA-10 contributed to the measurement of the latent construct. All Crit values for both monotonicity and IIO were below 80, indicating minor but acceptable violations. For the UCLA-8, items 3 (“I am an outgoing person”), 6 (“I can find companionship”), and 7 (“I am unhappy about being withdrawn”) had item-total correlations below 0.50 and H_i coefficients below 0.30, indicating that these items did not contribute to the measurement of the latent construct. These items were also identified by the factor loadings in CFA as misfitting items. The inter-item correlations also showed that items 6 and 7 did not contribute to the homogeneity of the items. Additionally, there were two violations of monotonicity and four violations of IIO for the UCLA-8 (i.e., Crit value > 80 in the sample comprising teachers and three violations of IIO in the sample comprising students). After removing these misfitting items, a five-item version of the scale was obtained that had acceptable indices at the item level in both samples, as shown in Table 4; the item total correlations were above 0.50, the H_i coefficients were above 0.30, and the violations of monotonicity and IIO were minor.

The Mokken and classical test theory indices for the three versions of the UCLA-20 at the scale level in the two groups of samples are presented in Table 5. The column labeled “Criteria” indicates the acceptable levels for these indices based on previous studies.

Table 5 The Mokken and classical test theory indices for the UCLA short-form scales at the scale level and correlations with related variables.

As shown in Table 5, all scale indices met the suggested cut-offs for satisfactory indices for the UCLA-10 and UCLA-5 in both groups of samples, except for the H coefficient of the UCLA-10 in the sample comprising students (H = 0.49), which indicated that it is a moderate scale. However, the H coefficient of the UCLA-8 for the sample comprising teachers indicated that it is a weak scale, whereas the H coefficient for the sample comprising students indicated that it is a moderate scale. In both samples of the UCLA-8, the AVE was below the suggested cut-off (>0.50), which reflected that the latent construct accounted for an insufficient proportion of variance in the eight items of the scale. All scales were satisfactorily reliable (0.81–0.91).

Since the results suggested that the five-item and 10-item versions were unidimensional, a CFA was used to further confirm this using a one-factor model. For the five-item version the fit indices were GFI = 0.95, TLI = 0.87, CFI = 0.95, and SRMR = 0.04 and for the 10-item version the fit indices were GFI = 0.96, TLI = 0.95, CFI = 0.98, and SRMR = 0.04, indicating acceptable fit.

The correlations between the UCLA-20 and the indices of psychological well-being were as follows: anxiety, r = 0.65 and p < 0.001; hopelessness, r = 0.55 and p < 0.001; depression, r = 0.65 and p < 0.001; life satisfaction, r = −0.53 and p < 0.001. As shown in Table 5, the various short-form scales had similar correlations with these indices for the two groups of samples (UCLA-10: 0.50–0.61; UCLA-8: 0.47–0.70; UCLA-5: 0.40–0.63).

4. Discussion

Although loneliness was considered to be a salient public health concern even before the COVID-19 outbreak, primarily due to its effect on mental and physical outcomes, the pandemic and its containment measures increased social isolation and loneliness. Many studies conducted in different contexts and among various population groups reached a similar conclusion [10,13,14]. Researchers generally conceptualize loneliness as either unidimensional (i.e., a unitary phenomenon that varies in perceived intensity but is similar across situations) or multidimensional (i.e., a multifaceted phenomenon that cannot be captured by a single global loneliness measure). Whether the unidimensional or multidimensional scale provides a more robust measurement of loneliness is not clear [29]. Multidimensional scales help to determine variations in the subjective experience of loneliness and might benefit targeted intervention; however, relatively more studies have investigated unidimensional measures of loneliness [29,79].

Although loneliness can be measured using different scales, for example, the Social and Emotional Loneliness Scale for Adults [80] and the Differential Loneliness Scale [81], consensus on the factorial structure of most of these scales or the factorial structure of each of their proposed dimensions is limited. Most analyses of loneliness scales are based on Cronbach’s alpha for scale reliability, the use of correlation to establish discriminant and construct validation, as well as exploratory factor analyses for developing the scale [29,82]. The UCLA loneliness scale is the most popular measure of loneliness globally [29]. This is mainly due to its unidimensional structure, short length, and satisfactory internal consistency reliability. However, the UCLA scale lacks a conceptual definition of loneliness, and the items included in the scale are based on face validity and inter-item correlation rather than a common conceptual understanding of the construct of loneliness [82].

In this study, we compared three short versions of the UCLA Loneliness Scale to extend the conceptual and theoretical approaches on which these instruments are grounded. The three scales compared were a 10-item version proposed by Russel [20], an eight-item version proposed by Hays and DiMatteo [35], and a five-item version developed in this study through the Mokken analysis. The reliability of all short-form scales exceeded 0.80 in the two sets of samples; thus, the reliability of these short-form scales was highly satisfactory.

Russel [20] proposed that loneliness is a unidimensional construct. For the UCLA-10 and UCLA-8, the EFA and CFA indicated that the items loaded on two factors. However, an ancillary bifactor analysis showed that the two subscales did not sufficiently account for the variance compared to that accounted for by the total scale; thus, the scale can be assumed to be unidimensional. The two-factor solution reflected a method artifact as all the positive items loaded on one factor and all the negative items loaded on another factor. Such an interpretation of unidimensionality in the case of the UCLA-8 was consistent with the findings of Hays and DiMatteo [35], who used EFA to identify the eight-item version of the scale. In the Mokken analysis, the AISP identified one scale underlying the 10-item and five-item versions; however, it identified that the items of the eight-item scale loaded on two scales. Additionally, unlike the UCLA-10 and UCLA-5, the H coefficient in the Mokken analysis indicated that the eight items of the UCLA-8 formed a weak scale. Thus, the results do not support a unidimensional interpretation of the UCLA-8. Our findings were consistent with those of Elphinstone [83] that the UCLA-8 did not have acceptable fit indices in a CFA analysis. However, this finding did not support the unidimensional structure reported in other studies [84,85].

The findings of item-total correlation greater than 0.50, inter-item correlations from 0.15 to 0.85, and H_i greater than 0.30 provided satisfactory support for the construct validity of the UCLA-10 and UCLA-5. Further support for construct validity was found based on acceptable Crit values for monotonicity and IIO. The items in these two short-form scales could differentiate between high and low scorers on the latent trait (monotonicity), and all items were treated similarly by respondents with the same level of the latent trait (IIO). The construct validity of the UCLA-8 was not strongly supported in this study. Two items in UCLA-8 had inter-item correlations below 0.15, three items had factor loadings below 0.60 on the total scale based on the CFA, and the H_i coefficients were below 0.30. Additionally, the UCLA-8 violated several assumptions of monotonicity and IIO, indicating that some items could not differentiate between high and low scorers, and some items were responded to differently by respondents with the same level of latent construct. The finding that the UCLA-10 demonstrated greater construct validity than the UCLA-8 were similar to the findings of a systematic review of the UCLA short-form scales, which indicated that high-quality evidence supported the internal structure of the UCLA-10, whereas the evidence supporting the construct validity of the UCLA-8 was of low-to-moderate quality [86].

The UCLA-10 and UCLA-5 demonstrated good convergent validity (AVE ≥ 0.50, CR > 0.70, and AVE < CR); however, in the case of the UCLA-8, the AVE indicated that the latent construct accounted for insufficient variance in the eight items. First, all short-form scales had satisfactory discriminant and concurrent validity. In all three short-form versions, the latent construct explained the variance in the items that contributed to its measurement (AVE) more than the variance they shared with other related constructs, including anxiety, hopelessness, depression, and life satisfaction (MSV and ASV). Second, the three short-form versions were significantly correlated with other indices of psychological well-being in the expected directions. These results supported the findings of other studies regarding the concurrent validity of the short-form scales [23,84,85,87].

More studies are needed to determine the dimensionality and validity of the UCLA-8. In this study, the UCLA-10 and UCLA-5 were identified as unidimensional measures of loneliness with satisfactory reliability and validity. The UCLA-5 might be a useful measure of loneliness when the longer 10-item or 20-item versions are not suitable, and this scale might be especially useful for telephone surveys. A limitation of the five-item version is that it only contains negative items, which makes it difficult to identify when participants exhibit acquiescence bias (i.e., they consistently select only one anchor of a scale). However, the advantages of including both positive and negative items do not necessarily outweigh the disadvantages of potential mistakes by respondents (e.g., accidentally agreeing with a negative item) or potential miscoding by the researcher, for example, failing to reverse score items [71,88]. A limitation of the study was stimulus familiarity, considering that the participants were only exposed to the 20-item version of the UCLA and not to the three separate versions, and this might lead to instrument bias.

5. Conclusions

Loneliness was described as the “signature” mental health issue of the COVID-19 pandemic. The reliable and valid assessment of loneliness should be prioritized. To avoid questionnaire burden and instrument fatigue, as well as prevent respondents from answering randomly to items due to the length of the questionnaire, short forms of the instrument to measure loneliness, such as the UCLA-10, should be investigated. A short-form instrument should have satisfactory reliability and validity that are similar to those of the original 20-item version. The classical test theory and the Mokken analysis provided satisfactory support for the reliability and validity of the UCLA-10 and UCLA-5 when used in South Africa. The UCLA-8 demonstrated satisfactory reliability regarding internal consistency; however, the validity was unsatisfactory. This was the first application of the UCLA-5, and further studies with more participants from different populations are needed to confirm the reliability, validity, and usability of this shorter version of the UCLA-20. The reliability and validity indices of the five-item version were satisfactory and comparable to those of the 10-item version in this study. The complementary evidence showing that the UCLA-5 had satisfactory psychometric properties from two different perspectives, including the classical test theory and the item response theory, provided further support for using the UCLA-5 as a suitable short-form version of the UCLA-20.

Author Contributions

Conceptualization and methodology: Anita Padmanabhanunni & Tyrone Pretorius; Analysis and data curation: Tyrone Pretorius. All authors discussed and interpreted obtained results. All authors contributed to the writing and editing of the manuscript over several iterations.

Competing Interests

The authors have declared that no competing interests exist.