The Trails Making Test. Does a Single Trial Reflect Performance Capability?

Kate McKeown ¹, Emma Richards ², Jessica Richardson ¹, Andrea Tales ^2,*

1. Department of Psychology, Swansea University, Swansea

2. Centre for Innovative Ageing, Swansea University, Swansea

* Correspondence: Andrea Tales

Academic Editor: Bart Ellenbroek

Special Issue: Brain and Visual Processing

Received: January 08, 2021 | Accepted: May 30, 2021 | Published: June 07, 2021

OBM Neurobiology 2021, Volume 5, Issue 2, doi:10.21926/obm.neurobiol.2102100

Recommended citation: McKeown K, Richards E, Richardson J, Tales A. The Trails Making Test. Does a Single Trial Reflect Performance Capability?. OBM Neurobiology 2021; 5(2): 100; doi:10.21926/obm.neurobiol.2102100.

© 2021 by the authors. This is an open access article distributed under the conditions of the Creative Commons by Attribution License, which permits unrestricted use, distribution, and reproduction in any medium or format, provided the original work is correctly cited.

1. Introduction

The trail making test (TMT) [1,2,3] is a one-trial, time-pressured neuropsychological test widely used to assess executive function-related information processing speed (reaction time, RT) across the lifespan. It is used as part of the assessment, diagnosis and monitoring of many clinical conditions, particularly those related to older age such as dementia, and is applied to functional assessments such as those related to driving ability [4]. In brief, Trails A of the TMT requires the participant to draw a continuous line joining a series of circled numbers in ascending order on a sheet of paper. Trails B, a task of greater difficulty and resource demand, entails the drawing of a continuous line joining a series of both circled numbers and letters alternately in ascending numerical and alphabetical order. For both sections, participants are instructed to perform the task as quickly but as accurately as possible, with test outcome the time taken (in seconds) for completion.

Whilst recognising the clinical utility of such a single trial (which allows results to be gained rapidly), and the fact that its single trial use has been validated, there are disadvantages to this approach. For example, the output from one trial of any test can be susceptible to spurious and extraneous effects, which if not taken into account, can mean that it fails to represent true performance. A one-trial test also precludes the examination of the functional integrity of other aspects of information processing such as practice effects which in turn can influence or reflect performance at cognitive, behavioural and neuropsychological levels [5]. Although it is well known that practice effects can influence neurocognitive assessment over both single test sessions and longitudinal studies and therefore need to be taken into account in [6,7,8,9,10,11], their potential for useful markers of cognitive decline and progression to dementia [7,8,9,12,13] is less well investigated. As there is evidence to suggest some degree of improvement in TMT performance with practice [14,15,16] and that TMT-B performance can be enhanced due to prior TMT-A exposure [17,18], the clinical usefulness of the TMT may be enhanced by a multi-trial approach, even during a single test session. Furthermore, as we describe below, administering more than one trial in a given testing session may also reveal the contribution to the information processing speed (RT) measure of potential spurious effects related to the use of a single trial.

Under typical administrative conditions, following a short practice trial consisting of a very small section of the whole test, the A and B sections of the TMT are administered as two single trials. It has been noticed by the authors with respect to both clinical and research application, that some individuals performing the TMT appear hesitant, or very slow in moving off from the starting point once instructed to do so (and having started timing), despite the prior practice session. We suggest that this means that the outcome of the single TMT test may contain a hesitancy effect that may affect the test’s ability to fully reflect a person’s functionality. This may be of particular relevance when information processing speed derived from the TMT is used as a component in the diagnosis of dementia and mild cognitive impairment), with disproportionate slowing (compared to healthy ageing for example) indicative of disease.

This hesitancy effect resembles the objectively measurable phenomenon called the ‘first response hesitancy effect’ found in some studies of sequential multi-trial processing [19,20,21], in which the response to the first in a sequence of test components can be significantly slowed compared to subsequent responses. This is an effect capable of significantly influencing information RT outcome in research studies [21] and appears related, at least in part, to set-up time for registering a new visual layout, response preparation, strategy development and natural variation in such components [19,20,21]. It may be the case that a similar effect occurs in response to the single trial (essentially a ‘first trial’) of Trails A and B resulting in an outcome measure that includes an extraneous hesitancy component that results in a slower information processing speed compared to that obtained if further trials had been administered, over which the effect is known to disperse [19,20,21]. The first response hesitancy effect that can have a significant detrimental effect upon information processing speed derived from a single trial of a test, yet this effect is not taken into account in either the analysis or the interpretation of TMT data. The fact that the TMT result is based only on a single, or first test, means that it may reveal more about hesitancy than speed of information processing per se and failure to take the potential influence of such effects into account possibly results in the misrepresentation of an individual’s actual level of function.

Furthermore, a single trial measure of a behaviour as complex as information processing speed, precludes the measurement of potentially important clinical information related to an individual’s range of performance during a given period of time; a behaviour which like longer-term practice effects, may also be indicative of the functional integrity of underlying brain processes. Failure to take account of this effect may result in the determination of person’s information processing speed performance as significantly slower (poorer) than it actually is, i.e., potentially causing poor performance to be mis-attributed to pathological change for example. In this study we administer 4 trials of the TMT to young adults to determine if within a single session, giving more trials over a short period of time may influence the outcome of such a test. This will help to determine whether as is current clinical practice, the first or single measure, only should be used as the marker of the functional integrity of an individual’s executive function-related information processing speed.

In the authors’ experience, after completion of the TMT it is common for some individuals to spontaneously report that they had felt self-conscious and anxious about performing in front of the test administrator. This anecdotal evidence is backed up by empirical data showing some evidence that self-consciousness, especially related to being watched, can modulate performance on cognitive tests [22,23,24,25,26]. It is possible that such anxiety and self-consciousness may contribute to the hesitancy sometimes observed at the beginning of the Trails A and B components of the TMT, thus whether TMT RT is correlated with anxiety and self-consciousness across the four trials is also examined.

1.1 Study Aims

The main aim of this study is to investigate how, within a single research session lasting no longer than thirty minutes (thus mimicking a typical clinical testing visit and one in which the potential for fatigue effects is minimized), information processing speed outcome may change with respect to the administration of four trials of the TMT-A and TMT-B components of the TMT. We hypothesize that there will be a ‘hesitancy effect’ in which the response to the first trial of both Trails A and Trails B, is slower than that in response to subsequent trials. We examine these effects in a group of healthy younger adults in order to examine performance in participants who typically form the control group in ageing studies. It also acted as a proof of concept study which would allow us to determine whether adding additional trials to the TMT paradigm might increases its ability to provide information about the functional integrity of other aspects of behavioral and brain processing, which may improve its clinical utility and its applicability to testing throughout the lifespan and in clinical populations.

A further aim is to determine whether the potentially extraneous factors of self-consciousness and anxiety are associated with any such hesitancy effect. Finally, we examine whether any practice effect continues after a ‘task-filled’ interval between the 3rd and 4th trials in which the questionnaires examining anxiety, and self-consciousness are administered.

2. Materials and Methods

2.1 Methods

2.1.1 Participants

The study involved a sample of 55 young adults recruited from the undergraduate student population of Swansea University, between the ages of 18 and 30 years [mean 22.11 years; sd = 3.41, with 31 females and 24 males]. See Table 1 for demographic details. All participants were right-handed and had no self-reported health or cognitive problems or substance or alcohol dependency. All had self-reported normal or corrected-to-normal vision and hearing, and the physical ability to perform the tasks. Participants were reminded to bring their reading glasses/ hearing aids if required. Appropriate vision, hearing and physical ability to perform the tests were confirmed by the administration of practice trials. The study took place in a research testing room within the Department of Psychology at Swansea University. Participants did not receive payment for taking part. Ethical approval was granted by Swansea University Department of Psychology Ethics Committee. All participants gave written informed consent.

2.2 Materials

2.2.1 TMT and Questionnaire Administration and Procedure

The TMT was administered according to standard administration instructions [27]. Trails A, requires participants to use a pen to connect a series of consecutively numbered circles that are presented in a random pattern on a piece of paper, whereas Trails B requires the connection of a series of circled numbers and letters alternately e.g. 1-A, 2-B, 3-C and so on. RT was indicated by the time taken (in seconds) to complete the test (with the time taken to self-correct any error included in the score). All participants completed four trials of the same version of the TMT (that included both parts A and B) in total, with each trial containing both Trails A and Trails B, in that order (according to normal practice [27]). No cut off-scores were used for data inclusion or exclusion. The standard practice run (a truncated version of each task) was administered only before the administration of the first trial. Participants completed 3 trials (of Trails A followed by Trails B) with no break in between (except for the time taken to provide new test papers and re-set the stop watch). On completion of the third trial of Trails A and B, participants were instructed on how to complete the questionnaires described below and, upon completion, the fourth trial or TMT-A and TMT-B was administered.

The researcher (who administered the tests and questionnaires) remained seated beside the participant for the duration of the test session. Participants were not provided with feedback about any aspect of their TMT or questionnaire-related performance. The questionnaires measured anxiety in the two weeks prior to testing, (GAD-7) [28] and self-consciousness (self-report Self-Consciousness Scale (SCS-R) including 3 components; private self-consciousness, public self-consciousness and social self -consciousness [29]. See Table 1 for Questionnaire data.

We did not administer the anxiety and self-consciousness questionnaires at the beginning of the research session, i.e., immediately before the commencement of the TMT task. This was in order to preclude effects associated with the increased awareness of, and thoughts related to, anxiety and self-consciousness that such questionnaires might induce, and which may have influenced the first trial per se. Instead, the tests were administered between the third and fourth trials. This paradigm meant that the information processing speed in response to the first trial could be compared to that of the second and third trials in order to examine practice effects over three trials in the absence of any effects related to the administration of the anxiety and self-consciousness questionnaires. Although practice effects related to repeated testing over weeks, months and years are well known, this paradigm allowed us to determine whether any practice effects continued after an interval filled by the performance of a difference task, namely the two questionnaires, within a single testing session. We recognise the possibility however that administering these tests after the third trial may mean that anxiety and self-consciousness had reduced as a result test familiarity which may in turn have influenced how participants filled in the questionnaires, and also that completing the questionnaires at this point may have influenced performance in response to the fourth trial of Trails A and B.

2.2.2 Data Analysis

For each participant, information processing speed (RT) across 4 trials of Trails A and Trails B of the TMT was compared. For each participant, for each trial, the mean, median, IQR and range of RT was determined. The RT of the first trial was compared with each of the subsequent second, third and fourth trials and changes in RT determined (see Table 2). Change in RT across the trials was examined with respect to the data from the anxiety and self-consciousness questionnaires (see Table 1). Descriptive and inferential statistical analysis were performed using parametric tests. Independent t-tests were used to examine gender differences, analysis of variance was used to examine the effects of practice across trials and correlational analysis was used to examine information processing speed between trials and self-consciousness. All analyses were performed using IBM SPSS statistics version 26.

3. Results

Table 1 Mean demographic and questionnaire data. Standard deviation in parenthesis.

3.1 Gender analysis of GAD-7 and SCS-R data

Independent T-test analysis revealed that females were more publicly and socially self-conscious than males, t [-2.11, p = 0.039] and t [-3.09, p = 0.003] respectively, but there were no other significant differences in gender with respect to age, GAD-7, and private SCS-R scores [all p values > .05].

Figure 1 Group mean RT in seconds over 4 trials for Trails A.

Figure 2 Group mean RT in seconds (Y axis) over 4 trials for Trails B.

Table 2 Group mean difference in reaction time (seconds) between Trial one and trials two, three and four on the Trail Making test A and B.

For Trails A, one way repeated measures analysis of variance (ANOVA) showed a significant main effect of practice (F (3, 162) = 45.67, p < .001; ηp2=.458). Planned Post Hoc pairwise comparisons revealed a significant improvement in RT (2.74 seconds) between trials 2 and 1 (p < .001: 95% CI, lower = 1.068, upper = 4.42); a significant improvement in RT (3.94 seconds) between trials 3 and 1 (p < .001: 95% CI, lower = 2.178, upper 5.71) and a significant improvement in RT (6.33 seconds) between trials 4 and 1 (p < .001: 95% CI, lower = 4.581, upper 8.08). There were no significant gender effects with respect to these results (all p values > .05).

Pearson Correlational analysis for all participants on difference in information processing speed between Trial 1 and 2, between Trial 1 and Trial 3 and between Trial 1 and 4 and self-consciousness revealed no significant relationships [all p values > .05] and similarly for anxiety [all p values > .05]. Although, as noted previously, females were more publicly and socially self-conscious than males, additional correlation between self-consciousness and difference in RT across all trials, resulted in no significant correlation for either females and males (all p-values > .05).

For Trails B, One-way repeated measures ANOVA revealed a significant main effect of practice (F (1,54) = 58.67, p < .001; ηp2=.521). Post hoc pairwise comparisons (Bonferroni adjusted for multiple comparisons) revealed a significant speeding of response (RT) for trial 2 compared to trial 1 (7.93 seconds faster) (p < 0.001, 95% confidence interval lower = 3.475, upper = 12.384); a significant speeding of RT for Trial 3 compared to trial 1 (12.205 seconds faster) (p < .001) 95% confidence interval lower = 5.387 upper = 19.023), and for trial 4 compared to trial 1 (p < .001] (18.516 seconds faster, 95% confidence interval lower = 12.282 upper = 24.750). There were no significant gender effects with respect to these results (all p values > .05).

Pearson Correlational analysis on difference in RT between Trial 1 and 2, between Trial 1 and Trial 3 and between Trial 1 and 4 and anxiety revealed no significant relationships [all p values > .05]. The same analysis for self -consciousness revealed only that Public self-consciousness was significantly correlated with the change in RT between trial 1 and 4 (r = .038, p = .038) i.e., a greater level of anxiety was associated with a greater degree of improvement (speeding) in RT between the first and last trial. This relationship failed however to survive Bonferroni correction (corrected for all three components of the self-consciousness scale and the three trial components) [p > .05]. Although, as noted previously, females were more publicly and socially self-conscious than males, additional correlation between self-consciousness and difference in RT across all trials, resulted in no significant correlation for either females and males (all p-values > .05).

4. Discussion

The aim of this study was to investigate how, in young adults, information processing speed (RT) outcome may change across the administration of four trials of the TMT-A and TMT-B components of the TMT. Specifically, we examined the potential existence of a hesitancy effect in response to the first administration of both Trails A and Trails B, and whether such an effect might be associated with an individual’s level of anxiety and/or self-consciousness.

As revealed in Figure 1, the time taken to complete the trails A component of the TMT improved significantly over successive trials compared to that obtained on the first trial, with the greatest benefit, i.e., faster information processing speed, seen in response to the fourth trial. The administration of even a second trial results in an average improvement in performance of 2.74 seconds; with a third trial facilitating an average improvement of 3.94 seconds, and a fourth an average improvement of 6.33 seconds.

The time taken to complete the Trails B component of the TMT (see Figure 2) also improved significantly over successive trials compared to that obtained on the first trial, with the greatest benefit, i.e., faster information processing speed, seen in response to the fourth trial. The administration of even a second trial results in an average improvement in performance of 7.93 seconds; with a third trial facilitating an average improvement of 12.02 seconds, and a fourth an average improvement of 18.53 seconds.

Our results are in agreement with previous studies indicating some improvement in TMT performance with practice but using different paradigms and time intervals to ours [14,15,16,17,18] and shows that practice effects for the TMT can occur even within a short single test session. In the present study, for each trial, Trails A was always administered before Trails B. Despite the potential, based on the work of Takeda, et al. [17], and Varjacic, et al. [18] for Trails B to be better performed because of this; i.e., despite this potential benefit, we still saw a significant improvement in trails B performance with increasing numbers of trials, indicating that repeated performance of Trails B per se also contributed to its improved performance.

For both Trails A and Trails B, the pattern of results illustrates an effect analogous to the first response hesitancy effect [19,20,21], in which the response to the first in a sequence of test components can be significantly slowed compared to subsequent responses, and which can significantly influence RT outcome in research studies across the lifespan [21]. This effect may be related, at least in part, to set-up time for registering a new visual layout, response preparation, strategy development and natural variation in such components as reported by [19] and [20] and to the fact that individuals are adjusting to their environment and situation [30,31].

Our results suggest that the short practice session before both Trails A and B does not remove this hesitancy effect; instead it forms a component of the information processing speed outcome of the single trial TMT test results. Arguably, the hesitancy effect in response to the first (or single) TMT trial mimics the slowing effects of pathological change or ageing, meaning that performance may be judged to be worse than it actually is; a potentially important factor when such a test is commonly used in clinical trials.

In the introduction we suggested that effects such as anxiety and self-consciousness may also be a contributory factor to the hesitancy effect. For Trails A however, the slowing in response to the first compared to subsequent trials was not significantly associated with either general level of anxiety or self-consciousness. Furthermore, neither the change in information processing speed across trials and any association with these measurements and anxiety and self-consciousness, varied with respect to gender. For Trails B, initial analysis indicated that higher levels of public self-consciousness were related to a greater improvement in RT (but only between the last, i.e., the fourth, and the first trials) with greater levels of self-consciousness associated with a larger improvement in performance as the number of trials increases. This suggests that those who are more self-conscious are slower to start with, i.e., in starting and/or performing the first trial, and thus have greater potential for getting faster as the trials progress. Although this result is in accord with previous studies showing some evidence of how self-consciousness can modulate performance on cognitive tests [22,23,24,25,26] this effect failed to survive Bonferroni correction.

It is clear that for both Trails A and B, significant speeding of RT also occurred between trials two and one and also between trials three and one. This means that practice-related speeding effects were apparent before the administration of, and thus any potential contamination from, the questionnaires, and indeed persisted and improved in response to trial four. This suggests a robust practice effect and one which occurs even when a different task interrupts test trials.

Irrespective of causality (which our present study cannot determine), our results support the idea that a single trial of both Trails A and Trails B, precludes the examination of additional potentially important clinical information related to an individual’s range of performance and the ability to improve with practice. Such an effect could lead to under-estimation of an individuals’ functional integrity both in terms of information processing speed and other processing components and behaviours; important factors with respect to the use of the TMT in clinical applications such as diagnosis and the determination of functional ability. This appears to be particularly important with respect to the more difficult Trails B task as the magnitude of the practice effects, and thus the advantage of allowing multiple trials and therefore the disadvantage of not doing so, being greater than those seen with respect to Trails A.

4.1 Individual Differences

From the data in Table 2, it is clear that for young adults there are individual differences in changes in information processing speed (RT) between the first and subsequent trials, in terms of minimum and maximum values and range. Indeed, examination of the data reveals that not everyone improves linearly across all trials, with variation in the number of trials needed to evoke an individual’s fastest response. The performance of some individuals remains almost constant, whereas for others, performance varies between improvement and decline or declines gradually. Some, but not all individuals who showed a large practice effect, had, compared to the group mean value, a disproportionately slower response to the first trial. This indicates that the effect was not simply a result of the greater potential for change. In terms of ceiling effects, others, for whom practice had little or no effect, did not necessarily display a faster response to the first trial. Although improvement with practice is a robust group mean effect, it is clear that individual differences need to be taken into account. One method for examining such effects in greater detail is to present trials until an individual’s asymptote is reached. Baseline information processing speed, ceiling effects, the potential for change, and the number of trials required in order for an individual to attain their best performance may all be important behavioral indicators of functional integrity. It is of course possible that so-called practice effects also, at least in part, reflect an individual’s ‘natural’ intraindividual variability across trials (a further measure of functional integrity with raised variability associated with older adulthood and morbidity [32]. Such effects need to be jointly investigated in future studies.

5. Potential Limitations and Future Directions

A limitation of this study was the number of trials used to examine the practice effect. In future we will examine these effects with respect to a greater number of trials; in particular in terms of an individual’s asymptote, i.e., the number of trials required to reach maximum performance: a potential behavioral marker of brain functional integrity which may have further clinical and research application. Further studies would also include the measure of performance across the lifespan and with respect to morbidity, and the potential clinical application of such tests. For example, using repeated administrations of the TMT as a performance measure might help individuals who fail performance cut-offs on first administration due to confounding variables but who would benefit from multiple exposures (e.g. [33]). Further investigations will include the examination of subcomponents of the information processing speed response, such as motor-initiation [34,35], response caution, response bias and indeed how such factors may interact [36]. It is possible that the magnitude of the TMT-related hesitancy effect per se is related to pathological change and the effects of ageing. This clearly needs further investigation as does the determination of its diagnostic value with respect to clinical conditions, what constitutes clinically meaningful results and how these vary with respect to the current one-trial TMT test, the number of trials administered and with respect to various clinical and research contexts. Such knowledge could also be useful in situations in which repeat testing, within a short time interval, is necessary such as in the study of acute drug effects. The potential influence of anxiety and self-consciousness upon TMT performance also needs to be further examined with respect to ageing and clinical conditions. The influence of such factors may have a greater role in older adults and in clinical populations (especially neurodegenerative disease) when individuals might be more aware of being tested for diagnostic/prognostic purposes and thus show higher levels of anxiety and self-consciousness. It is also possible that individuals taking part in this study did not report cognitive problems or substance or alcohol dependency. In future studies these and factors such as medication, would be more formally investigated.

It is widely accepted that TMT outcome can vary with respect to person-related (e.g. educational level and gender) and methodological factors (e.g. scoring methods and error analysis) [37,38]. Factors such as age and educational level tend to be taken into account by using standardized scores [33,39]. Nevertheless, it is increasingly apparent that TMT results can also be significantly influenced by additional factors (e.g. perceived task difficulty) that are not currently taken into account but which may in fact provide valuable additional data about brain integrity and clinical status [38,40,41,42,43]. It is also important therefore that future studies examine the hesitancy and practice effects related to the TMT with respect to these other factors. It would be interesting also to analyse the effects of repetitions in the context of the different outcome measures of the TMT that are sometimes applied, e.g. TMTB/TMTA and TMTB-TMA.

6. Conclusions

This study revealed how increasing the number of trials of the TMT, for both Trails A and Trails B from one to four trials can significantly influence the information processing speed outcome of these tests. The similarity of the results to the first trial hesitancy effect shows that the results of a single administration of the TMT test may not be fully indicative of a person’s actual or potential information processing speed. Such findings may have implications regarding the interpretation of such results in young adults, especially in clinical practice, with the possibility for some individuals that a single trial outcome significantly under-represents actual performance. This, together with the possibility that giving more trials also facilitates the measurement of other aspects of brain processing, e.g. the ability to improve performance with practice, which may also be a marker of brain and behavioural functionality; particularly in disease, leads us to suggest that it is time to consider the administration of more than one trial of the TMT.

Acknowledgments

The authors would like to thank all the study participants.

Author Contributions

Kate McKeown: Idea conception, study design, statistical analysis & interpretation, writing. Emma Richards: Idea conception, statistical analysis & interpretation, writing. Jessica Richardson: Idea conception, study design, interpretation of results, writing. Andrea Tales: Idea conception, study design, statistical analysis and interpretation, writing.

Funding

There was no source of funding for this work.

Competing Interests

The authors have declared that no competing interests exist.