Sports-related concussions (SRCs) are a form of mild TBI that are prevalent among athletes, especially those who participate in contact sports (Waltzman et al., 2020; Zuckerman et al., 2015). In the United States, between 1.6 and 3.8 million SRCs are estimated to occur annually (Zuckerman et al., 2015). The vast majority of SRCs occur in those aged 18 and younger, affecting an estimated 1.1 to 1.9 million youth athletes (Bryan et al., 2016). Between 2010 and 2016, the average annual emergency department (ED) visits for sport and recreation-related TBI (SRR-TBI) among children and adolescents was 283,000 (Waltzman et al. 2020). Beyond findings based solely on annual ED visits, data collected in 2017 from the Youth Risk Behavior Survey (YRBS), a cross-sectional study of those in grades 9–12, indicated that an estimated 2.5 million high schoolers in the US (~ 15%) reported having at least one SRR concussion in their lifetime (DePadilla et al., 2018).
SRCs are also prevalent among adults. Based on data from the National Collegiate Athletic Association Injury Surveillance Program (NCAA ISP), between the 2009–2010 and 2013–2014 academic years, the estimated prevalence of SRCs in NCAA sports was approximately 10,560 annually in the US (Zuckerman et al., 2015). For academic years 2014–2015 to 2018–2019, the NCAA ISP reported approximately 4.3 SRCs per 10,000 athlete-exposures (Chandran et al., 2021). Notably, these estimates do not include SRC prevalence in club sports or recreational sports occurring outside of school, and as a result, the reported incidences of SRCs are likely underestimated. Additionally, not reporting SRCs is common across settings, as some athletes may not realize they have experienced an SRC or they may have concerns about letting their team down or jeopardizing their ability to play (Beidler et al, 2018).
Anzeige
The prevalence of SRCs and underreporting tendencies highlights the need for valid and reliable concussion screening tools to help inform return-to-play decisions in a wide range of settings. Computerized cognitive assessments are one way to assess the sequelae of concussion to aid athletes, athletic trainers, coaches, parents, sports medicine physicians, and other medical professionals in making safer decisions. However, some of the more widely used computerized tests have been criticized for their psychometric shortcomings (Alsalaheen et al., 2016; Broglio et al., 2018; Gaudet et al., 2020; Resch et al., 2013; Resch et al., 2018) and limitations in symptom screening ability (Broglio & Puetz, 2008; Chin et al., 2016), which raises the question of whether other computerized tasks could provide useful information with respect to the cognitive sequelae of SRCs.
The Conners Continuous Performance Test (3rd Edition; CCPT3; Conners, 2014) is a computerized test assessing various dimensions of executive functioning (i.e., inattentiveness, sustained attention, impulsivity, vigilance) in those aged 8 and older (Multi-Health Systems Inc, n.d.). Users are instructed to respond to all letters other than “X” (commonly referred to as a non-X paradigm) using their spacebar or mouse (Multi-Health Systems Inc, n.d.). Current and previous versions of the CCPT (e.g., the 2nd Edition; Conners, 2000) have established good psychometric properties across the age span. For example, 1-week inter-class correlations (ICCs) for chronic adult stroke patients completing the second edition of the CCPT ranged from 0.7 to 0.9 (Chen et al., 2009) and from 0.62 to 0.82 for healthy children tested 3 to 8 months apart (Zabel et al., 2009). For the CCPT3, a sample of 120 individuals from the general population completed the measure between 1 and 5 weeks later, yielding corrected reliability coefficients ranging from 0.48 to 0.89, with a median of 0.67 (Multi-Health Systems Inc, n.d.; Conners et al., 2018) for the eight scores that will be discussed below. Split-half reliability figures for the CCPT3 were 0.92 and 0.94 for norm samples and clinical samples, respectively (Multi-Health Systems Inc, n.d.; Conners et al., 2018).
Since 1994, the most common application of the CCPT has been to help identify those with attention deficit disorder (Conners, 2000). However, the CCPT also appears to relate to some of the neurocognitive sequelae of mTBI, such as attention and executive functioning deficits (Galbiati et al., 2009; Levan et al., 2016; Paré et al., 2009; Zane et al., 2016). Specifically, scores on a previous edition of the CCPT have been shown to relate to the severity of TBI (Zane et al., 2016) as well as the course of TBI recovery as a function of treatment (Galbiati et al., 2009). The previous edition of the CCPT was also shown to relate to social problems in children with TBIs, with omission errors being the best predictor (Levan et al., 2016). More recently, the CCPT3 was found to account for 21.5% of the variance (Cohen’s d = 1.05) in CDC concussion symptom endorsement in a sample of 113 participants aged 6 to 17 years (Lecci et al., 2020) and CCPT3 scores, when combined with other measures, were able to predict the return/remove decisions of a pediatric neurologist, with an overall classification accuracy of 91% and 84.8% using general linear and deep learning models, respectively (Keith et al., 2019).
Most germane to the current study, recent research explored the viability of a shortened CCPT3 in predicting concussion outcomes. The shortened CCPT3 is comprised of the first half (180 trials) of the full CCPT3 (360 trials), which takes approximately 7.5 min to complete. It was recently found that in a sample of 925 individuals (108 with SRCs), including children, adolescents, and adults, the short CCPT3 produced scores that are highly consistent with the full CCPT3 and could explain 16.8% variance in CDC concussion symptom endorsement (Cohen’s d = 0.90) (Lecci et al., 2021). The variance accounted for was even greater when examining those who were concussed within 6 months of the assessment (explaining 22.1% variance, d = 1.07) (Lecci et al., 2021). These findings emerged despite individuals being evaluated an average of over 20 days following the SRC. The predictive validity of the short CCPT3 was further enhanced when assessing individuals within 1 week of their SRC (explaining 27.9% variance, d = 1.24), indicating that CCPT3 scores can differentiate symptom experience among those who are recently concussed (Lecci et al., 2021). Shortened CCPT3 scores also predicted concussion history for SRCs occurring more than 6 months prior to the assessment (Lecci et al., 2021). Finally, short CCPT3 scores could differentiate athletes who were completing a baseline (asymptomatic) assessment and those completing a post-concussion assessment with an overall 88.7% classification accuracy (97.2% correct for non-concussed and 77.9% for concussed) (Lecci et al., 2021).
Anzeige
Researchers achieved the above-noted CCPT3 findings using data collected by healthcare professionals in controlled medical settings using a Microsoft Surface Pro. However, there are several circumstances where individual assessments in controlled environments might be less likely to occur. For example, mass baseline testing can be a challenge, as it requires considerable personnel and equipment to be available at one time. It is also the case that following a suspected SRC, it is not always possible for individuals to see medical professionals to be evaluated in a timely manner. Even athletes who are involved in a return-to-play protocol may have several weeks between visits with a healthcare professional. Thus, there would be considerable value in determining if the above-reported findings would replicate when CCPT3 assessments are delivered remotely on a mobile device such as a smartphone.
The present study examines a mobile version of the short CCPT3 for a sample of youth and young adult athletes, some of whom were suspected of recently experiencing a concussion. We evaluate whether scores on the mobile version of the short CCPT3 can predict CDC concussion symptom endorsement and symptom severity, whether the findings are independent of ADHD diagnosis, and whether the emergent findings yield effect sizes similar to previous research.
Method
This is a retrospective study in which data were drawn from a registry involving assessments completed on mobile devices using the SportGait Mobile App (described below), with all evaluations taking place between July 2021 and January 2022.
For approximately half of the participants, the SportGait Mobile App was administered following a significant contact, where an athletic trainer suspected an SRC. These individuals were removed from play until deemed safe to return following National Athletic Trainers Association (NATA) guidelines (Broglio et al., 2014). Removal decisions were based on a variety of factors, including concussion symptom endorsement, scores on the short CCPT3, a gait assessment, behavioral presentation, history of past SRCs, and in some cases, scores on other measures (e.g., SCAT-5). For the remaining ostensibly healthy participants, the data were drawn from consecutively evaluated youth athletes competing in community and/or school-based sports leagues across multiple states, who, prior to the sports season, completed a baseline assessment using the SportGait Mobile App.
The rationale for including ostensibly healthy individuals completing baseline assessments and participants with suspected concussive injuries, each of whom were participating in youth sports programing, was to create variability in concussion symptom endorsement and severity ratings. Importantly, all those completing the SportGait Mobile battery were eligible for being selected for the study.
Participants
Those with a suspected SRC included a total of 63 consecutively evaluated youth athletes (28.6% female) with a recent head impact. Participants completed the SportGait Mobile battery between 0 and 24 (M = 4.26, SD = 4.84) days following the initial injury, with 62.3% being evaluated within 3 days, and 85.2% within 1 week. These participants were aged 11 to 23 years (M = 15.9, SD = 2.05) and 22.8% of participants reported a previous diagnosis of attention-deficit/hyperactivity disorder (ADHD). For these individuals, 43.9% reported no previous concussion (aside from the current suspected SRC), 33.3% reported one previous concussion, and 22.8% reported two or more previous concussions. Among those who had experienced a previous concussion, 28.1% occurred within the last 6 months, 5.3% occurred between 7 and 12 months prior, and 22.8% were more than a year prior to the current assessment.
A sample of 85 ostensibly healthy youth athletes were then randomly selected using a seed program from a database of 4589 baseline assessments. We removed two individuals from this group, as they were also included in the injured sample (i.e., tested twice). We removed three other individuals because of outliers on raw omission data, indicating inattentiveness and/or lack of motivation during the cognitive assessment (note: there were no outliers on raw omission data for the injured participants). The final sample of healthy athletes consisted of 80 individuals (30% female), aged 12 to 18 years (M = 15.36, SD = 1.35), with 11.8% of participants reporting a previous diagnosis of ADHD. For these individuals, 82.4% reported no previous concussion, 11.8% reported one previous concussion, and 5.9% reported two or more previous concussions. Among those who had experienced a previous concussion, none occurred within the last 6 months, 1.5% occurred between 7 and 12 months prior, and 16.2% occurred more than a year prior to the current assessment.
Participants were competing in a number of sports and some athletes were active in multiple sports. The sports included volleyball, boy’s and girl’s basketball, football, softball, girl’s swim/dive, wrestling, baseball, girl’s tennis, ice hockey, rugby, boy’s and girl’s soccer, golf, track and field, cross country, and cheer.
Anzeige
None of the participants was compensated for completing the SportGait Mobile App as it was part of standard practice for both the baseline testing and medical evaluations following a suspected concussion. Approval for this study was obtained from the University of North Carolina Wilmington’s Institutional Review Board (IRB #21–0047) as an anonymous, archival data analysis. The original data were therefore collected in the normal course of practice, either in clinical settings or as part of standard baseline testing.
Aside from the above-noted outliers, we only excluded data for individuals who failed to complete the full SportGait battery (i.e., incomplete data). Data requests can be made to the study’s first author.
Procedure
All participants completed the same standardized battery (known as SportGait Mobile), delivered on a mobile phone. The SportGait Mobile battery delivers tests in the following standardized order (though providers can opt to deliver the tests in a different order or omit some measures): the 31-item CDC concussion symptom checklist (which serves as our primary outcome variable), questions regarding the diagnosis and treatment of ADHD (which serves as a covariate), the mobile Conners Continuous Performance Test, 3rd Edition (CCPT3), which is the first half of the original CCPT3 and serves as the predictor variables, and a sensor-based gait assessment (three walks with the mobile device held to the chest). We did not examine the gait data for this study.
The 31-item CDC concussion symptom checklist (each item endorsed as present or absent) includes 11 danger items such as slurred speech, vomiting or nausea, persistent headache, and unequal pupil size and 20 less severe symptoms including confusion, sensitivity to light, clumsiness, and visual disturbance (Centers for Disease Control and Prevention, 2010). Participants then rated any endorsed items for severity from 0 (none) to 6 (severe). We used the total number of items endorsed to create a total symptom endorsement score, with higher scores denoting a greater number of experienced symptoms. We summed severity ratings to create a total severity score, with higher scores denoting a greater severity of experienced symptoms. We examined both total symptoms and total symptom severity as the outcome variables of interest.
Anzeige
The mobile app questions regarding ADHD diagnosis include whether the participant had ever been diagnosed with ADHD, whether they were prescribed medication for ADHD, and whether they had taken their ADHD medication at the time of the assessment. We categorized participants to reflect their “ADHD expression” as either no ADHD or having ADHD but medicated at the time of the assessment, together representing 90.4% of the entire sample. Alternatively, we categorized participants as having an ADHD diagnosis and either having no prescribed pharmacotherapy or not being medicated at the time of the assessment, with this representing 9.6% of the sample.
Participants then complete the half CCPT3, which includes the standard CCPT3 instructions and a 30-s practice trial, and then the 180 trial test (3 blocks of 60 trials) that takes 7 min. The mobile CCPT3 is a version of the Conners Continuous Performance Test that is completed on a mobile device. Unlike the computerized versions of the CCPT (3rd Edition; CCPT3; Conners, 2014), participants respond by tapping the phone screen, instead of tapping the spacebar or clicking a mouse, as would be the case in the original version of the test. The first 180 trials are fully counterbalanced for the inter-stimulus interval (time between each trial), which varied from 1, 2, or 3 s.
The mobile CCPT3 displays results as T-scores based on normative data from the administration of the half CCPT3 on the phone. Normative data, which are stratified by age and gender, are comprised of 2413 individuals (37.8% female) ranging in age from 8 to 72 (mean = 19.32, SD = 11.41). The normative sample is based on healthy individuals, with no history of concussion, who endorsed a non-significant number of concussion symptoms with limited severity and did not endorse any concussion danger signs at baseline.
The mobile CCPT3 produces 8 variables that serve as the predictor variables in the current study: hit reaction time, omissions, hit reaction time standard deviation, commissions, detectability, perseverations, variability, and hit reaction time inter-stimulus change. The full (15-min) (Conners et al., 2018) and half (7.5-min) versions of the CCPT3 (Lecci et al., 2021) administered on the computer have previously predicted concussion symptoms in children and adolescents.
Anzeige
Data are captured by the phone app and then transmitted to the cloud in a HIPPA compliant manner, and stored in Azure. The deidentified data was subsequently analyzed.
The analytic strategy for this research parallels that employed in previous research using the full CCPT3 and the shortened CCPT3 delivered on a Microsoft Surface Pro. All predictor variables (the 8 variable outputs from the short CCPT3) were entered into the regression equation simultaneously to evaluate the overall explained variance. The prediction model was evaluated based on the statistical significance of the explained variance (r-square), and Cohen’s d values are presented as standardized effect size coefficients. These values will also be compared to the same values calculated in previous research using the computer-based version of SportGait.
The sample size was determined by taking all of the athletes with suspected concussions who completed the mobile assessment within the specified 7-month window and then randomly selecting a sample of healthy individuals tested in that same timeframe to achieve a sample size of 140. Assuming effect sizes that are comparable to those from previous research, a sample size of 140 would achieve statistical power in excess of 0.90.
Results
Chi-square analyses indicate no significant difference between the healthy and injured samples in gender distribution (χ2 (1, N = 143) = 0.035, p = 0.852) or incidence of ADHD (χ2 (1, N = 143) = 2.71, p = 0.100). Independent samples t-tests indicated no significant differences in age, t(141) = − 1.90, p = 0.06, between the healthy (M = 15.36, SD = 1.35) and injured samples (M = 15.90, SD = 2.05). Further, age was not a significant predictor of our primary outcome variable, concussion symptom severity, R2 = 0.00, F(1, 78) = 0.00, p = 0.996. Thus, age was not included as a variable in any of the analyses.
Those with a suspected SRC were more likely to have a history of previous concussions (χ 2 (2, N = 125) = 20.30, p < 0.001), and they were also more likely to have had a previous concussion within the last 6 months (χ 2 (3, N = 125) = 28.28, p < 0.001).
Healthy participants reported an average of 2.74 symptoms (SD = 4.15) and an average total symptom severity rating of 5.26 (SD = 11.18). Injured participants with a suspected SRC endorsed more than three times the CDC concussion symptoms reported by healthy participants, with an average of 8.75 symptoms (SD = 6.25), and more than three times the severity rating, with an average total severity of 17.33 (SD = 16.01). t-Tests indicated that these values differ significantly, such that injured participants reported more CDC concussion symptoms, t(102.8) = 6.58, p < 0.01, and higher severity ratings, t(106.4) = 5.09, p < 0.01, relative to healthy participants.
Predicting CDC Symptom Severity and Symptom Endorsement with Mobile CCPT3 Scores
A linear regression predicting total symptom severity was performed, with the eight CCPT3 variables simultaneously entered as predictors. The eight CCPT3 variables combined account for 19% of the variance in total symptom severity, R2 = 0.19, F(8, 134) = 3.94, p < 0.001 (Table 1). This is characterized as a large effect size (Cohen’s d = 0.97). Of the eight CCPT3 variables, the significant predictors of total symptom severity included hit reaction time, omission errors, and commission errors. These findings are comparable to the achieved explained variance in an earlier study (R2 = 0.16, Cohen’s d = 0.87), in which two of the above three variables (hit reaction time and omission errors) were also significant (Lecci et al., 2021). In this previous study, additional CCPT3 variables emerged as significant (inter-stimulus change, perseverations, and hit rate standard error), likely due to the large sample size (> 900).
Table 1
Regression analysis predicting total severity of CDC concussion symptoms
R | R2 | Adjusted R2 | SE of est | F(df) | |
|---|---|---|---|---|---|
Model | 0.436 | 0.190 | 0.142 | 13.66 | 3.94(8)** |
Predictors | B | B 95% CI [LL, UL] | SE B | β | t |
Omissions | − 0.611 | [− 1.08, − 0.14] | 0.236 | − 0.398 | − 2.59* |
Commissions | 0.751 | [0.13, 1.37] | 0.312 | 0.500 | 2.41* |
D prime | − 0.196 | [− 1.05, 0.66] | 0.433 | − 0.130 | − 0.45 |
Inter-stim change | − 0.069 | [− 0.34, 0.21] | 0.139 | − 0.048 | − 0.50 |
Hit rate RT | 0.459 | [0.09, 0.83] | 0.186 | 0.275 | 2.46* |
Hit rate SE | 0.101 | [− 0.29, 0.49] | 0.199 | 0.087 | 0.51 |
Perseverations | 0.196 | [− 0.08, 0.48] | 0.141 | 0.181 | 1.39 |
Variability | 0.046 | [− 0.20, 0.30] | 0.127 | 0.043 | 0.37 |
We repeated the linear regression predicting total symptom severity using only the injured sample to determine whether CCPT3 scores can also differentiate severity among those who are injured. This model was also significant and predicted 27.6% of the variance in total symptom severity, R2 = 0.28, F(8, 54) = 2.76, p = 0.02. This is characterized as a large to very large effect size (Cohen’s d = 1.24). The increased predictive validity (r-square value) for the injured group is similar to the improved predictive validity documented for the CCPT3 in previous studies when likewise focused on a clinical/injured sample (Lecci et al., 2021).
Finally, we performed a linear regression predicting total number of endorsed symptoms, yielding a significant model that predicted 17.2% of the variance in the number of endorsed symptoms, R2 = 0.17, F(8, 134) = 3.48, p < 0.01, Cohen’s d = 0.91. Due to the strong correlation between total severity and total symptoms (r = 0.92, p < 0.01), only results predicting symptom severity, which includes information on both the number and intensity of symptoms, are discussed.
Mobile CCPT3 Predicting CDC Symptom Severity After Controlling for ADHD Expression
Because the deficits in executive functioning associated with the cognitive sequelae of concussion may overlap with those associated with ADHD, it is important to determine whether ADHD is a confound in the reported analyses. This can be addressed by assessing whether mobile CCPT3 scores can predict concussion symptom endorsement after statistically controlling for factors related to ADHD (i.e., using ADHD expression variable as a covariate).
We conducted a hierarchical regression with ADHD expression included in the regression equation in block 1, and all CCPT3 variables entered simultaneously in block 2 (see Table 2). ADHD expression significantly predicted symptom severity, explaining 4.6% of the variance, F(1, 142) = 5.92, p = 0.02, Cohen’s d = 0.44. The CCPT3 variables then add 19.8% to the explained variance for concussion symptom severity, ΔF(8, 115) = 3.77, p < 0.01, Cohen’s d = 0.99. Importantly, the explained variance for the mobile CCPT3 is virtually identical to the explained variance obtained when ADHD expression was not statistically controlled, indicating that the mobile CCPT3’s prediction of concussion symptom severity is independent of ADHD expression.
Table 2
Regression analysis predicting total severity of CDC concussion symptoms, controlling for ADHD expression
R | R2 | Adjusted R2 | SE of est | F(df) | ΔF | |
Model block 1 | 0.214 | 0.046 | 0.038 | 14.828 | 5.92(8)* | 5.92* |
Predictors | B | B 95% CI [LL, UL] | SE B | β | t | |
ADHD expression | 10.952 | [7.20, 12.73] | 4.502 | 0.214 | 2.43* | |
R | R2 | Adjusted R2 | SE of est | F(df) | ΔF | |
Model block 2 | 0.494 | 0.244 | 0.185 | 13.649 | 4.129(9)*** | 3.77*** |
Predictors | B | B 95% CI [LL, UL] | SE B | β | t | |
ADHD expression | 13.138 | [4.56, 21.71] | 4.329 | 0.257 | 3.03** | |
Omissions | − 0.624 | [− 1.12, − 0.12] | 0.252 | − 0.398 | − 2.48* | |
Commissions | 0.871 | [0.87, 0.33] | 0.326 | 0.556 | 2.67** | |
D prime | − 0.388 | [0.23, 1.52] | 0.463 | − 0.249 | − 0.84 | |
Inter-stim change | − 0.081 | [− 0.37, 0.21] | 0.146 | − 0.056 | − 0.56 | |
Hit rate RT | 0.394 | [0.01, 0.78] | 0.195 | 0.243 | 2.02* | |
Hit rate SE | 0.214 | [− 0.21. 0.64] | 0.214 | 0.188 | 1.00 | |
Perseverations | 0.178 | [− 0.12, 0.47] | 0.149 | 0.166 | 1.19 | |
Variability | − 0.033 | [− 0.30, 0.23] | 0.132 | − 0.030 | − 0.25 |
Discussion
The purpose of the current research was to determine whether mobile CCPT3 scores could predict CDC concussion symptom endorsement and symptom severity, and whether the findings are comparable to those obtained in controlled medical environments with data collected on computers. We also examined whether the findings were independent of ADHD status. Overall, the findings provide favorable evidence on all fronts.
The current research represents an important step in the validation of a brief concussion assessment tool that could be used to conduct baseline assessments and support rehabilitation and return-to-play decisions. The advantage of a concussion assessment tool delivered on a mobile device is that it allows for the ubiquitous adoption of the technology, whereby even individuals who are unable to see a health professional in person in a timely manner can nevertheless complete an objective assessment of their cognitive functioning within the convenience of their own homes, and those data can then be transmitted (in a HIPAA compliant format) for a health professional to review and interpret. These early data points could help inform conceptualizations of recovery when compared to subsequent data collected in healthcare settings during follow-up appointments.
Importantly, large effect sizes emerged for the prediction of concussion symptom severity for the entire sample of suspected concussed and ostensibly healthy individuals, with the findings paralleling those from previous research (with Cohen’s d = 0.90) that administered the full CCPT3 on a Surface Pro in controlled medical settings to a sample of over 900 participants (Lecci et al., 2021). This indicates that mobile CCPT3 scores may provide useful information to support initial diagnostic considerations as well as decisions during recovery, and this is also in keeping with previous research (Keith et al., 2019). In addition, mobile CCPT3 scores predicted CDC symptom severity even when focusing exclusively on those who sustained injuries, thereby suggesting that the cognitive data can provide information differentiating severity among those who are injured. This could then inform treatment recommendations during rehabilitation, readiness to return decisions, and recommendations for appropriate level of activity for the recovering athlete. Each of the findings replicate previous research using a version the CCPT delivered on a computer, showing that scores relate to the severity of mTBI (Zane et al., 2016) and to the course of recovery following an incident (Galbiati et al., 2009). Finally, replication of prior findings despite the switch in technology to mobile phones, and the fact that users had various versions of both iPhones and Android phones, suggests that the efficacy of the cognitive task is robust with respect to the technology employed.
In the current sample, participants were evaluated an average of just over 4 days following the suspected SRC, with several assessments occurring on the same day as the injury, and some upwards of 24 days following the injury. The extant literature has consistently shown that larger effect sizes emerge when concussed individuals are evaluated immediately following an injury (e.g., within a week) (Lecci et al., 2021). However, practical limitations often make timely assessments impossible, as injuries can occur when medical facilities are closed, and injured parties may have to wait for an available appointment that is days or weeks away. The advantage of the mobile assessment is that data can be collected almost immediately following an injury, even before healthcare personnel are available. Such data can be critical to better understanding the initial course and severity of the injury, and this in turn should better inform subsequent health decisions.
It is also noteworthy that even though the original CCPT was developed for use within the context of ADHD evaluations (e.g., Conners, 2000), and those with ADHD do have a greater lifetime history of concussion (e.g., Iverson et al., 2016), we found that the prediction of CDC concussion symptom severity was independent of the experience of ADHD (i.e., the predictive ability of the mobile CCPT3 did not change after statistically controlling for ADHD information). There are several reasons why this may have occurred. First, the original CCPT3, which is often used to assess for attention problems, is twice the length of the current mobile assessment (i.e., the original is a 15-min test with 360 trials), as the mobile version omits the second half of the original measure. It is certainly possible that the second half of the test is a more direct assessment of sustained attention, which is a construct relevant to ADHD, but potentially less relevant for concussion. Thus, when the last 180 trials are included, they may introduce “noise” with respect to concussion-relevant outcomes. Indeed, previous research demonstrates no decline in predictive ability for the CCPT3 when directly comparing the first 180 trials relative to the full-length (360-trial) test (Lecci et al., 2021). It is also possible that the mobile phone environment is less taxing with respect to sustained attention, when compared to the standard delivery on the computer, as individuals are largely accustomed to looking at their phones for significant periods of time (and at least for the 7.5 min it requires to complete the short CCPT3) (Ceci, 2022). Regardless as to the exact reason for the mobile CCPT3 predicting concussion symptom severity independent of ADHD, this finding should help in decision-making when dealing with patients for whom an ADHD diagnosis is applicable. Specifically, mobile CCPT3 scores appear to be functionally independent of ADHD when considering concussion outcomes. From a practical standpoint, it can also be argued that changes in CCPT3 scores during the return-to-play process would be most related to concussion recovery if the individual’s ADHD status is constant (e.g., every time they complete the mobile CCPT3 they are not medicated for ADHD if, at the time of their initial evaluation, they were also unmedicated).
The current work suggests that the CCPT3, and specifically the shortened, mobile CCPT3, exhibits utility that prior work has found to be additive when combined with other measures assessing neurobehavioral sequelae (e.g., gait and balance) (Lecci et al., 2021). That is, the information from the CCPT3 provides incremental information when combined with other domains, thereby allowing for a more comprehensive assessment of the broad range of concussion consequences.
The mobile CCPT3 adds to the literature on computerized concussion assessment instruments by exhibiting good overall psychometric soundness (e.g., test–retest reliability and validity coefficients) (Alsalaheen et al., 2016; Broglio et al., 2018; Gaudet et al., 2020; Resch et al., 2013; Resch et al., 2018), as well as by addressing limitations related to concussion screening and recovery, by capturing cognitive deficits at longer post-incident intervals (Broglio & Puetz, 2008; Chin et al., 2016).
Limitations and Future Directions
The current research focused on symptom endorsement and severity as the primary outcome variables. Previous research has linked CCPT3 scores to medical decisions (Keith et al., 2019), and future research could include such information, along with other medical data (e.g., imaging) to further evaluate accuracy. It is noted, however, that medical decisions involving concussed patients are strongly influenced by patient endorsements and/or experience of concussion symptoms (Keith et al., 2019), thereby highlighting the importance of mobile CCPT3 scores strongly predicting, but not being redundant with, concussion symptoms. It is also the case that the symptom experience of a patient is not always known or optimal. For example, research suggests that some athletes may be so motivated to return to sports that they under-report symptoms (Meier et al., 2015), making the non-endorsement of concussion symptoms in that context less helpful (and even counterproductive) to the decision-making of the healthcare professional. In addition, when dealing with young athletes (children and adolescents), research suggests that they may be less willing and/or able to accurately report their concussion symptoms (Leahy et al., 2018). In these and similar circumstances, having a rapid and available objective measure that is highly predictive of concussion symptoms is particularly useful. Of note, the present study included youth and young adults aged 11 to 23; therefore, the current findings may only generalize to this age group, and replication across a broader age range would be beneficial.
The current research also did not include comprehensive medical information or scores on other more extensive neurocognitive measures, and such data would help provide a more complete clinical picture, though we did collect information regarding concussion history and ADHD expression. Moreover, with respect to ADHD, our results indicated that the predictive ability of the mobile CCPT3 was independent of ADHD expression. Future research could further validate the mobile short CCPT3 by administering it along with a comprehensive neuropsychological battery to better characterize the specific functional deficits associated with concussion and other related neurocognitive conditions. A comprehensive neuropsychological assessment would also allow for a formal assessment of performance validity to ensure the accuracy of the cognitive data and better characterize any effects of effort.
Because the injured group was also more likely to experience a previous concussion within the last 6 months, it is possible that the emergent findings may capture some contributions from the previous concussion(s). However, this may be less relevant from a practical standpoint, as the ultimate goal of the mobile measure is to illustrate when cognitive sequelae of concussion are present, regardless as to when the incident occurred. Notably, previous research has shown that the shortened CCPT3 does predict concussion history (Lecci et al., 2021).
The current findings are especially promising because the observed effect sizes are considerably larger as compared to those emerging for other measures, and therefore represents the potential for marked improvement in understanding and characterizing concussion and the process of recovery. Moreover, the utility of a mobile phone delivery means that data can be collected more easily, more frequently, and with minimal resources, thereby providing athletic trainers, sports medicine physicians, neuropsychologists, and other medical professionals with critical objective data proximal to the injury to better inform their decisions.
It is also noted that the most recent consensus statement on concussion in sport suggests emphasizing reaction time (RT) data when using computerized neurocognitive tests (Patricios et al., 2022). Consistent with this recommendation, researchers have determined that reaction time data are among the slowest to return to baseline following a concussion (Broglio et al., 2023). Thus, the CCPT3, which is a reaction time-based measure with output that goes beyond simple RT, by also assessing multiple measures of variability in RT, is especially well suited for documenting the neurocognitive sequelae of concussion.
The current research represents an initial step in validating a mobile cognitive assessment platform, and the ease of data collection in a mobile environment also holds promise for future research that has the potential to better characterize the cognitive changes that occur immediately following a SRC. The demonstrated ability to collect reliable and valid data from the mobile CCPT3 also highlights its potential application to other conditions, abilities, and indices assessed in a mobile environment, and future research will explore these possibilities.
Acknowledgements
We would like to thank all the members of the research consortium, clinical partners, sports organizations, and athletes who contributed data to the study. The SportGait App used in the current research is available in the App Store and in Google Play.
Declarations
Conflict of Interest
Dugan graduated from the University of North Carolina Wilmington with an MA and was previously employed at SportGait Inc. Lecci is a professor of psychology at the University of North Carolina Wilmington and is a consultant and minor shareholder for SportGait Inc. He also serves as an associate editor for the Journal of Pediatric Neuropsychology. Woodley and Laney are graduate students at the University of North Carolina Wilmington and previously received part time research stipends through SportGait. SportGait is part of the University of North Carolina Wilmington’s Center for Innovation and Entrepreneurship.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.