Clinical Practice Guidelines About Screening for Disruptive Behavior Problems at Well-Child Visits: A Rapid Review of the Literature on the Accuracy of Parents’ Behavioral Concerns
Raymond H. Baillargeon 1,*, Marylène Charette 1, François Tessier 2, Kevin P. Brand 3
-
Interdisciplinary School of Health Sciences, Faculty of Health Sciences, University of Ottawa, Ottawa, Ontario, Canada, K1N 6N5
-
Public health, Gatineau, Québec, Canada
-
Telfer School of Management, University of Ottawa, Ottawa, Ontario, Canada, K1N 6N5
* Correspondence: Raymond H. Baillargeon
Academic Editor: Sanghamitra M. Misra
Special Issue: Supporting Children with Integrative Techniques During and After Public Health Crises
Received: March 08, 2022 | Accepted: July 24, 2022 | Published: August 03, 2022
OBM Integrative and Complementary Medicine 2022, Volume 7, Issue 3, doi:10.21926/obm.icm.2203031
Recommended citation: Baillargeon RH, Charette M, Tessier F, Brand KP. Clinical Practice Guidelines About Screening for Disruptive Behavior Problems at Well-Child Visits: A Rapid Review of the Literature on the Accuracy of Parents’ Behavioral Concerns. OBM Integrative and Complementary Medicine 2022; 7(3): 031; doi:10.21926/obm.icm.2203031.
© 2022 by the authors. This is an open access article distributed under the conditions of the Creative Commons by Attribution License, which permits unrestricted use, distribution, and reproduction in any medium or format, provided the original work is correctly cited.
Abstract
The current recommendations of the Canadian Paediatric Society about monitoring children’s disruptive behaviors at well-child visits call for screening if behavioral concerns are being raised by parents. But do parents’ concerns about their child’s behavior constitute a reliable means for primary care providers (PCPs) to decide either in favor or against screening? We conducted a rapid systematic review of the literature by identifying documents that cited the landmark study by Glascoe and her colleagues (1991) on the accuracy of behavioral concerns at identifying children with a disruptive behavior problem. Citation tracking was done using Web of Science (Core Collection; 17 October 2018) and SCOPUS (19 October 2018). Only one recent published study was identified. The calibration of concerns’ specificity (and other indices alike) yielded, at best, a fair value of the weighted kappa coefficient κ(0,0) (i.e., 0.255 and 0.094). Also, the calibration of concerns’ sensitivity (and other indices alike) yielded, at best, a moderate value of the weighted kappa coefficient κ(1,0) (i.e., 0.533 and 0.392). Overall, the results do not support the current recommendations. In fact, behavioral concerns do not provide PCPs with enough information to reach a decision about screening. We discuss different ways of gathering the necessary information.
Keywords
Infant health; practice guidelines; problem behavior; clinical decision-making; primary health care; preventive medicine; systematic review
1. Introduction
There is a consensus about the importance of monitoring children’s behavioral development during regularly scheduled well-baby/child visits with a primary care provider (PCP)--pediatrician, family physician, and nurse practitioner among others [1,2]. An integral part of the standard pediatric practice consists of routinely eliciting and attending to parents’ concerns about their child’s behavior. In many developed countries, attending to parents’ behavioral concerns involve the use of screening.
In 2017, in a joint statement with the Canadian Academy of Child and Adolescent Psychiatry (CACAP), the Canadian Paediatric Society (CPS) published clinical practice guidelines for PCPs about screening for disruptive behavior problems (DBPs) in 2-5 year-old children at regularly scheduled well-child visits [3]. According to these recommendations, PCPs should elicit parents’ concerns about their child’s behavior and perform behavioral screening whenever one or more concerns are being raised.
However, the mere presence of parents’ behavioral concerns may not constitute a reliable means for PCPs to rule in the presence of a DBP in children. PCPs may need more information before deciding in favor of screening. Indeed, many parents may be excessively worried about their children’s disruptive behavior. Also, the mere absence of parents’ behavioral concerns may not constitute a reliable means for PCPs to rule out the presence of a DBP in children. PCPs may need more information before deciding against screening. In fact, many parents may not fully appreciate the problematic nature of one aspect or another--e.g., frequency--of their children’s disruptive behavior. Overall, the information obtained by eliciting behavioral concerns in parents may be only marginally useful for PCPs to decide on the most appropriate course of action.
In their joint statement, the CPS/CACAP did not present evidence about the accuracy of parents’ behavioral concerns to support their recommendations. To our knowledge, there is at least one study that investigated the accuracy of parents’ concerns about their 24-77 month-old children’s behavior [4]. In that landmark study by Glascoe and her colleagues [4] (n = 95), 58.8% (i.e., 20/34) of parents who raised behavioral concerns had children without a clinically significant DBP. Also, the authors reported that 9.8% (i.e., 6/61) of parents who did not raise concerns had children with a clinically significant DBP. In practical terms, PCPs relying on behavioral concerns to decide whether or not to screen would be mistaken 27.4% (i.e., 26/95) of the time.
1.1 Objective
The first goal of this study is to conduct a rapid review of the literature for more recent evidence about the accuracy of parents’ behavioral concerns at distinguishing between children with a DBP and those without. To our knowledge, there has not yet been a systematic, either conventional or rapid, review of this literature published. The second goal of this study is to determine, based on the available evidence, whether: a) the mere presence of behavioral concerns provides PCPs with a reliable means to rule in the presence of a DBP; and b) the mere absence of behavioral concerns provides PCPs with a reliable means to rule out the presence of a DBP. If not, PCPs may need more information before they can decide in favor/against screening. Together, these two goals represent the main contribution of this study; namely, determine if the current CPS/CACAP’s recommendations are supported by the available evidence. Finally, options will be discussed on how to improve these recommendations.
2. Materials and Methods
2.1 Rapid Review of the Literature
Rapid reviews differ from standard systematic reviews over their use of ‘abbreviated’ methods at one or more stages of the literature review process [5,6,7]. For instance, they may use less comprehensive search strategies (e.g., searching fewer bibliographic databases) in the interest of expedience. This may be particularly important in the context where new clinical practice guidelines are being produced. PCPs need to make a decision about implementing the guidelines’ recommendations in their daily practice. Hence, the need for timely evidence to support the implementation of the CPS/CACAP’s recommendations. This is especially important given that the CPS/CACAP’s recommendations are due for a periodic review every year.
For this rapid review, we conducted a search of the literature on the accuracy of parents’ behavioral concerns by identifying documents that cited the study by Glascoe and her colleagues [4] (i.e., ‘forward’ citation tacking). Citation tracking was done using Web of Science (Core Collection; 17 October 2018) and SCOPUS (19 October 2018). In both databases, all document types were considered including article, review, letter, proceeding/conference paper, discussion (Web of Science only), book chapter (SCOPUS only), editorial material, short survey (SCOPUS only) and note. (Google Scholar is another electronic database used for ‘forward’ citation tracking, but it is not recommended [8].) Also, we searched the Cochrane Database of Systematic Reviews (19 February 2019) and PROSPERO (22 February 2019) for published systematic reviews on the accuracy of parents’ behavioral concerns. We adopted this streamlined approach as an alternative to a more comprehensive search of the literature given the narrow scope of this review. Otherwise, we conducted the review according to the guidelines developed by the Cochrane Screening and Diagnostic Tests Methods Group [9]. Also, we used the PRISMA-DTA guidelines for reporting the results of the review [10].
Among the documents identified by the search, we selected original studies that met the inclusion criteria (see below). We proceeded in two steps. In the first step, study selection was based on abstract and title alone. This step was carried out independently by all review authors (i.e., RHB, MC and FT). If a study was selected by at least one review author it was retained for the next selection step. In the second step, full-text selection was done independently by the three review authors. Any disagreement among review authors was resolved through discussion. Also, the reference lists of the selected studies were used for identifying other potentially relevant studies, and so on and so forth (i.e., ‘backward’ citation tracking).
To be selected, a study had to meet the following inclusion criteria: 1. Participants had to be primary-school children or younger. 2. Parents’ concerns had to be specific to behavior and systematically elicited. 3. The problem of interest had to be specific to behavior. 4. The target condition had to be disruptive/behavior problem in children. 5. The study design had to be naturalistic, with at least 10 children being identified as having a disruptive/behavior problem and at least 10 parents having behavioral concerns. 6. The four possible outcomes of interest (i.e., numbers of true positive, false negative, true negative and false positive outcomes) or their equivalents had to be given or at least deducible from the data.
For each study meeting these inclusion criteria (including the study by Glascoe and her colleagues [4]), data on the outcomes of interest were extracted for further analyses (see below).
2.2 Measuring the Accuracy of Parents’ Behavioral Concerns
One way to appreciate the efficiency of parents’ behavioral concerns at ruling in the presence of a DBP is to consider their predictive value positive (PVP) and specificity (SP) (see Results section for formulae). Another, though less familiar index can also be used for that purpose; namely, the risk ratio RR1 (see Results section for formula) [11]. It is the ratio of the posterior odds in favor of a DBP when concerns are being raised over the prior odds in favor of a DBP. (The value of RR1 expected by chance alone is equal to 1.) The PVP, SP and RR1 indices are expressed on different scales. For instance, the minimum value PVP can take (i.e., its value expected by chance alone) is equal to the proportion of children with a DBP. In comparison, the minimum value SP can take is different and is equal to 1 minus the proportion of non-concerned parents. (Note that PVP and SP share the same maximum value, namely 1.) For ease of interpretation, it is useful to calibrate or re-scale all indices on a common scale with values ranging from 0 to 1. Following Kraemer [11], these indices can be calibrated using the weighted kappa coefficient κ(0,0) (see Results section for formulae). It measures the extent to which the observed value of PVP/SP/RR1 exceeds that expected by chance alone. Once calibrated, PVP, SP and RR1 yield the same value of κ(0,0). For parents’ behavioral concerns to constitute a reliable means to rule in the presence of a DBP, the value of κ(0,0) needs to be at least substantial (i.e., 0.6 < κ ≤ 0.8) [12]. Otherwise, with poor/slight (i.e., κ ≤ 0.2), fair (i.e., 0.2 < κ ≤ 0.4) or even moderate (i.e., 0.4 < κ ≤ 0.6) values of κ(0,0), PCPs may need more information before making a decision in favor of screening.
Similarly, one way to appreciate the efficiency of parents’ behavioral concerns at ruling out the presence of a DBP is to consider their predictive value negative (PVN), sensitivity (SE) and risk ratio RR3 (see Results section for formulae) [11]. RR3 is the ratio of the posterior odds against a DBP when concerns are being raised over the prior odds against a DBP. Its value expected by chance alone is equal to 1. Because PVN, SE and RR3 are expressed on different scales, it is useful to re-scale them on a common scale with values ranging from 0 to 1. Following Kraemer [11], these indices can be calibrated using the weighted kappa coefficient κ(1,0) (see Results section for formulae). It measures the extent to which the observed value of PVN/SE/RR3 exceeds that expected by chance alone. Once calibrated, all three indices yield the same value of κ(1,0). The value of κ(1,0) needs to be at least substantial for parents’ behavioral concerns to constitute a reliable means to rule out the presence of a DBP. Otherwise, PCPs may need more information before making a decision against screening.
Finally, the overall efficiency (EFF) of parents’ behavioral concerns at distinguishing between children with a DBP and those without can be estimated as the sum of two proportions; namely, the proportions of true positive and true negative outcomes. Following Kraemer [11], this measure can be calibrated using the weighted kappa coefficient κ(0.5,0) (see Results section for formula). It measures the extent to which the observed value of EFF exceeds that expected by chance alone.
3. Results
3.1 Rapid Review on the Accuracy of Parents’ Behavioral Concerns
Before proceeding with our review, we consulted two international registries of systematic reviews to check for published systematic reviews on the accuracy of parents’ behavioral concerns. The Cochrane Database of Systematic Reviews was searched (19 February 2019) under the following types/topics: “Mental health” (585 reviews); “Developmental, psychosocial & learning problems” (179 reviews); “Diagnostic”/“Diagnosis”/“Diagnostic test accuracy” (112 reviews). One of the review author (RHB) checked for relevant published systematic reviews, and found none. Also, we searched PROSPERO (22 February 2019) using the following search words: “Population” (64 reviews); “Diagnostic” (40 reviews); “Test” (38 reviews); “Prevalence (25 reviews); “Concerns” (6 reviews); “Diagnostic accuracy” (5 reviews); “Test accuracy” (1 review). We used the following filters: Health area of review (“Mental health and behavioural conditions”); Type and method of the review (“Systematic review”); Source of the review (“Exclude animal”); Status of the review (“Published”). Once again, no relevant published systematic reviews were found. These results suggest that a systematic review on the accuracy of parents’ behavioral concerns has yet to be published.
Figure 1 (a flow diagram) presents the main stages of our rapid review of the literature. First, we proceeded with tracking citations for the study by Glascoe and her colleagues [4] (see Figure 1). Using Web of science (17 October 2018) and SCOPUS (19 October 2018), we identified 80 and 86 documents, respectively, with 69 documents identified by both databases.
Figure 1 PRISMA flow diagram for a rapid review on the accuracy of parents’ behavioral concerns. Adapted from Moher et al. [13].
All in all, 97 (i.e., 69 + 17 + 11) unique documents were identified via citation tracking. Each document was reviewed by the review authors based on title and abstract, except for 8 documents with no abstract (i.e., four letters, two chapters, one note and one editorial). The latter were reviewed based on full-text. From among the 97 documents, 22 studies were selected by at least one review author [14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35] (see Table S1). These studies were then reviewed for meeting the inclusion criteria by the review authors based on full-text. Of the 22 studies, 5 were excluded (4 were not original studies--and more like scoping reviews--and one was a letter) [26,27,28,31,33]. Of the remaining 17 studies, none met all six inclusion criteria (see Table S1). Many studies had either an index test for parents’ behavioral concerns (criterion #6) or a reference standard for disruptive/behavior problem with or without the corresponding target condition (criterion #4), but only one study by Mulhern and his colleagues [32] met these three criteria (see Table S1). However, it did not have a naturalistic study design. In fact, it was conducted with a sample of children referred to a pediatric clinic for ‘school-related learning and/or behavior problems’. [The target condition, ADHD (DSM-III-R)/ADD (DSM-III), was diagnosed in 38% of the children (i.e., 93/152). Particularly problematic is that a previous diagnosis of ADHD/ADD (mentioned by 7% of the parents) was included in the operational definition of ‘any parental concerns’]. At the end, none of the selected studies identified using citation tracking provided satisfactory evidence about the accuracy of parents’ behavioral concerns.
Second, 13 articles were identified as containing potentially relevant information about the accuracy of behavioral concerns by scanning the reference lists of the 22 studies mentioned above. Each cited article was reviewed by the review authors based on title and abstract, except for one article that could not be found. From among the 12 articles, 8 articles were selected by at least one review author [36,37,38,39,40,41,42,43]. They were then reviewed based on full-text. In the process, another article was identified as potentially relevant by reference lists scanning [44]. This article was then reviewed based on full-text. And, one more article was identified by scanning its reference list [45], and then reviewed based on full-text. Altogether, 10 articles were reviewed based on full-text, but none of the studies met all the inclusion criteria (see Table S2). In fact, not one study had both an index test for parents’ behavioral concerns and a reference standard for disruptive/behavior problem with the corresponding target condition (see Table S2).
Third, one article was identified by hand searching [46]. This article was reviewed by the review authors based on full-text. In the process, another article was identified as potentially relevant by reference lists scanning [47]. This article was then reviewed based on full-text. A third article was identified by scanning its reference list [48]. Finally, one more article was identified as it reported on the data from a more recent version of the survey on which the previous two articles were based [49]. Altogether, 4 articles were identified and reviewed based on full-text. One study by Reijneveld and his collaborators [46] met all the inclusion criteria (see below for more details), while the others met all inclusion criteria but the sixth (i.e., the authors did not provide their results separately for children and adolescents) (see Table S3). Overall, these results suggest that the study by Reijneveld and his collaborators [46] is the only published study since the landmark study by Glascoe and her colleagues [4] to provide direct evidence about the accuracy of parents’ concerns about their child’s behavior.
3.2 The Accuracy of Parents’ Behavioral Concerns
In the study by Glascoe and her colleagues [4] (n = 95), behavioral concerns were elicited using the Parents’ Evaluation of Developmental Status (PEDS) [50]. The PEDS is a widely used instrument for eliciting parents’ concerns in the primary care setting [51]. The proportion of concerned parents was estimated at 0.358 (i.e., 34/95; se = 0.049; 95% CI = 0.262-0.454). Also, DBPs were assessed using a screener for disruptive behavior in children as young as two years of age, namely the Eyberg Child Behavior Inventory (ECBI) [52]. The ECBI is recommended for use in pediatric care settings as a screening instrument for behavioral problems in children [3,53]. The proportion of children with a DBP was estimated at 0.211 (i.e., 20/95; se = 0.042; 95% CI = 0.129-0.293).
In the study by Reijneveld and his colleagues [46] (n = 4054), parents’ concerns about their 14-144 month-old children were elicited via a questionnaire using a question about the occurrence of behavioral concerns in the past 12 months for which parents felt they needed assistance/advice. The proportion of concerned parents was estimated at 0.247 (i.e., 1002/4054; se = 0.007; 95% CI = 0.234-0.260). Also, DBPs were assessed using two screeners for externalizing behavior problems, namely, the Dutch versions of the Child Behavior Checklist (CBCL) [54,55] and the Infant-Toddler Social and Emotional Assessment (ITSEA) for 14-month-old children [56]. The proportion of children with a DBP was estimated at 0.073 (i.e., 295/4054; se = 0.004; 95% CI = 0.065-0.081).
Table 1 presents the estimates for the four outcomes of interest and their standard errors (and 95% CI). For instance, in the study by Glascoe and her colleagues [4], the proportion of true positive outcomes was estimated at 0.147 (i.e., 14/95). Other estimates were obtained in the same way.
Table 1 Efficiency of parents’ behavioral concerns at distinguishing between children with a DBP and those without.
Table 2 presents indices measuring the efficiency of parents’ behavioral concerns at ruling in the presence of a DBP. As expected, estimated values of PVP, SP and RR1 did differ as these indices are expressed on different scales (e.g., 0.211-1.0 for PVP vs. 0.642-1.0 for SP in the study by Glascoe and her colleagues [4]). For instance, in the study by Glascoe and her colleagues [4], concerns’ PVP and SP were estimated at 0.412 (i.e., 0.147/0.358) and 0.733 (i.e., 0.579/0.790), respectively. Also, RR1 was estimated at 2.63 (i.e., 0.700/0.267); hence, in the study by Glascoe and her colleagues [4], the odds in favor of a DBP when a concern is being raised (i.e., 0.147/0.211 or 0.700) were 2.63 times higher than a priori (i.e., 0.211/0.790 or 0.267). To what extent did the estimated value of PVP/SP/RR1 exceed that expected by chance alone? The calibration of these indices yielded fair and poor/slight values of κ(0,0), with estimates of 0.255 and 0.094 in the studies by Glascoe and her colleagues [4], and Reijneveld and his colleagues [46], respectively. For instance, in the study by Glascoe and her colleagues [4], the difference between the estimated value of PVP/SP/RR1 and that expected by chance alone represented about one fourth of the maximum value this difference could actually take. Together, these results suggest that the mere presence of behavioral concerns does not provide PCPs with a reliable means to rule in the presence of a DBP in children.
Table 2 Efficiency of parents’ behavioral concerns at ruling in the presence of a DBP.
Table 3 presents the indices measuring the efficiency of parents’ behavioral concerns at ruling out the presence of a DBP. As expected, estimated values of PVN, SE and RR3 did differ as these indices are expressed on different scales (e.g., 0.790-1.0 for PVN vs. 0.358-1.0 for SE in the study by Glascoe and her colleagues [4]). For instance, in the study by Glascoe and her colleagues [4], concerns’ PVN and SE were estimated at 0.902 (i.e., 0.579/0.642) and 0.700 (i.e., 0.147/0.211), respectively. Also, RR3 was estimated at 2.45 (i.e., 9.17/3.75); hence, in the study by Glascoe and her colleagues [4], the odds against a DBP when a concern is not being raised (i.e., 0.579/0.063 or 9.17) were 2.45 times higher than a priori (i.e., 0.790/0.211 or 3.75). To what extent did the estimated value of PVN/SE/RR3 exceed that expected by chance alone? The calibration of these indices yielded moderate and fair values of κ(1,0), with estimates of 0.533 and 0.392 in the studies by Glascoe and her colleagues [4], and Reijneveld and his colleagues [46], respectively. For instance, in the study by Glascoe and her colleagues [4], the difference between the estimated value of PVN/SE/RR3 and that expected by chance alone represented about half of the maximum value this difference could actually take. Together, these results suggest that the mere absence of behavioral concerns does not provide PCPs with a reliable means to rule out the presence of a DBP in children.
Table 3 Efficiency of parents’ behavioral concerns at ruling out the presence of a DBP.
Finally, parents’ behavioral concerns’ overall efficiency, EFF, was estimated at 0.726 (i.e., 0.147 + 0.579) and 0.759 (i.e., 0.039 + 0.720) in the study by Glascoe and her colleagues [4], and Reijneveld and his colleagues [46], respectively (see Table 1). Note that the values of EFF expected by chance alone were 0.582 and 0.716, respectively. The calibration of EFF yielded fair and poor/slight values of κ(0.5,0), with estimates of 0.345 and 0.151, respectively (see Table 1). All in all, these results suggest that behavioral concerns do not provide PCPs with an accurate means to distinguish between children with a DBP and those without.
4. Discussion
There is a consensus about the importance of preventing DBPs in children [57,58]. Although regularly scheduled well-baby/child visits do represent an excellent opportunity to prevent DBPs via the monitoring children’s behavioral development [51,59,60], the current CPS/CACAP’s recommendations about screening for DBPs (i.e., screening if a behavioral concern is being raised by parents and otherwise not screening) appear to fall short in many respects.
First, evidence about the accuracy of parents’ concerns about their child’s behavior, although of good quality, appears rather limited. Our results suggest that the study by Glascoe and her colleagues [4], and the one by Reijneveld and his colleagues [46] are the only source of direct evidence bearing on this issue. Note that studies based on the (1999 and 2004) surveys of the mental health of children and adolescents in Great Britain did address this issue [47,48,49], but they did not report their results separately for children and adolescents. One reason for the paucity of studies may be publication bias against replication studies (e.g., exact or literal replications) [61]. Comparing the quality of the two studies in question, the one by Glascoe and her colleagues [4] may be better because its index test, the PEDS [50], is a well-validated test for eliciting parents’ concerns. That may explain, at least in part, why weighted kappa estimates were higher in the former than the latter. Otherwise, both studies suffer from what is a very common limitation of diagnostic test accuracy studies; namely, the absence of a ‘gold standard’ for the target condition (i.e., DBPs in children).
Second, parents’ behavioral concerns do not appear to provide PCPs with enough information to decide on the most appropriate course of action. Our results suggest that the mere presence of behavioral concerns does not provide PCPs with enough information to decide in favor of screening. Also, our results suggest that the mere absence of behavioral concerns does not provide PCPs with enough information to decide against screening. Most preoccupying is the lack of efficiency of parents’ behavioral concerns at ruling in the presence of a DBP in children. PCPs should be especially cautious when deciding in favor of screening given all the factors that might otherwise limit its usefulness (and, more generally, the one of secondary prevention of mental health problems in children) [62]. That puts the onus on the PCP to prove that screening is necessary. And, for that matter, the evidence in favor of ruling in the presence of a DBP needs to be strong, at least much stronger than the one provided by the mere presence of behavioral concerns. Hence, once behavioral concerns are being raised by parents, PCPs may need additional information before they can decide in favor of screening.
One way to gather the necessary information is to combine the eliciting of parents’ behavioral concerns with parents’ reports on their children’s disruptive problem behaviors. The latter can be used to identify children who, compared to other children of the same age and sex in the general population, exhibit disruptive problem behaviors on a frequent/extreme basis [63,64]. A modified algorithm for monitoring children’s behavior at well-baby/child visits would be the following: First, elicit parents’ behavioral concerns. If concerns are being raised, then proceed with obtaining parents’ ratings on specific disruptive problem behaviors. Refer a child for screening if both concerns and ratings suggest the presence of a DBP. Otherwise, do not refer him/her for screening.
Another way to increase the efficiency of parents’ behavioral concerns at ruling in the presence of a DBP in children is to combine the information gathered at two or more well-baby/child visits. A modified algorithm would be the following: First, elicit parents’ behavioral concerns at each visit. Refer a child for screening if concerns are being raised at all visits. Otherwise, do not refer him/her for screening. Also, this strategy could be combined with the previous one, using both concerns and ratings to rule in the presence of a DBP in children at any given well-baby/child visit. In this way, the usefulness of screening could be improved as there is emerging epidemiological evidence that the continuity of a particular DBP (at least during late toddlerhood and early childhood) may be much greater for children who exhibited the DBP in question earlier in life [65]. Future research should systematically investigate and compare the effectiveness of different monitoring strategies using, for instance, medical decision analysis [66].
4.1 Limitations
The search strategy used in this study was less comprehensive than the one generally advocated for in a systematic review of diagnostic test accuracy studies [9]. The focus was on peer-reviewed journal articles and not on the so-called grey literature (e.g., dissertations), with the risk of missing, for instance, some yet unpublished material. But there is emerging evidence that abbreviated searches may not affect the accuracy of rapid reviews’ conclusions [67]. At least, when, as in this study, researchers are combining the search of at least one database with references lists checking. Another limitation of this study is that it did not consider whether the CPS/CACAP’s recommendations were produced according to established standards [68]. In fact, the authors of the CPS/CACAP’s recommendations did not report on the methods used to develop their recommendations. In a recent review, Andrade and his collaborators [69] investigated clinical practice guidelines (published before June 2017) for disruptive behavior in children and youth. They found that relatively few guidelines met minimum quality standards for rigor of development (i.e., 9 out of 29; 5 out of 9 met high quality standards).
5. Conclusions
Regularly scheduled well-baby/child visits do represent an excellent opportunity to prevent DBPs via the monitoring children’s behavioral development. However, clinical practice guidelines about behavioral screening are not always trustworthy and/or supported by the available evidence. The current recommendations of the CPS/CACAP are undermined by the lack of accuracy of parents’ concerns about their child’s behavior. PCPs should gather additional information before deciding in favor of screening.
Acknowledgments
We would like to thank Karine Fournier for her advices on searching electronic databases. Also, we would like to thank Marc Bélanger for his help with the figure. Finally, we would like to thank Matthew McInnes for editorial advices.
Author Contributions
RHB Conception, databases/registries searching, citations tracking, reference lists scanning, manual searching, title/abstract and full-text study selection, data extraction and analysis, drafting, editing, revising. MC Title/abstract and full-text study selection, data extraction, editing, revising. FT Title/abstract and full-text study selection, data extraction, revising. KPB Editing, revising.
Competing Interests
The authors have no conflict of interest to declare.
Additional Materials
The following additional tables are uploaded at the page of this paper.
1. Table S1: Articles identified by citations tracking and selected by at least one review author.
2. Table S2: Articles identified by reference lists scanning and selected by at least one review author based on title and abstract.
3. Table S3: Articles identified via manual searching.
References
- Committee on Practice and Ambulatory Medicine, Bright Futures Periodicity Schedule Workgroup, Richerson JE, Abularrage JJ, Almendarez YM, Arauz Boudreau AD, et al. 2019 recommendations for preventive pediatric health care. Pediatrics. 2019; 143: e20183971. [CrossRef]
- Lipkin PH, Macias MM, Council on Children with Disabilities, Section on Developmental and Behavioral Pediatrics, Norwood Jr KW, Brei TJ, Davidson LF, et al. Promoting optimal development: Identifying infants and young children with developmental disorders through developmental surveillance and screening. Pediatrics. 2020; 145: e20193449. [CrossRef]
- Charach A, Bélanger SA, McLennan JD, Nixon MK. Screening for disruptive behaviour problems in preschool children in primary health care settings. Paediatr Child Health. 2017; 22: 478-484. [CrossRef]
- Glascoe FP, MacLean WE, Stone WL. The importance of parents' concerns about their child's behavior. Clin Pediatr (Phila). 1991; 30: 8-11. [CrossRef]
- Abrami PC, Borokhovski E, Bernard RM, Wade CA, Tamim R, Persson T, et al. Issues in conducting and disseminating brief reviews of evidence. Evid Policy. 2010; 6: 371-389. [CrossRef]
- Khangura S, Konnyu K, Cushman R, Grimshaw J, Moher D. Evidence summaries: The evolution of a rapid review approach. Syst Rev. 2012; 1: 10. [CrossRef]
- Thomas J, Newman M, Oliver S. Rapid evidence assessments of research to inform social policy: Taking stock and moving forward. Evid Policy. 2013; 9: 5-27. [CrossRef]
- Falagas ME, Pitsouni EI, Malietzis GA, Pappas G. Comparison of PubMed, Scopus, Web of Science, and Google Scholar: Strengths and weaknesses. FASEB J. 2008; 22: 338-342. [CrossRef]
- Deeks J, Bossuyt P, Takwoingi Y, Leeflang M, Flemyng E, Mellor L. Cochrane handbook for systematic reviews of diagnostic test accuracy (version 1.0.0) [Internet]. The Cochrane Collaboration; 2013. Available from: https://methods.cochrane.org/sdt/handbook-dta-reviews.
- McInnes MDF, Moher D, Thombs BD, McGrath TA, Bossuyt PM, Clifford T, et al. Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: The PRISMA-DTA statement. JAMA. 2018; 319: 388-396. [CrossRef]
- Kraemer HC. Evaluating medical tests: Objective and quantitative guidelines. Newbury Park: SAGE Publications; 1992.
- Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977; 33: 159-174. [CrossRef]
- Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 2009; 6: e1000097. [CrossRef]
- Spijkers W, Jansen DEMC, Reijneveld SA. The impact of area deprivation on parenting stress. Eur J Public Health. 2012; 22: 760-765. [CrossRef]
- Limbos MM, Joyce DP. Comparison of the ASQ and PEDS in screening for developmental delay in children presenting for primary care. J Dev Behav Pediatr. 2011; 32: 499-511. [CrossRef]
- Chung CY, Liu WY, Chang CJ, Chen CL, Tang SFT, Wong AMK. The relationship between parental concerns and final diagnosis in children with developmental delay. J Child Neurol. 2011; 26: 413-419. [CrossRef]
- Sices L, Stancin T, Kirchner L, Bauchner H. PEDS and ASQ developmental screening tests may not identify the same children. Pediatrics. 2009; 124: e640-647. [CrossRef]
- McGrew S, Malow BA, Henderson L, Wang L, Song Y, Stone WL. Developmental and behavioral questionnaire for autism spectrum disorders. Pediatr Neurol. 2007; 37: 108-116. [CrossRef]
- Hacker KA, Myagmarjav E, Harris V, Suglia SF, Weidner D, Link D. Mental health screening in pediatric practice: Factors related to positive screens and the contribution of parental/personal concern. Pediatrics. 2006; 118: 1896-1906. [CrossRef]
- Rydz D, Srour M, Oskoui M, Marget N, Shiller M, Birnbaum R, et al. Screening for developmental delay in the setting of a community pediatric clinic: A prospective assessment of parent-report questionnaires. Pediatrics. 2006; 118: e1178-1186. [CrossRef]
- Blanchard LT, Gurka MJ, Blackman JA. Emotional, developmental, and behavioral health of american children and their families: A report from the 2003 national survey of children's health. Pediatrics. 2006; 117: e1202-1212. [CrossRef]
- Ellingson KD, Briggs-Gowan MJ, Carter AS, Horwitz SM. Parent identification of early emerging child behavior problems: Predictors of sharing parental concern with health providers. Arch Pediatr Adolesc Med. 2004; 158: 766-772. [CrossRef]
- Chen IC, Lee HC, Yeh GC, Lai CH, Chen SC. The relationship between parental concerns and professional assessment in developmental delay in infants and children--A hospital-based study. J Chin Med Assoc. 2004; 67: 239-244.
- Glascoe FP. Parents' evaluation of developmental status: How well do parents' concerns identify children with behavioral and emotional problems? Clin Pediatr (Phila). 2003; 42: 133-138. [CrossRef]
- Malhi P, Singhi P. Can parental concerns detect children with behavioral problems? Stud Psychol (Bratisl). 2002; 44: 359-366.
- Glascoe FP. Evidence-based approach to developmental and behavioural surveillance using parents' concerns. Child Care Health Dev. 2000; 26: 137-149. [CrossRef]
- Glascoe FP. The value of parents’ concerns to detect and address developmental and behavioural problems. J Paediatr Child Health. 1999; 35: 1-8. [CrossRef]
- Glascoe FP, Bell RT. Behavioral and developmental problems in children. Patient Care. 1998; 32: 63-83.
- Glascoe FP. Parents' concerns about children's development: Prescreening technique or screening test? Pediatrics. 1997; 99: 522-528. [CrossRef]
- Glascoe FP, Sandler H. Value of parents' estimates of children's developmental ages. J Pediatr. 1995; 127: 831-835. [CrossRef]
- Glascoe FP, Dworkin PH. The role of parents in the detection of developmental and behavioral problems. Pediatrics. 1995; 95: 829-836. [CrossRef]
- Mulhern S, Dworkin PH, Bernstein B. Do parental concerns predict a diagnosis of attention-deficit hyperactivity disorder? J Dev Behav Pediatr. 1994; 15: 348-352. [CrossRef]
- Glascoe FP. The infant or young child with developmental delay. N Engl J Med. 1994; 331: 56. [CrossRef]
- Glascoe FP. It's not what it seems. The relationship between parents' concerns and children with global delays. Clin Pediatr (Phila). 1994; 33: 292-296. [CrossRef]
- Diamond KE. The role of parents' observations and concerns in screening for developmental delays in young children. Topics Early Child Spec Educ. 1993; 13: 68-81. [CrossRef]
- Sayal K, Taylor E. Detection of child mental health disorders by general practitioners. Br J Gen Pract. 2004; 54: 348-352.
- Lynch TR, Wildman BG, Smucker WD. Parental disclosure of child psychosocial concerns: Relationship to physician identification and management. J Fam Pract. 1997; 44: 273-280.
- Wildman BG, Kizilbash AH, Smucker WD. Physicians' attention to parents' concerns about the psychosocial functioning of their children. Arch Fam Med. 1999; 8: 440-444. [CrossRef]
- Oberklaid F, Dworkin PH, Levine MD. Developmental-behavioral dysfunction in preschool children. Descriptive analysis of a pediatric consultative model. Am J Dis Child. 1979; 133: 1126-1131. [CrossRef]
- Rogers BT, Booth LJ, Duffy LC, Hassan MB, McCormick P, Snitzer J, et al. Parents’ developmental perceptions and expectations for their high-risk infants. J Dev Behav Pediatr. 1992; 13: 102-107. [CrossRef]
- Scholle SH, Whiteside L, Kelleher K, Bradley R, Casey P. Health status of preterm low-birth-weight infants. Comparison of maternal reports. Arch Pediatr Adolesc Med. 1995; 149: 1351-1357. [CrossRef]
- Dulcan MK, Costello EJ, Costello AJ, Edelbrock C, Brent D, Janiszewski S. The pediatrician as gatekeeper to mental health care for children: Do parents' concerns open the gate? J Am Acad Child Adolesc Psychiatry. 1990; 29: 453-458. [CrossRef]
- Glascoe FP, MacLean WE. How parents appraise their child's development. Fam Relat. 1990; 39: 280-283. [CrossRef]
- Wildman BG, Kinsman AM, Logue E, Dickey DJ, Smucker WD. Presentation and management of childhood psychosocial problems. J Fam Pract. 1997; 44: 77-84.
- Horwitz SM, Leaf PJ, Leventhal JM, Forsyth B, Speechley KN. Identification and management of psychosocial and developmental problems in community-based, primary care pediatric practices. Pediatrics. 1992; 89: 480-485. [CrossRef]
- Reijneveld SA, de Meer G, Wiefferink CH, Crone MR. Parents' concerns about children are highly prevalent but often not confirmed by child doctors and nurses. BMC Public Health. 2008; 8: 124. [CrossRef]
- Ford T, Sayal K, Meltzer H, Goodman R. Parental concerns about their child's emotions and behaviour and referral to specialist services: General population survey. BMJ. 2005; 331: 1435-1436. [CrossRef]
- Meltzer H, Gatward R, Goodman R, Ford T. Mental health of children and adolescents in Great Britain. Int Rev Psychiatry. 2003; 15: 185-187. [CrossRef]
- Green H, McGinnity Á, Meltzer H, Ford T, Goodman R. Mental health of children and young people in Great Britain, 2004. New York: Palgrave Macmillan; 2005. [CrossRef]
- Glascoe FP. Collaborating with parents: Using parents' evaluation of developmental status (PEDS) to detect and address developmental and behavioral problems. Nolensville: PEDSTest.com, LLC; 2013.
- Hagan JF, Shaw JS, Duncan PM. Bright futures: Guidelines for health supervision of infants, children and adolescents. 4th ed. Elk Grove Village: American Academy of Pediatrics; 2017. [CrossRef]
- Eyberg S, Pincus D. ECBI™ and SESBI-R™. Psychological Assessment Resources; 1999. Available from: https://www.parinc.com/products/pkey/97.
- Grant R, Gracy D, Brito A. Developmental and social-emotional screening instruments for use in pediatric primary care in infants and young children. New York: Children’s Health Fund; 2010.
- Achenbach TM, Rescorla LA. Manual for the ASEBA school-age forms & profiles. Burlington: University of Vermont, Research Center for Children, Youth & Families; 2001.
- Achenbach TM, Rescorla LA. Manual for the ASEBA preschool forms & profiles. Burlington: University of Vermont, Research Center for Children, Youth & Families; 2000.
- Carter AS, Briggs-Gowan MJ. Infant toddler social and emotional assessment (ITSEA) manual (version 1.1). New Haven: The Connecticut Early Development Project; 2001.
- Petitclerc A, Tremblay RE. Childhood disruptive behaviour disorders: Review of their origin, development, and prevention. Can J Psychiatry. 2009; 54: 222-231. [CrossRef]
- Waddell C, McEwan K, Peters RD, Hua JM, Garland O. Preventing mental disorders in children: A public health priority. Can J Public Health. 2007; 98: 174-178. [CrossRef]
- Appendix S4: The case for routine mental health screening. Pediatrics. 2010; 125: S133-S139. doi:10.1542/peds.2010-0788J. [CrossRef]
- Weitzman C, Wegner L, Section on Developmental and Behavioral Pediatrics, Committee on Psychosocial Aspects of Child and Family Health, Council on Early Childhood, Society for Developmental and Behavioral Pediatrics, et al. Promoting optimal development: Screening for behavioral and emotional problems. Pediatrics. 2015; 135: 384-395. [CrossRef]
- Yong E. Replication studies: Bad copy. Nature. 2012; 485: 298-300. [CrossRef]
- Hall DMB, Elliman D. Health for all children: Revised fourth edition. Oxford: Oxford University Press; 2006. [CrossRef]
- Baillargeon RH, Zoccolillo M, Keenan K, Côté S, Pérusse D, Wu HX, et al. Gender differences in physical aggression: A prospective population-based survey of children before and after 2 years of age. Dev Psychol. 2007; 43: 13-26. [CrossRef]
- Baillargeon RH, Sward GD, Keenan K, Cao G. Opposition-defiance in the second year of life: A population-based cohort study. Infancy. 2011; 16: 418-434. [CrossRef]
- Baillargeon RH, Morisset A, Keenan K, Normand CL, Séguin JR, Japel C, et al. Development of disruptive behaviors in young children: A prospective population-based cohort study. Infant Ment Health J. 2012; 33: 633-650. [CrossRef]
- Hunink MGM, Weinstein MC, Wittenberg E, Drummond MF, Pliskin JS, Wong JB, et al. Decision making in health and medicine: Integrating evidence and values. 2nd ed. Cambridge: Cambridge university press; 2014. [CrossRef]
- Nussbaumer-Streit B, Klerings I, Wagner G, Heise TL, Dobrescu AI, Armijo-Olivo S, et al. Abbreviated literature searches were viable alternatives to comprehensive searches: A meta-epidemiological study. J Clin Epidemiol. 2018; 102: 1-11. [CrossRef]
- Qaseem A, Forland F, Macbeth F, Ollenschläger G, Phillips S, van der Wees P. Guidelines international network: Toward international standards for clinical practice guidelines. Ann Intern Med. 2012; 156: 525-531. [CrossRef]
- Andrade BF, Courtney D, Duda S, Aitken M, Craig SG, Szatmari P, et al. A systematic review and evaluation of clinical practice guidelines for children and youth with disruptive behavior: Rigor of development and recommendations for use. Clin Child Fam Psychol Rev. 2019; 22: 527-548. [CrossRef]