Comparison of a set of cognitive ability screening test for primary school-aged children in Indonesia

Meitha Togas,¹ Hartono Gunardi,¹ Rini Sekartini,¹ Sri Redatin Retno Pudjiati,² Eef Hogervorst³

Background

Monitoring children’s intellectual development is important to align their educational needs. This study compared nonverbal intelligence subtest of the cognitive test battery for individuals with or without intellectual disability (CIID) to measure visual logical reasoning, verbal intelligence subtest (verbal fluency [VF]–animal test) to assess vocabulary and semantic memory, and the modified Indonesian version of Hopkins verbal learning test (IHVLT) to assess learning ability and episodic memory, against the Wechsler intelligence scale for children (WISC) as an alternative cognitive screening tool in school-aged children.

 

Methods

A cross-sectional study was conducted in 145 grade 1–6 elementary school students in Jakarta between March and May 2017. Performance on the CIID subtest, VF, and modified IHVLT was assessed by a pediatrician, and the WISC test was performed by a psychologist. The associations between variables were evaluated using nonparametric Spearman’s rank correlations, regressions, and box plots.

 

Results

The correlation between total intelligence quotient (IQ) WISC and CIID was 0.42 (p<0.001) and 0.34 (p<0.001) for both IHVLT and VF. Box plots suggested possible discriminative capacity of CIID but insufficient specificity. However, CIID was associated with stunting (height) and health (weight) independent of age, sex, father’s education, or income.

 

Conclusions

The CIID subtest series, VF, and modified IHVLT correlated with total IQ WISC. However, CIID may be better to test predictors of poor cognitive performance in primary school-aged children than an IQ screening test.

 

Keywords

child, cognition, growth and development, intelligence

Assessment of cognitive ability early in life is important to establish the need for specialized support and pathways to promote optimal academic achievement. Low and borderline intelligence quotients (IQs) were reported in 88 (6.9%) of 1,284 Indonesian school-aged children in 2013.¹ Prevalence of stunting, which is associated with poor cognitive function and development, was found in 30.8% of under-five Indonesian children.² Learning difficulties due to cognitive issues, thus, need to be established in Indonesian primary school-aged children to identify their risk factors including stunting and help optimize children’s education and future socioeconomic prospects.

The most widely used cognitive measurement tool for children is the Wechsler intelligence scale for children (WISC).³ However, the WISC has limitations as it should be assessed by experienced psychologists. This can be an issue in developing countries, such as Indonesia, because there are limited numbers of pediatric psychologists, and they are unevenly distributed throughout the country. Moreover, the WISC takes approximately 45–60 min of examination time and is considered costly, in terms of purchase costs, time taken for children’s assessment, and time needed by a certified pediatric psychologist.4

A cognitive screening test called the cognitive computerized test battery for individuals with intellectual disabilities (CCIID) was developed to allow cross-cultural assessment of intellectual ability without the need for qualified psychologists.5,6 The CCIID is based on the same principles as other nonverbal IQ tests, such as the Cattell culture fair, Snijders-Oomen nonverbal intelligence test-revised (SON-R), and Raven’s matrices. It correlated well with the Wechsler adult intelligence scale, and SON-R and was found to have excellent psychometric qualities. This test was then modified as a paper-and-pencil test to be used for individuals, both with or without intellectual disability, and was renamed as the cognitive test battery for individuals with and without intellectual disability (CIID).⁷ The CIID test is easier to apply, faster, and cheaper than the WISC and could be applied by non-psychologists, such as general practitioners, nurses, teachers, pediatricians, and researchers. We used one of the nonverbal logical reasoning subtests called series to explore the possibility of a CIID subtest as an alternative to screen Indonesian children’s cognitive development. We added verbal tests including verbal fluency (VF) intelligence subtest, which recalls names of different animals (language and semantic memory), and the modified Indonesian Hopkins verbal learning test (IHVLT), which assesses verbal episodic memory and learning ability. We hypothesized that these tests could be used alone or together as a preliminary screening test of cognitive problems before children are referred to a psychologist for more in-depth cognitive assessments. This study aimed to determine the correlations between the CIID subtest series, VF, and modified IHVLT with WISC subtests on healthy Indonesian children aged 6 to 12 years and to assess predictors of their cognitive performance including stunting and weight as indicators of health.

 

 

Study population and design

A cross-sectional study was conducted in an elementary school of 03 Menteng in Central Jakarta between March 30 and May 4, 2017. The study included children from 6 to 12 years old who were enrolled in the school at the time of study. Children who had attention-deficit/hyperactivity disorder (ADHD), intellectual disability (including Down syndrome), congenital heart disease, and chronic diseases (such as severe asthma and thalassemia) were excluded by physical examination. Teachers and parents were informed about the study, and parents had signed informed consent before recruitment. All children and teachers consented to be included in the study. The study was approved by the Ethics Committee of the Faculty of Medicine, Universitas Indonesia, Cipto Mangunkusumo Hospital (No: 165/UN2.F1/ETIK/2017) before study onset.

The estimated required sample size for this study was 144 subjects. A total of 24 subjects were recruited by random sampling for each grade (grades 1 to 6), except for grade 6 in which 25 subjects were recruited. Demographic (age, sex, parental education, and income) and anthropometric (weight and height using the same standardized instrument) data were collected from each subject. Board-certified psychologists from the Faculty of Psychology Universitas Indonesia performed the WISC tests separately. The children’s WISC IQ was classified based on the Wechsler intelligence classification as follows: intellectually disabled (IQ≤65), borderline (IQ 66–79), low average (IQ 80–90), average (IQ 91–110), high average (IQ 111–119), superior (IQ 120–127), and very superior (IQ≥128).⁸ For analyses, the latter two groups were combined (see box plots) because of uneven sample distributions.

The CIID, VF, and modified IHVLT were tested together in one visit by a pediatrician. The CIID battery test measures nonverbal logical reasoning, which could impact educational performance, and includes fluid inductive reasoning and visual processing using subtest series versions A, B, and C. Participants had to choose one of four options to complete a set of four transformation forms. The test contains 39 questions, and each has an increasing gradient of difficulty levels, resulting in a maximum total score of 39 points.⁷

VF is a verbal IQ subtest and focuses on verbal semantic vocabulary and knowledge by asking participants to produce as many animal names as possible within a minute. One point was awarded for each animal named without repetition.7,9,10

The IHVLT, which was translated and back translated from the original version,7,9,11,12 focused on verbal episodic memory and learning ability. A list of 12 words from three categories, including animals (e.g., lion, horse, and tiger), human shelter (e.g., tent and hotel), and semi-precious stones (e.g., pearl and sapphire), was read out to participants. After recall, this list was repeated two more times. After the examiner finished reading out the list, participants were asked to recall the words including those mentioned at the earlier trials in any order. The maximum total immediate recall score possible is 36 points.7,9,11,12 The total verbal memory score (VS) was calculated as IHVLT total immediate recall plus the number of VF animals named correctly.

 

Data analysis

The data were processed using the SPSS software version 20 (IBM Corp., USA). The relationship between the CIID Series, VF, modified IHVLT, and WISC tests was evaluated using Spearman’s rank correlations and linear regression analyses to establish the mediation of demographic and health indicator variables on cognitive ability using stepwise backward analyses. None of the distributions were skewed, and all tests approached normal bell-shaped curves. Interpretation of the correlation coefficient (r) was based on the study of de Vaus¹³ where p<0.05 was regarded as significant: 0.00 no (linear) association, 0.01–0.09 trivial (linear) relationship, 0.10–0.29 low to moderate (linear) relationship, 0.30–0.49 moderate to substantial (linear) relationship, 0.50–0.69 substantial to very strong (linear) relationship, 0.70–0.89 very strong (linear) relationship, and 0.90+ near perfect.

 

 

The study included 145 students from an elementary school in 03 Menteng, Central Jakarta (Table 1). Almost half of the children were boys (66/145, 45.5%), with an average age of 9.7 (1.7) years.

 

 

The time to test the CIID, VF, and IHVLT ranged from 12 to 15 min. The mean score of the nonverbal CIID was 21.03 (5.79) (range 8–34), IHVLT 20.28 (5.96) (range 0–35), VF 16.66 (5.03) (range 6–29), and VS combined 37.01 (9.83) (range 16–64).

The average WISC test time was 60 min. The mean value of WISC nonverbal IQ performance was 101.82 (10.99) (range 79–138), WISC verbal IQ 108.28 (11.51) (range 80–142), and total IQ 105.72 (10.33) (range 83–141). Of 145 subjects, 99 (68.3%) had an average IQ classification as defined by WISC scores (Table 2).

 

 

Spearman’s rank correlation analysis was used to analyze the correlation between CIID and WISC scores. The correlation was 0.42 (p<0.001) between total IQ WISC and CIID and 0.34 (p<0.001) for both IHVLT and VF. Combining all tests (CIID + VF + IHVLT) rendered only a marginally higher correlation of 0.45 with total WISC IQ scores (p<0.001). The correlation of nonverbal CIID and performance IQ WISC score was 0.35 (p<0.001), with the regression formula Y = 87.75 + 0.67X (Y = WISC and X = nonverbal CIID; Figure 1). The correlation of VF and IHVLT combined and verbal IQ WISC score was 0.41 (p<0.001), with the regression formula Y = 90.56 + 0.48X (Y = WISC and X = verbal CIID; Figure 2).

 

 

 

 

Correlational analyses (Table 3) were used to assess relations between CIID series and IHVLT/VF and indicated that these were highly correlated with each other (0.48–0.59). The CIID series was significantly associated (p<0.01) not only with WISC subtests, such as the visuospatial block design (0.41) and object assembly (0.26) tests, but also with the verbal information (0.36) and Similarities (0.23) subtests. The IHVLT was also associated with the WISC Information (0.37) and similarities (0.25), but additionally with the verbal comprehension (0.19) and digit span (0.18) subtests. The IHVLT was also associated with the block design (0.20) subtest, possibly capturing its planning aspect. The VF was also associated with WISC information (0.37) and comprehension (0.32), and additionally with the coding (0.25) and block design (0.17) subtests. Arithmetic showed trends with all cognitive tests, but picture completion and arrangement were not significantly associated with any cognitive test.

 

 

Combining WISC IQ categories (“superior” and “very superior”) and comparing these WISC categories against CIID performance (Figure 3) suggested a reasonable indication of superior (>20 cut-off of CIID score) and very low IQ performance (<20), warranting further testing. However, specificity was low, with much overlap of scores in the middle categories. From the value of r = 0.45, we obtained a determinant coefficient of 20.4%, which means that only 20.4% of CIID scores predicted total IQ WISC scores. This may be because the CIID was not age adjusted, where the WISC was derived from age-adjusted scores. We did not have sufficient numbers of children to calculate age-adjusted scores in this study.

 

 

Stepwise backward linear regression analyses showed that the CIID raw scores were associated with age at testing (β = 0.23, p<0.05) and children’s sex (β = −0.14, p = 0.04, where especially younger boys outperformed girls overall, a difference that reduced with age). Educational level (class) had been excluded from analyses because of its high collinearity with age. The WISC total IQ scores were only associated with father’s education (β = 0.30, p<0.001; mother’s education had been excluded because of high collinearity with father’s education), and verbal IQ and performance IQ showed similar models. The IHVLT and VF were only associated with age of the child tested (β = 0.65, p = 0.001; β = 0.57, p<0.001), with high collinearity for weight and height, which were not entered in the model. The raw CIID scores in regression analyses were additionally (independently from age and sex) significantly associated with height (β = 0.57, p = 0.001, with taller children having better performance) and weight (β = −0.28, p = 0.02, with heavier children having slightly lower scores, but where curvilinear or cubic modeling would possibly have provided a better fit). There was only an independent trend for father’s education (β = 0.13, p = 0.06) to contribute to performance on the CIID. This model explained 39% of the variance (adjusted R squared). Combining CIID, IHVLT, and VF explained 55% of the variance with sex excluded and father’s education now contributing to analyses. When IQ was added, the model explained 70% of the variance. In post-hoc analyses, performance was also independent of parental income, which was not entered in the model, but showed collinearity with father’s education.

 

 

We found that the CIID subtest series could be used to provide an indication of risk predictors of children’s cognitive ability, such as height and weight. As an IQ screening test, it is probably less suited because of low specificity. The CIID, a modified paper and pencil-based test for individuals with or without intellectual disabilities, is a simplified version of the CCIID test.5,6 This study recruited normal and healthy school children. ADHD children were excluded as the disorder can affect their cognitive ability,¹⁴ similar to children with congenital heart disease,15,16 down syndrome,¹⁷ anemia,18,19 severe asthma,²⁰ and thalassemia.²¹ Children with other intellectual disabilities were also excluded because this study aimed to assess non-ID primary school children’s performance for a normative assessment. As the CIID was specifically developed for this lower end of the IQ spectrum, its performance would probably be more discriminative if these children had been included.

Based on this study, the CIID subtest series could be used as a screening method for predicting a child’s cognitive ability based on weight and height as indicators of health and stunted growth. Very high or low CIID scores (cut-off score 20) could identify when a child may need additional support or is exceptionally talented. However, such scores should then be followed up by a full WISC assessment as its specificity as an IQ screening test was inadequate.

In our subjects, VF scores ranged from 6 to 29, with a mean of 16.66 (5.03). This VF result was lower than another study involving 294 school-aged children aged 6–16 years in the Netherlands, which reported a mean of 18.76 (6.09).¹⁰ The lower mean VF scores in our subjects could be due to socioeconomic or crosscultural differences. In this study, the mean value of WISC nonverbal IQ performance was 101.82 (10.99) (range 79–138), WISC verbal IQ 108.28 (11.51) (range 80–142), and total IQ 105.72 (10.33) (range 83–141). The WISC performance IQ, verbal IQ, and total IQ in this study were higher than the WISC performance IQ, verbal IQ, and total IQ of school-aged children in a rural area in North Sumatra province, which were 81.08 (14.58), 88.10 (14.20), and 83.80 (13.14), respectively. These discrepancies might be due to the different socioeconomic status or cross-cultural differences (Sumatra’s population is mainly Minangkabau, whereas our cohort consisted mainly of Javanese and Sundanese children). Excluding children with anemia, which was not done in Sumatra, could also possibly explain differences in scores.²²

The subtest series from the CIID mainly focuses on nonverbal fluid reasoning ability and visual processing, which improve with age.⁷ Inductive and deductive reasoning are generally considered the hallmark of fluid intelligence. Visualization is a part of visual processing that involves the ability to generate, maintain, retrieve, and alter a well-structured visual image.²³⁻²⁵ As such, this CIID subtest only examined parts of the children’s intelligence compared with the WISC, which examines multiple cognitive abilities, including both fluid and crystalized intelligence.⁸ However, the advantage like other tests of its kind (Cattell culture fair and Raven’s matrices) is that the CIID subtest series is nonverbal and, therefore, more cross culturally applicable.

This study had several methodological limitations. First, we only used the pencil and paper test of the CIID subtest series but not the other subtests. A pencil and paper test is easy, fast to apply, and suitable for screening in daily practice, but using a computer or full battery may have improved correlations with WISC IQ scores. Second, the tests were not performed in the ideal quiet room without distraction, but in the library and school health room. Nevertheless, this drawback might increase its applicability in real-life situations. Despite the limitations, the study had several strengths. To our knowledge, this research was the first study to determine the correlation between CIID series, VF, IHVLT, and WISC in primary school-aged children in Indonesia. Moreover, the CIID series, VF, and IHVLT test were easy and cheap and could be done by a non-psychologist tester or clinician within a short time, requiring only 10–15 min. Using the CIID, but not WISC, may also allow identification of stunting as a predictor of poor cognitive function. In a large longitudinal study of over 1,500 children in different lower middle income countries, early stunting of growth was associated with poor cognitive performance at the age of 5 years.²⁶ The negative or curvilinear/cubic associations of weight (or body mass index) should be further explored. These associations with stunting and health were independent of the father’s education and income. The CIID could be combined with the HVLT and VF to explain more of the variance in models. Interestingly, the CCIDD and HVLT tests were individually identified as markers for later life dementia risk, even in individuals with learning disability,²⁷ and were shown to be modifiable by lifestyle interventions, such as exercise.²⁸ An earlier work suggested that poor cognitive function in childhood is associated with increased dementia risk in later life, but we found instable performance on traditional IQ tests over children’s development in Guatemala.²⁹ Further research is needed to establish risk and protective factors associated with better performance on the CIID in children.

In conclusion, this study showed a moderate correlation between CIID series, VF, and IHVLT test with the WISC. Given the box plot data, the CIID subtest could be used to estimate a child’s visual logical reasoning, although it requires further testing to develop age-appropriate norms. In addition, regression analyses suggested that the CCIID subtest could be used by itself to identify risk factors, like stunting associated with poor cognitive development and later life dementia risk. Its nonverbal element makes it possible to test children with hearing disability and different cultural backgrounds and languages, which is important in Indonesia with inhabitants of many different ethnicities. A large-scale study using the CIID test will be required in the future to assess its accuracy and usefulness in rural regions.

 

 

 

 

 

 

 

1. Sandjaja S, Poh BK, Rojroonwasinkul N, Le Nyugen BK, Budiman B, Ng LO, et al. Relationship between anthropometric indicators and cognitive performance in Southeast Asian school-aged children. Br J Nutr. 2013;110 Suppl 3:S57−64.
2. National Institute of Health Research and Development, Ministry of Health of the Republic of Indonesia. Basic Health Research (RISKESDAS) 2018. Jakarta: National Institute of Health Research and Development, Ministry of Health of the Republic of Indonesia; 2018. Indonesian.
3. Papalia DE, Martorell G. Experience human development. New York: McGraw-Hill Education; 2014. p. 325.
4. Elizabeth KE. Child development and related issues. In: Elizabeth KE, editor. Nutrition & child development. 5th ed. New Delhi: Paras Medical Publisher; 2015. p. 503–18.
5. Hogervorst E. The CIID: a cross cultural cognitive test to assess intellectual ability. Leicestershire: Loughborough University; 2011.
6. van Dijk A, Daďová K, Martínková I. Intellectual disability sport and paralympic classification. AUC Kinanthropologica. 2017;53(1):21–34.
7. Hogervorst E, Van der Wardt V. Cognitive test battery for Individual with and without ID (CIID). Leicestershire: Loughborough University; 2016.
8. Faculty of Psychology Universitas Indonesia. Manual WISC. Depok: Faculty of Psychology Universitas Indonesia; 1994.
9. Grenfell-Essam R, Hogervorst E, Rahardjo TB. The Hopkins Verbal Learning Test: an in-depth analysis of recall patterns. Memory. 2018;26(4):385–405.
10. Van der Elst W, Hurks P, Wassenberg R, Meijs C, Jolles J. Animal verbal fluency and design fluency in school-aged children: effects of age, sex, and mean level of parental education, and regression-based normative data. J Clin Exp Neuropsychol. 2011;33(9):1005–15.
11. Hogervorst E, Mursjid F, Priandini D, Setyawan H, Ismael RI, Bandelow S, et al. Borobudur revisited: soy consumption may be associated with better recall in younger, but not in older, rural Indonesian elderly. Brain Res. 2011;1379:206–12.
12. Hogervorst E, Sadjimim T, Yesufu A, Kreager P, Rahardjo TB. High tofu intake is associated with worse memory in elderly Indonesian men and women. Dement Geriatr Cogn Disord. 2008;26(1):50–7.
13. de Vaus D. Bivariate analysis for interval level variables, In: Survey in social research. 6th ed. New York: Routledge; 2014. p. 270–93.
14. Thomaidis L, Choleva A, Janikian M, Bertou G, Tsitsika A, Giannakopoulos G, et al. Attention deficit/hyperactivity disorder (ADHD) symptoms and cognitive skills of preschool children. Psychiatriki. 2017;28(1):28–36.
15. Marelli A, Miller SP, Marino BS, Jefferson AL, Newburger JW. Brain in congenital heart disease across the lifespan: the cumulative burden of injury. Circulation. 2016;133(20):1951–62.
16. Ryberg C, Sunnegårdh J, Thorson M, Broberg M. Intellectual functioning in children with congenital heart defects treated with surgery or by catheter interventions. Front Pediatr. 2016;4:113.
17. Edgin JO. Cognition in down syndrome: a developmental cognitive neuroscience perspective. Wiley Interdiscip Rev Cogn Sci. 2013;4(3):307–17.
18. Castro IPS, Viana MB. Cognitive profile of children with sickle cell anemia compared to healthy controls. J Pediatr (Rio J). 2019;95(4):451–7.
19. Chauhan U, Golhar S, Dahake P. Correlation between iron deficiency anemia and cognitive achievement in school aged children. Ann Int Med Res. 2016;2(4):178–80.
20. Irani F, Barbone JM, Beausoleil J, Gerald L. Is asthma associated with cognitive impairments? A meta-analytic review. J Clin Exp Neuropsychol. 2017;39(10):965–78.
21. Qurbani SN, Reniarti L, Chairulfatah A. Serum ferritin, serum nitric oxide, and cognitive function in pediatric thalassemia major. Paediatr Indones. 2017;57(3):148–52.
22. Lubis B, Saragih RAC, Gunadi D, Rosdiana N, Andriani E. Differences in the hematology response and cognitive development in iron deficiency anemia primary school age children who received iron therapy once and three times a day. Sari Pediatri. 2008;10(3):184–9. Indonesian.
23. Schneider WJ, McGrew KS. The Cattell-Horn-Carroll model of intelligence. In: Flanagan DP, Harrison PL, editors. Contemporary intellectual assesment: theories, tests, and issues. New York: The Guilford Press; 2012. p. 99–144.
24. McGrew KS. CHC theory and the human cognitive abilities project: standing on the shoulders of the giants of psychometric intelligence research. Intelligence. 2009;37(1):1–10.
25. Grey S, Green S, Alt M, Hogan TP, Kuo T, Brinkley S, et al. The structure of working memory in young children and its relation to intelligence. J Mem Lang. 2017;92:183–201.
26. Alam MA, Richard SA, Fahim SM, Mahfuz M, Nahar B, Das S, et al. Impact of early-onset persistent stunting on cognitive development at 5 years of age: results from a multi-country cohort study. PLoS One. 2020;15(1):e0227839.
27. Elliott-King J, Shaw S, Bandelow S, Hiremath A, Velayudhan L, Baillon S, et al. Dementia in individuals with intellectual disability; is there a better way to diagnose? Arc App Med. 2019;1(2):1–23.
28. Elliott-King J, Peel E, Hogervorst E. Acute cognitive effects of physical activity for people who have dementia. Int J Neurodegener Dis. 2019;2(1):009.
29. Mansukoski L, Hogervorst E, Fúrlan L, Galvez-Sobral JA, Brooke-Wavell K, Bogin B. Instability in longitudinal childhood IQ scores of Guatemalan high SES individuals born between 1941-1953. PLoS One. 2019;14(4):e0215828.

Copyright @ 2020 Authors. This is an open access article distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original author and source are properly cited. For commercial use of this work, please see our terms at http://mji.ui.ac.id/journal/index.php/mji/copyright.