(PDF) The Social Aptitudes Scale: an initial validation

Soc Psychiatry Psychiatr Epidemiol (2009) 44:508–513 DOI 10.1007/s00127-008-0456-4 ORIGINAL PAPER Elizabeth B. Liddle Æ Martin J. Batty Æ Robert Goodman The Social Aptitudes Scale: an initial validation Received: 10 July 2008 / Revised: 16 October 2008 / Published online: 1 November 2008 j Abstract Background Poor social skills are asso- ciated with a range of child and adolescent psychiatric Introduction disorders, with deficits being particularly marked in autistic spectrum disorders (ASDs). Here, we validate Impaired social ability is associated with child psy- a brief measure of social aptitudes where low scores chiatric disorders for several reasons. Qualitative are designed to index a substantially raised risk abnormalities in reciprocal social interactions are of ASDs. Method Parents of a national community defining characteristics of autistic spectrum disorders sample of 7,977 British 5–16 year olds completed (ASDs) [2, 17]. In addition, a broad range of child the Social Aptitudes Scale (SAS) as well as a general psychiatric disorders can potentially cause, and be questionnaire measure of psychopathology, the caused by, poor social skills. Thus behavioural and Strengths and Difficulties Questionnaire (SDQ). Psy- emotional disorders may lead to rejection or neglect chiatric diagnoses were assigned by clinical raters on by peers [12], reducing opportunities for the social the basis of detailed multi-informant information. learning of social skills. Conversely, peer problems Results All ten items of the SAS loaded onto a single can increase psychiatric vulnerability to adverse life latent factor, with a Cronbach’s alpha of 0.88. Corre- events or chronic adversities [11]. lations between the SAS and the SDQ were only Even brief assessments of children’s mental health modest, suggesting that the SAS measures different and wellbeing often include items on friendship and attributes to the SDQ. The SAS was significantly better popularity [e.g. 1, 10], whereas it is less common for than the SDQ at identifying ASDs. Conclusion Chil- social aptitudes to be assessed directly. This may re- dren and adolescents with low SAS scores are at flect the implicit assumption that success and failure increased risk of mental health problems, particularly in peer relationships are themselves adequate indices ASDs. of good and poor social skills. However, it is worth distinguishing between social inclusion (friendship, j Key words autistic spectrum disorder – diag- popularity, victimization) and social skill because nosis – methodology – screening they are conceptually distinct and may be dissociated in practice. Thus, some children with good social abilities have poor peer relationships, for example as the result of racial prejudice. Conversely, some chil- dren with poor social abilities have good peer rela- E.B. Liddle Æ M.J. Batty Developmental Psychiatry tionships, for instance if they attend schools that work University of Nottingham hard to foster the integration of children with dis- Nottingham, UK abilities. Ideally, a brief assessment measure of social R. Goodman aptitude would detect deficits in social ability at a Kings College London Institute of Psychiatry young enough age to provide remedial help before Dept. of Child and Adolescent Psychiatry these deficits lead to peer problems. Particularly low London, UK scores on a measure of social aptitude might also R. Goodman (&) provide useful pointers to unrecognized ASDs. P085 Institute of Psychiatry The Social Aptitude Scale (SAS) is a ten item scale London SE5 8AF, UK that forms part of the Development and Well Being SPPE 456 Tel.: +44-20/7848-0471 Fax: +44-20/7708-5800 Assessment [9]. The items were chosen to tap skills in E-Mail:

social understanding and behaviour that vary sub- 509 stantially in the general population and that are same age. The full measure in English and other languages can be usually markedly under-developed in individuals with viewed and downloaded from http://www.dawba.com/SAS. All ten items were completed on 7,504 individuals (94% of the sample); ASDs. The items focus on complex interactive skills only these individuals were included in the principal component rather than on relatively easily coached skills (such analysis. Eight or more items were completed on 7,768 individuals as remembering to say ‘‘please’’ and ‘‘thank you’’ or (97% of sample); total scores for the remaining analyses were introducing oneself by name). This paper is a pre- computed for all these individuals (prorating the score when only 8 or 9 items had been completed). liminary investigation of the psychometric properties Parents, teachers and 11–16 year olds also completed the of the SAS, based on a national community-drawn Strengths and Difficulties Questionnaire (SDQ) [10]. The SDQ sample of children aged 5–16 years. consists of 25 items relating to five symptom scales covering the following areas: emotions; behaviour; attention and activity level; peer relationships; and prosocial behaviour. The first four scales are added together to generate a total difficulties score. The SDQ’s Method impact supplement asks respondents whether they consider the child to have a significant mental health problem, and if so, enquires further about resultant distress and social impairment—permitting j Sample the calculation of an impact score. The Development and Well-Being Assessment (DAWBA) was The current study was carried out as part of a larger survey of a used to assess psychiatric disorder [9]. This structured interview nationally representative sample of British 5–16 year olds, which is was administered by lay interviewers to the parents of 5–16 year fully described elsewhere [12]. In Great Britain ‘‘child benefit’’ is a olds, and also to the 11–16 year olds themselves, while the inter- universal state benefit payable for each child in the family, and it viewers also recorded detailed verbatim descriptions of any prob- has an extremely high uptake. For this reason, the British Child and lem areas. An abbreviated version was mailed to the teacher. A Adolescent Mental Health Survey 2004 used the child benefit reg- small team of experienced clinicians used the information provided ister to develop a sampling frame of postal sectors from England, by all the informants, combining information as they would in the Wales and Scotland. After excluding families with no recorded clinic, to make diagnoses according to ICD–10 and DSM-IV criteria postcode or subject to current revision of their record, the sampling [2, 17]. The prevalence of the main diagnostic groupings was as frame was estimated to represent 90% of all British children [14]. follows: behavioural disorders 5.8%, emotional disorders 3.7%, This sampling frame was used to select a stratified random sample hyperkinesis 1.5%, ASD 0.9% [12]. of 12,294 families, of whom 9% opted out prior to their details Previously published data on children from this sample who being passed to the investigators, 5% could not be traced and 1% had been assigned ASD diagnoses [12] support the validity of these were ineligible [12]. Thus, 10,496 families were approached, of diagnoses. The prevalence of 0.9% is plausible: though higher than whom, 7,977 responded—representing 65% of those originally se- the commonly quoted range of 0.3–0.6% [3, 8, 15], it is lower than lected and 76% of those who were approached. All parents and 11– the prevalence of 1.2% reported by another recent British survey 16 year olds were invited to take part in face-to-face interviews with wide coverage and careful assessment [4]. The mean level of carried out in the family home. When the family consented, a parent-reported impact on the SDQ was higher for children with teacher nominated by the family was also mailed a questionnaire; ASDs than for those with hyperactivity, behavioural or emotional teacher data were available for 74% of the sample. disorders—demonstrating that the high prevalence was not the result of diluting true ASDs with large numbers of children with clinically insignificant symptoms. In addition, there was evidence j Measures of divergent validity, with ASDs having very different associations from other disorders, e.g. substantially higher rates of epilepsy and The ten item Social Aptitudes Scale (SAS) was completed by parents learning disability, but no evidence for the socioeconomic disad- about their children. The scale includes items such as ‘‘Able to vantage associated with other disorders (as judged by parental compromise and be flexible’’, ‘‘Easy to chat with, even if it isn’t on a education, housing tenure and area of residence). topic that specially interests him/her’’ and ‘‘By reading between the The ASD section of the DAWBA is governed by a ‘‘skip rule’’ so lines of what people say, s/he can work out what they are really that although all parents are asked some initial questions, only a thinking and feeling’’ (see Table 1). For each item, parents rated minority of parents is asked the more detailed questions. The full their child as ‘‘a lot worse than average’’, ‘‘a bit worse than aver- set of questions was only administered to 744 parents (9.3%) who age’’, ‘‘about average’’, ‘‘a bit better than average’’ or ‘‘a lot better met one or more of three criteria: the presence of potentially than average’’—the reference group being other children of the autistic features in the first 3 years that did not resolve completely; an SDQ profile with high peer problems and low pro-social behaviours; or a SAS score of 12 or less. Had the ‘‘skip rule’’ been Table 1 Factor loadings and item-score correlations for the 10 SAS items modified to exclude the SAS criterion, only one of the 67 children in the sample who was diagnosed with an ASD would have been Item (abbreviated) Factor Item-score missed. To avoid circularity, this one child was omitted from the loading correlation analyses when assessing how well the SAS screened for ASDs. (rho) There were no psychometric tests of children’s cognitive abili- ties or academic attainments. Parents and teachers were asked to 1. Responds appropriately to light-hearted teasing 0.64 0.60 estimate each child’s mental age, and teachers reported whether a 2. Easy to chat with 0.68 0.66 child had a written statement of special educational needs related to 3. Able to compromise and be flexible 0.75 0.70 cognitive and learning needs (including specific, moderate, severe 4. Can defuse a tense or embarrassing situation 0.75 0.70 and profound learning difficulties, but not distinguishing between 5. A good loser 0.62 0.61 them). For the purpose of these analyses, a child was considered to 6. Others feel at ease around him/her 0.76 0.75 have a generalized learning disability when one or both informants 7. Can work out what people are really thinking 0.68 0.65 estimated that mental age was 60% or less of the chronological and feeling age (e.g. a mental age of 6 or less at a chronological age of 10). 8. Can apologize and resolve matters to avoid 0.65 0.64 Such children were substantially more likely to have a written hard feelings statement related to cognitive and learning needs: 41% vs. 4%, 9. Can take the lead without seeming bossy 0.67 0.63 Odds ratio = 16.9 (95% confidence interval 11.6–24.3). 10. Aware of what is and is not socially appropriate 0.74 0.71 Analyses were conducted in SPSS version 13.0 (SPSS Inc., Chicago). 510 age, with the curves for different ages being very Results similar for the low scores that are particularly relevant to screening for clinical conditions. For ease of j Unidimensionality and internal validity of the SAS interpretation, therefore, we present the remaining analyses in terms of raw scores without age adjust- In order to quantify the degree to which the items ment. Almost identical results (not presented here) measured a single latent variable, principal compo- were obtained repeating the analyses using age-nor- nents analysis (PCA) was used to extract factors from malized T scores (mean of 50, standard deviation of the subset of questionnaires in which all ten items had 10) obtained by ranking the present data after re- been completed (N = 7,504). Only one factor had an weighting to be representative of the UK population Eigenvalue over 1: this factor had an Eigenvalue of 4.8 (conversion table displayed on http://www.daw- and accounted for 48% of variance; potential second ba.com/SAS). and third factors had Eigenvalues of 0.8 and 0.7 and were not included. As shown in Table 1, all 10 items had factor loadings over 0.6 on the single factor. j Overlap of SAS and SDQ Cronbach’s alpha was 0.88. All individual items cor- related strongly with total score: the lowest value of Table 2 presents the correlation of SAS score with 21 Spearman’s rho for any individual item was 0.60 different SDQ scores, reflecting three classes of (P < 0.001), and the highest value was 0.75 (P < 0.001). informants for each of seven possible SDQ scores. The The mean score was 24.6, with a standard deviation of SAS-SDQ correlations were higher for the parent- 6.3. The score was roughly a quarter of a standard based SDQ scores than for the corresponding teacher- deviation higher in females (mean 25.3, SD 6.1) than based and youth-based SDQ scores—not surprising males (mean 23.8, SD 6.4); t = 10.7, 7766df, P < 0.001. given that the SAS was completed by parents (‘‘shared rater effect’’). Equally unsurprising was the finding that the SAS score (higher scores are better) corre- j Score distribution by age lated positively with the SDQ pro-social score (higher scores are better) and negatively with all other SDQ The distribution of total SAS scores is shown in Fig. 1 scores (lower scores are better). It is noteworthy that for three age bands: 5–8 years old, 9–12 years old and none of the correlations was very high. 13–16 years old. From the plot, it can be observed that the scores for each age band are slightly negatively skewed. The modal SAS score for each age band is j Discriminating between different disorders around 20, representing a mean item score of 2, ‘‘about average’’, but parents are more likely to rate Figure 2 includes receiver operator characteristic their child as being ‘‘a bit better than average’’ (ROC) curves showing how well the SAS score and the or ‘‘a lot better than average’’ than ‘‘a bit worse SDQ total difficulties score detect each condition. The than average’’ or ‘‘a lot worse than average’’. This better the discrimination, the closer the curve ap- positive construal was slightly more pronounced with proaches the top left-hand corner, and the closer the increasing age (even though parents were asked to area under the curve (AUC) approaches unity. As a compare their children with others of the same age). guide to interpretation, an AUC of 0.5 reflects a Nevertheless, the distribution does not vary greatly by measure that is no better than chance at discrimi- nating between children with and without the relevant diagnosis, while an AUC of 1 would reflect perfect 10 discrimination. Allowing for the fact that both the 9 Aged 5-8 SAS and SDQ scores were obtained on the same Aged 9-12 sample [13], the SAS score was significantly better 8 Aged 13-16 than the SDQ total score at discriminating between 7 6 Table 2 Spearman correlations between the SAS score (parent-reported) and % 5 SDQ scores based on parent, teacher and youth report 4 SAS Rater SDQ Domain 3 Emotion Conduct Hyper- Peer Pro- Total Impact 2 active social 1 Parent N = 7758 )0.25 )0.39 )0.38 )0.23 0.42 )0.44 )0.32 Teacher N = 5799 )0.15 )0.20 )0.25 )0.18 0.22 )0.27 )0.24 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 Youth N = 1333 )0.06* )0.32 )0.17 )0.12 0.15 )0.26 )0.16 SAS score *Correlation significant at P < 0.05. All other correlations are significant at Fig. 1 Distribution of SAS score by age band P < 0.001 511 Fig. 2 ROC curves showing the Emotional Disorder Hyperkinesis discriminatory power of the SAS and 100% 100% SDQ total difficulties score (parent rating) for four diagnostic groupings 75% 75% sensitivity sensitivity 50% 50% 25% 25% SAS SAS SDQ SDQ 0% 0% 0% 25% 50% 75% 100% 0% 25% 50% 75% 100% 1-specificity 1-specificity Behavioural Disorder ASD 100% 100% 75% 75% sensitivity sensitivity 50% 50% 25% 25% SAS SAS SDQ SDQ 0% 0% 0% 25% 50% 75% 100% 0% 25% 50% 75% 100% 1-specificity 1-specificity children with and without ASDs (AUC = 0.98 for 80 SAS; AUC = 0.94 for SDQ; z = 3.01, P < 0.01 for the 70 difference). Conversely, the SDQ was a more dis- criminating measure than the SAS for the other three 60 disorders: hyperkinesis (AUC = 0.89 for SAS; AUC 50 % with ASD = 0.94 for SDQ; z = 3.019, P < 0.01 for the differ- 40 ence), behavioural disorders (AUC = 0.79 for SAS; 30 AUC = 0.88 for SDQ; z = 7.291, P < 0.001 for the difference) and emotional disorders (AUC = 0.65 for 20 SAS; AUC = 0.84 for SDQ; z = 10.967, P < 0.001 for 10 the difference). 0 Figure 3 shows that the likelihood of an ASD is 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 non-linearly related to the SAS score, with the prev- SAS score alence of an ASD varying between zero and 0.6% for SAS scores over 16, but rising rapidly as the SAS score Fig. 3 Probability of receiving an ASD diagnosis (vertical axis) according to SAS score (horizontal axis) falls further. Of the 61 individuals with ASDs and a valid SAS score, 57 (93%) had SAS scores of 16 or less, 46 (75%) had SAS scores of 11 or less, and 32 (53%) sis, behavioural disorders and emotional disorders as had SAS scores of 5 or less. No single cut-point is the independent variables. Allowing for the overlap ideal for all circumstances, since the choice of a cut- between these conditions, the independent reduction off depends on the population (e.g. community versus in the SAS score (and the 95% confidence intervals) clinic) and the relative importance assigned to false associated with each condition was as follows: a positive and false negatives. In this epidemiological reduction of 2.1 points (95%CI 1.4–2.8) for emotional sample, the screening properties for three potentially disorders, 4.1 points (3.1–5.1) for generalized learning useful cut-offs are as follows: with a cut-off of 16 or disability, 5.9 points (5.3–6.6) for behavioural disor- less: sensitivity = 0.936, specificity = 0.934, positive ders, 6.9 points (5.8–8.1) for hyperkinesis and 16.9 predictive value (PPV) = 0.104 and negative predic- points (15.4–18.4) for ASD. tive value (NPV) = 0.999; with a cut-off of 11 or less: sensitivity = 0.754, specificity = 0.981, PPV = 0.235 and NPV = 0.998; and with a cut-off of 5 or less: Discussion sensitivity = 0.475, specificity = 0.996, PPV = 0.525 and NPV = 0.996. The SAS was designed to tap the sorts of social apti- Forced-entry linear regression analyses were car- tudes that require a good ability to read social and ried out with the SAS score as the dependent variable emotional cues rapidly in complex situations in order and generalized learning disability, ASD, hyperkine- to guide socially skilled behaviour. In a large sample 512 of 5–16 year olds drawn from the community, the ten When it comes to detecting a relatively rare dis- constituent items all loaded onto a single factor, order in the general population, even measures with demonstrating the scale’s internal coherence. The SAS apparently impressive sensitivities and specificities score was roughly normally distributed, with no floor have unimpressive positive predictive values. Only effect, but with a small ceiling effect, particularly for 10% of those individuals in our sample who scored 16 older adolescents. or less on the SAS had an ASD (the cut-off that de- We anticipated that low SAS scores would predict tected 94% of those with an ASD). Lower cut-offs can ASDs, and that less marked deficits would be associ- improve positive predictive value, but at the expense ated with other psychiatric disorders or generalized of poorer sensitivity. For example, with a cut-off of 5, learning disabilities. These effects were empirically the positive predictive value rose to 52%, but this only supported. ASDs were associated with very low SAS detected 48% of those with an ASD. Community scores—typically more than two standard deviations screening programs will generally settle for fewer false below the mean. By comparison, the reduction in SAS negatives (missed cases of ASD) at the expense of score was around one standard deviation for hyper- considerably more false positives (‘‘false alarms’’), kinesis, and less than this for other psychiatric dis- provided these false alarms can be resolved rapidly by orders and generalized learning disabilities. These more detailed assessment without generating a lot of results demonstrate that the SAS score is not simply a parental anxiety. non-specific indicator of psychopathology or learning Indeed, the very notion of a false alarm may be disability. misleading since it assumes that the only purpose of There was clear evidence of divergent validity, with screening with the SAS is to detect ASDs. An alter- the SAS score being a more discriminating guide to native perspective is that the SAS provides ‘‘wide- the presence or absence of ASDs, and with the SDQ angle’’ screening that aims to detect ASDs in the total score being a more discriminating guide to the course of identifying a broader group of children with presence or absence of other disorders. Many of the low social aptitudes. With this broader group in mind, correlations between SAS and SDQ scores were professionals reviewing children with low scores on moderate to low, providing further evidence that the the SAS should consider a wide range of explanations: two measures were not tapping identical constructs. It an ASD is one possible explanation, but there are was striking, for example, that the parent-completed many others, including the consequences of other SAS score only correlated around )0.2 with the peer psychiatric disorders, a learning disability, or inade- problems score derived from the SDQ completed by quate opportunities to learn social skills. It is not a the same parent at the same assessment. This sup- false alarm if the professional rules out an ASD but ports the conceptual distinction between social apti- confirms a deficit in social skills, identifies a likely tude and peer relationships discussed in the cause, and advises the family or school on suitable introduction—good social aptitudes do not guarantee skill-building approaches. From this perspective, the good peer relationships, and neither do poor social SAS could appropriately be used as an initial aptitudes preclude good peer relationships. screening measure for general community samples, The SAS could potentially be used to screen for being followed up in some instances either by more ASDs, whether in the community or specialist clinics. definitive interviews or by a second screening phase Its screening properties are good: the area under the using autism-specific questionnaires. ROC curve was 0.98 (where a perfect screen would The study has significant limitations. Though the have a value of 1); and at a cut-off of 16 or less, sample was drawn from the community, only two- sensitivity was 0.94 and specificity was 0.93. While thirds of those originally selected took part, which these results seem to compare favourably with those may have reduced representativeness. The SAS was reported for other questionnaire measures covering only completed by parents—time constraints ruled social communication competence or autistic symp- out administering it to teachers as well, though their toms [5, 7, 16], it is potentially misleading to compare ratings would have been of great interest, and might screening properties based on different sorts of potentially form a better basis for universal screening. samples, and judged against different ‘‘gold standard’’ There was no assessment of the test-retest reliability measures. Direct head-to-head comparisons of dif- or inter-rater reliability of the SAS. The setting (a ferent screening measures are more instructive [6], large nationwide survey carried out by lay inter- and we would welcome the inclusion of the SAS in viewers) precluded validating the SAS by looking for any future head-to-head comparisons. It will also be correlations with direct observations of social important to compare the acceptability of the SAS behaviour, or with objective tests of Theory of Mind and other possible screening measures. We predict and the recognition of social and emotional cues. that the brevity of the SAS and its focus on compe- Similarly, the diagnosis of ASDs relied on informant tences will make it particularly acceptable for general reports but not direct observation. In the absence of population screening; other screening questionnaires IQ measures, the presence of generalized learning are longer and generally focus on symptoms and disabilities had to be judged from parent and teacher deficits. estimates of mental age. Finally, as already noted, the 513 SAS was not compared to other possible screening 6. Charman T, Baird G, Simonoff E, Loucas T, Chandler S, Mel- measures. Redressing the limitations in this initial drum D, Pickles A (2007) Efficacy of three screening instru- ments in the identification of autistic-spectrum disorders. Br J validation are important goals for the future. Psychiatry 191:554–559 In conclusion, the initial findings on the SAS sug- 7. Constantino JN, Gruber CP (2005) Social responsiveness scale gest that this short measure of social ability is (SRS). Western Psychological Services, Los Angeles promising—it is certainly more than a non-specific 8. Fombonne E (2003) The prevalence of autism. JAMA 289:87–89 9. Goodman R, Ford T, Richards H, Gatward R, Meltzer H (2000) measure of general psychopathology or learning The Development and Well-Being Assessment: description and disability. Children with low SAS scores are at an initial validation of an integrated assessment of child and adoles- increased risk of mental health problems, with ASDs cent psychopathology. J Child Psychol Psychiatry 41:645–655 becoming particularly likely at very low scores. 10. Goodman R (2001) Psychometric properties of the Strengths and Difficulties Questionnaire (SDQ). J Am Acad Child Adolesc Psychiatry 40:1337–1345 11. Goodyer I, Wright C, Altman P (1990) The friendships and recent life events of anxious and depressed school-age children. References Br J Psychiatry 156:689–698 12. Green H, McGinnity A, Meltzer H, Ford T, Goodman R (2005) 1. Achenbach TM (1991) Manual for the child behavior checklist/ Mental health of children and young people in Great Britain, 4 18 and 1991 Profile. University of Vermont Department of 2004. Palgrave Macmillan, Hampshire Psychiatry, Burlington 13. Hanley JA, McNeil BJ (1983) A method of comparing the areas 2. American Psychiatric Association (1994) Diagnostic and Sta- under receiver operating characteristic curves derived from the tistical Manual of Mental Disorders, 4th edn. (DSM-IV). same cases. Radiology 148:839–843 American Psychiatric Association, Washington DC 14. Meltzer H, Gatward R, Goodman R, Ford T (2000) Mental 3. Baird G, Charman T, Baron-Cohen S (2000) A screening health of children and adolescents in Great Britain. The Sta- instrument for autism at 18 months of age: a 6-year follow up tionery Office, London study. J Am Acad Child Adolesc Psychiatry 39:694–702 15. Rutter M (2005) Incidence of autism spectrum disorders: 4. Baird G, Simonoff E, Pickles A, Chandler S, Loucas T, Meldrum changes over time and their meaning. Acta Paediatr 94:2–15 D, Charman T (2006) Prevalence of disorders of the autism 16. Skuse D, Mandy WPL, Miller L, Goodman R, Lawrence K, spectrum in a population cohort of children in South Thames: Emond A, Golding J (2008). Social communication competence the special needs and autism project (SNAP). Lancet 15:210– and functional adaptation in a general population of children. 215 J Am Acad Child Adolesc Psychiatry (in press) 5. Berument SK, Rutter M, Lord C, Pickles A, Bailey A (1999) 17. WHO (1993) International statistical classification of diseases Autism screening questionnaire: diagnostic validity. Br J Psy- and related health problems: diagnostic criteria for research. chiatry 175:444–451 World Health Organization, Geneva