CELF-4: Potential for Bias Against Speakers of African American English
Janice Ellsworth and Akiko Fuse
Department of Biobehavioral Sciences, Speech-Language Pathology Program
Teachers College, Columbia University
Abstract
The CELF-4 is skewed to over-identify African America English speaking
(AAE) children as language disordered. It is widely used to establish
eligibility for language services, and placement in the special education
system. Bias is most apparent in assessment of syntactic, semantic and
morphological skills, all possible reflections of language environment,
rather than language disorder. Disproportionately low SES and parental
education levels among African American children are also predictive of
lower test scores, while not necessarily indicating language impairment.
Problem with an incorrect identification as language disordered
Problems with the Formulated Sentences subtest of the CELF-4
• Over-identification of language disorder in AAE-speaking children places a disproportional
number of these children on the special education track. Resulting problems include:
■ Perceived stigma
■ Reduced expectations
■ A self-fulfilling prophecy in which lowered expectation are associated
with lower performance levels (Jones & Healey, 1974).
• This subtest presents significant potential for error based on dialectal differences.
• “On average, AA children tend to score 1 SD below EUA (European American) peers on
standardized language measures” (Brooks-Gunn, Klebanov, & Duncan, 1996; Hammer et al., 2002; Kaufman & Kaufman, 1983;
Mercer, 1979; Reynolds, Lowe, & Saenz, 1999; Whitehurst, 1997, as cited by Restrepo, et al, 2006).
• Substantial evidence indicates that this difference is the result of environment (Jencks & Phillips, 1998).
• Procedure and scoring: The examinee is required to use a word provided by the
examiner to construct a sentence based on a picture in the Stimulus Book. The
examiner must rate the sentence from 0-2, based on grammatical correctness.
• The manual provides scoring examples for each sentence, and although a very
few items are identified as possible dialectal variations, the vast majority of
clearly AAE usages are not identified. In fact, many of the items indicated as
rating a score of only 1 (i.e., only half credit) represent common forms in AAE
dialectal usage.
Examples from just the first item for 5-6 year olds:
CELF-4 tests speakers of African American (AAE) dialect
on skills in Standard American English (SAE)
ƒ African American English (AAE) is a dialect of American English.
ƒ AAE is a regular and systematic language variation with unique structures and
patterns of grammar, pronunciation, and vocabulary.
ƒ AAE rules of syntax, morphology and phonology differ from those of SAE.
ƒ AAE is widely spoken across the US, with a wide range of dialectal variation.
Problems with use of standardized tests
in diverse populations
• Problems with standardized and norm-referenced tests have been identified
based on cultural and linguistic diversity. In addition to linguistic and dialectal
mismatches, these include content bias, and disproportionate representation in
normative samples (Laing & Kamhi, 2003).
• Test stimuli and procedures are based on white mainstream school practices
and standards. They assume exposure to similar concepts, vocabulary and
literacy experiences across cultural groups.
• Cultural lack of familiarity with the question-answer format or with the model of
providing labels for objects and rehearsing facts in the parent-guided interactive
manner of the mainstream may also skew results.
• Population samples used to establish norms for standardized tests may reflect
the current national balance of cultural and linguistic population groups, however
this may not in fact serve any of these groups. As long as a single cut-off score
on a single test is applied to all groups, no group can be accurately assessed.
Alternate approaches to assessment
of culturally & linguistically diverse populations
Use informal assessment methods
In-depth ethnographic interview
Dynamic assessment (test-teach-retest)
▪ Naturalistic language sample analysis
▪ Narrative assessment
▪ Portfolio review of child’s classroom work
Modify the CELF-4 to meet individual needs
Modify instructions
Provide additional examples and practice trials
Repeat stimuli as appropriate
Modify test administration
Exclude biased items
Test beyond ceiling
Allow extra time. Administer across several sessions.
■ Modify interpretation of responses based on cultural / linguistic factors
Vocabulary: Misleading indicator of language disorder on CELF-4
There is children in the living room (use of is/was for are/were).
The children is playing Nintendo all day (use of is/was for are/were).
The children is playing one of their favorite games (use of is/was for are/were).
The children playing the video game (omission of auxiliary is/are).
The children is playing (use of is/was for are/were).
They playin’ a game (omission of auxiliary is/are).
My baby-sitter children come to my house (omission of plural and of past tense).
• All are examples of common AAE dialectal usage, but were not indicated as such in
the manual. Only the sentence The children playing the video game (omission of
auxiliary is/are) was marked as a possible dialectal form.
• It must be noted that the manual advises the examiner to use clinical judgment in
scoring responses, and provides a list of dialectal variations in Appendix B. However
the failure to include some of the most common AAE dialectal forms in a section which
lists examples of dialectal variations, is problematic and is certain to result in incorrect
identification of language errors in assessments of African American children.
AAE Feature
SAE
AAE
Omission of auxiliary is/are
The boy is eating.
The children are playing.
The boy eating.
The children playing.
Omission of copula be
That dog is old.
That dog is a poodle.
That dog old.
That dog a poodle.
Substitution of is/was for are/were
They are playing baseball.
We were reading.
They is playing baseball.
We was reading.
Remote been (indicating distant past) He has been married a long time.
He been married.
Main verb be
He is on the phone right now.
He be on the phone right now.
Habitual be (actions over time)
Usually he is at school.
Usually he is at school.
Omission of noun plural
Dogs, chairs
Dog, chair
• Laing and Kamhi (2003) suggest that bias might be eliminated by developing
assessment instruments for each group. This however does not take into
account the diverse nature of diversity – in reality, cultural and linguistically
diverse groups are not clearly definable – they are infinitely diverse.
• Vocabulary is strongly influenced by language environment, and cannot reliably indicate
language disorder in children from non-mainstream language environments (Peña, Iglesias, & Lidz, 2001).
• Cultural difference has also been found to significantly impact children’s performance on tests
of vocabulary (Peña, Iglesias, & Lidz, 2001; Peña & Quinn, 1997).
Omission of auxiliary have
He has been there all day.
He been there all day.
Substitution of got for have
He has three dogs.
He got three dogs.
• Broad potential for bias has been found with respect to both socioeconomic
level and cultural background. African American children disproportionately
reflect low socioeconomic status and fewer years of parental education. Both of
these factors have been shown to be reflected in lower standardized test scores,
but do not necessarily indicate language impairment (Qi, Kaiser, Milan, & Hancock, 2006).
• Both the Receptive and Expressive Word Classes subtests of the CELF-4 depend in part upon
vocabulary size and content.
Omission of possessive “s”
I borrowed that boy’s book.
I borrowed that boy book.
Omission of 3rd person present “s”
She likes school.
She like school.
• Like the Formulated Sentences subtest, scores on these subtests also contribute to the Core
Language Score used to identify language disorder (see upper right box).
Omission of past tense.
She washed her hair.
She wash her hair.
Multiple negation
He doesn’t have any more.
He don’t have no more.
Examples: (From CELF-4 Receptive and Expressive Word Classes subtests)
References
Ballantyne, A. O., Spilkin, A. M., &Trauner, D. A. (2007). The revision decision: Is change always good? A comparison of CELF-R and CELF-3 test scores in children with language impairment, focal brain
damage and typical development. Language, Speech, and Hearing Services in Schools, 38, 182-189.
Charter, R. A. (2003). A breakdown of reliability coefficients by test type and reliability method, and the clinical implications of low reliability. The Journal of General Psychology,130, 290–304.
Genesee, F., Paradis, J., & Crago, M. B. (2004). Dual Language Development and Disorders: A Handbook on Bilingualism and Second Language Learning. Baltimore, MD: Paul H. Brooks.
Huaqing Qi, C., Kaiser, A. P., Milan, S., & Hancock, T. (2006). Language performance of low-income African American and European American preschool children on the PPVT–III. Language, Speech, and
Hearing Services in Schools, 37, 5-16.
Laing, S. P., & Kamhi, A. (2003). Alternative assessment of language and literacy in culturally and linguistically diverse populations. Language, Speech, and Hearing Services in Schools, 34, 44–55.
McFadden, T. U. (1996). Creating language impairments in typically achieving children: The pitfalls of “normal” normative sampling. Language, Speech, and Hearing Services in Schools, 27, 3-9.
McFadden, T. U., & Gilliam, R. (1996). An examination of the quality of narratives produced by children with language disorders. Language, Speech, and Hearing Services in Schools, 27, 48-56.
Paradis, J. (2005). Grammatical morphology in children learning English as a second language: Implications of similarities with specific language impairment. Language, Speech, and Hearing Services
in Schools, 36, 72-187.
Paul, R. (2006). Language disorders from infancy through adolescence: Assessment and intervention (3rd ed.). St. Louis: Mosby.
Peña, E. & Quinn, R. (1997). Task familiarity: Effects on the test performance of Puerto Rican and African American children. Language, Speech, and Hearing Services in Schools, 28, 323-332.
Peña, E. D., & Spaulding, T. J., Plante, E. (2006). The composition of normative groups and diagnostic decision making: Shooting ourselves in the foot. American Journal of Speech-Language
Pathology, 15, 247-254.
Peña, E.D., Iglesias, A., & Lidz, C. S. (2001). Reducing Test Bias Through Dynamic Assessment of Children's Word Learning Ability. American Journal of Speech Language Pathology, 10, 138-154.
Perona, K., Plante, E., & Vance, R. (2005). Diagnostic Accuracy of the Structured Photographic Expressive Language Test: Third Edition (SPELT-3). Language, Speech, & Hearing Services in Schools, 36,
103-115.
Plante, E. & Vance R. (1994). Selection of Preschool Language Tests: A Data-Based Approach, Language, Speech, and Hearing Services in Schools, 25, 15-24.
Restrepo, M. A., Schwanenflugel, P. J., Blake, J., Neuharth-Pritchett, S., Cramer, S. E., & Ruston, H. P. (2006). Performance on the PPVT–III and the EVT: Applicability of the measures with African
American and European American Preschool Children. Language, Speech, and Hearing Services in Schools, 37, 17 - 27.
Samel, E., Wiig, E., & Secord, W. (2003). Clinical Evaluation of Language Fundamentals (Fourth Edition) (CELF-4). TX, U. S.: The Psychological Corporation.
Spaulding, T. J., Plante, E., & Farinella, K.A. (2006). Eligibility criteria for language impairment: is the low end of normal always appropriate? Language, Speech, and Hearing Services in Schools, 37, 6172.
• In the ages 5-7 section, words such as lobster, seahorse, clothesline and garage
might not be familiar to the urban AAE child.
• In sections for ages 8-21, some of the words tested later in the test (on the receptive side)
clearly require mastery of higher level vocabulary. Examples include figurative, affirming,
assimilate, and zealous.
• And on the expressive side, it is equally unlikely that an examinee will be able to correctly
describe relations between those unfamiliar words, further impacting his language score.
Many thanks to Professor Cate Crowley, for her inspiration and for all the great things that she does.
Omission of interrogative do/does/did Where did you go?
Where you go?
Substitution of do for does
He does homework first.
He do homework first.
Done to indicate action completed
He walked the dog.
He done walked the dog.
Substitution of seen for saw
I saw the train coming.
I seen the train coming.
Use of ain’t for isn’t/aren’t / am not
The book isn’t here.
The book ain’t here.
Restatement of subject with pronoun My dog chewed up my sneaker.
My dog he chewed up my sneaker.
Use of additional auxiliary
He might have gone.
He might could have gone.
Substitution of them for those
Those bags are empty.
Them bags is empty.
Appending at with where
Where’s your bike?
Where your bike at?
Use of regularized reflexive
He fixed the car himself.
He fixed the car hisself.