Case: 14-70001
Document: 00512697728
Page: 1
Date Filed: 07/14/2014
No. 14-70001
____________________________________
IN THE UNITED STATES COURT OF APPEALS
FOR THE FIFTH CIRCUIT
____________________________________
JAMES LEE HENDERSON,
v.
Petitioner-Appellant,
WILLIAM STEPHENS,
Director, Texas Department of Criminal Justice
Correctional Institutions Division
Respondent-Appellee,
____________________________________
On Appeal from the United States District Court
for the Eastern District of Texas, Beaumont Division
Civ. Action No. 1:06-cv-00507-RHC
____________________________________
BRIEF OF AMICUS CURIAE
THE TEXAS PSYCHOLOGICAL ASSOCIATION
IN SUPPORT OF PETITIONER-APPELLANT
Mark K. Glasser
Texas Bar No. 8014599
David W. Denton Jr.
New York Bar No. 5169602
Sidley Austin LLP
1000 Louisiana St., Suite 6000
Houston, TX 77007
713.495.4623
[email protected]
[email protected]
Counsel of Record
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 2
Date Filed: 07/14/2014
RULE 26.1 DISCLOSURE STATEMENT
Pursuant to Rule 26.1 of the Federal Rules of Appellate Procedure the Texas
Psychological Association respectfully submits this Corporate Disclosure Statement
and states as follows:
1. The Texas Psychological Association is a nonprofit professional association
representing both licensed psychological practitioners and graduate students in
psychology in the state of Texas. The Texas Psychological Association is the state
affiliate of the American Psychological Association. The Texas Psychological
Association has no parent corporation, and no publicly held company has 10% or
greater ownership in the Texas Psychological Association.
i
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 3
Date Filed: 07/14/2014
STATEMENT REGARDING ORAL ARGUMENT
Amicus requests permission to participate in oral argument. Oral argument
would assist the Court, and the scientific expertise of amicus will help the Court
resolve questions pertaining to the psychometric testing at issue in this case.
Accordingly, amicus requests ten minutes of oral argument time to address the
scientific standards discussed in this brief in addition to the argument time granted to
the parties. Undersigned counsel is representing amicus pro bono in this matter.
ii
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 4
Date Filed: 07/14/2014
TABLE OF CONTENTS
Page
INTEREST OF AMICUS CURIAE ................................................................................... 1
INTRODUCTION................................................................................................................ 1
I.
A SHORT-FORM TESTING PROTOCOL IS
PROFESSIONALLY INADEQUATE TO MAKE A
DIAGNOSTIC JUDGMENT REGARDING
INTELLECTUAL DISABILITY................................................................. 5
II.
FAILURE TO MAINTAIN ADEQUATE RECORDS
DURING PSYCHOMETRIC TESTING OF
INTELLIGENCE SIGNIFICANTLY UNDERMINES THE
SCIENTIFIC METHOD. ........................................................................... 12
III.
PROPER APPLICATION OF THE CONCEPTS OF
VALIDITY AND RELIABILITY IS NECESSARY TO
EXTRAPOLATE AN ACCURATE FULL SCALE IQ. ....................... 15
CONCLUSION ................................................................................................................... 18
iii
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 5
Date Filed: 07/14/2014
TABLE OF AUTHORITIES
Page(s)
CASES
Atkins v. Virginia
536 U.S. 304, 317 (2002) ............................................................................................. passim
Blue v. Thaler,
665 F.3d 647 (5th Cir. 2011) ..................................................................................10, 11, 12
Ex parte Henderson,
No. 181-CR-12-93 (102nd Dist. Ct., Red River County, Tex. Oct. 5, 2005)
(testimony of Steve Gilliland) ........................................................................................6, 12
Ex parte Henderson,
No. WR-37658-03, 2006 WL 167836 (Tex. Crim. App. Jan. 25, 2006) ...............16, 17
Hall v. Florida,
134 S. Ct. 1986 (2014) ................................................................................................. passim
Rivera v. Dretke,
Civ. B-03-139, 2006 WL 870927 (S.D. Tex. Mar. 31, 2006), aff’d in relevant
part sub nom. Rivera v. Quarterman, 505 F.3d 349 (5th Cir. 2007) .............................11, 12
STATUTES
124 Stat. 2643 (2010) ...................................................................................................................2
Tex. Admin. Code § 465.22(a) .....................................................................................14, 15, 16
OTHER AUTHORITIES
Am. Psychiatric Ass’n, Diagnostic and Statistical Manual of Mental Disorders
(5th ed. 2013) ......................................................................................................................2, 3
Am. Psychological Ass’n, Ethical Principles of Psychologists and Code of Conduct
(2010).......................................................................................................................... 12, 14
Anthony P. Thompson et. al, Brief Intelligence Testing in Australia, Canada, the
United Kingdom, and the United States, 35 Prof’l Psychology Research and
Practice (2004).........................................................................................................................7
iv
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 6
Date Filed: 07/14/2014
Daniel J. Reschly et al., Mental Retardation: Determining Eligibility for Social Security
Benefits (2002) ...........................................................................................................................9
David Wechsler, WAIS-III Administration and Scoring Manual (1997) ..................................7
Gary Groth-Marnat, Handbook of Psychological Assessment (4th ed. 2003).............................8
Jerome M. Sattler, Assessment of Children (3rd ed. 1992) .......................................................10
Jerome M. Sattler & Joseph J. Ryan, Assessment with the WAIS-IV (2009) .......................13
v
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 7
Date Filed: 07/14/2014
INTEREST OF AMICUS CURIAE
The Texas Psychological Association (“TPA”) is the leading professional
association of psychologists in the state of Texas. Founded in 1947, the TPA is the
Texas state affiliate of the American Psychological Association (“APA”). The TPA
seeks to advance the field of psychology as a science and a profession. The TPA was
instrumental in the passage of the Texas Psychologist’s Certification and Licensing
Act in 1969, which led to the establishment of the Texas State Board of Examiners of
Psychologists and the promulgation of professional standards as part of the Texas
Administrative Code to ensure that all members of the psychological profession seek
to adhere to the highest standards of scientific achievement and professional conduct.
Amicus has a vital interest in ensuring that courts and administrative agencies
assess mental retardation in accordance with the best professional practices and that
legal determinations that depend on scientific evaluation strictly adhere to proper
clinical testing procedures and evaluations.1
INTRODUCTION
In Atkins v. Virginia, the Supreme Court recognized that the “widespread
judgment about the relative culpability of mentally retarded offenders” concluded that
1
All parties have consented to the filing of this brief. In addition, amicus confirms that
no counsel for any party authored this brief in whole or in part; and no party or
counsel for any party made a monetary contribution intended to fund the preparation
or submission of this brief.
1
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 8
Date Filed: 07/14/2014
the execution of those individuals did not promote “the penological purposes served
by the death penalty” and was prohibited by the Eighth Amendment. 536 U.S. 304,
317 (2002). At the same time, however, the Court recognized that “[t]o the extent
there is serious disagreement about the execution of mentally retarded offenders, it is
in determining which offenders are in fact retarded.” Id.
Although the terminology has changed and “mental retardation” is now
properly referred to as “intellectual disability,” the revised nomenclature “describe[s]
the identical phenomenon.” Hall v. Florida, 134 S. Ct. 1986, 1990 (2014); see also id.
(citing Rosa’s Law, 124 Stat. 2643 (2010) (changing entries in the U.S. Code from
“mental retardation” to “intellectual disability”); Am. Psychiatric Ass’n, Diagnostic and
Statistical Manual of Mental Disorders 33 (5th ed. 2013) (hereinafter “DSM-V”)
(“[I]ntellectual disability is the term in common use by medical, educational, and other
professions, and by the lay public and advocacy groups.”).
The Atkins Court declined to promulgate specific rules for determining
intellectual disability, but emphasized the importance of the use of proper clinical
practices for evaluating offenders who may be subject to the death penalty. First, the
Court highlighted the “much broader social and professional consensus” opposing
the imposition of the death penalty on intellectually disabled individuals, citing
approvingly to the briefs of “several organizations with germane expertise,” including
the American Psychological Association. Id. at 316, n.21. Second, and of particular
2
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 9
Date Filed: 07/14/2014
relevance to this case, the Court noted that the diagnosis of intellectual disability was
made only after proper clinical testing, specifically, the administration of “the
Wechsler Adult Intelligence Scales test (WAIS–III), [which was] the standard
instrument in the United States for assessing intellectual functioning.” Id. at 309 &
n.5.
Only this past May, the Supreme Court more clearly elucidated the requirement
that courts adhere to proper psychological standards of evaluation in implementing
Atkins’ prohibition on the execution of the intellectually disabled. In Hall v. Florida,
the Court considered a state law that prescribed certain inflexible rules for the
utilization of IQ scores in the diagnosis of intellectual disability for purposes of the
Eighth Amendment. Confronting a conflict between state law and the learned
opinions of the psychological profession with respect to the diagnosis of intellectual
disability, the Court unequivocally adopted the clinical judgment of medical
professionals.
The Court therefore held that “it is proper [for courts] to consider the
psychiatric and professional studies that elaborate on the purpose and meaning of IQ
scores.” 134 S.Ct. at 1993. In doing so, the Court rejected Florida’s determination
that the state was free to utilize a hard IQ cutoff of 70, above which individuals could
be subjected to capital punishment, without accounting for the proper clinical
standard error of measurement. The Court based its rejection of this practice on the
3
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 10
Date Filed: 07/14/2014
fact that “[t]he professionals who design, administer, and interpret IQ tests have
agreed, for years now, that IQ test scores should be read not as a single fixed number
but as a range.” Id. at 1995. The Court held that the state cannot adopt standards
that are at odds with those of the medical community. The Court stated that an
“awareness of the IQ test’s limits is of particular importance when conducting the
conjunctive assessment necessary to assess an individual’s intellectual ability.” Id. at
2000. And in making the determination that Florida’s rule violated the Constitution,
the Court relied on an amicus brief setting forth the professional expertise of the
psychological community, citing authoritatively the American Psychological
Association’s brief numerous times in its opinion.
This case presents a situation, similar to that in Hall, in which the State and the
courts below turned a blind eye to “the IQ test’s limits.” The TPA wishes to
emphasize three specific points applicable to this or any judicial or administrative
proceeding wherein the gravity of the determination would warrant attention to the
use of a complete instrument, properly administered so as to include all subtests, and
with such documentation as would permit later re-examination.
First, according to standards of professional practice, only the proper
administration of a complete standard test of intellectual functioning is sufficient to
make a diagnostic judgment that an individual is not intellectually disabled and
therefore susceptible to execution.
4
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 11
Date Filed: 07/14/2014
Second, even when the proper test is utilized, the administrator of any test
must keep a complete and accurate set of notes, known as “test data,” that both
records the individual’s test answers and indicates where professional judgment was
exercised during the administration of that test. Failure to preserve adequate records
of the test’s administration significantly undermines the scientific method necessary to
make a diagnosis of intellectual disability because it prevents any reexamination of the
test results and so reduces the testing procedure to nothing more than the number
that is its final result.
Finally, although it is always inappropriate to use a short-form test of the type
administered here for diagnostic purposes, even a professionally appropriate use of a
short-form test to approximate a full scale IQ score must correctly distinguish
between the reliability of the test across multiple administrations and the validity of the
short-form test result as an indicator of a full scale IQ score. Failing to account for
the difference between these two coefficients in determining the appropriate IQ range
results in statistical errors that are significant.
I.
A SHORT-FORM TESTING PROTOCOL IS PROFESSIONALLY
INADEQUATE TO MAKE A DIAGNOSTIC JUDGMENT
REGARDING INTELLECTUAL DISABILITY.
The standard psychometric test of intellectual functioning is the Wechsler
Adult Intelligence Scale (“WAIS”). The WAIS consists of a large number of subtests
designed to measure different levels of intellectual functioning, and the test is
5
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 12
Date Filed: 07/14/2014
“standardized” in order to give accurate and comparable IQ scores for most members
of the general U.S. population. The WAIS is updated over time to correct and
improve its accuracy as the standard measure of intellectual functioning. The most
recent edition, the WAIS-IV, was released in 2008.
The State’s experts testified that Mr. Henderson was administered an IQ test
when he was processed into prison in 1994. At that time, the standard IQ test was
the Wechsler Adult Intelligence Scale—Revised (“WAIS-R”). The WAIS-R consists
of six verbal subtests and five performance subtests. The verbal tests are:
Information, Comprehension, Arithmetic, Digit Span, Similarities, and Vocabulary.
The Performance subtests are: Picture Arrangement, Picture Completion, Block
Design, Object Assembly, and Digit Symbol. The scores derived from this test are a
Verbal IQ, a Performance IQ, and a Full Scale IQ. The Full Scale IQ is a standard
score with a mean of 100 and a standard deviation of approximately 15.
The particular test that was administered to Mr. Henderson was a portion of
the WAIS-R, consisting of the Vocabulary and Block Design subtests only. As this
evaluation took place long before Atkins was decided, it is unsurprising that the State’s
experts acknowledged that the purpose of this abbreviated testing was not to make a
diagnosis of intellectual disability; rather, it was merely a “screening” measure
designed to determine whether an inmate required further services consistent with his
special needs. See Transcript of Hearing at 158:2-6, Ex parte Henderson, No. 181-CR-
6
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 13
Date Filed: 07/14/2014
12-93 (102nd Dist. Ct., Red River County, Tex. Oct. 5, 2005) (testimony of Steve
Gilliland) (hereinafter “Gilliland Testimony”) (describing “screening process . . . to
determine special needs”). In other words, the State simply extrapolated Henderson’s
full scale IQ from his scores on two subtests extracted from the full form of the
WAIS-R to estimate what score he would have received had he taken the full test.
Despite this admission that Henderson’s test was not administered for the purpose of
making a diagnosis of intellectual disability, id. at 173:14-15 (“[W]e did not diagnose
mental retardation.”), the State now seeks to use that test for precisely this purpose.
This procedure is professionally inadequate to rule out a diagnosis of
intellectual disability. Criticism of the use of short-form tests, which are designed as
screening mechanisms and not as diagnostic tools, is universal in the psychological
community. See, e.g., Anthony P. Thompson et. al, Brief Intelligence Testing in Australia,
Canada, the United Kingdom, and the United States, 35 Prof’l Psychology Research and
Practice 286 (2004). Indeed, in superseding the WAIS-R used in this case, the manual
for the WAIS-III itself emphasized that “short forms should be used with extreme
caution. They should not be used in isolation for diagnosis or classification. In
general, they should not be used in examinations with a legal, judicial, or quasilegal
purpose (e.g., a statutorily mandated diagnosis or determination of a disability).”
David Wechsler, WAIS-III Administration and Scoring Manual 35-36 (1997).
7
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 14
Date Filed: 07/14/2014
The necessity for using a complete testing instrument for any diagnostic
purpose represents the consensus of psychological professionals that a variety of
factors can inform the appropriateness of a particular testing protocol. In limited
circumstances that do not require a clinical diagnosis, it is possible to administer a
short-form test in the interests of time and for purposes of making merely an initial
assessment of an individual’s intellectual functioning. “[S]hort forms can serve
appropriately as screening devices, which are best used when the purpose of
evaluation is other than for intellectual assessment. The results can be used either as a
rough indicator of intelligence, or as a basis for determining whether a more complete
cognitive assessment is necessary.” Gary Groth-Marnat, Handbook of Psychological
Assessment 191 (4th ed. 2003). This is with good reason: “Although time-efficient,
these short forms tend to provide less information about a person’s cognitive abilities,
produce a wider band of error than a full administration, result in less clinical
information, and are often of questionable accuracy when used for intelligence
classifications.” Id. From a professional viewpoint, “[a]t a psychometric level, the
development of short forms has been found wanting and results from abbreviated
measures are not synonymous with IQs from the standard batteries.” Thompson,
supra, at 288.
The clinical profession’s criticism of the testing protocol applied to Mr.
Henderson is not result-oriented; it is the point of that criticism that the protocol is
8
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 15
Date Filed: 07/14/2014
inappropriate for diagnostic purposes regardless of the diagnosis obtained. A shortform test is equally inappropriate as a clinical tool whether it is applied to prove
intellectual disability or to rebut such a claim. But certainly when the purpose of the
clinical evaluation is to make a determination as consequential as an individual’s
susceptibility to execution, nothing less than a full-test administration will suffice.
Even in the context of the far less consequential determination of eligibility for Social
Security benefits, it is settled that such short-form tests are unacceptable. “When
intelligence testing is conducted for high-stakes purposes, multidimensional, full-scale
instruments should be used in the diagnostic, decision-making process because these
instruments provide the most convincing evidence of technical adequacy, construct
sampling, and validity.” Daniel J. Reschly et al., Mental Retardation: Determining
Eligibility for Social Security Benefits 118 (2002).
As the Supreme Court recognized in Hall, “[a]n IQ score is an approximation,
not a final and infallible assessment of intellectual functioning.” 134 S. Ct. at 2000.
When evaluating an expert’s presentation of intelligence testing results, “awareness of
the IQ test’s limits is of particular importance when conducting the conjunctive
assessment necessary to assess an individual’s intellectual ability.” Id. In particular,
“in using these scores to assess a defendant’s eligibility for the death penalty, a State
must afford these test scores the same studied skepticism that those who design and
use the tests do.” Id. at 2001.
9
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 16
Date Filed: 07/14/2014
The state courts and the district court failed to apply the “studied skepticism”
of short-form tests as conclusive of intellectual functioning that is the professional
judgment of the psychological community. This error occurred even though the
courts had available to them evidence supporting the professional judgment that
amicus describes here. For example, the courts had available Jerome Sattler’s seminal
work, Assessment of Children, which explains:
When a small number of subtests is used, the estimate may
be far less adequate than that provided by the Full Scale.
Educational and clinical situations call for more, rather than less,
extensive cognitive evaluation. You are encouraged to administer the
Full Scale, unless there is some compelling reason to administer a
short form….The Full Scale should be administered so as to
maximize diagnostic information and minimize placement errors.
Short forms are not recommended for any placement, educational, or
clinical decision-making purpose.
Jerome M. Sattler, Assessment of Children 139 (3rd ed. 1992) (emphasis in original).
This Court has twice properly applied the professional skepticism amici urge
here to reject short-form tests. In Blue v. Thaler, 665 F.3d 647, 659 (5th Cir. 2011), this
Court affirmed the Texas courts’ rejection of short-form test results, which the state
courts chose simply to treat as if they did not exist at all. This Court held that “the
result of a short-form test is not a reliable substitute for a full scale IQ score” and
affirmed the Court of Criminal Appeals’ conclusions that it should not “‘try to
extrapolate an accurate IQ’ based on ‘an incomplete test score’ and instead chose to
10
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 17
Date Filed: 07/14/2014
‘simply regard the record as it comes to us as devoid of any reliable IQ score.’” Id.
(quoting Ex parte Blue, 230 S.W.3d 151, 166 (Tex. Crim. App. 2007)).
Similarly, in Rivera v. Dretke, Civ. B-03-139, 2006 WL 870927 (S.D. Tex. Mar.
31, 2006), aff’d in relevant part sub nom. Rivera v. Quarterman, 505 F.3d 349 (5th Cir. 2007),
the district court criticized the State’s proffered psychological evidence on the
grounds that “None [of the offered results] are [from] a WAIS-III test, however,
which both sides agree is the most reliable test for determining an adult’s IQ. The
Court notes that all four of these tests were shorter, screening-type tests.” Id. at *20.
It added, “all of the experts that testified in this case and all of the literature that this
Court has reviewed suggests that one sacrifices accuracy for speed when one
administers tests designed for screening purposes.” Id. The court granted Rivera’s
petition for habeas corpus, concluding that “None of the other available test scores
have a degree of sufficient reliability to satisfy this Court. Those that were
administered in prison were not gold standard tests given in an appropriate fashion to
establish a reliable IQ score. They were for the most part screening tests given in a
group setting to help guide the prison system in its mission to house and care for
prisoners.” Id. at *26. This Court affirmed, holding that the scientific fact of the
diminished reliability of short-form tests “offered a reasoned basis for how [the
district court] weighed the tests in its analysis.” 505 F.3d at 362.
11
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 18
Date Filed: 07/14/2014
The TPA agrees with the analysis of this Court in Blue and Rivera. Short-form
tests are professionally inadequate to make a diagnosis of intellectual disability. They
should not be accepted as competent proof in proceedings under Atkins and Hall to
make determinations about an individual’s capacity to be subject to execution.
II.
FAILURE TO MAINTAIN ADEQUATE RECORDS DURING
PSYCHOMETRIC TESTING OF INTELLIGENCE
SIGNIFICANTLY UNDERMINES THE SCIENTIFIC METHOD.
The State’s experts apparently acknowledge that it was a significant error for
Mr. Gilliland to fail to maintain complete notes of Mr. Henderson’s testing. See
Gilliland Testimony, supra, at 180:11-13 (“It was pretty important, and it was a mistake
on my part, and it’s not something that I intended to do.”). This was not merely an
error, it was a serious breach of the standards of conduct of the psychological
profession, so serious as to itself undermine reliance upon the result.
The APA’s Code of Ethics requires professional psychologists to maintain
records of “test data,” which it defines as “raw and scaled scores, client/patient
responses to test questions or stimuli, and psychologists’ notes and recordings
concerning client/patient statements and behavior during an examination.” Am.
Psychological Ass’n, Ethical Principles of Psychologists and Code of Conduct, Standard 9.04(a)
(2010). It is important that this information be preserved because an IQ examination
is not a purely standardized test that is administered without any semblance of
judgment. On the contrary, “the concept of intelligence, as represented by the IQ
12
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 19
Date Filed: 07/14/2014
score, can sometimes appear misleadingly straightforward. Intelligence test scores can
be complex, however, involving a variety of cognitive abilities, the influence of
cultural factors, varying performance under different conditions, and issues related to
the nature of intelligence.” Groth-Marnat, supra, at 5. When submitting a diagnosis
of either average intellectual functioning or intellectual disability, “[f]our major
influences on an individual’s performance on an intelligence test should be considered
in making diagnostic decisions or recommendations for intervention. Each of the four
poses threats to the validity of assessment results and all of them can be controlled to
some considerable degree: (1) examinee characteristics, (2) examiner characteristics,
(3) environmental influences, and (4) psychometric characteristics of tests.” Reschly,
supra, at 100.
WAIS testing inherently involves the application of clinical judgment.2
Although examiners strive for neutrality and standardization of test results, the
subjectivity inherent in scoring even a test as comprehensive as the WAIS requires
that subjective judgments be recorded precisely so that they can be evaluated by
supervisors and other reviewers. The WAIS administrative guidelines require other
2
For example, on the modern version of the Vocabulary subtest administered to
Henderson, the examiner is required to determine whether an individual’s answer to
the question merits 0, 1, or 2 points, based on factors such as whether the answer
“demonstrate[s] poverty of content,” uses “less pertinent synonyms,” or constitutes a
“correct figurative use of the word.” Jerome M. Sattler & Joseph J. Ryan, Assessment
with the WAIS-IV 98 (2009).
13
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 20
Date Filed: 07/14/2014
information to be recorded in testing notes as well. For example, if an individual
gives an unclear response, the examiner may prompt the individual further, but is
required to “record a ‘(Q)’ on the Record Form each time” he does so. Sattler &
Ryan, supra, at 97. The same is true if an examiner determines it is appropriate to
repeat a question for an individual who appears not to have understood, which is
supposed to be reflected in “an ‘(R)’ on the Record Form.” Id. at 100.
When an examiner’s notes are reduced to the mere recitation of his or her final
result, as in this case, it is impossible for any subsequent reviewer—whether a
supervisor such as Dr. Gillhausen or an expert reviewing the diagnosis for some later
purpose—to assess the degree to which clinical judgment affected an individual’s
score. This review and supervision is of such importance that the Texas State Board
of Examiners and Psychologists has promulgated standards for maintenance of
psychological records, including test data, in the Texas Administrative Code. As
relevant here, Section 465.22(a) of the Code provides that:
(1) All licensees shall create and maintain accurate,
current, and pertinent records of all psychological services
rendered by or under the supervision of the licensee.
(2) All records shall be sufficient to permit planning for
continuity in the event that another care provider takes
over delivery of services to a patient or client for any
reason, including the death, disability or retirement of the
licensee and to permit adequate regulatory and
administrative review of the psychological service.
(3) All licensees shall identify impressions and tentative
conclusions as such in patient or client records.
14
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 21
Date Filed: 07/14/2014
(4) All records and record entries shall be created in as
timely a manner as possible after the delivery of the specific
services being recorded.
(5) Records, test data and test protocols shall be
maintained and stored in a way that permits review and
duplication.
Tex. Admin. Code § 465.22(a).
It is therefore incorrect to say that we have available all the diagnostic
information necessary from the administration of the short-form test to Mr.
Henderson. What is available to the Court is only one individual’s assessment,
subsequent review having been rendered impossible, thus precluding any analysis of
the degree to which non-scored factors may have affected the score. This is a
significant failure that is inconsistent with the principles enshrined in the Texas
Administrative Code and the ethical standards of the psychological profession. The
court should not accept test results that are simultaneously unverifiable and
unfalsifiable, because they do not meet the criteria of reviewability required for a
principled diagnostic judgment.
III.
PROPER APPLICATION OF THE CONCEPTS OF VALIDITY AND
RELIABILITY IS NECESSARY TO EXTRAPOLATE AN
ACCURATE FULL SCALE IQ.
The TPA does not believe that it is ever professionally appropriate to use a
short-form intelligence test to extrapolate a full-scale IQ score for purposes of
diagnosing whether an individual is intellectually disabled. Even in the limited
circumstances in which it would be proper to approximate a full-scale IQ score from
15
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 22
Date Filed: 07/14/2014
a short-form test for screening purposes only, professional practice requires test
administrators to distinguish and apply correctly the mathematical coefficients of
validity and reliability that describe different aspects of a test’s accuracy.
Validity and reliability are terms of art in the psychological profession. Validity
“refers to the extent to which a test measures what it is supposed to measure, and
therefore the appropriateness with which inferences can be made on the basis of the
test results.” Sattler, supra, at 30. Reliability by contrast “refers to the consistency of
measurements . . . A test is considered unreliable if scores are subject to random,
unsystematic fluctuations; obviously a score is not dependable if readministration of
the test is likely to result in a randomly different score.” Id. at 25-26.
In his testimony, Dr. Gillhausen appears to have transposed these concepts.
The Court of Criminal Appeals noted that Dr. Gillhausen “testified that the reliability
of the short form WAIS-R is 94% which is ‘very acceptable.’” Ex parte Henderson, No.
WR-37658-03, 2006 WL 167836, at *3 (Tex. Crim. App. Jan. 25, 2006) (Cochran, J.,
concurring). That statement is accurate. Sattler’s seminal text notes that the reliability
coefficient of the “Vocabulary Plus Block Design” short-form WAIS-R administered
to Mr. Henderson is in fact 0.94 and this “reliability of the composite is impressive.”
Sattler, supra, at 234. Where Dr. Gillhausen appears to have erred, however, is in his
subsequent statement that “The reliability of applicant’s 83 I.Q. score ‘would allow us
to state that his I.Q. would fall within the range from seventy-six to ninety, about
16
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 23
Date Filed: 07/14/2014
ninety-five percent of the time, so that's fairly close.’” Ex parte Henderson, 2006 WL
167836, at *3 (Cochran, J., concurring).
That determination is controlled not by the test’s reliability, but by its validity
coefficient. A short-form test is supposed to approximate the score on a full test, and
so the validity coefficient of a short form test is a mathematical reflection of how well
correlated the short-form score is to the long-form score. The Vocabulary-Block
Design short-form WAIS-R has the highest validity coefficient of any two-part
subtest, at 0.90, but that is a lower coefficient than its reliability score of 0.94, cited by
Dr. Gillhausen. Sattler, supra, at 849 tbl. C-34. A short-form test only reaches the
0.94 validity coefficient once at least four subtests are administered. Id. In short, it is
professionally incorrect to use the reliability coefficient to calculate how well a shortform test score would correlate to a full-test score. That calculation must be made
using the proper validity coefficient.
It bears note that this error is yet another reason why it is inappropriate to
administer a short-form test for a high-stakes diagnostic purpose like that presented
here. First, no two-part short form test has a validity greater than 0.90. Even
assuming that all calculations are performed accurately, this comparatively low validity
expands significantly the standard error range and adds to the imprecision inherent in
IQ measurement. Second, because the full test is not administered, it becomes
necessary to perform these additional calculations relying on multiple different tables
17
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 24
Date Filed: 07/14/2014
and coefficients to extrapolate a full-scale score from a short-form test. Although
administration of the full test is not always a simple endeavor, it is markedly less
susceptible to the possibility of error along the many required axes of evaluation than
is the short-form test that was utilized here.
CONCLUSION
For the foregoing reasons, it is the professional assessment of the TPA that the
testing protocols used by the State in this case—including the test that was
administered, the recordkeeping procedures that were followed, and the mathematical
calculations used to describe the test results—were insufficient to support a diagnostic
judgment with respect to Mr. Henderson’s intellectual functioning.
Dated: July 14, 2014
Respectfully submitted,
/s/ Mark K. Glasser
Mark K. Glasser
Texas Bar No. 8014599
David W. Denton Jr.
New York Bar No. 5169602
Sidley Austin LLP
1000 Louisiana St., Suite 6000
Houston, TX 77007
713.495.4623
[email protected]
[email protected]
Counsel of Record
18
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 25
Date Filed: 07/14/2014
CERTIFICATE OF COMPLIANCE
In accordance with Circuit Rule 32(a) and Rule 32(a)(7) of the Federal Rules of
Appellate Procedure, the undersigned certifies that the accompanying brief has been
prepared using 14-point Garamond Roman typeface, and is double-spaced (except for
headings and footnotes).
The undersigned further certifies that the brief is proportionally spaced and
contains 4,270 words exclusive of the statement regarding oral argument, Rule 26.1
disclosure statement, table of contents, table of authorities, signature lines, and
certificates of service and compliance. This does not exceed 7,000 words, as
mandated by Rule 29(d) of the Federal Rules of Appellate Procedure. The
undersigned used Microsoft Word 2007 to compute the count.
/s/ Mark K. Glasser
Mark K. Glasser
19
ACTIVE 202636065v.2
Case: 14-70001
Document: 00512697728
Page: 26
Date Filed: 07/14/2014
CERTIFICATE OF SERVICE
I hereby certify that on this 14th day of July, 2014, I electronically filed the
foregoing with the Clerk of the Court using the CM/ECF System, which will send
notice of such filing to all registered CM/ECF users.
/s/ Mark K. Glasser
Mark K. Glasser
20
ACTIVE 202636065v.2