Session Code: 1435
Date/Time: Sat., Nov 17, 9:30 -10:30AM
Toward a Comprehensive Fluency
Assessment Framework:
Challenges & Solutions
Kenneth J. Logan, Ph.D.
University of Florida
Courtney T. Byrd, Ph.D.
University of Texas - Austin
Ronald B. Gillam, Ph.D.
Utah State University
Disclosure
• Kenneth J. Logan and Ronald B. Gillam are coauthors of the Test of Childhood Stuttering,
and receive royalties from the test publisher,
Pro-Ed, Inc.
Session Plan
Presenter
Topic
Outcomes
Logan
Model-based assessment of
fluency
Describe various perspectives on
fluency assessment and their
associated measurement options.
Byrd
Challenges associated with
measuring speech disfluency in
Spanish-English speakers who
stutter
Describe how stuttering is manifested
in bilinguals; ways to minimize the risk
of false positive diagnosis of stuttering
Gillam
Psychometric challenges
associated with comparing
people who stutter to people
who do not stutter
Describe development of normreferenced fluency scores and how
they should be used and interpreted in
fluency assessment.
All
Audience discussion
Model-Based Assessment of
Fluency Disorders
Ken Logan, Ph.D.
University of Florida
Assessment activities are integral in all phrases
of clinical service provision
Identify
Modify
Evaluate
Diagnose
treatment treatment treatment
disorders
goals
goals
outcomes
Deciding What To Assess
Inductive Processes:
What the patient tells
us about his or her
experience & quality
of life.
Deductive Processes:
(1) What we know about
a disorder.
(2) What we know about
normal fluency processes
After Onslow (2006)
Why base assessment on a model?
• Complete assessments
– Assessing all relevant
issues (validity)
• Consistent assessments
– Assessing variables
uniformly at all phases of
interaction (reliability)
• Efficient assessments
– Avoiding irrelevant issues
What kind of model?
We are assessing
Fluency Issues
So, we need a
model of
fluency!!
Primary Dimensions
Developmental
Stuttering
SLI
Epilepsy
Anomia
Cluttering
Second-language
learners
Acquired
stuttering
Bases for a Model of Fluency
Fillmore (1979)
Starkweather
(1987)
Martin &
Haroldson;
Smith et al.
• 1) Talkativeness; 2) Efficiency
• 3) Flexibility; 4) Creativity
• 1) Continuity; 2) Rate
• 3) Rhythm; 4) Effort
• Naturalness
• Stability
ICF Model, (WHO, 2001)
Health condition
(disorder or disease)
Body Functions
and Structures
Environmental
Factors
Activities
Participation
Personal
Factors
Continuity
A working model
of fluency
Rate
Rhythm
Effort
Naturalness
Talkativeness
Stability across activities
1. Assessing Continuity
“Continuity Breaks”
• Disfluency frequency
–
–
–
–
–
–
Rating Scales
• Formal Tools
• TOCS Observational Rating
% of words
Scales
# per 100 words
• Informal Tools
% of syllables
• Perceptions of Stuttering
Inventory (PSI)
# per time unit
• Self-Rating scales
(organized by type)
(analyze elicited samples)  How often was your speech interrupted by
disfluency in the past week?
1
Very
rarely
2
3
4
5
Very
often
Continuity: Analyze disfluency frequency
When -
- 70 syllables
-7 either contain or
are preceded by
disfluency
- (7/70)*100 = 10%
- 10 disfluencies
per 100 syllables
um
When the sunlight strikes raindrops in
the air, they act as a prism and form a
rrr
rainbow.
The rainbowb---is a division
of
llll
ffff
white light into many beautiful colors.
These take the shape of a long round
a- a arch, with
its
path
high
above,
and
its
are
two ends apparently beyond the
horizon.
Note added and deleted
words, too!
2. Assessing Rate
Speech Rate
Articulation Rate
• Based on fluent stretches of
speech
• Based on all utterances
(disfluent & fluent)
– Syllables (or words) per min
– Syllables (or words) per sec
– Syllables (or words) per min
– Syllables (or words) per sec
Analyze Elicited Speech Samples
Rating Scales – Informal Tools
1
Much
slower than
typical
2
3
4
Typical
5
6
7
Much
faster than
typical
What do clinicians use?
Rate Assessment
Measurement Method
Manual (w/ stop watch)
Manual (w/ digital imaging)
Subjective estimate (w/ time value)
Subjective estimate (w/ ordinal
descriptor)
Automated (via software)
% of Clinicians Using
Method “Always or
Often”
49
2
37
17
10
Al-Ghamedei & Logan (2012)
3. Assessing Rhythm
• Multi-dimensional, e.g.,
–
–
–
–
–
Continuity
Rate
Segment & pause duration
Stress patterns
Disfluency duration
Within repetitions
• Timing structure of partand whole-word reps
– Ratio of speech to silence
– E.g., Throneburg et al. (2001)
Disfluency duration
• Length (in sec) of selected
disfluencies
• Iterations per repetition
– [s- s-]some = 2 iterations
– [we-]we = 1 iteration
3. Assessing Rhythm (duration)
Informal Ratings
• Evenness of repetition (e.g.,
Objective measures
• Manually timed measures
Pindzola & White, 1986; Van Riper,
1982)
– Stop-watch based
– Spectrogram based
– Automated computation
• Estimated duration of
longest disfluencies (e.g, Riley,
• SSI-4 (Riley, 2008) software
2008)
• Scaled ratings of duration
1
Very
brief
2
3
4
5
Very
long
Disfluency consumes time.
Time affects rate.
1.82 s
The boy is polishing the shoes.
4.64 s
ThThThThe b- … the boy is p- p- p- polishing the shshshoes.
What do clinicians use?
Duration
Measurement Method
Manual (w/ stop watch)
Manual (w/ digital imaging_
Subjective estimate (w/ time
value)
Subjective estimate (w/ descriptor,
e.g., “very brief”)
% of Clinicians using
Method “Always or
Often”
54
3
56
40
Al-Ghamedei & Logan (2012)
4. Assessing Effort
Mental effort: Constructs
•
•
•
•
•
Attention
Concentration
Thinking
Planning
Rehearsal
Physical effort: Constructs
• Tension
• Force
• Struggle
• Activation
Measures
• Subjective/self-ratings
• Likert scales, PSI,
• Objective
• Neural activation
Measures
• Subjective (self-ratings)
• Likert scales, PSI, OASES,
TOCS, SSI-4
• Objective
• Muscle activation (EMG)
5. Assessing Naturalness
• Ingham & Riley (1998)
on speech naturalness:
Rate & Naturalness
– many constituents, e.g.,
•
•
•
•
•
•
Rate
Inflection
Articulation
Resonance
Loudness
Sentence structure
– prominent ones vary
across speakers and
circumstances
Logan, Roberts, Pretto, & Morey (2002)
5. Assessing Naturalness
1
2
3
4
5
Very
Natural
Very
Unnatural
Listener Perspective
Speaker Perspective
How natural does
speech sound?
How natural does
speaking feel?
Refs:
Refs:
6. Assessing Talkativeness
Talkativeness
Verbal
Output
How much?
How many
settings?
Succinctness
Flexibility
6. Talkativeness: Verbal Output
Talkativeness: By task
• # of conversational turns
• # of utterances
• # of words
• # of words per turn
• # of words per utterance
• Total time talking
• Average speech rate
Talkativeness: By setting
• # of distinct activities/partners
• # of new activities/partners
• Activity frequency (times per
week)
• # of abandoned activities
Talkativeness:
Succinctness/Efficiency
Message density
• # of clauses per utterance
• # semantic propositions per
words spoken
• # of semantic propositions
per time unit
Coherence/Focus
• Topic shifts/transitions
• Topic development
In Closing
• Fairly simple measures based on a model of fluency
• Many of the measures are relatively straightforward
to do – efficient.
• The model (and measures) provide a common
ground of comparison for fluency performance across
clinical populations and within individuals who
present with fluency problems that arise from
multiple sources.
Spanish-English Bilingual
Children
Differential Diagnosis Considerations
Courtney T. Byrd, Ph.D., CCC-SLP
Differential Diagnosis of Stuttering in
Bilinguals
• Critical need for bilingual fluency data to
enhance the cross-linguistic competence and
differential diagnosis of stuttering in this
rapidly growing clinical population
– e.g., Tetnowski et al. 2012; Shenker, 2011; Van Borsel et al.
2011; Bernstein Ratner, 2012
Increased Risk for
Development/Persistence
“Together, these findings suggest that if a
child uses a language other than English
in the home, deferring the time when
they learn English reduces the chance of
starting to stutter and aids the chances of
recovery later in childhood.” p.45
Howell, Davis, & Williams (2009)
Arch Dis Child2009;94:42-46
Increased Risk for False Positive
• If SLP cannot speak the language being
assessed (e.g., Finn & Cordes, 1997; Van Borsel & Pereira 2005)
• If the expected behaviors differ in type,
frequency or both…
• Linguistic uncertainty (e.g., Bedore et al. 2006)
leads to increase maze production…
Critical Need for Normative Data
• Typical disfluent behavior in bilinguals needs
to be determined in order to accurately
identify the atypical speech behavior (i.e.,
stuttering) in bilinguals.
Clinically Relevant Population
• Spanish English (SE) bilinguals
– Markedly rapid rate of growth…
– Uniquely vulnerable to false positive…
Purpose
• To identify types and frequencies of
disfluencies considered to be stuttering and
nonstuttering-like produced by bilingual SE
children who are typically fluent.
• To explore whether the disfluent speech
differs based on:
– language dominance
– language produced
Byrd, Bedore, & Ramos, in preparation
Method: Participants
• Mexican-American kindergarteners from central Texas
– No history of speech and/or language diagnosis or therapy
– No present or prior history of parent or teacher concern
about child’s fluency
– No present or prior history of parent or teacher concern
about any other speech and/or language impairment
Byrd, Bedore, & Ramos, in preparation
Method: Participants
• Dominance: Input/Output + Ability
• Parent report and teacher report regarding the
child’s language input and output on hour-byhour basis (Gutiérrez-Clellen & Kreiter, 2003; Restrepo, 1998).
• Completed experimental version of the Bilingual
English Spanish Assessment (BESA) (Peña et al., in
development).
Byrd, Bedore, & Ramos, in preparation
Method: Participants
• Balanced bilinguals (BB)
– using Spanish and English 40-60% of the time
– 6 (3 males, 3 females), age 5;6 – 6;7
• Spanish dominant (BSD)
– using Spanish 61-80% of the time
– 6 (3 males, 3 females), age 5;6 – 6;7
• English dominant (BED)
– using English 61-80% of the time
– 6 (3 males, 3 females), age 5;6 – 6;7
N = 18 (9 males, 9 females; age 5;6 to 6;7)
Byrd, Bedore, & Ramos, in preparation
Method: Data Collection
• Task: two narratives - tell & retell
– collected in each language on different days w/in 4 weeks
– one of four wordless picture books
•
“A Boy, a Dog, and a Frog” (Mayer, 1967); “Frog, Where are you?” (Mayer, 1969); “Frog On His Own” (Mayer,
1973); “Frog Goes to Dinner” (Mayer, 1974).
• Collection
– digital audio recorder (Sony MS-515 or ICD-P320) with
external microphone (ECM 115)
• Transcription
– Sony digital voice editor (v 2.4.04)
* See Bedore, Peña, Ho, & Gillam (2010) for more info re: sampling procedures
Byrd, Bedore, & Ramos, in preparation
Method: Data Preparation
• Transcribed and coded the narratives using the
Systematic Analysis of Language Transcription
(SALT: Miller & Iglesias, 2008).
– Trained bilingual SE graduate research assistants
• Each disfluent word/phrase was coded for
– specific type of disfluency
– category of SLD or NonSLD
• Reliability checks completed for all data
Byrd, Bedore, & Ramos, in preparation
Results: Language Sample Characteristics
BB
BSD
BED
English
Spanish
English
Spanish
English
Spanish
5.41
4.76
4.975
4.39
4.545
4.535
64
46.5
57.5
34.5
43.5
50.5
NTW
357.5
226
308.5
172.5
211
232.5
NDW
93
78.5
80
57
75
72.5
MLUw
NU
Language Dominance: No significant difference
Language Produced: No significant difference
Language Dominance by Language Produced: No significant difference
Byrd, Bedore, & Ramos, in preparation
Results: Types produced - SLDs
• 16/18 participants produced monosyllabic word repetitions in both
samples
– 1 produced monosyllabic word repetitions only in Spanish sample
– 1 produced monosyllabic word repetitions only in English sample
• 12/18 participants produced sound repetitions in either English or
Spanish
– only 1 produced sound repetitions in both samples
– 5 participants did not produce sound repetitions in either sample
• No participants produced inaudible or audible sound prolongations
Byrd, Bedore, & Ramos, in preparation
Results: Types produced - NonSLDs
• 17/18 participants produced revisions in both of samples
– 1 produced revisions in Spanish only
• 17/18 participants produced interjections in both samples
– 1 produced interjections in Spanish only
• 15/18 participants produced phrase repetitions in both samples
– 1 produced phrase repetitions in only one of sample
– 2 did not produce any phrase repetitions in either sample
• 14/18 participants produced unfinished words in both samples
– 1 produced unfinished words in one sample
– 3 did not produce an unfinished word either sample
Byrd, Bedore, & Ramos, in preparation
Results: Rhythm, Tension, Iterations
• No secondary behaviors
• No other atypical tension during speech
• No atypical rhythm in the repetition
• ….however, high number of iterations
• Monosyllabic word repetitions
– Range = 4 to 10, M = 6
• Sound repetitions
– Range = 4 to 8, M = 5
Results: Frequency
24
Mean percent disfluent
20
16
BB
BSD
BED
12
8
4
0
English
Spanish
% SLD
English
Spanish
%NonSLD
English
Spanish
%Total Disfluencies
SLDs higher than 3%/100 words in 14/18 participants
Byrd, Bedore, & Ramos, in preparation
Results: Frequency
24
Mean percent disfluent
20
16
BB
BSD
BED
12
8
4
0
English
Spanish
% SLD
English
Spanish
%NonSLD
English
Spanish
%Total Disfluencies
TDs higher than 10%/100 words in 13/18 participants in at least one sample
Byrd, Bedore, & Ramos, in preparation
Results: Language Dominance
16
Mean percent disfluent
12
F(2,30) = .883, p =
.424, ηρ² = .056
F(2,30) = .264, p = .769, ηρ² = .017
BB
BSD
BED
8
4
0
English
Spanish
% SLD
English
Spanish
%NonSLD
No significant difference in SLDs or non-SLDs produced relative to language
dominance.
Byrd, Bedore, & Ramos, in preparation
Results: Language Produced
F(1,30) = 3.357, p = .077, ηρ² = .101
16
Mean percent disfluent
12
* F(1,30) = 5.312, p = .028, ηρ² = .150
BB
BSD
BED
8
4
0
English
Spanish
% SLD
English
Spanish
%NonSLD
Participants produced a significantly higher percentage of stuttering-like disfluencies
in their Spanish samples than their English samples.
Byrd, Bedore, & Ramos, in preparation
Results: Language Dominance by Language Spoken
16
F(2,30) = 1.250, p = .301, ηρ² = .077
Mean percent disfluent
12
F(2,30) = .381, p = .686, ηρ² = .025
BB
BSD
BED
8
4
0
English
Spanish
% SLD
English
Spanish
%NonSLD
No interaction between language dominance and language sample for SLDs or non-SLDs
Byrd, Bedore, & Ramos, in preparation
Discussion: Language Dominance
• Dominance within a language may not be as
critical to disfluency as the nature of the
language being spoken
• Inconsistent dominance criteria across
previous studies may have compromised
interpretation of role of dominance in
stuttered speech.
– Coalson, Pena, & Byrd, in review
Byrd, Bedore, & Ramos, in preparation
Discussion: Language Dominance
Byrd, Bedore, & Ramos, in preparation
Discussion: Language Dominance
• Stuttering differs in balanced bilinguals
– Dale (1997) 4 adolescents - Spanish only
– Bernstein Ratner & Benitez (1985) – 1 adult , >English
• Stuttering  in more dominant language
– Carias & Ingram (2006)- 4 children (ages 4-10), 2 > Spanish, 2 > English
• Stuttering  in less dominant language
– Ardila et al. (2011) – 1 adult, > Spanish
Byrd, Bedore, & Ramos, in preparation
Discussion: Language Produced
• More speech disfluencies in Spanish than in English
because more grammatical and syntactic restrictions
(Bedore et al., 2006). For example…
– In Spanish, choose form with appropriate level of
definiteness, gender, and number.
– In English, only choose definiteness (a vs. the)
– Motor considerations?
Byrd, Bedore, & Ramos, in preparation
Differential Diagnosis Considerations
• Need for further evaluation of monosyllabic word
repetitions as a clinical marker for stuttering
• Inaudible or audible prolongations may also be
indicative of stuttering in bilinguals
• Atypical tension and rhythm may prove to be
more discriminating than type, frequency, and/
number of iterations
Byrd, Bedore, & Ramos, in preparation
Differential Diagnosis Considerations
• Frequency and related severity guidelines
• Adult versus child
• Family history
• Bilingual SE parent versus monolingual teacher
Byrd, Bedore, & Ramos, in preparation
Final Comment
• Continued exploration of typical versus
atypical fluency patterns in both of the
languages of bilingual speakers will inform
theoretical constructs regarding the
underlying nature of stuttering and help to
determine whether the characteristics that
have been used to date to identify stuttering
are as universal as is this complex disorder.
Developing a Comprehensive
Measure of Stuttering
Psychometric Considerations
Ron Gillam, Ph.D., CCC-SLP
Aspects of a Speech Fluency Measure
• Rapid picture naming
– Single word production under time pressure
• Modeled Sentences
– Sentence production with a syntactic model
– Elicits complex structures
• Structured Conversation
– Questions about pictures
• Narration
– Creating a short story about a sequence of pictures
Rating Scales
• How often (never, rarely, sometime, often)
children demonstrate
– Speech Fluency Rating Scale
• Get stuck or “blocked” when trying to say a sound
• Repeat the same sound or word more than once
– Disfluency-Related Consequences Rating Scale
• Seems to avoid saying words that he or she might not
say smoothly or fluently
• Becomes frustrated when he or she has trouble
speaking fluently.
Scoring Reliability
• Poor scoring reliability associated with
multiple disfluency categories (Cordes &
Ingham, 1994; Cordes, Ingham, Frank, &
Ingham, 1992).
• 2 broad classes of disfluencies
– Repetitions
– Prolongations/blocks
• Interval-based measures
– First 3 words (Modeled Sentences, Conversation,
Narration)
180
160
140
120
100
CWS
80
CWNS
60
40
20
0
10
20
30
40
50
60
70
80
90 100 110 120 130 140
Normative Scores
• Transform raw scores from the general
population into z-scores
• Convert z-scores into a distribution that
represents a normal or bell-shaped curve.
• Appropriate when the construct being
measured is expected to be normally
distributed in the population the test was
designed for.
Skewed Distributions
• Construct of disfluency does not have a
normal distribution in the general population.
• Behaviors that characterize stuttered speech
occur relatively infrequently in CWNS.
Standardization
• Standard scores developed from raw scores
for the typically developing sample
• Direct linear transformation of the raw scores
that maintained the shape of the original
distribution
• Polynomial regression to fit the progression of
means, standard deviations, skewness and
kurtosis across age groups.
Percentile Ranks
• Calculated directly from the raw score
distributions
• Indicate the percentage of the distribution of
the typically developing sample that is equal
to or below any particular percentile
– Percentile of 66 means that 66% of the CWNS
sample earned a raw score at or below that value.
• Roid (1989) – percentile ranks converted into
standard scores.
Diagnostic Accuracy
• Sensitivity = .87
• Specificity = .84
• Positive Predictive Value = .79
Psychometrics
• Non-normal distributions require different
approaches to creating standard scores
• Fit index scores to means, standard deviations,
skewness and kurtosis across age groups.
• Applies to stuttering, aphasia, dysarthria,
apraxia and other communication disorders