INTERNATIONAL STUDY AND
LANGUAGE INSTITUTE
WHAT IS 'EAP SPEAKING',
EXACTLY?
Revisiting an EAP Speaking test, the
‘TEEP Speaking’
Bruce Howell, University of Reading
EALTA 2016
1
Copyright University of Reading
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
EAP SPEAKING MEANS…
•presentations
•seminar discussions
•tutorial discussions
(maybe)
Conference theme
Assessment of what....? - Revisiting the issue of construct(s)
2
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
TEEP SPEAKING IS …
• not ‘TEAP’!
3
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
TEEP SPEAKING
• TEEP = Test of English for Educational Purposes
• test of EAP (English for Academic Purposes)
• ‘exit test’ on pre-sessional EAP courses at University
of Reading – i.e. ‘entrance test’ for degree study
• 1000~1500 candidates per year (incl. some external)
• a new Speaking component introduced in 2010
• two-phase structure:
• monologue
• dialogue
4
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
TEEP SPEAKING - FORMAT
• 25-minute timeslot (23 minutes + 2 mins’ turnaround)
• 2 candidates, interlocutor, assessor
• scripted, timed
• recorded
5
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
PLANNING TIME: 1 MINUTE
phrased simply, to
avoid construct
irrelevance
6
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
PLANNING TIME: 4 MINUTES
+ MONOLOGUE: 3 MINUTES
EACH
originally the
instructions asked
for a “presentation”
7
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
PLANNING TIME: 2 MINUTES
+ DISCUSSION: 4 MINUTES
representative
of a ‘seminar’ (?)
8
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
DISCUSSION : 2 MINUTES
more of an open ‘intellectual discussion’
we hope students normally have such discussions!!
9
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
TEEP SPEAKING – STAGES
stage of test
what happens
what EAP construct?
1 minute to think about
focus question +
1
the general topic + 4
roles (planning time)
mins to plan own topic
study skills – preparing,
making notes
3 minutes each to
each candidate’s talk
2
introduce viewpoints, as
(monologue)
a role play
simulated ‘presentation’
scenario
3
(planning time)
2 minutes to understand
a specific situation
study skills – preparing,
making notes, combining
ideas
4 minutes for an analysis
and discussion
simulated ‘seminar’
4
discussion about
scenario (dialogue)
return to focus question:
more general
simulated ‘intellectual
5
2 minutes to discuss the
discussion (dialogue)
discussion’
‘bigger picture’ question
10
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
TEEP SPEAKING – VALUES
• test created to operationalise values underpinning
existing (very established!) teaching and materials
• fundamental aim = replicate the experiences of
speaking as a student in English
• test should be based on teaching materials/syllabus
• topic-based approach – suitably challenging and
motivating 'academic' topics
• ∴ planning time essential
• to feature both ‘explaining’ and ‘discussing’
• dyad pairing for genuine interaction; management of
speaking considered vital (e.g. initiating, clarifying)11
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
TEEP SPEAKING – GRADING
performance in specific tasks
- more ‘EAP-like’
Global criteria, given by both interlocutor and assessor:
• Explaining ideas and information
• Interaction
Analytical criteria, given by assessor only:
• Fluency
apparently tending
• Accuracy, range
towards representing
• Intelligibility
‘general’ proficiency
… also linked to values
Weighting = global 50% / analytical 50%
12
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
TEEP SPEAKING = AN ‘EAP
SPEAKING TEST’
• do you what ‘EAP Speaking’ is?
• does your local university have a definition?
• in European universities, more course using EMI
(Colman 2012; Dimova et al 2015)
• so 'EAP Speaking' has increasing importance in
Europe (and elsewhere)
(right?)
I have to be satisfied my EAP Speaking
test really is ‘EAP’
13
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
QUESTIONS TO ASK
OURSELVES
easy ones:
• Are we happy that our test matches our syllabus?
• Are we happy that our test is ‘EAP’?
slightly more difficult questions (from ‘EALTA Guidelines
for Good Practice in Language testing and Assessment’):
• How clearly is/are the test purpose(s) specified?
• Is there a description of the test taker?
• Are the constructs intended to underlie the
test/subtest(s) specified?
14
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
REVISITING THE CONSTRUCT
DEFINITION
Bachman & Palmer (2010:43):
“… we can define the construct from a number
of perspectives, including everything from the
content of a particular part of a language
course to a needs analysis of the components
of language ability that may be required to
perform language use tasks in the target
language domain, to a theoretical model of
language ability.”
15
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
SO, WE’VE GOT SOME WORK
TO DO
• Frameworks such as Weir's (2005) define ‘Speaking’
• but where “… tasks … are designed to collect evidence
in a small sub-set of contexts … test purpose should
drive the definition of the construct” (Fulcher 2003)
• So, what aspects are relevant to ‘EAP Speaking’??
What does it include??
Douglas (1997:1): “ … international students … need to be able to
speak proficiently to accomplish many tasks and purposes. First,
they need basic interpersonal communication skills (BICS) to get
around campus, buy books and materials in the bookstore, get
meals in cafeterias and restaurants, ask a friend about a homework
16
assignment … “
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
PART OF EAP SPEAKING?
17
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
ONE WAY OF LOOKING AT IT:
BALEAP CAN-DO FRAMEWORK
18
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
BALEAP CAN-DO
FRAMEWORK (SPEAKING)
•
•
•
•
•
•
•
•
•
•
•
•
•
examples of the 50+ descriptors:
that’s
S1.3.1 Demonstrate high level presentation skills
interesting!
S1.2.1 Take part in group discussion (and socializing)
S1.2.4 Debate and communicate/share thoughts or feelings
S1.2.3 Engage fully in discussion rather than providing superficial contribution
S2.1.1 Ask for clarification
S1.1.5 Take part in group work analysing and solving problems
S1.1.2 Work independently as well as collaboratively
S1.1.1 Demonstrate critical thinking
‘academic
S1.1.6 Tell other people when they are wrong
culture’ – part
S1.4.5 Engage in ‘Socration’ dialogue with supervisors
of EAP (?)
S1.4.8 Challenge a lecturer
S1.1.8 Introduce cultural insights
19
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
MORE QUESTIONS
(NOT MANY ANSWERS YET)
• is our ‘ingrained’ definition of EAP speaking valid?
• have things changed?
• how many relevant sub-skills does TEEP Speaking
already feature?
• are any omissions justifiable for practical reasons?
• we need to defining the ‘academic quality’ of topics
• how to express scores or profiles
• “the overall score is meaningless” (O’Sullivan 2012)
• have to extract the ‘EAP’ elements from CEFR, in
order to make an equivalence claim
20
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT
REFERENCES
• Bachman, L. and Palmer, A. (2010) Language Assessment in Practice.
Oxford: Oxford University Press.
• Colman, J. (2012) in Doiz, A., Lasagabaster, D., Sierra, J. M. (Eds.) EnglishMedium Instruction at Universities: Global Challenges. Multilingual Matters
• Douglas, D. (1997) Testing Speaking Ability in Academic Contexts:
Theoretical Considerations TOEFL Report MS-08 ETS available at
https://www.ets.org/research/policy_research_reports/publications/repo
rt/1997/icia
• Dimova, S. , Hultgren, A. and Jensen, C. (Eds.) (2015) English-Medium
Instruction in European Higher Education. English in Europe, Volume 3.
Berlin, Boston: De Gruyter Mouton.
• EALTA (2006) EALTA Guidelines for Good Practice in Language testing and
Assessment available at http://www.ealta.eu.org/guidelines.htm
• Fulcher, G. (2003). Testing Second Language Speaking London:
Longman/Pearson Education.
• Weir, C. J. (2005) Language Testing and Validation: An evidence-based
21
approach Houndgrave: Palgrave-Macmillan
LIMITLESS POTENTIAL | LIMITLESS OPPORTUNITIES | LIMITLESS IMPACT