What characterises the B2/C1 levels?
Anthony Green
© Copyright ALTE 2015
Manual for Language Test
Development and Examining
Chapter 2
2.7 Key questions
Who decided that there should be a test —what can they
tell you about its purpose and use?
What will the educational and social impact of the test be?
What type and level of language performance needs to be
assessed?
What type of test tasks are necessary to achieve this?
2
© Copyright ALTE 2015
Where to start?
In [insert name of school/company/district/country]
learners must demonstrate their language competence at
[B2/C1] level in order to…
•enter/graduate from university
•get a visa/ citizenship
•qualify as a [insert name of professional level job]
3
© Copyright ALTE 2015
The measurement metaphor
Body temperature 25°C – 45°C
Domestic oven 50°C – 250°C
Air in Bergen -50°C – 10°C
4
© Copyright ALTE 2015
K2
Senior childcare administrator
Undergraduate student of marketing
Production manager
Hotel concierge
5
C2
C1
B2
B1
A2
A1
IELTS 6.5
PTE-G Level 2
TOEIC 225
© Copyright ALTE 2015
Where to start?
Proficient
Independent
Basic
6
C2
C1
B2
B1
A2
A1
© Copyright ALTE 2015
B2
1. effective argument: account for and sustain his opinions in discussion by providing relevant
explanations, arguments and comments; explain a viewpoint on a topical issue giving the advantages
and disadvantages of various options; develop an argument giving reasons in support of or against a
particular point of view; take an active part in informal discussion in familiar contexts, commenting,
putting point of view clearly, evaluating alternative proposals and making and responding to
hypotheses.
2. hold one’s own in social discourse: e.g. understand in detail what is said to him/her in the
standard spoken language even in a noisy environment; initiate discourse, take his/her turn when
appropriate and end conversation when he/she needs to, though he/she may not always do this
elegantly; interact with a degree of fluency and spontaneity that makes regular interaction with native
speakers quite possible without imposing strain on either party.
3. a new degree of language awareness: correct mistakes if they have led to
misunderstandings; make a note of “favourite mistakes” and consciously monitor speech for it/them;
generally correct slips and errors if he/she becomes conscious of them.
7
© Copyright ALTE 2015
B2+
represents a strong B2 performance. The focus on argument, effective social discourse and on
language awareness which appears at B2 continues.
However, the focus on argument and social discourse can also be interpreted as a new focus on
discourse skills.
This new degree of discourse competence shows itself in conversational management (cooperating strategies): give feedback on and follow up statements and inferences by other speakers
and so help the development of the discussion; relate own contribution skilfully to those of other
speakers.
It is also apparent in relation to coherence/cohesion: use a variety of linking words efficiently to
mark clearly the relationships between ideas; develop an argument systematically with appropriate
highlighting of significant points, and relevant supporting detail.
8
© Copyright ALTE 2015
C1
A broad range of language, which allows fluent, spontaneous communication, as
illustrated by the following examples:
Can express him/herself fluently and spontaneously, almost effortlessly. Has a good command of a
broad lexical repertoire allowing gaps to be readily overcome with circumlocutions. There is little
obvious searching for expressions or avoidance strategies; only a conceptually difficult subject can
hinder a natural, smooth flow of language.
The discourse skills appearing at B2+ are more evident at C1, with an emphasis on
more fluency, for example:
select a suitable phrase from a fluent repertoire of discourse functions to preface his remarks in order
to get the floor, or to gain time and keep it whilst thinking; produce clear, smoothly flowing, wellstructured speech, showing controlled use of organisational patterns, connectors and cohesive
devices.
9
© Copyright ALTE 2015
Measurement concerns
 Vague formulation
 Local norms
 Interlinguistic variation
 Teacher judgement of Can Do descriptor
difficulty does not equate to empirical task
difficulty
 Equivalence
10
© Copyright ALTE 2015
Degrees of precision
 Can understand a wide range of demanding,
longer texts, and recognise implicit meaning.
 Can understand the main ideas of complex text
on both concrete and abstract topics, including
technical discussions in his/her field of
specialisation.
11
© Copyright ALTE 2015
Teacher judgements
B2 Can take a series of follow up questions with a degree of fluency
and spontaneity which poses no strain for either him/herself or the
audience.
Choices presented to teachers:
2. Could be expected to perform this activity provided that circumstances are
favourable, for example if he/she has some time to think about what to say,
or the interlocutor is tolerant and prepared to help out.
3. Could be expected to perform this activity without support in normal
circumstances.
4. Could be expected to perform this activity even in difficult circumstances,
for example when in a surprising situation or when talking to a less
cooperative interlocutor.
12
© Copyright ALTE 2015
Progression in the CEFR


13
Progress is not merely a question of moving up a vertical
scale. p.26
The set of descriptive categories introduced in Chapters
4 and 5 map out a ‘horizontal dimension’ made up of
parameters of communicative activity and
communicative language competence. … just the
addition of domain, for example, would give a third
dimension turning… a grid into a notional cube. A full
diagrammatic representation of the degree of
multidimensionality involved would in fact be very
challenging, if not impossible. p.16
© Copyright ALTE 2015
Test equivalence
What does linking different tests to the CEFR imply?
Equating (results essentially interchangeable)
Equivalent forms of same test: built to the same specifications; equal in difficulty
Calibration
Tests designed to measure same content. Different levels of difficulty or reliability.
Prediction
Tests not assumed to be measuring same construct : different task types, different
conditions, Regression used to predict scores on one test from the other. Requires same
test takers for both tests.
Moderation (statistical or social)
Tests not assumed to be measuring same construct, but considered desirable to
compare scores. Tests administered to different groups of students – in statistical
moderation, links made via third measure, or via score distribution.
Linn, R. L. (1993). Linking results of distinct assessments. Applied Measurement in
Education, 6, 83-102.
14
© Copyright ALTE 2015
The Common European Framework of Reference for Languages
Chapter 4
15
© Copyright ALTE 2015
Illustrative scales
three perspectives on communicative language use
Competences
Linguistic
Sociolinguistic
Activities
Pragmatic
Discourse
Flexibility
16
Taking the
Floor
(Turntaking)
Thematic
Development
Strategies
Functional
Coherence
Propositional
Precision
Spoken
Fluency
© Copyright ALTE 2015
Which competences?
Syntactic sophistication
B2 Good grammatical control.
Adverbial subordinate clauses with -ing
Talking about spare time, I think we should go to the museum.
Pseudo-cleft with wh- as subject
What fascinated me was that I was able to lie on the sea surface.
C1 Consistently maintains a high degree of grammatical accuracy.
Subject to object raising with assume
The low cost was assumed to be an advantage.
Might for permission
Might I tell you what we discuss?
17
© Copyright ALTE 2015
Which competences?
A rich vocabulary?
B2 Has a good range of vocabulary for matters connected to his field and most
general topics?
C1 Has a good command of a very broad lexical repertoire including idiomatic
expressions and colloquialisms
New words
B2
B2
B2
joy
dynamic
bored to death
C1
C1
C1
distress
keen
before you know it
Established words: new meanings
B2
B2
B2
B2
18
Let’s take a chance.
I swallowed a bone.
Dare to dance!
junk food
C1
C1
C1
C1
Let’s take this matter further.
Small firms are swallowed up by multinationals.
Dare Dave to dance!
junk mail
© Copyright ALTE 2015
Which competences?
Phonological control
C1 Can vary intonation and place sentence stress correctly in order to
express finer shades of meaning.
B2 Has a clear, natural, pronunciation and intonation.
Orthographic control
C1 Layout, paragraphing and punctuation are consistent and helpful.
Spelling is accurate, apart from occasional slips of the pen.
B2 Can produce clearly intelligible continuous writing, which follows
standard layout and paragraphing conventions.
Spelling and punctuation are reasonably accurate but may show
signs of mother tongue influence.
19
© Copyright ALTE 2015
Which activities?
Creative writing
C1 Can write clear, detailed, well-structured and developed
descriptions and imaginative texts in an assured, personal, natural
style appropriate to the reader in mind.
B2 Can write clear, detailed descriptions of real or imaginary events
and experiences marking the relationship between ideas in clear
connected text, and following established conventions of the
genre concerned.
Can write clear, detailed descriptions on a variety of subjects
related to his/her field of interest.
Can write a review of a film, book or play.
20
© Copyright ALTE 2015
Which activities?






completing forms and questionnaires;
writing articles for magazines, newspapers, newsletters,
etc.;
producing posters for display;
writing reports, memoranda, etc.;
making notes for future reference;
taking down messages from dictation, etc.
Listed as activities in the CEFR, but no illustrative scales
provided
21
© Copyright ALTE 2015
Domains
spheres of action or areas of concern in which social life is
organised




22
the personal domain, in which the person concerned lives as a
private individual, centred on home life with family and friends, and
engages in individual practices such as reading for pleasure,
keeping a personal diary, pursuing a special interest or hobby, etc.;
the public domain, in which the person concerned acts as a
member of the general public, or of some organisation, and is
engaged in transactions of various kinds for a variety of purposes;
the occupational domain, in which the person concerned is
engaged in his or her job or profession;
the educational domain, in which the person concerned is engaged
in organised learning, especially (but not necessarily) within an
educational institution.
© Copyright ALTE 2015
Communication themes
topics which are the focus of attention in particular
communicative acts
1. personal identification
5. travel
2. house and home, environment
6. relations with other people
3. daily life
7. health and body care
4. free time, entertainment
8. education
4.1 leisure
9. shopping
4.2 hobbies and interests
10. food and drink
4.3 radio and TV
11. services
4.4 cinema, theatre, concert, etc.
4.5 exhibitions, museums, etc.
4.6 intellectual and artistic pursuits
4.7 sports
12. places
13. language
14. weather
4.8 press
23
© Copyright ALTE 2015
Situations
external situations which arise may be described in terms of…
24
1.
the locations in which, and the times at which, they occur;
2.
the institutions – the structure and procedures of which control much
of what can normally occur;
3.
the persons involved, especially in their relevant social roles in relation
to the user/learner;
4.
the objects (animate and inanimate) in the environment;
5.
the events that take place;
6.
the operations performed by the persons involved;
7.
the texts encountered within the situation
© Copyright ALTE 2015
Domain
Locations
Institutions
Persons
Objects
Events
Operations
Texts
Home: : house, rooms, garden
own
of family
of friends
of strangers
Own space in hostel, hotel
The countryside, seaside, etc.
The Family
Social networks
(Grand)Parents, Offspring,
Siblings, Aunts, Uncles,
Cousins,
In-laws, Spouses
Intimates, Friends,
Acquaintances
Furnishing & furniture
Clothing
Household equipment
Toys, tools, personal hygiene
Objets d'art, Books, Pets, Wild
/domestic animals,
Trees, Plants, Lawn, Ponds,
Household goods, Handbags,
leisure/sports equipment
Family occasions
Encounters
Incidents, accidents,
Natural phenomena
Parties, visits
Walking, cycling, motoring
Holidays, excursions
Sports events
Living routines
(dressing, undressing
cooking, eating, washing, etc.)
DIY, gardening
Reading, Radio & TV
Entertaining
Hobbies
Games & sports
Teletext
Guarantees
Recipes
Instructional material
Novels, magazines,
Newspapers
Junk mail
Brochures
Personal letters
Broadcast and recorded
spoken texts
Public spaces:
street, square, park, etc.
Public transport
Shops (super)markets
Hospitals, surgeries, clinic
Sports stadia, fields, halls
Theatre, cinema,
entertainment
Restaurant, pub, hotel
Places of worship
Public authorities
Political bodies
The law
Public Health
Services clubs
Societies
Political parties
Denominations
Members of the Public
Officials
Shop personnel
Police, army, security
Drivers, conductors,
Passengers
Players, fans, spectators
Actors, audiences
Waiters, barpersons
Receptionists
Priests, Congregation
Money, purse, wallet
Forms,
Goods
Weapons
Rucksacks
Cases, Grips
Balls
Programmes
Meals, Drinks, Snacks
Passports, Licences
Incidents
Accidents, Illnesses
Public meetings
Law-suits, Court trials
Rag-days, Fines, Arrests
Matches, contests
Performances
Weddings, Funerals
Buying and obtaining public
services
Using medical services
Journeys by road/rails/ship/air
Public entertainment and
leisure activities
Religious services
Public announcements and
notices
Labels & packaging
Leaflets, Graffiti
Tickets, Timetables
Notices, Regulations
Programmes
Contracts
Menus
Sacred texts
Sermons, Hymns
Offices
Factories
Workshops
Ports, railways
Farms
Airports
Stores, shops, etc.
Service industries
Hotels
Firms
Civil Service
Multinational
Corporations
Nationalised
industries
Trade Unions
Employers/ees
Managers
Colleagues
Subordinates
Workmates
Clients
Customers
Receptionists, Secretaries
Cleaners, etc.
Business machinery
Industrial machinery
Industrial & craft tools
Meetings
Interviews
Receptions
Conferences
Trade fairs
Consultations
Seasonal sales
Industrial accidents
Industrial disputes
Business admin. Industrial
management
Production operations
Office procedures
Trucking
Sales operations
Selling, marketing
Computer operation
Works office Maintenance
Business letter
Report Memorandum
Life & safety notices
Instructional manuals
Regulations
Advertising material
Labelling & packaging
Job description
Sign posting
Visiting cards, etc.
Schools: Hall
Classrooms, Playground,
Sports fields, corridors
Colleges
Universities
Lecture Theatres
Seminar rooms
Student Union
Halls of Residence
Laboratories
Canteen
School
College
University
Learned
societies
Professional
Institutions
Adult education
bodies
Class teachers
Teaching staff
Caretakers
Assistant staff
Parents
Classmates
Professors, lecturers
(Fellow) Students
Library & laboratory staff
Refectory staff, cleaners
Porters, Secretaries, etc.
Writing material
School uniforms
Games equipment & clothing
Food
Audio-visual equipment
Black-board & chalk
Computers
Briefcases & School bags
Return to school/entry
Breaking up
Visits and Exchanges
Parents' days /
evenings
Sports days, Matches
Disciplinary problems
Assembly
Lessons
Games
Playtime
Clubs & societies
Lectures, Essay writing
Laboratory work
Library work
Seminars & tutorials
homework
Debates & discussions
Authentic texts (as above)
Textbooks, Readers
Reference books
Blackboard text
OP text
Computer screen text
Videotext
Exercise materials
Journal articles
Abstracts
Dictionaries
Situations
Educational
Occupational Public
Personal
Table 5. External contexts of use
25
© Copyright ALTE 2015
Situations
Table 5. External contexts of use
Occupational
Domain Locations
Offices
Factories
Workshops
Ports, railways
Farms
Airports
Stores, shops, etc.
Service industries
Hotels
26
Institutions
Firms
Civil service
Multinational corporations
Nationalised industries
Trade unions
Persons
Employers/ees
Managers
Colleagues
Subordinates
Workmates
Clients
Customers
Receptionists, Secretaries
Cleaners, etc.
© Copyright ALTE 2015
Situations
Table 5. External contexts of use
Objects
Business machinery
Industrial machinery
Industrial & craft
tools
27
Events
Meetings
Interviews
Receptions
Conferences
Trade fairs
Consultations
Seasonal sales
Industrial
accidents
Industrial disputes
Operations
Business admin.
Industrial
management
Production
operations
Office procedures
Trucking
Sales operations
Selling, marketing
Computer operation
Works office
Maintenance
Texts
Business letter
Report Memorandum
Life & safety notices
Instructional manuals
Regulations
Advertising material
Labelling & packaging
Job description
Sign posting
Visiting cards, etc.
© Copyright ALTE 2015
Multiple scales, multiple dimensions
simple syntax
complex syntax
frequent lexis
infrequent lexis
familiar genre
unfamiliar genre
concrete topic
abstract topic
known topic
standard
non-standard
context neutral
context specific
straightforward
involved
implicit information
explicit information
28
unknown topic
© Copyright ALTE 2015
Conditions and constraints
imposed by external conditions under which communication occurs
Physical conditions:
a) for speech:
clarity of pronunciation;
ambient noise (trains, aircraft, ‘static’, etc.);
interference (crowded street, markets,
pubs, parties, discos, etc.);
distortions (poor telephone lines, radio
reception, public address systems);
weather conditions (wind, extreme cold,
etc.).
b) for writing:
poor reproduction of print;
difficult handwriting;
poor lighting, etc.
29
Social conditions:
number and familiarity of interlocutors;
relative status of participants;
presence/absence of audience;
social relationships between participants.
Time pressures:
different pressures for speaker/listener and
writer/reader;
preparation time for speeches, reports, etc.;
limitations on time allowed for turns and
interactions (e.g. by rules, expense,
competing events and commitments, etc.);
Other pressures:
financial; anxiety-producing situations, etc.
© Copyright ALTE 2015
Relating test tasks to target language use situations
Task characteristics (Bachman and Palmer, 2010, pp.66-68)
Setting
Input/ Expected Response
physical characteristics
format: channel, language, length etc.
participants
language: organisational
time of task
(grammatical, textual),
Rubric
instructions
structure
pragmatic etc.
topical
Relationship between Input and Response
time allotment
type of external interactiveness
recording method
scope of relationship
directness of relationship
30
© Copyright ALTE 2015
CEFR Grids
i)











31
Task input/prompt
Language of input/prompt
CEFR level of input/prompt
Time permitted or suggested for this task
Control/guidance
Content
Genre
Rhetorical function(s) of input
Imagined audience
Mode of input/prompt
Topic or theme of input
Integration of skills for input
ii)
Response (description of written
response elicited by the prompt(s)/input)










Number of words expected
Rhetorical function(s) expected
Text purpose
Register
Domain
Grammatical competence expected
Lexical competence expected
Discoursal competence expected
Authenticity: situational
Authenticity: interactional…
© Copyright ALTE 2015
The map metaphor
32
© Copyright ALTE 2015
Where to start?
[the CEFR] views users and learners of a language primarily as
‘social agents’, i.e. members of society who have tasks (not
exclusively language-related) to accomplish in a given set of
circumstances, in a specific environment and within a particular
field of action
33
© Copyright ALTE 2015
Can I predict the domains in which my learners will operate and the
situations which they will have to deal with? If so, what roles will they have
to play?
What sort of people will they have to deal with?
What will be their personal or professional relations in what institutional
frameworks?
What objects will they need to refer to?
What tasks will they have to accomplish?
What themes will they need to handle?
Will they have to speak, or simply listen and read with understanding?
What sort of things will they be listening to or reading?
Under what conditions will they have to act?
What knowledge of the world or of another culture will they need to call
on?
What skills will they need to have developed? How can they still be
themselves without being misinterpreted?
(p.44)
34
© Copyright ALTE 2015
Communicating at work
some common requirements (p.62)
As temporary residents learners should be able to:
35


seek work permits etc. as required;


read employment advertisements;


understand and follow joining procedures;

etc.
enquire (e.g. from employment agencies) about the nature, availability and
conditions of employment (e.g. job description, pay, laws of work, free
time and holidays, length of notice);
write letters of application and attend interviews giving written or spoken
information about own personal data, qualifications and experience and
answer questions about them;
understand and ask questions concerning the tasks to be performed on
starting work;
© Copyright ALTE 2015
British Council/ EAQUALS
core inventory
A tool for developing teaching and learning activities (and/or
assessments) from real-world task types: business meeting
36
© Copyright ALTE 2015
37
© Copyright ALTE 2015
38
© Copyright ALTE 2015
The BC/EAQUALS core inventory
The CEFR-based scenario concept offers a means to:
Work top down from real world contexts to identify the competences needed to
be successful in them, plus criteria to judge that success. Both the
communicative language activities listed and the aspects of competence
needed to be successful in them suggest classroom activities. These in turn
suggest specific lessons or lesson sequences, with self-study and project
tasks.
[Offers a means to] develop a tasks specification for a test, with the
assessment procedure, assessment conditions, marking and grading
procedures etc. specified on the second page.
North, B., Ortega, A., & Sheehan, S. (2010). A core inventory for general
English. British Council/EAQUALS.
39
© Copyright ALTE 2015
Where to start?
Linking of a test to the CEFR cannot be valid unless
the examination or test that is the subject of the linking
can demonstrate validity in its own right.
Council of Europe (2009). Relating Language Examinations to the Common
European Framework of Reference for Languages: Learning, Teaching,
Assessment (CEFR). A Manual. Strasbourg: Language Policy Division.
Validity: the degree to which evidence and theory
support the interpretations of test scores entailed by
proposed uses of tests
40
© Copyright ALTE 2015