When might good teaching practice become a testing problem? Diane Schmitt Nottingham Trent University Task authenticity “Authenticity is the degree of correspondence of the characteristics of a given language test task to the features of a TLU task” (Bachman and Palmer, 1996:23) Authenticity Characteristics of the TLU tasks Characteristics of the test task Describe the target language use domain • What are the characteristics of the type(s) of writing students do at university? • Demonstrate knowledge, understanding or applicability of disciplinary content • Timed exams – 1-3 hours • Coursework essays – work done at home/library • Essay task set by lecturer • Guidance on reading may be given, source selection is up to student • Normally requires references to external sources • 2000+ words in length • Plagiarism heavily penalised “Authenticity… • Provides a means for investigating the extent to which score interpretations generalize beyond performance on the test to language use in the TLU domain” (Bachman and Palmer, 1996: 24). Generalisability “If the test tasks reflect real-life tasks in terms of important identified conditions and operations it is easier to state what a student can do through the medium of English…unless steps are taken to identify and incorporate such features it would seem imprudent to make statements about a candidate’s ability to function in normal conditions in his/her future target situation” (Weir, 1993:28 in Shaw and Weir, 2007: 63). Generalisability “If the test tasks reflect real-life tasks in terms of important identified conditions and operations it is easier to state what a student can do through the medium of English…unless steps are taken to identify and incorporate such features it would seem imprudent to make statements about a candidate’s ability to function in normal conditions in his/her future target situation” (Weir, 1993:28 in Shaw and Weir, 2007: 63). Context validity (Weir, 2005) • Context validity takes account not just of the linguistic content of an assessment task, but also the social and cultural contexts in which the task is performed. • purpose • content knowledge • setting • linguistic demands • time available • writer/reader relationship • text length (input/output) • marking criteria Pre-sessional students’ writing experience What’s the longest piece of writing you have ever done in English? IELTS essays (250-300 words) x 11 TOEFL essays (250-300 words) x 2 1000 word final year dissertation x 1 1000 word essay for a friend x 1 Have you ever had to use sources in your writing before? Yes x 2 No x 13 The teaching task At the end of this course, students should be able to… Write increasingly longer essays – 1500-2000 words Select, read, understand and synthesize source material Integrate relevant information and ideas into essays Develop a coherent organisational structure and stick to it Correctly use in-text citation Correctly format a bibliography Ensure that language use effectively conveys ideas and does not obscure meaning Effectively proofread and self-edit A recent thread on the BALEAP listserve • At university, language skills are used in an integrated way, e.g. students listen to lectures and take notes, discuss and refine their ideas with peers, increase their understanding through reading, notetaking and summarizing and provide evidence of their learning through written and oral work. However, on many EAP courses, these language skills are timetabled and taught separately, possibly to facilitate different teachers sharing classes. • To what extent do we risk under-preparing students for their university experience if we separate out the skills? • Olwyn A common approach to writing • An set of integrated classes which culminate in students having to write a long-source based essay (and possibly a lead a seminar and/or give a presentation). These aim to simulate the university environment and “scaffold” the task of writing an academic essay. • Cazden (1983) defined a scaffold as “a temporary framework for construction in progress” (p. 6). • In Vygotsky’s words, “what the [learner] is able to do in collaboration today he will be able to do independently tomorrow” (Vygotsky, 1987, p. 211). • (I confess, I got these last two quotes off Wikipedia!) The pre-sessional at NTU • 21 contact hours per week • 9 hours of integrated skills – leading towards writing one source-based essay on a shared topic • Assessment • 50% coursework essay • 25% coursework presentation • 25% listening/reading exam Blurring the boundary between teaching and assessment Teaching to the test? Source texts are the basis for reading instruction so substantial comprehension work is done Virtually all writing work is related to this one assignment Writing process is staged with students getting both written and tutorials feedback along the way Is the picture any clearer elsewhere? • 44 BALEAP members responded to a survey • 32 use a coursework essay in their final assessment for progression • The essays’ weightings range from 17.5 – 80% of the final mark • 12/17 of these use the same essay to teach essay writing as they do to assess essay writing • Students commonly write only 1 source-based essay during their course (Course lengths range from 4 – 10 weeks) • The amount of support provided is considerable although it varies from programme to programme “The essay is given a lot of weight because the vast majority of our students will have • to write long, take home essays on their future programmes.” Painting by numbers Questions?
© Copyright 2026 Paperzz