Faculteit Letteren & Wijsbegeerte Hannah Janssens
Using the dictogloss procedure to improve the retention and use of L2 academic English formulaic sequences Masterproef voorgedragen tot het behalen van de graad van Master in de Meertalige Communicatie Academiejaar: 2015 – 2016 Promotor Prof. Dr. June Eyckmans Vakgroep Vertalen Tolken Communicatie 2 3 PREFACE
The process of writing a master’s thesis has acquired a certain reputation of being the most
difficult task of one’s university education. Upon writing this note of thanks, I can confirm
this statement. Perseverance and overcoming your fear of failure is essential to succeed, but
this is not always so simple, as one would expect. There were some moments that I doubted
myself or lacked motivation. Knowing that I am easily inclined to procrastinate, I feel a sense
of pride upon presenting this paper. It has been a period of intense learning for me, not only
because I extended my knowledge about second language acquisition, but also because it has
taught me a lot on a personal level. I would like to reflect on the people without whom I
would not have reached this final point. They helped and/or supported me throughout this
period in different sorts of ways:
Professor June Eyckmans, who introduced me to the subject, sparked my interest and
shared her knowledge about the dictogloss method. She provided feedback and valuable
guidance when needed and was always willing to answer questions when I had some doubts.
Ms Alexandra Rosiers, a GOLLD researcher active in the domain of foreign language
acquisition and language pedagogy. She was so kind as to make time for the dictogloss
experiment during the writing course she teaches.
My family, who love me unconditionally and support me no matter what. I would like
to thank my parents for their sympathetic ear. They gave me that extra bit of encouragement
when I needed it most: as the deadline approached. My sister Eva was able to give me some
practical advice. As she is writing her doctoral dissertation at the Free University of Brussels
(VUB), she has some experience with writing high-quality academic texts. Finally, my
brother Jesse often made me smile (without even trying) when I had an off day.
My dear friends, who have played a major role in my life and without whom I would
not have become the same person I am now. We have been friends for quite some time now
and no matter where we will be in future stages of our lives, I am convinced that our
friendship will be a lasting one. I would like to thank Alexandra Koljaj in particular, because
she is the closest to me and she always has my back. Talking about things other than my
paper or having a catch up with a glass of wine made a welcome change.
4 Nicolas Knudde, an engineering student at Ghent University. I have only known him
for seven months and even though he is currently on an Erasmus exchange in Stockholm, he
was still very much present in my life. We talked daily and he was a prop and stay to me.
Although his research domain is quite different from mine, he was writing his master’s thesis
as well and it still felt like we went through the same. Whenever I was down, he tried to
motivate me or make me laugh and he succeeded in doing just that.
Last but not least, I would like to thank you, the reader, for your interest and for
taking the time to digest my paper.
Hannah Janssens
22 May 2016
5 TABLE OF CONTENTS
1. INTRODUCTION .......................................................................................................................... 6 2. THEORETICAL FRAMEWORK ............................................................................................... 10 2.1 Formulaic sequences ..................................................................................................................... 10 2.2 The dictogloss method .................................................................................................................. 13 2.2.1 What is dictogloss? .................................................................................................................................... 13 2.2.2 The four phases ........................................................................................................................................... 14 2.2.3 The benefits of dictogloss ....................................................................................................................... 15 2.3 (C)LREs: (critical) language related episodes ....................................................................... 16 3. METHODOLOGY ........................................................................................................................ 17 3.1 Participants ....................................................................................................................................... 17 3.2 Materials ............................................................................................................................................ 17 3.3 Data collection procedure ............................................................................................................ 21 3.4 Data analysis procedure ............................................................................................................... 23 4. RESULTS ...................................................................................................................................... 25 4.1 Pretest, pretreatment, immediate posttest and delayed posttest .................................. 25 4.2 Participant feedback ...................................................................................................................... 28 5. CONCLUSION AND DISCUSSION ........................................................................................... 29 5.1 Possible explanations for the lack of effect ........................................................................... 29 5.2 A more general interpretation of the results ........................................................................ 33 5.3 The potential of future research ............................................................................................... 35 6. BIBLIOGRAPHY ......................................................................................................................... 36 7. APPENDICES .............................................................................................................................. 38 Appendix (1): pretreatment worksheet – comparison group ................................................. 39 Appendix (2): pretreatment worksheet – experimental group ............................................. 40 Appendix (3): dictogloss treatment (three readings) – experimental group .................... 41 Appendix (4): feedback form – identical for both groups ........................................................ 42 6 1. INTRODUCTION
Some people might mistakenly think that learning a language is a matter of learning lists of
isolated words by heart. Obviously, this is not the case and all kinds of things enter into
mastering a language, such as grammar, morphology, spelling, phonology, syntax,
pragmatics et cetera. Nonetheless, lexis remains crucial. When learning vocabulary second
language (L2) learners should not simply focus on learning single words but they should also
know how to combine these single words in an idiomatic way. By means of using formulaic
sequences, i.e. multi-word units, learners can become more proficient in a foreign language.
This holds true for both oral and written usage.
Expanding one’s stock of words can happen in a few different ways. Direct, intentional
teaching of vocabulary is one of them (Nation, 2001). This way of explicit, intentional
learning is often seen as something opposed to learning vocabulary via guessing from context
(Kelly, 1990, cited in Nation, 2001, p. 232). The latter is an example of incidental learning of
vocabulary, since the student mainly focuses his or her attention on the general message of
the text while reading or listening. Nation (2001, p. 232) feels that the opposition some
people make between intentional and incidental learning is unfortunate and unjust. According
to him they simply constitute two different ways of processing and learning vocabulary,
which are complementary activities because they each enhance the learning that comes from
the other. In his opinion, direct teaching of vocabulary should not occupy more than 25% in a
well-designed language learning programme.
As mentioned before, intentional teaching of L2 vocabulary can be effective and this may
also be true for formulaic sequences (FSs). FSs are ubiquitous (Schmitt & Carter, 2004) both
in everyday language and in English for academic purposes (EAP). A word-centred
conceptualisation of vocabulary has resulted in FSs seldom being taught in an appropriate
manner nor being tested as part of overall vocabulary knowledge, which is unfortunate
knowing that a large proportion of any discourse consists of FSs. Erman and Warren (2000)
calculated that FSs of various types constituted 58.6% of the spoken English discourse they
analysed and 52.3% of the written discourse (cited in Schmitt & Carter, 2004, p. 1). The
word-centred conceptualisation has also meant that there has been little research on the
7 instruction of formulaic sequences and the effect some pedagogical procedures or exercise
types might have in their acquisition (Alali & Schmitt, 2012, p. 154). Few are known to be
effective. Some researchers (Haywood & Jones, 2004; Cortes, 2006; Li & Schmitt, 2009;
Pauwels & Peters, 2015) found that explicit teaching of targeted academic FSs was effective,
especially with regard to later recognition rather than autonomous use in later writing (cited
in Lindstromberg et al., 2016, p. 13). Seeing that language teachers’ ultimate goal is to pass
on their knowledge to others and eventually improve students’ level, it is important teachers
do not neglect multi-word units. In fact they should do the opposite because L2 students who
can not only understand FSs but also know how to accurately and appropriately use them are
more likely to be judged by others as being relatively proficient (Lindstromberg et al., 2016,
p. 12). This is true for oral (Boers et al. 2006) and written discourse (Lewis, 2008; Ohlrogge,
2009 as cited in Alali & Schmitt, 2012, p. 154).
In that context a few questions come to mind: Can we develop a method that helps language
learners to remember formulaic sequences better? If so, what does the method exactly look
like? Which factors can contribute to improved retention of those formulaic sequences? Is it
best to work individually or collaboratively and why is that? Can mastery ever truly be
achieved?
For quite some time now, methodologists have recommended small group work (including
pair work) in the second language classroom (Long & Porter, 1985, p. 207), because it gives
peers a chance to interact, which serves as an attractive alternative to the traditional teacherfronted classroom interaction (Doughty & Pica, 1985). The key element of interaction is
negotiation defined by Pica (1992, p. 200) as an activity in which interlocutors work
linguistically together to resolve the communication difficulty identified by one of the
interlocutors. Moreover, negotiation has an impact on comprehension: it helps to make
meaning more comprehensible for L2 learners (Pica et al., 1987, cited in Nabei, 1996, p. 59).
Consequently, negotiation is believed to be a useful context for language learning (Nabei,
1996, p. 2). Some goal-oriented classroom activities incorporating negotiation such as the
information gap, dictogloss, jigsaw and problem-solving stimulate L2 learners to exchange
information and communicate for the purpose of a meaningful outcome (Nunan, 1989, p. 10).
My master’s thesis will focus on comparing two versions (standard vs. modified) of one
collaborative production task type, the so-called “dictogloss”. Moreover, I would like to
8 contribute my bit towards helping to bridge the gap in the relatively limited research
concerning the dictogloss method as a means of contributing to the acquisition of academic
formulaic sequences.
We will conduct our research by building on the scientific research that has already been
done. We refer to the research by Lindstromberg and his colleagues in 2016. They carried out
a scientific experiment that we will attempt to replicate in order to check its reliability. They
made a comparison between the standard dictogloss and a modified version. The latter was
developed to direct the attention of the learners to the FSs. In this study, we will make use of
another modification and in that sense it is not an exact replication. The specific differences
between both experimental investigations will be discussed in more detail in the chapter on
methodology. Participants were divided into a comparison group (CGr) and an experimental
group (EGr). The latter group received a modified version of the exercise, which ultimately
resulted in them achieving better results, i.e. they used more FSs during a delayed posttest.
As the modified dictogloss showed promise, we wanted to optimise the method so that the
EGr would not only perform better, but the contrast between the EGr and the CGr would
become stronger as well. Hence, we wanted to attract learners’ attention to the FSs in two
different ways instead of only one: a similar way and a new way. This meant we made a
double modification to the modified version of the previous experiment.
With all this in mind, the aim of the present study is to assess the effectiveness of the
dictogloss as a collaborative task type in order to teach/learn academic formulaic sequences.
The general research question is as follows:
When relatively proficient learners of academic English do a dictogloss and their attention is
directed to the forms of targeted phrases in two different ways (on the one hand by
highlighting the initials of the FSs typographically and instructing students to finish the
words’ onsets during a pretreatment and on the other hand by handing out a worksheet on
which the FSs were in part already given and instructing students to complete them during
the dictogloss exercise itself), does this make a difference in later uncued production of those
phrases?
9 Given that we are interested in the difference between the modified version and the standard
dictogloss, this raises several more specific questions:
1. Will the experimental group make a longer reconstruction, i.e. use more words,
than the comparison group?
2. Does the experimental group use more formulaic sequences than the comparison
group in their attempt at reconstructing the text?
In order to answer these questions we formulated the following hypothesis: participants of the
modified version of the dictogloss will achieve better results. In other words, we expect them
to use more formulaic sequences as well as more words than the participants of the
comparison group.
This paper is organised as follows: in the introduction we briefly touch upon the terms
“formulaic sequences” and “dictogloss”. Next, we enter into more detail in the theoretical
framework and reviewed the literature on the subject matter. We tried to define formulaic
sequences and dictogloss in sections 2.1 and 2.2 respectively. In section 2.3 we talk about
LREs. A LRE is defined by Jackson (2001, p. 298) as any part of a dialogue where language
learners use metatalk. We collected different scientific sources that we found on the Internet,
especially by searching Google Scholar or consulting the websites www.unicat.be and
www.jstor.org, in the library of our Faculty of Arts and Philosophy (Ghent University) and in
the library of a university in Brussels (VUB). Chapter 3 is subdivided as well and concerns
the methodology used for this study. The results are set forth and discussed in chapter 4. We
try to give an answer to the research question propounded in the introduction and check
whether or not the hypothesis has been confirmed. Chapter 5 summarises the results in the
conclusion and comprises the discussions of our findings. Finally, the bibliography makes up
chapter 6, followed by a few appendices in chapter 7.
10 2. THEORETICAL FRAMEWORK
2.1 Formulaic sequences
Formulaic sequences are omnipresent in almost all types of discourse. Research by Howarth
(1998), Erman & Warren (2000) and Foster (2001) suggested that at least one-third to onehalf of a language is composed of formulaic elements (cited in Conklin & Schmitt, 2008, p.
72). Yet, it is not surprising that their results varied to some degree. Still, we can assume that
proficient language users store a number of prefabricated chunks in their long-term memory,
which they can access whenever they want to use these ‘ready made’ expressions (Conklin &
Schmitt, 2008, p.75). This is intuitive, so note that the above claims are made by assertion
seeing that little research has been done to substantiate those claims. In spite of this
assumption we know of no study that has directly attempted to quantify how many FSs one
should know to be viewed as proficient. The widespread existence of formulaicity in many
languages allows multilingual learners to transfer the meaning of FSs across their native
language and other foreign languages they learn (Conklin & Schmitt, 2008, p.74).
Words that constitute a formulaic sequence are not simply words that ‘belong together’,
because this description is too vague and formulaic sequences are much more than strings of
words linked together with collocational ties. FSs are multi-word expressions that occur as
phrases and as coherent semantic units at a relatively high frequency (Jiang & Nekrasova,
2007, p. 433). According to Halliday (1966) this implies that learners may be able to predict
words that co-occur “with a probability greater than chance” (cited in Barfield & Henrik,
2009, p. 4). An example of a FS is “on the other hand” and this FS expresses the single
meaning of “conversely”. Sometimes FSs are also linked to a specific pragmatic function.
Their high frequency distinguishes FSs from less frequent phrases, such as “on the wall”
(Jiang & Nekrasova, 2007, p. 433). As from when can we speak of “a high frequency of
occurrence”? Obviously, determining which multi-word expressions belong to the category
of high frequency words/the category of FSs is something quite subjective. Take the case of
idioms for example. They are an example of multi-word units that express a single meaning.
Yet, some consider them FSs (Schmitt & Carter, 2004) while others do not. Jiang &
Nekrasova (2007, p. 433), for example, use “formulas” as a synonym for “formulaic
11 sequences”, after which they go on to oppose formulas against idioms. By combining the
meanings of all the words that compose a formula, the overall meaning can be derived, but
this is not the case for idioms. In order to be able to correctly use idioms, such as “beat
around the bush” or “kick the bucket”, one should learn their meanings by heart. In addition
to that, are there any other criteria that determine which multi-word expressions are
considered FSs and which are not? Moon (1997, p. 44) claims that institutionalisation, noncompositionality and fixedness are other key characteristics of what she calls “multi-word
items”.
Overall, it is clear that it is difficult to define formulaic sequences because of their diversity
(Schmitt & Carter, 2004, p. 3). Firstly, the length of FSs is variable. They can be short, long
or anything in between. Secondly, they can be used for a variety of different purposes, such
as to express sympathy (“I am (very) sorry to hear about …”), to make a request (“Would
you be kind enough/so kind as to …”), to comply with a request (“I would be happy/glad to
…”), to exchange specific information very fast and in a precise way in order to avoid
communication misunderstandings (“Wind 28 at 7” = this phrase is used in aviation language
to state that the wind is 7 knots per hour from 280 degrees) and so on. In other words they are
often linked to recurrent communication needs and consequently this gives them considerable
pragmatic utility (Concklin & Schmitt, 2008, p. 73). Thirdly, some FSs are totally fixed
(“ladies and gentlemen”), while others have a number of slots that need to be filled with (an)
appropriate word(s) (“[someone/something, usually with authority] made it plain that
[something as yet unrealised was intended or desired]”) (Schmitt & Carter, 2004, p. 3).
Besides carrying a denotative meaning and realising a pragmatic function, FSs are often also
associated with semantic prosody. This semantic prosody reflects the speaker/writer’s
positive or negative attitude or evaluation. An example of a positive semantic prosody would
be FSs that include the word “provide”, such as “provide information” or “provide services”
(Concklin & Schmitt, p. 74).
Seeing that FSs are so diverse, it does not come as a surprise that there is no unanimity about
how to define formulaic sequences and there is not even unanimity about the denomination of
the phenomenon. Wray (2002, p. 9) discovered over 50 terms to describe the concept of
formulaic language.
12 Some examples are listed:
• chunks
• collocations
• conventionalised forms
• formulaic speech
• formulas
• holophrases
• multi-word units
• prefabricated routines
• ready mate utterances
• multi-word items
For our research we have decided upon the term “formulaic sequence”, defined by Wray as
follows:
A sequence, continuous or discontinuous, of words or other elements, which is, or
appears to be, prefabricated: that is, stored and retrieved whole from memory at the
time of use, rather than being subject to generation or analysis by the language
grammar. (Wray, 2002, p. 9)
Ultimately, the following question comes to mind: how do language users process formulaic
sequences? We refer to the holistic hypothesis that posits that FSs are stored, processed and
retrieved holistically. This is consistent with the definition provided above and with the
research of Jiang & Nekrasova (2007). Moreover, formulaic language has processing
advantages over equivalent nonformulaic language. Conklin & Schmitt (2008) carried out a
study and their main aim was to explore the commonly asserted and widely accepted notion
that FSs are processed more quickly and easily than creatively generated language. The
results of their study provide significant evidence that confirms this widespread belief, at
least for reading. However, to date there is no direct evidence supporting this belief for
auditory processing. In addition, the results of the study by Conklin & Schmitt (2008, p. 84)
showed that the processing benefit was observed for both natives and nonnatives, even
though the L1 English speakers processed the targeted FSs even faster. This is not surprising
since people are generally most fluent in their mother tongue. Still, it is especially good to
hear that the processing advantage is the case for L2 speakers as well, because it is common
knowledge that people who are not brought up bilingual who want to master a second
language will generally run up against more difficulties and having productive knowledge of
FSs (when this includes knowing when and where to use them) is a skill that is generally
considered to come quite late in the acquisition process.
13 2.2 The dictogloss method
2.2.1 What is dictogloss?
The concept of the dictogloss is simple: learners listen to a passage and jot down key words.
They are asked to write down as much as possible and in the correct order. Next, language
learners are motivated to work together in small groups and create a reconstructed version of
the text they have previously heard and taken notes on (Smith, 2012, p. 70). The
reconstructed text will unavoidably not be an exact replica of the original version of the text,
but it should be a linguistically acceptable text similar in content and style (Murray, 1994, p.
69). Dictogloss differs from pure dictation because of the interaction and collaboration. The
task uses negotiation, which is believed to be useful in the context of second language
acquisition (SLA) (Nabei, 1996).
Dictogloss is an example of a collaborative production task type that promotes ‘focus-onform’ (FonF). Other task types include jigsaw and text reconstruction. Alegría de la Colina
and García Mayo carried out a study in 2007 in which they compared the three abovementioned task types to see which one was the most effective in promoting FonF. FonF,
thought to occur in in the context of meaning-focused language use, has been contrasted with
‘focus-on-forms’ procedures (Lindstromberg et al., 2016, p. 14). Both instructional
approaches can be used to teach grammar or vocabulary. In the latter case, FonF attends to
lexical items (single words as well as multi-word units) within a communicative task
environment, whereas FonFs teaches lexical items within non-communicative and nonauthentic language tasks (Laufer, 2006, p. 150). This can be tied in with Ellis’s view (2001).
According to him the distinction is related to how a language learner perceives oneself and
the language. In the case of FonF the student views oneself as a language user who is of the
opinion that language is a tool for communication. In the case of FonFs on the other hand, the
student views oneself as a learner of a language and considers the language as an object of
study (cited in Laufer, 2006, p. 150).
In this study, we are particularly interested in the learners’ internalisation and retention of
academic FSs and dictogloss is suitable for this purpose. According to Lantolf & Appel,
(1994, cited in Alegría de la Colina & García Mayo, 2007, p. 91) knowledge is constructed
14 through interaction and then internalised. In other words language is viewed as a social and
cognitive activity and both aspects are essential to reach complete understanding. Dictogloss
tasks are traditionally used at a relatively advanced level of English, because learners need
sufficient linguistic resources to be able to take part in the small-group discussions about
language (Murray, 1994, p. 69). Moreover, it is fairly easy to check whether or not learners
will use the targeted FSs in later writing, because the method promotes the production of
language, i.e. ‘pushed output’. We assume the knowledge amassed by learners will not be
limited solely to later recognition.
2.2.2 The four phases
Dictogloss was first presented by Ruth Wajnryb in 1990. The task traditionally follows a
four-stage procedure when used in a classroom:
Phase 1: Warm up/preparation: A topical warm-up prepares learners for the subject of the
text that they are about to hear. This is advisable because people listen more attentively when
they are able to anticipate what they are about to hear and ideally their interest has been
attracted as well. Then, learners are prepared for some of the vocabulary (Wajnryb, 1990, p.
7). Lastly, students are informed of the rationale of the exercise
Phase 2: Dictation (three readings): A text is read multiples times at normal spoken speed and
the instructor makes short pauses between the sentences. The pace should be comparable to
that of a news broadcast on television or on the radio and the readings should be as identical
as possible. Wajnryb (1990, p. 5) says the text should be read twice. The first time learners
listen and focus on the overall meaning of the text while allowing the words to “wash over
them”. The second time simultaneous note-taking is allowed. Smith (2012) is an advocate of
a third reading to help leaners to expand their notes. When taking notes, students should be
encouraged to write down words that will help them reconstruct the text. Typically, those
words will be content or information words. Grammar of function words such as “the”, “his”,
“and”, by contrast, are not likely to serve as memory triggers. When a student has written
down enough content words, he or she will be able to complete the text with function words.
15 Note that in the previous experiment by Lindstromberg et al. (2016) the text was read out
four times instead of three times like in our experiment. The first and second reading were
identical (listening followed by note-taking from memory) and the third and fourth reading
were identical to each other as well (simultaneous note-taking).
Phase 3: Reconstruction: The third phase is the most collaborative phase in which students
pool their notes, negotiate what is correct, edit their notes and work together on their version
of the text. Students are compelled to engage in metatalk (Swain, 1998, cited in Lim &
Jacobs, 2001, p. 4) and strive to reconstruct a version with correct grammar, content and
textual cohesion. Each group of students produces a reconstructed text from their shared
resources and one of the student in each group functions as a scribe (Wajnryb, 1990, p. 5).
Depending on the teacher’s objective, the reconstruction may focus on replication or
similarity of meaning (Smith, 2012, p. 70). When the focus is on content, the students can use
their own words to reformulate the same meaning.
Note that in our experiment each student handed in their own version of the text at the end of
the three readings. Between each reading learners relied on their own fragmentary notes first,
but if they whished to perfect their reconstructions they were allowed to consult a fellow
student.
Phase 4: Analysis and correction: During the last phase, students participate in a class
discussion: they analyse and correct their texts. Under the guidance of their teacher, students
are encouraged to discuss the language choices they made and compare the various versions
of the other groups with their own and the original version. According to Tedick (2001)
students not only discussed grammatical issues, but focused on orthographic and semantic
aspects as well (cited in Smith, 2012, p. 73). This concurs with the three types of LREs
(Language Related Episodes) discussed further below.
2.2.3 The benefits of dictogloss
Dictogloss lends itself to this study because of its benefits to language learners and teachers
alike.
16 A few are listed below (Smith, 2012, p. 74-75):
! Students are encouraged to reflect consciously on the language they produce (FonF)
and this may lead to greater retention, an opinion that concurs with research done by
Kowal and Swain (1997) and by Tedick (2001).
! Students are encouraged to think critically.
! Dictogloss leads to synchronous interaction, meaning that more speakers speak more
often. They may also overcome their reluctance to speak and interact because they are
so preoccupied with succeeding in reconstructing the text through cooperative
learning.
! Dictogloss can be used as alternative assessment, such as peer-assessment and selfassessment.
! Dictogloss promotes communicative competence through the use of all four skills
(reading, listening, writing and speaking).
! As knowledge is shared, dictogloss is suitable for students with a different level of
proficiency. They can learn from each other. However, they should be able to
participate in the conversations about language and need sufficient linguistic
resources in order to do so (Murray, 1994, p. 69).
2.3 (C)LREs: (critical) language related episodes
We focus a bit more on the third phase, i.e. the reconstruction phase. There are plenty of
CRLEs to be found in the interaction between students during the dictogloss exercise. The
term “CLREs”, sometimes also called “LREs”, is an abbreviation for (Critical) LanguageRelated Episodes. LREs are utterances in which language is the focus of discussion. Students
talk more or less explicitly about how the English language works, i.e. they engage in
metatalk, and question their own or their interlocutor’s language use (Alegría de la Colina &
García Mayo, 2007). An LRE begins when a linguistic problem is identified and ends when
the discussion is completed. Kowal and Swain (1994) make a distinction between three major
categories of LREs: meaning based episodes, grammatical episodes and orthographic
episodes (cited in Nabei, 1996, p. 63). Not all these three categories are present in other
collaborative task types as well. In the case of the jig-saw exercise, for example, only
meaning based communication is required (p. 71). Overall, we may say that language
17 learners benefit from LREs because language use and language learning take place at the
same time (Alegría de la Colina & García Mayo, 2007, p. 91).
3. METHODOLOGY
3.1 Participants
The participants in this experiment were students majoring in English and another foreign
language in the Department of Translation, Interpreting and Communication at Ghent
University in Belgium. Their additional language was either Spanish, German, Italian or
Russian and their native language was Dutch, except for two Erasmus students. Their level of
proficiency was estimated as C1 (CEF), which corresponds to an IELTS score of 7-8. They
all enrolled on an English writing course, which focused on academic writing, and more
specifically abstract writing. Since the participants were third year students, the course can be
helpful for them when they are writing their own abstract for their paper in order to obtain
their Bachelor of Arts in Applied Language Studies. Thirty-nine students participated in the
first session, which consisted of a pretest, a pretreatment, the dictogloss exercise, the
immediate posttest (i.e. the first individual text reconstruction) and a moment for them to
provide feedback. During the second session, which included the delayed posttest, two
participants were absent meaning thirty-seven students participated. The participants were
randomly assigned to one of the two groups: an experimental group (EGr) that experienced
the modified dictogloss and a comparison group (CGr) that experienced the exercise
following the standard dictogloss method. The division was as follows: nEGr = 20, nCGr =
19 in the first session and nEGr = 19, nCGr = 18 in the second session.
3.2 Materials
For the dictogloss we used an abstract of 152 words by Yar (2005) containing 18 FSs, as seen
just below this paragraph. The 18 FSs of interest are relatively evenly spread throughout the
text and are in bold. During the pretest, however, the text was free of any kind of
highlighting. During the pretreatment, on the other hand, both groups (CGr & EGr) were
given the initials of the targeted FSs, shown in two typographically different ways.
18 Intellectual property1 (IP) crime remains a marginal topic2 in sociological and
criminological investigation. This neglect is due in part to3 the perception that such offences
are ‘non-serious’ and/or ‘victimless’. This paper sets out to challenge4 such assumptions by
examining a particular instance of5 intellectual property crime, namely the counterfeiting of
dangerous goods, and in particular6 the counterfeiting of automotive components.7 It is
argued that8 such activities, now globally widespread9, carry significant economic costs10,
and pose substantial risks to11 public health and safety. Some of the key drivers of12 this
trade are analysed, along with recent developments in13 law, policing, crime control14 and
technological innovation that aim to curtail the counterfeiting of these and other dangerous
counterfeit products15. It is argued that intellectual property crime comprises a socially
significant16 and sociologically challenging phenomenon17, one that deserves concerted
attention18 from those working in the sociology of crime, law, health, risk, technology and
political economy.
One of the conditions that need to be fulfilled for the dictogloss procedure to succeed is that
the text used should be short and dense (Wainryb, 1990, p. 5). Logic suggests that the human
brain can easily remember a small amount of text, which also means it becomes harder the
longer the text is. Since my master’s thesis is concerned with EAP, an abstract seemed an
obvious choice. An abstract serves as a summary of the content for a paper or an article and
thus tells a more or less complete story in only a few sentences. Researchers often skim
abstracts to verify whether that particular one could be relevant to their own research and if
that is not the case they will disregard it. In that respect it is highly important for the writer of
an abstract to make sure that the information density is high. The length of an abstract is quite
variable depending on the discipline, but normally it ranges from 100 to 500 words. The
abstract chosen for our research was in line with the average length of abstracts in many
fields, hence it seemed suitable to serve as a good example of the genre. Even though the
subject of the abstract, namely sociology, was outside the participants their field of study,
their level of competence was estimated high enough to comprehend the text. When choosing
the FSs, we targeted the ones that were at least relatively fixed. Moreover, we selected FSs
that, although they appeared in a text about sociology, quite possibly can be used in other
contexts as well, because most of them have a more general meaning.
As mentioned briefly above, during the pre-treatment each student received a worksheet.
Even though both the CGr and the EGr had basically the same task, i.e. adding the missing
ends of the words, the format for the two groups was different. The initials were shown in a
typographically different manner: a version with more highlighting for the EGr versus a
rather traditional look for the CGr. In accordance with our hypothesis, this means minimised
19 highlighting and attention direction for the CGr. With the intention of drawing the attention
of the students in the EGr to the FSs, the initials were printed in bold, and were bunched
together to emphasize their close association. The type size was also slightly bigger. The
CGr, on the other hand, simply had to fill in the blanks. In spite of their intention, the
researcher never explicitly said that the participants had to pay more attention to these FSs
than to rest of the text. After all, they had to reconstruct the entire text. The text was triple
spaced in both formats of the worksheet, so as to make sure the students had enough space to
complete the words. Both formats of the worksheet can be found in appendices (1) and (2).
During the dictogloss itself, i.e. the three readings, every student belonging to the CGr
received a blank sheet of paper on which they wrote down as much as possible while
experiencing the standard dictogloss procedure. Conversely, the members of the EGr were
given a worksheet on which the FSs of interest were partly already there. In that regard, the
participants who experienced the CGr treatment were at a disadvantage. The EGr members
still had to complete the FSs, but were given more than just the initials. The latter was the
case during the pre-treatment in both groups. Now the FSs were centred on the page,
appeared in chronological order and each FS was shown on its own line. The 18 FSs are
given just beneath this paragraph. The slashes indicate where we pressed enter on the
keyboard, in other words they represent where the line breaks were on the worksheet.
However, neither the numbering, nor the slashes appeared on the actual pages. In order to
form a better idea of what the worksheet looked like, it is included in appendix (3).
intellectual property1 / a marginal topic2 / is due in part to3 / this paper sets out to challenge4 /
a particular instance of5 / and in particular6 / automotive components7 / it is argued that8 /
globally widespread9 / economic costs10 / pose substantial risks to11 / the key drivers of12 /
recent developments in13 / crime control14 / counterfeit products15 / socially significant16 /
challenging phenomenon17 / concerted attention18
During the immediate posttest and the delayed posttest, the participants were given a blank
sheet on which they tried to reconstruct the entire text. Seeing that the delayed posttest took
place a week later, a plausible scenario would be that participants would have forgotten the
text to a large extent by then. To help them remember the content of the original abstract, a
translation in their first language (L1) was provided before the start of the delayed posttest.
This was also a good idea in terms of making sure the delayed reconstruction was a good way
of testing the participants’ productive memory of forms and retention of the FSs rather than
20 their reproduction of content. When the delayed posttest started, the L1 translation had
already disappeared again, to avoid that the exercise would turn into a translation exercise.
As was the case for the preceding experiment by Lindstromberg et al. (2016), which we base
our research on, the Dutch translation was also projected briefly onto a large screen just after
the students indicated unknown words. By doing so, we saw to it that all the participants
could better understand the text. The translation in the participants’ mother tongue is shown
just below. The Dutch equivalents of the targeted English FSs are printed in bold and the
English FSs are bracketed. Please note that there was no form of highlighting present in the
translation as the participants saw it. Moreover, the interpolated English expressions were
deleted from the L1 translation.
Schending van intellectuele eigendomsrechten1 [intellectual property] is nog steeds een
nauwelijks behandeld thema2 [a marginal topic] binnen sociologisch en criminologisch
onderzoek. Deze nalatigheid is deels te wijten aan3 [is due in part to] de visie dat zulke
overtredingen niet ernstig zijn en/of geen slachtoffers veroorzaken. In dit artikel zullen deze
veronderstellingen worden betwist4 [this paper sets out to challenge] aan de hand van het
bestuderen van een specifieke soort van5 [a particular instance of] intellectuele
eigendomsrechtenschending, namelijk het vervalsen van gevaarlijke stoffen, in het
bijzonder6 [and in particular] het vervalsen van motorvoertuigonderdelen7 [automotive
components]. We zullen betogen dat8 [it is argued that] zulke activiteiten, die ondertussen
wereldwijd in gebruik9 [globally widespread] zijn, aanzienlijke economische schade10
[economic costs] met zich meebrengen en een belangrijk gevaar vormen voor11 [pose
substantial risks to] de volksgezondheid en de veiligheid. Een aantal sleutelfactoren van12
[the key drivers of] deze handel zullen worden belicht, samen met nieuwe ontwikkelingen
in13 [recent developments in] het recht, de wetshandhaving, de misdaadbestrijding14 [crime
control] en de technologische innovatie, die erop gericht zijn het vervalsen van deze en
andere gevaarlijke namaakproducten15 [counterfeit products] te beperken. We besluiten dat
de schending van intellectuele eigendomsrechten een maatschappelijk relevant16 [socially
significant] en sociologisch uitdagend verschijnsel17 [challenging phenomenon] behelst, één
dat de gezamenlijke aandacht18 [concerted attention] verdient van wetenschappers die zich
bezighouden met misdaad, de wet, de volksgezondheid, risico-analyse, technologie en
politieke economie.
Some of the original English multi-word units include words that have Dutch equivalents that
are (partly) similar in form, e.g. “intellectual” and “intellectueel” or “economic” and
“economisch”. However, we opted for an alternative translation for words that resembled
each other whenever possible, e.g. “nieuwe ontwikkelingen” rather than “recente
ontwikkelingen” for “recent developments”. We did this because L1 FSs that are formally
21 similar to their English equivalents can cue guessing or prompt students’ recall too directly
and that was not our intention.
3.3 Data collection procedure
Given that we base our research on a previous experiment by Lindstromberg et al. (2016), the
procedure followed for collecting data was for the most part similar to the procedure they
used, yet not identical. In particular the pre-treatment was an additional step, which was not
included in their experiment.
Prior to the actual experiment, the first session started off with a short introduction on
abstract writing: students recapitulated what an abstract is, what its function is and they were
presented with a definition. The instructor also explained that the exercise could be helpful
for them to enhance their own abstract writing skills. After that, the actual experiment began,
consisting of five major steps. The first step of the experiment was the so-called pretest. All
students read the entire English text and were asked to underline all the words or word
combinations they did not know already, which gave us an idea of their familiarity with the
vocabulary. Then, the sheets of paper were collected and the Dutch translation was displayed
on the screen. After a few minutes of reading through it, the L1 translation disappeared again.
Next, the students were divided at random in two different groups: the comparison (standard
dictogloss) group and the experimental (modified dictogloss) group. However, please note
that both groups did the exercise simultaneously and in the same classroom. Since the total
number of participants was an odd number, the division was not entirely equal, yet almost.
Even though it was not clarified why they were divided in different groups and the researcher
did not explain why they received different sheets of paper, the participants might have had
an inkling.
The second step during the first session of the experiment was the pretreatment. All the
participants were given a worksheet with initials, but these sheets were somewhat different as
mentioned earlier. The abstract was divided in two, marked by a double forward slash in the
middle. The first half of the text was read out loud and students were instructed not to write
until the researcher reached the slash marks. After that, students picked up their pen or pencil
and tried to recover from their memory all the words for which they were only given the first
22 letter. Shortly thereafter, the researcher continued with the second half of the text, following
the same procedure, whereupon the students handed in their sheet of paper.
The dictogloss treatment itself (standard or modified) functions as the third step and consist
of three phases. The students remained split up into the same two groups. The CGr provided
with a blank sheet of paper, while the EGr received the worksheet with parts of the FSs down
the middle, as described above. The instructor read the text aloud a first time, while the
participants were instructed not to write simultaneously. When the instructor finished
reading, the participants were given some time to jot down all the words or word
combinations they could remember. The second reading was identical to the first one, i.e.
listening followed by note-taking from memory. The last reading differs from the previous
two, because now the participants were also allowed to take notes while the text was read out.
Moreover, in between the three readings, students were also allowed to pair up and turn to the
other person’s notes, but only after they first relied on their own notes. Ideally they worked
together in groups of two, which was not always possible because of odd numbers, so in
some cases groups of three were formed. When they had finished their reconstructions, all the
participants’ copies were collected. Then, the original English text (without any kind of
highlighting) was briefly projected on the screen so as to give participants a chance to form
an idea of how they performed.
Before the participants moved on to the fourth step of the first session’s experiment, they
were invited to walk around the classroom, stretch, wave their arms et cetera for
approximately two and a half minutes. After that, they sat down again for the immediate
posttest, but not right next to each other to avoid they could copy from their neighbour.
Participants of both groups were handed a blank sheet of paper on which they attempted to
rewrite the full text on their own as precisely as they could, after which they handed in their
sheet.
The elicitation of feedback was the fifth and final major step of the first session. The
participants could give their opinion openly, yet anonymously on three questions. First they
were asked to rate the exercise overall regarding its potential helpfulness in learning English
academic phrases on a scale of 1 – 7, with 1 being the lowest and 7 being the highest. To
answer the second question, they could express their feelings, if they had any, about a
specific part or parts of the exercise. Lastly, they were asked whether or not they considered
the exercise they just did one that has good potential to promote learning of any other
23 aspect(s) of English and they could make a suggestion. In order to make sure the anonymous
feedback forms could be linked to one of the two groups, participants were asked to write
down either “gaps” for the pretreatment worksheet with the gaps and the lines (= CGr) or
“initials” for the version with the initials in bold (= EGr). A copy of the feedback form can be
found in appendix (4). At the end of the first session the students were thanked for their
cooperation and not a single word was mentioned about the fact that a second session would
follow. This was a conscious decision, because we wanted to avoid that the participants
would intentionally try to remember phrases and/or other parts of the text.
The turnout during the second session was satisfying: only two students were absent, one
belonging to the CGr and one to the EGr. The second session took place exactly one week
later, because this allowed us to work out whether the participants had learned something. As
was the case before, we were particularly interested in how much phrases they recalled and
the number of words used for their reconstruction. If you want to measure the effect that both
types of the dictogloss had, an immediate test is not sufficient. Two and a half minutes after
the dictogloss (standard or modified) is rather fast, which means participants will rely heavily
on their short-term memory. Only a week later it is plausible to assume that the information is
processed and consolidated, which means the participants will have to depend on their longterm memory. During the second session the Dutch translation was displayed on a screen for
one minute followed by a final test. Showing the L1 translation again helped to refresh the
participants’ memory and recall the gist of the abstract. During the delayed posttest
participants were once again asked to reconstruct as much of the text as possible. They did
this individually and were provided with a blank sheet to write their version of the abstract
on. As always, the worksheets were collected.
3.4 Data analysis procedure
At that moment in time, the collected data were ready to be read into the computer, processed
and analysed. A Microsoft Excel file helped us to keep all the information neatly organised
and came in handy when making calculations. Since we were mostly interested to see
whether the results of the comparison group and the experimental group would diverge, the
one group was always contrasted with the other.
For the pretest, the number of words each participant was not familiar with was simply
counted, whereupon the average for each group was determined. Note that the total number
24 of unknown words only took into account different words. The repetition of the word
“counterfeiting”, for example, was counted as one unknown word. Similarly, words that had
the same stem, such as “counterfeit” and “counterfeiting” were considered as one unfamiliar
word as well. In the case of word combinations such as “key drivers” and “intellectual
property” third year university students are expected to know those individual words. By
underlining those words, they meant they were not familiar with the word combination as a
whole and therefore the word combinations were considered as one unknown word. After all,
FSs are multi-word strings that behave as single units: they realise a single meaning and/or
function.
The layout of the worksheets for the pretreatment, immediate posttest and the delayed
posttest were practically the same. The worksheet of our Excel file for the pre-treatment
differed slightly, because it did not include the category “total number of written words” as it
was not applicable. During the pre-treatment participants were not instructed to reconstruct
the whole text, but solely complete the end of the words. To determine the length of the
abstract in words, the words were counted manually and all individual words written by
participants were counted, regardless of their correctness or aptness. All participants also
achieved a score that reflected how many phrased they had used in total. By means of the
binary number system, each student was assigned a score for each FS. In other words, if the
FS was not present or unacceptable this would correspond to a score of 0, whereas 1 meant
that the FS was correct. Nonetheless, spelling mistakes were allowed since we reasoned that
the participant succeeded in remembering the FS, which was after all the main objective, but
simply did not know how to spell the word(s) in question. The scores per participant were
added up and this resulted in a total, with 18 being the highest score possible. As was the case
for the total number of written words, the average was calculated per group.
Remember that participants experienced either one of the treatments (standard versus
modified) simultaneously while they were in the same classroom. Also recall that in order to
preserve the participants’ anonymity, they were simply asked to indicate if they belonged to
the CGr or the EGr by writing “version with gaps” in the former case and “ version with
initials” in the latter. In that way, we would be able to compare the ratings of both groups
with each other. Unfortunately, it was found that the feedback forms of the two groups
became mixed together, in the sense that too much students mistakenly claimed that they
belonged to the group that received the version with the initials (printed in bold), i.e. the EGr.
25 We assumed this happened because the term “initials” probably caused some confusion. It so
happens that both pre-treatment worksheets used initials, even though only students who
were given a page with initials in bold were instructed to write down “initials”.
However, this problem could be solved. First of all, we assumed that students who wrote
“gaps” on their worksheet were indeed right about that. Now it was only a matter of detecting
the few students who wrongfully wrote “initials” among the students who rightfully did so.
After the experiment ended during the second session, i.e. a week after the students had
provided feedback, they were asked to come to the front of the classroom and the problem
was briefly explained to them. They all agreed to confirm which rating form was theirs on
condition of anonymity. So as to avoid we would lose much time by doing this, we tried to
link up each feedback form to the corresponding student beforehand. We did this by looking
at the sheets of the pretreatment, the dictogloss itself and the immediate posttest, because
these were not anonymous. By focusing on the participants’ handwriting we were able to
connect each name to a feedback form. In some cases it was quite easy, because the
handwriting bore a strong resemblance, because it was characteristic and was very distinct
from the others or because some students wrote with pencil or wrote in a different colour of
ink. When their handwriting was very similar, by contrast, it was rather hard and timeconsuming. In some cases we had to guess which form belonged to which participant.
Naturally, we needed more confirmation for our suspicions, which is why we asked the
participants to simply confirm which feedback form was theirs. As a result, the problem was
solved and the participant feedback was still of use. Next, we calculated the mean ratings of
the exercise in terms of how helpful it could be in teaching and learning phrases. In addition,
we tried to report the contents of the participants’ answers to the two other questions in a
shortened form.
4. RESULTS
4.1 Pretest, pretreatment, immediate posttest and delayed posttest
With regard to the pretest, which served to determine whether both groups were similar
regarding the knowledge they had of the vocabulary used in the abstract. The average number
26 of unknown words amounted to 2.16 in the CGr and 2.70 in the EGr. On average, the
comparison group knew more words than the experimental group before the start of the
dictogloss exercise. Yet, the difference was rather small and thus not significant. From this
we could not conclude that the CGr had a headstart with regard to the knowledge of lexis.
Moreover, the data also revealed that the text was not difficult for the participants to
understand in terms of lexis.
The results of the pretreatment, immediate posttest and delayed posttest are summarised in
the following two tables and are expounded underneath them.
" Table 1:
Test
CGr
EGr
Raw mean
Group n, Mean (Mn), SD
Group n, Mean (Mn), SD
difference (MD)
Pretreatment
19n, 12.11Mn, 3.49SD
20n, 12.74Mn, 3.36SD
0.632MD
Immediate posttest
19n, 9.58Mn, 4.23SD
20n, 8.85Mn, 4.49SD
0.728MD
Delayed posttest
18n, 5.83Mn, 3.45SD
19n, 5,47Mn, 2.89SD
0.360MD
Table 1 shows the participants’ performance in terms of how many target phrases they
reproduced, with 18 being the maximum score possible. The students achieved the highest
score on the pretreatment, the second highest on the immediate posttest and the lowest on the
delayed test. This corresponds to the chronological order of the experiment and tied in with
what we expected. However, not everything we expected was confirmed. As illustrated in the
table above, the members of the CGr used more targeted FSs in both the immediate and
delayed posttest. Even though, the opposite was true for the pretreatment: the EGr
outperformed the CGr ever so slightly. Recall that our hypothesis was twofold. These results
do not support the first half of our hypothesis, which conjectured that EGr participants would
reproduce more FSs than CGr participants. When looking at the raw mean differences, it is
notable that there were only slight differences between the CGr and the EGr, namely <1.
Because the CGr outperformed the EGr ever so slightly during the delayed posttest and
because the mean differences were so small, it was unnecessary for us to calculate the
significance. The MD was also smaller in comparison to the experiment by Lindstromberg et
al. (2016) we based our research on. There the raw mean differences varied between 1 and 2
rounded off. Note that the standard deviation was higher in our experiment than in the
previous one during the delayed test. This means that the there were quite some differences
27 between the results of individuals: for some the exercise worked well and for others it did
not. We also have to take into account that there were two exchange students who
participated (one belonging to the CGr and one to the EGr). It is not surprising they generally
achieved a lower score, because all participants were shown the Dutch translation of the text
to jog their memory before the start of the delayed test. As exchange students do not know
the language, they could not benefit from this and had to rely entire on what they
remembered from the first session (one week ago). According to their teacher we may also
assume that their level of English is a bit lower than the level of the other students. This may
have an impact on their results as well.
" Table 2:
Test
CGr
EGr
Raw mean
Group n, Mean (Mn),
Group n, Mean (Mn),
difference (MD)
Range
Range
Immediate posttest
19n, 90.16Mn, 36-135
20n, 76.15Mn, 27-129
14.008MD
Delayed posttest
18n, 69.00Mn, 14-114
19n, 70.63Mn, 24-137
1.632MD
Table 2 features the total number of written words in both posttests. Recall that the original
abstract counted 152 words. All words written by students were counted, even when they
were not correct or appropriate. As table 2 shows, on average the CGr used more words than
the EGr in the immediate posttest, but this was not true for the delayed posttest, our prime
concern. In the latter case, the EGr performed better, but the mean difference was minimal
(1.632). By contrast, the raw mean difference for the immediate test amounted to 14.008. By
way of comparison: in the experiment by Lindstromberg et al. (2016), the mean differences
diverged less from each other (8.35 in the immediate test versus 12.76 in the delayed one).
Yet, we also have to take into account that they used a shorter text of 120 words. When
looking at the word range, the widest range was observed in the EGr during the delayed
posttest: the length of the abstract ranged between 24 and 137 words. Whereas the average
number of words used by students decreased after a week, i.e. compared to the immediate
test, the average range of words increased for both groups. All things considered, we cannot
conclude that the second half of our hypothesis, which postulated that the EGr would use
more words when reconstructing the text, was fully confirmed.
28 4.2 Participant feedback
At the end of the first session the participants were allowed to give feedback about the
exercise. The distribution of the ratings for the comparison group (n = 19) was as follows: 11
– 22 – 32 –
3.5
1 – 46 – 56 – 61 – 70, where each superscript represents a rating and the
corresponding frequency is written in normal font. One participant of the experimental group
(n = 19) had to leave class early, so both groups were even. The distribution was as follows:
1
2 – 24 – 32 – 45 –
4.5
1 – 55 – 60 – 70. For both groups there was not one modal class, seeing
that the ratings 4 and 5 were the most frequent scores. The mean rating for the CGr was 3.92
and 3.45 for the EGr.
When we look at the feedback and the ratings for both groups combined and compare them to
the ratings of the experiment by Lindstromberg et al. (2016, p. 18) we noticed a difference.
The median of the ratings was lower in our experiment, namely 4 for each of the two groups
of students. Knowing that the feedback forms of both groups became mixed by accident in
the preceding experiment, we only know that the mean was 5 for the CGr and the EGr
combined. In addition to that there was a difference that was even more noticeable. In their
experiment nearly 60% of the ratings were 5 or above and only 10% were 3 or less. In ours,
by contrast, 12 of the 38 (31,58%) of the ratings were 5 or above, while 13 of the 38
participants (34,21%) gave the exercise a rating of 3 or less. In other words, the percentage of
ratings that were 5 or more was not only lower than in the previous experiment, it was in fact
even lower than the percentage of people who rated the exercise a 3 or less.
These rather negative ratings were mainly reflected in their response to the first question,
which asked whether the participants had a particular feeling about any part(s) of the exercise
and if so which. Reoccurring comments were either about the immediate test being too
difficult or about the focus of the exercise being too much on participants’ short-term
memory. They felt it was only a useful exercise for people with a good short-term memory
and some even claimed they had a bad short-term memory themselves, including one student
of the EGr who commented that the exercise was almost impossible for him/her because of
an attention deficit disorder and another student who talked about experiencing a blackout
during the immediate posttest. The second question asked whether they thought the exercise
had good potential to promote learning of any other aspect(s) and if so which. 18 of the 38
participants (47,36%) responded they thought the exercise was likely to promote vocabulary
and/or multiword expression learning. This result was in line with the result of the previous
29 experiment where 45,76% answered the same. Some students thought the exercise could be
helpful to train your brain to memorise better in general and others thought it could improve
their listening skills. The former comments made up the second largest category, followed by
the latter comments about listening skills. All in all, we concluded that participants were, in
comparison with the previous experiment, generally less enthusiastic and sometimes even
negative about the dictogloss method.
5. CONCLUSION AND DISCUSSION
5.1 Possible explanations for the lack of effect (modified versus standard dictogloss)
The goal of this study has been to determine the effectiveness of the modified dictogloss
treatment versus the standard dictogloss. In order to do so, our main interest were the results
of the posttests and in particular those of the delayed posttest. We wanted to verify if the
modified treatment would be a better means of helping L2 EAP learners to recall and use L2
FSs that are conventional in AE. In summary, the data revealed that the CGr outperformed
the EGr in terms of how many FSs they reproduced, which contradicts our hypothesis.
Because the mean differences were very small and not significant, this also led us to
generalise that the treatment did not seem to produce an effect (neither the pretreatment, nor
the different versions (standard vs. modified) of the dictogloss treatment) on the results of the
delayed test. In other words, the CGr and the EGr performed equally as well/bad, both in
terms of the number of target phrases they reproduced and in terms of the total number of
words used for their reconstruction as well.
Our hypothesis predicted that participants belonging to the experimental group would use
more formulaic sequences and more words than the comparison group during the delayed
posttest. As our hypothesis was not confirmed, we looked into possible explanations as to
why the modified treatment did not work as expected. Please note that this list is not
exhaustive and there might as well be other explanations we have not thought of.
The first explanation is quite simple: when we compared the feedback of both groups, we
noticed that the mean rating the EGr gave was lower than the mean rating of the CGr. It goes
30 without saying that students who are less enthusiastic or did not see the point of the exercise,
will not do their absolute best and subsequently they will achieve a lower score.
With regard to the pretreatment we cannot compare our results to the experiment by
Lindstromberg et al. (2016) because their experiment did not include that particular extra
step. During the pretreatment the EGr outperformed the CGr ever so slightly, which was not
the case in the immediate or delayed posttest. However, the difference was not significant,
from which we concluded that both formats of the worksheet had the same effect. A possible
reason which would explain why the worksheet the experimental group received (i.e. the
version with the initials printed in bold, a bit larger and bunched together) did not work better
than the worksheet the comparison group received, is the following: de facto, both formats
did not differ all that much. Both versions included initials and the participants’ task was
exactly the same: finish the onsets of the words. The main difference was that the initials
were shown in a typographically different way. Even though we expected that typographic
highlighting would draw more attention to the FSs and although we thought that by bunching
together the initials this would stress their close association, this did not turn out to be the
case.
As mentioned above, the worksheets for the pretreatment were not very different from each
other. By contrast, when the students tried to reconstruct the text (three readings), their sheets
of paper differed a whole lot more. The CGr received a blank sheet of paper while the EGr
was given a worksheet with the FSs shown centred, in order of their occurrence and each on
its own line. Note that a similar worksheet was used during the previous experiment. The
layout was identical, but the participants were provided with the targeted FSs in their entirety,
while in our experiment one or more words were missing from each FSs. We assumed this
would engage the students of the EGr more than when they were given the entire phrase.
A logic line of reasoning would be that if the participants of either group already performed
well during the dictogloss treatment itself, this would also result in better results in both
posttests and vice versa. Interestingly, this reasoning is not consistent with our findings, that
is to say, it is not in line with the results of the EGr. In fact, the EGr performed exceptionally
well during the three readings in terms of how many targeted FSs they could complete. The
EGr counted 20 students and all but one achieved the highest score possible, namely 18, with
the one student who did not being an exchange student who achieved a score of 16. This
resulted in a mean score of 17.90 versus 13.63 for the comparison group. Note that the CGr
31 in itself did not achieve a mean score that was particularly low, which did not contradict the
line of reasoning for them. Even though the EGr achieved better results, the scores of the CGr
were in fact still fairly high. In other words, this led us to believe that there must have been
another reason that might explain why the EGr performed better than the CGr during the
dictogloss treatment, but this did not lead to better results in the end, i.e. the delayed posttest.
As mentioned before, the worksheet for the EGr was distinct from the worksheet used in the
previous experiment. This may have made a difference. While we assumed that the
worksheet used in our experiment would have a positive effect, because we thought it would
engage participants more, it might as well have had a negative effect and may have been less
effective than the worksheet used in the previous experiment. In the six following paragraphs
we enumerate a couple of reason that might explain why our worksheet did not achieve the
desired effect. Each paragraph deals with a different possible explanation. Please note that the
possible reasons we think of are quite complex and even more so than other possible
explanations for the lack of effect we listed above (about the pretreatment and the different
dictogloss methods). We tried to place ourselves in the participants’ position and as a result
the reasons mostly have to do with how they listen and perceive words as belonging together
or not, hence they are very difficult to substantiate on the basis of figures.
First of all, recall that the two groups were given exactly the same instructions in the same
room and at the same time. The instructor asked all participants to reconstruct the entire text.
However, both groups may have had a different objective in mind. We believe the
participants of the CGr their objective was identical to what they were instructed to do. The
EGr, on the other hand, altered their main objective upon receiving their worksheet. Their
main objective became filling in the blanks, while reconstructing the rest of the text was of
minor importance. Fill-in exercises with gaps or lines are very often used as a teaching
method. They are examples of traditional exercises all students are accustomed to: they
immediately know what to do and what is expected of them. In our experiment there was one
student of the EGr who at a certain moment asked the instructor whether they also had to
reconstruct the rest of the text (probably because he was confused why some other students
kept writing while he did not need as much time). The dictogloss exercise started some time
before he posed the question, meaning that he was only focused on filling in the blanks until
then. Obviously, we cannot generalise based on one participant, but this might have been the
case for some other students of the EGr as well. However, the possible explanation we
32 propounded concerning a different main objective remains feasible. By contrast, in the
previous experiment the worksheet for the EGr was slightly different. The complete FSs were
given. This meant that both groups had the same main objective, which was reconstructing
the (rest of) the abstract.
In summary, we may say that the CGr and the EGr perceived both versions of the dictogloss
treatment as different exercises with a different main objective, which meant that their
primary focus was different. This leads to our second reason why the worksheet for the EGr
used in our experiment may be less effective than the worksheet of the previous experiment.
A different central focus may translate into a different way of listening to the text. The CGr
will listen more attentively to every word of the abstract. They have to reconstruct the
abstract from scratch and as they are given a blank page all words are equally as important to
achieve their main goal. This does not mean the EGr will not listen attentively, it only means
that for the them some words are more important than others, namely the words needed to
achieve their main goal, which is filling in the blanks. By mainly focusing on those words,
they might filter out some other words – unconsciously or not – when listening to the text.
In the third place, the focus of the EGr may shift. First they will focus on the words that are
already on their worksheet and they will listen intently to the word(s) that precede or follow
the given word(s). As soon as they here the word(s) they need to fill in the blanks, their focus
shifts to that word/those words, probably meaning that they already pay less attention to the
words present on the worksheet. Moreover, as soon as they filled in the gaps, they achieved
their principal objective and their focus shift again. Now they have to accomplish their
secondary objective, which is completing the rest of the text. They no longer pay attention to
the FSs, but focus on all the other words.
Fourthly, as the EGr their primary focus was listening to the FSs and in particular to the
words that were missing, they may have failed to understand the structure of the text because
they were too focused on something else. It is important to grasp the overall meaning of an
abstract or else it will be a lot more difficult to remember and reconstruct the text, especially
one week later.
Fifthly, students of the EGr may not perceive the FS as a whole. In a certain sense, de FSs are
split into two parts: the word(s) given and the word(s) that need to be completed. As a result,
all the words that constitute the FSs are not typographically shown as a whole: in the same
33 font, the same colour, the same size. When students complete the gaps with their own
handwriting, the FSs may not register in their brain as words that belong together because it
visually does not look like a whole. By contrast, the EGr of the previous experiment received
a worksheet on which the complete targeted FSs were given and all words were
typographically the same. According to our way of thinking this would mean that the FSs
were more likely to be perceived (and remembered) as a whole, which would also explain
why the EGr performed better in terms of the reproduction of the targeted FSs. In both
experiments the CGr had to start off with a blank sheet of paper. Still, they were not given the
FSs, which implies the FSs could not be split into two parts either and as a result they may
have registered some FSs as a whole while simply listening closely. Once a FS is perceived
as a whole, it will be more likely to embed itself as a whole into one’s memory.
The final possible reason we are listing has to do with the advantages a blank sheet of paper
might have over a worksheet. A blank page gives the writer a lot of freedom, in the sense that
the writer is not dependent on a structure that is fixed beforehand. When listening to a text
every individual will mentally structure a text in a different way: in a way that he or she finds
helpful in order to remember and reconstruct the text later on. As the EGr received a
worksheet, the underlying structure of the text was decided for them. If this structure did not
correspond to their own structure they had of the text, they could not deviate from the outline
already provided on the worksheet. For some people a blank sheet of paper might work well,
but for others the opposite may be true. Some may find it too stressful, because they have to
do without the aid of the worksheet on which the targeted FSs were already given. In addition
to that, for some participants a blank page will represent a bigger challenge. As the CGr has
to start from scratch and is not provided with an outline, enthusiastic participants will
probably set the bar higher for themselves.
5.2 A more general interpretation of the results
The results we just talked about at micro level create a whole other and surprisingly different
image when we interpret them at macro level. By macro level we mean when comparing both
experiments. Knowing that the modified version of the dictogloss lacked in effect, some may
find it tempting to assume that this also meant that in general the participants performed
poorly when comparing our experiment to the previous one. However, it was found that this
34 is an inaccurate generalisation. In fact, the general results for all the participants together
were quite all right in relation to the previous experiment.
First, we decided to calculate the mean number of FSs reproduced during the final
reconstruction for the CGr and EGr combined. We wanted to know how many FSs
participants remembered on average in relation to the total number of FSs present in the
abstract (14 in the previous experiment and 18 in the current experiment). So we expressed
this result as a percentage and these percentages could be compared. Whereas participants
averagely reproduced 31.41% of the targeted FSs in our experiment, they only remembered
23.00% in the previous experiment. Although we did not achieve the main objective we had
in mind for this study, namely develop a modified dictogloss method which would not only
work better than the standard dictogloss, but would also work better than the modified
version of the previous experiment, on average the participants remembered more FSs. To
put it differently, the contrast between the CGr and the EGr was very small, but their results
combined were better than the results of both groups combined in the experiment by
Lindstromberg et al. (2016). This is even more surprising, knowing that the general feedback
of our experiment was less positive. Two factors may have had a positive influence. Firstly,
the difficulty of the FSs. As the participants of the two experiments were all third year
students majoring in English at the same university, we can assume their level of English is
roughly the same. The better results in our experiment for both groups combined may simply
be because the FSs were considered less difficult or the participants already knew more FSs
prior to the experiment. Another explanation concerns the change of the procedure. The
pretreatment, an additional step that was not included in the previous experiment, may have
had a positive impact on the retention of the FSs. As mentioned before, the different formats
of the worksheet did not lead to a contrast between the CGr and the EGr, but an additional
step implies that both groups were exposed to the FSs once more. As we all know, repetition
is crucial to amass knowledge in general and to learn a language as well. Either one or both
of these explanations may be true.
In a similar way, we calculated the mean total number of words participants used in their
delayed reconstruction (without contrasting the CGr with the EGr) as a percentage of the total
words of the abstract. In our case a 152 word text was used, in the previous experiment a 120
word text. It is striking that the percentages were in fact identical: in both experiment
participants generally used 45.93% of the total number of words the abstract counted. At first
35 sight some may conclude this means they the participants of the previous experiment
performed equally as well as the participants of the current experiment. Yet, we argue that the
participants of our experiment actually performed better. Logic suggests that longer texts will
be harder to remember in their entirety, i.e. the percentage will probably be higher when
participants are presented with a shorter text. This is not consistent with our findings. From
this we concluded that the participants of our experiment generally performed better: it is
harder for them to reach the same percentage given that the abstract was longer.
In addition to that, we may not forget that both experiments were quite different in terms of
variables. Precisely because there are so many variables, reaching at a conclusion is more
difficult. It is quite hard to give possible explanations because it is hard to find out which of
the following variables have had an impact on the results.
! A different number of participants
! A different abstract containing different FSs
! A different number of FSs
! A difference in the length of the abstract
! An additional step: the pretreatment
! A different modification to the standard dictogloss and subsequently a different
worksheet
All in all, if there is one thing that our research has shown, it is definitely that results viewed
from a micro level point of view may sketch a very different picture when they are
approached at a macro level. Because of that, comparing with other/previous research can be
useful and is to be recommended.
5.3 The potential of future research
More research is advisable, because results will be more reliable in an experiment with more
participants. It is commonly held that the more people participate, the more reliable results
will be. Moreover, our research would have even more potential when fewer variables are
changed in relation to previous research. In that way it will become clearer what the effects of
specific variables are and as a result the researcher can try to manipulate these variables.
36 A suggestion for future research would be to use the exact same abstract with the same FSs in
two experiments while doing one’s best to make sure the participants are as similar as
possible, both in terms of the number of participants and in terms of their level of English.
When those variables are stable, the researcher could, for example, change only one element
of the procedure. If there is a difference in results between both experiments, there is a fair
chance that this difference can be linked to that one particular adjustment made to the
standard dictogloss method.
After verifying what the specific effects of variables are, being able to manipulate those
variables will enable the researcher to maximise the effects. Following this line of reasoning
one comes to the conclusion that this will eventually contribute to reaching the objective in
mind: the development of a method that optimally helps students to remember and use more
academic formulaic sequences in the long run and facilitates language acquisition.
6. BIBLIOGRAPHY
Alali, F. A. & Schmitt, N. (2012). Teaching formulaic sequences: The same as or different
from teaching single words? TESOL Journal, 3(2), 153-180. Retrieved from
http://www.hetutors.com/uploads/alali-f-and-schmitt-n-(2012)-teaching-formulaicsequences-the-same-or-different-from-teaching-single-words-tesol-journal-3-2-153180.pdf
Alegría de la Colina, A., & García Mayo, M. P. (2007). Attention to form across
collaborative tasks by low-proficiency learners in an EFL setting. In M. P. García
Mayo (Ed.), Investigating tasks in formal language settings (pp. 91-116). Clevedon,
UK: Multilingual Matters.
Barfield, A. & Gyllstad, H. (2009). Researching collocations in another language: Multiple
interpretations. Basingstoke, UK: Palgrave Macmillan.
Boers, F. & Eyckmans, J. & Kappel, J. & Stengers, H. & Demecheleer, M. (2006). Formulaic
sequences and perceived oral proficiency: putting a lexical approach to the test.
Language Teaching Research, 10(3), 245-261.
37 Conklin, K. & Schmitt, N. (2008). Formulaic sequences: are they processed more quickly
than nonformulaic language by native and nonnative speakers? Applied Linguistics,
29(1), 72-89.
Connabeer, R. & Eyckmans, J. & Lindstromberg, S. (2016). A modified dictogloss for
helping learners remember L2 academic English formulaic sequences for use in later
writing. English for Specific Purposes, 41, 12-21.
Doughty, C. & Pica, T. (1985). The role of group work in classroom second language
acquisition. Studies in Second Language Acquisition, 7(2), 233-248.
Jiang, N. & Nekrasova T. M. (2007). The processing of formulaic sequences by second
language speakers. The Modern Language Journal, 91(3), 433-445.
Laufer, B. (2006). Comparing focus on form and focus on FormS in second language
vocabulary learning. The Canadian Modern Language Review, 63(1), 149-166.
Lim, W. L. & Jacobs, G. M. (2001). An analysis of students’ dyadic interaction on a
dictogloss task. Retrieved from http://eric.ed.gov/?id=ED456649
Long, M. H. & Porter, P. A. (1985). Group work, interlanguage talk, and second language
acquisition. TESOL Quarterly, 19(2), 207-228.
Nabei, T. (1996). Dictogloss: Is it an effective language learning task? Working Papers in
Educational
Linguistics,
12,
59-74.
Retrieved
from
http://files.eric.ed.gov/fulltext/ED401759.pdf
Nation, P. (2001). Learning vocabulary in another language. Cambridge, UK: Cambridge
University Press.
Nunan, D. (1989). Designing tasks for the communicative classroom. Cambridge, UK:
Cambridge University Press.
Moon, R. (1997). Vocabulary connections: Multi-word items in English. In N. Schmitt & M.
McCarthy (Eds.), Vocabulary: description, acquisition and pedagogy (pp. 40-63).
Cambridge, UK: Cambridge University Press.
Murray, H. (1994). Inside dictogloss – an investigation of a small-group writing task. Bulletin
Suisse
de
Linguistique
Appliquée,
60,
67-84.
Retrieved
from
http://doc.rero.ch/record/23145/files/67-84_Murraybis.pdf
Pica, T. (1992). The textual outcomes of native speaker-non-native speaker negotiation: What
do they reveal about second language learning? In C. Kramsch & S. McConnell-Ginet
(Eds.), Text and context: Cross-disciplinary perspectives on language study (pp. 198237). Lexington, MA, USA: D.C. Heath.
38 Schmitt, N. & Carter, R. (2004). Formulaic sequences in action: An introduction. In N.
Shmitt (Ed.), Formulaic sequences: Acquisition, processing and use (pp. 1-22).
Amsterdam, The Netherlands: John Benjamins Publishing Company.
Smith, K. (2012). Dictogloss: A multi-skill task for accuracy in writing through cooperative
learning. Retrieved from http://tht-japan.org/publications/2011/069-080_smith.pdf
Wajnryb, R. (1990). Grammar dictation. Oxford, UK: Oxford University Press.
Wray, A. (2002). Formulaic language and the lexicon. Cambridge, UK: Cambridge
University Press.
Yar, M. (2005). A deadly faith in fakes: Trademark theft and the global trade in counterfeit
automotive
components. Internet
Journal
of
Criminology.
http://www.internetjournalofcriminology.com/
7. APPENDICES
Appendices (1), (2), (3) and (4) can be found on the following pages, each one on its own
page. They are exact copies of the original worksheets and feedback form used in this
experiment.
39 Appendix (1): pretreatment worksheet – comparison group
I _________________ pr_________________ (IP) crime remains a m_________________
t_________________ in sociological and criminological investigation. This neglect i_______
d____________ i______ p____________ t_______ the perception that such offences are
‘non-serious’ and/or ‘victimless’.
Th__________ p_________________ s_____________ o___________ t________
ch_________________ such assumptions by examining a p_________________
i_________________ o_________ intellectual property crime, namely the counterfeiting of
dangerous goods, a________ i_________ p_________________ the counterfeiting of
a_________________ c_________________.
I______ i_______ a_________________ th_________ such activities, now
gl_________________ w_________________, carry significant e_______________
c____________, and p_____________ s_________________ r_______________ t_______
public health and safety. //
Some of th_____ k_____________ dr_________________ o_________ this trade are
analysed, along with r_____________ d_________________ i_____ law, policing,
c_________________ c_________________ and technological innovations that aim to
curtail the counterfeiting of these and other dangerous c_______________
pr_________________.
It is argued that IP crime comprises a s_________________ s_________________ and
sociologically ch_________________ ph_________________, one that deserves
c_________________ a_________________ from those working in the sociology of crime,
law, health, risk, technology and political economy.
40 Appendix (2): pretreatment worksheet – experimental group
I. pr. (IP) crime remains a m. t. i. sociological and criminological investigation. This
neglect i. d. i. p. t. the perception that such offences are ‘non-serious’ and/or ‘victimless’.
Th. p. s. o. t. ch. such assumptions by examining
a p. i. o. IP crime, namely the counterfeiting of dangerous goods, a. i. p. the counterfeiting
of a. c.
I. i. a. th. such activities, now gl. w., carry significant e. c.,
and p. s. r. t. public health and safety. //
Some of th. k. dr. o. this trade are analysed, along with r. d. i. law, policing,
c.c. and technological innovations that aim to curtail the counterfeiting of these and other
dangerous c. pr. It is argued that IP crime is a s. s. and sociologically
ch. ph., one that deserves c. a. from those working in the sociology of crime, law, health,
risk, technology and political economy.
41 Appendix (3): dictogloss treatment (three readings) – experimental group
intellectual …………………..
………………….. topic
is due in ………………….. to
this paper sets out to …………………..
a particular ………………….. of
and in …………………..
automotive …………………..
it is argued …………………..
………………….. widespread
economic …………………..
pose ………………….. risks to
the ………………….. drivers of
recent ………………….. in
crime …………………..
counterfeit …………………..
socially …………………..
challenging …………………..
………………….. attention
Appendix (4): feedback form – identical for both groups
Feedback about the exercise you did…
On a scale of 1 to 7, how would you rate the exercise overall regarding its potential
helpfulness in learning English phrases?
(1 lowest, 7 highest) Your rating here: _____
Do you have a particular feeling about any part(s) of the exercise? Which?
Do you think this exercise has good potential to promote learning of any other aspect(s) of
English? Which?
42