Available online at www.sciencedirect.com
Journal of Second Language Writing 21 (2012) 284–293
Focus article: Replication in second language writing research
Graeme Porte a,*, Keith Richards b
a
Depto. de Filologı́as Inglesa y Alemana, Campus Cartuja, Universidad de Granada, 18071 Granada, Spain
b
Centre for Applied Linguistics, University of Warwick, Coventry CV4 7AL, UK
Abstract
This paper discusses the meaning and range of replication in L2 research from both quantitative and qualitative perspectives. In
the first half of the paper, it will be argued that key quantitative studies need to be replicated to have their robustness and
generalizability tested and that this is a requirement of scientific inquiry. Such research is seen as a useful way of examining
previous outcomes and of introducing novice researchers to the research process and academic discourse. The L2 writing field is
seen to present an increasingly diverse nature of scope and topics, which can easily result in conflicting, fragmented, and
consequently confusing research outcomes. Replication can help bring some order to this situation by focusing on the ‘‘why’’ of
previous findings. In the second half of the paper, we extend our discussion to qualitative research, where the idea of replication is
seen as inherently problematic. While recognizing the important differences between quantitative and qualitative research in this
respect, we argue that an outright rejection of the concept of replication may have diverted attention away from considering ways in
which qualitative research projects can build on one another and contribute to a developing understanding of L2 writing. Our
arguments are based on developments in qualitative research methods and online technologies over the last quarter of a century,
which undermine traditional objections to replication. We also propose using links to different types of related studies (‘Reference
Studies’) as a means of making essential connections among related projects.
# 2012 Elsevier Inc. All rights reserved.
Keywords: Quantitative research; Qualitative research; Replication studies; Reference studies; Active citation; Rigor; Transparency
Ferris (2005) recounts being inspired by Leki’s (1992) call for replication studies in L2 writing research. She recalls
going out that very day and finding a study to replicate – tellingly, it did not take long to find. Twenty years later,
replication studies in L2 writing remain scarce – almost non-existent. The present journal, to our knowledge, has only
seen the appearance of one study explicitly labeled a replication in its abstract. Given this apparent scarcity of research,
together with the circumstances that prompted this focus article, one might have expected more debate to have been
initiated. Surprisingly, however, at the 2011 Symposium on Second Language Writing (SSLW), specifically devoted to
‘‘getting published,’’ ‘‘improv[ing] the productivity of researchers from various parts of the world,’’ as well as
‘‘challenging the status quo in academic publishing’’ (http://sslw.asu.edu/2011/), replication of quantitative or
qualitative research appears to have been little discussed.
Unfortunately, a robust tradition of replication studies is missing from L2 studies in general. One might surmise
that:
a. Replication is not often undertaken as it is understood to be limited to exact replication studies which exist to
confirm the outcomes of the original studies, or
* Corresponding author.
E-mail addresses: [email protected] (G. Porte), [email protected] (K. Richards).
1060-3743/$ – see front matter # 2012 Elsevier Inc. All rights reserved.
http://dx.doi.org/10.1016/j.jslw.2012.05.002
G. Porte, K. Richards / Journal of Second Language Writing 21 (2012) 284–293
285
b. Replication is being carried out but is regarded as low-prestige, mundane, ‘‘unoriginal,’’ or ‘‘non-academic,’’ and
therefore not encouraged by faculty; consequently, such papers are rarely submitted or accepted for publication.
Both scenarios have worrying implications for the field of L2 writing research, but both quantitative and qualitative
replication is argued here to be feasible, necessary, and publishable. However, as replication in both paradigms may be
seen to have different aims and approaches, we have divided this paper into discussions of each approach, with Porte
looking at quantitative research and Richards at qualitative research. While we are in full agreement on the arguments
advanced in both sections, we have chosen to present the paper in this way in order reflect our belief that, although the
two perspectives are complementary, they should not be conflated.
Replication in a quantitative paradigm
The debate over the effectiveness of error correction in L2 writing (e.g., Ferris, 2010; Truscott, 2007) is an
instructive example. After 15 years of debate and a considerable accumulation of evidence, we appear to have reached
an impasse. In one sense this is encouraging since the area has clearly aroused the interest of numerous researchers,
both experienced and novice, and many data have been obtained. Viewed from another angle, however, and
particularly for those looking for guidance from research outcomes into policy and actual practice, the sheer amount –
and possible interpretations – of those data is at best confusing, and at worst overwhelming. Have we reached a point
where excessive data are being accumulated from an uncoordinated research agenda, thereby complicating the
interpretation of results across similar studies? I conjectured at SSLW 2010 that this may be a consequence of the
perceived demand for ‘‘original’’ research, or the ‘‘TDLR syndrome,’’ wherein researchers feel it is an essential
element of the genre of research writing to account for their present study as a response to the fact that ‘‘To Date Little
Research’’ exists on the subject.
Novelty thus comes to be seen as the key to progress or publication in the field. No one would argue against the
position that the more research data we gather about a subject and from various contexts, the better; it is doubtless valid
information and very interesting, but it also encourages data accumulation rather than knowledge construction (Porte,
2010). At some point a critical mass is reached (such as that perhaps attained in written error correction research)
which should trigger something more than the further amassing of empirical findings. It should also cause us to stop
and think about just why that mass has accumulated in the first place, and whether it contributes the kind of information
really needed to move us forward both in our research and in the eventual goal of that research – in the way we apply it.
Space does not allow me to detail key L2 writing studies which may be candidates for replications as I did at SSLW
2010, but it is pleasing to note that, since then, Polio (2012) has admirably taken up the challenge of formulating a
possible replication research agenda for written error correction studies.
L2 writing is – in terms of scientific enquiry – still a relatively young field, with systematic research beginning only
in the early 1990s. Much, therefore, remains to be explored, but there is still more to be discovered about our present
knowledge. As in any methodical endeavor, our research should take us on a journey of continuous discovery, and one
that not only seeks out novel areas of study but also routinely pauses to re-examine previous work in the light of new
information. In so doing, we are also contributing to the research community by revisiting earlier assumptions or
conclusions, confirming or otherwise what has previously been produced or thought (Porte, 2012). My aim in this
section is to present a case for replication studies in L2 writing research within a quantitative paradigm and, hopefully,
encourage a section of the readership to consider whether they might make a useful contribution in this way.
Worryingly, however, there might still exist an historical bias in many quarters against both undertaking and
publishing replication studies. Thus, there is little or no chance for what is supposed to be the self-correcting nature of
science to be nurtured in our field. The consequent absence of replication studies is especially problematic if we accept
that error, both of a methodological and conceptual kind, inevitably accompanies much of the research we do. Such
unattended error is particularly disquieting in a field with the kind of practical objectives we espouse and in which we
might reasonably expect research to regularly and reliably inform educational policy on L2 teaching. It is also
reasonable to imagine that basing key pedagogical decisions on the outcomes of a series of studies with undetermined
and/or unverified amounts of error may even have proved counterproductive in some learning contexts in the past
(Porte, 2012).
Even at the level of faculty we may be faced with direct or indirect opposition to actually carrying out replication
studies. Culturally, both the social sciences and society at large value the ‘‘ground-breaking,’’ the ‘‘innovative,’’ and
286
G. Porte, K. Richards / Journal of Second Language Writing 21 (2012) 284–293
the ‘‘original.’’ I have heard of faculty being discouraged from undertaking replication studies because, with their
presence on that person’s CV, they might be perceived as not having undertaken ‘‘original’’ work. Faculty might then
pass on these beliefs to graduate students, and so the cycle continues. In fact, as Fitzpatrick (2012) and Abbuhl (2012)
show, graduate programs are prime environments for implementing replication studies and introducing novice
researchers to appropriate research methodology, and the experience is almost guaranteed to act as a springboard to
innovative questions of one’s own.
What kind of replication?
Confusion still surrounds the different approaches to replication (see Polio, 2012 for a useful summary). In this
paper I will present those most commonly cited in the literature. Exact (or literal) replication is difficult to carry out in
our context since, by definition, it requires examining the same subjects employed in the original study. While
researchers might want to repeat their own studies (see below), it is often argued that there can be no such thing as
exact replication (or mere repetition). There are bound to be dissimilarities in the conditions of the different studies,
and a replication which verified a previous outcome would need to show that the result held again despite these
differences.
A more common venture, and one which allows for enough flexibility to accommodate the realities of conducting
research in educational settings, is an approximate (or partial) replication. The original study’s fundamental operations
are followed closely, but one or two of the non-major variables are changed to allow for eventual comparability
between both studies and their outcomes. Conceptual (also known as constructive) replication tests previous
hypotheses or results using a different experimental design. Different data collection procedures might be employed,
such as observation instead of self-report, or qualitative methods used alongside the quantitative methods relied on in
the original study. Successful conceptual replication provides stronger support for the original findings precisely
because evidence shows outcomes were not just artefacts of the original methodology.
Semantics does not help here – the term ‘‘replication’’ may itself be partly to blame for the unpopularity of what it
signifies. Both ‘‘replication’’ and ‘‘replicate’’ have connotations of producing something which is the ‘‘same’’ in any
subsequent study. In fact, this interpretation of sameness applies to only the first of the above kinds of replication and is
the least common of the three – namely exact replication. The essential component one is looking for is enough
similarity – rather than sameness – across the replicated and original studies to permit their satisfactory and
constructive comparison across one or more areas.
The purpose of replicating a study is not necessarily to obtain similar results. If these do not obtain, some doubt may
be thrown on the original outcomes, but there should at least be the positive result of encouraging further investigation
about why there was this apparent discrepancy. In other words, whatever the kind of replication undertaken,
knowledge will inevitably be obtained about the outcomes of the original study which will help the community move
forward and contribute to the self-correcting nature of scientific enquiry. Thus, while replications aim to encourage
reliability in the methodological aspects of a study and underline the robustness or generalizability of initial findings
when consistent data result, I wish to dissuade the reader from the assumption that such consistency should be
considered the sole aim of replication. Contradictory outcomes are just as interesting as consistent ones and, it must be
emphasized, just as worthy of dissemination.
The focus in any replication is always on the original study being replicated and what the results of our replication
tell us about that study, its robustness, and perhaps its generalizability. The aim is not to extend the methodology to
another context and report new outcomes in a ‘‘What happened when we did the same thing in. . .’’ situation. This is
because we assume that the objective in the replication was closely to assess the findings of that original in some way,
rather than simply transfer the original study’s methodology and procedures to produce a new study in a different
context and with different subjects. In such ‘‘follow-up’’ studies one usually finds a number of new variables entering
the mix: In general, the more variables changed from the original study, the more distant the new study is from what
was done originally. Therefore the two become less comparable and the focus shifts from the original study to the
outcomes in the new context studied.
Replication studies, because of their comparative nature and aim, need to be presented for publication in a way in
which the reader is able to appreciate the comparisons made between the papers. Readers are encouraged to consult
Brown (2012) for an excellent breakdown of how to present such a paper for publication.
G. Porte, K. Richards / Journal of Second Language Writing 21 (2012) 284–293
287
A replication research agenda for L2 writing
I suggest that coordination of a replication research agenda is now critical in L2 writing studies and that a number of
agents need to be involved. The excellent annotated bibliographies in JSLW would be a good place to start the search
for target studies. As they stand, however, the JSLW bibliographies remain interesting compilations of abstracts and
little more. One might usefully take this beyond the current abstracting stage and orientate it towards the signposting of
studies which could profitably be replicated. The compilers might consider incorporating authors’ comments on
limitations and/or future research (where the need for further replication is often proposed) or, indeed, themselves
briefly suggest how and why the study in question might benefit from approximate or conceptual replication.
Learned journals have always had a key role to play both in gate-keeping and setting research agendas. Further
encouragement to replicate studies could be provided by a formal call from the main journals that such studies are
indeed welcomed in their pages. JSLW is devoted ‘‘to publishing theoretically grounded reports of research,’’ but
specific encouragement to submit replication studies also serves to send a message out to those working in the field that
replication research is valued in the profession and sanctioned by the publication outlets of that profession.
While reviewing journal submissions in recent years, I have noticed an apparent trend to dispense with detailed
suggestions for ‘‘further research’’ or at most include them almost as an after-thought. Perhaps this is the current
fashion, but it is not one I endorse. Like many journals these days, JSLW does not state anywhere in its submission
guidelines that there should be any specific address of ideas for future research. My suggestion is that it should be
incumbent on authors to suggest not only what limitations there have been in their own research but also to encourage
others to revisit their work through replication with a view to establishing its reliability or generalizability. By doing
so, they would not be admitting doubt in their own work but rather appropriately recognizing the eminently
contributory and recursive nature of the scientific endeavor.
At Language Teaching we also commission authors to draw up research agendas for specific topic areas which
provide a ‘‘way in’’ to research for those interested in contributing to the field. Something similar – or more specific –
could be done for L2 writing, whereby an author might be asked to draw up a research agenda of studies that merit
replication because of their current relevance, their significance, or the perceived visibility of their results. Other key
candidates for replication in the L2 writing field might be those studies which have yielded unexpected or unexplained
results, or which would benefit from improving their generalizability. This could include studies which have already
made a significant contribution in terms of content and/or impact on the field.
Finally, much of the literature on replication research in the ‘‘pure’’ sciences assumes that some kind of independent
corroboration is needed when a replication of a previous study is carried out. However, it is in the original researchers’
interest to see their study outcomes validated or generalized through replication. Journal editors should welcome and
actively encourage studies wherein each researcher is responsible for establishing the reliability and validity of their
own research, and carrying out replication studies is an important part of that. The opportunity to replicate or improve
upon one’s own study would encourage laudable conduct: Researchers would not feel as much need to move too soon
from one topic to another, but instead learn from their experience.
Replication in a qualitative paradigm
If, as the foregoing suggests, quantitative researchers in our field have shied away from replication studies despite
the contribution such studies can make, what hope is there that qualitative researchers might be persuaded to take on
the challenges of replication? After all, we have an escape clause: replication and qualitative research (hereafter
QUAL) are antithetical. Yet this position sets aside all the advantages of replication highlighted above. In what follows
I argue that developments in QUAL over the last 25 years mean that there is no need for such pessimism and that the
time is ripe for reassessing the possibility of replication in QUAL.
One consequence of a shift in perspective away from the quantitative versus qualitative ‘‘paradigm wars’’ (Bryman,
2006) has been to expose divisions within QUAL which have long been present but which tended to be set aside in the
context of what was seen as a more important debate. Researchers within and beyond our field note that QUAL has become
‘‘increasingly fragmented’’ (Atkinson, 2005, p. 2; see also Holliday, 2004, p. 731), and there is a clearly discernable split
between those adopting a broadly realist position and those insisting on the primacy of the particular, or what one opponent
has described as ‘‘militant particularism . . . that privileges a ‘local’ of shared settings and face-to-face encounters and
treats them as the sites of situated, indexical knowledge about teaching and learning’’ (Nespor, 2006, p. 122). While this
288
G. Porte, K. Richards / Journal of Second Language Writing 21 (2012) 284–293
approach has been criticized for its ‘‘restlessly innovative’’ orientation (Atkinson, 2005, p. 5), it has opened up new ways
of understanding which should not be lightly dismissed. Nevertheless, its stance is inherently antithetic to replication,
however conceived.
Realist approaches, on the other hand, hold out the prospect of the sort of incremental advances discussed in the first
half of this paper, thereby deepening our understanding in ways not accessible through the mere accumulation of
research findings, so it is to these that attention will be directed. The argument will address four questions: (1) Is
replication possible in QUAL? (2) What conditions are necessary for it to succeed? (3) How might it be approached?
(4) What are the implications of this for the presentation of such research? The first two questions are addressed in the
following section, which develops in response to Le Compte and Goetz’s classic (1988) paper arguing that replication
in QUAL is not possible; the remaining two are more practical and are followed by a brief consideration of why
research into L2 writing may be ideally placed to exploit the potential of replication studies.
Is replication possible in qualitative research?
The relationship between qualitative and quantitative research is fraught with conceptual, definitional, practical,
and political difficulties, which leads some researchers to see the distinction itself as inherently problematic (e.g.
Allwood, 2011). Because quantification is too crude an instrument for distinguishing the two approaches, researchers
have tended to take up positions similar to that of Maxwell (2010, p. 477), who sees them as representing two different
ways of making sense of the world: ‘‘in terms of variables and correlations [i.e., QUAN] and in terms of events and
processes [i.e., QUAL].’’ However, for many QUAL researchers the nature of such events and processes makes
replication impossible.
This is the position LeCompte and Goetz (1982) take up in their paper on ethnography that has become a standard
reference point for those regarding QUAL and replication as incompatible. Their objections fall broadly into three
categories: the problem of change over time; the ‘‘personalistic’’ nature of ethnography; and methodological and
representational limitations. However, I shall argue that advances in methodological rigour and representational
possibility since 1982 have undermined the force of these objections.
LeCompte and Goetz’s first and primary objection is that ‘‘because human behavior is never static, no study can be
replicated exactly, regardless of the methods and designs employed’’ (1982, p. 35). This is certainly true, but there is no
reason to assume that the problem is unique to QUAL. In any case, the assumption that replication must necessarily be
exact replication is a misapprehension which my co-author has taken pains to correct both here and elsewhere (Porte,
2012). Le Compte and Goetz also go on to note that changes in features of the physical, social, and interpersonal
context over time make replication problematic, but if detailed delineation of specific contexts is possible (see below)
this could be seen as an opportunity for replication rather than a barrier to it.
LeCompte and Goetz’s second objection – the personalistic nature of ethnography – poses different challenges. The
most serious of these relates to the researcher’s unique positionality and involvement in the conduct and outcome of
the research, and the flow of information associated with it. LeCompte and Goetz argue that ‘‘[multiple] researchers
will fail to obtain comparable findings unless they develop corresponding social positions’’ (1982, p. 37). This could
be dismissed as a problem that applies only to ethnography and not other kinds of qualitative research, but even here it
is not necessarily telling. In L2 writing, for example, most researchers have experience as teachers, so it would be
perfectly possible to identify a common teacher-researcher role (e.g., ‘‘EAP tutor’’) and work/task specifications (e.g.,
‘‘following syllabus X using book Y at level Z’’) that could be replicated in future research. That said, LeCompte and
Goetz also note that the ‘‘ethnographic process also is personalistic; no ethnographer works just like another. A
researcher’s failure to specify precisely what was done may create serious problems of reliability’’ (1982, p. 36). Any
attempted replication would need to address this issue, but to a large extent this is a matter of setting appropriately
rigorous standards in terms of how data collection procedures are developed and represented. When LeCompte and
Goetz proposed this objection, data collection procedures were for the most part hidden in the ‘‘black box’’ of
researchers’ experience and intuitions, but, as I argue below, developments in research methodology since that time
have drawn the sting from this claim. Refinements in analytical procedure, for example, mean that no current doctoral
thesis based on a ‘‘vague, intuitive, and personalistic’’ analysis (1982, p. 40) could hope to pass muster.
These developments are particularly pertinent to the methodological and representational challenges advanced by
LeCompte and Goetz. Their claims that ‘‘replication may remain impossible if the constructs, definitions, or units of
analysis which informed the original research are idiosyncratic or poorly delineated’’ and that it depends on ‘‘precise
G. Porte, K. Richards / Journal of Second Language Writing 21 (2012) 284–293
289
identification and thorough description of the strategies used to collect data’’ (1982, pp. 39 & 40) are indisputable, but
while these might have been simply desiderata 30 years ago, today they are requirements. Prompted by criticisms such
as these, attention has shifted over the past 20 years to establishing standards for QUAL, leading to the publication of
key statements and guidelines (e.g., American Educational Research Association, 2006; Chappelle & Duff, 2003).
Attempts to improve standards have benefited from developments in technology, allowing online access to everexpanding databases. LeCompte and Goetz’s claim that journal-length articles do not allow for complete descriptions
of research design, data collection, and analytical procedures remains true only where associated websites do not allow
direct access to the data themselves. We are fast approaching a point where researchers wishing to replicate studies
will no longer ‘‘depend on fugitive monographs, technical reports, and personal communications’’ (1982, p. 40). Most
AL and SLA journal editors consulted for this paper confirmed that they can provide links to banks of supplementary
data, and it would be a pity not to exploit what most claimed to be a currently underused facility.
While technological advances are clear for all to see, advances in the rigor demanded of qualitative researchers are
less obvious, and there is certainly no reason to be complacent. In a recent meta-analysis of 44 ethnographic studies in
the field of medicine (Atkins et al., 2008), for example, while around three-quarters included adequate details of the
methods of data collection, less than half included acceptable details of sampling, only 12 provided a clear description
of the role of the researcher and, lamentably, only 12 adequately described the process of analysis – and there is no
reason to think that research in TESOL would present a rosier picture. Nevertheless, the problem has been recognized
and addressed in QUAL generally, some researchers tackling broader issues of criteriology (e.g., Tracy, 2010, p. 837),
others homing in on matters of research procedure (e.g., Onwuegbuzie & Leech, 2010). It would be impossible to
review developments on all fronts, but a consideration of two key elements will provide a sense of the sort of progress
which has been made to date in QUAL and which would be possible in TESOL.
In a valuable paper on reliability and validity in QUAL, Morse, Barrett, Mayan, Olson, and Spiers (2002) insist that
qualitative researchers should focus on rigor while researching, rather than relying on post hoc checks on
trustworthiness, and they identify categorization (a process involving the identification and coding of specific elements
within the data, and the organization of these into categories) and saturation (ensuring that nothing is missing from the
categories) as important elements in the process of verification. These elements offer valuable illustrations of the sort
of increasing transparency in QUAL that provides a foundation for replication studies. Recently, for example,
Wasserman, Clair, and Wilson (2009) proposed an analytical refinement that would allow greater transparency in the
ways in which relations between concepts are established, making clear links between categories and themes.
Alternative methods of thematic analysis have also been developed (e.g. Attride-Stirling, 2001; Braun & Clarke,
2006), meeting the rigorous structural and procedural demands essential for reliable research (Payne & Williams,
2005, p. 310) and encouraging a level of detailed representation that – crucially – allows comparison across studies. By
determining that the inclusion of further examples of a particular feature will contribute nothing new to the description
already developed, the second element, saturation, provides a necessary complement to categorization. Until recently
this was left to the judgment of the individual researcher, but workable criteria are now available. Bowen (2008), for
example, provides not only a guide to determining when categories are saturated but an example of the process of
categorization via coding.
How might replication in qualitative research be approached?
These developments and others like them have moved QUAL into a position where there need be no barriers to
replication in terms of methodological rigor and transparency, but this still leaves unresolved the issue of how it is to be
implemented. One line of approach would be to argue that there is no fundamental difference between quantitative and
qualitative research, which is broadly what Tacq’s (2011) claim that they share the same ‘‘experimental logic’’
amounts to, but a less radical approach involves establishing a principled way of achieving replication across cases.
This is the basis for the concept of ‘‘replication logic,’’ in which each case stands on its own as an analytic unit,
preserving the rich local context, but can be combined in a series ‘‘sampled for theoretical reasons, such as revelation
of an unusual phenomenon, replication of findings from other cases, contrary replication, elimination of alternative
explanations, and elaboration of the emergent theory’’ (Eisenhardt & Graebner, 2007, p. 27; see also Johnson, 1997).
This might also be represented as a reconfiguration of transferability issues in QUAL, moving away from questions
about whether findings might be transferable and toward the issues of transferability itself: In what respects cases
might be transferable, for what reasons they might be transferable, etc.
290
G. Porte, K. Richards / Journal of Second Language Writing 21 (2012) 284–293
Such approaches may not hold out the prospect of exact replication, which, as my co-author has pointed out above,
is probably practically as well as conceptually impractical in QUAL (e.g., Schofield, 2002, p. 174), but they do allow
for the sort of adjustments that create the possibility of approximate replication. In strengthening the breadth of claims
that might be made on this basis, they also make it less likely that qualitative researchers will feel the need to make
unwarranted generalizations on the basis of single studies (see Onwuegbuzie & Leech, 2010; Payne & Williams, 2005
for discussions of generalization practices).
How should qualitative replication studies be presented?
If the arguments to this point have at least established a case for considering replication in QUAL ‘‘as a systematic
process of extending, limiting, or simply reconsidering previous outcomes in the light of what we subsequently
discover rather than the accumulation of uncorroborated conclusions’’ (Porte, 2012, p. XX), there remains only the
issue of how it might best be presented in order to establish the essential links between studies that will generate the
sorts of cumulative insights and understandings that Porte refers to above.
The two essential connections that need to be made in presenting qualitative replication studies are those between
claim and data (where the data may be drawn from studies being replicated) and those linking related studies.
Moravcsik (2010) offers a way of establishing the former. Drawing on the dimensions of rigor and transparency
highlighted earlier, he argues that active citation, ‘‘the use of rigorous, annotated (presumptively) primary source
citations hyperlinked to the sources themselves,’’ provides a means of assuring ‘‘transparency and replicability in
selection, presentation, and preservation of qualitative evidence’’ (p. 31). The examples of primary sources he gives
include documents, oral histories, interview transcripts, and notes based on participant observation – all data sources
that might form part of a qualitative case study of L2 writing and that could be stored online.
Active citation establishes important connections to data and other studies within the text, but if we are to enable
readers to see at a glance how a published study relates to others, then it might be worth considering the inclusion of an
additional entry under the paper abstract along the same lines as keywords. Included here might be hyperlinked
citations to Reference Studies (or R-Studies) linked to the published paper. R-Studies could include not only
replications (where the relationship between the studies involved is an important concern) but also repetition and
(secondary) ‘‘resource’’ studies. In this way, connections could be made across different studies, whether qualitative or
quantitative.
The potential for replication in qualitative research on L2 writing
Research on L2 writing offers unique opportunities for replication because one of its key interests, as
exemplified in work on genre, lies in how language and context relate, and its products (texts) offer the prospect of
precise comparison. To illustrate what replication might involve, I draw on the most recent issue of JSLW at the
time of writing. Kibler (2011) focuses on four adolescent L2 writers and two of their teachers in a U.S. high school,
aiming to understand their expectations for assigned writing tasks (a humanities persuasive letter and a biology lab
report) and the extent to which these overlap. The data are drawn from fieldnotes, interviews, and student writing
samples, which form part of a larger ethnographic study. At the end of the paper the author makes a number of
suggestions for further research including how the teaching aligns with the writers’ and teachers’ own motivations
and goals.
Taking this as a basis for replication, it would be possible to identify other 10th grade classes in similar schools in
the U.S. having teachers and students with similar profiles (or profiles different in specific ways) involved in writing
biology lab reports. Drawing on online primary data from Kibler’s study, it would be possible to use the same data
collection procedures but code interviews and fieldnotes independently and compare results under common
categories, linking these to precisely defined categories in feedback on, and revisions to, the original assignments. This
could deepen our understanding of the nature of teacher-student alignment and might then form the basis of
(replicable) interventionist studies aimed at identifying practices that improve such alignment, or, over time, form part
of a broader understanding of the teaching and production of biology lab reports through school to higher education
and ultimately the workplace.
In advancing such arguments I am conscious that some qualitative researchers will instinctively recoil from what
they see as a Brave New World in which qualitative research conforms to the dictates of the quantitative orthodoxy and
G. Porte, K. Richards / Journal of Second Language Writing 21 (2012) 284–293
291
the agendas of those who promote it. But as St. Pierre and Roulston (2006, p. 678) argue, it is no longer sufficient
simply to claim to be a qualitative researcher: Qualitative research, like science itself, ‘‘is not just one thing.’’ I wish to
insist that nothing I have proposed diminishes the importance or value of the sort of qualitative studies that will never
be susceptible to replication. My aim has been merely to suggest that we should no longer assume that there is no place
for replication in QUAL. As Mackey (2012) insists, qualitative replication is most certainly not easy, but this is no
reason for rejecting it outright, and QUAL is a sufficiently broad approach to embrace very different forms of
engagement, including those suggested here.
Conclusion
The first half of this paper focused on replication in quantitative research. It noted the scarcity of such work in
L2 writing and that the perception of replication research in the community is not conducive to wider dissemination
of these studies. It was suggested that current practices might be encouraging the accumulation of data rather than
the construction of knowledge and that the time is propitious for a replication research agenda in certain areas of L2
writing research. This would see researchers revisiting previous assumptions or conclusions, confirming or
otherwise what has been produced or thought by others before us. Coordination of such an agenda was seen as
critical and suggestions were made regarding how learned journals and faculty might jointly promote more
replication research.
The second half of the paper examined the possibility that such benefits might be extended to work in qualitative
research, where replication is often seen as antithetical to fundamental concerns. Recognizing that by their very nature
some forms of QUAL will never be susceptible to replication, the argument focused on realist approaches, where
advances in technology have combined with increasingly rigorous methods of data collection and analysis to
undermine traditional claims about the ‘‘personalistic’’ nature of QUAL and representational limitations. The
relevance of replication in QUAL to research on L2 writing was also illustrated and it was suggested that journals
might encourage relevant links by introducing active citation and using the ‘‘key words’’ feature to identify relevant
reference studies.
It is in the nature of the scientist to be skeptical about research outcomes, and while some in our field may only
see red at the very thought of being labeled a ‘‘scientist,’’ it is this systematic, recursive, inquisitive, and critical
approach to research outcomes in L2 writing that is being encouraged in this paper. By addressing the results of
previous research through suitable replication approaches, where these are feasible and studies susceptible to such
an approach, the outcomes of that research can only have their scope better defined, extended, or limited.
Inevitably, by doing so we stand to gain more than we do from amassing uncorroborated evidence based on the
outcomes of ‘‘a self-contained, unalterable body of work’’ (Editorial (2006, July 27), Let’s replicate. Retrieved
from http://www.nature.com/nature/journal/v442/n7101/full/442330b.html). Indeed, as the L2 writing field
expands into novel areas of study, we would argue for greater investment in replication research in order to reflect
that increasingly diverse nature of scope and topics. If our research is left without a sound replication agenda, both
qualitative and quantitative, we might expect at best considerable conflict among research outcomes and, at worst,
confusion and stagnation. Replication of key studies will promote knowledge transfer and construction across that
research rather than mere knowledge accumulation. Together with a coordinated replication studies agenda and a
more responsive and receptive attitude from those at the publishing and teaching ends, it will encourage the kind of
healthy curiosity and skepticism towards research outcomes that will help move us forwards – rather than
sideways.
We have tried to underline the usefulness of replication research in terms of both its contribution to the research
scene and what we potentially lose by not undertaking it. This is not the first call for this kind of research (cf., Polio and
Gass, 1997) and may not be the last. It would doubtless be more comfortable for us again to ignore this call and simply
go on researching as we always have been doing. We would not want to suggest, of course, that replication research
will provide the answer to all our questions and the cure for all our conflicting evidence. Nor is this a call for us all to go
forth and replicate. We are suggesting – and of course not only with regard to L2 writing research – that as we extend
upwards the building that houses the sum of our expanding research knowledge we must, prior to beginning the
construction of each additional floor, ensure the stability of the levels below. It then becomes incumbent on us to
regularly revisit the entire structure to satisfy ourselves of its overall integrity. Building on top of initial weaknesses or
lacunae in our knowledge would forever leave that structure exposed to doubt and potential collapse.
292
G. Porte, K. Richards / Journal of Second Language Writing 21 (2012) 284–293
References
Abbuhl, R. (2012). Practical methods for teaching replication to applied linguistics students. In G. Porte (Ed.), Replication research in applied
linguistics. Cambridge: Cambridge University Press.
Allwood, C. M. (2011). The distinction between qualitative and quantitative research methods is problematic. Quality & Quantity http://dx.doi.org/
10.1007/s11135-011-9455-8 Advance online publication.
American Educational Research Association. (2006). Standards for reporting on empirical social science research in AERA publications.
Educational Researcher, 35(6), 33–40.
Atkins, S., Lewin, S., Smith, H., Engel, M., Fretheim, A., & Volmink, J. (2008). Conducting a meta-ethnography of qualitative literature: Lessons
learnt. BMC Medical Research Methodology, 8, 21http://dx.doi.org/10.1186/1471-2288-8-21 Retrieved from http://www.biomedcentral.com/
bmcmedresmethodol.
Atkinson, P. (2005). Qualitative research—unity and diversity [25 paragraphs]. Forum Qualitative Sozialforschung Forum: Qualitative Social
Research6(3) Art. 26. Retrieved from http://www.qualitative-research.net/index.php/fqs/index.
Attride-Stirling, J. (2001). Thematic networks: An analytic tool for qualitative research. Qualitative Research, 1(3), 385–405http://dx.doi.org/
10.1177/146879410100100307.
Bowen, G. A. (2008). Naturalistic inquiry and the saturation concept: A research. Qualitative Research, 8(1), 137–152http://dx.doi.org/10.1177/
1468794107085301.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(1), 77–101http://dx.doi.org/10.1191/
1478088706qp063oa.
Brown, J. D. (2012). Writing up a replication report. In G. Porte (Ed.), Replication research in applied linguistics. Cambridge: Cambridge University
Press.
Bryman, A. (2006). Paradigm peace and the implications for quality. International Journal of Social Research Methodology, 9(2), 111–126http://
dx.doi.org/10.1080/13645570600595280.
Chappelle, C. A., & Duff, P. A. (2003). Some guidelines for conducting quantitative and qualitative research in TESOL. TESOL Quarterly, 37(1),
157–178.
Editorial. (2006). Let’s replicate. Nature, 442, 330http://dx.doi.org/10.1038/442330b.
Eisenhardt, K. M., & Graebner, M. E. (2007). Theory building from cases: Opportunities and challenges. Academy of Management Journal, 50(1),
25–32http://dx.doi.org/10.5465/AMJ.2007.24160888.
Ferris, D. (2005). Tricks of the trade: The nuts and bolts of writing research. In P. Matsuda & T. Silva (Eds.), Second language writing research (pp.
219–230). Mahwah, NJ: Lawrence Erlbaum.
Ferris, D. (2010). Second language writing research and written corrective feedback in SLA. Studies in Second Language Acquisition, 32(2), 181–
201http://dx.doi.org/10.1017/S0272263109990490.
Fitzpatrick, T. (2012). Conducting replication studies: Lessons from a graduate program. In G. Porte (Ed.), Replication research in applied
linguistics. Cambridge: Cambridge University Press.
Holliday, A. (2004). Issues of validity in progressive paradigms of qualitative research. TESOL Quarterly, 38(4), 731–734.
Johnson, R. B. (1997). Examining the validity structure of qualitative research. Education, 118(2), 282–292.
Kibler, A. (2011). ‘‘I write it in a way that people can read it’’: How teachers and adolescent L2 writers describe content area writing. Journal of
Second Language Writing, 20(3), 211–226http://dx.doi.org/10.1016/j.jslw.2011.05.005.
LeCompte, M. D., & Goetz, J. P. (1982). Problems of reliability and validity in ethnographic research. Review of Educational Research, 52(1),
31–60.
Leki, I. (1992, March). Changing perspectives on L2 reading and writing. Plenary speech presented at TESOL conference.
Mackey, A. (2012). Why (or why not) when and how to replicate research. In G. Porte (Ed.), Replication research in applied linguistics. Cambridge:
Cambridge University Press.
Maxwell, J. A. (2010). Using numbers in qualitative research. Qualitative Inquiry, 16(6), 475–482http://dx.doi.org/10.1177/1077800410364740.
Moravcsik, A. (2010). Active citation: A precondition for replicable qualitative research. Political Science & Politics, 43(1), 29–35http://dx.doi.org/
10.1017/S1049096510990781.
Morse, J. M., Barrett, M., Mayan, M., Olson, K., & Spiers, J. (2002). Verification strategies for establishing reliability and validity in qualitative
research. International Journal of Qualitative Methods, 1(2), 13–22 Retrieved from http://ejournals.library.ualberta.ca/index.php/IJQM/index.
Nespor, J. (2006). Methodological inquiry: The uses and spaces of paradigm proliferation. International Journal of Qualitative Studies in Education,
19(1), 115–128http://dx.doi.org/10.1080/09518390500450268.
Onwuegbuzie, A. J., & Leech, N. L. (2010). Generalization practices in qualitative research: A mixed methods case study. Quality & Quantity, 44(5),
881–892http://dx.doi.org/10.1007/s11135-009-9241-z.
Payne, G., & Williams, M. (2005). Generalization in qualitative research. Sociology, 39(2), 295–314http://dx.doi.org/10.1177/0038038505050540.
Polio, C. (2012). Replication in published applied linguistics research: An historical perspective. In G. Porte (Ed.), Replication research in applied
linguistics. Cambridge: Cambridge University Press.
Polio, C., & Gass, S. (1997). Replication and reporting: A commentary. Studies in Second Language Acquisition, 19(4), 499–508.
Porte, G. (2010). Appraising research in second language learning (2nd ed.). Amsterdam: John Benjamins.
Porte, G. (2012). Replication research in applied linguistics. Cambridge: Cambridge University Press.
Schofield, J. W. (2002). Increasing the generalizability of qualitative research. In A. M. Huberman & M. B. Miles (Eds.), The qualitative researcher’s
companion (pp. 171–203). Thousand Oaks: Sage.
St. Pierre, E. A., & Roulston, K. (2006). The state of qualitative inquiry: A contested science. International Journal of Qualitative Studies in
Education, 19(6), 673–684http://dx.doi.org/10.1080/09518390600975644.
G. Porte, K. Richards / Journal of Second Language Writing 21 (2012) 284–293
293
Tacq, J. (2011). Causality in qualitative and quantitative research. Quality & Quantity, 45(2), 263–291http://dx.doi.org/10.1007/s11135-009-9293-0.
Tracy, S. J. (2010). Qualitative quality: Eight ‘‘big-tent’’ criteria for excellent qualitative research. Qualitative Inquiry, 16(10), 837–851http://
dx.doi.org/10.1177/1077800410383121.
Truscott, J. (2007). The effect of error correction on learners’ ability to write accurately. Journal of Second Language Writing, 16(4), 255–272http://
dx.doi.org/10.1016/j.jslw.2007.06.003.
Wasserman, J. A., Clair, J. M., & Wilson, K. L. (2009). Problematics of grounded theory: Innovations for developing an increasingly rigorous
qualitative method. Qualitative Research, 9(3), 355–381http://dx.doi.org/10.1177/1468794109106605.
Graeme Porte is senior lecturer in English language and applied linguistics research design at the University of Granada (Spain). He has been the
editor of Language Teaching (Cambridge University Press) since 2004 and has published widely in international journals. His book on teaching
research critique Appraising Research in Second Language Learning (John Benjamins, 2010) is a core reader on many postgraduate courses
worldwide and received its second edition in 2010. He has recently edited Replication Research in Applied Linguistics (Cambridge University Press,
2012) in the Cambridge Applied Linguistics series.
Keith Richards is an associate professor in the Centre for Applied Linguistics at the University of Warwick (UK). He has worked in a number of
countries and has been involved in teacher development around the world. His main research interests lie in the area of professional interaction and
his publications include Qualitative Inquiry in TESOL (Palgrave, 2003), Language and Professional Identity (Palgrave, 2006), Applying
Conversation Analysis (edited with Paul Seedhouse, Palgrave, 2006), Professional Encounters in TESOL (edited with Sue Garton, Palgrave,
2008) and Research Methods for Applied Language Studies (with Steven Ross and Paul Seedhouse, Routledge, 2011).