Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
Qualitative Resear ch Genr es in the I S L iter atur e:
Emerging Issues and Potential Implications
Suprateek Sarker
Department of Information Systems
Washington State University
Pullman WA 99164
[email protected]
Abstr act
In this opinion piece, we provide a critical
commentary on the arena of qualitative research in
IS. We reflect on why reviewer or editorial
evaluations of manuscripts, with respect to
methodological issues in qualitative studies, often
come across as “prejudiced.” By viewing the
adoption of qualitative research in the IS discipline
as an evolutionary process, and by highlighting key
differences among genres of qualitative manuscripts,
a number of implications for both authors and
evaluators of qualitative manuscripts emerge.
1. I ntr oduction
Whether or not we, as members of the IS
research community, consider ourselves to be
“qualitative researchers,” it is more than likely that
we have been involved at some time or the other: a)
as (co-)creators of qualitative manuscripts, b) as
recipients of evaluations on qualitative manuscripts
from referees and editors that have prompted us to
feel that we have been methodologically
misunderstood (or discriminated), and/or c) as
evaluators providing advice for undertaking revisions
on a manuscript and/or (input to) editorial decisions,
that may not have been appreciated by the authors,
ironically often for the same reasons as we have had
as authors receiving evaluations. While there is no
simple explanation for or solution to the above
experiences, one way to start unearthing and
addressing the underlying causes is to reflexively
consider the following questions: 1)What criteria or
values do we hold dear when conducting or
evaluating a qualitative study? 2) What are common
concerns regarding qualitative research? 3) What
methodological references do (or should) we utilize
in our own qualitative work and/or suggest to authors
whose work we may be reviewing? While each one
of us may have specific and, in many cases, welleducated responses to these questions (e.g., “richness
of contextualization” for the first question, “lack of
validity, reliability, and transferability” for the
second, and Yin [32] or Eisenhardt [7] for the third),
the goal of this paper is to argue emphatically for the
position that “it really depends…”
In our own experience, as authors and evaluators,
the most commonly expressed concerns of evaluators
of qualitative manuscripts 1 , irrespective of the nature
of the qualitative manuscript, indicate that many in
the IS research community view (or least categorize
for convenience) qualitative research as a monolith
under the labels of “case studies” or “interpretive”
research. While there is a gradually increasing
awareness in the discipline regarding the differences
between “interpretive” and “positivist” case studies,
and “action research,” especially among qualitative
researchers, the variations within interpretive
research are seldom acknowledged (e.g., [20]). In
fact, we believe, by treating all interpretive research
(or case or qualitative research) as being
fundamentally the same, we may be signaling that all
of this work needs to satisfy the same set of
conditions/criteria. The overall point is that unlike in
positivist research community where there is a clear
recognition of the differences between surveys, lab
experiments, event studies and so on, the distinction
between qualitative research “genres” are not widely
discussed, and thus remain unknown to many
colleagues.
This poses a huge challenge in
conducting, presenting, and justifying one’s work
from a methodological standpoint [10].
The
objective of this paper is to take a first step in
clarifying the similarities and differences among the
most prevalent genres of studies in the evolving
qualitative IS research arena, and to elaborate on
some of the key implications for both authors and
evaluators.
1
e.g., manuscript “lacking in richness and deep insights,” the
presence of “bias” in the manuscript, an absence of a clear
“sampling” and “coding” scheme for texts, lack of evidence that
the “conclusions resonate with those of the participants,”
incomplete information about the who were interviewed, when,
and for how long, and whether the interviews were transcribed, and
finally, the perennial lack of generalizability.
Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)
0-7695-2755-8/07 $20.00 © 2007
1530-1605/07 $20.00 © 2007 IEEE
1
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
The rest of the paper is structured as follows. In
the following section, we present a perspective on the
evolution of qualitative research within the IS
discipline, providing brief descriptions of the distinct
stages of the development. Next, we discuss some of
the innate characteristics of a study that are indicative
of the qualitative research genre. Based on the
linkages established between the characteristics of
the study and its genre, we argue how the criteria for
excellence associated with each genre can be
radically different. Finally, we offer our views on the
implications of our discussion of genres in targeting
work to IS research sub-communities in different
stages of their development (with respect to
qualitative methodologies).
2. The Evolution of Qualitative Resear ch
in the I S Discipline
In this section, we present an impressionist
account of how the state of qualitative research
methodologies has evolved (and continues to evolve)
within the IS community. We deliberately use the
term “impressionist” to characterize our description,
since we do not intend it to be an objective historical
account with frequency counts of papers published in
different journals in different years but rather an
imaginative interpretation (e.g., [3]) that captures the
essence of the evolution. In order to enable this
story-telling, we conceptualize qualitative research as
an innovation in a social system, and represent the
evolution process using the S-curve (Fig. 1).
The initiation stage refers to the patterns
observable soon after the introduction of qualitative
studies into the mainstream IS research community.
During this stage, “research” meant “quantitative
research,” and studies without hypotheses and
statistical analysis were excluded from the definition
of research itself. Lee and Liebenau [17, p. 3] capture
the patterns of behaviors in this phase eloquently in
their statement, “.. IS researchers had pejoratively
and imperialistically dismissed all qualitative
research as ‘unscientific’.” In this era, while
quantitative researchers enacted a “supremacist”
view, dominating the methodological discourse in the
discipline and its prestigious publication outlets, the
qualitative researchers enacted an “isolationist”
viewpoint, by shying away from the mainstream,
communicating research within their own subcommunity in journals and conferences that the
majority of mainstream scholars were unaware of.
While in general most qualitative researchers in this
era were content with their isolationist approach,
some of them led the struggle to legitimize this form
of research, by establishing the “scientific” nature of
case research [14], by utilizing the rhetoric of
“diversity” [24], by making qualitative research
methodologies more accessible and understandable to
mainstream audience, and by arguing for the virtues
of methodological “pluralism” in the research
community (e.g., [8]). As a result of these (and many
other similar) efforts, which are far too many to list,
the transition to the “contagion” stage was finally
made possible. Looking back to what we refer to as
the isolationist stage, Markus [18, p. 12] declared, in
her keynote address of a IFIP 8.2 conference
dedicated to qualitative methodologies, that
“achieving academic acceptance for qualitative
methods was no small feat.”
The transition to the contagion stage established
qualitative research as a legitimate alternate form of
research, which was fuelled by the rhetoric of
“exploration” (e.g., [30]). The underlying idea was
that there were many ill-understood IS phenomena
not captured by earlier theories, and there could be no
progress unless exploration, that was flexible and
primarily inductive, was undertaken by researchers.
This era brought a mindset of openness from a
methodological standpoint – everyone invited,
anything goes. The consequence was a proliferation
of case studies, many in leading outlets, often
authored by well-known quantitative researchers.
Typically, such studies used symbolic/obligatory
citations (e.g., prior editions of [32], [2,7,14,29])
perhaps to create an academic veneer, and indeed a
review published by Dube and Pare [6] shows that
few studies in this era actually followed
recommended methodological guidelines.
As in other instances of diffusion of innovation,
eventually the social system (here, the IS research
community) had to self-correct the degree of
enthusiastic openness to the innovation (here,
qualitative research), and this signaled the gradual
end for the free-for-all era, and the coming of the
control stage. “Method talk” then took a technical
turn, and this was characterized by increasing
concern for quality methodological rigor, invitations
for (and certification of) “exemplary” works, and the
emergence and reinforcement of “criterialogy. 2 ”
While this control mechanisms had a number of
positive effects, such as the development of some
shared values and criteria in the mainstream
community, and the sophistication of the
methodological discourse, the undesirable unintended
consequence was the creation of a bewildering set of
guidelines and criteria faced by an author, any of
2
Exemplars and criteria featured in Special Issues on
Intensive/Interpretive/Qualitative research/Action Research in
several prominent journals edited by experts in the field.
Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)
0-7695-2755-8/07 $20.00 © 2007
2
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
which could be potentially applied on his/her
manuscript by a referee, often due to inherent bias
(where certain values of qualitative research, say the
belief that any qualitative study must involve
“coding” is used as the prominent criterion
irrespective of the nature of the study), or
inexperience and partial education (e.g., where
despite Klein and Myers’ [13] insistence that the
criteria they propose are not universally applicable to
all studies, their criteria are still used as an inflexible
yardstick). To give a sense of the enormity of the
problem for a potential author, we list some of the
commonly held expectations: internal validity,
procedural validity, reliability, dependability,
auditability, thick description, conceptual density,
contextualization,
multiple
interpretations,
hermeneutic circle, coherence, theoretical saturation,
authenticity, credibility, plausibility, criticality, selfreflexivity,
suspicions
regarding
distortions,
transferability/external validity, penetration of texts,
novel insights, experiential and evocative texts, and
so on (e.g., [1,14,15,2,32,19,11,13,22,27,30]).
Admittedly, all of the above represent virtues, but the
question is one of emphasis and tradeoff. While we
do not suggest that any one evaluator of manuscript
would impose all of these requirements, it is
altogether plausible that a panel of 5 reviewers might.
And even if they don’t, the author is burdened with
anticipating and preparing defenses for the
“generalized” hypothetical reviewer who potentially
is free to invoke any of these criteria as challenges.
Recognizing the frequent mismatch in the criteria-inuse of authors and of gatekeepers, Markus [18, p. 14]
expresses her disapproval. She states “... When we
review the research of qualitative research
colleagues, we ‘diss’ [i.e., disrespect] those who do
not do qualitative research exactly as we do… this is
pur e and simple pr ej udice” [emphasis added]. An
important point worth emphasizing is that, in many
cases, it is the qualitative researchers, arguably some
of the same ones who were distraught about the
supremacist attitudes of the quantitative researchers
during the initiation phase, who appear to be unable
to transcend their own style and assumptions
regarding qualitative work, while serving in the role
of an evaluator. As Markus (above) rightly points
out, this is nothing but “prejudice,” whether
intentional or otherwise.
We present two illustrations drawing on
evaluators’ comments at prominent research journals
in the field. We would like to clarify upfront that our
goal is to not to criticize the comments or the
commentators 3 , but to highlight some of the patterns
of behaviors associated with the control stage. The
first example pertains to a positivist case study
manuscript (its author is unknown to us). We will
focus only on the comments offered on the
methodological aspects of the study. One reviewer
(Ri), for example, expressed his/her expectation from
the case study as follows “..One of the strengths of
theory building and theory testing using a case study
approach is the ability to j uxtapose contr adictor y or
par adoxical evidence that might gener ate novel
insights.” On similar lines, an editor, summarizing
comments of the review panel, stated “The authors’
use of the case study approach is found to be lacking
in r ichness and deep insights that one normally
expected of high quality case studies. [Rj] finds the
narrative very … “ fact-or iented” while [Rk]
describes it as “ distant” [emphasis added].
Is anything out of place here? Well, one could,
on behalf of the authors, argue that their positivist
case study was really not about generating novel/deep
insights by juxtaposing contradictory and paradoxical
evidence. It was much more about looking for
consistent patterns through triangulation of data
sources ([21]) that would enable the falsification of a
theory/proposition ([14]). Thus, the study was by
design not concerned with richness of description or
interpretation but instead was (rightly) attempting to
mimic the scientific attitude of being fact-oriented
and distant.
Consider a second example, pertaining to two
related interpretive studies, involving hermeneutics
and interpretation using the Actor-Network theory.
Some of the concerns expressed by the review panel
included: a) What are the potential sources of
“bias”? How did you select your texts? How were
your conclusions “extracted” from the data? Does
the interpretation resonate with the subjects?
Again, there are a number of potentially
legitimate responses to the objections. In the first
place, the term “bias” and its usage here reveals that,
unlike the authors of the study, the evaluator may
have adopted a realist ontological stance, where bias
is seen as a threat to validity. The question that arises
is: in an ANT or a hermeneutic study, would bias
necessarily be a threat? One could argue that it is
because of the so-called “bias” brought in through the
perspectives that we are able to generate insights that
are otherwise hidden from a common-sense
perspective held during the examination of data. For
3
Indeed, we have ourselves been “guilty” of similar “prejudice,”
and it is only through reflection over time, that we have developed
an understanding on some underlying issues, and wish to
communicate our perspective to colleagues through this paper.
Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)
0-7695-2755-8/07 $20.00 © 2007
3
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
example, the fact that IT and humans are analytically
treated the same in ANT, provides us the ability to
see how IT plays “socially constructed” roles usually
associated with individuals, such as those of traitor
and manipulator. Indeed, noted commentators, such
as M. R. Montgomery, NY Times Book Review on
Aramis, an exemplary work on ANT by Bruno
Latour, have noted that “Mr. Latour.. is creating-- a
new genre of fiction and reality (Scientifiction) that
tells a larger truth...” In other words, in some genres,
such as “scientifiction,” theoretical and even
empirical “bias” can be an asset.
Regarding the selection (sampling) of texts in a
hermeneutic study for example, it is worth noting that
the hermeneutics methodology is based on how
people come to understand texts in light of their
situations [16]. Specifically, when we read a book,
we rarely develop our linear understanding by
placing equal importance on each word or paragraph,
but instead attempt to iteratively make sense of the
“parts” in the context of the changing “whole”(e.g.,
[9,15]. Similarly, the expectation of “extraction”
presumes a “grounded” stance to data, where the
implied process of analysis is conceptualized by the
evaluator as follows: data Æ coding Æ inferences.
While this is the model of analysis in some
qualitative studies, in others, interpretation is viewed
as “productive”, not merely “reproductive” [22], and
there are no known rules of coming up with
(reproductive) interpretive “guesses” [12]. Finally,
with respect to the expectation of conclusions
resonating with participants, the authors might argue
that they are addressing their work to the scholarly
community, and focusing on higher-level rather than
first-level constructs of the natives (e.g., [15,30]),
and toward revealing “larger truths.” Clearly, the
high-level constructs in the IS literature, such as
“interpretive flexibility,” “distantiation,” “spirit of
technology,” and “inscription,” as also wisdom
embedded in Latour’s “scientifiction” works cannot
always be validated through “member checking”
[19]. In summary, we hope to have established above,
that “Qualitative research is a diverse enterprise…
[however] it is often portrayed in broad strokes that
blur differences” [10, p. 5].
For some colleagues, who are frustrated by the
“control,” there is good news on the horizon -- there
are signs that the discipline is approaching the
maturation stage with respect to qualitative research.
In this stage, we are likely to see a widespread
recognition of the different genres of qualitative
research, beyond the label of “case studies,” and even
“positivist” and “interpretive” case studies. For such
a transition to happen, there will be a need for
enabling a clear understanding in the IS research
community of the objectives, strengths, limitations,
and criteria pertaining to different genres, and the
creation of a critical mass of specialists capable of
evaluating
each
genre (e.g.
hermeneutics,
ethnography) and its sub-genres (e.g., philosophical,
critical,
phenomenological,
and
validation
hermeneutics, or realist, confessionalist, and
impressionist ethnography) (e.g., [28, 23,12]).
As the IS discipline matures with respect to its
adaptation to incorporate qualitative research,
secondary S-curves corresponding to similar
processes of evolution for qualitative sub-genres are
likely to become evident.
We wish to make a final point regarding our
above discussion of the stages. While for
convenience, we have treated the entire IS discipline
as a monolith, it is clear to us that different subcommunities in IS, whether defined in terms of
interest areas, conferences, journals, or even
geographical locations, are moving through this Scurve different at different rates and with different
trajectories. For example, a technical conference on
software engineering tools may be at an initiation
stage, and this sub-community may reject a work
based on the application of hermeneutic circle,
because the methodology is seen as “repetitive” in
the way it revisits and interprets the same data. On
the other hand, sub-communities focused on inviting
researchers to participate in knowledge building
within emerging areas of interest (e.g., e-Gov) may
be enacting Contagion. Sub-communities associated
with prestigious journals for which “rigor” is of
utmost concern may be engaging in some of the
actions described for the control stage. Finally,
specialized qualitative research communities (e,g.,
IFIP WG 8.2) may be in the process of achieving
maturation, and experiencing the arrival of the
secondary S-curves.
3. I ntr oducing qualitative r esear ch genr es
While there are many formal definitions of the
term genre, in this paper we use the term broadly to
differentiate among the different qualitative research
methodologies with different conventions for
conducting, representing, and justifying the studies.
Not only are assumptions underlying the
methodologies distinct, but the argumentation and
rhetorical style are also different (e.g., [11]).
On Figure 2, we map some of the prominent
genres visible in the qualitative research arena in the
IS literature. The two dimensions of the map are:
data-centric ÅÆ interpretation-centric, and
inductive ÅÆ deductive. The first dimension is
Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)
0-7695-2755-8/07 $20.00 © 2007
4
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
roughly equivalent to objective or subjective
approach to data. The second dimension is useful in
differentiating between genres of empirical studies
depending on whether they are data-driven or theorydriven. On the right hand side of the map, we show
two well-known quantitative approaches: labexperiments and surveys, to provide a point of
reference for some readers familiar with quantitative
approaches. Within the dotted oval, we show
positivist case methodology, grounded theory
methodology, exploratory case study (not really a
methodology, but nevertheless a popular approach
used by many authors for describing a situation and
unearthing some “lessons learned”), interpretive case
studies, ethnography, and hermeneutics.
We would like to emphasize that while the map
does highlight key differences in the methodologies
in terms of the two chosen dimensions, there are
many differences that are not obvious, primarily
because of the overlaps among methodologies and
significant differences within them. For example, in
hermeneutic tradition, a fundamental divide exists
between scholars holding relativist and absolutist
approaches to texts. Relativists question the authorintentionality of texts, a position that is vigorously
upheld by the objectivists. Further, labels such as
“interpretive case studies” (e.g., [30]) draw upon
ethnography, hermeneutics, grounded theory
methodology, and exploratory case studies, which
themselves partially overlap. It is easy to see from
the map that the design of criteria covering the entire
field would be very challenging, if at all possible.
For example, the celebrated set of criteria for
interpretive field studies [13] that is based on the
hermeneutic philosophy includes two criteria: “the
principle of contextualization” and “the principle of
interaction between researchers and the subjects.” For
a hermeneutic study that adopts a non-authorintentional view of texts, and where the texts
themselves are not generated through an interaction
of the study’s authors and the originators of the texts
[22], it is clear that the two above principles will not
be applicable. While experts in qualitative research
with minds open to different approaches to texts
would be able to recognize this point, it is not clear
that other evaluators would be able to adjust their
application of the Klein and Myers’ criteria,
considering the precise nature (genre) of the study
they are assessing.
4. Deter mining the Appr opr iate Genr e
Clearly, a host of factors can potentially
contribute to the decision of what genre to adopt,
from the author’s perspective. Likewise, a number of
factors would indicate to a knowledgeable evaluator
regarding the appropriate genre and appropriate
criteria for the genre. Below, we discuss four key
factors: a) The researcher’s conception and use of
data, b) The nature and role of theory in the study, c)
The analysis strategy used in the study, and d) The
nature of claims regarding findings of the study.
4.1 The r esear cher ’ s conception/use of data
Researchers adopt various perspectives on data.
In different studies, “data” may represent:
x Facts (e.g., XYZ Corp’s IT budget was $ 6,000,
000 for 2004).
x Subjective understanding (e.g., an individual
discussing her simultaneous sense of freedom and
bondage as a result of mobile device use).
x Socially constructed reality (e.g., a group of
process designers developing a shared understanding
of CASE tool as a “manipulator” and acting
accordingly with respect to the system)
x Negotiated meanings by subjects and researchers
(e.g., an interviewer asking an interviewee to imagine
situations specified by interviewer (“what if”
explorations), and then the interviewer and
interviewee jointly exploring different possibilities).
x Persistent text, distantiated from the context.
This is a critical point, especially for those
researchers conducting research within the
phenomenological hermeneutic tradition (e.g., [23,
3]). We provide a brief elaboration on the last point.
Consider the following text originating from a
redesign team-member in an organization
undertaking BPR. This text was captured by us in an
interview in 1996:
Some of the things [inefficiencies] that we
found can be fixed immediately... I call it the
hatchet in the head... if somebody has a
hatchet in their head, pull it out...
sometimes they are called quick-hits… low
hanging fruit... [emphasis added]
The fact that the interview segment had been
textualized at some point means that it is now
addressed to an indefinite number of possible readers
who bring in very dissimilar perspectives to the
reading of the text [23], leading to distantiation and
autonomization [16]. It is worth noting that the
quotation is being discussed here, almost a decade
later, and whether we, as authors re-using the text, as
readers, or as discussants of this paper misinterpret
the quotation or not, the author cannot “rescue” the
meaning [23]. Moreover, the intention of the
originator (speaker) is difficult to reconstruct [12]. It
Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)
0-7695-2755-8/07 $20.00 © 2007
5
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
may not be possible to track down him down and
verify what he meant, for a variety of reasons (e.g.,
he may have retired, or moved to a different city).
What is more intriguing is that the author may not
have known what he meant in the first place – in the
interview, he may have uttered some expressions that
he had read in a trade journal on BPR, for example,
and understood them only partially. Finally, the
meaning of the text may have changed for the author
over time [12].
The point here is that with
distantiated texts, there is a strong possibility of
dissociation between what the text means and what
the originator meant. In addition, through penetration
of a text, often an interpreter understands texts better
than the originator/author of a text [23].
4.2 The natur e and r ole of theor y in the study
The genre of a qualitative study also in part
depends on the manuscript authors’ definitions-in-use
of theory. For example, theory can be seen as a set of
falsifiable laws or statements linking independent and
dependent variables [15]. It can also be seen as a
narrative (“an account of a social process” [5]), a
product of disciplined imagination (e.g., [31]), an
“enlightenment” or “artful and exciting insights” [5],
or even as a “lens” or “scaffolding” [30].
It is reasonable to expect, given the intricate
relationship between theory and methodology, that a
study engaged (for instance) in building artful and
enlightening theory would be methodologically
distinct from a study that is engaged in developing
well-defined falsifiable propositions connecting
independent and dependent variables.
4.3 The Analysis str ategy used in the study
The nature of analysis in a study plays an
important role in defining the character of the study
and the appropriate conventions for writing,
justifying, and evaluating it. Some qualitative studies
rely on “polyphonic” presentation, where authors
allow subjects with multiple perspectives speak for
themselves. Others use induction, following the
prescribed steps, for instance, in the “Ladder of
Analytic Abstraction” [19], or the open coding, axial
coding, and selective coding recommended by some
grounded theory methodologists [27]. Yet others use
deduction, through the use of hypothetico-deductive
logic for the purpose of falsification [15]. Finally,
analysis takes the form of interpretation, appearing in
at least two variations: a) Elaboration using a (meta)theoretical lens or as a scaffolding [30] and b) as a
“guess” followed by “validation”, which respectively
represents the humanistic and the scientistic moments
of the hermeneutic circle [12, 23]).
4.4 The natur e of claims r egar ding findings
Qualitative studies have different aims and claim
different types of contributions. Different types of
claims have lesser or greater compatibility with
different genres. A study may, for example, seek to:
a) uncover what really happened (“the search for
truth”) [12], b) represent reality accurately (e.g.,
[32]), c) develop a plausible understanding of a
poorly understood phenomenon, d) generate new
concepts, novel insights [30], or propositions,
construct evocative, experiential text to “capture,
even reenact, the subject’s experience and to describe
that in full emotional color” ([10, p. 9]), or e) seek to
influence the views of a specific audience, reflecting
a “moral commitment” to a cause [26].
4.5 L inking the factor s to genr es and cr iter ia
We have discussed four key aspects of studies
underlying the “fit” with different qualitative
research genres. Given that the number of
combinations conceivable with different conceptions
of data, theory, analysis, and claims is huge, we are
not in a position to create a simple decision table that
provides guidance on suitable genre indicated by
each combination. Nevertheless, in Figure 3, we
attempt to capture the some of the combinations that
are associated with genres that are commonly seen in
the IS literature. For example, a (deductive) positivist
case would involve theory as a set of generalizable
propositions, data would be viewed as facts or shared
socially constructed reality, the analysis would
involve the use of hypothetico-deductive logic, and
the claim would be a validation or falsification of a
theory. In contrast, a phenomenological hermeneutic
study would view theory as a guess or enlightenment
attained through disciplined imagination, data as
distantiated texts, analysis as the second variant of
interpretation discussed above (guess followed by
validation), and the claim as a plausible but novel
understanding of the phenomenon being studied
(e.g., [25]). It is worth noting that an objectivist
hermeneutic study would differ from a
phenomenological hermeneutics study on all of the
four
aspects,
thereby
requiring
different
representation style, rhetoric for justification, etc.
Awareness of the differences in the genres (see
Figure 3) automatically allows one to appreciate the
fact that the associated criteria would be different as
Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)
0-7695-2755-8/07 $20.00 © 2007
6
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
well. Figure 4 presents key (not exhaustive) criteria
associated with the genres. For example, for the
deductive positivist case study genre, criteria that
would make sense include internal validity, construct
validity, reliability, and adequate degrees of freedom
[32,14]). Hermeneutic studies on the other hand
would need to be judged on, among other issues, the
level of coherence achieved in the interpretation by
blending the parts and the whole of the data. The
principle of the hermeneutic circle [13] is central
here. However, all hermeneutic studies do not share
key assumptions – different hermeneutic traditions
constitute different sub-genres. Thus depending on
the type of hermeneutics tradition a study aligns with
criteria
would
vary.
For
example,
a
phenomenological hermeneutic study can be more
imaginative, given that its goal is not to find what the
author meant but to deliver a creative plausible
understanding by penetrating the text [23]. Also
given the emphasis on validation (not verification),
the quality of argumentation during the validation
moment of the hermeneutic circle is also important.
We recall that objective hermeneutics is concerned
with the discovery of the original meaning of the text.
Thus, inferring the author’s intent is of paramount
importance, and this may be accomplished in many
ways, including the examination of and triangulation
with other texts by the same author or authors of the
same tradition and time period, who address the issue
of interest, directly or indirectly [1]. Similarly,
inductive exploratory case studies and realist
ethnographies concerned with representing reality as
accurately as possible, would need to adhere to a
different set of standards: high degree of authenticity,
auditability, and validity through member-checking.
Further, the richness (not thickness, which involves
peeling off multiple layers of meaning)) of
description would be a key attribute of such studies,
and the authors would be expected to provide
“confessions” to inform the reader how inaccuracies
in representation may have crept in.
5. Addr essing audience at differ ent stages
of development: I mplications for author s
Up to this point, we have discussed the
underlying characteristics of qualitative research
genres that mediate the communication between the
researcher and the reader. In this section, we discuss
some implications for authors in communicating with
their target audience, who may be located at different
points of the S-curve discussed earlier, having
different rates and trajectories in their evolution
process. Thus, authors of qualitative manuscripts
need to adjust their methodological justification,
application, and discussion based on stage of
evolution the audience sub-community may be in.
Clearly, authors must first diagnose which stage their
audience is currently in. Having completed such a
diagnosis, the following tactics may be useful:
x
When targeting work to a sub-community
that is in the initiation phase, the authors could focus
on establishing the value of diversity of research
approaches. For example, the authors may explain
basic differences between qualitative and quantitative
research, and demonstrate how their work adds to the
community’s body of knowledge. The key point is to
appeal to the audience regarding the value of some
(“controlled”) diversity in their sub-community,
while tangibly demonstrating the value of the
submitted (qualitative) work with respect to the
audience members’ own terms and priorities. Despite
such efforts, sometimes qualitative research authors
may find sub-communities in the initiation stage to
be unsympathetic to their methodologies. Their work
may be dismissed as subjective storytelling, and
considered not objective or rigorous, arbitrary,
unscientific, etc (e.g., [4]); in this case our advice
would be to avoid (or withdraw) submission, since no
amount of convincing is likely to work.
x
When targeting qualitative work to a subcommunity in the contagion stage, the authors are
likely to face few methodological objections and
challenges. Sophisticated methodological discourse is
usually unnecessary and not helpful. Basic obligatory
citations should work (e.g., [32,7]), and in most
cases, evaluators would not object to the mixing of
elements from different genres, under the “case
study” or “interpretive study” labels.
x
When targeting qualitative work to a subcommunity in the control stage, the authors are likely
to face significant methodological challenges, with
evaluators bringing up a variety of objections, some
of which may not be applicable to the “genre” the
authors may be using. In this case, the authors may
need to “educate” evaluators about what
characteristics their genre has and does not have. This
can be done by making explicit the assumptions
regarding data, theory, analysis, and the
objectives/knowledge claims of the study, and
relating these assumptions to criteria that would be
appropriate to apply.
x
Finally, when the target audience is
diagnosed to be in “maturation,” the authors need to
convince evaluators regarding the sophistication in
their methodological discussion and application. In a
“mature” sub-community, evaluators are likely to be
experts in the genre adopted by the authors, and the
Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)
0-7695-2755-8/07 $20.00 © 2007
7
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
experts will likely expect evidence regarding the
authors’ mastery of the genre: awareness of
contemporary trends and dilemmas in the chosen
genre, and most importantly, the ability to
consistently adhere to the genre. Mixing of genres or
even sub-genres, which may have been be acceptable
in the earlier stages [11], is likely to meet with
substantial resistance and criticism.
4.
5.
6.
6. Conclusion
In this manuscript, that may be characterized as
an “opinion” piece, we have attempted to provide
some reflection on why peer evaluations with respect
to methodological issues in qualitative studies often
come across as “prejudiced.” Our premise has been
that the authors as well as evaluators are generally
well-intentioned, and are merely enacting their own
understanding of what qualitative research is. In
particular, there is often a mismatch between the
genre (or genre combinations) implicitly adopted by
the authors and the genre (or genre combinations)
implicitly expected by the reviewers/editors. What
makes the situation worse is that the evolutionary
state of sub-communities and its members are not
static but evolving – authors face a moving target
while they are themselves in motion.
We hope that this paper will contribute in
bridging this mismatch between the creators and the
gatekeepers. With each side understanding (and
articulating) the assumptions/criteria associated with
different genres and also appreciating the
evolutionary process/states of the other side, the
authors and evaluators will not communicate past
each other, but with each other, leading to a more
constructive dialogue and better scholarship in the
collaborative spirit for which all of us yearn.
7.
8.
9.
10.
11.
12.
13.
14.
7. Refer ences
1.
2.
3.
Alvesson, M. and Skoldberg, K. Reflexive
Methodology: New Vistas for Qualitative
Research. Sage, London, UK, 2000.
Benbasat, I., Goldstein, D. K., and Mead, M.
“The Case Research Strategy in Studies of
Information Systems,” MIS Quarterly (11),
1987, pp. 369-386.
Boland, R. J. "Information system use as a
hermeneutic process," in Information Systems
Research: Contemporary Approaches and
Emergent Traditions, Elsevier, North Holland,
1991, pp. 439-457.
15.
16.
17.
Denzin, N. and Lincoln, Y. Handbook of
Qualitative Research Second Edition. Sage,
Thousand Oaks,CA, 2000.
DiMaggio, P. J. "Comments on 'What
Theory is Not'," Administrative Science
Quarterly, (40:3), Sept. 1995, pp. 391-397.
Dube, L. and Pare, G. "Rigor in Information
Systems Positivist Case Research: Current
Practices, Trends, and Recommendations,"
MIS Quarterly, (27:4), Dec. 2003, pp. 597635
Eisenhardt, K.M. “Building Theories from
Case Study Research,” Academy of
Management Review (14:4), 1989, pp.532550.
Fitzgerald, B. and Howcroft, D. “Towards
Dissolution of the IS Research Debate: From
Polarisation to Polarity,” Journal of
Information Technology, (13:4), 1998,
pp.313-326.
Gadamer, H. G. Truth and Method. Seabury
Press, New York, NY, 1975.
Gubrium, . F. and Holstein, J. A. The New
Language of Qualitative Method, Oxford
University Press, New York, NY, 1997.
Harvey, L. “A Discourse on Ethnography,” in
Information Systems and
Qualitative
Research, A.S. Lee, J. Liebenau, and J.I.
DeGross (Eds.), Chapman and Hall, New
York, 1997, pp. 207-224.
Hirsch, E. D. Validity in Interpretation, Yale
University Press, New Haven, Conn., 1967.
Klein, H. K. and Myers, M. D. “A set of
principles for conducting and evaluating
interpretive field studies in information
systems,” MIS Quarterly, (23:1), 1999, pp.
67-93.
Lee, A. S. “A Scientific Methodology for MIS
Case Studies,” MIS Quarterly (13:1), 1989,
pp. 33-50.
Lee, A. S. "Integrating positivist and
interpretivist approaches to organizational
research," Org. Sci., (2:4) , 1991, pp. 342-365.
Lee, A. S. “Electronic mail as a medium for
rich
communication:
An
empirical
investigation
using
hermeneutic
interpretation,” MIS Quarterly, (18:2), June
1994, pp. 143-157.
Lee, A.S. and Liebenau, J. “Information
Systems and Qualitative Research,” in
Information Systems and Qualitative
Research, A.S. Lee, J. Liebenau, and J.I.
DeGross (Eds.), Chapman and Hall, New
York , 1997, pp. 1-8.
Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)
0-7695-2755-8/07 $20.00 © 2007
8
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
18. Markus, M.L. “The Qualitative Difference
in Information Systems Research and
Practice,” in Information Systems and
Qualitative Research, Chapman and Hall,
New York, 1997, pp. 11-27.
19. Miles, M. B., and Huberman, A. M.
Qualitative Data Analysis, Sage, Thousand
Oaks, CA, 1994.
20. Nandhakumar, J and Jones, M. “Too close
for comfort? Distance and engagement in
interpretive information systems research,”
Information Systems Journal, 7, 1997, pp.
109-131.
21. Paré, G., and Elam, J. J. “Using case study
research to build theories of IT
implementation,” in Information Systems and
Qualitative Research, A. S. Lee, J. Liebenau,
and J. I. Gross (Eds.), Chapman and Hall,
London, 1997, pp. 542-568.
22. Prasad, A. “The contest over meaning:
Hermeneutics
as
an
interpretive
methodology for understanding texts,”
Organizational Research Methods, (5:1),
2002, pp. 12-33.
23. Ricoeur, P. From Text to Action: Essays in
Hermeneutics II. Northwestern University
Press, Evanston, 1991.
24. Robey, D. "Diversity in Information
Systems Research: Threat, Promise, and
Responsibility,"
Information
Systems
Research, (7:4), 1996, pp. 400-408.
25. Sarker, S., and Lee, A. S. “Does the Use of
Computer-based BPC Tools Contribute to
Redesign Effectiveness? Insights from a
Hermeneutic Study," IEEE Transactions on
Engineering Management, (53:1), 2006, pp.
130-145.
26. Schweizer, T. “Epistemology: The Nature and
Validation of Anthropological Knowledge,”
in Handbook of Methods in Cultural
Anthropology, H. R. Bernard (ed.), Altamira
Press, Walnut Creek, CA, 1998, 39-87.
27. Strauss, A. and Corbin, J. Basics of
Qualitative Research, Newbury Park, CA,
1990.
28. Van Maanen, J. Tales of the Field.
University of Chicago Press, Chicago, 1988.
29. Walsham, G. Interpreting Information
Systems in Organizations, John Wiley and
Sons, New York, NY, 1993.
30. Walsham, G. “Interpretive case studies in IS
research: nature and method,” European
Journal of Information Systems, 4, 1995, pp.
74-81.
31. Weick, K. E. "What Theory is Not,
Theorizing Is," Administrative Science
Quarterly, (40:3), Sept. 1995, pp. 385-390.
32. Yin, R. K. Case Study Research, Second
edition. Sage, Thousand Oaks, CA, 1994.
Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)
0-7695-2755-8/07 $20.00 © 2007
9
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)
0-7695-2755-8/07 $20.00 © 2007
10