Thinking About: Research quality and constitutional changes in Higher Education Institutions.
To the Network 21 Symposium “Criticism and Collegiality in the field of educational research: the case of Sweden in Nordic perspectives”, Nordic Educational Research Association
Conference in Gothenburg, March 4 – 6, 2015.
Rita Foss Lindblad, Faculty of Librarianship, Information, Education and IT, University of
Borås, [email protected]
Abstract
Reports citing scientific and relevance deficiencies in educational research is numerous and can be
seen as indicating on-going and constitutional tensions and changes in the field of educational research more generally (Ranis, 2009). While the identification of such constitutional changes can help
us make sense of those kinds of standards that come to the fore in particular judgmental processes of
educational research (institutionalized forms of research assessments), it leaves out information about
research quality that pertains to the scientific process. The dimension of ”quality of research” has
come to be equal to ”evidence quality” (Mosteller & Boruch, 2002; Shavelson & Towne, 2002.). By
arguing the importance of a distinction between the two, this think-piece aims to bring changes in the
epistemic dimensions of quality of research to the fore. What if present constitutional changes of academic research has left us with poor understandings of the methodological and epistemological foundations of our research practices? Does this make a difference – and for what?
The elusiveness of Research Quality
Even though there are different views on what ‘quality’ means and refers to in empirical research there seems to be a consensus that it refers to something internal to the production of
scientific knowledge; elusive in kind but encompassing everything from study design to theoretical and methodological rigor and relevance (for example fit between research questions,
object under study, theories and methods used etc.).
However, in most of the debates that since at least the mid 1990s has haunted social and educational research with accusations of being of poor quality, there are surprisingly few explications of what, more exactly, the deficiencies of research quality are. What we find is instead
more sweeping conceptualizations where poor quality relates to un-abilities to address “external” dimensions of research and its uses and effects in society at large. With regards to educational research this will mainly mean its uses for teachers and policy-makers.
The very idea of the usefulness of scientific knowledge has been heavily debated ever since
its institutionalization and the organization of the sciences has come to meet up with this expectation differently from time to time. Strong parallels between the conceptualization of science and research and its social organization and social embedding has been obvious. Long
held beliefs in demarcations between “science” and “society” (scientific and lay knowledge)
has for example come in tandem with the autonomy of universities and the institutional imperatives of norms such as those expressed by Robert Merton (1942) and in ideas about the
objectivity and value-neutrality of scientific knowledge. Thus, and for long, research quality
was regarded as an internal affair of the sciences - defended by beliefs about the possibility of
its societal use and utility.
Such a situation is difficult to see as relevant for describing the situation today, and we have
now a substantial number of studies that bear witness of erosions of academic norms a nd
altered power structures in higher education institutions. It is a situation described in times of
intensified reform activities within contexts of globalization, marketization, new public management etc. (Enders et.al. 2011, Hasselberg et.al, 2013, Kehm & Teichler, 2013).
The politization of Research Quality
We could thus, from this very sweeping view of the relation between internal research quality
and external uses, conclude that we are living in times with radically new expectations of how
the external/internal dimensions of universities and research activities are supposed to be balanced and organized. Even though the situation never has been stable, the political and social
interest in research has been intensified and the use and users of research has come in more
direct focus also for actors within the research community (see Kaiserfeld, 2013).
This has meant, for example, a blurring between internal/external dimensions of research and
made ”quality of research” equal to ”evidence quality” and a tendency to put blame on “quality” when the “betterment” of practice has failed (Mosteller & Boruch, 2002; Shavelson &
Towne, 2002.). It has, also, come to mean new disciplines as well as the socialization of new
target groups of researchers (teachers) as well as the establishment of Educational Science at
the Swedish Research Council (much seen as an initiative with the direct purpose of increasing research quality according to societal needs).
But are there, also, more substantial changes within the epistemic dimension of research,
changes that are of concern for those internal dimensions of research quality so seldom discussed?
It is, I think, far easier to demonstrate changes in the social setting of educational research and
far more difficult, and maybe dangerous from the perspective of a researcher, to take into
consideration epistemic changes of a kind that concerns the way we conduct our research
practice. We might be fully aware of new pressures and changed power relations when it
comes to what kind of educational research should be supported and valued and how (and by
whom) it should be assessed. However, our reflexivity might have shortcomings when it
comes to our ability (or will) to observe what effects the drifts of academic culture and norms
have on such things as theories, methods and the very ‘doings’ or research.
Theoretical and meta-theoretical dislocations – in search for significances
I started my talk by claiming that such a thing as an “internal” dimension of research quality
makes sense and is in common uses. Even though discourses on research quality seldom have
this dimension in focus, talk about it hasn’t vanished; on the contrary it seems to be utterly
present as something naturalized or highly threatened.
On one side, when talk follows routes of criticism and assessment, we can observe that “research quality” seems to be poorly conceptualized within frameworks within which its meaning is taken for granted. Thus, in several cases of establishing national criteria for research
assessments (such as the Research Quality Framework in Australia or the Research Assessment Exercise in UK), explicit taken-for-granted references are for example made to such
things as “originality”, “excellence” and “rigour”. From our own immediate context within
the Swedish Research Council, the critique directed to educational research - for example in
terms of theoretical weaknesses and lacks of explanatory powers – implicitly presupposes the
existence of a “standard view” of science and research (for example Aasen et.al., 2005).
On the other side, and now within more social understandings of research where current
changes within higher education and research policy are observed, research quality refers to
something that is threatened and, indeed, is of risk of corrosion. The above-mentioned distinction between “quality of research” and “evidence quality” bear witness of this, as do notions
and studies about “epistemic drift” (Elzinga, 1997), signaling the tendency of scientists to
attend more to the external relevance of their research. Here, internal/external dimensions of
research as well as “quality” has been highly criticized but nevertheless maintained as important, not only symbolically but also because of the conviction that standards and views on
research practices historically do change. Change within present situation of radical restructurings of the higher education systems thus seems highly unlikely.
It is however neither within the discourses of “Social studies of the sciences”, “Sociology of
Knowledge”, “Higher Education” or studies that more directly tries to capture current (and
society-driven) changes within the Educational Sciences1 that we can find witness of changeses that more directly address research quality in an internal meaning. We have, instead, to
go to literature that has its focus on theoretical and meta-theoretical dimensions of research, to
ideas of theories of post-structuralism and meta-theoretical thought in the post-Kuhnian era.
Given these notions, I will end this think-piece by referring to two, what I find to be interesting and challenging observations, that could concern the difficulties of addressing the internal
dimension of research quality (quality of research) in conceptually more rich ways. Each
reference (despite their difference) relate to what I think is a missing dimension in current
debates on what bearings present constitutional changes of Higher Education have on ‘quality
of research’ and in relation to which I will put forwards questions that I wish to bring forwards to this symposium.2
In the introduction to his book “Method in Social Science. A Realist Approach” from1992
Andrew Sayer claims that social science is seriously in doubt. While outsider’s attitudes are
claimed often to be hostile, social sciences themselves have no common view of what constitutes proper approaches to social research. He put blame on philosophy of social science for
1
Such as Educational Research on Trial. Policy Reform and the Call for Scientific Rigour, by Pamela Barnhouse
Walter, Annette Lareau, and Sheri H Ranis from 2009, Educational Restructuring. International Perspectives on
Traveling Policies by Sverker Lindblad & Tom Popkewitz from 2004, or Disciplines of Education. Their Role in the
Future of Educational Research, by John Furlong& Martin Lawn from 2011.
2
The examples could be multiplied, but two will hopefully be enough for putting forwards reflections and
questions.
few constructive contributions to method in empirical research, and text on method for ignoring developments at the philosophical level (philosophy of science) and social theory. As a
consequence, he states that:
“…the lack on work on alternative methods has actually discouraged some of the critics
and their supporters from even venturing into empirical research. Meanwhile, many of
the empirical researchers whose works has been under attack have been content to conclude that the debate is not really relevant to them, or else that philosophical discussions
in general threaten empirical research and should therefore be avoided.”
In his book A nice derangement of epistemes. Post-Positivism in the study Science from Quine
to Latour from 2004 John H Zammito introduce us to post-positivist philosophy and theory of
Science since 1950s. As he says, this is no neutral book and what he wants to suggest is that:
“…some theorist have drawn upon post-positivism to initiate an attack upon the practice
of empirical inquiry itself. I consider and contest certain extravagant gestures in philosophy of language which, when taken seriously, threaten to undermine indispensible
canons of empirical inquiry. Probably the most extravagant theses in philosophy of language have been “the theoretical uptakes of Thomas Kuhn’s notion of “incommensurablity”, of Willard van Orman Quine’s dogma of “indeterminacy or translation, of Donald Davidson’s modulation of that into “indeterminacy of translation”, and – in a different tradition – of Jaques Derrida’s disquisition on difference.
What is suggested, by Sayer as well as by Zammito, is the failure of social science generally
to address “meta-theoretical” questions of empirical research. If true, this indicates that social
scientists will indeed have difficulties, even to formulate, some kind of defense towards “external” pressures of all kinds. This failure could, as suggested by Sayer, be related to
(miss)understandings of method itself, as well as meta-theoretical as well as theoretical to
miss-readings.
To me this is something that could be claimed to be the case also in Educational Research.
Thus, I put forwards the following questions:


We need to Do we suffer of theoretical and meta-theoretical miss-readings, are we in
lack of vocabulary or of showing little or none interests in questions of what we mean
when we address issues that concern quality of research?
Are we, indeed, even discussing these questions – and could a lack of such discussions
in fact be a consequence of the changes we find ourselves in?
References:
Enders, J; de Boer, H. & Westerheijden, D (Eds.) (2011) Reform of Higher Education in Europe. Sense Publisher
Hasselberg, Y, Rider, S och Waluszewski, A (2013) ’Introduction’ in S. Rider et al (eds)
Transformations in Research, Higher Education and the Academic Market. Dordrecht:
Springer.
Kehm, B & Teichler, U (Eds.) (2013) The Academic Profession in Europe: New Tasks and
New Challenges. Dordrecht: Springer.
Merton, R. (1942). Science and technology in in a Democratic Order. Journal of Legal and
Political Sociology. 1:15-26.
Lindblad, S. & Popkewitz, T. (Eds.) (2004). Educational Restructuring. International Perspectives on Traveling Policies. Information Age Publishing.
Mosteller, F., & Boruch, R. (Eds.). (2002). Evidence matters: Randomized trials in education
research. Washington, DC: The Brookings Institute.
Ranis, Sheri H. (2009). Blending Quality and Utility: Lessons Learned from the Education
Research Debate. Sid. 125 – 143. I Walters, P.B., Lareau, A., and Ranis, S.H. (Eds.) Education Research on Trial. Policy Reform and the Call for Scientific Rigor. Routledge.
Shavelson, R. J., & Towne, L. (Eds.). (2002). Scientific research in education. Washington,
DC: National Research Council, National Academy Press.
Swedish Research Council (2014): Research evaluation in Sweden – FOKUS. Stockholm: VR
Walters, P.B., Lareau, A., and Ranis, S.H. (Eds.) Education Research on Trial. Policy Reform
and the Call for Scientific Rigor. Routledge.
Zammito, J.H. (2004). A Nice Derangement of Epistemes. Post-positivism in the Study of Science from Quine to Latour. The University of Chicago Press.