Assessing Research Quality
An editor’s perspective
Chris Chapman
2nd October 2013
Accounting, Organizations and Society
http://ees.elsevier.com/aos/
• An International Journal Devoted to the
Behavioural, Organizational & Social Aspects of Accounting
• Methodologically diverse
Financial Times Journal List
ABS Journal List – 4*
8 Issues a year, in the 30’s articles
The editor’s job is to publish quality work
• Rejection rates – Producing knowledge not rationing prestige
Rigour and Relevance
There is no single nor stable optimum in terms of this tradeoff
Finding a shifting Goldilocks zone determined in relation to a research
community whose boundaries also shift over time
There are many forms of “rigour”
Lindbeck, A. (2007) The Sveriges Riksbank Prize in Economic Sciences
in Memory of Alfred Nobel 1969-2007
• It may also be argued that the Prize-awarding authority has
demonstrated that there are many different ways to advance a
science like economics:
• rigorous deductive theorizing, whether by way of verbal or
mathematical techniques; the development and application of new
concepts and methods of analysis; rigorous empirical testing of
existing hypotheses,
• as well as less formalized confrontation of various hypotheses with
empirical fact; or "simply" profound observation and nonformalized
innovative thinking about economic issues.
My bulleting: text at
• http://nobelprize.org/nobel_prizes/economics/articles/lindbeck/index
.html
Discussing the nature of Relevance
Alvesson, M., and J. Sandberg. 2011. Generating Research
Questions Through Problematization. Academy of Management
Review 36 (2):247-271.
• Interesting is something that challenges assumptions
» “Gap spotting” – under problematises
» “Critical” – Over problematises?
• Problematising is a question of making explicit what assumptions
might “fruitfully be challenged”
Impact on practice?
• I have never seen work turned away because it has an impact on
practice, but that is neither necessary for all work, nor sufficient
for any in the context of an academic journal
• The impact of academic knowledge (on anything) rarely arises
most powerfully through individual articles
Creeping instrumentality is a threat to research quality
Salami slicing of studies?
• Maximise number of publications subject to minimum
scope/quality
• Pernicious at the aggregate and individual publication level
Citation strategy?
• Ritualistic currying favour or constructing an audience with whom
to engage in debate and discovery?
Research quality begins with individual’s passion for a topic and their
ability to communicate how and why that can be useful to a wider
audience
• Enthusiasm for teaching is a useful marker
We must be careful not to over-individualise and over-homogenize our
understanding of research therefore