Realities Toolkit #
Practical considerations for leading and working on
a mixed methods project
Vanessa May and Hazel Burke, Morgan Centre, University of Manchester
July 
Introduction
The aim of this toolkit is to highlight key issues that might arise out of leading or
working on a mixed methods research project. It will be useful for both Principal
Investigators or project leaders and other team members alike, and will offer a practical
guide to help prepare for, design and carry out a mixed methods project. The focus of
this toolkit is in other words on the practical aspects of such work, covering some of the
more common pitfalls that mixed methods projects might face: the importance of
teamwork; the need to allow for extra time; issues around data analysis and integration;
and publishing from mixed methods projects. By highlighting these potential challenges
as something worth considering at the outset of a project, we by no means intend to put
you off from embarking upon mixed methods research, but rather hope to make the
experience even more enjoyable.
Acknowledgments
This toolkit draws on a Realities ‘Interdisciplinary Dialogue’ event that focussed on the
processes and products of mixed methods research, held on 11 May 2010 at the
University of Manchester. The event was attended by researchers with experience of
working on mixed methods projects within a range of disciplines in the social and health
sciences. We present some of the key points raised during the discussions that day. We
wish to offer our special thanks to Sue Heath, James Nazroo and Alicia O’Cathain, who
led the discussions that day, as well as to all the other participants1.
The importance of teamwork
Good teamwork is necessary for any project to be successful. Achieving this in a mixed
methods project is particularly important but can also prove to be especially challenging
(O’Cathain et al., 2008). Whereas researchers working on projects involving a single
1
Our thanks go to the participants at the workshop for their valuable contributions: Tarani
Chandola; Angela Dale; Simon Duncan; Rosalind Edwards; Nissa Finney; Alexandra Greene; Sarah
Irwin; Jennifer Mason; Stewart Muir; Margaret O’Brien; Mike Savage; Simone Scherger; Julius Sim;
Carol Smart; Mandy Winterton. (This toolkit does not necessarily represent participants’ views.)
1
Realities toolkit #11: Practical considerations for leading and working on a mixed methods project
methodological approach can often make certain assumptions about the team’s shared
vocabulary and approach, mixed methods teams that are likely to involve researchers
from different disciplinary and methodological backgrounds do not have this luxury. A
particularly tricky aspect of mixed methods team work is not being able to rely on team
members understanding each others’ work, or agreeing with each others’
methodological motivations. In addition, in some disciplines there is an established
convention of working in teams, whereas in other disciplines this is not necessarily the
case. Consequently, team members may be faced with a foreign vocabulary as well as
different assumptions about what counts as evidence and what constitutes knowledge,
in addition to which the members may have varying expectations regarding how team
work should proceed.
In other words, mixed methods work often involves working outside of ones comfort
zone, which can give rise to tensions and misunderstandings, that, if handled badly, can
lead to some team members feeling exposed, slighted, undervalued or undermined. If
this happens, not only does successful research become difficult to achieve, but it can
also be an unpleasant and stressful experience for the researchers concerned. This is why
it can be useful to explicitly address the interpersonal aspects of mixed methods team
working from the outset. Good teamwork need not mean that everyone always agrees,
but rather that any tensions that do emerge are handled in a respectful manner in order
to create an environment where creative tensions (Mason, 2006) can flourish and provide
impetus for innovative research.
One suggestion to help develop good interpersonal relationships within a team is to
budget for one or several away days for all team members.
The role of the principal investigator or project leader in determining the success of a
mixed methods project should not be underestimated. Project leaders can do much to
help support their team and its research by making it clear that they value mixed
methods work and by encouraging and facilitating this kind of work. A principal
investigator plays an important role in setting the tone of a project in terms of working
relationships, or can help ensure that some ground rules of respect are agreed at the
outset. It can also be worth thinking about how junior team members are to be
mentored, particularly if senior members are unfamiliar with the methods they use, or
how team members from a ‘minority’ methodological background are to be supported.
Setting out a clear division of labour regarding where, when and by whom the ‘mixing’
should occur can also be helpful. The principal investigator’s attitude towards mixed
methods can ‘make or break’ a mixed methods project. Generally those projects where
the principal investigator is supportive of all the methods involved tend to be those that
are experienced as positive working environments. Such projects might also be more
successful in terms of managing to integrate findings because no single method is
allowed to ‘take over’.
Different academic disciplines tend to gravitate towards different models and styles of
teamwork. For example, being part of a project within the health sciences is likely to be a
very different experience compared to being part of a human geography project. If team
members come from different disciplinary backgrounds, it is worth considering the
different expectations that they might bring to the project and to explicitly address how
the team is expected to work, in relation to, for example, different members’
responsibilities, or communication between team members. This can help avoid future
misunderstandings, or simmering resentments.
July 2010
2
Realities toolkit #11: Practical considerations for leading and working on a mixed methods project
The time challenge
A general tip is that more integrated projects are likely to place more demands on the
time of team members compared to projects where the different methods comprise
parallel sub-projects with little integration or communication taking place between them.
Perhaps the main reason why highly integrated projects will take more time is because
team members must be able to understand and interpret the data derived through the
different methods, which in many cases entails that they learn a new vocabulary. Perhaps
a good analogy is to imagine that all the team members have different first languages:
some speak English, some speak Russian, some use sign language. The various members
may, or may not, speak the other languages. How much extra time will have to be
allowed will depend on how much communication between the different team members
is needed for the project to be able to move forward and for successful integration to
take place.
Translating, for example, one form of academic reasoning, approaches to sampling, or
data, can involve lengthy and repeated meetings where methods, methodologies and
concepts are explained to those team members who are unfamiliar with these and who
may have some doubts, and important questions, about their robustness. If team
members are expected to in any meaningful way understand or analytically engage with
‘non-native’ data, enough time has to be allowed for them to learn how to interpret
these unfamiliar data. Generally such translation work is not required in single method
projects.
Moreover, if any integration or synthesis of findings is to take place, even a mixed
methods project involving two or three fairly discrete sub-projects that function
relatively independently is likely to require more time than would be necessary in a
project where everybody was familiar with the method used.
Analysing, interpreting and integrating different data
Even when a project leader has allowed enough time and has managed to marshal a
rambunctious mixed methods rabble into a creative, intellectually-challenging-yetsupportive team, one key problem may still remain: the data and how to integrate them.
Even if the team has so far worked together perfectly harmoniously, the different
methods may produce data that are wilfully contradictory, or when team members’
interpretations of the data may contradict each other. In the first case, the team might
disagree over how to handle such contradictions. Some may view any contradictions as
inherently problematic and as a sign that something has gone ‘wrong’ in the study, while
others can perceive the contradiction as an interesting intellectual puzzle in its own right
(please see our toolkit on contradictory data for a more detailed discussion of different
ways of handling contradictory data).
See also Realities Toolkit #12: ‘What to do with contradictory data?’ by Vanessa May,
available from www.manchester.ac.uk/realities/resources/toolkits
In the case of contradictory interpretations among team members, some teams are
comfortable with making their divergent analyses public (for example in the form of a
‘minority report’ that presents interpretations that differ from the main report), while
other teams prefer to ‘solve’ the contradiction and present a more unified set of findings.
This may of course not always be for the team to decide; funders in particular may not
July 2010
3
Realities toolkit #11: Practical considerations for leading and working on a mixed methods project
react well to a project report that presents contradictory findings. If this is the case, the
only option might be to present the common ground and avoid highlighting the
differences, especially if the funder is looking for a set of recommendations. In other
cases, contradictions or differing perspectives may be more acceptable, and can be made
public. In practice, this may be less problematic than it sounds. For example, it is possible
that the team members would be targeting different journals with different audiences.
Project leaders might find it useful to think about ‘ownership’ of data, especially in cases
where team members’ interpretations diverge. Do team members have the same rights
to use all the data collected within a mixed methods project? In other words, can team
members publish their own divergent interpretation of data that they were not directly
involved in collecting? Can the different team members go on to publish their conflicting
interpretations of the data once the project is completed?
Publishing from mixed methods projects
A final challenge that mixed methods teams may face is that of finding a publication
outlet for their findings. Even when different types of data combine easily to tell one
story, or when team members seamlessly agree on how to handle contradictions in the
data or conflicting interpretations, finding places to publish mixed methods research can
prove to be more difficult than is the case for single method research.
Different methodological traditions tend to have different styles of writing, and journals
have their in-house styles or even templates that authors are expected to follow. Fitting
in with a particular tradition, style or template of writing – usually designed for writing
up findings from a single method study – might not always be possible for mixed
methods work. Because it can be difficult to find a ‘home’ for mixed methods
publications, many mixed methods projects end up producing single method publications.
Agreeing a publication strategy may help. You can use this to set out what publications
you anticipate will come out of the data, where you will aim to publish them, and who
will write them.
It is worth considering that authorship of a mixed method publication is likely to take
more time. As mentioned above, mixed methods writing cannot always readily follow an
existing format, which is why it can take extra time to work out how to structure a paper.
There can also be the need to integrate different literatures, or perhaps iron out
differences in writing styles. Another reason why mixed methods studies often end up
writing single method publications is because teams simply run out of time and are
therefore unable to integrate their findings in any meaningful way.
July 2010
4
Realities toolkit #11: Practical considerations for leading and working on a mixed methods project
References
Mason, J. (2006) ‘Mixing methods in a qualitatively driven way’, Qualitative Research,
6(1): 9-25.
O'Cathain, Alicia, Murphy, Elizabeth & Nicholl, Jon (2008) ‘Multidisciplinary,
interdisciplinary, or dysfunctional? Team working in mixed-methods research’,
Qualitative Health Research, 18(11): 1574-1585.
Feedback welcome! If you have any comments on this toolkit or if you
can tell us how you have used it in your research or teaching please do
drop us a line at [email protected] and let us know.
Join our email newslist for updates on our toolkits, events and other activities:
www.manchester.ac.uk/realities/aboutus/newslist
@
Realities is part of the National Centre for Research Methods, which aims to improve
research methods across the UK social science community. We are based in the Morgan
Centre for the Study of Relationships and Personal Life at the University of Manchester.
Realities, Morgan Centre, Sociology, University of Manchester, Manchester M13 9PL
+44 (0) 161 275 0265 | [email protected] | www.manchester.ac.uk/realities
July 2010
5