How to...
Use mixed methods research

Why use mixed methods research?

Back in the 1970s and 1980s, qualitative and quantitative research were seen as two opposing paradigms, each with their own supporters, most of whom would have denied the possibility of mixing them – this period is often referred to as "the paradigm wars".

During the last two decades, however, researchers in both camps have begun to see the importance of each other's data.

While researchers have been mixing quantitative and qualitative techniques for years, what is new is the deliberate attempt to provide theoretical background and formal design(s).

Undoubtedly, what has fuelled the interest has been the increasing complexity of research problems, and the desire for an understanding that is both deep and broad (afforded by qualitative and quantitative data respectively). 

The increasing importance of the link between research and policy has also played its part, as has the greater availability of data provided by the internet, which enables multiple data sets to be integrated in different ways, providing a richer and more informed picture of social phenomena.

Advantages and disadvantages of using mixed methods

Mixing data sets can give a better understanding of the problem and yield more complete evidence – the investigator gains both depth and breadth. 

Amalgamating statistics with thematic approaches can help avoid over-reliance on the former and can also capture "soft-core views and experiences" (Jogulu and Pansiri, 2011) and the subjective factors necessary to elucidate complex social situations.

It can also strengthen findings – a process known as triangulation.

On a more philosophical level, mixed methods research combines paradigms, allowing investigation from both the inductive and deductive perspectives, and consequently enabling researchers to combine theory generation and hypothesis testing within a single study (Jogulu and Pansiri, 2011). 

Having to use mixed methods also helps researchers to develop their skills, which is particularly important for those at an early stage of their career. 

Mixed methods are not without their drawbacks, however.  An obvious one being the resources and skills required – one researcher may not be skilled in both qualitative and quantitative methods and may have to call on the expertise of someone else, or another team, which will increase the cost.

While mixed methods are used widely throughout the social and behavioural sciences, there are a number of particular areas where the approach has become popular.

• Multidisciplinary research, including research which focuses on a substantive field, such as childhood or disability.
• Research on complex social issues, particularly populations which would be difficult to reach with a questionnaire, but where it is still necessary to employ quantitative methods to demonstrate scale and thereby achieve change (Brannan, 2005).
• Complex and pluralistic contexts, for example, studies involving schools, cross-national studies, and studies where it might be important to bring in a different perspective to counterbalance, say, a rather individualistic bias from informants.
• Practical and policy-related academic research. (Both sorts of research are needed for the reasons given above.)

Undertaking mixed methods research

A research methodology needs a philosophical and epistemological underpinning, i.e. what is the nature of the knowledge you are trying to uncover?

Quantitative research is based on positivism; qualitative on interpretivism or social constructivism: the distinction is between objective and subjective knowledge.

Johnson and Onwuegbuzie (2004) advocate pragmatism as the philosophical partner for mixed method research, citing the pragmatist philosophers William James and John Dewey who believed in settling metaphysical disputes by tracing the practical consequences.

Johnson and Onwuegbuzie's article provides a useful introduction to the philosophical debate, and to pragmatism. Tashakkori and Teddlie (2003) also have a chapter on pragmatism in their book. Cresswell (2009) provides a useful introduction to the different world views that accompany different research methodologies. De Loo and Lowe (2011) discuss mixed methods in the contexts of the philosophy of science. 

The research question

Most researchers would see the research question(s) and/or hypotheses as fundamental to selecting the type of research methodology.  Typically, a mixed method research design should be chosen when these are best answered by both qualitative and quantitative information.

Salfi (2011) conducted a study which looked at successful leadership practices of headteachers in Pakistan.  His research questions (RQs) were: 

• RQ1. What type of leadership practices are employed by the headteachers of successful schools in Pakistan?
• RQ2. Is there any similarity among leadership practices of headteachers of these schools?
• RQ3. Is there any difference between the opinions of headteachers and their subordinates regarding their leadership practices?

The author obviously needed to obtain as wide a profile as possible so his design included a survey with demographic, Likert-scaled, and open-ended questions; however, he also needed opinions, which are subjective, and best dealt with by qualitative methods. 

Questions of frequency may best be explored by quantitative methods, and perception and opinion by qualitative. If the questions deal with both of these, then mixed methods are likely to be preferred. It may also be the case that the investigator wants to explore a subset of a population in greater depth, perhaps after a wider survey.

See Creswell (2009) for a discussion of how to frame mixed method questions and hypotheses. 

Rationales

As early as 1989, Greene et al. (cited by Johnson and Onwuegbuzie, 2004) suggested five main rationales for conducting mixed methods research:

1. Triangulation – the confirmation of results by different methods. 
2. Complementarity – results from one method are used to enhance, elaborate or clarify results from another method.
3. Initiation – where new insights are obtained which will stimulate new research questions.
4. Development – results from one method shape another method.
5. Expansion – expanding the breadth and the range of the research by using different methods for different lines of enquiry.

Murphy and Maguire (2011) used mixed methods to evaluate the costs and benefits of conducting clinical trials. They devote considerable space to the analysis of their data, which they had trouble triangulating: in the end, they decided that their data yielded complementary findings. 

Wedawatta et al. (2011) used a mixed methods approach (questionnaire and case study) to investigate the impact of extreme weather events on the construction of small to medium-sized enterprises. Their design could be described as both complementary, as results from the case studies threw further light on the findings from the survey, while it also initiated new knowledge and further research questions.

The important point to note here is that using different methods is not simply a matter of amalgamating the data and corroboration of results; rather, the investigator needs to think very carefully as to how the results are being combined, and what findings, or questions, arise.

Strategies & typologies

There are a number of different types of mixed method research design.  The most common terms researchers use are:

• Concurrent or simultaneous – different research methods are incorporated into the same research study and the information integrated to interpret the results.
• Sequential – one method is used initially, followed by another method, possibly followed by a third.

The methods may have equal status, or one may be dominant.

Jogulu and Pansiri (2011) provide the following matrix, adapted from other researchers, which illustrates the typology outlined above: 

De Silva (2011) describes the adoption of mixed methods in in voluntary environmental reporting research, and her own experience of using a QUAN→qual design.

Jogulu and Pansiri (2011) analyse two management PhDs which use mixed methods research (p. 691ff). Both used a sequential strategy, using quantitative data followed by qualitative data – QUAN→qual.

Harrison and Reilly (2011) analyse the use of mixed method research in major marketing journals, categorizing their findings into exploratory (qual→QUAN), explanatory (quan→QUAL), embedded (in which one form of data play a supporting role), concurrent, and hybrid.

The type of design used and, in particular, the dominance of qualitative or quantitative will affect the analysis.

Combining quantitative and qualitative research is not simply a matter of using different methods and then adding up the results to form a set of findings. 

Mixed method research has emerged as a new research paradigm alongside qualitative and quantitative research, with its own methodology and procedures, which the investigator would do well to acquaint themselves with. 

Once they have done so, mixed methods can be an exciting and rewarding field, one that not only yields solid research, but research that can be used.

References

• Brannen, J. (2005) "NCRM methods review papers, NCRM/005. Mixed methods research: a discussion paper", unpublished discussion paper, available at: http://eprints.ncrm.ac.uk/89/ [accessed 25 July 2011].
• Cresswell, J.W. (2009), Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (3rd ed.), Sage Publications Inc.
• De Loo, I. and Lowe, A. (2011), "Mixed methods research: don't – 'just do it'", Qualitative Research in Accounting & Management, Vol. 8 No. 1, pp. 22-38.
• De Silva, T.A. (2011), "Mixed methods: a reflection of its adoption in environmental reporting", Qualitative Research in Accounting & Management, Vol. 8 No. 1, pp. 91-104.
• Greene, J.C., Caracelli, V.J. and Graham, W.F. (1989), "Toward a conceptual framework for mixed-method evaluation design", Educational Evaluation and Policy Analysis, Vol. 11 No. 3, pp. 255-274.
• Harrison, R. and Reilly, T. (2011), "Mixed methods designs in marketing research", Qualitative Market Research, Vol. 14 No. 1, pp. 7-26.
• Jogulu, U.D. and Pansiri, J. (2011), "Mixed methods: a research design for management doctoral dissertations", Management Research Review, Vol. 34 No. 6, pp. 687-701.
• Johnson, R.B. and Onwuegbuzie, A.J. (2004), "Mixed methods research: a research paradigm whose time has come", Educational Researcher, Vol. 33 No. 7, pp. 14-26.
• Murphy, L. and Maguire, W. (2011), "Applying mixed methods research in evaluating clinical trials", Qualitative Research in Accounting & Management, Vol. 8 No. 1, pp. 72-90.
• Salfi, N.A. (2011), "Successful leadership practices of head teachers for school improvement: Some evidence from Pakistan", Journal of Educational Administration, Vol. 49 No. 4, pp. 414-432.
• Tashakkori, A. and Teddlie, C. (Eds) (2003), Mixed Methods in Social and Behavioural Research, Sage Publications Inc.
• Wedawatta, G., Ingirige, B., Jones, K. and Proverbs, D. (2011), "Extreme weather events and construction SMEs: Vulnerability, impacts, and responses", Structural Survey, Vol. 29 No. 2, pp. 106-119.