ISSN: 2052-9368 Health Economics Working Paper Series HEWPS Number: 201501 Microfinance, health and randomised trials Neil McHugh, Olga Biosca and Cam Donaldson April 2015 www.gcu.ac.uk/yunuscentre Microfinance, health and randomised trials Neil McHugh1* * Corresponding author Email: [email protected] Olga Biosca1 Email: [email protected] Cam Donaldson1 Email: [email protected] 1 Yunus Centre for Social Business and Health, Glasgow Caledonian University, Glasgow, Scotland, UK Contact information: Yunus Centre for Social Business and Health Glasgow Caledonian University Level 3 - Buchanan House 58 Port Dundas Road Glasgow Scotland United Kingdom G4 0BA 1 Abstract Background There is a growing need to identify upstream public health interventions as an alternative means of acting on health. Microfinance has been portrayed in this regard in public health journals. Furthermore, microfinance has spread to more-advanced economies where it offers potential as an alternative means of acting on persistent and growing health inequalities. However recent considerations establish that randomised controlled trials (RCTs) represent the ‘gold standard’ in impact evaluation of microcredit. Discussion In seeking to evidence ‘microcredit as a public health initiative’ in a Western context we question the notion of trials as the apparent gold standard for microcredit evaluations by comparing this approach to developments in thinking about study design for complex public health interventions. This field has moved onto a more eclectic approach to study design for complex interventions with recognition given to insights drawn from non-randomised study designs and qualitative studies. Furthermore, we consider ethical issues currently missing from the global debate on impact assessment, raised by the notion of mounting microcredit RCTs in Western contexts. These issues relate to equipoise and the screening devices used for randomisation and raise the question of whether it would ethically be possible to conduct a microcredit RCT in a Western context. Summary RCTs represent an attempt to improve the internal validity of microfinance impact evaluations. However, questions remain regarding the appropriateness of adopting this method to the exclusion of other important sources of evidence. Thus while microfinance may hold potential in the field of public health, the sector may first have to heed methodological lessons from public health to enhance its evidence base. 2 Background The importance of identifying upstream public health interventions has heightened given the general acceptance that there are limits to population health improvement through traditional means [1, 2]. Microcredit, the provision of small loans to those excluded from mainstream financial institutions, has been portrayed in this regard, with papers published in public health journals [3, 4]. Furthermore, such initiatives have even been spreading to more-advanced economies, due to, not only, being seen as an alternative means for stimulating economic growth in deprived communities, but also for potentially, in the longer-term, acting on persistent and growing health inequalities [5, 6]. This is a very positive development, as microcredit can be seen as part of a set of initiatives which take us beyond the mid-stream path of individual risk factor reduction and towards tackling the genuine upstream ‘causes of the causes’ of illness. Ironically, this recognition comes when randomised controlled trials (RCTs) have been adopted as somewhat of a ‘gold standard’ by many academics in the microcredit field, and, moreover, trial evidence questions the credibility of claims for the impact of microcredit. The aim of this paper is to call the adoption of RCTs as a ‘gold standard’ into question, especially in the context of mounting such studies in Western contexts, which is where our interest lies. In seeking to evidence ‘microcredit as a public health initiative’ we have been contemplating how the notion of trials, as the apparent gold standard for microcredit evaluations, compares with developments in thinking about study design in public health. The basis of our case is that, given developments in the evaluation of complex community based public health initiatives, this field has moved onto a more eclectic approach to study design for complex interventions [7]. Thus, as well as microcredit having a potential role as a public health initiative, public health, in return, may provide methodological lessons to the microcredit research sector where the pursuit of trial-based evidence alone, we contend, may be hampering the production of relevant evidence on its public health (and other wider) impacts. Randomised trials of microfinance The use of RCTs in development economics and, particularly in the microfinance arena, continues to increase in popularity. As seen in Box 1 [8-10], recent initiatives such as the Abdul Lateef Jameel Poverty Action Lab (J-PAL), Innovations for Poverty Action (IPA) and the International Initiative for Impact Evaluation (3ie) have promoted the use of RCTs as the best way of assessing impact in development. As a result, RCT-based impact evaluations on microfinance programmes have multiplied; recently a full issue of the American Economic Journal was dedicated to microfinance RCTs [11]. Since the first randomised evaluation of microcredit took place [12], these have been applied to evaluate not only the impact of microloans, but also several other related financial initiatives and credit-plus programmes [3, 4, 13-25]. The growth of RCTs has, in part, been a reaction to concerns around the quality of initial microcredit impact evaluations, specifically in disentangling causation from correlation [11, 26]. Further, the proliferation of RCTs reflects a growing trend in other areas of development research where evaluations of social interventions seek to replicate methods previously thought to be largely the domain of the medical field. Critical papers have been written on the use of RCTs in development and microfinance [2734]. Most of these critiques focus on the lack of generalisability of RCT results [27], but 3 others reflect on flaws with respect to their internal validity and the difficulties of replicating clinical-type experiments in a broader social context [8, 28]. It has also been recognised that the economics literature on RCTs has, in the past, ignored reports on mistakes in the design, conduct and reporting of medical trials [29], some of which is related to randomisation no longer being considered the ‘gold standard’ in public health research [8, 30, 31]. Likewise there are serious ethical implications to consider in RCTs of socio-economic interventions [29, 30]. It is these issues, of the gold standard and ethics, that we wish to build upon in reflecting on how best to generate evidence on the impacts of microcredit. Discussion Learning from public health: the move from RCTs as the gold standard The clinical origins and application of RCTs guarantee their credibility and rigour amongst the scientific community. The enthusiasm with which they have been adopted by social scientists has contributed to the belief, particularly amongst development economists, that this study design overrides any other. However, advocates for the use of RCTs in the social sciences are formulating arguments that are very similar in nature to the claims that were made when the evidence-based medicine movement began in the 1980s and 1990s. Since RCTs started to be used in medicine around 60 years ago [29], their legitimacy as the ‘gold standard’ has dissipated and the ‘weight’ assigned to this research method has been rebalanced in favour of more epidemiological, or observational, approaches. This is particularly so in the area of public health, where the causal chains that exist between the intervention and the outcome are much more complex than in clinical interventions [32]. Context tends to matter more, and the results of RCTs cannot be easily extrapolated to similar interventions in different areas. Additionally, the causal pathways for public health interventions involve behavioural responses that go beyond the strictly biological ones that exist in clinical research [30]. With causal pathways similar in complexity, microfinance could benefit from following the evolution in public health research, as documented in the guidelines on ‘Developing and evaluating complex interventions’ produced by prestigious UK funder, the MRC [7]. This new phase for public health research, reflected in the MRC guidelines, highlights the need to go beyond scientific rationalism to tackle modern problems such as obesity and mental health. A significant contribution to this debate identifies as a starting premise that population health improvement is conditional on a health-promoting societal context characterised by a culture in which healthy behaviours are the norm, with supportive institutional, social, and physical environments [33]. This implies that environmental and social factors play a determining role in understanding mediating mechanisms that underlie the intervention-effect relationship. There is a general recognition that RCTs, on their own, cannot achieve a full understanding of such relationships. Research methods in epidemiology, the core science for public health, have been adapting to these shifts in the nature and main purpose of public health and the limitations of the oftassumed linear pathway, generating new and wider frameworks [34]. In public health, as in social interventions such as microfinance, research evaluation methods need to be able to assess effects deriving from complex pathways. While RCTs can identify direct short-term effects reasonably well, their efficacy falls when the routes from the intervention to the effects are more indirect and circuitous. This makes other research methods equally necessary and valid to evaluate a complex intervention, as these might identify subjective 4 and environmental correlates of behaviour that could inform the design of interventions in different contexts. An example of this can be found in the public health literature with the successful sudden infant death syndrome (SIDS) campaign ‘Back to Sleep’. The success of this campaign was identified through observational evidence that highlighted behavioural factors (position of baby) and poverty as risk factors for SIDS. Moreover, the success of this campaign could have been established two decades earlier if non-trial evidence had been synthesised in the same way that RCT evidence is [34]. Thus, microfinance researchers need to be critical and aware of the limitations of RCTs, particularly in the area of complex interventions, as well as of the fact that RCTs are not always feasible [11] or present severe ethical dilemmas. Ethical dilemmas in Western contexts As well as considering whether an RCT approach is appropriate to answer a particular research question in a given context, ethical issues associated with such a study must be examined. Key issues to consider from an ethical standpoint are those of equipoise and consent [35]. While literature has addressed the serious ethical implications that RCTs of socio-economic interventions present in practice, we feel considering the issue of equipoise in relation to the mounting of microcredit RCTs in Western contexts is illustrative and adds to this debate. Equipoise exists at the personal or clinical level and reflects uncertainty regarding the outcome of an intervention. This uncertainty, within personal and clinical equipoise, has been described as “the principle, moral and practical, required to justify ethically a randomised controlled trial” [36]; it is key to the staging of an ethical RCT. Personal equipoise is the uncertainty of an individual physician regarding the outcome of a trial [35]. Awareness of an inferior treatment option combined with the physician’s ethical obligation to treat patients to the best of their ability would constitute a moral obligation for a physician to prevent a patient from partaking in a RCT. Similarly, clinical, or collective, equipoise reflects uncertainty or indifference regarding treatment types amongst the profession as a whole – no consensus exists concerning a preferred mode of treatment [35, 37, 38]. This begs the question as to what this means for randomised trials of microcredit. Precisely who is in equipoise and what is the relevance of this for policy? Of course, government and donors may be in equipoise and require best evidence about what works best for lowincome communities. However, in policy terms, is it really the case that consideration could be given to stopping financial services which, as indicated by demand, are taken up by lowincome groups? Furthermore, should at risk groups be denied access to credit-plus programmes that offer a health intervention alongside a loan for income generating activities until the end of a study period? On the ground, it is doubtful whether individual credit officers (in many respects the corollary of the physician in the medical context) would be in equipoise. Anyone would likely be judged as having the right to be considered for a loan, and, also, to receive such a loan should the credit officer judge them to be creditworthy. This, in part, might explain why some individual RCT designs were typically implemented using a form of randomisation connected to credit scoring in which randomly selected applicants previously deemed marginal rejections were reconsidered; thus applicants received a microloan who otherwise would have been rejected [4, 13, 14]. However, giving 5 loans to those who have failed to meet screening requirements could also be judged ethically dubious, the medical parallel being to offer inappropriate treatment to a patient for the sake of research. Likewise, cluster randomisation (e.g. randomising villages [12]) might be seen as a way of conducting a trial whilst avoiding the need to randomise at the individual level. The problem with this becomes one of relevance – the need to look at results at the cluster level when the uptake of microcredit at such a level might not even be considered by many in the population, thus dampening any potential impacts observed. Given the contextual nature of many RCTs of social and public health interventions, such as microcredit, it is then doubtful whether results from RCTs will bring resolution to equipoise, due to a lack of external validity [39, 40]. Thus serious questions remain around the ethics of subjecting trial participants to research where the value of the results could be questioned, especially given people’s rights to access such services and the possibility of error in judgments arising from trial results. It is our belief that these considerations call into question whether it would be ethically possible to conduct a microcredit RCT in a Western context. These arguments are of course relevant to all forms of microcredit RCTs and thus a further question is raised of why individuals in Western contexts are treated differently to those in developing countries? Summary Recognition must be paid to the methodological development that RCTs represent in the microcredit research sector in terms of seeking to improve the internal validity and so quality of impact evaluations. However, as this article outlines, questions remain regarding the appropriateness of adopting this method to the exclusion of other important sources of evidence. Our purpose has been to draw attention to these questions by focusing on issues preventing the implementation of RCTs in Western contexts rather than to argue against rigour in impact evaluations. Accordingly, we reiterate the call for more rigorous microcredit impact evaluation studies that are suitably designed to correspond to their particular setting and see RCTs as an important part of that [41]. We also recognise that despite the arguments provided regarding the benefits of implementing other (non-RCT) study designs identified in the public health research field, concerns may still exist due to the higher face validity of RCT approaches. Therefore, we conclude by highlighting methodological innovations which can enable researchers, practitioners and policy-makers to better judge and identify rigorous studies conducted using other research methods. For example, following developments in thinking within public health research, outlined previously in this article, non-randomised study designs have also achieved greater recognition within prestigious evidence-synthesis movements, such as The Cochrane Collaboration [42]. Recognising the importance of moving beyond RCTs, corresponding alternative synthesis approaches to the ‘specialist’ synthesis method of meta-analysis have been developed, such as narrative synthesis [43, 44]. Furthermore, of particular relevance to this article are the specialist tools available to assess the quality of non-randomised study designs [45]. Specifically tools now exist in public health research, such as that developed by the Effective Public Health Practice Project, which enable the methodological quality of randomised and non-randomised study designs to be assessed [46]. Such developments mean tools now exist whereby the quality of randomised and nonrandomised study designs can be compared. Thus the rigour and quality of non-randomised study designs can be assessed in relation to RCTs to allow informed judgements about how much weight to attach to the results of particular studies. 6 These developments stem from the notion of systematic reviews, commonly used in the medical field to summarise the results of randomised trials in order to examine the effectiveness of an intervention in attaining a particular impact. Ironically, although not prevalent, their use has recently become more common within the microcredit research sector [47-51] with increasing recognition given to the benefits of pooling studies and undertaking formal synthesis [11]. This itself is a significant development given that systematic reviews, in general, are noted for their ability to inform policy and practice [52]. In summary, we do recognise that use of RCTs in the microfinance sector may be suitable for answering specific research questions, in specific contexts but importantly this method is not suitable for examining all potential areas of impact. This view is beginning to be acknowledged by others within the sector who are realising that for particular outcomes of interest, such as health, RCTs may not be appropriate due to longer gestation periods requiring longer study designs which may cause problems withholding participants’ access to microcredit [11]. As such it is our belief that other study designs such as panel data methods, observational studies and qualitative methods, which can form part of the required preparatory work to an evaluation to determine adequacy or plausibility, should also be seen as a necessary part of any subsequent main study. The lessons and tools garnered from the public health research field, identified here, illustrate that the quality and rigour of non-randomised study designs can be upheld and that the insights obtained from their use mean they deserve the same level of attention as has been bestowed on RCTs. Heeding such methodological lessons would enable the evidence base around microfinance to be enhanced and increase its potential in the field of public health. Authors’ contributions NM, OB and CD contributed to the conception and writing of this article. Furthermore, NM, OB and CD read and approved the final version. Declaration of interest All authors declare that there were no conflicts of interest in the writing of this article. Role of funding source The authors declare that there were no funding sources connected to the writing of this article. Acknowledgements This article emerged from conversations with colleagues at the Yunus Centre for Social Business and Health, Glasgow Caledonian University. We, the authorship team, thank these colleagues for their insights and critical conversation. 7 Box 1 “Creating a culture in which rigorous randomized evaluations are promoted, encouraged, and financed has the potential to revolutionize social policy during the 21st century, just as randomized trials revolutionized medicine during the 20th“ (Duflo, The World Bank is finally embracing science, 2004 Lancet editorial) (in Deaton 2010)5 “The cleanest way to answer these questions is to mimic the randomised trials that are used in medicine to evaluate the effectiveness of new drugs” (Poor Economics, pg. 8, Banerjee and Duflo)6 “IPA uses randomized evaluations to measure impact because they provide the most credible and reliable way to learn what works and what does not. Randomized evaluations use the same methods frequently used in high quality medical research and rely on the random assignment of a program or policy to measure its impact.” (IPA’s website)7 References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. Commission on Social Determinants of Health: Closing the gap in a generation: Health equity through action on the social determinants of health. Edited by Final Report of the Commission on Social Determinants of Health. Geneva: World Health Organisation; 2008. Marmot M: Fair Society: Healthy Lives. Strategic Review of Health Inequalities in England Post-2010. In The Marmot Review. 2010. Hamad R, Fernald LC, Karlan DS: Health education for microcredit clients in Peru: a randomized controlled trial. BMC public health 2011, 11(1):51. Fernald LCH, Hamad R, Karlan D, Ozer EJ, Zinman J: Small individual loans and mental health: a randomized controlled trial among South African adults. BMC Public Health 2008, 8(409):1 - 14. Mohindra KS, Haddad S: Women's Interlaced Freedoms: A Framework Linking Microcredit Participation and Health. Journal of Human Development 2005, 6(3):353 - 374. Donaldson C, Baker R, Cheater F, Gillespie M, McHugh N, Sinclair S: Social business, health and well-being. Social Business 2011, 1(1):17-35. Medical Research Council: Developing and evaluating complex interventions: new guidance. Medical Research Council; 2008. Deaton A: Instruments, randomization, and learning about development. Journal of Economic Literature 2010, 48:424-455. Banerjee A, Duflo E: Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty PublicAffairs; 2011. About IPA [http://www.poverty-action.org/about] Banerjee A, Karlan D, Zinman J: Six Randomized Evaluations of Microcredit: Introduction and Further Steps. American Economic Journal: Applied Economics 2015, 7(1). Banerjee A, Duflo E, Glennerster R, Kinnan C: The Miracle of Microfinance? Evidence from a Randomized Evaluation. American Economic Journal: Applied Economics 2015, 7(1):22-53. Karlan D, Zinman J: Expanding Credit Access: Using Randomized Supply Decisions to Estimate the Impacts. The Review of Financial Studies 2010, 23(1):433-464. Karlan D, Zinman J: Microcredit in Theory and Practice: Using Randomized Credit Scoring for Impact Evaluation. Science 2011, 332:1278-1284. 8 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. Karlan DS, Zinman J: Credit elasticities in less-developed economies: Implications for microfinance. The American Economic Review 2008, 98(3):1040-1068. Field E, Pande R, Papp J, Park YJ: Repayment flexibility can reduce financial stress: a randomized control trial with microfinance clients in India. PloS one 2012, 7(9). Field E, Pande R, Papp J, Rigol N: Does the classic microfinance model discourage entrepreneurship among the poor? Experimental evidence from India. The American Economic Review 2013, 103(6):2196-2226. Giné X, Karlan DS: Group versus individual liability: Short and long term evidence from Philippine microcredit lending groups. Journal of Development Economics 2014, 107:65-83. Ashraf N, Karlan D, Yin W: Female empowerment: Impact of a commitment savings product in the Philippines. World Development 2010, 38(3):333-344. Ssewamala FM, Ismayilova L, McKay M, Sperber E, Bannon Jr W, Alicea S: Gender and the effects of an economic empowerment program on attitudes toward sexual risk-taking among AIDS-orphaned adolescent youth in Uganda. Journal of Adolescent Health 2010, 46(4):372-378. Pronyk PM, Hargreaves JR, Kim JC, Morison LA, Phetla G, Watts C, Busza J, Porter JDH: Effect of a structural intervention for the prevention of intimate-partner violence and HIV in rural South Africa: a cluster randomised trial. The Lancet 2006, 368:1973-1983. Kim JC, Watts CH, Hargreaves JR, Ndhlovu LX, Phetla G, Morison LA, Busza J, Porter JDH, Pronyk P: Understanding the Impact of a Microfinance-Based Intervention on Women's Empowerment and the Reduction of Intimate Partner Violence in South Africa. American Journal of Public Health 2007, 97(10):1 - 9. Pronyk PM, Harpham T, Busza J, Phelta G, Morison LA, Hargreaves JR, Kim JC, Watts CH, Porter JD: Can social capital be intentionally generated? A randomised controlled trial from rural South Africa. Social Science & Medicine 2008, 67:1559 - 1570. Jan S, Ferrari G, Watts CH, Hargreaves JR, Kim JC, Phetla G, Pronyk PM: Economic evaluation of a combined microfinance and gender training intervention for the prevention of intimate partner violence in rural South Africa. Health policy and planning 2011, 26(5):366372. Karlan D, Valdivia M: Teaching entrepreneurship: Impact of business training on microfinance clients and institutions. Review of Economics and Statistics 2011, 93(2):510527. Roodman D, Morduch J: The impact of microcredit on the poor in Bangladesh: Revisiting the evidence. Journal of Development Studies 2013, 50(4):583-604. Cartwright N: A philosopher's view of the long road from RCTs to effectiveness. The Lancet 2011, 377(9775): 1400-1401. Shaffer P: Against excessive rhetoric in impact assessment: overstating the case for randomised controlled experiments. Journal of Development Studies 2011, 47(11):16191635. Eble A, Boone P, Elbourne D: Risk and Evidence of Bias in Randomized Controlled Trials in Economics (January 6, 2014). In Mimeo. Brown University; 2013. Barrett CB, Carter MR: The power and pitfalls of experiments in development economics: some non-random reflections. Applied Economic Perspectives and Policy 2010, 32(4):515548. Cartwright N: Are RCTs the gold standard? BioSocieties 2007, 2(1):11-20. Worrall J: Evidence in medicine and evidence-based medicine. Philosophy Compass 2007, 2(6):981-1022. Davies SC, Winpenny E, Ball S, Fowler T, Rubin J, Nolte E: For debate: a new wave in public health improvement. The Lancet 2014, 384(9957):1889-1895. Ogilvie D, Craig P, Griffin S, Macintyre S, Wareham NJ: A translational framework for public health research. BMC Public Health 2009, 9(1):116. 9 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. 45. 46. 47. 48. 49. 50. 51. 52. Toroyan T, Roberts I, Oakley A: Randomisation and resource allocation: a missed opportunity for evaluating health care and social interventions. Journal of Medical Ethics 2000, 26:319-322. Enkin MW: For and against Clinical equipoise and not the uncertainty principle is the moral underpinning of the randomised controlled trial. British Medical Journal 2000, 321:756-758. Edwards SJL, Lilford RJ, Hewison J: The ethics of randomised controlled trials from the perspectives of patients, the public, and healthcare professionals. British Medical Journal 1998, 317:1209–1212. Cook C, Sheet C: Clincial equipoise and personal equipoise: two necessary ingredients for reducing bias in manual therapy trials. Journal of Manual and Manipulative therapy 2011, 19(1):55-57. Osrin D, Azad K, Fernandez A, Manandhar DS, Mwansambo CW, Tripathy P, Costello AM: Ethical challenges in cluster randomized controlled trials: experiences from public health interventions in Africa and Asia. Bulletin of the World Health Organization 2009, 87:772779. Deeming C: Trials and Tribulations: The ‘Use’ (and ‘Misuse’) of Evidence in Public Policy. Social Policy and Administration 2013, 47(4):359-381. Hulme D: Impact Assessment Methodologies for Microfinance:Theory, Experience and Better Practice. World Development 2000, 28(1):79 - 98. Higgins JP, Green S: Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. Cochrane Collaboration; 2011. Pope C, Mays N, Popay J: Synthesizing Qualitative and Quantitative Health Evidence. A guide to methods: Open University Press; 2007. Popay J: Moving beyond effectiveness in evidence synthesis: Methodological issues in the synthesis of diverse sources of evidence. National Institute for Health and Clinical Excellence,; 2006. Deeks JJ, Dinnes J, D’Amico R, Sowden AJ, Sakarovitch C, Song F, Petticrew M, Altman DG: Evaluating non-randomised intervention studies. Health Technology Assessment 2003, 7(27). Quality assessment tool for quantitative studies [http://www.myhamilton.ca/myhamilton/CityandGovernment/HealthandSocialServices/Res earch/EPHPP.] Stewart R, van Rooyen C, Dickson K, Majoro M, de Wet T: What is the impact of microfinance on poor people? A systematic review of evidence from sub-Saharan Africa. In Technical Report. Edited by EPPI-Centre SSRU, University of London; 2010. van Rooyen C, Stewart R, de Wet T: The Impact of Microfinance in Sub-Saharan Africa: A Systematic Review of the Evidence. World Development 2012, 40(11):2249–2262. Duvendack M, Palmer-Jones R, Copestake JG, Hooper L, Loke Y, Rao N: What is the evidence of the impact of microfinance on the well-being of poor people? . London:: EPPI-Centre; 2011. Arrivillaga M, Salcedo JP: A systematic review of microfinance-based interventions for HIV/AIDS prevention. AIDS Education and Prevention 2014, 26(1):13-27. Vaessen J, Rivas A, Duvendack M, Palmer-Jones R, Leeuw F, van Gils G, Lukach R, Holvoet N, Bastiaensen J, Hombrados JG et al: The Effects of Microcredit on Women’s Control over Household Spending in Developing Countries: A Systematic Review and Meta-analysis. Campbell Systematic Reviews; 2014. Dixon-Woods M, Bonas S, Booth A, Jones DR, Miller T, Sutton AJ, Shaw RL, Smith JA, Young B: How can systematic reviews incorporate qualitative research? A critical perspective. Qualitative Research 2006, 6(1):27-44. 10 Yunus Centre for Social Business and Health Glasgow Caledonian University Cowcaddens Road Glasgow G4 0BA Scotland, United Kingdom T: +44 (0) 141 331 8330 E: [email protected] www.gcu.ac.uk/yunuscentre Glasgow Caledonian University is a registered Scottish charity, number SC021474, © Glasgow Caledonian University 2014.
© Copyright 2026 Paperzz