United Nations Educational, Scientific and Cultural Organization UNESCO Education Research and Foresight Occasional papers 02 March 2012 Beyond the Conceptual Maze The notion of quality in education Concern with quality improvement in education Conceptualizations of quality and analytical frameworks Implications for assessing the quality of education ISSN 2304-7747 Sobhi Tawil Abdeljalil Akkari Beatriz Macedo Senior Programme Specialist Education Research & Foresight UNESCO Professor Faculty of Psychology and Education University of Geneva Programme Specialist Education Research & Foresight UNESCO Abstract Discussions relative to the quality of education often remain blurred by the lack both of clarity, as well as of a common understanding of what is actually meant by the term quality. Arguably, this has to do with the fact that, rather than an operational concept, quality in education is a notion which commands a seemingly intuitive understanding. As such, there is no single definition or approach, but rather diverse possible conceptualizations and multiple approaches, each based on widely differing assumptions. This paper thus proposes to take stock of some of the conceptualizations of the notion of quality in education, and of possible analytical approaches as well as their underlying assumptions. This appears crucial to UNESCO’s support of Member States as the notion of quality frames the organization’s efforts for the development of education worldwide. While this paper will focus on conceptualizations developed while monitoring progress towards the Education for All (EFA) goals, it will also refer to other approaches put forward to better understand and examine the quality of learning and the performance of education systems. In reviewing these, this paper offers three categories of frameworks in what might be called (i) the Learner-centred Approach, (ii) the Inputs-process-outputs Approach, and (iii) the Multidimensional Social Interaction Approach. The purpose of this paper is not to propose yet another model to conceptualize quality in education, but rather to chart a way through the conceptual maze by reviewing the array of existing approaches. 2 UNESCO Education Research and Foresight • Occasional papers > Concern with quality improvement in education The centrality of the notion of quality in the framing of UNESCO support in the area of education reflects the progressive shift in the discourse on international education over the past decade. While international focus on the quality of education appears to be recent, discussions at the 1990 World Conference on Education for All (WCEFA) in Jomtien had already placed significant emphasis on learning achievement and had guarded against focusing too narrowly on issues of access, retention and completion. In spite of this, the Education for All (EFA) movement that followed in the 1990s largely placed its focus on expanding access to basic education in general, and to primary schooling in particular. This quantitative bias was clearly a response to the declining trends in primary enrolment observed in 25% of developing country contexts in the 1980s1. Since 2000, however, the notion of quality has permeated the discourse on international education, in line with the explicit goal formulated in the 2000 Dakar Framework for Action: →→ … improving all aspects of the quality of education and ensuring excellence of all so that recognized and measurable learning outcomes are achieved by all, especially in literacy, numeracy and essential life skills.2 The quantity-quality trade-off in international education development This more recent prominence of the theme of quality since 2000 is undoubtedly a reaction to the more quantitative vision that prevailed during the 1990s, when emphasis was predominantly placed on expanding access to primary education. During that time, much of the focus in educational development had been on increasing the number of children and youth able to access basic education, with much less attention being paid to the outcomes of the educational process. Even today, and despite the formidable progress observed in expanding access to basic schooling in much of the developing world since 2000, including in many of the poorest countries, higher net primary enrolment rates have yet to fully translate into higher literacy rates in many countries.3 This is largely due to the high dropout rates observed in many contexts signifying that many children do not complete the full primary education cycle and, consequently, do not acquire basic literacy and numeracy skills. Moreover, and perhaps more worryingly, one can observe that completion of a full cycle of primary education of poor quality does not always ensure the acquisition of basic numeracy and literacy skills.4 As a result, patterns of primary school dropout and the often low quality of learning in primary schools observed 1 Berstecher & Carr-Hill (1990). 2Dakar Framework for Action (2000). 3 See report of the Secretary-General of the United Nations on “Implementing the Internationally Agreed Goals and Commitments in Regard to Education” presented at the 2011 ECOSOC Annual Ministerial Review (Geneva, 4-8 July 2011). in many countries worldwide contribute to the reproduction of illiteracy among young adults. The dilemma thus became increasingly framed in terms of a quantity-quality trade-off; that is, that the expansion of education systems and opportunities to learners thus far excluded, and often belonging to more disadvantaged communities, had translated into an overall decline in average levels of learning acquisition at the end of the primary education cycle. Quality improvement as a broad concern within national education reforms This shift in the international education discourse is also reflected in the increased concern with improving the quality of education shown in national education reforms worldwide. The rationales used to justify increasing concern with quality, relevance, results, and performance in national education and training systems are many and wide-ranging. Rationales for quality improvement are based, firstly, on the need to increase the effectiveness of national education and training systems, as they are essential levers for societal development in an increasingly globalized world; secondly, on the need to reduce discrepancies observed between educational outlays and the relevance of what is actually learned to the lives of learners, their families and their communities in particularas well as to national development efforts more broadly. Rationales for improving quality in the discourse on national education reform also have to do with the need to reduce the observed inequalities in performance between schools within education systems. The disparities observed in the social and gendered distribution of learning outcomes among various categories of the population also need to be addressed. Finally, these rationales are also related to concerns regarding formal education systems’ capacity to deal with the growing diversity and complexity of national societies, resulting from the increasing pace of migrations, urbanization, cultural globalization, and the broadening access to expanding sources and channels of transmission of information, knowledge and values.5 Growing interest in external assessments of learning outcomes This shift in discourse is concurrent with a growing interest in international and national assessments of learning outcomes. Traditional approaches to the quality of education rely 5 See, for example, Pigozzi (2006) in Ross & Jurgens Genevois (Eds.) Crossnational studies of the quality of education. Paris: IIEP pp. 39-49. 3 UNESCO Education Research and Foresight • Occasional papers essentially on proxy indicators relative to educational provision and inputs that affect the teaching and learning process; there is, however, a growing interest in the results of the process in terms of the levels and distribution of cognitive skills acquired. Indeed, beyond the mere delivery of more schooling to greater numbers, the levels, relevance, and distribution of skills that children and youth acquire through formal education could be said to constitute the most immediate measure of the success or failure of individual and collective investment in education. It is within this perspective that international and regional assessments such as the Programme for International Student Assessment (PISA), the Trends in International Mathematics and Science Survey (TIMSS), the Progress in International Reading Literacy Study (PIRLS), the Programme d’Analyse des Systèmes Educatifs de la CONFEMEN (PASEC), the Southern and Eastern Africa Consortium for Monitoring Educational Quality (SACMEQ) and the Second and Third Regional Comparative and Explanatory Studies of student achievement in Latin America and the Caribbean (SERCE, TERCE) have been testing levels of cognitive achievement in mathematics, science and language skills.6 Moreover, an increasing number of countries are implementing large-scale external national assessments to test achievement in various areas of learning, and at different levels of schooling. The data produced by such large-scale assessments aims to answer three main sets of questions: (i) what are learners actually learning? what knowledge do they possess? what do they master in terms of skills? (ii) Are education systems performing well in producing the intended learning outcomes? (iii) What are the characteristics of students’ learning environments that explain this performance? A narrow focus on cognitive learning outcomes But such large-scale external assessments based on standardized testing have also been criticized for a number of reasons. Many have argued that such testing narrowly reduces the vision of results of educational processes. Indeed, standardized testing tends to focus on more easily measurable ‘academic’ learning outcomes, neglecting other important social dimensions of learning. Having said this, there is increasing worldwide effort to measure value orientation, attitudes and emotional development, despite the important methodological challenges this represents.7 6PISA measures mathematics, science and reading skill levels among 15 year olds in OECD countries and beyond; TIMSS measures mathematics and science skill levels among 9 and 13 year olds; PIRLS measures reading skill levels and attitudes among 9 year olds; PASEC measures reading and math skill levels among grade 2 and grade 5 pupils; SACMEQ measures reading and math skill levels and knowledge of basic health issues among grade 6 pupils; SERCE (2008) measured language and mathematics skill levels and knowledge of natural sciences among pupils in grades 3 and 6. The Third Regional Comparative and Explanatory Study (TERCE) of student achievement in Latin America and the Caribbean was in preparation at the time of writing. 7An important example of this is the International Civic and Citizenship Education Study (ICCS) conducted by the International Association for the Evaluation of Educational Achievement (IEA) in 2008 and that measures knowledge of, understandings in, and attitudes towards civic and citizenship education among 14 year olds (grades 8 or 9). Rankings and the risk of the “commodification” of education Moreover, it has been argued that cross-national assessments encourage a focus on the international ranking of education systems, resulting in a temptation to propose “model” schooling systems whose characteristics could theoretically be exported to other contexts. From a methodological point of view, the scale and the margin of error inherent to national scores combined with the weak range of performance observed between more developed countries renders any international ranking of participating countries, on the basis of a standard measure of skill levels, meaningless.8 The question then is not to so much to use such data to naïvely attempt to identify the characteristics of an effective schooling model to implement/ impose in widely differing contexts and educational traditions, but rather to aim for a better understanding of how the achievements of specific categories of students are related to characteristics of their learning environment in particular contexts. Finally, more radical critiques “challenge the sacrosanct idea that learning and intellectual capacity can and should be measured like any other quantifiable commodity” and question the pervasive role of “edumometer” technology in public education. In the context of more developed countries, for instance, the use of national data assessment to compare and rank educational institutions can, in some contexts, run the risk of the “commodification” of education not only in the “transformation of standardized tests into data points, but also in the way that aggregations of these scores at the school level are used to compare schools as though they were educational production sites.”9 Beyond rankings: generating data on system equity and effectiveness While cross-national data resulting from the assessment of learning outcomes are not devoid of their shortcomings, both conceptually and methodologically, they nevertheless represent a unique body of comparative evidence. This evidence can be used to assess the average performance of national education systems over time, the degree of equity in the social distribution of learning outcomes, and to identify possible factors and system characteristics that determine both the level and distribution of these outcomes. Indeed, more recent analyses of such assessment data also seek to gain a better understanding of the social distribution of learning, and the systemic and contextual determinants of the performance of systems over time.10 Indeed, beyond the initial fervour around cross-national comparisons and rankings, more recent analyses of PISA results, for instance, illustrate the usefulness of these large-scale assessments in plotting inequality in the social distribution of learning results over time. A 2009 analysis of the social distribution of PISA results in France, for instance, 8 Baudelot & Establet (2009: 19). 9 Corbett (2008). 10 See, for example, Mourshed, Chijoke & Barber (2010), or the current World Bank Benchmarking Education Systems for Results (SABER) initiative being piloted in the Asia-Pacific region. 4 UNESCO Education Research and Foresight • Occasional papers indicated that while the average levels of learning outcomes are rising, inequalities are also becoming wider. This leads the authors to conclude that while French schooling contributes satisfactorily to the formation of elites, it fails to ensure a sound level of basic skills to all learners.11 Other research based on learning assessment data has been exploring the performance of school systems over time. A recent study on school improvement in twenty countries has attempted to understand what cluster of factors explain progress in average levels and more equitable distribution of learning outcomes in a variety of contexts.12 Both national and international external assessments of learning outcomes have the advantage of making national authorities take a fresh look at their education systems, helping them understand their system’s strengths and weaknesses, and to differentiate between in-school, systemic, and contextual factors of improvement that could inform policy interventions.13 Changing paradigms in the pursuit of better quality education Whatever the conceptualization of quality, the analytical framework adopted, or the set of factors focused on in any particular context, the recent shift in international and national education policy focusing on quality improvement reflects a changing paradigm relative to educational processes. This change of paradigm has important implications in terms of conceptualizations of the learning process, the teaching/ learning methods used, curriculum design, as well as the ways in which educational processes are planned and managed. In terms of implications for teaching and learning processes, education is no longer equated solely with teaching, and more concern is being given to what is being learned and how this is being done. This is mirrored in the shift in discourse from a focus on schooling to a view of lifelong learning in which the acquisition of “basic learning tools” (as defined in the 1990 Jomtien view of “basic learning needs”) constitute the essential building blocks. Learning is no longer seen as being time-bound, and increased attention is being given to the intended outcomes of learning. This shift also translates into a greater interest in active learner-centred approaches and the construction of knowledge, the development of analytical, synthesis skills and other competencies, rather than the traditional passive teacher-centred approaches based on the transmission of information and knowledge relying heavily on memorisation and rote learning (Table 1). Table 1 Shifting Conceptualizations of Teaching and Learning14 From focus on : ➠ To concern with : Teaching Learning Schooling Life-long learning Time-bound learning Outcomes-based Transmission of information Learning to learn Passive learning Active learning Memorisation & rote learning Understanding, analysis, synthesis and application Teacher-centered approaches Learner-centered approaches Summative evaluation Formative evaluation In terms of curricula, this shift in the conceptualization of teaching and learning implies a move away from a focus on knowledge as the organizing principle of curriculum content, to one based on the development of skills and the acquisition of competencies. As a result, there is a move away from fragmented conceptions 11 Baudelot and Establet (2009: 30). 12 Mourshed, Chijoke & Barber (2010). 13 Baudelot & Establet (2009: 34). 14 Tables 1-3 are adapted from Tawil (2006) “La qualité et la pertinence de l’éducation: un défi mondial”, in : Toualbi, N. & S. Tawil (Eds). Réforme de l’éducation et innovation pédagogique en Algérie. Alger: UNESCO-ONPS, pp. 27-50. of curricula content based on compartmentalized knowledge and specialised subject areas, towards more integrated curricula content based on learning areas and possibilities of diversification of curricula at regional, local or school levels in order to ensure greater relevance (Table 2). 5 UNESCO Education Research and Foresight • Occasional papers Table 2 Shifting Conceptualizations of Curriculum Design From focus on : ➠ To concern with : Fragmented curricula Integrated curricula Specialized subjects Learning areas Content defined as knowledge Development of skills and competencies Centralized curricula Diversification: regional and local curricula Beyond teaching/learning processes and curriculum design, this shift also has implications for the planning and management of educational processes. Indeed, rather than traditional quantitative approaches that focus more on supply-side factors, inputs, access issues, and investment efforts, attention has been increasingly placed on qualitative approaches, social demand for education, system efficiency, as well as outcomes, results and impact of educational processes. Such shifting conceptualizations of teaching and learning have also often gone hand in hand with efforts to diversify educational delivery, often through more decentralized forms of planning and management, if not through greater devolution of power to the local level and greater decentralization of educational decision-making (Table 3). Table 3 Shifting Conceptualizations of Educational Planning and Management From focus on: ➠ To concern with: Supply Demand Quantitative approach Qualitative dimensions Access Quality & Relevance Inputs Outputs, Outcomes, Results & Impact Investment efforts System efficiency Centralized management Decentralization The elusive notion of quality in education “If no one asks me, I know: If I wish to explain it to one that asketh, I know not”.15 What is actually meant by the notion of quality in education? Much like Saint Augustine’s statement in reference to time, the notion of quality in education commands a seemingly intuitive understanding and yet seems to escape definition. The fact that it is a notion, rather than an operational concept, implies that it cannot be captured through any single definition or approach. This explains why diverse conceptualizations have been proposed and multiple approaches developed in an attempt to reach a common understanding of what quality education 15 Saint Augustine of Hippo on the notion of time in Book 11 of his Confessions. is actually about, how best to analyse and monitor it, and how to inform policy interventions to improve it. The section that follows thus proposes to take stock of some of the differing conceptualizations of the notion of quality in education, as well as of possible analytical approaches. The range of conceptualizations and approaches proposed often cloud our understanding in our stubborn attempt to reduce this complex and multidimensional notion to one single all-encompassing model. The purpose of the next section is not to propose yet another model to conceptualize quality in education, but rather to present and review the array of approaches that exist and their multiple possible entry points. In doing so, we hope to help chart a way through the conceptual maze resulting from the diverse attempts to approach and better understand this multi-faceted notion. 6 UNESCO Education Research and Foresight • Occasional papers > Conceptualizations of Quality and Analytical Frameworks A review of existing conceptualizations of the notion of quality in education and their corresponding analytical frameworks may be categorized into three types of approaches. The first of these is the Learner-Centred approach characteristic of UNICEF, inspired by a rights-of-the-child perspective which places the child-learner at the centre of concerns. A second set of more technical, rational approaches are variations on the Input-process-output model inspired by an industrial view of educational production, seeking to assess the quality of educational “products” and the “performance” of education systems. Finally, a third set of approaches can be described as based on a Multidimensional Social Interaction perspective, which incorporates a more sociological perspective on education as a “public good” that must necessarily take into account a range of dimensions and the dynamics of interaction between various stakeholders. This third approach implies a continuous process of redefinition regarding the consensus on what constitutes relevant education, from the multiple perspectives of diverse groups of stakeholders in a specific context at a given time. This section briefly presents these approaches, illustrating each by sample frameworks, including some of those proposed by UNESCO as contributions to the international “quality improvement debate”.16 thus resides in the fact that it distinguishes clearly between “the level of the learner in his or her learning environment” (or the “inner learner level”) and that of “the level of the education system that creates and supports the learning experience” (or the “outer system level”). In this framework, teaching and learning methods, the learning environment, as well as organisational dimensions of the system are necessarily conceived as being learner-centred. Figure 1 Framework for the Quality of Education (UNESCO 2006) Managerial and administrative system Implementation of good policies Seeks out learners A first approach may be defined as Learner-centred in that it locates learning at the centre of thinking about quality in education. Inspired by human rights norms and the rights of children in particular, this approach is sensitive to issues of inclusiveness in access to education, and educational experience alike, emphasizing: →→ … rights to education, rights in education and rights through education in equal magnitude. Rights-based frameworks not only necessitate the active elimination of all barriers to opportunities to learning, but also note that the learning experience itself has intrinsic worth and should promote child rights.17 Processes Supportive legislative framework Learning Content The Learner-Centred Approach: A rightsof-the-child perspective What the learner brings Environment Resources Means to mesure learning outcomes Source: Pigozzi (2006). The 2006 framework proposed by the former UNESCO Division for Quality Education is based on this approach (see Figure 1 below). While the primary concern of this framework is placing learning at the centre of the model, “the inputs, processes, environment and outputs that surround and foster (or hamper) learning are important as well.”18 The merit of the framework Rather than an operational model per se, the proposed framework translates a view of quality in education based on principles of inclusiveness and equity inspired by a rightsbased approach in general, and by the Convention on the Rights of the Child (CRC) in particular. This is very much in line with the view of quality education inherent to the model of Child-Friendly Schools (CFS) developed by UNICEF.19 Indeed, UNICEF (2009) suggests that key principles based on the CRC can be used to generate the desired features, or characteristics, of child-friendly schools. It is the interpretation of these CRC principles that define both the desired features of educational processes in any given context, and the CFS standards of 16 See, for example, discussion in Nikel and Lowe (2010). 17UNESCO (2011). “EFA End-of-Decade Note on Quality (Goal 6) for the Asia Pacific Region”, July 2011 (draft). 18Pigozzi (2006: 42-43). 19It is useful to point out that Mary-Joy Pigozzi had prior experience at UNICEF before serving as Director of the Division of Quality Education at UNESCO. 7 UNESCO Education Research and Foresight • Occasional papers quality education. These standards of quality relate to at least four dimensions:20 The Input-Process-Output Approach: The technical-rational vision →→ Inclusiveness and equality: Including children’s equal rights, respect for diversity and gender equality A second set of frameworks may be said to represent variations on the more technical- rational Input-process-output approach inspired by an industrial view of educational production seeking to measure the quality of educational “products” and the “performance” of education systems. While there are numerous available frameworks inspired by this approach, they are all based on a perspective in which the nature of the results of the educational process (outputs, outcomes, impact…), are determined by a number of input factors (ranging from the characteristics of learners, to curriculum, teachers and school environment…), as well as by other contextual factors. →→ Effective teaching and learning: Including teachers’ motivation and support systems, relevant curricula “infused with life skills”, student-centred teaching and learning methods →→ Safe, healthy and productive learning environments: Including skills-based safety education, the adoption of healthy growth standards, and the organization of physical education →→ Participation and harmonization in school management: Including student and teacher participation, family-community and school partnerships, and school leadership These dimensions are at the heart of the CFS Framework proposed by UNICEF (2005) which further highlights the importance of gender responsiveness by presenting it separately from the broader dimension of inclusiveness and equality. In the Asia-Pacific region, the framework was later expanded to include an additional important dimension of “systems and policies” (see Figure 2). The point of the CFS Framework is that it “provides a comprehensive understanding of quality and, perhaps more importantly, an integral approach to recognizing the diversity of learners and the contextual influences”.21 Having said this, the relevance of this general approach is constrained by the fact that it tends to focus on primary schooling, particularly in low-income countries. Figure 2 Child Friendly Schools Framework Inclusive schools Healthy, protective & safe Systems & policies The World Bank Vision and the EFA Global Monitoring Reports (GMR) Rooted in this approach, the World Bank vision, for instance, proposes a model to identify factors that determine effectiveness at the school level; factors which are related to “enabling conditions” (such as effective school leadership and a capable teacher force) that shape both the teaching and learning processes, as well as the school climate.22 These enabling conditions within the school are dependent on inputs such as support from parents and the community, curriculum and teaching/learning materials, and facilities. School effectiveness is ultimately assessed through outcomes in terms of the participation, academic achievement, social skills and economic success of students.23 The framework proposed in the first EFA Global Monitoring Report (GMR 2002) (Figure 3) is largely inspired by this model, building on the ideas developed by Heneveld & Craig (1995), as well as those of OECD/INES (2001), and Scherens (2002). Much like the World Bank vision, the GMR framework makes explicit the most tangible input and contextual factors that affect the “black box” of the teaching/learning process. The GMR 2002 framework is useful in going a step further in that it allows for a clear distinction between the levels of the classroom, the institution, and that of national policy.24 The six dimensions of child friendly schools Involving students, families and communities Effective learning Gender responsive Source: UNICEF (2005). 20 See Framework for National Child-Friendly School (CFS) Standards in UNICEF (2009). 21Idem. 22Heneveld & Craig (1995). 23 See World Bank conceptual framework relative to “Factors that Determine School Effectiveness” presented in Heneveld & Craig (1995). 24UNESCO (2002: 80). 8 UNESCO Education Research and Foresight • Occasional papers Figure 3 GMR 2002 Input-Process-Outcome Framework INPUTS School curriculum content textbooks and learning materials teacher qualifications, training, morale and commitment adequate facilities parent/community support PROCESS School climate high expectations strong leadership positive teacher attitudes safe and gender-sensitive environment incentives for good results flexibility/autonomy Student characteristics aptitude, ability perseverance/commitment nutrition and health school readiness attended ECCE gender Household/community characteristics parental attitudes household income community economic and labour market conditions cultural/religious factors OUTCOMES Achievement cognitive development literacy, generic skills good citizenship personal development positive attitudes towards learning healthy behaviour Attainment formal completion diplomas/qualifications Teaching/learning sufficient learning time active teaching methods integrated systems for assessment and feedback appropriate class size appropriate use of language Standards official learning objectives (desired outcomes) Contextual factors Macro-economic and fiscal policies, political stability, decentralization and governance, civil service quality National goals and standards for education, curricular guidelines, sources of funding and allocation, teacher recruitment/deployment Education system management Participation, progression, completion and transition Engagement and use of time Peer effects, parental support, promotion policies Source: derived from Heneveld and Graig (1995), OECD/INES (2001), Scheerens (2002). Source: UNESCO (2002). Global Monitoring Report. Education for All: Is the world on track? Several years later the 2005 GMR, devoted to the theme of quality in education, also proposed a variation on this general input-process-output framework, reorganising the various clusters of factors and offered for consideration the following five dimensions (Figure 4): learning processes as well as the learning environment, such as infrastructure and equipment, curriculum, syllabi, textbooks and other instructional material, the training and motivation of teachers, school directors, inspectors and administrators, and modalities of school management. →→ Learners’ characteristics: This first dimension encompasses characteristics of the learner and his/her household that largely determine the effectiveness of the educational process. They include characteristics such as the learners’ aptitude, health, gender, perseverance, school readiness, prior knowledge, family and socioeconomic obstacles to learning. →→ Outcomes: Outcomes of the learning process represent the most tangible expression of the results of the teaching and learning processes in terms of the levels and social distribution of literacy and numeracy skills, life skills, creative and emotional skills, values, respect for diversity and social cohesion, social well-being and impact on societal development. →→ Teaching and learning process: The process of teaching and learning is centrally located within the framework and includes essential pedagogical factors such as teaching/ learning methods, actual instructional time, assessment methods, average class size, languages of instruction, and class management strategies. →→ Facilitating inputs: This dimension includes a range of factors that directly influence and shape the teaching and →→ Context: In attempting to fully understand the dynamics of the teaching/learning process, it is essential to consider social, cultural, economic and political dimensions of the local context. These dimensions include such factors as the structure and needs of the labour market in a context of globalization, socio-cultural, linguistic and religious factors, public demand, modalities of national governance, support structures, and public resources allocated to education. 9 UNESCO Education Research and Foresight • Occasional papers Figure 4 GMR 2005 Framework for Understanding Education Quality Enabling inputs Teaching and learning Learning time Learner characteristics Aptitude Teaching methods Assessment, feedback, incentives Class size Perseverance Teaching and learning materials School readiness Physical infrastructure and facilities Prior knowledge Human resources: teachers, principals, inspectors, supervisors, administrators Barriers to learning Outcomes Literacy, numeracy and life skills Creative and emotional skills Values Social benefits School governance Context Economic and labour market conditions in the community Socio-cultural and religious factors (Aid strategies) Educational knowledge and support infrastructure Philosophical standpoint of teacher and learner National standards Public resources available for education Peer effects Labour market demands Parental support Competitiveness of the teaching profession on the labour market Globalization Time available for schooling and homework Public expectations National governance and management strategies Source: UNESCO (2005). EFA Global Monitoring Report. The quality Imperative. Paris. Development of an Analytical Framework to Monitor Quality (UNESCO) Building on and attempting to refine the input-process-output approach, UNESCO is currently testing an analytical framework to help monitor quality in general education systems. Initially inspired by a variation on the output-process-output approach, ‘quality education’ in this framework is broadly conceptualized as one which is “effective for purpose, development relevant or responsive, equitable, resource-efficient and as denoting substantive access.”25 While still in development, the framework aims at creating an instrument for systemic analysis and monitoring that attempts to go beyond prior approaches that have tended to focus on selected aspects of education inputs, most often in isolation from one another. Indeed, it is argued that: the best part, only cognitive outcomes receive attention. Even then, cognitive outcomes have been narrowly defined as test scores.26 Measuring the quality of education: Limits and challenges The most analyzed inputs are finance, teachers, curricula, school infrastructure and furniture, books and instructional materials. However, it is very rare that even these select aspects receive a comprehensive, articulated and interactive/iterative analysis. Likewise, process factors that often receive isolated attention are assessment, management and governance. For With the input-process-output approach, the conceptualization of the quality of education remains essentially a pursuit of measurement through quantitative indicators. This raises two sets of concerns. The first is technical and raises concerns about data availability and reliability, as well as about the existence of robust statistical national/local information systems to produce them. There is also the additional challenge of common definitions of indicators that allow for international comparisons and to which the International Standard Classification of Education (ISCED) is a response. Beyond this, however, the most direct, albeit limited, internationally comparable data on the quality of “outputs” or results of educational processes are arguably generated by international/regional assessments of learning outcomes. Such data, however, do not allow for a global snapshot as the various large-scale assessments concern diverse groups of participating countries. 25 Marope (2011: 9). 26Ibid., p 5. 10 UNESCO Education Research and Foresight • Occasional papers In the absence of such direct measures on a global scale, a range of possible proxy indicators for the quality of learning and the performance of education systems may be used. Indeed, remaining within the Input-process-output model, and with reference the framework of international efforts to monitor Education for All, GMR reports on a range of indictors relative to inputs (such as, total public expenditure on education as % of GNP, total public expenditure on education as % of total public spending, teachers…), to process (e.g. primary repetition, drop out and completion rates), and to outputs (e.g. youth and adult literacy rates). However, given data deficits for international comparisons, the first GMR launched in 2002, for instance, proposed the following two proxy indicators for quality, both of which are input indicators; (i) share (%) of public expenditure allocated to the primary education sector, and (ii) pupil-teacher ratios in primary education. In the most recent GMR (2011) two proxy indicators are used in monitoring progress towards EFA Goal 6 which is the input indicator of pupil-teacher ratios in primary education, as well as the process indicator of survival rate to grade 5. Beyond the issue of the availability, reliability and comparability of data relative to direct and indirect measures of educational quality, the attempt to assess the quality of educational systems solely through measurable indicators also raises broader conceptual concerns. Indeed, the focus on measurement encourages a reductionist view of the educational process, and its social and political dimensions. The use of indicators to measure the quality and performance of education systems is but one, albeit practical, approach to assessing the multiple dimensions of educational quality. It must, however, be stressed that not all dimensions of the quality and relevance of learning and the performance of education systems can be assessed solely through measurement and quantifiable indicators. The Multidimensional Social Interaction Approach: Perspectives of local stakeholders Indeed, the idea of quantifiable educational performance as translating the essence of quality in education is not shared by all educationists. In order to transcend the instrumental and technical-rational vision on which the input-processoutcome model is based, another set of approaches adopt a wider societal approach. Based on a vision of education as a public good, these approaches focus on the dynamics of interaction between the various dimensions of education and the necessary process of continuous redefinition of consensus or social contract among the stakeholders involved relative to what national education systems should achieve and how this is to be done. The ‘fabric’ of quality in education One recent illustration of such an approach is that of the ‘fabric’ of quality in education (Nikel and Lowe 2010).27 The approach proposes seven conceptual dimensions – effectiveness, efficiency, equity, responsiveness, relevance, reflexivity, and sustainability (see Figure 5 below) – arranged so as to emphasize that the quality of education is much like a ‘fabric’: that is, it is at its strongest when ‘stretched’ or maintained in tension. The framework emphasizes the need to seek a contextually relevant balance among the seven dimensions, where ‘balance’ does not imply a simple equalizing across all dimensions, even if that were conceptually possible. The needs and the possibilities for action within different educational contexts will vary and decisions must be made over what is desirable and feasible within a specific situation. The model represents a radical departure from the input-process-output model, in that it conceptualizes quality improvement in education as attempts undertaken in a context defined by tensions between different dimensions and on different systemic levels. Figure 5 The ‘Fabric’ Model of Quality in Education Efficiency Effectiveness Equiity The Quality of Education Sustainability Responsiveness Relevance Reflexivity Source: Nikel & Lowe (2010). Tikly & Barret (2007) and Tikly (2010) This explicit recognition of possible differing values amongst the various stakeholders involved in education (learners, parents, teachers, government ... is precisely what the analytical framework developed by Tikly and Barret (2007) is based on. Within this conceptualization, the issue of the quality (and effectiveness) of education cannot be understood without an overall analysis of the historical, socioeconomic, political 27Nikel, J. & J. Lowe (2010). “Talking of fabric: a multi-dimensional model of quality in Education. Compare: A Journal of Comparative and International Education, 40(5), 589-605. 11 UNESCO Education Research and Foresight • Occasional papers and cultural contexts within which a given education system is operating. It is in this perspective that the framework aims to help assess and understand the role of a given education system in the exacerbation, reproduction or narrowing of inequalities, in particular those based on traditional factors of discrimination such as gender, income, and (ethnic, linguistic or linguistic) minority status. Two elements are important in their framework. The first is the importance of the contextual factors (historical, socioeconomic, political, and cultural) that shape education policy. Secondly, the perspectives of local stakeholders, including those of teachers, are considered key in understanding quality in a given context. Indeed, the model stresses the importance of taking into consideration the perceptions of stakeholders at the local level in any attempt to improve the quality of education. The model locates the issue of good quality education at the intersection of factors that define the policy, home and community, as well as school enabling environments at the local level. These are summarized in Figure 6. Figure 6 Tikly 2010 Model of Good Quality Education Enabling policy environment • National debate • Teacher development and incentives • Headteacher training • Assessment, monitoring and evaluation • Relevant and inclusive curriculum • Textbook procurement and distribution • Targeted financial support for schools • Stakeholder engagement • Local support for schools • School based professional development • School self evaluation • Infrastructure and resources • Structured & inclusive pedagogy GOOD QUALITY EDUCATION • School governance & parental ‘voice’ • Home school links/ parental education • School meals & child health Enabling school environment • Stakeholder engagement and community ‘voice’ • Parental education • Parent support for learning • Books in the home • A place to study Enabling home and community environment Source: Tikly (2010). The UNESCO Santiago model (2007) The 2007 UNESCO Santiago model proposes five dimensions of the quality of education that attempt to capture the perspectives of the various educational stakeholders concerned and the social action that they are engaged in at the local level. In addition to the dimensions of effectiveness, efficiency and equity that characterize an educational system, the model stresses the dimension of relevance, which is analysed at two distinct levels: (i) the relevance of the educational system in responding appropriately to collective societal concerns, and (ii) the relevance (or “pertinence”) to the daily conditions of individual learners, their families and communities. The five dimensions of this model are defined in the following manner: 12 UNESCO Education Research and Foresight • Occasional papers Table 4 The Five Dimensions of the Quality of Education28 Relevance Need for educational experiences to guarantee the kinds of learning that truly prepare people for modern life, in line with the vision of four pillars of learning as proposed by the UNESCO (2006) “Delors Report”. Pertinence Flexibility of educational experiences so that they can adjust to the particular conditions of individuals, can value diversity, and can provide venues for participation. Equity The extent to which the universal right to education is effective for all, in view of creating more just societies. Effectiveness The extent to which public action is effective in expanding access to basic education (comprehensive early childhood care and education; universal completion of primary education; ensuring student learning achievement). Efficiency The extent to which public services are efficient in the use of resources for the delivery of education as a public good. Gauging relevance through the “four pillars” of learning While indicators do exist to monitor system effectiveness, efficiency and equity (as we have seen above), data on the relevance of the educational process is much scarcer. The Santiago model proposes the four pillars of learning, as outlined in Learning: The treasure within (UNESCO 1996), as a reference to gauging the relevance (and “pertinence”) of educational processes: learning to know; learning to do; learning to live together; learning to be. In doing so, the framework highlights the underlying tension between “learning to live together” (or the “socialization” function of education), and the “learning to be” (or the “individuation” function of education). Individuation may be seen as the movement of individual intellectual, emotional and spiritual differentiation and the activation of unique individual potential. This is opposed to the constraining process of socialization which may be seen as one of integration of an “asocial” being into a given social group (space) at a given moment in its evolution (time) through the activation of an arbitrary set of values, norms, beliefs, cognitive schema, knowledge, and skills encompassed in any given cultural system. It is this process of reaching consensus between stakeholders with differing perspectives, on what selection of values, norms, beliefs, knowledge, and skills which is at stake in the process of educational policy formulation within each specific context. The lifelong learning perspective It is important to stress that the four pillars of learning29 were envisaged against the backdrop of the notion of lifelong learning, itself an adaptation of the concept of Lifelong Education as initially conceptualized in the landmark Learning to Be (UNESCO 1972). As such, assessing the quality of learning 28UNESCO (2007). 29Learning to be; Learning to do; Learning to know; and Learning to live together (UNESCO 1996). cannot be reduced to schooling or to formal education and training sectors. Bearing in mind that learning happens both within and outside of school, analysis of the social and economic context of learners is essential to reaching a better understanding of quality in education.30 Nor can employability and socioeconomic integration be seen as the only purpose or result of (successful) learning. In the lifelong learning perspective, learning is about the development of the whole person: →→ It is about allowing every individual to participate in society and making our society more cohesive. Learning enables people to develop to their full potential and to play an active role in their environments. It allows them to try new things and to harness untapped talents. Along with enhancing employment opportunities and professional standing, learning lays the groundwork for fulfilment in life.31 The challenges of operationalizing a vision of lifelong learning and monitoring the quality and relevance of education through the four pillars of learning are daunting as they suppose system-wide frameworks and interdisciplinary approaches. But there have been some interesting recent attempts to do so in the Canadian and European contexts. Indeed, drawing on the pioneering work of the Canadian Council of Learning in developing the Composite Lifelong Learning Index (2010), the Bertelsmann Stiftung has developed an index for European countries based on indicators that cover the various domains of lifelong learning as defined by the four pillars of learning. These European Lifelong Learning Indicators (ELLI) and composite index are an attempt to capture at least a portion of the concept of lifelong learning in the European context. The selection of relevant indicators is necessarily conditioned by the social, economic and cultural realities of each context and it would be interesting to see how this composite index may be adapted (or not) to ensure relevance to non-European context-specific realities. 30 Resnick (1987). 31ELLI: European Lifelong Learning Indicators. Making lifelong learning tangible. (Bertelsmann Stiftung, 2010). 13 UNESCO Education Research and Foresight • Occasional papers > Implications for Assessing the Quality of Education The centrality of the issue of relevance Arguably, the subjective concept of relevance is central to the understanding of quality in education in any given context. The quality of any educational process can only be assessed from the particular standpoint of a specific category of stakeholders. It is on the basis of what they believe to be the priority goals assigned to education systems and the nature of the outcomes desired, that each category of stakeholders shapes its perception of what constitutes relevant education. Indeed, each category of stakeholders – whether learners, teachers, parents, employers, education authorities, local and/or national government officials, international aid agencies and donors – has its perception of what constitutes relevant educational processes and how effective they are in contributing to the desired outcomes they attach value to. Whose perspective on goals and outcomes of education? Parents, teachers and educational authorities, for instance, may value educational outcomes defined essentially in terms of learning achievement and the cognitive development of learners. Parents and communities also value social and economic outcomes of education such as social promotion, access to information and employment. Other stakeholders such as national government officials, employers and international aid agencies may value socio-economic development outcomes such as increased productivity of workers, economic growth, poverty reduction, reduction of inequalities, and stronger integration of national economies in globalized knowledge societies. Yet others may value the civic and political outcomes of education in terms of greater freedom, strengthened social cohesion, respect for cultural diversity, and the development of active and responsible citizenship. It is also important to stress the often overlooked perception of the learners themselves and the meaning they attribute to learning with reference to their expectations, their identity, and their prospects for the future.32 The “Nash Equilibrium”: Negotiation and consensus This understanding is clearly echoed in the Communiqué of the 2003 Ministerial Round Table on Quality Education33 which highlights the need to take into account the multiple perceptions 32 See, for example, Charlot, E. Bautier, J.-Y. Rochex (1992). 33 Communiqué issued by 141 countries at the Ministerial Round Table on Quality Education (Paris, 3-4 October 2003) organized during the General Conference of UNESCO. of different stakeholders on what constitutes relevant/quality education. Indeed, the Communiqué emphasizes the need for transparent consultations among the various stakeholders in view of reaching a consensus on what the desired educational outcomes are and on how best to reach them. Drawing a parallel with game theory, education quality in any particular context at any given point in time may be understood as the consensus that best responds to the diverse interests of the multiple stakeholders concerned and their perceptions of the relevance of education required to fulfil them. The notion of quality in education may thus be considered to be a sort of Nash Equilibrium resulting from broad and transparent consultations. The question of the quality of education can therefore not be reduced to a set of purely technical issues, but must necessarily be seen as a dynamic process of continuous adjustment resulting from negotiations between the various stakeholders involved. →→ Quality cannot be proclaimed, simply because it cannot be reduced to several general quantifiable figures and values, neither to a set of processes of “quality control”. Quality must be continuously observed, analyzed and adjusted in an interactive manner by all the stakeholders involved. Such an approach encourages all stakeholders to claim their rights and fulfill their obligations with regard to this public good.34 Social pact on education as a public good As a result, and beyond the various conceptualizations and multiple analytical approaches, the starting question in any discussion about quality in education is that of determining what the desired outcomes of any educational process are to be. What is the combination of individual and collective social, civic, economic and political outcomes that are desired in a given context? This is necessarily related to the question of the range and nature of stakeholders involved in education, their perceptions of what are the most important outcomes of the education process at the individual, local and national levels, and the necessary process of negotiation required in order to reach consensus on a social pact35 on education as a public good. Given the importance attached to educational development by such a large range of stakeholders and the significant individual and collective efforts made and resources invested in the sector, it is only legitimate that sound attempts be sought to determine how well educational provision is producing the range of desired outcomes. 34 Meyer-Bisch (2009: 11). Our translation. 35 See for example, Tedesco (1998). 14 UNESCO Education Research and Foresight • Occasional papers Assessing quality in education: beyond measurement The need for multiple and complementary approaches Having said this, assessing the desired outcomes of educational processes cannot be reduced to measurement. Indeed, not all educational outcomes can (or even should) be measured quantitatively. Even when considering one category of educational outcomes such as learning, only some types of learning outcomes (of the cognitive type) lend themselves easily to measurement. Other types of educational outcomes in particular, social, civic and political outcomes of educational processes - can and must be assessed, even if they cannot necessarily be measured. The question then is to determine what assessment methodologies are available and are most appropriate to capture the effectiveness of educational processes in producing diverse educational outcomes. As we have seen above, while dimensions such as equity and efficiency of educational systems can indeed be measured and quantified, the essential dimension of relevance cannot be measured in the same fashion, but rather, through other methodologies guided by such references as the four “pillars of learning”. In assessing the multidimensional notion of quality in education, it is thus imperative to remain open to multiple approaches, and to the necessary combination of complementary methodologies that help best capture the complexity of the issues at hand. The differing perceptions among diverse stakeholders of what constitutes the main goals and outcomes of education, and thus, the relevance of any specific educational and training processes, dictate the nature of the analysis (rights-based, technical economic or sociological), the importance of various dimensions and clusters of factors to be examined, as well as the selection of methodologies and indicators to use. It would be sterile, in our view, to oppose the three conceptualizations and approaches to quality; that is, the Learner-centred, the Input-process-output, and the Multidimensional Social Interaction approaches. If the technical-rational Input-processoutput approach dominates international discourse on the quality of education, it is most certainly because it lends itself to a relatively simple methodology in approaching a complex phenomenon. Having said this, and despite the methodological challenges that it implies, the third approach can invigorate the global debate on quality improvement in education. 15 UNESCO Education Research and Foresight • Occasional papers >References Barrett, M., Chawla-Duggan, R., Lowe, J., Nikel, J. & E. Ukpo (2006). “The Concept of Quality in Education: A review of the ‘international’ literature on the concept of quality in education”. EdQual Working Paper No. 3. University of Bristol /University of Bath. Baudelot, C. & R. Establet (2009). L’élitisme républicain. L’école française à l’épreuve des comparaisons internationales. Paris: Le seuil. Bertelsmann Stiftung (2010). ELLI: European Lifelong Learning Indicators. Making lifelong learning tangible. Blanco, R. (2007).Quality Education For All: A human rights issue. EFA/PRELAC Background Document. Santiago de Chile: UNESCO Regional Bureau for Latin America and the Caribbean. Charlot, B., Bautier, É., & J.-Y. Rochex (1992). Ecole et savoir dans les banlieues... et ailleurs. Paris: Armand Colin. Corbett, M. (2008). “The Edumometer: The commodification of learning from Galton to the PISA.” Journal for Critical Education Policy Studies, 6(1). Encinas-Martin, M. (2009). Learning Counts: International seminar on assessing and improving quality learning for all. (Paris, 28-30 October 2008). Paris: UNESCO. ECOSOC. (2011). Implementing the Internationally Agreed Goals and Commitments in Regard to Education. Report of the United Nations Secretary-General. New York. Heneveld, W., & H. Craig (1995). Schools Count: World Bank Project Designs and the Quality of Primary Education in Sub-Saharan Africa. Washington, DC: World Bank. Kellaghan, T., & V. Greaney (2001). Using Assessment to Improve the Quality of Education. Paris: UNESCO International Institute for Educational Planning. Marope, M. (2011). “General Education Quality Diagnostic/ Analysis and Monitoring Framework”. Concept Note: Paris: UNESCO (February 2011). Meyer-Bisch, P. (2009). La qualité de l’éducation: l’accomplissement d’un droit culturel dans l’indivisibilité des droits de l’homme. Fribourg: University of Fribourg (Switzerland). Mourshed M., Chijioke C. & Barber M. (2010). How the World’s Most Improved School Systems Keep Getting Better. McKinsey & Co. Nikel, J. & J. Lowe (2010). “Talking of Fabric: A multidimensional model of quality in education”. Compare: A Journal of Comparative and International Education, 40(5), 589-605. Pigozzi, M. J. (2006). “What is the ‘quality’ of education’? (A UNESCO perspective)”. in Ross & Jurgens Genevois (Eds.) Cross-national studies of the quality of education: Planning their design and managing their impact. Paris: International Institute for Educational Planning pp. 3949. Pigozzi, M. J. (2004) “The ministerial viewpoint on the quality of education”, in Prospects, no. 130, Policy Dialogue and Education, pp. 141-149. Resnick, L. B. (1987). “Learning in school and out”. Educational Researcher, 16(9), pp. 13-20. Robin, A. (2008). Education for All, the Quality Imperative and the Problem of Pedagogy. London: University of London/Institute of Education. Ross, K. N. & L. Mählck (1990). Planning the quality of education: The collection and use of data for informed decision-making. Paris: UNESCO International Institute for Educational Planning. Saito, M. & F. van Capelle (2009). “Approaches to Monitoring the Quality of Education in Developing Countries – Searching for Better Research-Policy Linkages”. Paper based on the presentation during: The International Symposium on Quality Education for All – Approaches to Monitoring and Improving the Quality of Education (Berlin, 11-12 May 2009). Paris: UNESCO International Institute for Educational Planning. Tedesco, J. C. (1998). The New Educational Pact: Education, competitiveness and citizenship in modern society. Paris, Geneva, UNESCO International Bureau of Education. Tikly, L. & A. M. Barrett (2007). Education Quality; Research priorities and approaches in the global era. Bristol: University of Bristol. UNESCO. (2011). End-of-Decade Notes for Education for All, Goal 6. Bangkok: EFA Technical Working Group for Asia Pacific Region. (September 2011 draft). UNESCO. (2007a). The State of Education in Latin America and the Caribbean: guaranteeing quality education for all: A regional report, reviewing and assessing the progress toward Education for All within the framework of the Regional Education Project (EFA/PRELAC). Santiago de Chile: UNESCO Regional Bureau for Latin America and the Caribbean. 16 UNESCO Education Research and Foresight • Occasional papers UNESCO. (2007b). Dakar +7: EFA Top Priority for Integrated Sector-Wide Policies. Dakar: UNESCO Regional Office for Education in Africa (BREDA). UNESCO. (2007c). Enhancing Learning: From access to success. Report of the first experts’ meeting: Defining areas of action (Paris, 26-28 March 2007). Paris: UNESCO. UNESCO. (2005). EFA Global Monitoring Report 2005: The quality imperative. Paris: UNESCO. UNESCO. (2002). EFA Global Monitoring Report 2002: Is the world on track? Paris: UNESCO. UNESCO. (2000). Dakar Framework for Action. UNESCO. (1990). World Declaration and Framework for Action to meet Basic Learning Needs. Jomtien, Thailand. UNICEF. (2009). Child Friendly Schools. New York: UNICEF. UNICEF. (2007). Quality of Education. New York: UNICEF. UNICEF. (2000). Defining Quality in Education. A paper presented at the meeting of the International Working Group on Education, Florence, Italy. June 2000.
© Copyright 2025 Paperzz