Towards a “Mixed Economy” of Head Teacher Development Evaluation Report to the Scottish Government on the Flexible Routes to Headship Pilot Julia Davidson, University of Glasgow Christine Forde, University of Glasgow Peter Gronn, University of Glasgow (Co-Principal Investigator) John MacBeath, University of Cambridge (Co-Principal Investigator) Margaret Martin, University of Glasgow Margery McMahon, University of Glasgow (19 September 2008) Table of Contents I. TABLES AND FIGURES................................................................................................................................. 1 I. ACRONYMS AND ABBREVIATIONS ......................................................................................................... 2 III. ACKNOWLEDGEMENTS........................................................................................................................... 3 1. INTRODUCTION ............................................................................................................................................ 4 2. EXECUTIVE SUMMARY .............................................................................................................................. 6 3. RECOMMENDATIONS ................................................................................................................................. 7 4. BACKGROUND TO THE PILOT ............................................................................................................... 10 5. THE PROGRAMME MODEL IN THE PILOT ......................................................................................... 11 5.1 DESIGN PRINCIPLES .................................................................................................................................... 11 5.2 CONCEPTION OF LEADERSHIP ..................................................................................................................... 12 6. THE STANDARD FOR HEADSHIP AND THE PILOT ........................................................................... 12 7. MANAGEMENT OF THE PILOT ............................................................................................................... 13 7.1 FLEXIBLE ROUTES OPERATIONAL GROUP ................................................................................................... 13 7.2 INTERFACE WITH LOCAL AUTHORITIES ...................................................................................................... 14 8. QUALITY ASSURANCE IN THE PILOT .................................................................................................. 15 8.1 QUALITY ASSURANCE OF COACHING.......................................................................................................... 15 8.2 QUALITY ASSURANCE OF ASSESSMENT ...................................................................................................... 16 9. SELECTION OF CANDIDATES FOR THE PILOT ................................................................................. 17 10. THE OPERATION OF THE PILOT ......................................................................................................... 19 10.1 PROGRAMME OVERVIEW: VALUE OF THE FRH ......................................................................................... 19 10.2 PROGRAMME OVERVIEW: FLEXIBILITY OF THE FRH ................................................................................ 21 10.3 PROGRAMME COMPONENTS: TEACHING ................................................................................................... 23 10.3.1 Residentials ...................................................................................................................................... 25 10.4 PROGRAMME COMPONENTS: COACHING................................................................................................... 27 10.4.1 Introduction ...................................................................................................................................... 27 10.4.2 Recruitment and Selection of Coaches ............................................................................................. 29 10.4.3 Definition of Coaching ..................................................................................................................... 30 10.4.4 Coaching to a Standard ................................................................................................................... 32 10.4.5 Support for Coaches......................................................................................................................... 34 10.4.6 Experiences of Coaching.................................................................................................................. 36 10.5 PROGRAMME COMPONENTS: ASSESSMENT ............................................................................................... 41 10.5.1 Emotional Competence Inventory .................................................................................................... 41 10.5.2 Professional Learning Plan ............................................................................................................. 43 10.5.3 Local Authority CPD........................................................................................................................ 45 10.5.4 Learning Log: .................................................................................................................................. 46 10.5.5 Portfolio (Reflective Commentary and Evidence) ............................................................................ 47 10.5.6 Field Visit ......................................................................................................................................... 53 10.5.7 Panel Presentation ........................................................................................................................... 54 2 11. THE IMPACT OF THE PILOT ................................................................................................................. 57 12. NATIONAL IMPLEMENTATION OF THE FLEXIBLE ROUTES TO HEADSHIP ......................... 64 13. CONCLUSION ............................................................................................................................................. 68 14. REFERENCES ............................................................................................................................................. 69 APPENDIX I: TERMS OF REFERENCE....................................................................................................... 71 APPENDIX II: METHODOLOGY .................................................................................................................. 73 APPENDIX III: SELECTED RESEARCH INSTRUMENTS ....................................................................... 76 1 i. Tables and Figures Figure 1: Flexible Route to the SfH: Pilot Programme Table 1: Candidates’ understandings of mentoring and coaching Table 2: Assessment Process in the FRH Pilot Programme 23 30 48 2 i. Acronyms and Abbreviations APL CPD ECI FRH FROG GTC HEI HMIe LA LL PLP OECD SEED SfH SQH Accreditation of Prior Learning Continuing Professional Development Emotional Competence Inventory Flexible Routes to Headship Flexible Routes Operational Group General Teaching Council Higher Education Institution Her Majesty’s Inspectorate of Education Local Authority Learning Log Professional Learning Plan Organisation for Economic Co-operation and Development Scottish Executive Education Department Standard for Headship Scottish Qualification for Headship 3 iii. Acknowledgements To be able to bring a report to fruition on a project of this scale requires a collective effort. For this reason, the members of the evaluation team would like to single out a number of people for particular mention. First, we are grateful to all of the candidates and coaches who agreed to take part in the evaluation. An unavoidable consequence of the pilot nature of a project is that those who volunteer to participate at times assume “guinea pig” status, both in their own eyes and those of the evaluators. The fact that the future of a pilot can stand or fall depending in part on how they perform exacerbates the sense that they are being scrutinized. For these reasons, we thank the candidates and coaches most sincerely for being so generous with their time, and also for their frankness. We wish all of them success in the next steps of their leadership journeys. Next, we would like to thank John Gunstone and Scott Brand of the Teachers Division of the Scottish Government, for their advice, patience and understanding; Rosa Murray and her colleagues at the General Teaching Council (GTC) for their assistance and hospitality during the panel assessments of the candidates; representatives of participating local authorities (LAs); and members of the Flexible Routes Operational Group (FROG), in particular the project manager, Jim Keegans, for his patience, candour, co-operation and assistance in liaising with the evaluation team members. Lastly, our deepest debt of gratitude goes to our team of transcription typists, all of whom worked under extreme pressure to meet very tight deadlines: Yvonne Cadell, Jodie Cask, Barbara Gronn, Cheryl Marshall, Kirsty Paterson, Lorna Peedle and especially Rosaleen Maguire, who shouldered the biggest burden of all, and did so with such speed, efficiency, thoroughness and grace. 4 1. Introduction From a developmental point of view, the most promising [leadership] intervention, coaching, has a mixed record when it comes to its efficacy. The research picture is unclear (Conger, 2004, p. 139). As is evident from Jay Conger’s assessment of the impact of coaching, the jury of the research community is still out. In Scotland it is a relatively recent leadership development initiative, and the centre-piece of the Flexible Routes to Headship (FRH) pilot. While the evaluation of the FRH was not entered into as a validity test of Conger’s assessment, the findings certainly have a direct bearing on his claim. As will be evident in the findings documented in this report, we believe that coaching has a significant contribution to make to the formation and development of school leaders, as does the FRH itself. Indeed, with the particular interests of Scottish schooling and its leaders in mind, our hope is that both the letter and spirit of our recommendations will be interpreted as strengthening the contribution that may be made by the availability of this alternative route. The life-span for the FRH pilot was envisaged by the then Scottish Executive as 20 months. It commenced in February 2007 and, at the time of drafting this report (June 2008), it had nearly four months left to run. The nine months allotted by the Executive for the conduct of this evaluation, therefore, fell broadly across the mid-point of that life-span. By the time of the completion of this report, about half the candidates were deemed to have satisfied the requirement for the Standard for Headship (SfH), and may be thought of as the first cohort of graduates of the pilot. The remainder of their peers are expected to complete towards the end of 2008. The evaluation team has been aware that the FRH is not the only programme of its type and that two similar schemes are operating in Scotland. Our assumption is that, either immediately or in the not too distant future, the option for teachers to pursue a flexible route to the SfH will be available nationally, if this is seen as serving their development needs and those of the school system more broadly. For this reason, we have chosen as our report title the words used by one of our informants, “mixed economy”, because, as we see it, these encapsulate the future of head teacher development in Scotland. As is evident in the broad thrust and detail of our recommendations, we see a revised version of the FRH as having a legitimate place alongside the existing Scottish Qualification for Headship (SQH)―one that incorporates the amendments we have suggested—as an alternative to satisfying the SfH. Prospective head teachers would then be able to choose from one of two available two routes, each with their different and unique emphases, but equivalent in the demands they make, the quality they endorse and in the status they confer. As suggested by the Table of Contents, the sequence of our discussion observes a fairly conventional format. Immediately following the Executive Summary of our findings there is a consolidated list of the Recommendations embedded throughout the report. The next six sections of report (sections 4-9) summarise aspects of the “infrastructure” of the pilot programme. The FRH itself really comes alive, hopefully, in sections 10 and 11 where we concentrate on the participants’ experiences of the programme and its impact on them. In section 12 we discuss issues and implications for the adoption of the FRH by Scottish LAs. During the data collection phase of the evaluation the team adhered fairly closely to the methodology it originally proposed in its submission to the Scottish Executive’s call for 5 tenders. A summary of the tender specifications and our methodology, along with selected examples of our research instruments are to be found in the Appendices. Throughout this report, we cite extracts from our data, in particular the transcripts of interviews and focus groups. To retain the anonymity of our informants we have de-identified these extracts. Finally, a quick perusal of our reference list will confirm that we have made only sparing use of both the research and development literature on coaching and leadership. This was a decision partly forced on us by time pressures but also because the opportunity for a more extensive engagement with this material will arise when we consider the wider significance of our findings in scholarly fora. Notes on Usage: 1. Although what was previously known as the Scottish Executive is now the Scottish Government, depending on the timing of particular documents we are considering in the report we have retained both titles in our discussion. 2. As is evident from section ii, we have been sparing in our use of acronyms and abbreviations. Most of these are well-known. The convention we have observed is to cite the full title on the occasion of first usage in the text. This is followed immediately by the bracketed acronym which is used from that point on in the text, with the only variation occurring when we may have quoted from original sources. Julia Davidson Christine Forde Peter Gronn John MacBeath Margery McMahon Margaret Martin 19 September 2008 6 2. Executive Summary This report provides an evaluation of the FRH pilot. The evaluation was undertaken during 2007-8. The Glasgow-Cambridge evaluation team which conducted the evaluation and authored this report responded to a call for tenders by the Scottish Government during 2007. Essentially, the terms of the evaluation (see Appendix 1) required the team to document the perspectives of a range of participants in the pilot, particularly in respect of the development of candidates in satisfying the SfH, the development of coaches, the nature and sustainability of assessment and the capacity implications associated with a potential “roll out” of the FRH across Scotland by LAs. The timing chosen by the Government for the evaluation occurred at about the mid-point of the pilot’s duration. A series of qualitative procedures were used to obtain data (e.g., documents, interviews, focus groups and observations). Using each of these sources, the evaluation team was able to research all aspects of the operation of the pilot. The team sees the FRH playing an important role in the future development of Scottish head teachers. On the basis of the evidence it acquired, the evaluation team has made 21 recommendations. These recommendations are intended to strengthen that development role. In the framing of the recommendations, the team drew liberally on the reflections of a number of informants and, where appropriate, elaborated and aligned its arguments with existing bodies of research literature. Throughout the pilot, the centrepiece and unique feature of the FRH was the coaching provided to aspiring (and some practising) head teachers by a dedicated group of coaches. The coaching experiences documented in this report, the team believes, together comprise an important case study in coaching and provide significant lessons and insights for this globally emerging approach to school leader development. 7 3. Recommendations This section provides a cumulative list of the recommendations in the sequence in which they are to be found in the report. The numbers of the pages on which each recommendation appears in the text have been included in brackets. Recommendation 1 That the pedagogy, design principles and conceptual view of leadership which make the FRH distinctive be clearly articulated for the provision of future programmes and be accessible in all programme documentation. (p. 12) Recommendation 2 That, in regard to formalising the FRH as an alternative pathway to satisfying the SfH, with parity to the SQH, consideration be given to bringing the SfH and both pathways within the purview of the GTC. (p. 13) Recommendation 3 That FROG be disbanded and the FRH be aligned with SQH and managed in one of two ways: first, by incorporating it within the existing GTC-accredited consortia arrangements for the SQH; or, second, establishing a national management structure by the GTC or under its aegis, which constitutes a management group with overall responsibility for the design, structure and implementation of the FRH, and a programme group which is responsible for delivery of the FRH. (p. 15) Recommendation 4 That for future FRH programmes the management group prepare guidelines that spell out clear expectations of coaches and coaching styles as part of an overall statement of the coaching methodology. (p. 16) Recommendation 5 That assessment of candidates as having satisfied the SfH in future FRH programmes be undertaken by an independent panel of assessors appointed by, and in accordance with assessment criteria determined by, the management group. (p. 17) Recommendation 6 That, in view of the fact that about 20% of the current pilot candidates have already acquired head teacher appointments, consideration be given by the management group to more accurately identifying potential FRH programme beneficiaries at earlier points in the leadership career cycle. (p. 18) Recommendation 7 That, in the light of identified concerns about the accuracy of “flexibility” in the programme title, consideration be given to re-titling the FRH for future provision in a way that more accurately captures the spirit and letter of the programme. Examples worthy of consideration might include the “alternative” or “coaching” qualification for headship. (p. 22) Recommendation 8 That (i) residentials be retained in future FRH programmes; (ii) in preresidential enrolment packages distributed to new candidates and in the residentials themselves the management group communicate clear and unambiguous information about programme components and outcomes, and expectations, engagement and commitment of candidates. (p. 25) Recommendation 9 That, in determining the desired profile of an FRH coach, the management group consider a range of potential coaching recruitment sources (e.g., serving head teachers, recently retired head teachers, LA officers), particularly given the likelihood of increased integration of services for children at LA level across Scotland. (p. 29) 8 Recommendation 10 That, in its statement of coaching methodology, the management group clarify the distinctions between various potential aspects of the coaching role (e.g., tutoring, mentoring) and who should perform them, acknowledge that coaching is directed specifically towards the SfH, and ensure that these aspects are incorporated into the training and development of FRH coaches. (p. 33) Recommendation 11 That the management group establish clear processes for the preparatory training, ongoing supervision and support of coaches. (p. 35) Recommendation 12 That, as part of its guidelines for future FRH programmes, the management group devise a protocol for allocating coaches and candidates that includes strategies for managing unsuccessful or defective coaching, and for the termination of, and exit from, coaching relationships. (p. 40) Recommendation 13 That the management group provide clear guidelines on the administration of future ECIs, paying particular attention to: the number and categories of respondents to provide information for the initial and subsequent administration of the instrument (particularly as some candidates take up new appointments in the interim); the need for sensitivity in utilising feedback and follow-up strategies by coaches. (p. 42) Recommendation 14 That, the management group clarify the extent of the detail of the SfH to be included in PLPs, and the extent to which the PLP is a plan that encompasses both personal learning and professional action orientations linked to school improvement. (p. 44) Recommendation 15 That the management group clarify the link between the CPD needs identified in PLPs and the facilitation of LA-based CPD provision. (p. 45) Recommendation 16 That the management group clarify the purpose and status (mandatory or voluntary) of the LL, and its connection to the assessment and development requirements of the FRH. (p. 46) Recommendation 17 That the management group undertake a thorough review of the FRH assessment process, giving particular attention to: • • • • the balance of formative and summative assessment the extent to which the provision of formative and summative assessment ought to be part of the coaches’ role the purpose and length of portfolios, commentaries and evidence, and the nature of the links between these elements the question of whether and how portfolios, commentaries and evidence relate to the entirety of the SfH (p. 51) Recommendation 18 That the management group review the place of the field visit in the coaching relationship and assessment process, and explore the possibility of it being part of the summative assessment of candidates. (p. 53) 9 Recommendation 19 That the management group consider fine-tuning the role of the presentation panels, giving particular attention to the panel composition (especially the validity of parent representation), choice of questions in relation to the SfH, provision of feedback to candidates and length of presentations by candidates. (p. 56) Recommendation 20 That, as part of its review of the concept of leadership underpinning the FRH, and particularly with a view to enhancing the participants’ effectiveness in school improvement, the programme group consider utilising research in the areas of professional, adult and workplace-based learning to assist in the development and re-development of the practice of senior and experienced professionals. (p. 63) Recommendation 21 That, prior to future negotiations with potential participating LAs, the management group consult with the GTC with a view to resolving issues concerned with the accreditation of coaches, assessment protocols and the maintenance of comparable standards of assessments of candidates. (p. 66) 10 4. Background to the Pilot As an alternative approach to ensuring that prospective head teachers had satisfied the requirements of the SfH, the FRH began to take shape during 2005-6. The Scottish Executive had decreed that, from August 2005, all initially appointed head teachers would be expected to meet the SfH (SEED, 2006a, foreword). The context in which the idea of an alternative route emerged included discussion within government and within continuing professional development (CPD) circles about existing leadership capacity, development and succession planning. In Ambitious, Excellent Schools: Leadership―A Discussion Paper, the Executive committed itself to a “leadership agenda”, based on the view that “effective leadership makes for an effective school” (SEED, 2005a, both quotes p. 1). A number of developmental approaches were canvassed in the document, including opportunities for mentoring and coaching. By early 2006, there had been “feedback from the profession”. The words “flexibility” and “flexible approach” were in currency, and the possibility of an alternative to the existing SQH was beginning to solidify. The core idea was to capitalize on experiential learning opportunities, in which case “flexibility should emerge from a plan of activity developed individually” by each candidate for headship (SEED, 2006a, all quotes foreword & p. 3). At this time, in Achieving the Standard for Headship―Providing Choice and Alternatives: A Consultation Document, the Executive outlined the rudiments of a developmental model in which, for a period of between two and six years, candidates were expected to devise and work towards implementing a development plan (SEED, 2006a, p. 5). There were a number of leadership capacity-related issues influencing official thinking during this formative period. These emerged in an extensive national consultation process from February onwards comprising focus groups, round tables and informal meeting. One analysis of the evidence was unable to detect any clear patterns among the disparate consultation submissions, aside from a broad agreement that flexibility should possibly be considered (Forde, 2006). SEED (2006b, section 1.3) suggested four reasons for offering a new alternative route: the need to take account of teachers’ “individual lifestyles and professional commitments”; the recognition that provision for teachers’ professional development should reflect their individual approaches to learning; the likelihood that a “significant drop out rate” of SQH participants would leave head teacher vacancies unfilled during the next decade, so heightening the need for attracting applicants via a flexible alternative; and, finally, it was believed that LAs had surplus funding available to resource a new initiative. There were two other influential factors. The first was feedback from practitioners that highlighted the importance of supportive colleagues in helping future leaders in decisionmaking: at the heart of programmes of development for these teachers seemed to be the flexibility of choice, individual ownership of the programme and many of the successful heads talk[ed] about the power of having someone coach them. They might not have used the term coach but they talked about people who had been their mentors or their supporters or who had helped them to make hard choices and decisions so that was one kind of thread that influenced their thinking. The second was the perspective of leading international scholars, one of whom had sowed the seed of the idea that relationships were “the heart of things, your relationships being 11 important and so on”. This notion may have helped lay the foundation for the subsequent emphasis in the FRH pilot on interpersonal skills. By late 2006, the possibility of an alternative had crystallized around the idea that “aspiring head teachers would benefit from more focussed support” provided by a dedicated coach. Thus, “an individual aspiring head teacher would benefit from an opportunity to reflect on their own practice, to share their self-evaluation to engage in quality dialogue with an individual whose clear purpose was to support, but to support and challenge”. A sub-group of the National CPD Advisory Group (which advised the National CPD Advisory Team) had conducted further consultations. Its idea of a flexible alternative was then formalised by the Executive in the shape of an 18-month pilot in which about 30 candidates from three LAs would attend a preparatory residential, engage with a coach, and produce a reflective commentary and portfolio of evidence. LAs were to identify coaches from among experienced teachers in management posts (SEED, 2006b, sections 2.2 & 2-8). In summary, then, “we were providing a choice for individuals who might not otherwise have come forward for whatever reason”. The outcome of the lead-up period was a second route to achieve the SfH. “That was the rationale, to provide choice, to provide flexibility within a programme and to enhance the existing provision to plug a gap which the Executive had identified”. 5. The Programme Model in the Pilot By their very nature, programme pilots are highly susceptible to modification during both the start-up and implementation phases. Such are the limitations of individual and collective human cognition that at the outset it is impossible to predict, let alone control, every conceivable contingency that might be likely to affect a programme’s success or lack of it. The evaluation team recognizes both the force of these constraints and that pilots provide opportunities to learn from experience. Specifically in relation to this pilot, there were occasions when, in the minds of the participants we spoke to, decisions seemed to have been made in haste or for reasons that were less than entirely clear. In some cases the impression was created that components had not been fully thought through (see section 10 below). These proved to be a source of at least temporary confusion. Having acknowledged these caveats, we offer some observations on the model of development in the pilot. 5.1 Design Principles For both the period prior to and during the nine months of this evaluation, it is fair to say that there was no precisely specified model of the FRH underpinned by a clearly articulated set of design principles. In essence, the conception or model of the programme evolved or emerged. To foreshadow our later detailed discussion, this work-in-progress status of the programme was particularly significant for four features: the initial residential at Peebles, the status of the Learning Log (LL), the utilization of coaches for site visits instead of field assessors, and the degree of coherence and overall integration of the various programme components. The closest approximation to a set of specifications and design principles was the Scottish Executive’s seven-page circular entitled “Achieving the standard for headship―Providing choice and opportunity” (SEED, 2006b). This document outlined a series of programme parameters. It listed, for example, a number of elements and 11 key tasks which it referred to as “stages”. Without summarising or elaborating on these in this report, the circular provided 12 scant pedagogical justification for the nominated programme elements. Moreover, there was no statement about the nature of the model of coaching to be adopted in the pilot and why this was the preferred model. Nor were there any references to the origins or sources for the elements, apart from about half a dozen mentions of existing practices in the SQH (in sections 2.14, 2.16. 2.19 and 2.20). In this respect, the circular assumes that the assessment process in the FRH would “broadly reflect existing SQH arrangements” (section 2.14), that a nominee of an SQH consortium would be a member of candidate assessment boards (ibid) and that the choice of formal assessment elements should also broadly reflect existing SQH practice (section 2.16). 5.2 Conception of Leadership Our general observations about pedagogy, design principles and a preferred coaching model apply equally to leadership. While the evaluation team is acutely conscious that the FRH was intended for prospective head teachers and directed at their requirement to satisfy the SfH, we were unable to find any explicit statement about leadership or reference to a preferred leader prototype that informed the thinking behind the FRH pilot. As suggested previously in section 4, these heads are intended to be leaders and this was an integral element in the conception of the pilot. As the evaluation proceeded, it became progressively clearer to us that an emerging hallmark of the pilot was an emphasis on what might be termed personal transformation. The switch in terminology from the professional action plan, originally foreshadowed by the Executive to “Professional Learning Plan” (PLP), was the first sign that alerted the team to this emphasis. Other indicators included the significance attached to personal feedback from the Emotional Competence Inventory (ECI), the value attached by coaches to interpersonal relations during field visits to the candidates’ schools, the personal journey focus of the candidates’ portfolios and reflective commentaries, and the content of the questioning at the final presentation panels on the Professional Actions section of the SfH. What eventually crystallized our awareness of the emerging emphasis on personal attributes was a panellist’s remark following a candidate’s presentation at the GTC: “Transformed people transform society”. While the validity of that statement is not at issue, if that is the FRH’s distinctive approach to leadership development then that uniqueness should be clearly expressed in any future documentation made available to Scottish educators and especially to prospective candidates. In both instances (i.e., the underlying view of leadership, and the pilot design and accompanying principles), the length of such documentation need not be extensive but should express concisely the thinking that underpins and provides the rationale for a flexible route. Recommendation 1: That the pedagogy, design principles and conceptual view of leadership which make the FRH distinctive be clearly articulated for the provision of future programmes and be accessible in all programme documentation. 6. The Standard for Headship and the Pilot In view of the fact that the FRH is a new and potentially alternative option for prospective head teachers to satisfy the SfH, a few words are in order about it. This standard is one of four in Scottish schooling. Only three of these, however, are controlled by the GTC: the Standard for Initial Teacher Education, the Standard for Full Registration and the Standard for Chartered Teacher: 13 The standard for headship is not owned by the GTC. It is a government function. Now in some ways that’s just historical, I think, because the standard for headship was the first of the standards to be developed back in the, almost 10 years ago now. And the government has hung onto the standard for headship ever since and when it was reviewed, which is what three or four years ago now, there clearly was a desire to continue with government ownership of that standard. Now that, I think, is because they see leadership and headship as being different from being a teacher. While it is not within the evaluation remit to consider control of the SfH (see Appendix 1), our recommendation concerned with the management of the FRH in the next section of this report has implications for the current oversight of the SfH. Likewise, while we were not asked to comment on the SQH in relation to the FRH―although, as we indicated in the previous section, the Executive did so frequently in its seven-page circular―there is little point in thinking of the FRH as a self-contained entity. Some candidates, for example, were prepared to be part of the FRH although not the SQH and, if the flexible provision is to be made available nationally, then LAs will have to provide for one or both routes. Scottish education cannot afford to institutionalize and commit resources to two rival pathways to meeting a standard. If, however, it is accepted that the SQH and FRH should be viewed as equal in status, although different in emphasis, then in our view it makes sense to have both pathways controlled by the one authority. Advantages include consistency of managerial oversight, commonality of quality control and assurance, a surer guarantee of equivalence of assessment demands on participants, a stronger likelihood that those involved in each pathway will learn from the experiences of the other, reduction of duplication in marketing and promotion, and so on. The linchpin of the argument is assessment. It is now well acknowledged internationally that standards-based approaches to evaluating professional performance stand or fall according to the quality of their assessment systems (Ingvarson & Hattie, 2008, pp. 15-16). Because the credibility of assessment systems is strengthened by the independent status of both the process of assessing evidence and the robustness of assessment outcomes, the case is strong for aligning the location the SfH and the two pathways to its attainment with the GTC (and see section 7.2 on this point). Recommendation 2: That, in regard to formalising the FRH as an alternative pathway to satisfying the SfH, with parity to the SQH, consideration be given to bringing the SfH and both pathways within the purview of the GTC. 7. Management of the Pilot Two of the key implementation stages originally identified by the Executive were the creation of a steering group to manage the pilot and the appointment of a project manager. The project manager was appointed, and throughout the pilot he has played a central role in initiating and coordinating all aspects of the programme. Early in the pilot, a project plan drawn up by the project manager was shared and discussed by FROG members. 7.1 Flexible Routes Operational Group FROG was established at the inception of the pilot and since then has met regularly. The core of this group consists of three coaches, the coaches’ coach and the project manager. At the beginning of the pilot, meetings were held weekly although these became less frequent in the later stages. From time to time, other people are invited to attend (e.g., the external moderator, members of the National CPD Team). FROG’s deliberations mostly focus on 14 overall management and delivery of the project, with documents for discussion tabled by the project manager and coaches providing ongoing operational updates. A Flexible Routes Steering Group was also set up and consists of representatives from the Executive and the National CPD Team. This does not appear to have scheduled meetings but convenes on an asneeds-be basis. As an illustration of FROG proceedings, one meeting of about two hours duration observed by an evaluation team member discussed the recently-conducted field visits in candidates’ schools. The discussion ranged freely across the following topics (Field notes, 29 February 2008): • • • • • • • • • • detailed arrangements for field visits requisite number of visits per week for coaches (given the emotional and physical demands of the half-day duration of a visit, and the need to quickly finalize reports) aspects of the process for assessing candidates (including format of records and the timing of blocks of formative and summative assessment) length of (some) candidates’ portfolios and the time required to read these (in one instance over 100 pieces of evidence had been submitted) relative demands on candidates of portfolio writing and fields visits, along with the respective merits of each process of external moderation and its robustness quality of candidates’ submissions and scales for grading them discussion of fellow coaches’ reports on candidates provision of feedback to candidates on the day of the visit the overall progress being made by candidates 7.2 Interface with Local Authorities At the commencement of the pilot, five LAs were identified and agreed to take part. These were Fife, South Lanarkshire and a partnership of Scottish Borders, Midlothian and East Lothian. Coaches were recruited by LAs for up to about 23 months. Suitable candidates for the pilot were expected to be in senior management positions in a school or LA. Originally, provision was also made for a Flexible Routes Advisory Group, on which there were to be representatives from the Executive, higher education institutions (HEIs), the GTC, coordinators from pilot LAs, National CPD Team co-ordinator, the project manager and head teachers. This group does not appear to have met. If the FRH is to become a permanent feature in Scottish head teacher development, then our view is that these arrangements require revision. The FROG structure that emerged may have been satisfactory to steer the implementation of the pilot but, for long-term sustainability of the programme as a credible alternative to the SQH, we do not regard it as fit-for-purpose. The acceptability of what we now propose rests in part on the fate of Recommendation 2 and, if the FRH and SQH are to be aligned as alternative routes to the SfH, this proposal also has implications for the existing arrangements for managing the SQH. The evaluation team sees two options. On the hand, future provision of the FRH could be brought in under the existing partnership provisions of the SQH. Currently, the GTC accredits consortia partners comprising LAs and HEIs as providers. An example is the Western consortium. As award-conferring bodies, HEIs provide an academic award for completion of the SQH while the Scottish Government provides a professional award. We see 15 no reason why the FRH should not be managed as part of these partnerships, with completion of an HEI practice-based award in coaching (e.g., the University of Stirling) being sufficient for the GTC to accredit coaches. On the other hand, the SQH and FRH could be aligned and managed nationally, either directly through the GTC or a management structure under its auspices. For the FRH, this option would entail some kind of two-tiered arrangement in which decisions about programme structure and delivery would be separated. Whereas the present FROG structure coalesces the two functions, there should be a management group and a programme group which is responsible to it. The composition of the management group with responsibility for both routes should include one or more representatives of: the GTC, HEIs, participating LAs and project managers (or equivalent appointees). The responsibilities of the management group should include programme design (in accordance with Recommendation 1); design and conduct of assessment; appointment of project manager(s); accreditation and appointment of coaches; quality assurance of coaching; selection and appointment of candidates; liaison with LAs. For the FRH, the composition of the programme group should include the project manager, coaches, coach of the coaches (if appropriate) and a representative of the National CPD Team. The programme group’s responsibilities should then include receipt of regular feedback reports from coaches, field assessors and external moderator(s); determination of assessment requirements and deadlines; and provision of regular reports to the management group. Recommendation 3: That FROG be disbanded and the FRH be aligned with SQH and managed in one of two ways: first, by incorporating it within the existing GTC-accredited consortia arrangements for the SQH; or, second, establishing a national management structure by the GTC or under its aegis, which constitutes a management group with overall responsibility for the design, structure and implementation of the FRH, and a programme group which is responsible for delivery of the FRH. 8. Quality Assurance in the Pilot A number of quality assurance issues have emerged in the evaluation of the FRH pilot. Here we highlight quality assurance in coaching and assessment. 8.1 Quality Assurance of Coaching Prior to the implementation of the pilot the Scottish Executive stipulated that the identification and appointment of coaches would be the responsibility of participating LAs. While it was conceded that “the selection process for the coach has to be high standard, it has to be quality assured”, it was also the case that the LAs also “know their own people, they know the coaching skills from their own people”. The three coaches appointed were to undergo coaching training by a nationally appointed reputable provider (SEED, 2006b, sections 2.8 & 2.17). The role of the three coaches was envisaged as (Keegans, 2007, unnumbered): a broad developmental support role including assisting candidates develop and implement a professional action plan, meeting candidates on a regular basis to review progress and supporting candidates in a coaching capacity for the duration of the pilot. With the coach of the coaches appointed, SEED was seeking both individual and group skill development as part of tailored skill training for the coaches, along with “consistency of 16 approach on coaching purposes, values and commitments within the pilot group” (ibid). The project manager was to liaise with the coaches to evaluate their impact as well as the outcomes for individual candidates and the coaches as a group. Assessment of the quality of individual coaches was not part of the terms of the evaluation. Nonetheless, given the central role of coaching their credibility, skill and impact are highly significant. We are able to confirm a strong consensus among our informants as to the high quality of coaching provided by the three current appointees. And while each of the LAs are familiar with, and already make provision for, some form of coaching as part of their own internal leadership development programmes, we are influenced by the observation that “it’s essential to ensure, I think, that the people who are recruiting have the right type of knowledge”. In our view, quality is more likely to be assured in the future selection and appointment of coaches if, as we have already suggested, these functions are controlled by the management group. This, in turn, requires the management group to determine clear national guidelines and expectations. Recommendation 4: That for future FRH programmes the management group prepare guidelines that spell out clear expectations of coaches and coaching styles as part of an overall statement of the coaching methodology. 8.2 Quality Assurance of Assessment Originally, following the project manager’s initial liaison with the GTC to make arrangements concerning the assessment of candidates, the GTC was to assemble a team of head teachers who would act as “trained field assessors” (SEED, 2006b, section 2.14). It was envisaged that field assessors would undertake field visits in candidates’ schools, prepare field assessment reports and, as members of assessment boards, assist in preparing final assessments of candidates’ portfolios and presentations (ibid, section 2.16). Moreover, 30 field assessors are referred to in section 5.2 of the terms of reference for this evaluation (see Appendix I). Field assessors, however, were not appointed. Apparently, both FROG and the Scottish Government decided that the FRH was “too big a programme” and that the appointment of assessors was “too much to do for the pilot”, although this “might be something that was looked at if it was rolled out nationally”. The question that was asked was: “well, do we need field assessors because we’ve got a head teacher, we’ve got a participant and we’ve got the coach”? Instead, the three coaches also acted as assessors, a change that was accepted by the GTC: in fact what we have now is a much more manageable programme. Where previously we were going to have to train something like 60 field assessors, now the coaches are doing that, so coaches then have a formative assessment role for their own participants and a formative assessment role for participants other than their own. Later on in this report, we consider issues related to formative and summative assessment processes (see sub-section 10.5.5) and the role of assessment in field visits (see sub-section 10.5.6). For the moment our concern is with assuring and enhancing the credibility of the FRH assessment process. It was put to us that the assessment in the pilot was generally sound “because the three coaches worked through the assessment criteria together, they developed them together, they knew exactly what they were looking for and they worked closely together cross-marking and supporting one another through the process”. While, on the one 17 hand, this kind of close collaboration is clearly a strength, on the other hand, it can also be a weakness. This is because like-mindedness and mutual support can be self-sealing. In this regard, they act as potential disincentives when assessing candidates coached by a coaching colleague, constraining objective judgments about candidates, no matter how scrupulous or well intentioned a coach might be as an assessor. For this reason, we believe that in future FRH programmes, the roles of coaching and assessment should be clearly differentiated, and be fulfilled by two distinct groups of people. Recommendation 5: That assessment of candidates as having satisfied the SfH in future FRH programmes be undertaken by an independent panel of assessors appointed by, and in accordance with assessment criteria determined by, the management group. 9. Selection of Candidates for the Pilot The Executive anticipated that prospective candidates for the pilot “will be at different stages of their leadership development i.e. between 1 and 3 years away from being in a position to present themselves for assessment against the SfH” (SEED, 2006b, section 2.8). Selection began in November 2006. Participating LAs employed different means of selecting candidates. In one instance, there was a recognition that, while candidates had to be chosen quickly for the pilot, and therefore had to be people ready to assume leadership roles, for the future they could be further back in respect of experience and readiness. In another case, the LA conducted a briefing session at which about a dozen people attended, some of whom were later accepted into the pilot. In another LA, teachers were invited to nominate themselves, they were then interviewed and the selection was made by LA officers. A more practical and less academic option The evaluation team asked the candidates about why they participated in the FRH pilot, the significant people who may have encouraged them to apply, the timing of the offer in terms of their personal and professional lives, and whether the FRH was a tempting alternative to the SQH. For those candidates who were selected, the introduction of the FRH was timely in a number of senses. For some it came at a time when they had been contemplating SQH, had decided against it or had been rejected. Some participants had been turned down for a sequence of jobs and had recognised, and/or had been advised of, the need for further CPD. In some cases the recognition of gaps in professional knowledge came from suddenly finding themselves in the position of acting head, bringing home dramatically the complexity of the job. It was often a time when people needed a new impetus in their professional lives. For some this was prompted by a change in appointment, such as moving from an LA post or secondment to a senior position in a school. For some it was personal and domestic circumstances, such as a marital separation, children growing up, a partner taking a new job, or the need to look after an elderly or disabled relative, which precluded travel. For others still it was the combination of a personal and professional inertia that led to a restless desire for something new―“being bored, looking for a challenge” or, as one candidate said, having “put her life on hold”, the timing was right to move forward again. Others, having found themselves as acting heads, recognised that they lacked the breadth of experience for that role. The FRH was not universally seen as the pathway to headship. A common theme was that it would provide a broader experience and enhance leadership skills, whether or not one went 18 for headship: I felt that at the beginning of the journey no matter what job I was going to do, it would make me a better person because leadership should be within all aspects of the school and not just with the head teacher. While the availability of the programme was incentive enough for some to apply to be part of the pilot, more typically it was the advice and influence of a friend or colleague. Key influential people cited were LA advisors, CPD co-ordinators, directors of education and HMIe. One candidate described being “cajoled, persuaded and encouraged” by a range of people, including his wife, who looked to him to “push his career”. More than any other source it was head teachers who supplied the push, and they were cited as a seminal influence by about half of those we interviewed. The FRH was frequently compared favourably with the SQH on the basis of what had been heard from friends or colleagues who had completed, or dropped out of, the SQH, sometimes together with advice from heads who had or had not been enrolled in it. For the majority, the driving motive for enlisting in the FRH was its flexibility, closely aligned with its perceived practical and “non-academic” thrust, avoiding “the drudge of having to do loads of reading”. Being able to do it in one’s own time, without rigid external demands, and rooted in practice, proved to be the strongest attractions. Some spoke of themselves as not academic, despite having a post-graduate university degree. One such candidate opted for the FRH on the ground that SQH was “too theoretical” and involved “having to write essays”. Others said: Although people say you must be academic to be a teacher, I wasn’t and never have been. The whole academia bit, it’s really not me. The attraction of the FRH as not academic and descriptions of self in similar terminology may be seen as a reflection of a wider discourse in Scottish education, in which the terms “academic” and “non-academic” have been framing concepts in reference to curriculum subjects and to pupils who follow them. The selling or promotion of the FRH appears, from a substantial majority of interviewees, to have played into this dichotomous discourse, a view that was soon to be confounded for the candidates within the first few weeks, or even days, of the commencement of the pilot (as we indicate in more detail in section 10). Summary These reflections raise a number of issues to do with flexibility (see section 10.2 below) and the future selection of candidates. For the moment, we highlight just one selection issue concerned with those potential applicants whom the FRH is intended to benefit. As we have indicated, due to the shortness of the lead-up time to the commencement of the pilot, the candidates who were admitted were those who, in all probability, would have secured head teacher appointments had they not been selected. Moreover, a number were already acting up or were appointed as heads at some stage during the pilot. Our view is that the FRH might be more usefully targeted at prospective and aspiring leaders who, in developmental terms, are located somewhere earlier in the career cycle rather than on the cusp of headship. Recommendation 6: That, in view of the fact that about 20% of the current pilot candidates have already acquired head teacher appointments, consideration be given by the management group to more accurately identifying potential FRH programme beneficiaries at earlier points 19 in the leadership career cycle. 10. The Operation of the Pilot In this extensive section of the report, we provide a detailed account of the candidates’ and the coaches’ experiences and perspectives on the pilot, with occasional interspersed reflections by LA informants. We begin with an overview and then a detailed consideration of the components of the FRH. Stylistically, we have endeavoured throughout to combine a narrative of the participants’ experiences with a more analytical perspective as the foundations for our specific recommendations. 10.1 Programme Overview: Value of the FRH The Incredible Journey Despite a rather shaky start (documented in more detail below in sub-section 10.3.1), there was virtual unanimity among the candidates that the FRH had been a hugely positive experience. “FRH is the greatest thing that’s happened”, said one candidate, who described the experience as a roller coaster ride from the “fantastic highs” of the programme and the return to day-to-day realities with new insights and self-confidence. The metaphor of the journey was a common strand in the candidates’ interview comments, a progressive path from indecision about leadership to an affirmation of unrecognised and unrealised capacity. For some, the impact of the pilot experience was immediate: Even at that first residential weekend I came back with my attitudes completely changed. You come back thinking: “I can take on the world”. Much of this euphoria is explained by the candidates’ relief from what was variously described as the “hurly burly”, the “juggling”, the sense of being “caught up in the cogs of the wheel” or “lost in a blizzard of paper”. These vivid metaphors were complemented by frequent references to, “stepping back” from the incessant demands, the routine that allowed little time for reflection, “to develop strategic awareness away from the nuts and bolts. In the daily routine, there was never enough time to give to others or to focus on oneself and one’s own needs. Some candidates spoke about the loneliness of their jobs. Some had had few confidantes or friends, and felt isolated by their status, overtaken by the demands and pressures of the day, and were rarely confirmed, supported or challenged. One commented on feeling “isolated”, as there was no-one to whom he was close enough to “let off steam” or “somebody to sound off to”. Another said: The opportunity to have that dialogue with somebody else is so rare in the busy professional lives that we all lead, that actually carving out that space is the thing that I think has the potential to make the greatest difference. For those who felt lonely or oppressed by the unrelenting demands of their roles the coach filled the void and was someone to speak to, an oasis of calm and unconditional listening. Such was the quality and intensity of the interaction with their coaches that many candidates spoke of being able to forget the demands and pressures of the job to focus on the here and now. 20 Sustaining and enriching the journey There were three key elements that were seen as working powerfully together to sustain and enrich the journey: the coaching, the residentials and the collegial networking with others in the pilot. While coaching was given pre-eminent place as the centrepiece of the programme by most candidates, the residentials were a complementary form of support and renewal, opportunities to share and network with colleagues. These were occasions for meeting others facing similar challenges or like-minded people. While the experience across cohorts was a positive feature of residentials, it was the bonding within the cohorts that created a strong sense of solidarity and fellow-feeling. Opportunities to explore and find common ground with others in a similar position were widely seen as strengths of the pilot, which underlines the need for this form of lateral exchange on a face-to-face basis as well as on an ad hoc basis, thereby serving a quasitherapeutic purpose: If I send out an e mail saying: ‘I’ve had the most horrendous day. Has anyone finished the 3000 words?’ And I get seven back saying: ‘Totally agree’, kind of thing, you know. This continuing virtual exchange between candidates strengthened as the FRH progressed, with a number of them commenting during the second round of interviews on the lasting value of these collegial networks: Just having contacts out there, both of a personal and friendly nature, but also contacts that I would now feel very comfortable in getting in touch with at a professional level. Speakers at the residentials who were head teachers were seen as particularly helpful and inspiring, and as credible witnesses to the day-to-day demands and dilemmas of headship. Finding the balance between expert input, the impatient demands of the programme and opportunities for more open collegial exchange was also a recurrent theme. Finally, for some, but not all, of those candidates who were deputes, their key source of support was their head teacher. One claimed that “it would have been impossible for me to be successful within the programme, within the flexible routes without the support and the vision of the head teacher”. This person suggested that “candidates in the future may well be either hampered or incredibly helped in the course depending on who their head teacher is”. Summary Hopefully, we have given a broad indication of the candidates’ positive feelings about the FRH pilot as a whole. Their affirmation of the FRH was also shared by the other groups of participants in the pilot. Before exploring the individual components of the pilot in more detail and highlighting elements of it worthy of a re-think or fine-tuning, the use of “flexibility” in the title of the pilot was an issue returned to consistently in candidates’ review of their expectations and experiences. 21 10.2 Programme Overview: Flexibility of the FRH A flexible route: Expectations confounded For the majority of the candidates, the extent of flexibility within the FRH programme confounded their expectations. Our concern is three-fold: first, the mental association of “flexible” with a non-academic and non-theoretical programme emphasis; second, the accuracy of the word flexible as a description of the pilot programme when its components are mandatory and therefore the applicability of flexible in the naming of the pathway itself; third, an absence of clarity regarding accreditation of prior learning (APL). Non-academic? On the first concern, one candidate recalled the lead-up days prior to the first residential at Peebles: We were all emailed the Harvard referencing thing before we went and I really do think at that point there was a feeling of: ‘Gosh! This is far more than I expected’. At least one LA had been up-front at the outset with candidates about the reading and requirement to be familiar with research, but the candidates “hadn’t expected there to be as much reading, as much emphasis on reading”. Towards the end of the evaluation, however, a very marked shift had taken place. Candidates were talking about doing more reading of educational journals and previously unopened educational books: Before I started the course I wasn’t a deep reader of literature research. I now do basically. I try to whenever possible source evidence to back up my gut feelings about things and if the research evidence isn’t there I really give things a second thought. Latterly, then, the practice-theory divide is much less in evidence. As one candidate put it: “On a day-to-day basis, the theory very much becomes practice”. Flexible? On the second concern, while a primary initial attraction of the FRH was its self-proclaimed flexibility, and while in many aspects of its operation flexibility proved to be a strength (for example, in the negotiation of the timing, location and format of coaching sessions), there was a commonly expressed view that the FRH was not as flexible as people had imagined: Well, it’s been much less flexible than I thought it would be, right, much less flexible. This remark was echoed by the thinking of an LA informant, who said that: “I think my feeling is that the word ‘flexible’ is probably wrong. I don’t think it’s been as flexible as some people might have thought it was going to be”. At the same time, “the amount of what they’ve had to do is still quite substantial, it’s not an easy option” for candidates”. One candidate described the tension between being told to do “what you’re comfortable with”, but then “what we were comfortable with wasn’t quite what they wanted”. Indeed, what “they wanted” was initially a source of some confusion and the formulaic nature of what was required was seen by many as disappointing. This feature was cited as particularly challenging in the prescriptive guidance for the PLP and in relation to readings and assignments: 22 I think there was a lot of tension maybe on the second day [at Peebles] because it wasn’t clear beforehand that we would be expected to have such a weighty exercise at the end and there was a lot of input from the university on the portfolio of evidence and I think a lot of people were feeling: ‘Oh! This is not what I expected’. This realisation was ascribed by some to the perception that the programme was still being worked out and decisions were made on the hoof, so that flexibility was being lost in the need for greater structure and direction. FROG was apparently wrestling with the paradox of the candidates’ desire for flexibility and less of the “airy fairy”, which acted as a “subtle pressure” on coaches to create more rigid structures and progressively minimize the flexibility. From the perspective of the coaches, a number of different facets of the programme contributed to the impression and reality of flexibility. These included the pace of completing the programme, format and time options for assessment submissions, choice of focus for school-based work and written reflection, and the flexibility offered by the coaches’ own multi-dimensional roles. In interviews and a focus group, the coaches explored the issue of whether increased future flexibility in the FRH was possible, including: the participation of people in other roles in education (e.g., education officers); candidates pursuing the programme over an extended period of time on a longer programme (not just 18 months); different assessment procedures; and additional ECIs, field visits and alternative ways of assembling the evidence. Prior learning? The third concern about prior learning was the most disappointing aspect for many candidates because, while appearing to promise to take account of prior learning, the pilot did not in fact do so. This perception was owed in part to a legacy of the Chartered Teacher route and a widely shared perception that the FRH had been sold, in part, on that premise of APL: What appealed to me about the flexible routes was that I could look at examples of work that I had done previously. The candidates’ sense of grievance was occasionally more forcibly expressed: I don’t think I would be lying if I said we were all slightly duped. Summary These doubts about the bases on which FRH was originally promoted raise questions about the ways in which it should be promoted in future, in terms of its flexibility, the place (if any) of APL, its alleged non-academic nature or, more honestly, as an alternative coachingintensive route to satisfying the SfH. Recommendation 7 That, in the light of identified concerns about the accuracy of “flexibility” in the programme title, consideration be given to re-titling the FRH for future provision in a way that more accurately captures the spirit and letter of the programme. Examples worthy of consideration might include the “alternative” or “coaching” qualification for headship. 23 10.3 Programme Components: Teaching In this sub-section we review the individual programme components. These are mapped out in Figure 1. Although as we suggest in our detailed consideration of the coaching there was an element of teaching in the relations between candidates and coaches (in the form of tutoring), here we concentrate solely on the residentials. 24 Figure 1: Flexible Routes to the SfH: Pilot Programme Supporting the coaches Coaching of coaches Candidates’ Programme CPD Opportunities LA CPD Provision e.g.: sessions with coaches Selection of coaches Selection of candidates: Financial Training by LAs LA applications; references Coaching and interviews Timetabling individually every 6/8 weeks approximately Legal Issues Training of coaches: ongoing contact through 2 day programme Residential Launch: February January 2007 – Peebles supervision FROG meetings: coaches, Coaching sessions 1 per coach of coaches, project fortnight per candidate of manager, SEED coaches: FROG meetings and representatives coach networking meetings Other activities: good practice visits, meeting with LA email and phone ongoing Leadership development LA Residentials: Assessment Conference weekly initially and less regular latterly Advisers Network meetings – 1 per month in LA 0.5 day or twilight Harburn House Transformational leadership 2 day residential courses organised for LA cohorts 25 10.3.1 Residentials There were three residentials at which the entire cohort of candidates assembled: the first, was a three-day at Peebles in February 2007; the second a two-day in Edinburgh the following May and the third a three-day at Harburn House (East Lothian) in October. In addition, individual LA cohorts held their own residentials. It was the Peebles gathering which occasioned most comment among the candidates. Changing goalposts: The first meeting The disjuncture between the promise of the programme which we have just discussed in section 10.2 and the immediacy of the first experience came to almost everyone as something of a shock. Words used by the candidates to describe this initial Peebles meeting were highly emotional: “bombarded”, “befuddled”, “agitated”, “shocked”, “washed out”, “confused”, “panicked”, “scary” and “overwhelming”. While these feelings dissipated over time, candidates typically looked back on the Peebles experience as being thrown in at the deep end and struggling to make sense of it. This was the aspect most often pinpointed as the nadir of the programme and as the most important lesson to be learned for the future: The key to change would be in the whole format in the way the course was introduced. It was far too much too soon. That knocked everybody’s confidence. We all came away from Peebles thinking: ‘I don’t know what’s going on. I don’t know what they’re talking about. I can’t do this’. I actually sat in that room and thought: “I just want to go in my car now and go”. Some referred to exhaustion at the end of a “full on”, “overloaded” day, with too much detail to absorb, that was too fast paced, and with too much ground to be covered. Apart from very positive comments from nearly all candidates about inspirational speakers, the data suggest that Peebles created three difficulties for them: first, they were confused about the segment on assessment; second, this confusion helped fuel the impression which many came away from Pebbles with that the FRH was being thought out as it went along; third, there was concern with the ECI, not so much with the instrument itself but with the way in which the feedback it provided was treated. (This third point is considered later on in subsection 10.5.1.) Difficulties: Assessment and evidence The assessment segment “did knock people off their feet”: I was even more confused by the end of it…and really more worried about it than anything. A huge frustration on my part was the couple of days we had where the explanation was very much a waste of time. The particular source of bewilderment here related to the nature and use of evidence. One candidate, for example, found the idea of evidence trails “very confusing”: [The candidates] were completely balled over by what we did. And I don’t blame them. You could see them panicking, so on that basis I would say that they, that they didn’t know, that their expectations were not what in fact what the pilot became. They 26 were taken aback. I don’t think they had much of an idea of what they were to do in order to demonstrate that they met the standard for headship…actually what we were doing was really giving the meat of what [assessment] was about. And I can remember their faces. They were shocked. Difficulties: A work in progress In relation to the second point about adhockery, feelings of confusion and consternation were recalled in the tranquillity of a retrospective interview but clearly with some residual emotion which, for some, had lasted well beyond the first meeting: [There were] a few months when I was not very clear about what was expected of us. It was just like children at school. You share your learning intentions at the beginning and you share your success criteria and I don’t think we did that. We didn’t know exactly what we were listening to and why we were listening to it. “Nobody knew what the expectation was”, claimed one candidate, recognising that this was to some extent inherent in the nature of a pilot, so that in general, allowances were made, with some participants emphasising the positive factors in pioneering and learning together. Interview comments were made about organisers, residential presenters and coaches still on their own steep learning curves. While generally appreciated and compensated for by the positive impact of the programme, the experience was also unsettling for some. One candidate claimed that at this point in the pilot: “We always felt that the coach was one step ahead rather than five or six steps ahead”. Another participant commented that the “goalposts kept changing” and concluded: “I think they’ve learnt themselves that if you don’t get it right at the beginning it will affect the course throughout”. While acknowledging the exploratory nature of the pilot, participants also commented on its political significance: Because it’s a pilot I think there’s a kind of political pressure to get people through and to get them through quite quickly because they [?] have to have evidence and they have to have exemplar material and they have to have the stuff to use as a base for the following group because they obviously want, I think they want to roll the programme out. Summary From a programme management perspective, the experiences and reactions documented in this sub-section (and subsequent sub-sections) of the report highlight the importance of residential away-days or retreats for communicating programme-wide expectations and information. For candidates who may be aspiring head teachers, such occasions are equally significant for the fostering of group-level support, morale, a shared sense of identity and common purpose, and the seeding of professional networks. Recommendation 8 That (i) residentials be retained in future FRH programmes; (ii) in preresidential enrolment packages distributed to new candidates and in the residentials themselves the management group communicate clear and unambiguous information about programme components and outcomes, and expectations, engagement and commitment of candidates. 27 10.4 Programme Components: Coaching In this sub-section we consider five aspects of coaching. The three FRH coaches also played a significant role in the assessment of candidates, at both formative and summative stages, as we indicated earlier in section 8.2. We have held over consideration of this component of their role until we discuss assessment in its entirety in section 10.5. 10.4.1 Introduction Coaching relationships, in particular coaching sessions between candidates and coaches, are confidential and certainly beyond the direct scrutiny of evaluators. In that respect, coaching is not dissimilar to a doctor-patient consultation, and the relations between counsellors, therapists and their patients. The core element that set FRH coaching apart from these other confidential relations, as we suggest in more detail later on, was the building of relationships and the establishment of trust between coaches and candidates. The practice of the coaching component in the FRH took the form of one-on-one sessions between a candidate and a coach held every two to three weeks. These encounters usually lasted for up to two hours but could be longer, and these took place at times and in places that were mutually convenient. Some coaching sessions occurred during the school day, either in school or at a local teachers’ centre, but in other cases at other venues before the working day commenced or in the evening to fit in with the candidates’ commitments. It is important at the outset to be clear about the contractual way in which FRH coaching is framed. (This idea of contractual framing, it has to be said, existed mostly in the minds of the coaches. There were very few instances in which candidates talked about framing.) There are two dimensions to contractual framing. First, the coaching relationship itself is structured by a sequence of six-monthly verbal agreements between a coach and each candidate that is intended to set boundaries (i.e., ground rules for engagement) and anticipated outcomes (e.g., progress on assessment tasks). Second, within each coaching session this sense of contractualism is evident in the setting of what the coaches refer to as “the container” in which the areas for exploration are agreed. The container or “space” refers figuratively to the idea of a relationship of trust and confidentiality (see sub-section 10.4.6) which allows for open and challenging discussion on an agenda, negotiated focus and content for particular sessions. Two coaches describe what happens when they coach. Secret gardens In a typical coaching encounter, there are a number of phases. These set out the parameters for that session―“the kind of contract for that day” and then listening―“I’m not naturally listening…real listening”. This is followed by an exploration of the issues and then the actions the candidate will undertake before they meet again to address this situation. But this is a flexible process depending on the issues or circumstances. This coach identified the approach as dialogic. The coach rejects a purely questioning-based approach and instead focuses on articulating―“voicing” and “reframing”―what the candidate is saying or feeling and using this to challenge the candidate: It’s about voicing what you see and voicing what you hear and it’s not about any judgements, no judgements, and for me I found that so much easier and people accept it, people accept the challenge because they know you’re not judging, they know 28 you’re really feeding back to them so for me that’s been really powerful. Another coach gave an outline of a typical coaching session. It usually starts with a general talk: We tend to just talk generally about: ‘How are you?’ and ‘What’s been happening?’ And so it’s a sort of, just a kind of calming thing and then, depending on what’s happened at the last meeting, or what they’ve brought into the room, we might talk about aspects of their work that they’re finding challenging. So it’s a case of, you know, at the outset it would be, you know: ‘What do you hope to get from this meeting?’ And it could be difficulties with their work, it could be difficulties with a member of staff, it could be challenges they’re facing, it could be a meeting they’re planning that, they’re not sure about how to handle it. A lot of it’s about leadership and really management of issues. It could be concerns they have about their submission, it could be sometimes they want to talk about what they’ve been reading and, but sometimes it’s around, it’s around: ‘What do you see in me?’ ‘You know, I was in this meeting, this is what happened, this is how I behaved; can you talk to me about what you see in me?’ That sort of, and that’s how they’re moving, that’s how they’re moving. It’s, if you know yourself, if you see aggression in somebody or if you see negative, if you hear a lot of negatives, some people just aren’t aware that they do it or they say it. So, it’s about honesty really, so that’s really a typical…and then as the meeting moves on, they will come, they will know what they have to do or what they should do the next time, because they’ll say it themselves, because there’s a lot of listening going on and I’ll often re-frame things and give them back what they’ve said and I’ll say things to them like: ‘Do you realise this is the word you used? You’ve just used this word. Did you mean that? And what does that word make you think?’ So, it’s that type of thing where, it’s a lot of listening, a lot of, sort of reframing things and challenging. But there is also a seasonality or life cycle to coaching, for the focus of the sessions seems to have changed over the course of the programme. At the outset the focus seemed to be very much on the candidates’ leadership role: “It’s changed. At the beginning it was very much around: ‘Who I am in the setting I’m in. What my responsibilities are. How I can improve on what I’m doing just now’”. The initial focus was also on the candidates’ self-evaluation and the feedback from the ECIs for the PLPs. More recently there seems to have been a focus on the assessment submission, the review of draft work and issues around the material to be submitted and reading. In addition, for some there are now other issues that are discussed in coaching sessions: I’ve now got people going for headship interviews and other things happening, you know, there’s a number of them talking about headship interviews, so I think even then, I think the relationship changes because sometimes the coaching meetings are nothing to do with their professional development or their writing. The coaching meeting might be about: ‘I’ve got an interview, can we talk about that?’ ‘Well, yeah, we can talk about that. How are you feeling about it? Have you got any concerns about it? How are you preparing for it? Talk me through what type of presentation you would do’. You know: ‘How could you improve that?’, that type of thing. This coach is also aware, as are the other coaches, of the danger of being overly involved in the lives of the candidates, for some candidates find the coaching sessions to be a very emotional experience. 29 10.4.2 Recruitment and Selection of Coaches Who should coach? The first issue we consider concerns the selection of coaches for the FRH. Selection was viewed by the coaches themselves as critical in ensuring the quality of the programme. One of the issues with which they wrestled in our interviews with them was credibility: whether a coach needed to have been a head teacher and the importance of coaches at least having an understanding of the role of head teachers: Understanding the demands of headship, because the demands of headship are like no other job I know, and I think these people who we’re supporting and helping to develop are going into one of the most demanding jobs you will ever get. On balance, coaches thought that anyone in that role should be an experienced and successful head teacher, partly so that he or she would appreciate the specific demands of the role of head teacher, and its underpinning values and purposes in relation to children: “It’s children that makes it a unique environment…it’s the moral aspect as a leader; you are trying to juggle all the paperwork things but with the impact on children”. Implicit in this viewpoint is a construction of the head teacher as a learner and the need for coaches to mirror this in their practices: The person who’s the coach has got to have been an effective head teacher, I think, or a very effective senior manager. They’ve got to have very, very good people skills; they’ve got to understand the whole, the whole aspect of what it means to be a really, really good head teacher. And there has to be this academic rigour attached to it…I mean, a person who runs a school must be a learner, they must read and develop and want to learn. If your head of your school’s not a learner then God help your school. So these things for me are, I mean that’s come over, my thinking’s changed over a period of time. What does/should a coach do? Clarifying the role of the coach at the outset of the pilot was important as there were expectations that the role was more akin to that of mentor (see section 10.4.3): “Because it’s not about me in that expert model which I think I…confused it with in the beginning”. Credibility, which engendered trust in the coaching relationship, was seen as paramount: “It’s not a soft option”. The coaches referred to the unique circumstances of the pilot in bringing together a particular group of people with a range of complementary skills and experiences. This was seen as an important aspect in the success of the programme: “If you only have people coming along that don’t have much experience and don’t have particularly much credibility in their professional standing, don’t be surprised if you don’t get the standard you are looking for”. Summary As an evaluation team, we have an open mind as to whether coaches of prospective head teachers need to be either serving or recently retired head teachers themselves. After all, the coaches’ coach was not an appointment from within the education and schooling sector. At the same time, we are aware (although none of the participants in the evaluation mentioned this point) that in a number of LAs the work of education officers is increasingly nested 30 within, or aligned with, the work of professionals from other sectors (e.g., social welfare). The reason we draw attention to this trend is that prospective head teachers’ responsibilities extend beyond children’s learning and, in the context of inter-agency work and new community schools, the leadership repertoire is inevitably growing in scope and expectation. Recommendation 9 That, in determining the desired profile of an FRH coach, the management group consider a range of potential coaching recruitment sources (e.g., serving head teachers, recently retired head teachers, LA officers), particularly given the likelihood of increased integration of services for children at LA level across Scotland. 10.4.3 Definition of Coaching Coaching is one of a number of closely related support roles that figure prominently in the career development and leadership development literatures. Apart from coaching itself (Witherspoon & White, 1996), these roles include mentoring (Higgins & Kram, 2001), critical friendship (Swaffield, 2007), counselling (Gray, 2006) and network-based learning in communities of practice (Reeves, et al., 2003). As we suggested at the outset of this report, the scholarly community is divided over the merits and contribution of coaching (Berglas, 2002; Hall et al., 1999). While this report is not the occasion to arbitrate these disputes, our data shed some light on the way the support sought by candidates required the coaches to blur the distinctions between, or even fuse some of, these roles. (Because the respective views of coaching held by candidates and coaches are considered in this and the succeeding subsection of the report, we have held over our next recommendation until section 10.4.4.) Coaching and mentoring The best illustration of the blurring of role perceptions is the way the terms “coaching” and “mentoring” run as constant themes in the candidates’ interview narratives. While virtually all candidates distinguished between these two forms of support there is, nonetheless, a lingering ambiguity and confusion as to when one or the other is to the fore in the relationship with their coaches: I think initially I really have to say that I didn’t know the difference between coaching and mentoring and I’m still not 100 per cent sure I have a better idea now. These ambiguities are evident in statements such as: “A coach type mentor” “Mentoring within coaching” “Coaches have stepped over the line and gone into mentoring” One candidate, commenting positively on her relationship with her coach, nonetheless perceived it as mentoring: [The coach] was superb…was very good at giving advice but that’s not what a coach is there for…was more of a mentor and…was superb. This statement reflects a general perception that mentoring is advice while coaching is nondirective and encourages people to come up with their own solutions, and solve problems for themselves. One candidate drew a parallel with Rogers’ idea of non-directive counselling: 31 Mentoring A mentor’s someone you rely on who has similar experience Coaching A coach should be a head with real experience Affirmation and reassurance Reassuring Advising, guiding Suggestion and advice Supportive Someone to turn to strictly for advice Practical Practical Why don’t you try this…it works Suggests things to try out Somebody to sound off to Sounding board Let off a bit of steam Explaining rationale for decisions taken Venting You can share problems with Relaxed Ad hoc Highly structured and focused Questioning Asking searching questions Mentoring is role modelling Probing, challenging Someone you don’t want to model your behaviour on Teasing things out Suggesting targets and deadlines Role modelling Looking at the first draft was mentoring Not role modelling Advice on the way things should be written up Setting targets and deadlines Verbalising thoughts to clarify thinking A coach is something completely neutral In the first 10 minutes the coach shouldn’t speak Recommending reading Table 1: Candidates’ understandings of mentoring and coaching. I had read some similar stuff before, Carl Rogers you know, person-centred counselling and stuff like that. It’s kind of similar in many ways. While the distinction is made between counselling as therapeutic support and coaching as performance orientated, it is clear from many of the interviewees’ accounts that “unloading”, “sounding off” and “letting off steam” serve a therapeutic or quasi-therapeutic purpose (see Gray, 2006). Coaching sessions were described by one candidate as “ritualistic cleansing”. This therapeutic effect was most noticeably to the fore in the early days of the pilot when there were confusion and anxiety about the programme, and coaches did appear to play a 32 therapeutic and demythologising role. Table 1 compares statements which candidates attributed to either coaching or mentoring. These distinctions between mentoring and coaching raise a question as to how helpful they are in headship practice. While strong emphasis is clearly placed on allowing people to solve their own problems, as distinct from giving advice, it appears that candidates use this distinction to impose mental boundaries around different kinds of relationships with colleagues, as revealed in the statement: “When you’re talking to the person you judge whether it’s mentoring or coaching you’re going for”. This may actually prove to be less than helpful in the day-to-day interactions with a range of colleagues within and outside the school. Summary The significance of this conceptual looseness for the future of the FRH is that it re-inforces what we expressed earlier in Recommendation 4 about the need for clarity about the coaching methodology in the programme. 10.4.4 Coaching to a Standard This need for clarity is strengthened even further by re-emphasizing that coaching in the FRH is not simply generic coaching. Rather, it is coaching geared specifically to enabling candidates to achieve a professional standard through the completion of specific assessment tasks. The outcome of the coaching-induced mastery at the heart of this achievement is enhanced self-efficacy to perform role tasks (Popper & Lipshitz, 1992). For this reason, the FRH coaching role has been characterised as that of “programme coach” or a “professional development coach” with two areas of focus: “it’s realising that it’s a programme coach to enable people to actually bring about some behavioural change but also there’s a task involved here…getting through the standard for headship”. It is for these reasons that the FRH coaches played multiple roles. Wearing a number of hats The coaches’ views on the distinctions between coaching, mentoring and tutoring provide additional evidence of role blurring to that in the previous sub-section (10.4.3). “Coach” was the main activity, “65/70 per cent” of the time, but there were times when the coach acted as a mentor, tutor or facilitator (by organising CPD opportunities in the LAs, for example) as well as assessor: “I see it definitely as tutoring, maybe I’m not right but that’s how I see it, I see it definitely as tutoring and sometimes a mentor”. The common feature across these roles was seen as “that of ‘change’”. Coaches thought that avoidance of directive approaches in mentoring and tutoring was important. Being a tutor was a significant part of the work of the coaches. Early in the pilot, for example: [The relationship was] more task focused, getting the professional learning plans in place, ensuring that people knew what was expected, they knew what the standard was, they knew how to draw on their past experience, how to pull that information together, what they were reflecting, what they were learning… Later on, work with the participants appears to have been around the development of the 33 assessed tasks: “there was no coaching done, it was really tutoring around their submission”, and here tutoring activities included giving feedback and advice on written material: …somebody brought me for their discreet claim―they’ve to do a discreet claim on 3.4.4 [of the SfH]―and this person had done a draft and it was very, very much a draft and all they wanted from me was to know: ‘Am I on the right track? Here are the types of quotations I’m using. This is the reading I’ve been doing. Do you think that’s broad enough?’ or whatever. I would give them assessment, I would formatively assess that but it would be on an informal basis and it wouldn’t be in any depth really at all. The coaches, however, saw this support as “light touch tuition” with any instructional element removed. The coaches viewed these different roles as a strength―seeing coaching as one set of skills in their repertoire complemented by other strategies: I’m very concerned…about the whole coaching being the buzz. It really concerns me because it is not the be all and end all of life coaching. It is yet another tool and I think people who think it is [the be all and end all] are very naïve and I think it could actually be quite dangerous because it’s like: ‘Oh! We’ll be coaching now’. And that’s not what coaching is about. It’s, the lines are blurred and it’s a skill that you have in your repertoire of skills. Another aspect of the coaches’ role is mentoring and this is where the previous professional experience of the coaches as head teachers gave them the expert status they could draw on to fulfil this role. The question of whether the roles of coach and mentor might be separated, with the mentor providing specific guidance in relation to aspects of practice or projects, was raised with the coaches. The relationship between the two roles is another aspect that requires consideration as part of Recommendation 4. How seamless is seamless? More generally, there were differences in views regarding the movement from one role to another. One coach, for example, who felt a coach would move easily from role to role, had not “found any tensions personally, I don’t think any of us have, but I do understand the roles are complex and I would really admit that”. In contrast, another coach remarked that moving across the different roles was not seamless: “Lots of tensions in it. I haven’t felt it’s as seamless as [colleague]…perhaps in my moving from one role to the other”. It was agreed that moving in and out of the different roles should be transparent and discussed with the candidates, such as moving, for example, from a coaching to a mentoring role with the candidate’s permission: “I say: ‘Okay [candidate’s name]; right, now let’s move from coaching, let’s have a look at the commentary and I’ll give you feedback’, which is mainly through questions; so even the feedback on the commentary, I think, if you’re doing this as a coach needs to be interrogative rather than instructive really”. The unique combination of the sets of skills and strengths of the particular group of coaches in the pilot raised questions for the coaches about future sustainability: “when you are looking to put all of these roles into one individual you have to question the sustainability in that and also the capacity with which to hold it”. (Sustainability is discussed in section 12). Finally, a critical issue linked to the multiple roles raised by the coaches in focus group 34 discussions concerned the underpinning purpose of the programme was the coaching intended to enable candidates to complete the programme or to develop and change as leaders as well?: If our prime purpose is to encourage people to develop as leaders, never mind what they write, then there’s less of a tension. I think that what we need to be more frank about is that this is probably a coaching and tutoring role and actually say so, and formative assessment becomes part of what a tutor does within the process. Within this programme, we can’t only be coaches nor should we be only tutors”. If in fact, as this last observation suggests, tutoring for assessment tasks plays such a prominent role in FRH coaching, then consideration needs to be given to two issues. First, the proportions of tutoring, mentoring, coaching and other forms of support require clarification. Second, people other than coaches may have to be introduced to the FRH to teach and tutor candidates. Summary As Poglinco & Bach (2004, p. 400) point out, coaching is frequently an ambiguous role and its boundaries are ill-defined. In the view of the evaluation team, the evidence in this, and the preceding, sub-section of the report re-inforce this observation and highlight the need for conceptual clarity regarding the manifold nature of coaching. For the FRH in particular, such clarity should be accompanied by an explicit recognition that coaching facilitates the attainment of a comprehensive standard of professional performance. Recommendation 10 That, in its statement of coaching methodology, the management group clarify the distinctions between various potential aspects of the coaching role (e.g., tutoring, mentoring) and who should perform them, acknowledge that coaching is directed specifically towards the SfH, and ensure that these aspects are incorporated into the training and development of FRH coaches. 10.4.5 Support for Coaches Just as support was decisive for the candidates, so too for the coaches it was regarded as a vital factor, both in their development as coaches and in the need to take a rigorous approach in the practice of coaching. For each individual, a number of available forms of support were identified, each of which we discuss briefly in this sub-section: • • • the coach of the coaches FROG, and fellow coaches in the pilot group. Supported As a source of support, the coach of the coaches provided both training and ongoing coaching, and this contribution was seen as very positive with several references to this work as: “supporting, guiding, teaching, reassuring but actually letting you get on and do it” and “totally non-judgemental”. Although the involvement of a coach of coaches was acknowledged to be “the luxury model”, this support provided a breadth and depth to the coaching process. Support from the coaches’ coach included one-to-one meetings (about once a term initially), and e-mail and telephone contacts. This support helped the coaches to 35 develop their skill and confidence: I think one develops all the time and you feel quite secure in what you are doing and so you’re stepping out all the time. You feel much better about it and it isn’t just because I had stepped outwith [the coach of coaches’] authority and one event didn’t go as well as I would have hoped. I was able to critically reflect that would have happened. I wouldn’t have done in the past, I was able to, with [the coach of coaches’] support, tease out what was wrong with it and why and what have you, and then make a better bash at it next time round. So by the modelling of the practice and the relationship you have with [the coach of coaches], I think that has really made the whole thing. Specific training for the coaches was provided at the beginning of the programme by the coach of the coaches and subsequently similar provision was made for the replacement of a coach who came into the programme later. This training was seen as important, even for the coach with previous training and experience in coaching role: There were new bits in it because coaching is vast as you know, it’s wide, the range of practice in coaching is wide so I still learn new things and I think what I learned most at that training with [name] was about other things to read that I hadn’t read…and I’ve read quite a few other things that [name] has recommended which haven’t really been necessarily about coaching, much more about life and much more about values and things like that. There were several areas the coaches needed to develop. These included: “understanding the standard for headship, understanding behaviours and aspects of school life, understanding what is required to reach the standard for headship in terms of what it takes to run a good school, how you actually challenge and support each other …” and these areas were seen to be well in place through training as well as participation in the ongoing development of the pilot. The training was seen as enabling the coaches to develop their own styles of coaching. Anchored The second support vehicle was meetings of FROG. The coach of coaches also played a supervisory role, particularly in the second year of the pilot project through the FROG meetings. This was seen as particularly important―an aspect of a duty of care for coaches and candidates alike: “There is something about ensuring that whoever is coaching actually has a really good supervision in place because if something goes wrong you have a responsibility, a liability…”. These meetings also provided opportunities for discussion about the development of the various aspects of the pilot project: being able to share it with people who have an empathy towards that situation, as in the FROG group, really helps. It’s like, I suppose it’s a coaching session for me so I’m able to articulate and they’re going: ‘Oh! That could happen to us’. The FROG meetings continued through the programme although less regularly than initially. Here the coaches expressed disappointment at the later infrequency of these meetings, and instead the coaches themselves met together: “Your thinking moves on so much when you are with other like-minded, challenging people”. Overall, the experience of being a coach on this programme was overwhelmingly a positive one for all coaches concerned: “a wonderful opportunity for growth for me”. A great deal of personal satisfaction was derived from this experience: “There’s been a huge personal change for me based around awareness of myself, 36 based around the reading that I’ve had the luxury...the debate within the FROG group”. Networked Working together as a group of coaches provided the third avenue for support. Indeed, working together has engendered a strong and important network: “We’ve become a very close group. We think very alike. We’re different people but I think our values are very in line with one another, and what we believe about education and teaching and learning”. There were concerns about how this mutual support might be managed if there was a greater geographical dispersal of the LAs involved in future FRH programmes, particularly when limited interaction among some of the coaches at the earlier stage of the project had acted as a barrier. Sharing and collaboration by the three current coaches was vital: “a lot of high quality interaction and discussion between the coach group and a huge support”. Summary It is clear from the experiences of the coaches that their support was two-fold: first, they were each individually coached, a process which they valued highly in shaping their own roles as coaches; second, they were each supported through the supervisory role that defaulted to FROG as the pilot progressed. In the event that the programme group envisaged in Recommendation 4 is established, we expect that it would assume the supervisory role that fell to FROG. Alongside that consideration will need to be given to providing opportunities for the coaching of coaches as well as networking. Recommendation 11 That the management group establish clear processes for the preparatory training, ongoing supervision and support of coaches. 10.4.6 Experiences of Coaching It is also important for us as evaluators to acknowledge the coaches’ positive experience of participating in the FRH programme. Blooming Coaching was also personally challenging experience, enabling each of the coaches to reflect on their own ways of working: “It’s made me think an awful lot more about the way in which I talk to people professionally” and “I’ve been challenged to face things that maybe before I had been quite frightened of, situations with very, very able practitioners where I’ve had to challenge them and it’s been scary for me”. Involvement in the project as a coach was also seen as a very powerful learning experience―here, the coaches talked of having been on a steep learning curve initially in developing their skills and understanding, and also in gaining an understanding of their role in the pilot: It was quite insightful for me for I was quite narrow. So now when I go out, the type of behaviour I have, and I now realise that I am much more person focussed. Fledging As we have suggested already, only one of the coaches had previously been involved in coaching, but all of the current coaches indicated they were building on skills developed in their earlier roles as head teachers, and all had experienced a growth in their confidence and 37 skill as coaches. Indeed, being a coach had crystallized many of their previous understandings and experiences: I just see the coaching element as a great way for everything, for encouraging pupils to learn, encouraging staff and the whole thing about people taking ownership and responsibility to themselves for their own learning”. The coaching sessions provided one of the most satisfying and exciting aspects of the programme for the coaches: You never knew what you were going into and for me that was exciting because you felt, I felt that I was really meeting a need in a lot of people, that it wasn’t a dependency thing, it was: ‘This is what I’ve done. Can I talk you through it? Can I talk it through with you’. According to Gray (2006, p. 477), a distinguishing characteristic of coaching is that “it is the coaches themselves who originate and commit to their own strategies, meaning that ‘buy in’ is more likely”. The evaluation data suggest that a pre-condition of buy in is trust. The coaching experience: A question of trust Through the coaching process the relationship evolved with the coaches acknowledging the development of a close bond between themselves and the candidates. The coaching process, however, was not about creating dependency but instead it was about challenging individuals to enable them to find their own solutions: They might miss me but it would definitely not affect their practice. They’re confident, capable individuals. They’ve built up a good strong network but they don’t rely on one another... They appreciate one another and they will talk to one another but they are independent learners and I’m quite confident about that. Some candidates were apprehensive about the coaching, given its centrality in the programme. Getting the wrong person, it was said, could jeopardise success and worries were expressed by some about the “done deal. We had no choice”. Indeed, one candidate commented that the role of coach was so central that the first question asked was: “Who is the coach?”. Establishing a positive and trusting relationship occurred at an early stage in the pilot. Trust appeared in most cases to move very quickly from provisional to professional trust―in other words, from a period of tentative testing to unconditional confidence in the coach’s expertise. In some cases, this was because there was prior knowledge of the coach but also because there were early returns from the coach-coachee relationship. This was attributed variously to “openness”, “personality” and “style” and seeing “the person behind the coach”. “Striking up the relationship” was helped, in this instance, with the coach providing background professional and personal life details―“a nice relaxed way of meeting” for the first encounter. For some candidates trust remained provisional for a longer period. One commented that the programme underestimated the length of time needed time to establish a relationship―“a lot of building up and breaking down of barriers and things to go through”. This was not helped by the destabilising effect of changing coaches. Another candidate described the early stages of the relationship, as having to prove yourself to the coach, to show “you were good enough 38 to be there”. With the recognition that the coach was not there to assess (i.e., to make personal judgments about candidates) but to support, it became possible to disclose weaknesses and share problems. Building and sustaining trust The separation of coaching and assessment was seen as a critical component despite some initial uncertainty about assessment and having to “prove yourself”: You can see the person is not your assessor, which I kind of think that, you always had that in the back of your mind at the beginning―were they assessing you and now your realise that: ‘No. There is a supporting role’. So you are more likely to say that you are having difficulty with this than you were at the beginning. Some candidates had experience of two different coaches and were able to comment positively about the different benefits they acquired from both. For some the switch was upsetting, while most saw the change as beneficial―describing it as from “chit chat to the nitty gritty” and from “ticking along” to “moving along”. In interviews towards the end of the pilot a number of candidates commented on the strength of coaching sessions in terms of their structure or pattern. Rather than engaging in an openended chat, the coach and candidate would sit down, agree on the issues to be covered and allocate time accordingly. As well as this structure, the sequence and flow of issues were also seen as strengths, ranging from immediate concerns to longer term professional issues, from surface to deep or, as some put it (alluding back to our earlier discussion), from mentoring to coaching. This tendency was also related, in part, to the timing of sessions. In many cases these were once every two weeks, but some preferred a long time span in between sessions so as to allow time for embedding of what had been learnt between visits. Not every coaching session was necessarily seen in a positive light. As some candidates put it, they had to be in the right frame of mind: “Your head has to be in the right place” and it was crucial not to be overwhelmed by the volume of daily work. In the midst of busy lives candidates used words such as “pushing”, “driving”, “keeping me on task” and referred to a need for structure, deadlines and targets. There is also a suggestion of the coaching relationship becoming quite addictive. One candidate commented, for example, that by the next meeting a fortnight later there was a need for a further visit because the effects have “worn off”. Another said: I’m going to miss it if I don’t have it any more, I’ll probably go to a shrink instead. These last remarks raise the important question of how participants in intense support and helping relationships generally, and coaching in particular, negotiate exit. We re-visit this issue of disengagement, along with engagement, at the conclusion of this sub-section. The power of coaching The power of the coaching experience is explained by a number of factors. At the most simple level it is to have someone who is willing to listen for an extended period, a privilege rarely if ever afforded within the routine business of the school day or week. Allied to this is the time-out factor, away from “whizz bang” and the incessant knock on the door. Being able to share a practical problem and talk it through with a colleague who was “there for you” 39 was, for almost all, an opportunity which was unique, empowering and to some extent therapeutic. The “luxury” of dedicated time with the coach allowed lack of information, confusion or misunderstanding to be discussed and clarified, with the coach picking up and helping to make the links between the taught sessions, the literature, assessment criteria and the day-to-day pressures of the job. Trust in the coaching relationship was consistently emphasised as the key ingredient because it permitted a deeper form of sharing without risk or judgement. The neutrality of the coach, with little or no knowledge of each candidate’s school―what Czarniawska-Joerges (1997) terms “outsidedness”―was described as bringing unfettered insights to accounts of events free from prior history. Sometimes the neutrality was encapsulated in a neutral space outside a school or in parts of a school that were off limits to others. The distinctive contribution of the coach which rises above these pre-conditions is the quality of listening: listening for a story than just to a story, affirming, probing and challenging. This is the essential difference between supportive coaching and talking things through with one’s partners or families. While frequently mentioned by participants, this domestic discourse was expressed more as an emotional release and a need for unreserved acceptance, whereas the coach was valued because of the challenge that accompanied the support: [The coach was] brutally honest and I really appreciate that looking back now. It’s what I needed. It’s not somebody to tell me I’m great. It’s somebody to tell me and advise me and to make me reflect on where I am. The coaches also affirmed the significance of listening. They regarded it as paramount and closely connected to a sense of complete confidentiality: candidates “know they can say almost anything that they want to and also somebody that they know will challenge them and it’s not just about listening, the listening skill is paramount obviously but it’s about listening and then finding the right questions to ask and probing in order to help them move on”. Above all, then, it is the coaching experience that is credited with building confidence, sometimes shaken by other forms of feedback, such as ambivalent ECI feedback or unexpected and destabilising demands of the course. As members of an LA or CPD team in touch with many schools and with other course members, coaches provided individual candidates with a benchmark and a normative reference point against which to assess their own practice. Being “privy to a lot of personal information” was seen both as adding to the coach’s credibility but also raising a doubt about whether or not to divulge very personal issues. Distinctions were made between ad hoc, formal, strict and supportive coaching, the e mail or phone cry for help, the formally agreed space and the strict adherence to a protocol such as talking through a pre-defined issue. Examples of informal ad hoc meetings with the coach were meeting in shopping centres such as ASDA and playing golf together. There were many comments about the coach “being there for you” at the end of a phone, responding quickly and helpfully to e-mails―“[The coach] just, I would say, worked above and beyond the call of duty for us all…was just always readily available”. The extent to which this is a realistic and reasonable expectation has, however, to be considered in relation to the wider institutionalisation of the role and what Elmore (1996) terms “going to scale” if and when coaching is implemented nationally. 40 Emotional labour A common theme in the discussions of the coaching process was the emotional content of these coaching sessions. Coaches found the experience to be emotionally demanding, as coaching is a very intensive role: “At this time of their project, we are right in there with these people and that…is why I am so emotionally tired”. The candidates determine what is discussed, including personal issues, but the coaches drew a clear line between this as a process of allowing the candidates to “off-load” and their adoption of a counselling role: “I’m happy to listen because it’s what they want, to get it all out, ageing parents, difficult children, just the usual gamut of life…They’re not expecting me to fix it”. One of the distinctive features of the role of the coach in this programme is the length of time they are coaching a candidate without a break―over an extended period of up to 18 months: This is coaching within a programme, a context, so one of the things, and also it’s a slightly false scenario where often you wouldn’t coach someone for 18 months without a break…because what you can end up doing is colluding and you will begin to notice things in people because, you accept it because that’s the way that X does things. However, when someone fresh comes in they don’t know any of that. So, you can be around someone too long. The coaches indicated that they were aware of this danger. In this respect the coaching contracts, agreed on every six months, provided a structure for the coaching relationship. The programme had distinct phases partly marked by these periodic coaching contracts, and also by the demands of the assessment process and the candidates’ progress towards submission of their portfolios: In the first phase there was a coaching style adopted...the way in which you set the conversation up, there’s the way in which you listen, there’s the way in which you challenge and support…[also] a bit probably of facilitation, a little bit more around mentoring, a bit of directive coaching because you are laying out what a PLP is…phase 2 there was more coaching...more time to start to look at the ECI in terms of behavioural work…in phase 3, however, there’s been more, there may have been elements of tuition…what does the structure look like, a bit more times of mentoring and perhaps coaching is now again beginning to kick off again. Just as with mentoring (Kram, 1984), then, the coaching relationship is staged or phased. Generalising on the basis of our evidence, there appear to be four sequential phases. The first was largely related to setting up the programme and covering aspects such as the planning of learning and understanding the requirements of the pilot. The second was where coaching proper took place and included a greater focus on the ECIs. The third entailed preparation for the assessment, and included a mixture of tutoring and mentoring. The fourth stage included more coaching again. The six-monthly contracts were seen also as playing a vital role in the ending of the coaching relationship though there was some discussion of this: “I find that quite tricky to do. We discussed that as a group quite recently and obviously we perhaps need to have an explicit and reasoned exit strategy”. 41 Summary In this sub-section we have provided a comprehensive description of the demands coaching places on coaches and candidates alike, and the depth of the personal engagement entailed by good quality coaching. A couple of features that set the FRH pilot apart were that candidates were allocated a coach, they did not choose one, and the coaching continued for a fixed predetermined period, rather than ending when the two parties may have deemed it to be over. Fortunately, by the end of the evaluation, each of the 29 coach-candidate pairings was deemed to be highly successful. This cannot, however, be vouchsafed in the transition from a small controlled pilot to large scale reform. For future programmes, therefore, we believe that some attention should be paid to the possibility that candidates and coaches may not always hit it off as well as the likelihood that there will be coaches who do not meet the required standards. Moreover, as the observation by the coach preceding this paragraph suggests, there is probably also a need to think about both exiting the coaching relationship and life after coaching. Recommendation 12 That, as part of its guidelines for future FRH programmes, the management group devise a protocol for allocating coaches and candidates that includes strategies for managing unsuccessful or defective coaching, and for the termination of, and exit from, coaching relationships. 10.5 Programme Components: Assessment As part of the evaluation, in addition to the interviews with all of those involved in the FRH, we analysed samples of the written tasks completed by the candidates and observed the presentation panels. In this sub-section we discuss those tasks undertaken at the beginning of the programme for self-evaluation and planning―the ECIs, the PLPs for ongoing reflection and the LLs―as well as the assessment tasks―the reflective commentary, the portfolio, the field visit and the panel presentations at the end of the programme. We also explore the significant issues that emerged from the analysis of these materials, and from the views obtained from the interviews and focus groups. 10.5.1 Emotional Competence Inventory Doing the Scottish thing The ECI is 360 degree evaluation instrument introduced to candidates at the beginning of the pilot. The ECI focused on personal qualities and interpersonal skills, and was consonant with the thrust of the coaching to support behavioural change. The ECI had been distributed prior to the first residential and candidates received their feedback at Peebles. It is evident that there was considerable confusion and concern about the use of this instrument in the early part of the programme. This ECI was familiar to some who had already used a 360 degree feedback instrument, while for most it was a new and challenging exercise. While viewed positively by the majority (“it’s a really, really brilliant thing that”, its purpose, usefulness and impact were unclear. One candidate commented on the bad timing of its introduction and period for completion, compounded by the view that “it wasn’t really put across to our cohort just how important that was”. In the context of information overload, it was not given the time it merited: 42 A boiling cauldron of hundreds of [bits of] information and: ‘By the way, here’s feedback on your competence and it has been given to you by your colleagues’. The usefulness of the ECI was also compromised by the lack of clarity regarding who should provide the responses. “Some people were asked to include family members, some people were asked to include only a certain number, others more and that probably kind of skewed the results”. Left largely to their own devices, some candidates had deliberately chosen people who would provide challenging feedback, whom they “did not see eye-to-eye with”, later regretting the lack of guidance: I exclusively selected people who I thought might be critical, seriously I did, and that was bad news for me. It dented my confidence…I felt kind of drained. Some had given the ECI to a small group of close associates while yet others did include some family members. This disparity caused some angst: I felt ill through the talk because I felt: ‘Oh my goodness!’ Because it suddenly came to light that people had asked about five people and I was sitting here with 18 and I was sitting and I felt quite faint. There was also a wide spectrum of difference in responses to the feedback from a deep sense of disquiet to a morale boosting affirmation (‘Oh God! People think I’m okay!’). A number of depute heads revealed their own lack of confidence, their modest perception of self rating being significantly lower than that of their colleagues: I think when I filled it in I did the Scottish thing. I thought: ‘I can’t say I’m good at that’. And I rated myself low… which was interesting for my own development because I didn’t always perceive what I was good at. Problems and Potential In virtually every case, candidates commented that they knew who the “anonymous” respondents were by, for example, recognising their handwriting. While compromising the anonymity principle, this response could also be a way of rationalising the discrepancy between one’s own and others’ views. The lack of clarity in framing the activity left some people feeling discomfited, while the aftercare was not always helpful either. As one candidate put it: 10 minutes to digest it and then have a conversation with someone one had never met. This meant that some people went home and back to their schools left feeling very dispirited by carrying the legacy of “two pages of what was wrong with me”. The coaches were also keenly aware of the impact this process had had on candidates. Part of the difficulty had been that this element stood independently of other aspects and the coaches at that point had not received the required training to debrief the candidates with the outcomes which, as the coaches reported, had caused real concern: “They got their ECI first week in February and they were stunned”. There is clearly a need for better direction and, as one candidate put it, “a health warning”. On the other hand, the ECIs were referred to regularly in the coaching sessions and it was anticipated that the candidates would revisit them in preparation for the presentation panels. Many of these early mistakes and lack of direction were addressed when it came to the second round of the ECI. With the beneficial, or “traumatic”, legacy of the first ECI, the 43 second administration of it was more thoughtful, measured and generally yielded much more helpful and encouraging outcomes. Most had found a better mix of respondents, “a wider pool of people”, “some randoms just to give it a better balance”. It also posed new questions for things that you could work on so it meant that the development work you then undertook over the next 18 months was really focussed on what you needed rather than what you thought you needed. One candidate chose to focus on a specific theme in the second round, using a questionnaire of the person’s own choosing to complement the Hay Group’s ECI protocol and inviting robust critical appraisal. Many candidates commented on having a better sense of perspective on the negative comments and greater resilience in being able to handle them. Recommendation 13 That the management group provide clear guidelines on the administration of future ECIs, paying particular attention to: the number and categories of respondents to provide information for the initial and subsequent administration of the instrument (particularly as some candidates take up new appointments in the interim); the need for sensitivity in utilising feedback and follow-up strategies by coaches. 10.5.2 Professional Learning Plan Difficult and unexpected tasks In Achieving The Standard for Headship―Providing Choice and Opportunity (SEED, 2006b, section 2.10), an outcome of the preparatory stage of the programme “would be the production of a professional action plan that is signed off by the candidate’s local authority”: Plans will also be timed and costed at this stage to ensure that they are realistic and affordable. The local authority will also ensure that where possible that there is a tie in to the teacher’s individual CPD programme for the coming year, and that the professional action plan is compatible with the school improvement plan. In the FRH, this process evolved into the development of a PLP in which, based on their selfevaluation against the SfH, the candidates drew up proposals to address their identified learning needs. Here there were a number of issues identified by candidates and coaches which echoed the findings from the analysis of a sample of 19 PLPs. There were wide variations in the presentation of the PLPs in terms of: (1) layout, (2) length, and (3) detail, as well as in the extent of the coverage of the SfH. Most significantly, there seems to have been a blurring in some plans between the development of an action plan for a school-based project and a comprehensive plan detailing the candidate’s development across the entire SfH, including the three essential elements. There was also slippage in relation to terminology. Throughout the report, we have used PLP as an acronym “Professional Learning Plan”. As is evident in the first quotation in the next paragraph, however, PLP was sometimes used for “Personal Learning Plan”. The PLPs were seen as an important element by the coaches but they, too, were aware of this confusion and had to address this with candidates: Some people had written up an action plan which is quite different from a personal learning plan and that actually confused the issue when it came to writing the 44 commentary. There had to be some disentangling of the purpose, but I agree, the personal learning plan is very important. From the candidates’ perspectives, the development of a PLP was largely unexpected, and to some extent an unwelcome, part of the programme. The expectations of the PLP did cause some confusion and some “consternation”. Strong words were sometimes used to describe the plan―“pain and torture”. One candidate expressed resentment at “strict guidelines”, making an invidious comparison with a more mature approach adopted by this teacher with senior pupils. It was a “bigger headache than it ought to be”, claimed another candidate, because “nobody seemed to know the right way”. Variations on a theme The apparent lack of clarity led to people undertaking it in a range of different ways: I know some people who wrote it up in a story format. I know some people who wrote it up with a school development plan. And I chose to look at it from the point of view of having done the audit and the self-evaluation ECI. What I did was, I looked and waited until we did our action plan for the school, the school improvement plan, and knew what was coming from that and linked projects that I had to do anyway for the improvement plan, and just tweaked them slightly for the learning plan. Our analysis indicated that the predominant mode for presenting the PLPs was the use of some form of grid, although a small number of candidates have presented their plan as a narrative or a combination of narrative and tabular format. A number used the format of a school improvement plan or some adaptation of this format, while others produced their own grid. The length varied considerably―the shortest was a two-page grid, while others ran to several pages in length. The level of detail concerning the areas of the SfH to be addressed and the ways in which the candidates developed in these areas also varied enormously. Some plans were very complex. One plan, for example, made links between the SfH, ECI, the school development priorities and the quality indicators from HMIe’s How Good Is Our School? Most plans, however, adopted a tabular format akin to a school improvement plan and identified areas of the Professional Actions to be covered. Some plans included links between the Professional Actions and essential elements of the SfH in specific development opportunities, but areas of the SFH, such as Strategic Vision, Values and Aims (4.1) and Knowledge and Understanding (4.2), were frequently not included. Once again, issues highlighted from the analysis of the PLPs reflected the views of candidates. Some candidates reported that they had assiduously tried to cross-reference their skills to the SfH, embedding these in their PLP, while others expressed difficulties in coming to terms with what was expected in relation to the SfH. Reflecting on the SfH and matching oneself against it, with an emphasis on the evidence, was widely perceived as a both a difficult and unrewarding task. Evidence trails were described as “heavy going” and “discouraging”. Matching every piece of evidence to the SfH involved “hours of annotation”, claimed one, while another commented: “By the time I got to 20 and 25 I’m thinking: ‘Why am I doing this?’”. While the point of the exercise is to self-evaluate against the SfH and identify development needs, it was widely suggested by the candidates that there might be less mechanistic and tedious ways of accomplishing the desired end. 45 Planning for what? A key issue highlighted by these data is understanding the purpose and significance of the PLPs in the shaping of coherent individualised learning programmes. In only a small number of plans was there any clear sense of the process, and progress, of learning, in which the candidates indicated their intention to develop knowledge and understanding to support their practice. Few emphasized the need for reflection, critical understanding, the use of LLs, reflective journals and other learning opportunities embedded in the FRH. Further, impact and evaluation of the candidates’ learning tended to be constructed in terms of the success of a project rather than their development as a school leader. The distinction between a project (which would involve others) and their own practice was not always clear. There was little reference to the learning processes embedded in the FRH. A tension is evident in the balance of prior learning and the need to demonstrate the achievement of the SfH. The focus for most PLPs was on the planning of learning experiences, while a minority had summarised previous experiences and related these to the achievement of the SfH. In these cases, the PLP was more akin to a plan for a portfolio, with the identification of evidence that could be put forward to demonstrate achievement of specific areas of the SfH. This tension between what was deemed to have been achieved previously and need to demonstrate these areas of competence through either evidence gathering or completion of further development opportunities, was highlighted by one of the coaches: “We’ve been asking people to cover every competence and then they’ve done the self-audit on the ECI which says: ‘Oh! You’re great. This is rich, but you’ve still to prove it’. You know?” Recommendation 14 That, the management group clarify the extent of the detail of the SfH to be included in PLPs, and the extent to which the PLP is a plan that encompasses both personal learning and professional action orientations linked to school improvement. 10.5.3 Local Authority CPD Following learning needs? PLPs have a significant role in providing a framework to support the development of individual candidates. An important aspect of the FRH is the opportunity for candidates to undertake CPD that is relevant to their specific learning needs: “The development opportunities will depend on the professional action plan of the individual candidates” (SEED, 2006b, section 2.12). The idea of accessing a wide range of CPD opportunities matched to the specific learning needs of individual candidates has the potential to provide flexibility, but this brings with it the danger of fragmentation and a sense of gap filling. Originally, it was to be the job of the project manager to group candidates according to their identified needs and to establish learning networks (SEED, 2006b, section 2.11). While there was some CPD provision for a full cohort via the residential courses, enabling access to CPD opportunities became a responsibility of each coach who acted as a “facilitator” of CPD activities. In one LA, a common programme of CPD events became the means of networking. Provision was largely available on a LA basis with coverage of areas such as timetabling, legal issues and coaching skills. The range of possible learning opportunities included in the PLPs varied greatly. Some were 46 largely school-based opportunities or tasks to be completed as part of the candidate’s remit, while other candidates have looked more broadly at other types of opportunities. About twothirds of the plans included course-based CPD opportunities―typically technical aspects of management such as: financial training, finance, health and safety knowledge of legislation in personnel issues, and additional support needs or CPD opportunities in the specific area of their school-based project. Some candidates planned personal skills training―coaching courses, such as presentations skills, were the predominant ones included. Recommendation 15 That the management group clarify the link between the CPD needs identified in PLPs and the facilitation of LA-based CPD provision. 10.5.4 Learning Log: Fallen by the wayside Clarity in the understandings of the learning process underpinning the FRH on the part of the candidates was an important issue in the development of the LL. Generally, the LL was accepted as “a good thing” and its completion paved with good intentions. In the event, however, it was, as some candidates said, “ignored” or “intermittent”. This was partly explained by the open and unstructured way in which it was introduced and because it was left open-ended with no “push” or expectation compared to other highly structured expectations (e.g., the PLP). “I don’t think anybody in our cohort took it seriously”, claimed one candidate, “because it wasn’t presented to us as being a really good tool to help you when it comes to writing up your reflective commentary”. Another candidate said that to take time out in the course of a busy day “felt like cheating, you’re cheating your employer”. Another candidate reiterated a concern expressed in the first interview that, in a turbulent school marked by frequent violent incidents, “you feel like you’re leaving someone else to deal with it, and you should be kind of fighting fire, or doing something. It’s quite a selfish thing to do, really, in school time”. The failure to sustain the LL is explained by its voluntary nature, set against the urgency of other demands back in the school situation and by the priority given to those elements of the programme which were to be assessed. This raises the question of whether LL is a vital or peripheral aspect of the FRH, whether it needs to be more structured or whether there are alternative means of serving the same ends. There is some ambivalence as to whether it should be a requirement, encapsulated in one response towards the end of the pilot: I wouldn’t have been very pleased if somebody said to me: ‘You’re going to do it whether you like it or not’…Somebody could have me do it for a short period of time to see if I benefited from it. There is, however, broad agreement that the it is personal and private, and that to require its contents to be shared would alter the nature of what one wrote: The situation in there that, had it fallen into the wrong hands―you know, I hadn’t thought about until I thought I’d lost it. The importance of privacy, as well as the therapeutic nature of the log, is brought home vividly in the following confessional: 47 I’ve got a terraced garden and each terrace is named after a member of staff who, I came home and channelled my aggression out on the soil, so I used to call it my wet weather journal. Some candidates found it difficult to write something down rather than thinking or talking about it with others. As one candidate said, pointing to her head: “The learning log is just up here now rather than written down”, while others commented on its benefits in stimulating dialogue: The learning log helped me a great deal when it came to talking to my colleagues, especially at senior management level, about what I was doing and why I was doing it. One candidate described how the log could be very helpful in preparing for a coaching session: I did use the learning log for that, the little book. I did kind of note down things that I wanted to speak to [my coach] about before I went, not a formal agenda or anything like that, but just a few notes of things I wanted to share with [the coach]. One candidate had found the LL so useful that she had introduced it as a weekly task for pupils to record their learning and built it into school self-evaluation. Towards the end of the pilot, some candidates regretted that there had not been a greater emphasis on maintaining the log as a source of evidence for the portfolio or the PLP. Recommendation 16 That the management group clarify the purpose and status (mandatory or voluntary) of the LL, and its connection to the assessment and development requirements of the FRH. 10.5.5 Portfolio (Reflective Commentary and Evidence) The main elements in the assessment of the candidates on the FRH were a reflective commentary, a portfolio of evidence, a field visit and panel presentation at the conclusion of the programme. Members of the evaluation team have been involved at each stage of this process, reviewing the written material submitted and observing the presentation interviews. In addition, the experiences of both candidates and coaches were explored through the interviews and focus groups. The following summary provides an overview of the collated findings in which each of the tasks is considered as well as the assessment process as a whole. The process of assessment: Formative and summative The coaches played a significant role in both the formative and summative stages of the assessment process. They provided the formative feedback on the first 3,000-word task and then on the 10,000-word reflective commentary. The coaches and other FROG members then acted as summative assessors for the candidates of the other coaches. As we noted earlier in the report, the original intention had been to recruit and train field assessors through the LAs (typically experienced head teachers). It was decided, however, that the assessment should be conducted by the coaches and other FROG members. The role of the coach as assessor was a matter of considerable discussion among the coaches and some indicated their concern at the beginning of the process, particularly as they had no training in 48 the role. At the same time, their involvement as assessors was seen as a significant development opportunity in this pilot for the coaches, as it allowed them to appreciate the standard of work required in the submissions and to sample different examples of work (through the process of moderation). Whether the coaches should continue to play this role of assessor in the future, however, is something that was questioned: “The jury’s still out about who does the summative assessment”. As the coaches reported to us, steps were taken to ensure that the role of coach did not influence the assessment of the submitted work, particularly the stipulation that coaches should not undertake the summative assessment of their own candidates and that the moderation was to be provided externally. Nevertheless, the acknowledged strong connections in the group of coaches and in the FROG, can lead to the external perception that rigour and standards may be compromised (as we suggested earlier in section 8.2). Indeed, there was a strong view on the need for the visible checks and balances for an assessment process that was not served by the blending of the role of coach and assessor. Conflict of interests? While there were advantages in involving the coaches in this assessment process, for example by furthering for their own development during the pilot, the decision to allow them to assess candidates did raise significant issues. As we recommended in section 8.2, their involvement in the assessment of candidates can create tensions, carrying the risk of undermining the reputation of the programme as rigorous and having parity of esteem with the SQH―an aspiration of those both involved in the pilot project and wider stakeholders: Now I think the SQH in the universities provides a very, very positive route to good head teachers for many, but I think having some sort of flexibility within these programmes…is not a bad thing to have either. The issue of rigour in the assessment process was reiterated by a number of those involved and the appointment of an external moderator was welcomed. There was another concern about rigour with regard to the balance of formative and summative assessment in the programme in relation to the standard of the candidates’ work. The “task”, as it was referred to, was a significant area of discussion during the individual coaching sessions - part of the tutoring role which was significant in both the second and the third six-monthly contract phases of the coaching. Here, the candidates had opportunities to seek advice on reading, discuss any relevant issues as well as guidance on writing. While there is considerable merit in the provision of tutoring feedback, within the assessment component as a whole, the balance lies with formative feedback in addition to that provided through the coaching process. A map of the assessment process is to be found in Figure 1, stages 1-3 of which were deemed to be formative while, stages 4-5 were summative. Table 2 shows that at stage 1 candidates completed an initial written task of 3,000 words which they submitted to their coach for written formative feedback. In stage 2, the full reflective commentary of 10,000 words and the portfolio of evidence submitted for formative assessment to the coach and to the second coach who acted as assessor. An outcome of this second stage was the generation of a list of themes by the coach and the assessor to be pursued in stage 3 (the field visit), which again was a formative process. Written feedback 49 was then provided to the candidates to enable them to revise the reflective commentary and portfolio of evidence to meet the assessment criteria in stage 4 of the assessment process. In the view of the coaches, the balance between formative and summative assessment was seen as about right, echoing their stance on the coaching approach. The practical issue they raised, however, was the necessity of seeing the full 10,000 words, given that not many candidates were required to make significant revisions. Furthermore, the balance between formative and summative assessment did raise the question of where the line should be drawn, particularly when there may be some candidates who seem unable to meet the demands of the programme―how long, for example, should a coach go on supporting a candidate by providing feedback on the written work? And: at “what point do you say: ‘Well I’ve tutored enough and I’ve coached enough and now the person’s on their own, they’ve passed or failed?’ It’s a tricky one”. Table 2: Assessment Process in the FRH Pilot Programme Stage Activity Role Assessment Initial stage ECI coach Formative Initial stage PLP coach Formative coach Formative 1 3,000 word task: Introduction, rationale and critical incident 10,000 word critical reflection and portfolio of evidence coach and coach/assessor Formative 2 Field visit coach/assessor [written feedback discussed through own coach] Formative coach/assessor Summative 4 Revised 10,000 word critical reflection and portfolio of evidence Panel Presentations – 15 minute presentation and questions Panel includes GTCs rep, LA/sch rep and coach/assessor Summative 5 3 The “academic” nature of the assessment requirements There seem to have been a number of different implicit understandings about the purpose of the FRH regarding expectations of appropriate assessment. The planning document for the pilot which we quoted towards the beginning of this report stated that “the assessment process envisaged would broadly reflect existing SQH arrangements” (SEED, 2006b, section 2.14). Further, the assessment tasks themselves were created through a specific commission 50 to the engagement of people (not members of FROG) who previously had been involved in other professional qualifications. It is here in the review of this process that we witnessed one of the few explicit discussions about the nature of leadership and the outcomes to be achieved through the FRH: “Now one of the, I think, great opportunities of an alternative route to headship was to define a vision for what the head, a head should be for the twenty first century, not: ‘What does the standard say?’ but: ‘What kind of head do we need for the twenty-first century?’ and: ‘How does that articulate with the standard? How then do we work toward developing this kind of leader?”’. Despite the aspiration for parity with the SQH, however, and the stated need to reflect the assessment processes of that professional qualification, the FRH programme―to re-inforce the point we made earlier―had been presented to candidates as “non-academic”. Consequently, there was significant disquiet about the demand for written assignments as part of demonstrating one’s achievement of the SfH. One coach thought it had been a mistake to present the programme as non academic: “The first page of the leaflet that I first came back with this [has the claim that] this is not an academic programme. That was a fundamental error to write that because it is academic”. Describing the FRH as non-academic had created false expectations at the outset, particularly regarding the assessment process. The concerns among some candidates about undertaking written assignments were clearly evident, as we have already highlighted. These concerns prompted the development of another mode of assessment: an oral mode, in which the candidates could write up tasks according to the rubric and then present their work orally for formative and then later for summative assessment. The possibility of other forms of assessment was also raised by coaches subsequent to the completion of the first full contract cycle. A third approach might have included provision for an extended school visit and greater use of the ECI. At the time of the evaluation, however, candidates in the first group to complete the final assessment had undertaken the written mode. (One candidate was about to embark on the oral mode, and so it was not possible to include this in the evaluation.) Reflective commentary As part of the evaluation, the reflective commentaries and the portfolios of evidence submitted in the first wave of assessment were reviewed by the evaluation team using the SfH and the criteria for assessment agreed by FROG. There is no doubt about the considerable effort, hard work and investment of time which has gone into the reflective commentaries and the portfolios of evidence. There are, however, a number of issues that emerged from this analysis. The reflective commentary was seen as a worthwhile task by the coaches: “I think the commentary is an excellent exercise provided it is really based on personal reflection on [...] leadership development”. And, as we suggested previously, the development of the two formative written tasks was a key focus in the coaching sessions. The support through the coaching sessions was important in producing the very marked shift towards the end of the pilot (i.e., for those who presented in May) in the candidates’ understanding of the potential of more academic forms of learning. The practice-theory divide was also much less in evidence. As one candidate put it: “On a day-to-day basis, the theory very much becomes practice”. 51 From the analysis of the reflective commentary undertaken by the evaluation team, three issues emerged: (1) coherence, (2) use of reading, and (3) criticality and consideration of the wider context of headship as detailed in the SfH. There were also differences in the quality of some of the written reflections. There was a small number of examples of strong submissions, in which the candidates reflected on their practice critically and used some literature to assist in that analysis, but these helped to expose the flaws in others: in many cases the submissions lacked the cutting edge one would hope for in this standard of work, a view also expressed in the interviews: “The overall quality was good”, although there was “one that I would have said, though, would have definitely failed”. Lack of coherence There was often a lack of coherence in the commentaries, and sometimes a lack of clear connection to the portfolio of evidence. In the construction of the rubric for the reflective commentary, the apparent blurring of the need for candidates to claim competence (the justifications for which, by their nature, should be descriptive and assertive) and to be critically reflective of their practice and learning as leaders, contributed to the overall lack of clarity. There was a wide variation in the quality of the candidates’ reflections, ranging from some thoughtful submissions to many largely descriptive pieces of work. The quality of reflections was affected by the use made by the candidates of literature to inform their analyses. Many were operating at a fairly superficial level here and there was not a lot of evidence of encounter with, and reflection on, quality research evidence. (This is partly why earlier in sub-section 7.2 we recommended that HEIs be represented on the proposed FRH management group, in order to provide academic guidance on robust research evidence and reading.) There was a marked tendency for candidates to drop in quotations and specific terms without explanation or reference, rather than engaging in conceptual analysis. Readings were taken as largely unproblematic and the ideas in them as largely uncontested. The limited criticality was also evident in the understanding of wider issues, particularly related to elements 4.1 Strategic Vision, Values and Aims and 4.2 Knowledge and Understanding of the SfH. There was a strong sense here of a personal journey rather than an all-round demonstration of how the standard had been met. Nor was a holistic understanding of the SfH always evident. Critical incidents sometimes seemed like appendages and unrelated to key themes. There was little sense of the candidates’ work being set in the wider framework of a whole school’s capability for change or improvement―for example, in terms of school culture and the importance of shifting attitudes and practices in a fundamental way. Similarly, the sense of external influence was very localised, with little evidence of awareness of the wider political context and levers for change which influence school leadership. There was a lack of evidence of the leadership process being contested or problematic and a lack of understanding of dilemmas of headship which, from other data gathered in the course of the FRH evaluation, was clearly a significant focus for the coaching sessions. The portfolio: Over-egging the pudding The need for the portfolio in its current construction was questioned by the coaches after they had completed the first wave of summative assessments. Those portfolios submitted for summative assessment were viewed as over-lengthy and making a significant demand on candidates. The concerns of coaches are clearly echoed in the views of candidates: I ended up with 60 pieces of evidence, some of which were single pages. Some might 52 be ten pages long. This candidate’s portfolio required a four gigabyte memory stick to hold every letter, every memo and every policy document which the candidate professed to sifting through to match to the SfH. A widely held view was that much more guidance could have been given as to what constituted good evidence and what a solid portfolio should look like. The emphasis on what appeared to some as mechanistic busywork, it was said, could have been better served by other forms of communication: I think the degree to which you have got to annotate the portfolio and the hours that I feel I put into doing that, when I feel I could just talk somebody through it. I’m sort of toying with the idea of even just having a wee tape recorder myself at home and taping it, which would be much better for me than writing it down. These issues raised by both coaches and candidates concerning the portfolios were also confirmed by our analysis of them. The portfolio evidence provided was extensive, but often descriptive, rather than evaluative, and the claims for competence made against it were sometimes dubious. It was often limited to evidence of action taken―for example, meetings held―rather than evidence of impact (e.g., on pupil learning). Much of the evidence seemed almost random and there was often no sense of an evidence trail related to evidence-informed practice and evaluation processes. This may have been due partly to the process adopted in the construction of the portfolio. This process seemed to have started with individual items of evidence which were then matched to many areas of the SfH. This practice can lead to sense of randomness in the processes followed. The general lack of evaluation mechanisms for the work carried out meant that impact on learning and teaching was rarely discussed. Reviewing its helpfulness towards the end of the programme, one candidate commented that the portfolio might have been of use to the GTC and the assessors, but that “it certainly is the least beneficial in terms of my learning”. There was also a strong recommendation from some candidates to those who might follow to collect evidence more strategically as they go along, and for it to be a key aspect of the support offered by coaches. Being told at the outset the number of copies required of the portfolio would also have been helpful. Among the coaches there was agreement that the candidates did have to provide evidence, but that the construction of the portfolio adopted in the pilot in which evidence is gathered to cover all aspects of the SfH―Professional Actions and Three Essential Elements, rather than merely the Professional Actions―was seen as an issue, as this focus had given rise to the production of very large portfolios. The original guidance had proposed that the portfolio of evidence should focus solely on Professional Actions. Presenting the portfolio, using the Professional Actions (and the processes implicit in these) supported by evaluative evidence, however, would reduce the number of items of evidence and address the issue of fragmentation in the portfolio, which were clearly concerns for coaches and candidates alike. Summary This sub-section has provided an extensive coverage of a range of aspects of assessment and learning tasks in the FRH. The intention has been to illuminate features concerned with the quality of the candidates’ work and to highlight the assessment role played by coaches. On the basis of what we have seen and observed, our conclusion is that the assessment process 53 would benefit significantly from re-thinking some of the role relationships, elements, size and bulk of the task required of the candidates, and particularly the link between the portfolio and the candidates’ current and future practice. Recommendation 17 That the management group undertake a thorough review of the FRH assessment process, giving particular attention to: • • • • the balance of formative and summative assessment the extent to which the provision of formative and summative assessment ought to be part of the coaches’ role the purpose and length of portfolios, commentaries and evidence, and the nature of the links between these elements the question of whether and how portfolios, commentaries and evidence relate to the entirety of the SfH 10.5.6 Field Visit A reality check The field visit was seen as a vital component of the assessment process and one which all coaches had found very rewarding―it completed the picture: “Outstanding. That’s the best part of the process by a million miles. For the field assessor it’s wonderful and I think for the participants they enjoy it”. The format seemed to have been quite flexible in terms of the number of people with whom the coach-assessor discussed the work of the candidates and whether it would entail a tour of the school. The field visit was broadly welcomed, in retrospect, as a further source of evidence and a reality check as to one’s leadership style and self-awareness: If your perception of what you do is not what everybody else has then it’s critical that somebody’s going to come out and speak to people to find out what they think. One candidate remarked that much of what is written for Masters degrees for HEIs has to be taken on trust, while in the FRH programme “triangulation is one of the best forms of assessment, even though it was a bit nerve wracking”. This candidate added: That was the first time anybody had actually ever checked up to see if what I was writing about my work was actually true. Getting feedback at the end of the visit was critical. Candidates commented favourably on feedback as constructive and helpful, even when it cast their own self-concepts or practice in a different light: The field officer had rigorously checked through the materials that I had sent in and made some very detailed comments and observations which she teased out in the field visit about things that she felt were lacking or not fully developed. This, however, was not true in all cases. Some candidates did not get feedback after the field visit and felt uneasy at having to guess at what had been said or not said: It was lunchtime and [the assessor] had to go and I would like to have sat with [the assessor] again and got a bit of feedback…about what other people had said about me. 54 Issues had also arisen in instances of candidates who had changed schools and the work in the submission was based on the school where a candidate no longer held a post: it was felt that there was not the same immediate sense of commitment and support. Nevertheless, the assessment had been completed successfully. Written feedback from the field visit was given to the coach who then worked this through with the candidate in coaching sessions. It is clear that this feedback was valued. It bears repeating, however, that the field visit is only a formative process. Potentially, this is a process in which a number of people who have been involved in the work of the candidate are able to provide verification of the quality and impact of the work in accordance with the demands of the SfH. This is not possible, however, with the current status of the field visit as a formative task. The consequence of this is that the written tasks are the central focus of the summative assessment process, which potentially runs counter to the aspiration of creating a development programme, the centre of which is experiential learning supported by coaching. Further, as is evident from the interviews with coaches, the preparation of the written tasks also absorbed valuable coaching time. The positive features of the field visit led the coaches to consider ways in which these could be extended to include a tour of the school, as this would allow the coach-assessor to see the candidate in context interacting with a range of people―staff, fellow managers and pupils. This process would become part of an evidence-based approach to coaching and allow for feedback to be given in context. No doubt there is merit in this idea, but it would be important to clarify roles and purposes. This type of visit could be part of the ongoing coaching work (the coaches reported that they had undertaken this type of activity as part of coaching rather than for the assessment process). Equally, this type of visit could be part of the summative assessment processes in which the emphasis is on verification of practice, thus rebalancing not only formative and summative assessment but also placing experiential learning and practice at the heart of the entire assessment process. Recommendation 18 That the management group review the place of the field visit in the coaching relationship and assessment process, and explore the possibility of it being part of the summative assessment of candidates. 10.5.7 Panel Presentation The final hurdle As part of this evaluation, the views of coaches, candidates and other stakeholders were collected on the panel presentations. In addition, members of the evaluation team attended as observers of 12 of the 14 presentations in May 2008. The interview at the presentation was generally seen in a positive light by the candidates, and was described as “supportive”, “informal” and as “a professional conversation”. The preparation time and the presentation were seen as very helpful both in the short term and as an investment in the longer term when being interviewed for a headship post. Rehearsing presentations and getting feedback from peers during ad hoc mini courses was also appreciated. Some candidates, however, did still feel unclear about the purpose or focus of the interview and did not know quite how to “pitch it”. Was it meant to be about the learning journey or about a demonstration of headship competences, about speaking as a head or as a depute? “They weren’t trying to catch you out or make you look stupid”, commented one 55 candidate, adding that it was a fine line between the formal and informal nature of the event, although that the balance had been well managed. There were, however, three kinds of caveats for the candidates: • • • The anonymity of the panel Length of time for presentations Lack of formative feedback A few candidates commented on the anonymity of the panel, not knowing who the members were and not being introduced to them: As a matter of courtesy it might have been nice for me to have a nameplate and for them to have nameplates because there were five or six people and it would have been nice to know who I was discussing these issues with. A 15-minute presentation was seen by some as too short to allow them time to develop their ideas, while a number of candidates commented on the lack of feedback at the end. As one candidate described it: “‘You’ve passed. Well done. Off you go”’: There’s a stock sentence that they use at the end, something along the lines of: ‘I’m now pleased to tell you that you have passed and achieved the standard for headship, congratulations’. And you say: ‘Oh! Thank you very much.’ And you look at them and they go: ‘That’s it’. There’s got to be a better way of finishing off. Some candidates did get full and helpful feedback from their coaches but still felt that they would have liked something more at the time. Written feedback sometime later was also seen as less helpful than a face-to-face response: It’s quite different just getting a piece of paper. I would have preferred to have sat down and at the end: ‘How did that go?’ and ‘What do you think’? Having better knowledge of the physical set-up would, apparently, have helped some who said a clearer visualisation of procedures would have made the experience less daunting, while the added value of a cup of tea would have been welcomed or some form of celebration afterwards rather than, as depicted by some candidates, walking off into the sunset, with no real grasp on whether their performance had been a borderline, good or excellent pass: “Was it a scraped pass as opposed to a rubber stamp? ‘Yes. Absolutely fantastic!’”. Leaving the GTC building, there was a kind of sadness as well and: “Gosh, that’s it. Over”. In our observations of the presentations, the following aspects were considered: (1) the presentation itself, including its quality, and its connection with being able to demonstrate achievement of the SfH; (2) interview questions and responses underpinning understanding of the contested nature and purpose of headship, and demonstration of the achievement of the SfH; and, (3) the process of the interview, particularly its purpose and strength as an assessment tool. Presentations of self Most candidates seemed very well prepared and professional in their presentations to the panel. They had obviously put in a great deal of time preparing for this final part of the assessment. Discussion of personal change was a common thread and many candidates were 56 impressive in their ability to analyse the journey they felt they had made personally and professionally. The effect of the coaching was clear to see for many candidates who were able to reflect with some insight on their development, especially in relation to interpersonal skills and abilities and self awareness. The standard set of questions used by the panel also reflected the emphasis on coaching and the focus on interpersonal skills and abilities, all of which were drawn from only one element in the SfH (4.3 Personal Qualities and Interpersonal Skills). This priority meant that it was difficult to assess the candidates’ understanding of the SfH in a holistic sense, because so many aspects of the SfH were not addressed in the presentation or the questions. The focus on interpersonal skills and abilities and the candidates’ personal journeys resulted in a lack of depth in probing their awareness and understanding of conceptual models of leadership, the wider political context of education and leadership, and the tensions inherent in the role of head teacher, all of which compounded some of the concerns we have already raised about the reflective commentary. On the other hand, the process was very supportive of the candidates and was well managed throughout. Candidates were treated with courtesy and encouraged to do their best. Two issues emerged, however, during the discussion with stakeholders: first, the issue of timing and, as we have already suggested, the need for immediate feedback (within 15 minutes); second, the inclusion of a parent representative as a panel member. (In addition, the issue of longer term sustainability of these panels arose, because of the potential burden of organising and resourcing imposed on the GTC: see section 12 below). Assessing presentations The time given over for discussion of the candidates’ performance was quite short and, together with the fact that they were waiting outside the room to hear the result, could have led to a fairly speedy conclusion by the panel with almost the assumption of success. Indeed, any negative feedback was seen as the basis for follow-up coaching, rather than the possibility of a decision to fail the candidate. There was often discussion about the candidates’ presentation of self, rather than their achievement of the SfH, which sometimes made the interview feel more like a job interview for the candidates than an assessment of their capabilities against the SfH. It is the specific nature of this panel interview as an assessment process through which an individual will be adjudged to have met or not met a professional standard that raises questions for us. Strengths were identified: “That seemed to work really well. I was very pleased with that. I was very pleased with the contribution of head teachers and the Local Authority reps, and they had been excellent and really gone to the heart of the candidate’s submission and the candidate’s referencing the standard”. As this presentation panel was an assessment process, however, the justification for the membership and role of a parent representative on the panel is not entirely clear. I do have real qualms about the Executive or the government’s desire to have parents in these panels. I’m not convinced, I haven’t heard a rationale for having them there first of all […] the analogy is perhaps made with an interview panel for a job where there’s a parent on the interview panel and I can see the role there […], the parent would have a real knowledge of the school, a real understanding of how things work, 57 real vision of how the school should be developed. That’s fine. But this is measuring somebody against a standard, a professional standard. Another interviewee echoed this concern, citing an incident in which the parent member of one of the panels pursued a series of questions that were not related to the assessment schedule: “One parent just went off on their own issue about hearing impaired and that was not relevant. But that’s exactly why I didn’t think parents should be there [...] Now that’s alright at an interview but it’s not in this part of referencing the standard. I also think it could be very unethical and dangerous for us that, it’s possible that a parent could have a casting vote on deciding whether a candidate meets the standard and if you take that to appeal, and somebody will, what knowledge or experience does that parent have to make an informed decision, and would be found wanting”. Recommendation 19 That the management group consider fine-tuning the role of the presentation panels, giving particular attention to the panel composition (especially the validity of parent representation), choice of questions in relation to the SfH, provision of feedback to candidates and length of presentations by candidates. 11. The Impact of the Pilot Measurement of the impact of programme interventions, in order to be able to demonstrate changes in attitudes, behaviour, actions and the like, in a preferred direction and within a desired time frame, is a notoriously difficult exercise. From a strictly technical point of view, there are protocols that need to be observed. Thus, provided that two group samples, otherwise alike in every respect in their attributes and characteristics, perform or react differently when subjected to an intervention treatment, on the one and hand, and nonintervention, one the other, with both groups under similar conditions, then there are firm grounds for confidence that an outcome has been caused by the treatment. Likewise, in the absence of matched samples, the existence of two sets of data, one at a baseline and the second at time 2 following an intervention may, when compared, demonstrate some degree of change in a desired direction. Evaluation of the FRH pilot lent itself to neither of these conditions. As we indicated in the Introduction, the timing of the evaluation began about eight months after the pilot’s commencement and it ended before it was completed. In the methodology that we proposed we conducted two separate rounds of interviews, interspersed with focus groups, in order to ascertain shifts in the rhythm of the programme and changes in the thinking, attitudes and perceived readiness for headship development of the candidates. We also tried to track the progress of candidates by having them complete regular electronic diary entries. A few attempted to, but on reflection this was perhaps too ambitious, given the busy lives that they all lead. In the end, then, we had three main sources to draw on for evidence of impact: first, direct self-reports by the candidates on their experiences; second, indirect reports on the candidates by coaches and workplace colleagues; third, the candidates’ written submissions and oral presentations. Even these sources were complicated by one other factor, to which we drew attention earlier in Recommendation 6. This was that the sample of candidates with whom we were working was slightly biased, in that at least some of those selected to take part were considered very likely to have been appointed heads in the near future. We begin with a summary of the overall impact of the programme on the candidates. We then suggest some of the more specific ways in which aspects of the pilot had an impact. 58 Impact: Shedding the comfy slippers A recurrent theme among candidates is that the FRH pushed them out of their comfort zones or “comfy slippers”. The word “forced” is frequently used. Impact was described in three ways: as personal, professional and as organisational. For those who had not seen themselves as headship material, the FRH had challenged their reticence or complacency, while for others it had given impetus to initiate school-wide change and challenge their colleagues. The main personal and professional impact was widely described as a growth in self confidence and self belief: I don’t think I’m a different person at work. I think I’m more of the same person though. This view of the added value of the FRH may be counter-pointed with another perspective: that of finding the person within: Quite a lot of my values have changed and I have become much stronger in things I believe in. Referring to others, one candidate described watching them growing, “seeing how far they had come in six or nine months was actually quite startling”. Rather than “shying away from conflict”, as another candidate put it, a number of others talked about a new willingness to see conflict in a positive light and to welcome it as growth-promoting. At the same time, being less judgemental or suspending judgement was cited as a major professional benefit of the course. Many contrasted their previous styles with a new found attempt to be more reflective, listening and supportive of their colleagues. Self-confessed, high energy, driven heads or deputes found it both difficult and salutary to “give people more room”, “create the space”, “stand back” and “listen more”―“a fundamental mind shift”: I’ve completely changed my stance in the way that I do things, from feeling that I have to answer everything to now being somebody who sees themselves as facilitating others to find the answers for themselves so that they will actually move their lives forward. Impact was described as going beyond the personal and professional to organisational change, what might be described as capacity building and mining the treasure within (as the Delors Report put it). There were recurrent references to a greater self-efficacy in developing colleagues and distributing leadership. There are descriptions of school-wide change as candidates took the initiative, for example, in reforming CPD or recasting roles and responsibilities. There was, in some narratives, a Damascus Road element, as in accounts of “taking on the world” and “having the power to change things”, yet this was combined with an element of deskilling, for there was in many accounts a sense of inadequacy. Some spoke, for example, of previously doing everything “hellishly wrong”: It made me feel quite bad about myself, you know, in the sort of person I was…I was expected to deconstruct myself. 59 “Some people thought they were fantastic and then found out: ‘No you’re not’”, said one candidate. The impact still being experienced six months later: “That really hurt to think I wasn’t the person I thought I was”. Frustration or a sense of inadequacy are revealed in descriptions of schools in highly challenging circumstances (external and internal), troubled communities, split staff, internal rivalries and inadequate head teachers. Some accounts in highly challenging schools reveal a struggle to cope with the distance between high aspiration and the day-to-day realities of their schools and communities. As one candidate commented, “you can’t move everything”. This is a reminder of the old adage―the insight to know what you can change, what you can’t change and the wisdom to know the difference. This tension between the highs of the pilot and the lows of life back with the nuts and bolts of school life, leaves open to question the extent to which prospective heads are enabled to manage the politics and micro-politics both internally and externally, which requires a quite different skill-set from a softer coaching approach. This dilemma is mentioned in one or two of the interviews but is a missing element in most. While this suggests that it has been a theme in coaching sessions, it does not appear to be one that has been to the fore in people’s thinking and accounts of the programme’s impact. The candidates’ stories are predominantly about their relationships with colleagues, with their new-found ability to listen more, delegate more and team-build more effectively. By contrast, there are few references to the role of leadership beyond the school, in work with other agencies, with parents, with pressure groups, and the delicate negotiation and prioritisation skills required in managing external relations. Inter-professional politics and politics at LA and national levels appear only in the margins of their accounts. While handling conflict within the staff emerges as a theme, conflict externally with other professional and voluntary groups, with school governors, parent councils, examination boards or trades unions, is conspicuous by its absence, although there is one reference to a mini-training event on handling the media. In a context of extended schools, new community schools and the increasingly contested nature of the interface between school and community, these issues would appear to merit a much higher priority. Levels of impact Having provided this general overview, we see three potential levels of impact: first, the development of self-confidence and personal skills of candidates to take on leadership roles; second, the outcomes achieved through the candidates’ practice in schools in enhancing the work of their colleagues; third, the overall capability of schools to provide quality learning environments for the improvement of student learning. The evaluation team has evidence of impact at the first two levels (see below) although it has no evidence of level 3 impact. With these caveats in mind, we offer four illustrations of the programme’s level 1 and 2 impact on candidates as a result of their participation in the FRH pilot. As we have suggested throughout this report, a key strength of the FRH in the eyes of its proponents is its transformative potential. In addition to the evidence of transformation sprinkled throughout the discussion so far, each of these four examples to some extent affirms that potential. 60 Identity in transition Our general impression, from our informants’ own testimonies is that, for the large majority of candidates, their confidence in themselves and their leadership capacity has been boosted significantly thanks to their participation in the FRH. It is difficult to be precise about the numbers of candidates for whom this is the case, because not all of them expressed their feelings equally openly or forcibly. The kind of shift in attitude that we became aware of is evident in the following example. One teacher, who would have been “happy to sit in the post that I was in, happy trundling along”, had to be strongly encouraged by a head teacher to apply for admission to the pilot. This candidate also had no aspirations in applying for a promoted post and was, “like most people in the west of Scotland, very conscious of my weaknesses”. Indeed: I had very little self-confidence. I was aware that I was hard-working and I knew that I tried my best but I wasn’t altogether sure that that was, you know, that I had particular strengths in any areas, you know, in spite of colleagues saying: ‘You’re exceptionally good with that child’. ‘You can do this X, Y, Z, with parents’. I never particularly felt that I had, if I’m honest, deep down I knew I must have had strengths somewhere but I was never particularly conscious of my strengths. Contrast the tone of the self-scrutiny in those remarks with that in the following extract from the same candidate in the second round interview: I don’t recognise myself sometimes, truly. I appreciate that I still have a long way to go but there’s some days that I do not recognise myself. Today, as an example, I was at inter-agency training today. Now that is very much out of my comfort zone…Social workers, health workers and talking about self evaluation and, I mean previously even if I had attended something like that, which would have taken a lot for me to do, I know that I would have sat and I would have contributed very little, not because I don’t have anything to contribute, but because I wouldn’t have been confident that what I was saying had any value in it. And today I found myself spouting. It might not be perfect but I was conscious of kind of pausing and thinking: “That’s you”. And that’s a huge thing for me, it definitely is. Here is a candidate who is clearly beginning to reflect. Taken together, these extracts provide clear evidence to us of a candidate’s identity in transition and of that candidate’s growing sense of professional mastery. A wake up call An arguably more powerful influence on personal change than reflection is feedback. When feedback is about oneself as a person, rather than task performance, negative feedback or disconfirmation can be more significant in its effects than positive feedback. As Hattie & Timperley (2007, pp. 98-9) argue in their review research on feedback, it is because “individuals will go to great lengths to confirm their self-perceptions by attending most closely to feedback information that fits their view of self and by trying arrange their environment to acquire further self-confirming evidence”. As our next example indicates, dealing with negative feedback also demands deep reserves of resilience. Prior to arrival at Peebles for the first residential, one candidate was “walking on air, you know, I just felt really good about things and about how things were going”. This person then received the 61 ECI results: [These] weren’t quite what I was expecting and the realisation about the operational management and so on, and it was like a, I think I fell to pieces that weekend actually. I really realised that I was going to have to massively move my perspective in order to become an effective manager. And it was hard, because I was sitting with people who by and large were, sitting with lovely people who were very happy about themselves. It took a month for this candidate to “start feeling confident again about myself”. The significance of this example in respect of evidence of programme impact is two-fold. On the one hand, the personal pain revealed in these extracts reinforces the points we made earlier in Recommendation 12 about the need for feedback to be managed sensitively when using diagnostic instruments such as the ECI. On the other hand, the candidate’s mention of “operational management” is a trigger for questions about the target at which the FRH is directed. Much was made in the post-interview discussions at the presentation panels of the need for candidates to make the transition from thinking operationally to thinking strategically, if they were to rise to challenges of headship. The question for us is: How well does personal transformation sit with being strategic? At the panel presentation, there was agreement that, 15 months after the incident recounted in the first quotation, this particular candidate had been transformed by the programme and had made the transition to a “strategic” approach. But this case may be the exception rather than the rule, as it is possible to be personally transformed by a development programme and still not to be “strategic”. Likewise, to think strategically is a cognitive capability. It has no necessary connection with transformed character and personal qualities, which are about self-hood. In short, there is evidence here of a possible disconnect. Unintended consequence This next example of evidence of impact is a very graphic and heartening one. It will be a source of comfort to proponents of the potential power of personally transformed people, although for the evaluation team its significance is slightly different. When they were interviewed, a number of candidates suggested that a spin-off of having been coached was that they were beginning to adopt a coaching style in their dealings with colleagues. This is a classic example, perhaps of an unintended consequence. That is, something transpires that is not necessarily envisaged by anyone and no-one anticipated that it would occur. Selfconfession by candidates of changed behaviour is one thing, but when a person other than the candidate relates a story such as the following about a field visit, that is a qualitatively different matter: I did [a field visit] in [name of LA] and two of the staff were in tears. I met the head obviously, I met the participant, I met a principal teacher…and I met the head librarian who had been heavily involved in art weeks and book weeks and things like that. The person I was assessing had also been involved…and this principal teacher…said that the work of this [candidate] over the past sort of 15 months had totally changed [this teacher’s] life…hated [the] job…was disillusioned, everything, you name it…just a worn out teacher [who] said this [candidate] came in and the first thing…was listen to [this teacher] and the relationship was built up, started to look at what was good about the department, what you need to improve, looking at attainment, looking at achievement of children and [this teacher] said: ‘I’ve never known anything like it’…in tears... Then the librarian…said similar things, you know: 62 ‘I’ve never been valued before. Nobody’s listened to me’. This example is evidence of a number of things, including role modelling, personal style and influence as a leader. Proponents of transformational leadership, for example, might see it as evidence of one of the four “Is” that originally defined this model (i.e., individualized consideration), although interestingly the coach thought it exemplified “distributed leadership in its true sense, in that it’s about listening to people, it’s about giving them opportunities, it’s about working with people, helping to develop other people”. However it might be interpreted, it is unsolicited evidence of impact. Crossing the Rubicon At the outset of this report we cited the Executive’s hope that an initiative offering flexibility to prospective heads might begin to have a demographic impact on Scottish head teacher supply (SEED, 2006b, section 1.3). The bottom line for us is that there is no way of knowing whether an FRH candidate’s success in securing a head teacher appointment, either during the programme or subsequent to it, has resulted from taking part in the pilot or would have occurred in any case. As we have seen, a number of the FRH candidates did become heads during the pilot and others certainly began applying for posts. But not everyone had such aspirations at the outset: I had no real conception in my own mind about being a head teacher. I am more than aware, looking at my own head teacher here and the experiences of others, how difficult a job it is. I don’t know whether at time I felt as if I was prepared to put myself through that or my family through that […] the idea of leadership and head teacher as an integral part of leading the school and leading change, that’s quite a pressure to take on board. So the initial aspect was to give me an insight into the job I was doing here already. To make me better at that, to make me more aware, to make me think more about what I was doing. Because the job I’m doing just now is very much or can be very much just day-to-day led, and it’s administration, it’s management, it’s not strategic planning. So the areas that were in my remit of CPD and of enterprise, I felt that I wanted to step back and do a bit more sort of reading and research and thinking about them, rather than just sort of doing things, you know, crashing through them because that was the next thing to be done. What about later on in the pilot—had anything changed for this candidate?: I’ve started applying for jobs: bottom line. I put in for the first one about three weeks ago, which I didn’t get on the list because my CV is not detailed enough yet, so that’s I suppose kind of crossing the Rubicon for me. I have still a lot of issues […] those issues are still kind of bubbling about, but I have started to apply for head teacher posts, well one, and I now know kind of, where the land lies in terms of where I’m going, so I will in the next sort of, six months I think, start to push the boat out and start to do more thorough preparation in advance of the expectation to get on lists and be interviewed for head teacher posts. Whether I ever get there’s another story. Once again, as with the former example, this quotation provides evidence of a candidate’s identity transition. Unlike the first candidate in this section, however, confidence was not something lacking for this candidate and there is not quite the same sense of self-reflection having taken place. On the other hand, the issues here were to do with questions such as: What for? And, why? The internal mental conversation about the FRH in this person’s mind―evident in the transcript segment omitted here; i.e., [..]―was “Why am I doing this?” 63 This is evidence of a candidate trying to be honest about her/his motivation. Other candidates had also applied for admission to the pilot as a form of CPD, as personal growth, as expanding their professional repertoire, often without ambition for promotion, or in “order to become a better [leader]”. Such ambition, or lack of it, had to be concealed at the point of interview, it was said by one candidate. Others who hoped to be better leaders in their own senior positions were prepared to challenge themselves as opportunities for headship arose. There were at least five categories of response: • • • • • Actively seeking headship posts Planning a career elsewhere in the education service (e.g., SEED, LA) Securing headship posts during the programme Planning a longer term strategy Content to stay unless all the conditions are right While most candidates said they were now ready to take on a headship and while most who had completed the FRH in May were actively applying for headship, others were happy to bide their time. “If one doesn’t come up I’m happy doing what I’m doing”: this statement seems to speak for a substantial number for whom the various conditions had to be right―the travel, the location (with family in mind), and the type and size of school. The limitations imposed by this cluster of factors are expressed by one candidate as: “It would have to be this side of the city”. For some the challenge of headship is seen as an internal struggle to shed the “comfy slippers”, while others are happy to take a measured approach: I would like to have a few more years of deputy under my belt before I would consider headship. Personal, domestic and family circumstances loom large in many of the candidates’ accounts and have a major impact on issues of work-life balance and intensification of the role. Unexpected, or planned, events such as marriages, pregnancies, separation and divorce were all cited as factors that needed to be factored in to their decision-making: [Due to family considerations] I’m not wanting a head teacher’s post straightaway. I’m not thinking about it for another three, four, five years time kind of thing. Despite the enhanced self-assurance brought about by involvement in the programme, there remains an element of being encouraged and “needing a push” into applying for headship and a lingering reluctance among some to leave the comfort of the familiar. A candidate who became a head teacher during the course described it as “a tale of two experiences”: the first highly positive, the second the inability to cope with the “steep learning curve” of becoming a head and then residual energy needed to complete the demands of the programme, and then concluding: It was stimulating, got me really focus on it, it probably got me to write better application forms and make better presentations for the job and I ended up where I wanted to be as a head teacher. I just don’t have a certificate on the wall. At the end of the pilot a powerful metaphor was used by one candidate to convey the integrity and depth of the experience of becoming truly ready and equipped for headship, as like a stick of rock that you could cut through at any point and find the consistency of leadership qualities. 64 CODA There was a final plea from candidates for some form of recognition, a certificate in return for the hard work and huge emotional and intellectual investment perhaps, and “the amount of work and sweat and tears we put into it”, while the (ultimately academic) standard achieved was seen as needing a commensurate accolade: Three of us have got M.Eds and we felt that, although it wasn’t M.Ed standard, it wasn’t a kick in the pants off it. You know the integrity of the assessment and the depth of assessment and the level of training and the level of research…I do feel that it should be professionally recognised. Summary The four examples summarised in this section could be multiplied. Taken together with the general overview, their significance for us is that they reinforce the need to attend to some of the threads of leadership and learning touched on earlier in the report, particularly in Recommendations 1 and 14. In the first of these we highlighted the importance of conceptual clarity about leadership and in the second we mentioned school improvement. If, as part of the thinking that sits behind this FRH pilot, a way could be found of bringing these two elements together more explicitly in the rubrics and practice of the programme, then the FRH might be able to tackle the third and elusive level of impact we articulated at the beginning of this section: the enhancement of school-wide capability. To this end, some recognition of existing knowledge concerned with adult and related areas of learning might be useful. Recommendation 20 That, as part of its review of the concept of leadership underpinning the FRH, and particularly with a view to enhancing the participants’ effectiveness in school improvement, the programme group consider utilising research in the areas of professional, adult and workplace-based learning to assist in the development and re-development of the practice of senior and experienced professionals. 12. National Implementation of the Flexible Routes to Headship Section 5.4 of the evaluation specifications required us to examine the possibility of the implementation of the FRH (see Appendix i). We took this to mean across Scotland, in the same way that the SQH is available across the country. For this reason we used “national” in the sub-heading for this section, although given our description in sub-section 7.2 of how the current arrangements operate between LAs and other stakeholders in relation to the SQH, our choice of national may be inappropriate. At the end of the day, if our recommendations are adopted and the FRH does become an alternative pathway to the SfH, it will be LAs which will decide whether or not to opt in or out, by directing or not directing resources to the FRH (see Recommendation 3). In this section we have synthesised the principal considerations that are likely to influence those decisions. They form part of what Elmore (1996, p. 4) has described as getting to scale or scaling up with educational reforms or innovative practices: “understanding the conditions under which people working in schools seek new knowledge and actively use it to change the fundamental processes of schooling”. While the specific context of Elmore’s (1996, p. 18) remarks is changes in the core of teaching practice, and the reasons why these may or may or 65 not “travel” from setting to setting, scaling up in head teacher development is concerned with the factors that might facilitate or inhibit the adoption by LAs and diffusion of the FRH as a virtuous innovation. Finance The major concern of LAs is funding, because “we have to take decisions on an annual basis now which are budget-driven”. The evaluation specifications did not require us to consider budgeting and costs associated with the FRH pilot, but on the other hand, it is impossible to avoid this area (or any other area for that matter) when our informants raised it with us during interviews. There was no opposition in principle to the idea of making two routes―SQH and FRH―available to prospective head teachers. “I think this [pilot] is a good opportunity to see if it works at minimal cost to us and then we would hope to be able to go down both routes and to offer that kind of choice to people if it’s going to be successful”. The bottom line was that there needed to be “parity of esteem between these two”. “The standard is the standard and they have to be equitable”. Moreover, the availability of the two routes was viewed positively: “I actually think by having the two routes within the next year and a half, if it rolls out, I will say […] will double the volume of people” who apply to participate. The stumbling block, however, was money: “The problem I’ve got, then, is: How on earth do I finance it? You know, if we can currently finance it”. Similarly, another informant asked: “Is it [i.e., the FRH] rolled out with additional funding”? In the absence of sufficient funding to afford both routes, LAs were inclined to forego offering the FRH and would continue funding the SQH: [The FRH has] certainly generated a lot of interest. I mean, I’ve taken, I’m only going through the cycle for the kind of first time really, but I’ve certainly taken a lot more calls this time than I did last year and I think people are interested in the two different routes. And it’s nice to be able to come on and say: ‘This is your choices’. So, I’d like to say that they wouldn’t be in competition, obviously. Until we were further down the line it would be really hard to say: ‘Yea’ or ‘Nay’ to that, but I would hope not. Because I think they would suit different people and I think it would actually increase the numbers that we could have potentially. I think where there might be a problem is if all has to come from the same pool of money, because then obviously we would have to probably be sending the same number of people but just within, you know, the places. I hate to be kind of always harping on about money but it’s probably true. Infrastructure There were a closely associated set of concerns about “infrastructure”. There was some doubt about an LA’s capacity to provide trained coaches: “Do we have trained coaches to that level?”: At the moment, no I don’t think we do. We have people who are still building up their coaching through the Diploma [?] but most of them are experienced leaders and to take them out of their situation gives us another problem in terms of filling it. So there would be big implications and things to think about there. One unanticipated cost saving resulting from the pilot, of course, was the decision to use coaches instead of field assessors. If in future this decision was rescinded and the FRH reverted to the original idea of trained field assessors, this would introduce an additional cost 66 burden on LAs: The field assessor, I’m not sure about that and there was no discussion with us about that. That simply happened and we were told about it and while I can see some benefits, in that the coaches are steeped in the programme, I think in terms of the integrity of the programme it would actually be good to involve people, like head teachers who are outwith it. Because I think the coaches, although they are going to be assessing different people, obviously who they are not coaching they are very close to the programme and it will be more difficult for them and I have no difficulty with their professionalism, but it is more difficult for them to be objective and I am not sure that it is positive. And I think that in terms of the robustness of the programme I think the perception is that this a way of saving money because it would cost money to train people… So if that is the reasoning I don’t think that’s so good. A dilemma here is that while the FRH offered so much through the intense coaching it provided prospective heads, that intensity may not be “financially viable” or to “fund your people coming into school for half a day every week”. A related infrastructure issue for LAs, though of considerably lesser magnitude, was the additional provision of library support (i.e., books, literature) for candidates’ ongoing CPD generated by needs identified during the coaching. Because this was a parallel structure to that of the SQH, the normal provision of such resources through HEI libraries was unavailable. One other significant issue with wider implications was raised about the sustainability of the presentation panels, particularly if the number of participating LAs increased sharply. With 15 candidates completing their final assessments in May 2008, there were concerns raised about the sustainability of the assessment process. With candidates able to opt for one of three assessment points in the cycle, on the one hand, this gave them flexibility of choice, but on the other hand, this was likely to create difficulties when the numbers of candidates increased, as one interviewee indicated: I have got concerns, I have to say, about the future, because it’s how it’s managed: that’s going to be really important. How this actually happens and how it’s managed that’s going to be important because I think at the beginning we can support one another, but as the numbers get bigger and geographically the spread gets wider it’s not going to be so easy. A suggestion was made that the three assessment points should be reduced to two. Cutting the assessment to two times in a year, however, could cause undue pressure for the management of the presentation panels. These were managed through the GTC and unease was expressed about the pressures this process might create: We’ve got 30 candidates: we’re going to bring them forward in groups, and so on, they’re not all going to get to the end point at the same time. That’s okay, we will cope with that, but if you had 32 local authorities with lots and lots of candidates coming forward a different times of the year there is a huge implication [for the GTC]. 67 Quality Quality assurance in relation to aspects of the FRH has already been considered at a number of places in this report. A handful of additional quality assurance issues related to implementation of the programme, however, is worth mentioning. Two concerned the conduct of field assessments. In some instances, there may not have been sufficient guarantees of confidentiality and privacy: The discussions and interviews, if you want to call them that, with the peer deputes were undertaken in the school library which was a public space. That was an issue. I felt that because the peer depute and the principal teacher came in together there was no opportunity for the principal teacher to confirm or deny what the depute was saying, because it was in a public space and there were a couple of pupils present. Although they were quite some distance away I felt somewhat uncomfortable that there may have been a, should there be a difference of opinion between the PT and the depute, I wouldn’t have liked it to have been played out in the school library where pupils could hear that. There is a need, therefore, for choice of non-public venues to communicate feedback and consideration should also perhaps be given to the provision of staged feedback. A related issue for assessment, in the event of scaling up the FRH, concerned the consistency of evaluative judgements: My concern would be as with any process that, should this be rolled out, as the number of coaches, assessors, tutors increases, that we have to keep as much, we potentially have less consistency. I think it worked because the three coaches worked through the assessment criteria together: they developed them together, they knew exactly what they were looking for and they worked closely together cross-marking and supporting one another through the process. There were only 15 or so portfolios to consider so each one of these was looked at, at least twice, maybe three times, so it was tight. If this was rolled out to be looking at say 60 portfolios a year, I think there would have to be other checks and balances in the process somehow. This possibility might arise “because the danger is you could have 30 different versions of this [FRH] if it goes out to local authorities”. In these circumstances, with so many LAs likely to be involved and responsible for developing coaches, “how to we ascertain or support coaches in their own growth to being the best they can”? In short, one key challenge here is to ensure that “coaches are of the same standard” and a related one is “to make sure that the coaches’ bosses’ management structure is also aware just how important this is”. That, in turn, might mean that an agency such as the GTC is required to accredit provision within LAs in much the same way as it already accredits providers for the SQH and the Chartered Teacher. Recommendation 21 That, prior to future negotiations with potential participating LAs, the management group consult with the GTC with a view to resolving issues concerned with the accreditation of coaches, assessment protocols and the maintenance of comparable standards of assessments of candidates. 68 13. Conclusion Across the world, the development of school leaders is now a major national policy priority of governments. The member countries of the Organisation for Economic Co-operation and Development (OECD), for example, have recently participated in a comparative international project entitled Improving School Leadership. The 20 or so Country Background Reports, along with the series of follow-up conferences, where the implications of the findings have been analysed in detail, provide ample testimony to the significance attached to the activity of leadership and the challenges associated with replenishing school leaders. In an effort to maintain and improve the quality of their leadership, a number of these countries now see development programmes for school leaders, in particular head teachers and principals, as inextricably linked to sets of standards for professional performance. In that regard, Scotland is no exception. As we have indicated in this report, since 2005, prospective head teachers in Scotland have been required to satisfy the criteria set out in the SfH. Currently in 2008, the sole route for meeting the SfH is completion of the SQH. With 15 candidates having completed the FRH pilot at the time of writing, and another 14 expected to follow soon, the SQH’s monopoly on head teacher accreditation may be coming to an end. Whether or not it does, will depend in part on the adoption or modification of the recommendations in this evaluation report. If, indeed, these are accepted then future school leaders in Scotland will have a choice of development options available to them. As we have sought to make clear, our idea of a “mixed economy” model of provision is one of complementarity rather than competition between the existing SQH and the new FRH. That is, we are suggesting there should be a choice of options and that that choice means a difference in programme content and emphasis. For this reason, an alternative title for our report might have been “horses for courses”, because we recognise that there different means available for people with different sets of needs to achieve similar ends. The thrust of our findings in this evaluation is that the FRH has much to commend it. Clearly, the coaching that forms the centrepiece of the FRH is a significant mechanism for forming leaders and offers an equally significant set of learning contexts. Earlier, we cited Hattie’s & Timperley’s (2007) argument that feedback is a powerful instrument for maximizing learning. Its purpose, Hattie & Timperley (2007, p. 86) argue, is to reduce discrepancies between current understanding and a goal. Feedback will be effective, provided it answers three questions for the student: Where am I going? How am I going? Where to next? In short: feed up, feed back and feed forward. As evaluators, we can be fairly confident that, if these were not the precise questions the candidates were asking themselves during this pilot, then they were probably thinking along these lines when they met with their coaches and as they progressed through the programme. In relation to Hattie’s and Timperley’s claims, the adeptness of the coaches, as we hope we have shown, lay less in providing direct answers and more in assisting candidates to assemble and wrestle with feedback evidence of their performance so that they might be able to arrive at answers to their own self-generated questions, at their own pace and in a manner of their own choosing. If so, then this is effective learning. In summary, then, on an imagined scale of revision ranging from major surgery at one end 69 through to tweaking at the other, we are suggesting modifications to the FRH that while weightier than cosmetic by no means call for a major re-think. In short, the core idea of coaching that informed this pilot is an highly admirable one, and the progress and life cycle of the pilot, to the extent that we have been able to capture them adequately within these pages, indicates that a huge amount has been achieved and knowledge acquired in a remarkably short time. We believe this is a programme that deserves a place in the landscape of Scottish school leader development and our hope is that our recommendations, which are intended to strengthen the FRH, will help secure it that place. 14. References BERGLAS, S. (2002) Dangers of executive coaching, Harvard Business Review, 80(6): 8792. CONGER, J. (2004) Developing leadership capability: What’s inside the black box, Academy of Management Executive, 18(3): 136-139. CZARNIAWSKA-JOERGES, B. (1997) Narrating the Organization: Dramas of Institutional Identity (Chicago: University of Chicago Press). ELMORE, R.F. (1996) Getting to scale with good educational practice, Harvard Educational Review, 66(1): 1-26. FORDE, C. (2006) Alternative routes: Summary. Agenda paper for NSQH Consortia meeting, 22 August. (unnumbered) GRAY, D.E. (2006) Executive coaching: Towards a dynamic alliance of psychotherapy and transformative learning processes, Managerial Learning, 37(4): 475-497. HALL, D.T., OTAZO, K.L. & HOLLENBECK, G.P. (1999) Behind closed doors: What really happens in executive coaching, Organizational Dynamics, 27(3): 39-53. HATTIE, J. & TIMPERLEY, H. (2007) The power of feedback, Review of Educational Research, 77(1): 81-112. HIGGINS, M.C. & KRAM, K.E. (2001) Reconceptualizing mentoring at work: A developmental network perspective, Academy of Management Review, 26(2): 264288. INGVARSON, L. & HATTIE, J. (Eds) (2008) Assessing Teachers for Professional Certification: The First Decade of the National Board for Professional Teaching Standards (Oxford: Elsevier/JAI), pp. 1-21. KEEGANS, J. (2007) Flexible routes to the Standard for Headship: Update January 2007” (unnumbered). KRAM, K. (1983) Phases of the mentor relationship, Academy of Management Journal, 26(4): 608-625. POGLINCO, S.M. & BACH, A.J. (2004) The heart of the matter: Coaching as a vehicle for professional development, Phi Delta Kappan, 85(5): 398-400. POPPER, M. & LIPSHITZ, R. (1992) Coaching on leadership, Leadership &Organization Development Journal, 13(7): 15-18. REEVES, J., TURNER, E., MORRIS, B. & FORDE, C. (2003) Culture and concepts of school leadership and management: Exploring the impact of CPD on aspiring headteachers, School Leadership & Management, 23(1): 5-24. SEED (2005a) Ambitious, Excellent Schools: Leadership―A Discussion Paper (Edinburgh: Scottish Executive). SEED (2005b) Ambitious, Excellent Schools: Standard for Headship―November 2005 (Edinburgh: Scottish Executive). 70 SEED (2006a) Achieving the Standard for Headship―Providing Choice and Alternatives: A Consultation Document (Edinburgh: Scottish Executive). SEED (2006b) Achieving the Standard for Headship―Providing Choice and Opportunity (Edinburgh: Scottish Executive). (unnumbered) SWAFFIELD, S. (2007) Light touch critical friendship, Improving Schools 10(3): 205-219. WITHERSPOON, R. & WHITE, R.P. (1996) Executive coaching: A continuum of roles, Consulting Psychology Journal, 48(2): 124-133. 71 Appendix i: Terms of Reference 5. Aims and Objectives of the Evaluation* 5.1 The principle aims of the evaluation will be: 5.1.1 to examine, from the perspective of all relevant stakeholders, how participants have:- • • • • • • • • developed towards the SfH; developed their attitude and behaviour towards leadership and management practice; been supported in their leadership development and in constructing their own professional learning plans; viewed the support of the coach in development of their professional practice and personal development; accessed quality leadership development opportunities and what these were; impacted on their school community (including pupils and colleagues) in carrying out their development; used the pilot to work flexibly, taking account of life styles and professional commitments, towards the SfH; and viewed and handled the assessment process (including the impact of formative assessment procedures). 5.1.2 • • • developed during the pilot; been supported in their duties; and viewed the support of the coach trainer 5.1.3 • • to examine, from the perspective of all relevant stakeholders:- the sustainability of formal assessment procedures; and the contribution of field assessors to the assessment process. 5.2 • • • • • • to examine, from the perspective of all relevant stakeholders, how coaches have:- The evaluation will need to collate information from various sources including:all 30 candidates; all 3 coaches; all 30 field assessors; principal assessor; the project manager; and coaches’ coach 5.3 The project manager has evaluated parts of the programme to date and this information will be made available. 5.4 To consider the views of the following stakeholders not only on the aims above but the capacity issues around rolling out this pilot: • • local authority pilot co-ordinator; local authority CPD co-ordinator; 72 • • • • candidates’ headteacher; candidates’ colleagues (including the senior management team); National CPD Co-ordinator; Other interested individuals and groups. * Extracted from “Research Specification: Evaluation of Flexible Routes to Headship Pilot” (Teachers’ Division, Scottish Executive Schools’ Directorate), 2007. 73 Appendix ii: Methodology The pilot evaluation comprised a series of broadly sequential phases of qualitative research Data collection commenced November 2007 and concluded in June 2008. Among them, the evaluation team members shared a set of generic research experiences and specialist competences. To capitalize on the team’s collective skills base, every attempt was made to align specialisms with data collection on corresponding components of the pilot. While the joint principal investigators exercised overall responsibility for the evaluation, day-to-day management of the project fell within the remit of the Research Fellow. The team conferred regularly to review progress, analyse the data and to discuss emerging themes. The wealth of data generated by the range of methods permitted cross-checking, consistency, correction and validation of information. Ethical clearance for the evaluation was obtained from the Faculty of Education, University of Glasgow (see p. xx). Prior to the commencement of data collection, the Research Fellow and one of the principal investigators conducted briefing sessions with the coaches and the three cohorts of candidates. The methods were: • Content Analysis: The evaluation team conducted a review of the background documentation pertinent to the FRH provided by the project manager (FROG). 17 candidates provided individual PLPs which were analysed systematically. In addition, the candidates and coaches were invited to complete electronic journals. Except for 2 coaches and 3 candidates the majority did not take up this option. 2 other candidates provided hard copies of journal entries. • Interviews: There were two rounds of 1:1 interviews with the candidates and coaches. In the first round, the candidates were interviewed face-to-face and in the second round over the telephone. Interviews were also conducted with 3 coaches and the coaches’ coach, staff of the GTC, LAs, FROG members and others. The breakdown of categories and numbers of interviewees was: • • Candidates: Rd 1: 29; Rd 2: 28 Coaches: Rd 1: 4; Rd 2: 4 FROG: Local Authorities: GTC: Rd 1:2; Rd 2: 1 External Moderator: Other: = 57 =8 =2 =4 =3 =1 =1 TOTAL: = 76 Focus Groups: Four focus groups were conducted. 19 candidates, 3 coaches and the coaches’ coach participated. The breakdown of categories and numbers of focus groups was: Candidates: Coaches: 3 1 TOTAL: 4 Field Observations: The evaluation team observed the final panel interviews at the GTC for the 14 candidates who submitted in May 2008. In addition, at least 1 member of the 74 research team was also present at most FRH management meetings and FROG meetings held during the evaluation. Unfortunately, the timing of the evaluation period precluded team members’ participation in FRH residentials and due to other commitments the team was unable to be present during assessors’ field visits. • Analysis of Candidates’ Written Submissions: The portfolios of evidence and reflective commentaries submitted by the first 14 candidates for the final assessment were analysed by the research team in May 2008. 75 UNIVERSITY of GLASGOW Faculty of Education Ethics Committee For Non Clinical Research Involving Human Subjects EAP2 NOTIFICATION OF ETHICS APPLICATION FORM APPROVAL Application No. (Research Office use only) E849 - 3 Period of Approval (Research Office use only) 19/09/2007 to 01/07/2008 Date: 19 September 2007 Dear Peter I am writing to advise you that your application for ethical approval, reference E849 for 'Evaluation of Flexible Routes to Headship Pilot’ has been fully approved following your fulfilment of the required amendments. You should retain this approval notification for future reference. If you have any queries please do not hesitate to contact me and I will refer them to the Faculty’s Ethics Committee. Regards, Terri Hume Ethics and Research Secretary 76 Appendix iii: Selected Research Instruments As the coaching experience played such a significant role in the candidates’ experience of the FRH, Round 1 interview schedules (G and I) for both candidates and coaches have been included for illustrative purposes. The schedule of interview questions for local authority representatives (K) has also been included. Attachment G: Interview schedule for candidates (Time 1) Evaluation of Flexible Routes to Headship Pilot Motivation for participation in the programme 1. Originally, before being accepted into the FRH pilot, you were required to submit an application. What influenced your decision to do so—were there key individuals, experiences or events? 2. Was the SQH an option for you at the time? If it was, why did you apply for the pilot rather than the SQH? Expectations of the programme 3. Thinking back to the circumstances when you applied, were there particular outcomes that you hoped for by being accepted into the pilot? What were these? 4. At the time, did you have any leadership aspirations? What were those aspirations? Was there any connection between these and your wish to participate in the programme? 5. Were you conscious, for example, of any particular strengths and weaknesses you may have had as an aspiring school leader? Experiences so far 6. Looking back over the period since your acceptance into the programme, describe for me in general terms what your overall experience of the pilot has been like. What, for example, has been the most beneficial aspect for you? And the least beneficial? 7. Leaving aside the coaching which we’ll deal with in a moment, let’s turn now to the specific components of the programme… What can you tell me about the residentials: How did these work and were they useful? Why? Why not? 8. What about the Professional Learning Plan? Have you tried to link this to your original aspirations for yourself as a leader? How did you do this? Give me some idea of the goals you have set yourself. 9. What about the Emotional Competences Inventory (ECI): did you learn anything new about yourself? 10. How about the Learning Log that FRH pilot participants have to maintain: Is this helpful? In what way? Coaching 77 11. How often do you and the coach meet, and for how long each time? Describe a typical coaching session for me.12. Think back to when you were initially allocated a coach… did you hit it off straight away or were there teething problems? How did you get to know one another? 12. To what extent has your relationship with the coach gone through phases? Why might this be? Describe these for me. 13. Tell me what has been the most beneficial outcome for you of the coaching. Why is this? 14. Do you have a mentor or mentors? To what extent do you see coaching as similar to, or different from, mentoring? 15. Now tell me what the least beneficial experience has been. Why? 16. Looking back, if coaching for leadership development is going to be effective, what are the “bottom-line” conditions that have to be met? 17. Summing it up, then, has the coaching made a difference to the development of your leadership and management skills? To your attitudes and behaviour? Why? Why not? Where to from here 18. At this stage of the pilot, has your experience made any difference to your leadership aspirations? Describe its impact on those aspirations for me now. 19. Do you talk about your aspirations with other people? Who? 20. To what extent can you visualise yourself in a headship role? Is this something that has only occurred as a result of the experience of the programme? 21. At this stage, is it likely when you have completed the programme that you will apply for a headship? Why? Why not? 22. Is there anything you want to add? 78 Attachment I: Interview schedule for coaches (Time 1) Evaluation of Flexible Routes to Headship Pilot Growth as a coach 1. I will explore your work with the candidates in a moment… Can we begin by discussing your own development as a coach…? Is this the first time you’ve taken on the role of coach? 2. So far, are you enjoying being a coach? What is so rewarding about it? 3. Is there a downside to coaching? 4. At this stage, do you think this is something you would do again? Support for coaches 5. As a new coach, what support have you most needed from those responsible for this project? 6. Do you feel you have had their support so far? 7. If you were to be asked to undertake this role again, are there any pre-conditions you would insist on by way of support for coaches? Coach of coaches 8. Explain to me how you have been prepared or trained for the role of coach? 9. Has this relationship between the coaches and their trainer been a profitable one? Why? Why not? Coaching of candidates 10. Turning now to the coaching itself ..., what does coaching mean to you? Now summarise a typical coaching session for me. 10. There is an expectation that coaches will provide candidates with formative assessment in the pilot. Can you explain for me your understanding of “formative assessment” and how this works in the programme? 11. Outline for me the role of a coach in relation to the development of these aspiring school leaders? 12. Is there any difference between coaching and mentoring or are they one and same? 13. How would you describe the quality of your relationship with your candidates? 14. Do you think coaching relationships go through stages of development? If yes, what are those stages? Broadly speaking, at which stage are your relations with the candidates at the moment? 15. Do you think your candidates are making good progress in their development towards the SfH? What would you point to as evidence of that development? 16. Have you visited them in their schools yet? What happens during a school visit? 17. Summing it up… At this stage, what is your bottom line if coaching is going to be a success? 18. Is there anything you want to add? 79 Attachment K: Interview schedule for local authority personnel Evaluation of Flexible Routes to Headship Pilot Local authority leadership development 1. Let’s begin by you summarizing for me the council’s policy on school leadership development and giving me an indication of the range of programmes on offer. 2. Given what you just said, why did the council agree to take part in this pilot? Attractions of the FRH pilot 3. Turning specifically to the FRH pilot…, from the council’s point of view, what is it hoping will be achieved with this FRH scheme? 4. What is it that makes this pilot programme different and distinctive? What does it do that other alternative council programmes don’t or can’t do? 5. How are these other council programmes that you mentioned faring at the moment? 6. Do you any sense of why people have put themselves forward for the pilot? Progress so far 7. At this stage, do you have any evidence about how well the pilot is travelling? 8. What is the explanation for the modifications that have been made, such as the use of residentials, the change in coaching personnel and the adoption of “medical rounds”? Components of the Programme 9. As you are aware, the programme has a number of components to it, of which the main one is coaching… In your view, what might the coaching of leader aspirants achieve that can’t be accomplished by other means? 10. Do you have any feedback or intelligence about whether the coaching is being well received by coaches and candidates? Assessment of the coaching model 11. Given what we know from the research evidence about how complex and stressful headship roles can be… how confident are you and the council about the robustness of coaching as a basis for headship preparation in this local authority? 12. Playing devil’s advocate for a moment… what about the argument that the most effective form of learning or preparation for a headship might be workplace-based? You don’t think these candidates would be better off with responsibility for, say, a major school-based development project? 13. Do these candidates lose any sense of being part of a learning community, given that they are not working as part of a group? 80 14. Does it concern you that, unlike chartered teachers and graduates of the SQH, the candidates who complete this programme will not come away with a Masters degree or a graduate diploma? The future 15. Do you have any confidence that, having taken part in this pilot programme, the candidates will apply for headship vacancies? Is there an expectation that they will and are they under any pressure to do so? 16. Suppose the evaluation of this pilot judges it to be a success and it is “rolled out” nationally… Can you identify for me what the implications would be for this local authority? Does the authority, for instance, have the necessary infrastructure (e.g., training provision, sufficient coaches) for such a roll-out? 17. What about the provision of support for the range of activities candidates might undertake for their Professional Learning Plans? How would these be funded? 18. Would there be any other resource implications of such a roll-out for this local authority? 19. Do you think that this FRH programme is better suited the needs and capacities of some local authorities rather than others? 20. What might be the impact of a national roll-out of the FRH on existing council programmes (e.g., the SQH)? Do you have any reason for thinking that the two programmes (i.e., the SQH and the FRH) would widen the pool of prospective heads and produce a surge of applications OR would the two be in competition with one another by drawing from the same size aspirant pool and the same source of funding? 21. Is there anything you would like to add? ISBN 978 0 7559 1825 6 (web only publication)
© Copyright 2026 Paperzz