Striving for Success: Insights from Succeed 2020 Partners Third Report of the Succeed 2020 Documentation Project Caitlin Rose Dailey and Monica Mean FHI 360 July, 2016 Table of Contents Executive Summary .................................................................................................................................................. 1 Introduction and Background .............................................................................................................................. 3 Methodology ...................................................................................................................................................... 4 Findings ......................................................................................................................................................................... 6 Organizational Capacity ................................................................................................................................ 6 Implementing Succeed 2020 ...................................................................................................................... 8 Relying on Data...............................................................................................................................................11 Impact of Succeed 2020 ..............................................................................................................................13 Planning for Sustainability.........................................................................................................................15 Shifting toward Sustainability ............................................................................................................................19 D Disclaimer: The Succeed 2020 documentation team is separate and independent from the FHI 360 technical assistance team that works directly with REAs and other Succeed 2020 partners. The findings and opinions contained in this report are solely those of the documentation team and are not meant to represent the FHI 360 technical assistance team. Executive Summary Since 2011, Succeed 2020 has helped North Dakota students transition from school into college and careers. Eight Regional Education Associations (REAs) have led this effort, working with each other and with their member schools and districts, local businesses and higher education institutions, a statewide steering committee, and FHI 360. The 2014–2015 school year was the first in which all eight REAs had entered the full implementation phase of the initiative. This report is the third in a series of annual documentation reports and features stakeholders’ perspectives on continuing progress and hurdles in Succeed 2020 during the 2014–2015 academic year. REAs have been able to offer more and, they believe, better professional development services to their member districts because of Succeed 2020 resources. They have recognized the need for collecting and analyzing data on these services to make decisions about their priorities. While they are negotiating these programmatic decisions, they are also working to strengthen their management acumen and become stronger organizations. The information presented in this report derives from interviews with REA staff members, FHI 360 technical assistance providers, and steering committee members as well as surveys of school and district leaders from North Dakota’s public schools. Data was collected from December, 2015 to March, 2016, meaning that respondents were reflecting on the previous school year. Key findings from this report follow. Organizational Capacity Increased staffing levels have enabled REAs to develop their capacities to deliver services to member districts. REAs were working to become more strategic and develop stronger planning skills to conceptualize their services as a cohesive body of work. Succeed 2020 has prompted many REAs to develop greater organizational complexity to meet the demands of the initiative. Succeed 2020 has invested significantly in helping REAs develop their capacities for human resources management. REAs were developing a stronger collective identity and ability to advocate for themselves. Implementing Succeed 2020 REAs have developed a strong grasp of the Succeed 2020 strategies. Their progress was evident in stronger professional development services for their members and more sophisticated requests for technical support. REAs continued to struggle with staffing issues. With experience, REAs were refining their understanding of the Succeed 2020 strategies and the activities in each. REAs also abandoned some activities or priorities they deemed unsuccessful. The specific types of technical assistance from FHI 360 that REAs requested and received varied. REAs addressed a variety of topics in their professional development. School and district staff members reported that professional development that focused on using data was both the most available and the most useful to them. 1 Relying on Data The collection and use of data has simultaneously been one of the greatest areas of progress for the REAs and the greatest area of need. FHI 360 has increasingly provided targeted support for developing REAs’ data capabilities. REAs struggled to link their activities directly to student outcomes. They relied on several alternative sources of information to understand the effectiveness of their professional development. Succeed 2020 stakeholders recognized REAs’ progress in relying on data but acknowledged that they have a long way to go. Impact of Succeed 2020 Succeed 2020 helped some REAs provide services to new audiences. School and district leaders reported that REAs effectively addressed their needs. Their satisfaction with REA services increased during the grant period. Planning for Sustainability REAs were still in the beginning stages of planning for sustainability, but many were beginning to think about potential future funding sources. Almost all the REAs would like to receive technical assistance around grant writing and identifying additional funding sources in the final years of Succeed 2020. REA staff members hoped to maintain their growth in using data. REAs needed support in using data to help them determine which services to sustain. Professional development, events for students, and project staff emerged as the features of Succeed 2020 REAs most hoped to sustain. Collaboration among the REAs has increased dramatically because of Succeed 2020 and most REAs would like to sustain this stronger collaborative spirit. There was consensus among Succeed 2020 stakeholders on the different roles each should play in planning for sustainability. Over the course of Succeed 2020, REAs have strengthened their own organizations and the services they offer member districts. They are responding to calls to enhance their data capabilities, both to plan their ongoing work and to prove its worth. School and district leaders have recognized the growing strength of their REAs. This report concludes with continuing challenges and questions, many centered on data and REAs’ roles, that will need to be addressed as the initiative pivots to focusing on sustainability. 2 Introduction and Background Since 2011, Succeed 2020 has helped North Dakota students transition from school into college and careers. Eight Regional Education Associations (REAs) have led this effort, working with each other and with their member schools and districts, local businesses and higher education institutions, a statewide steering committee, and FHI 360. The 2014–2015 school year was the first in which all eight REAs had entered the full implementation phase of the initiative. REAs have been developing improved services for educators in their regions in each of Succeed 2020’s strategies: Ongoing college and career counseling and planning Access to and success in rigorous academic and CTE programs Targeted and coordinated supports In tandem with efforts to improve their professional development services, REAs have been building their capacities to collect and analyze data that will help them understand the efficacy of these services. With considerable prompting from the Succeed 2020 steering committee and FHI 360 and significant technical support they have recognized the importance of relying on data for their programs and developed their skills in doing so. Likewise, many REAs have been pushed to quickly strengthen their organizational capacities to manage and support complex programs like Succeed 2020. REAs received Succeed 2020 planning grants in the fall of 2011. Implementation of Succeed 2020 began in the 2012–2013 school year with four REAs: Missouri River Education Cooperative (MREC – Mandan); Northeast Education Services Cooperative (NESC – Devils Lake); Roughrider Education Services Program (RESP – Dickinson); and South East Education Cooperative (SEEC – Fargo). After an intensive planning phase, the remaining REAs moved into implementing Succeed 2020 programming. Three REAs entered the full implementation phase in the 2013–2014 school year: Great Northwest Education Cooperative (GNWEC – Williston); Mid Dakota Education Cooperative (MDEC – Minot); and North Central Education Cooperative (NCEC – Bottineau). Red River Valley Education Cooperative (RRVEC – Grand Forks) began full implementation in the 2014–2015 school year. The Succeed 2020 steering committee comprises representatives of state agencies, the gubernatorial administration, professional organizations, and the legislature and advises on the initiative. FHI 360, a nonprofit organization, provides technical support to REAs and manages the Succeed 2020 grant from the Hess Corporation. The Succeed 2020 documentation project collects and records information on the progress of Succeed 2020 as it unfolds. To document progress toward Succeed 2020 goals, the documentation team collects information from REA staff members, FHI 360 staff members, steering committee members, and school and district leaders. The documentation project seeks to: Capture perspectives on progress toward the goals of Succeed 2020 from multiple stakeholders Communicate this progress to the Hess Corporation, stakeholders involved in Succeed 2020, and the North Dakota public Inform and improve program implementation by REAs and the technical assistance provided to REAs This report is the third in a series of annual documentation reports. The first report, Succeed 2020: Perspectives on Progress, featured findings from the Succeed 2020 planning phase, begun in 2010, through the first full year of implementation in the 2012–2013 school year. The second report, Succeed 2020: 3 Perspectives on Continued Progress, highlighted continuing successes, challenges, and questions that emerged in the 2013–2014 school year as most REAs moved into Succeed 2020 implementation. That report detailed a set of ongoing challenges facing the REAs and their partners, including: Staffing shortages in REAs and their member districts The need to develop stronger organizational policies and procedures in many REAs The need to develop stronger data collection and analysis skills among REA staff members Concern about maintaining REAs’ current level of services for member districts after Succeed 2020 grant funding ends Encouraging more substantive engagement from businesses and higher education Ensuring REAs’ have access to technical assistance from the entire FHI 360 team Clarifying the role of the steering committee Assessing the impact of Succeed 2020 within the constraints of the grant period This report features stakeholders’ perspectives on continuing progress and hurdles in the 2014–2015 academic year. REA staff members’ understanding of Succeed 2020 and its strategies has become deeper and more substantive as they move further into implementation. Although they continue to grapple with ongoing challenges, with continued support REAs have made significant progress. They and their members are recognizing positive changes in the level and efficacy of services they are able to offer North Dakota educators. Methodology The information presented in this report derives from interviews with REA staff members, FHI 360 technical assistance providers, and steering committee members as well as surveys of school and district leaders from North Dakota’s public schools. Data was collected from December, 2015 to March, 2016, meaning that respondents were reflecting on the previous school year. Now that all eight REAs have progressed in implementing Succeed 2020, this report explores five interim research questions: Where have REAs made the greatest progress in implementation? What progress have REAs made in addressing implementation challenges? What impact has Succeed 2020 had on REAs’ organizational capacities, individually and collectively? What have they been able to do and provide because of Succeed 2020? Who do REAs’ regard as the primary target/audience for their services? How has Succeed 2020 affected school and district administrators? Counselors? Teachers? Students? What Succeed 2020 services/activities do REAs plan to sustain after the initiative, individually and collectively? Why these services/activities? What plans are in place to fund these continuing services/activities and what role should Succeed 2020 partners play in planning for sustainability? Data sources for this report include: Interviews with REA staff members: REA or Succeed 2020 directors from each of the eight REAs participated in interviews. They shared their perspectives on implementation progress and challenges, work with their FHI 360 liaisons, developments in their organizational capacities, the impact thus far of their Succeed 2020 activities, and their plans for sustainability. Interviews with FHI 360 technical assistance providers: Ten FHI 360 staff members, including REA liaisons and members of the initiative’s leadership team, participated in interviews with the research team. They were asked to share their perceptions of REAs’ progress, the overall impact of Succeed 2020, and the effectiveness of FHI 360’s technical assistance strategies. 4 Interviews with steering committee members: Ten members of the steering committee gave interviews; seven were randomly selected from the committee and an additional three volunteered to participate. They were asked about Succeed 2020’s impact across the state and how or whether Succeed 2020 services and activities could be sustained after grant funding. Surveys of school and district administrators: Sixty-eight district and school administrators, nearly all principals or superintendents, from the state’s 179 school districts responded to surveys. They answered questions on their relationship with their REA and the effectiveness of REA services. Respondents came from 54 different school districts and included members of all eight REAs.1 This is the second year school and district leaders have been surveyed, meaning that results from the 2013–2014 and 2014–2015 school years can be compared. Document review: The documentation team reviewed documents including REA progress reports, work plans, and meeting agendas and presentations. Qualitative and quantitative data were analyzed for key themes related to the research questions above. The report presents these themes from a cross-REA perspective because study participants were promised anonymity. 1 Survey respondents may not be a representative sample of all North Dakota school and district leaders. 5 Findings REAs have been able to offer more and, they believe, better professional development services to their member districts because of Succeed 2020 resources. With considerable prompting and support from the steering committee and FHI 360, they have also recognized the need for collecting and analyzing data on these services to make decisions about their priorities. While they are negotiating these programmatic decisions, they are also working to strengthen their management acumen and become stronger organizations. Although these efforts are accompanied by ongoing challenges, Succeed 2020 stakeholders reported that REAs have made, and continue to make, significant progress toward Succeed 2020 priorities. This section highlights findings on key topics including: Efforts to strengthen REAs’ organizational capacities REAs’ activities in Succeed 2020’s strategies challenges they have encountered, and strategies they have used to address those challenges REAs’ developing abilities in collecting and using data Succeed 2020’s impact to date Nascent plans for sustaining some of the REA services developed or expanded under Succeed 2020 after grant funding ends Organizational Capacity The programmatic and reporting requirements of Succeed 2020 have helped many REAs identify structural challenges within their organizations. FHI 360 has increasingly provided specific support for developing each REA’s organizational capacities to manage larger staffs and more complex programs. Increased staffing levels have enabled REAs to develop their capacities to deliver services to member districts. Professional development services for member districts have historically been REAs’ “bread and butter”; Succeed 2020 has enhanced their capacities to deliver those services. Succeed 2020 has allowed most REAs to hire additional staff members to deliver services. Put simply, they have been able to “hire more staff who can do more things.” Grant resources have also helped REA staff members deepen their individual expertise. The grant has increased REAs’ capacities both in sheer numbers of staff and in the knowledge and experience of those staff members. Consequently, REAs are now able to offer more, and more robust, services to their member districts. REA staff members take pride in the availability and utility of their professional development offerings under Succeed 2020. Succeed 2020 directors reported that their member districts were increasingly recognizing the value of their services and turning to the REAs as important regional resources. According to one REA Succeed 2020 director, “Our members are used to an extremely high level of service right now, as far as we have kind of become the one- “We've been able to spread our wings and find a niche of where we can really help schools really well [by] getting our people trained.” -REA staff member 6 stop shop for member schools. We're the first entity they call for support, for answers to their questions, for guidance, for professional learning services, for resources, you name it.” REAs were working to become more strategic and develop stronger planning skills to conceptualize their services as a cohesive body of work. Many REA and FHI 360 staff members described how they were moving away from providing “one and done” professional development offerings. Instead they were planning and offering coherent series of professional development sessions that followed on one another, collecting information on the effectiveness of their services, conducting on-the-ground follow-up in schools, and seeking to tailor subsequent offerings in response to the information they collected on professional development sessions. As one FHI 360 staff member portrayed it, REAs were becoming less “knee jerk” in planning their professional development and making strides toward incorporating individual projects into a larger vision and overall goals for their organizations. Some FHI 360 staff members speculated that this development was at least partly a result of the comprehensive Succeed 2020 work plans required by the grants and the intensive technical assistance FHI 360 has offered over the years in strengthening these plans. Where in prior years some REA staff members appeared to view these work plans as merely a compliance exercise or a paperwork burden, now many have come to see them as useful living documents helping to guide their work. Some REAs also requested and received technical assistance from FHI 360 on developing strategic plans for their organizations as a whole. One FHI 360 staff member with significant experience in strategic planning has provided this assistance for several individual REAs as well as for North Dakota Regional Education Associations (NDREA) as a collective entity. Succeed 2020 has prompted many REAs to develop greater organizational complexity to meet the demands of the initiative. Although REA staff members highlighted increased staffing as perhaps the most positive feature of Succeed 2020 funding, added people and added work had also made managing their organizations more complicated. Some were grappling with balancing their programmatic work—delivering services to their members—with the administrative and managerial tasks of running the REA. Increased organizational complexity under Succeed 2020 drew attention to missing or inadequate policies and procedures for financial systems, managing employees, and working effectively with their governing boards. Succeed 2020 has invested significantly in helping REAs develop their capacities for human resources management. Based on their developing knowledge of REA needs and specific requests from some REA staff members, FHI 360 has provided ongoing and intensifying support for REAs to develop their capacities for human resources (HR) management. In earlier years, FHI 360 offered workshops during regular cross-REA meetings; this year the Succeed 2020 team invested more heavily by hiring two consultants to work with REAs to assess their HR needs and develop individual plans for meeting those needs. “The capacity has definitely been increasing in what we’ve been able to provide, but the way that we operate has not grown as quickly.” -REA staff member 7 The consultants conducted thorough assessments, aligned with AdvancED standards, of needs related to HR and organizational systems for five REAs. REAs’ needs assessments highlighted the necessity of focusing on performance management, including establishing clear job descriptions, clarifying lines of reporting and authority, and instituting systems for performance assessment and accountability. The assessments also underscored the need for leadership development within REAs. Executive coaching was offered to REA directors who agreed to implement specific recommendations from the assessment and pay a portion of the cost as one avenue for growth following their needs assessment. Based on finding from these needs assessments, some REAs will also need help on effective communication and collaboration with their governing boards. This support was helping REAs develop policies and procedures that account for their increased organizational complexities. The push to develop their organizational capacities and strategic acumen has also helped the REAs begin to plan for the optional AdvancED accreditation process, which will require them to meet standards for organizational systems. REAs were developing a stronger collective identity and ability to advocate for themselves. Nearly all REA staff members reported that building stronger individual capacities has helped make the REAs stronger partners to one another. They were becoming less isolated and more collaborative even as they focused on meeting the particular needs of their respective regions. According to one REA director, “It’s definitely more of a collaborative atmosphere. Everybody seems more willing to work together and actually wants to work together to improve not only their own REA but other REAs and education in general. That has definitely changed for the better.” This commitment to stronger collaboration is evidenced by the development of a new strategic plan for NDREA, the REAs’ collective organization. “As we are able to provide opportunities for REAs to really showcase and demonstrate what value they play, their role will be valued even more.” -Steering committee member Many REA staff members, and some steering committee members, reported that these changes mean that REAs are potentially positioned to be more prominent players in the statewide education infrastructure. One steering committee member noted that REAs will need “to continue to focus on the long range plans of a systematic NDREA system and how that NDREA system fits within all of the other systems…in the state that contribute to K-12 education.” Most of the REAs were making efforts to align their activities with broader statewide priorities and make a case for how they are suited to advancing statewide priorities. Implementing Succeed 2020 In the 2014–2015 school year, all eight of North Dakota’s REAs had received full grant funding and moved into implementation of Succeed 2020. After intensive collective efforts toward planning and developing shared resources (for example, the NDSS Implementation Guide and the Work Based Learning Manual) in the early years of Succeed 2020, the REAs are now comfortably established in running Succeed 2020 programming in their own regions. They have turned to a focus on fine-tuning and improving the programming delivered to their members and are beginning to rely more heavily on using data in these efforts. REAs have developed a strong grasp of the Succeed 2020 strategies. Their progress was evident in stronger professional development 8 services for their members and more sophisticated requests for technical support. A major theme across interviews was the positive progression in the quality of REAs’ services afforded by Succeed 2020. First, the REAs have been able to provide higher quality professional development to their member districts than before Succeed 2020. Additional staff and better communication with their member districts have allowed REAs to align their professional development offerings with districts’ needs. The result is that the member districts now see the REAs as very valuable sources of teacher and administrator improvement. Second, REAs have improved their grasp of reporting requirements for Succeed 2020 and have begun to focus their requests for technical support more on the content of their Succeed 2020 work. Many respondents shared their perception that the level of depth in REAs’ technical assistance requests, either in formal written reports or more informally in conversation with their liaisons, has increased over the years. Many FHI 360 liaisons noted that earlier technical assistance requests from REAs were often focused on administration of the grant, compliance with reporting requirements, and understanding the strategies of Succeed 2020. There were more requests in 2014–2015 for content-based training, strategic planning, and collecting better data for decision making. These perceptions are illustrated in Figure 1. “[REAs are] getting much more sophisticated with their questions. They’re becoming good questioners to make sure they are doing what they want to be doing.” -FHI 360 staff member Figure 1. Perception of Succeed 2020 Technical Assistance Requests by Year Year 1 Reporting and Adminstration Year 2 Understanding Succeed 2020 Strategies Year 3 Collecting Data and Strategic Planning In addition, the quality of the quarterly reports and work plans that REAs submitted to FHI 360 has increased substantially. Succeed 2020 leaders were pleased with the more robust information and data REAs were including in these reports. Although the quality of data still needs improvement, FHI 360 and the REAs have made sure that including data on the progress of Succeed 2020 activities is part of the normal process of completing these reports. REAs continued to struggle with staffing issues. Some REAs struggled with staff turnover and finding a robust candidate pool to fill open positions. This was particularly problematic with leadership positions, as the turnover of leadership had a significant impact on the progress of Succeed 2020 implementation in some REAs. As a result, there has been uneven implementation of Succeed 2020 programming across the REAs; staff turnover was associated with slower progress. With experience, REAs were refining their understanding of the Succeed 2020 strategies and the activities in each. REAs also abandoned some activities or priorities they deemed unsuccessful. All REAs had challenges with at least one of the strategies of the Succeed 2020 initiative. Implementation of the strategies was also inconsistent across the REAs. In college and career planning, despite their best intentions and efforts, some REAs were stymied in institutionalizing college and career planning in their 9 member schools. They have found that counselors in schools are already preoccupied with social and emotional counseling and lack the necessary time and training to do college and career counseling. As one REA staff member explained, “In none of our schools do we have an identified counselor as just a college and career counselor. They are…counselors of emotional-behavioral issues, emergency situations, and then academic counseling as well.” There appears to be limited buy-in from some school and district leaders to focus on college and career counseling. REA staff members were developing and delivering some direct student services—for example, college or career fairs—to fill the gaps. In contrast with previous years, REA respondents made little mention of the Roads to Success college and career readiness curriculum, perhaps reflecting the difficulty in establishing it in schools. In academic and CTE preparation, respondents across REAs reported that REA services focused on academic preparation were robust and shared a focus on the North Dakota State Standards. CTE, however, had not received the same level of focus especially in REAs where it had not historically been a priority. An FHI 360 team member described one REA’s challenge as, “They just don’t have a handle across the REA of what their CTE offerings are and how people are using them. They don’t even have the gaps identified.” Some REAs were doing little or no work around CTE and staff members noted that they did not have the capacity to support CTE. Targeted and coordinated supports appropriately varied widely according to the needs of students in each REA. For example, some of the REAs have large Native American populations and have provided targeted supports to that subgroup while others have concentrated on other areas. Many reported struggling with this strategy, in part because there was no consistent understanding of what targeted and coordinated support looks like. As a result, there was little consistency across REAs. The specific types of technical assistance from FHI 360 that REAs requested and received varied. Each REA is assigned a liaison from the FHI 360 technical assistance team who fields their questions and requests and offers the first line of technical support. The liaisons’ level of involvement and the extent to which REA staff members considered their liaisons a crucial part of Succeed 2020 decision making varied quite a bit. Some REAs had strong relationships with their liaisons and communicated with them often while others communicated more minimally or were less likely to request technical assistance and involve their liaison in making decisions. As one REA staff member explained, “I don’t really think that the challenges that we’ve had are anything that we could have taken advantage of FHI [360]’s resources in order to [address].” In some instances, REAs saw growth in precisely the area that was their liaison’s greatest strength. Because of different needs and requests by individual REAs and different strengths of individual liaisons, the support provided across REAs has not been entirely consistent. 10 REAs addressed a variety of topics in their professional development. School and district staff members reported that professional development that focused on using data was both the most available and the most useful to them. Results from the survey administered to school and district leaders indicated that 82 percent of those surveyed received professional development from their REA around the use of data, as indicated in Table 1. Other common topics for professional development included reading and writing in content areas (81 percent), North Dakota State Standards (78 percent), and teacher evaluations (78 percent). In addition, use of data (50 percent) and reading and writing in content areas (50 percent) ranked as the most useful professional development topics covered by REAs, as indicated in Table 2. The North Dakota State Standards (41 percent) and teacher evaluations (37 percent) were also reported as useful professional development topics. “This year we have seen a big increase of courses that are offered for our teachers...we are very appreciative for this.” -School/district leader Table 1. Results from the School and District Leadership Survey—Professional Development Topics Covered by REAs Professional Development Services Percent of Respondents (N= 68) Received from REA Use of data 82% Reading and writing in content areas 81% North Dakota State Standards 78% Teacher evaluations 78% Table 2. Results from the School and District Leadership Survey—Most Useful Professional Development Professional Development Services Percent of Respondents (N= 68) Received from REA Use of data 50% Reading and writing in content areas 50% North Dakota State Standards 41% Teacher evaluations 37% Relying on Data Succeed 2020 has pushed the REAs to develop their capacities for collecting, analyzing, and using data as a means to both plan their services and demonstrate their own effectiveness. The steering committee has focused heavily on accountability by REAs for the results of their Succeed 2020 work, while FHI 360 has put significant investment into helping REAs build a “culture of evidence.” The collection and use of data has simultaneously been one of the greatest areas of progress for the REAs and the greatest area of need. Many Succeed 2020 stakeholders spoke about the progress REAs are making with data and how this has become part of an ongoing, regular culture within REAs. Not only were they collecting more data in general, they were more strategic about the types of data they were collecting. The REAs were seeking more technical assistance specifically around data and improving their evaluation tools; during this period FHI 360 hired a new staff member largely to respond to these requests. “We have never had the capacity prior to Succeed 2020 to really triangulate the data to the extent we are now, or to be collecting data on our members so that we can better design our services and plan for more efficient and more successful opportunities.” REA staff member 11 Going forward, REA and FHI 360 staff members posited that the REAs would need to focus more closely on using the new data they are collecting to drive decisions. They would also need to consider how best to use the data to communicate about their effectiveness to various audiences. FHI 360 has increasingly provided targeted support for developing REAs’ data capabilities. FHI 360 technical assistance providers have offered training on how to collect and analyze data and use it for making decisions. During this period, the FHI 360 team provided a series of ongoing trainings for REAs across the state. With this assistance, REAs were planning their assessment tools and calendars more carefully. This work was also helping to promote more consistency across REAs in data collection and use. According to an FHI 360 staff member, FHI 360’s data work with the REAs has focused on “creating evaluation instruments and using them effectively… That need [for REAs to strengthen their data capabilities] probably would have emerged on its own, but because we’ve [FHI 360] really made it a focus, they’ve been very open about needing help.” REA staff members valued this support; according to one, “help with the data has been, at least to me, the most useful in terms of …[realizing] we can work with our schools, and learn what they need, and try something and figure out if it’s working or not working.” REAs struggled to link their activities directly to student outcomes. They relied on several alternative sources of information to understand the effectiveness of their professional development. REAs deliver the bulk of their services to educators, rather than directly to students, so it is difficult to attribute demonstrable changes in student outcomes to their activities. While they were building their capacities to analyze student data, REAs were also relying heavily on perception data from educators to assess their services. REAs’ have relied on data sources including: Follow-up surveys from professional development participants: With technical assistance, REAs have developed improved surveys asking educators how useful they find professional development and how they have implemented their learning in the classroom. Many teachers have responded to the implementation surveys positively, indicating that they have made changes in classroom practice as a result of professional development. Some REAs have also conducted focus groups with educators to understand what they need from professional development and how useful they find it. Participation rates in professional development: Several REA staff members reported that they have seen increasing enrollments in their professional development both by individual educators and entire districts. They take these increases as evidence that their services have increased in quality. Classroom observations: REA staff members conducting school visits have observed changes in classroom practice that they ascribe to their professional development. According to one Succeed 2020 director, REA staff members were “seeing the benefits of that professional learning [that we provide]. They’re seeing the teachers using the framework, providing objectives, and pushing the students toward what we would consider best practices.” “[Our] ultimate goal is that our impact is on the students, by changing the practice of those in the classroom and those heading up the leadership of the school.” -REA staff member 12 Succeed 2020 stakeholders recognized REAs’ progress in relying on data, but acknowledged that they have a long way to go. REA staff members reported that they were collecting more data and relying on it more heavily in their decision making. They also described how they were beginning to use data from their members to plan and deliver services, instead of going on “gut.” Some steering committee members were frustrated by the lack of concrete data showing the impact of REA services, but others saw growth in REAs’ data literacy. According to one steering committee member, “[REAs are] using data in a much better way to drive their decisions. They’ve really grown in their understanding over the last 14 or 15 months of how important it is.” Impact of Succeed 2020 With REAs now comfortably settled in running their Succeed 2020 programming, and becoming more comfortable with collecting and using data, they were increasingly focused on the impact, or perceived impact, of their professional development. Their feedback from educators has been largely positive, as were the results of the survey of school and district leaders collected for this report. It appears that REAs’ member districts were increasingly coming to value the services provided by their REA. “Once you’ve been pushed to really start thinking…from that perspective of, ‘What are our outcomes and how are we going to measure it?’ I don’t think you can ever go back.” -REA staff member Succeed 2020 helped some REAs provide services to new audiences. Ultimately, REAs’ activities are intended to improve student outcomes in North Dakota, but with some exceptions their services are focused on teachers and school administrators and, to a lesser extent, counselors. Most REA staff members regarded teachers as the primary audience for their services; Succeed 2020 resources have enabled REAs to offer services to more teachers in their regions but also to new audiences. In at least one REA, Succeed 2020 enabled them to offer professional development to administrators and counselors, where before they had only worked with teachers. School and district leaders reported that REAs effectively addressed their needs. Their satisfaction with REA services increased during the grant period. Survey responses collected from school and district leaders for this report indicated that their satisfaction with REA services increased from the 2013–2014 school year to the 2014–2015 school year. Although it is not possible to link this increased satisfaction directly to Succeed 2020 services, it is worth noting that members’ satisfaction with their REAs increased as Succeed 2020 implementation deepened. Reinforcing REA staff members’ reports that they were working hard to pinpoint the specific needs of their member districts using data, survey respondents indicated that REAs were effectively identifying schools’ needs. According to one school or district leader, “The [REA] does an excellent job of collaborating with us to address the need specific to our district as well as other schools. We appreciate their willingness to get direct input [from] us when making programming decisions.” 13 Figure 2. How effectively do REA staff members work with your school and/or district to identify needs of students and teachers? Percent Who Answered Very Effectively 0.6 0.5 0.4 0.3 0.2 59% 39% 0.1 0 2013-2014 2014-2015 Correspondingly, as REAs’ became better at identifying their members’ needs, they were also addressing those needs more effectively. Figure 3. How effectively do REA staff members work with your school and/or district to address needs of students and teachers? Percent Who Answered Very Effectively 0.6 0.5 0.4 0.3 0.2 57% 39% 0.1 0 2013-2014 2014-2015 Compared with the 2013–2014 school year, in 2014–2015 more school and district leaders were very satisfied with the services they received from their REA. 14 Figure 4. How satisfied are you with the quality of services that your REA provides to your school and/or district? Percent Who Answered Very Satisfied 0.8 0.7 0.6 0.5 0.4 0.3 56% 68% 0.2 0.1 0 2013-2014 2014-2015 These survey results dovetail with REA staff members’ perceptions that member districts were increasingly seeing the value of their services. Hopefully REAs will be able to capitalize on their increasing facility with data and stronger professional development capabilities to maintain this upward trend in members’ satisfaction with their services. Planning for Sustainability REAs were beginning to think of what Succeed 2020 services they would or could sustain after grant funding, and potential avenues of funding for doing so. In spite of their growing reliance on data for programmatic decisions, they had not yet turned to those data in identifying priorities for sustainability, although many hoped to do so. Technical assistance from FHI 360 also began to focus on planning for sustainability during this period. The need to plan for sustainability will become increasingly urgent as the end of the Succeed 2020 grant approaches. REAs were still in the beginning stages of planning for sustainability, but many were beginning to think about potential future funding sources. Almost all the REAs would like to receive technical assistance around grant writing and identifying additional funding sources in the final years of Succeed 2020. Although none of the REAs had developed full sustainability plans, there was some variation in their thinking about sustainability. Some REAs were still in the “conversation” phase of planning for sustainability, meeting with their staff and their board to discuss options and alerting staff about how the end of Succeed 2020 might affect them. Other REAs had brainstormed additional funding sources and were beginning to decide which services to sustain. Staff members from these REAs shared some ideas about potential sources of future funding, including: Dedicated funding from the state legislature Increased member dues from districts “We haven't had the full-blown [sustainability planning discussions yet]. We…just started saying, ‘Okay, here's where we're at. We got to start thinking about this [sustainability] and what are we going to do.’" -REA staff member 15 Establishing or expanding fee-for-service charging members for specific services Funding through the Department of Public Instruction (DPI) External grants Shared services among several REAs It appeared that most of the REAs have little to no experience applying for external grants. REA staff members posited that FHI 360 would be in the best position to help them identify external funding sources and assist with effective grant writing. REAs would be interested in workshops around this topic and many were committed to learning more about applying for external grants. Most REA staff members mentioned developing their grant writing capability as a priority for their final years of the project. REA staff members hoped to maintain their growth in using data. Although many respondents did not explicitly reference data to help them determine what REAs should sustain, REA staff members themselves have recognized the importance of data and would like to continue to sustain a data culture beyond the duration of Succeed 2020. Many continuously acknowledged how far they had come in their ability to collect and use data. Specific practices they hoped to sustain included continuously collecting data about their services, keeping data experts on staff, accessing and using the statewide longitudinal data system, and using data to improve programs and services. However, among the items listed as services the REAs would like to sustain, more than half of the respondents did not use data to help them determine what those services should be. REAs needed support in using data to help them determine which services to sustain. Several FHI 360 liaisons highlighted the need for REAs to use data to be more strategic in selecting which services to sustain. FHI 360 staff members were also pushing for REAs to develop a plan reflecting their vision for the future of their organizations. Among the REAs, there was as yet little clarity on how to accomplish this task. REAs will need support around using data to help them determine which services to sustain. Many REA staff members understood that they should use data, but it was unclear whether they knew how to do so effectively. FHI 360 is planning a set of technical assistance tools and strategies to help the REAs in this effort. Professional development, events for students, and project staff emerged as the features of Succeed 2020 REAs most hoped to sustain. Several REA staff members noted how much their member districts appreciated the improved and aligned professional development services the REAs were providing under Succeed 2020. They hoped to maintain this level of service beyond the Succeed 2020 initiative. Most REAs also listed at least one of their annual events as something they hoped to sustain. These included events like the Scrubs Camp and career expos. These events were very popular among their member districts and communities, and thus worth sustaining according to REA staff members. Many were also fairly certain that they 16 would find the resources to sustain these events because of their relatively low cost and buy-in from regional business communities. A few REA staff members described their desire to maintain current staffing levels in order to sustain the services they provide for their member districts, particularly with professional development and maintaining their data culture. Few REAs had a plan in place for funding these positions beyond Succeed 2020 but acknowledged the need to start looking for alternative funding. FHI 360 staff members were less optimistic about the REAs’ ability to keep current staffing levels, but all offered support in helping the REAs find additional funding sources. Collaboration among the REAs has increased dramatically because of Succeed 2020 and most REAs would like to sustain this stronger collaborative spirit. A common theme among respondents was how much more the REAs were communicating and working with each other because of Succeed 2020. The REAs operated in silos before, but now worked with other REAs on an ongoing basis. Not only has the amount of collaboration increased, but most REA staff members spoke of the importance of collaboration going forward for sustainability. Specifically, nearly all REA respondents mentioned sharing resources with each other and their support for the strengthening of NDREA. Some specific services or positions they believed could be shared among the REAs included professional development offerings, data staff, and technical expertise in various subjects. There was consensus among Succeed 2020 stakeholders on the different roles each should play in planning for sustainability. When prompted about what role each of the Succeed 2020 partners would play in sustainability, most respondents agreed about what those roles should be. Succeed 2020 stakeholders hoped that each partner would fulfill their roles as follows: Steering committee: Many respondents saw the steering committee as advisors to the REAs and advocates and champions for the REAs to the state legislature. They hoped that REAs’ development under Succeed 2020 and the outcomes of their Succeed 2020 work would help make the case to the legislature that REAs were vital to education in North Dakota. External budget realities may impact the likelihood of REAs obtaining additional funding from the state legislature. State agencies: Steering committee members, some of whom represent state agencies, reported that Succeed 2020 has helped them realize the value of REAs. A slight majority of steering committee respondents reported that REAs should make sure they are contributing to state-level collaboration and goals. Some REA staff members believed that a partnership with DPI would be beneficial. A few steering committee members went further and suggested that DPI or some other agency should provide oversight to the REAs. There were mixed feelings about this idea from REA leadership, with some strongly opposing the idea. Some REA staff members were content with their current organizational structure and would prefer reporting to their board versus an entity like DPI. FHI 360: REA staff members overwhelmingly reported that FHI 360 should provide technical assistance around writing grant proposals and identifying funding sources to assist with sustainability. Some REA staff members also mentioned that FHI 360 should continue to provide assistance with collecting and using data to help with sustainability purposes. “The other greatest aspect of this work is the collaboration across REAs. It gave us all something in common to talk about, beyond what we had in common before.” -REA staff member “[REAs need to] continue to focus on that state level collaborative contribution.” -Steering committee member 17 NDREA: Most REA staff members saw the value of operating as a cohesive unit as NDREA. They referenced NDREA as an entity that could provide a common voice for the REAs in the search for additional funding as well as providing future services. However, some REA staff members were more committed to the idea than others, perhaps because REA directors have been most involved in NDREA while some of the REA staff members that were interviewed have not. 18 Shifting toward Sustainability Over the course of Succeed 2020, REAs have strengthened their own organizations and the services they offer member districts. They are responding to calls to enhance their data capabilities, both to plan their ongoing work and to prove its worth. School and district leaders have recognized the growing strength of their REAs. Nonetheless, REAs and their partners contend with continuing challenges and questions, many centered on data and REAs’ roles, that will need to be addressed as the initiative pivots to focusing on sustainability. REAs were developing a stronger collective identity and working to strengthen NDREA as a vehicle for their communal voice. Most REA staff members hope to maintain this collaborative spirit, and NDREA may become an important channel to advocate for the REAs’ relevance as the state considers future funding for REAs. Increased staffing levels have allowed REAs to enhance the level of services they provide to their members. As grant funding ends, REAs and their partners may need to consider new staffing models if they hope to maintain a similar level of services. REAs struggled to link their activities directly to student outcomes. If they plan to seek further funding, they and their partners will need to determine how to demonstrate the effectiveness of REA services. While Succeed 2020 has invested heavily in developing REAs’ data capacities, it was universally acknowledged that most REAs have significant room to grow. They will need to continue developing their skills in collecting, analyzing, and using data to drive their decisions. Crucially for sustainability, they will also need support in learning how to use data to share their successes. Most REA staff members hoped to continue at least some of their Succeed 2020 services and activities, perhaps in a reduced or somewhat different capacity, with new sources of funding. Almost all the REAs would like to receive technical assistance around grant writing and identifying additional funding sources in the final years of Succeed 2020. It is likely that REAs will not be able to sustain every facet of Succeed 2020 programming they currently provide. REAs will need support in using data to help them determine the most effective services to sustain. 19
© Copyright 2024 Paperzz