Prepared by Education Analytics, Inc

December 2023

This report describes the initial findings of a longer interim evaluation of support being provided by UnboundEd/CORE to two school districts implementing high-quality middle grades mathematics curricular materials: Guilford County Schools (GCS) and the Monterey Peninsula Unified School District (MPUSD). The evaluation is being conducted by Education Analytics, Inc, and was commissioned by UnboundEd/CORE. This report addresses two evaluation questions:  

1. Was UnboundEd/CORE support delivered as intended?
2. Did this support contribute to building the districts’ capacity to implement high-quality mathematics curricular materials?

The findings reflect UnboundEd/CORE activities occurring between the initiation of support in 2021 to the end of the 2022-23 school year.

The Effective Implementation Cohort Initiative and Partner Districts

UnboundEd/CORE is providing support for curricular materials implementation through the Effective Implementation Cohort (EIC) initiative. The official launch of the three-year grant started in the summer of 2021 and will conclude in the summer of 2024.

UnboundEd/CORE selected GCS and MPUSD to participate in the EIC grant, through a selection process in the winter of 2020. GCS is based in Greensboro, North Carolina, where they are the third largest district in the state. The district serves grades K-12 and has a population of 68,000 students and 24 middle schools. Guilford County Schools selected Open Up Math as their curriculum. MPUSD is based in the Monterey Peninsula of California. The district serves grades K-12 and has a little less than 10,000 students and five middle schools that serve grades 6-8. MPUSD selected i-Ready as their curriculum. MPUSD selected their curricular materials during the 2018-19 school year and started implementing them in the 2019-20 school year. GCS selected their curricular materials during the 2017-2018 school year and started implementing them in the 2018-2019 school year. UnboundEd/CORE began working with both districts during the 2021-22 school year.

The UnboundEd/CORE Support Model 

The curriculum implementation model being implemented in GCS and MPUSD through the EIC project focuses on building the capacity of both district and school leaders to support the implementation of high-quality curricular materials. Because the effective implementation of high-quality materials is dependent on systemic change, the UnboundEd/CORE model requires multiple years of work—starting in year one with building leadership capacity to support educators as they implement materials as and continuing through year three to build capacity to sustain the change at the district-level and across schools and to scale to additional grade levels and subject areas, as appropriate. The specific goals of the curriculum implementation support model include: 

• Building sustainable district- and school-level curriculum implementation systems
• Providing professional learning experiences that enable district and school leaders to effectively support the implementation of high-quality materials
• Engaging district and school leaders to monitor and improve curriculum implementation

What support did UnboundEd/CORE deliver in the first two years of the EIC grant?

UnboundEd/CORE provided substantial support to both district implementation teams and school-level site leadership teams. At the district level, support provided included: 

• Helping districts complete implementation plans, which included an instructional vision, implementation goals, communication and professional learning plans, and progress measurement methods.
• Providing professional learning to district implementation teams through a two-day Systems Leader Academy as well as through providing thought partnership, sharing resources as well as experiences or learnings from other districts, etc.   
• Through weekly meetings with district leads as well as district implementation team meetings, helping the district teams to make decisions and keep on track, check progress, and revisit and refine implementation goals.
• Helping both districts develop data systems, especially around collecting and analyzing implementation data from classroom walks.

Because materials are implemented at the school level, UnboundEd/CORE also provided school-specific support. At the school level, all leadership teams received or participated in each of the planned support components – onboarding days, professional learning days, various types of classroom walks, and end-of-year program review days. UnboundEd/CORE supported school leadership teams to analyze data and develop plans to prepare leaders to directly monitor and support curriculum implementation. Throughout the partnership, UnboundEd/CORE worked to build the capacity of school leaders around the adopted curricular materials and supported them with creating the conditions necessary for successful implementation.

There was some variation in how many of the different types of walks schools conducted, but all schools conducted at least two in the first two years of UnboundEd/CORE support. There appeared to be little other variation in support across schools, though two additional schools were added in MPUSD and four GCS schools moved from the first to the second cohort of GCS’s phased implementation.

The findings below justify further analyses to examine potential impacts of UnboundEd/CORE support on teacher and student-level outcomes. Unlike many interventions where implementation is weak, in both districts there is evidence of a strong effort to provide support that one would reasonably expect to influence these outcomes.

How was the quality of UnboundEd/CORE support perceived by those who received it?

Despite some UnboundEd/CORE staffing issues in the first year of the engagement, in both districts, leaders seemed quite happy with UnboundEd/CORE’s district implementation team support and appreciated its value. Flexibility, thought partnership, and effective communication were positive aspects noted by leaders in both districts. 

At the school level, substantial majorities of respondents to post-activity surveys agreed or strongly agreed that the activities were well planned and conducted, relevant, and meaningful. For example:

• Eighty-eight percent or more of Onboarding Day participants agreed or strongly agreed that the session was well facilitated and well planned, materials were well prepared, time was used efficiently, and the work done in the sessions was relevant and meaningful.
• Eighty-five percent or more of those who attended the Professional Learning Days for school leaders agreed or strongly agreed that the sessions were well facilitated and well planned, materials were well prepared, time was used efficiently, and the work done in the sessions was relevant and meaningful. 
• Ninety-six percent or more of the respondents to surveys administered by UnboundEd/CORE after the classroom walks had positive perceptions of the quality of the sessions. 
• Eighty-three percent or more of respondents to UnboundEd/CORE’s program review day surveys had positive perceptions of session quality (e.g., relevance, facilitator knowledge, opportunities for collaboration). 

Did UnboundEd/CORE support build district and school-level capacity to implement high-quality curricular materials?

In both districts, there is evidence that the level of support provided influenced implementation capacity.

At the district level, support helped to increase knowledge of what effective implementation looks like and provided processes and tools to assess it. UnboundEd/CORE’s support was especially important for MPUSD, which due to its limited resources would otherwise not have been able to organize a sustained and coherent implementation effort. Some highlights of the capacity building included:

• The classroom walks organized by UnboundEd/CORE improved the capacity to assess the degree of implementation. Neither district appears to have had a systematic way of doing so before UnboundEd/CORE support. These data were perceived to provide more proximal information than the test results GCS was tracking.
• The support helped the districts develop standard operating processes that can be used after the grant to continue to support curriculum implementation in the schools.
• In GCS, UnboundEd/CORE support helped to unify communication with schools about what effective curriculum implementation looks like, and encouraged consistency in communication among district staff, UnboundEd/CORE staff, and coaches, so that all were giving consistent feedback to schools. 
• In GCS, UnboundEd/CORE support built the capacity of district leaders to carry on with classroom walks and continue support for schools.
• In MPUSD, especially given district size, UnboundEd/CORE support helped build district data capacity for curriculum implementation. In GCS, UnboundEd/CORE support helped the district build data capacity by supporting the consolidation of implementation-related data into a dashboard that facilitates implementation team conversations and allows better tracking of progress. 

At the school level, the specific UnboundEd/CORE services helped school leadership teams build implementation capacity. By the second year, most school teams had developed and were leveraging implementation plans. Responses to UnboundEd/CORE’s satisfaction surveys also provided evidence of capacity building. For example:

• Eighty percent or more of survey respondents agreed or strongly agreed that they were confident they were prepared to take the next steps toward implementation of the new curricular materials, that their learning had increased, and they were prepared to implement the new learning. 
• All respondents to surveys after year one in both districts perceived that the classroom walks helped them norm on a problem of practice, identify patterns of practice across the school, identify staff learning needs, and plan next steps in curriculum implementation. 

What’s Next

UnboundEd/CORE is in the final year of service delivery with MPUSD and GCS, with the EIC project ending in the summer of 2024. There will be two additional reports as Education Analytics continues to gather data and develop reports around the impact of the curriculum work with GCS and MPUSD. One report will focus on the impact of the support on classroom practice and the other on the overall impact of the work on student achievement results. Those reports will be available over the next year.