Inclusive Classrooms Survey

Inclusive Classrooms Survey as a Practical Measure

Characteristics of Practical Measures	Descriptions of the Inclusive Classrooms Survey Measure
Is closely tied to a theory of improvement	In partnership with Panorama Education and the Rennie Center for Education Research & Policy, Teaching Matters has developed the Inclusive Classrooms Survey to monitor shifts in a primary driver. The measure has six factors and taps into domains of inclusive classroom practices that the NSI theorizes that, if fostered, can help students stay on track in 8th grade. The six domains of focus are classroom belonging, classroom engagement, cultural awareness and action, rigorous expectations, teacher-student relationship, and valuing of subject.
Provides actionable information to drive positive changes in practice	The information provided by the measure is actionable for improvers at different levels. Using the dashboards created by Panorama and data analysis protocol designed by the Teaching Matters hub, teachers can individually and collectively surface glows (i.e., bright spots) and grows (i.e., areas of concern) and work with school leaders and coaches to identify strategies — change ideas coming from the Teaching Matters NSI Change Menu and the Panorama Playbook — to better student experience. Meanwhile, the Teaching Matters hub uses the survey data to populate p-charts as the team seeks to unveil and understand variation among classrooms and schools. Through attending to variation systematically, the hub makes progress in answering the questions of what works, for whom, and under what conditions, as well as coming up with plans to support schools in addressing variation in student experience.
Captures variability in performance	Depending on the individual teacher’s or school’s choice of change idea, different domains of inclusive classroom practices may be targeted. If a teacher chooses a change idea that is designed to help students build a sense of classroom belonging, the teacher may not expect positive changes in student valuing of subject. Because the data are broken down by domains, improvers can study variability in student experience across domains and check if the observed variability aligns with their predictions.
Demonstrates predictive validity	According to the Teaching Matters NSI’s theory of improvement, inclusive classroom practices should predict 8th grade on track. To test this hypothesized causal link, the Teaching Matters hub examined the relationship between the two constructs using the Inclusive Classrooms Survey and ELA screener assessments (i.e., iReady/MAP Growth). The results suggested that improvement in inclusive classroom practices was positively related to change in the percentage of students meeting ELA growth goals. Although further investigation is warranted, these results are evidence that the survey has predictive validity. This is important because it means that if growth in survey results is detected, it can be assumed that the network is making progress toward its aim.
Is minimally burdensome to users	It takes intentional planning to ensure smooth administration of surveys. It is no exception in the case of the Inclusive Classrooms Survey. Teaching Matters hub members described rostering as being a “medium-level lift” for schools, although they provide schools with rosters pulled from the New Visions for Public Schools data platform. Reviewing and revising rosters and coordinating as needed can be time-consuming. Also, given that the Inclusive Classrooms Survey is a 33-item measure that includes three open-ended items, doing the survey more than once a year can feel burdensome to students, especially if teachers do not share with students what they are learning from their feedback.
Functions within social processes that support improvement culture	The Teaching Matters hub acknowledges the importance of building the capacity of network members to engage in collective sensemaking of the survey data. First, the hub rolled out a data analysis protocol at a network convening. The protocol centers around three questions: 1) What are your students telling you? 2) What does that mean they are asking you to do? 3) How might you address their ask? At the convening, network members had an opportunity to practice using the protocol to examine network-level data and experience the power of reflecting as a group on the three questions. They were then asked to apply the protocol to analyze school-level data at school improvement team meetings. Through this social learning process, network members can have a better understanding of both the survey data and data analysis protocol and how they can embed both in their improvement routines.
Is reported on in a timely manner	Given that the Inclusive Classrooms Survey is designed to assess students’ general perceptions of their classroom experience (that are relatively stable and take time to change) and not their specific perceptions of lesson(s) at a particular moment in time, the survey is administered three times a year in Teaching Matters NSI schools. Following survey administration, schools will receive results consolidated in dashboards. This allows schools to consistently monitor changes in student perceptions of inclusive classroom practices. Moreover, survey administration is often situated before and after action periods when schools engage in active testing of change ideas. This can make connecting improvement in change ideas to improvement in drivers easier.

Question on Practical Measures Inspired by the Inclusive Classrooms Survey

What are the ways to build buy-in around a shared practical measure?
Within an improvement network, there are often schools with different instructional models and priorities. Sometimes it may be hard for schools or classrooms to commit to administering common measures in the testing of a theory of improvement.

The Teaching Matters hub addresses this challenge by making the measurement work student-centered. Because the Inclusive Classrooms Survey is student-reported and contains open-ended questions, educators can hear and learn from students in their own perspective and voice. Elevating student voice is crucial in doing equity work. Such a focus brings together educators who are dedicated to dismantling inequities.

Also, the Teaching Matters hub invests time and resources in explicating a shared why. Building inclusive classroom practices may sound like the right approach to achieving equity, but each educator and student may envision such practices differently. To help the network establish a shared vision and commitment, the hub created a slide deck that describes six key domains of inclusive classroom practices and how they are measured by the Inclusive Classrooms Survey. This resource was shared with network schools, and schools that used this resource reported receiving more actionable data that they could learn from to inform their practice. The more schools can utilize data for improvement, the better the buy-in to collect and analyze the data.

Furthermore, the Teaching Matters hub develops structures and routines to support administration and analysis of the survey. The hub works with schools to set up a survey coordinator role and provides training to the coordinators. Additional support is also offered by the Teaching Matters improvement coaches. Besides helping teachers and school leaders learn from the survey data, coaches share what they learn from working with school teams with hub members to facilitate continuous process improvement.

The case of the Teaching Matters NSI illustrates that it takes strategic hub leadership moves to lead a network to arrive at a shared practical measure that resonates with improvers. It may be a challenging journey, but it will be worth it in the long run, as having common measures helps improvers learn from and reduce variation in performance, which is at the core of the problem of inequity.