Introduction

For a measure to be considered practical, it must be minimally burdensome to users and attend to the social processes of use that help activate a strong culture of improvement. While technical considerations are certainly important when developing and testing any practical measure, it is just as critical to build and nurture social routines that engage an improvement community of diverse expertise. It is also integral that we remain intentional and proactive about empowering all team members to stay present as active agents of change.

The Student Engagement Exit Ticket measure was developed out of such commitment to create an improvement-focused culture that enables a generative, trusting community of improvers to come together to openly share insights and ideate solutions through a collaborative design process. This rich practice of collective brainstorming not only allows for greater alignment among individuals focusing on a shared problem of practice but is also instrumental to designing a minimally burdensome, user-friendly measure that best serves the needs of front-line practitioners.

The Need for Measurement

The Student Engagement Exit Ticket measure is a simple survey designed to provide teachers with direct feedback on student engagement around specific problems of practice. Annie Wilhelm, a professor at Southern Methodist University, collaborated with Dallas Independent School District leaders and community members to develop a new STEM-focused PK-8 school. To support this work, the design team worked with a local elementary school to improve teacher learning around using data and continuous improvement methods in their practice. However, shortly after they began this work, COVID-related school closures shifted their focus to helping teachers better understand important classroom dynamics like student engagement during remote and hybrid learning. From student feedback data, the design team and teachers hoped to not only learn about what was working for whom and under what conditions, but also to identify critical areas of unmet need and use that data to inform changes in practice. Testing this measure in the context of the pandemic provided timely learning opportunities for teachers grappling with the adjustment to remote instruction.

The Measure: Student Engagement Exit Ticket

The Student Engagement Exit Ticket is a short survey that collects high-level data on student engagement with content. It is intended to be regularly administered at the end of class (see Figure 1) so students can communicate their needs to teachers by providing feedback and educators can take the next steps to adapt their instructional practices.

Figure 1. Screenshot of Student Engagement Problem of Practice Measure

When considering measures for improvement, the social routines and processes guiding how a measure is developed and used are just as critical as the data it produces. It is important to note the unique nature behind the development of this measure, which saw teachers taking a lead not just in the implementation but also in the design of the survey tool itself. The teacher-led design of this measure speaks to the team’s intentionality towards nurturing social learning routines that energize diverse stakeholders, particularly teachers as front-line practitioners, to actively drive and engage in the improvement process. Additionally, because the measure utilizes an online survey platform (Google Forms), the data is immediately available and teachers can easily review the results, thus providing actionable information that teachers can use to improve their instructional practices.

Further, to ensure that the measure was truly practical for teachers, the professional learning design team was committed to minimizing the burden of implementation on teachers – from creating a low-lift survey that is easy to incorporate into their daily work routines to facilitating the creation and data aggregation process for them. Following the data aggregation, the PD sessions provided a low-stakes, safe environment for data to be shared and for important peer-to-peer learning to occur.

Insights from Using the Measure: Co-creating measures with teachers

The Student Engagement Exit Ticket survey, though not currently in use, offers improvement practitioners important considerations for the development and implementation of practice-based measures. The collaborative design and use of this measure spotlights a countercultural, simple approach that is accessible and can alleviate the burden on practitioners while also elevating their voice in the development process. To develop this measure, Annie and several other collaborators met virtually with the school principal and 15-20 teachers to identify problems of practice they wanted to work on, ranging from the use of technology for instruction to relationship building with students. Based on shared problems of practice, teachers worked together on a virtual whiteboard (Jamboard) to develop look-fors and brainstorm statements they wanted students to say at the end of a lesson with respect to their focal problem areas.

Figure 2. Screenshot of Brainstorm Session Jamboard

Building on the various problems of practice and insights gathered from their brainstorming exercise, the professional learning design team generated multiple short surveys in English and Spanish using Google Forms. Using Google Forms, which was already a familiar tool to teachers, made it easier for the measure to be built into their practice. In so doing, the design team aimed to develop initial measures that could be iterated on as necessary, with hopes of making the measures as close to the teachers’ practice and needs as possible. The teachers then took these surveys to pilot with their students. Though they did not employ a formal protocol for data analysis, the team is aware of their blind spots and would like to use this routine again to revise the measures and to better understand if they can provide more information around variability. The entire group came back together for a PD session following their first testing cycle, where Annie and her team synthesized emergent findings from the data analysis in an effort to facilitate discussion among teachers and engage them in a collective sense-making process. The integration of research, professional, and improvement knowledge in this measure’s development also helped foster a sense of shared responsibility around the improvement aim and collective learnings. 

So, what key takeaways can practitioners glean from this measure? Collective engagement and robust social processes are necessary for maximizing the potential of a practical measure to accelerate change.

Impact of Measurement for Improvement

While the Student Engagement Exit Ticket is not currently in use and has yet to be iterated on, it provides a strong example of a counterculture approach to measure development, as front-line improvers took an active role in designing measures to support their learning and improvement efforts. This collaborative development process has the power to influence a shift in mindset about what it looks like to design and use measures for improvement. By expanding who is involved in measure development, this process also helps build collective will among front-line practitioners and drives their enthusiasm to improve their practice. At the core of improvement is a shared commitment to learning. Improvement demands the active, full engagement of teams and its success is predicated on the ability of a team or network to manage both the technical and human sides of change. This team implemented intentional practices to engage the diverse community of practitioners, researchers, and leaders in their collaborative effort, and we can learn from their best practices as we attend to the social routines that support the development of an improvement culture at large.