A Home Run for Quality Improvement

We know that the quality of afterschool programs can have an impact on learning. We know that the quality is measureable. But is quality improvement malleable? That is, do we know how to improve and sustain the quality of “expanded learning” across the constellation of community programs? Is it marketable? That is, are the tasks and resources required to continuously improve quality within the reach of the average program manager? The average community?

After too many years of waiting for these answers, we have the first study that gives us not one, but two solid yeses.

The study looks at the Youth Program Quality Intervention (YPQI), a data-driven continuous improvement model developed for afterschool systems by the David P. Weikart Center for Youth Program Quality. (Full disclosure: The center is a division of the Forum for Youth Investment.) A summary and link to the report are here, and you can take 15 minutes to read the four-page brief about the Implications. I’ll give you the top notes now.

Employing the kind of rigor that researchers insist on (such as use of a control group), the study shows that continuous improvement:

  • Works well – Compared with staffers in afterschool programs in the control group, the site leaders and frontline staff who used the continuous improvement practices well and often improved the quality of instruction. Those practices include assessment, planning, training and coaching.
  • Works everywhere – The afterschool networks participating in the study represented different sizes and types of communities, organizations, funding mechanisms and policy structures.
  • Works with everyone – Afterschool programs frequently have staffing challenges – high manager turnover, low staff education levels, high youth/adult ratios. The YPQI worked as well in challenged programs as it did in better-equipped ones. In fact, staff turnover in YPQI sites was lower than in control group sites.
  • Works over time – The improvements were maintained one year after the post-intervention data were collected. Sites had maintained a focus on improvement, staff had still used the continuous improvement practices, and staff employment tenure remained high.
  • Works with modest staffing plans and budgets. The time commitment associated with the YPQI is modest compared with other interventions which use external coaches and require more staff time in training to produce similar effects. An average site team of one manager and three front line staff spent a combined total of 123 hours on YPQI activities over 18 months. That’s an average of less than three hours a month for managers and 1.5 hours a month for frontline staff.

This study couldn’t have been released at a better time. Consider these opening lines from a recent article in Education Week on expanded learning:

“A variety of efforts have sprung up across the country to define and improve the quality of after-school staff, some of which bear resemblance to the quest to improve the effectiveness of classroom teachers. But given that many out-of-school programs face limited funding and their staffs tend to be young, part-time workers who rarely commit to the job for long, questions remain over how to provide professional development in a cost-effective way.”  

The continuous improvement model is the best professional development solution to the afterschool quality challenge, because professional development needs to be directly linked to and demonstrated through the daily work of doing the job. The closing sentences of the Implications brief are worth highlighting. They speak not only to the power of this particular intervention, but to the broader power of placing information in the hands of those empowered to use it:

“Despite the fact that performance data was not made public and programs were not threatened with sanctions for low scores, program quality improved in response to standards and supports designed to empower site managers to enact the four continuous improvement practices.”

So here we have standards to meet, solutions to fulfill them and ways to measure success. That’s the power of adopting a continuous improvement approach to increasing program quality.