Choosing the Right Tool: How Youth Programs Can Measure Their Impact on ‘Soft Skills’
May 21, 2014
Nicole Yohalem and I started out in this field nearly two decades ago as youth workers who cared deeply about helping young people develop skills that mattered. Working in separate programs (later becoming colleagues at the Forum for Youth Investment), we knew in our hearts that the work done by our nonprofits helped young people gain practical, real-world skills, such as collaboration, communication, perseverance and decision-making. We tried our best to come up with proof to back that belief – but we lacked tools sophisticated enough to make the connections.
Today, the number of easily accessible tools is still small, but their sophistication has caught up with growing demand, as more people say not only that these skills matter, but that they should be measured.
Two years ago, the Forum for Youth Investment collaborated with two of our research colleagues, David DuBois and Peter Ji, to create a guide reviewing user-friendly, rigorous youth outcome measures of socio-emotional skills. Socio-emotional skills include things like communications, collaboration, critical thinking, decision making, initiative and self-direction. These skills are at the core of many current buzzwords, such as soft skills, 21st century skills, and even “grit.”
We created From Soft Skills to Hard Data to cut through those buzzwords and the plethora of reports and commentaries that trumpet the importance of these skills. We wanted to give youth programs, which are increasingly under pressure to demonstrate outcomes, practical guidance for measuring their effect on the development of these skills.
The guide proved so popular (it’s the most downloaded report on the Ready by 21 website) that in January we released an updated version, done in partnership with the Weikart Center for Youth Program Quality. And on May 28, we’ll discuss practical ways to measure the impact of afterschool programs during this free webinar.
The guide is well-used because it provides straightforward information and gives professionals a basis on which to compare tools and understand the measurements on which they are built. The guide offers:
- Summaries of 10 rigorously designed instruments measuring 21st century skills. These summaries drill down to the “scale level”: a set of statements that hang together to measure particular skills and attributes, like empathy or problem-solving.
- Comparative information on user considerations for administering the tools. We created a Consumer Reports-style comparison of the features of different tools, looking at cost, target age, companion tools and content covered.
- A quick guide for program leaders who are just beginning to delve into youth outcome measurement tools.
- A primer on psychometrics, which deals with the validity and reliability of psychological measuring. Understanding the mechanics behind the tools (what makes them work) and having assurance that they reasonably measure what they say they measure is important for programs to know if these tools can weigh effectiveness.
We see programs using the guide in three primary ways:
- To narrow their choices among commonly accessible tools to measure effectiveness, or to help construct customized tools based on existing measures.
- To clarify their priorities and position themselves with funders, partners and other stakeholders.
- To help design performance improvement systems that provide data that generates meaningful feedback, in order to create actionable plans to serve young people better.
For example: The Washtenaw Alliance for Children and Youth, a county-wide provider network in southeast Michigan, used the instrument summaries to refine its community-level outcome objectives for all youth, and to facilitate alignment between programs in measuring common outcomes. The guide helped the alliance address its most difficult measurement challenge: identifying how to measure the indicators that the programs agreed to track. Agreeing to some common measures allowed the programs to coordinate their efforts to report on outcomes collectively to funders and other stakeholders.
The call for better supporting and measuring youths’ 21st century skills is popping up in all kinds of places – perhaps most noticeably in those places characterized, rightly or wrongly, as narrowly concerned with academics: schools. It turns out that soft skills, socio-emotional skills, 21st century skills and grit are all important to both the academic bottom line and to the full range of outcomes for young people.
We hope that offering some practicality and precision for determining how youth programs instill those skills will help providers in their quest to get all young people are ready for college, work and life.
Alicia Wilson-Ahlstrom is senior program manager for research and development at the Forum for Youth Investment.