Aligning Evidence
Best Practices Discussed at our Children’s Cabinet Network Summits
Focus on continuous improvement.
One of the biggest concerns with this emphasis on evidence is the fear that if results show less than promising outcomes, programs will lose their funding and will be eliminated. A culture shift is needed to allow leaders the space to take risks and find out what works, balanced by ways to maintain accountability. Cultivating trust, regular feedback, mutual accountability and capacity building can ensure that partners do not feel threatened by evaluations. Instead, it should be viewed as a method to help practitioners get to the outcomes we all want to see. As one participant noted, evidence can be used to consider, “How do we pick what’s right for us? How do we implement it effectively?”
Context matters.
In the past, leaders often skipped right over looking at research and evidence before creating policies. It seems now we have gone to the other extreme of the spectrum and everyone has jumped on the evidence bandwagon. There has been a clear shift in desire at the state and local level to finance evidence-based programs, but evidence-based is a very new discipline for a lot of leaders in the field. While the increased use of evidence in policy is encouraging, you cannot drop a program into a community just because it was successful in a randomized control trial. You have to have buy-in from the community and it has to be relevant for the particular community context. Policymakers need to consider the populations a specific program is designed to address and look at ways to change their home-grown programs so that they meet new standards as the evidence-base for certain approaches grows.
Utilize a broader definition of evidence.
Leaders at all levels of government are beginning to recognize the need to use a range of evidence types in their decision-making processes. Public officials mentioned their use of evaluations, statistics, administrative or survey data, performance improvement data and behavioral analytics. The federal government is also using a tiered evidence approach that focuses more funding for programs backed by a RCT, but encourages other experimental analyses as well. The federal government also uses behavioral insights to improve program efficiency and save taxpayer dollars. Local governments also recognize that much of the data they collect can be used to improve programs or tie funding decisions for homegrown programs to outcomes when expensive RCTs are unavailable.
Capacity-building remains a priority.
Local and state agencies spend a large amount of their time building their own capacity and the capacity of their partners to use evidence when making policy choices or setting priorities. Officials in places like New York City have started using protocols to allow multiple agencies to share data, question methodologies and clean their data sets. This allows agencies to have the most up to date and useful evidence when reforming programs. Other local agencies work with their grantees to increase their capacity for evaluation. This allows agencies to focus on improving the work of their grantees and shift funding to more effective evidence-based programs or practices.
Examples Shared at our 2016 Children’s Cabinet Network Summit
Rhode Island updated their procurement process so that partners were demonstrating evidence that procured items would get a specific outcome and not take up valuable resources.
The Pew-MacArthur Results First Initiative is working to build inventories of programs and how much they cost in a number of jurisdictions. This helps localities understand what the return on investment is for various evidence-based programs. The organization has also developed a clearinghouse of clearinghouses in order to make evidence more accessible to local partners.
The Annie E. Casey Foundation’s Evidence2Success framework helps jurisdictions use evidence- based policies through a youth survey that demonstrates local needs. This information can be used by policymakers to understand risk factors in their communities and identify programs that mitigate those risks.
Examples Shared at our 2015 Children’s Cabinet Network Summit
Providence Child and Youth Cabinet is using data and evidence in innovative ways as a member of the Annie E. Casey Foundation’s Evidence2Success initiative. The Cabinet, a coalition of 60 organizations, established an Evdience2Success action plan through 2018. The Evidence2Success Prevention Framework emphasizes data from local youth and the evidence-based programs.
Broward County’s Children’s Services Council uses what they call “relational contracting” as a means of allowing program providers to use evidence for continuous improvement. As a funder they provide support to providers and encourage them to share the barriers and problems they come across as opposed to threatening pulling their funding. This has been a cultural shift for their community stakeholders who have not been funded in this way before. The Council believes they don’t succeed unless the grantees succeed – so they work in partnership and provide technical assistance to help them get to the agreed upon outcomes.
Connecticut’s legislature is at the forefront of using results based accountability in their appropriations committee.
Illinois’ Governor’s Office of Management and Budget has developed Budgeting for Results a publicly transparent and accessible site created by a Commission responsible for advising the Governor in setting outcomes and goals and timelines associated with the budget.
Congress has established an Evidence-Based Policymaking Commission charged with conducting a comprehensive study of the data inventory, data infrastructure and statistical protocols related to federal policymaking and the statistical and programmatic agencies responsible for maintaining that data.
The White House Office of Management and Budget has placed significant emphasis on evidence and evaluation, including in the FY16 President Budget, and other federal programs.