[Keesha]

Evaluation can improve youth programs because it centers youth voices and experience which can enhance service delivery. Evaluation allows for reflection, it offers understanding of how service is perceived by participants, and the impact of services. It lets you know what you’re doing right and where the gaps are. It allows you to come up with a plan, to take action and implement these changes so services can be improved. It gives youth programs opportunities for improvement.

[jiannaquickstad]

I think this question could be answered in many ways depending on the type of evaluation being utilized. For example, we are working on an implementation process evaluation. So we aren’t really trying to “prove” anything, but rather understand what is going on. In this case, I think for the evaluation to be impactful and truly improve the program, there needs to be buy-in from all stakeholders. After the evaluation, there will need to be support to implement the identified changes and work to improve the program and the experience for end-users. Additionally, I believe beyond “buy-in” there needs to be established trust during the evaluation process so that the youth and other partners involved will be comfortable and honest sharing their genuine experiences so that the program can truly be improved and tailored to their needs.

[Gwen]

Evaluation is essential for improving programs. Evaluation should engage with all stakeholders of a program – program participants, program staff, community members impacted directly/indirectly by the program, etc. These stakeholders should all have the opportunity to share their feedback and satisfaction with a program, as well as any other stories or observations they might have. From this information, we can learn what went well, what didn’t go well, and how we can improve from next time. Evaluation also allows us to see if the program is achieving the outcomes we hoped it would achieve – and if we’re not achieving them, then evaluation gives us the opportunity to reflect and understand why. Maybe we’re missing a key activity in our programming, maybe participants were confused about an activity we did run, or maybe that outcome isn’t actually relevant to us. Evaluation is an opportunity to reflect on a program as a whole, to understand how it works currently, and to learn from our experiences and the feedback of participants and staff so that we can make future versions of a program even better.

[Emma]

I believe that evaluations can provide integral information for improving programing – especially when youth who are participating in the program are engaged with it directly. I think it is important for evaluations to center the voices of those being served and ensure that the program is meeting their needs. An evaluation may be able to ask participants directly what could be improved or changed to better serve them. These changes can then be shared and implemented. There are definitely challenges to this but there are ways for programs to pivot or adjust based on findings from evaluation.

[Montse]

In previous years, we have found that the turnover of staff + lack of responsibility / ownership of the evaluation aspect made it so that it got forgotten.

I believe that the staff needs to be informed of the importance of the evaluation, why we are doing, why it is important for them to help us answer these questions / run these activities.

Besides, because we constantly have new staff coming in, we are trying to make it part of the onboarding materials and it is also mentioned as part of their responsibilities in their contract.

Having someone responsible of collecting, analyzing, communicating the data and steps for the next cycle is vital.

[Mourer]

By using the Conclusions reached by interpreting the Evaluation of previous iterations of the programs and applying them on newer iterations and then evaluating what those changes did or didn’t do.

[Kathe]

Building equity considerations into all aspects of program design, development, and evaluation is critical to improving our programming and our work with youth!

Our team recently developed and shared a resource on Best Practices for Planning & Facilitating Anti-Oppressive Focus Groups, summarizing each step of the planning and facilitation process and highlighting specific equity considerations.

You may also want to check out a series of guides developed by the W.K Kellogg Foundation on Doing Evaluation in Service of Racial Equity: Debunk Myths, Diagnose Biases and Systems, and Deepen Community Engagement.

What resources support your work to advance equity through program evaluation?

[Lulu]

Given program evaluation is an on-going, dynamic process, it allows us to pause and rework on evaluation methods as we go. For example, a qualitative analysis of a focus group might inform us that there are certain questions that might not have worked as we had anticipated. Consequently, measures can be taken to reformulate/rewrite the evaluation questions for the next rounds of feedback sessions in order to better improve the programming of the given youth program.

[Caroline]

Absolutely, Lulu! And you bring up an excellent point here- evaluating our programs can simultaneously allow us to “evaluate our evaluations”. Are we asking the right questions? Are we using the right methods? Sometimes an unexpected learning curve can happen that enables us to rework evaluation components for the future!

[Kathe]

Using evaluation to ‘improve’ youth programs (rather than just ‘prove’ that they work!) is core to YouthREX’s Framework for Evaluating Youth Wellbeing.

If you haven’t already, you can take our FREE 10-week online certificate course exploring the main concepts, approaches, and practices relevant to conducting evaluation of youth sector programs and initiatives, Program Evaluation for Youth Wellbeing. Registration for our next cohort of learners closes on Wednesday, October 26th, and the certificate will open on October 31st. The best part? All course content is available to you at once so that you can work through the modules at your own pace!

Have you already taken this certificate with us? Great! Maybe you want to strengthen your skills and learn how to manage, analyze, and visualize quantitative data? Even better! Registration for our FREE Using Spreadsheets in Program Evaluation certificate course (which runs online for four weeks, beginning November 21st) doesn’t close until Wednesday, November 16th, so there’s lots of time for you to decide to join us this fall!

Unsure which certificate is right for you? Looking for other resources to support your program evaluation? Get in touch with our Learn team!

[Kathe]

YouthREX recently participated in an online event hosted by the National Youth Council of Ireland, “Rethinking Impact: Legacy and Evaluation in a Global Youth Work Context”, during which we discussed how we can broaden definitions of ‘evaluation’ and move beyond ‘impact’ to thinking about ‘legacy’. You can learn the key takeaways and watch a recording of the presentations. 🙂

[Sophia]

Improving youth programs and evaluation is critical to hearing from the youth themselves. Allowing them to use their voice and have their voice count, be heard and taken into consideration when evaluating programs for youth is very important. Through their lens and views other youth will feel comfortable to share their thoughts and feelings. Having youth evaluate and give input critical to understanding what works and doesn’t work. When youth are involved in planning, evaluation and programming the outcome will hopefully be more positive and this is the true stages of “improving” rather than just “prove”. Collection of voices coming to together expressing their views/ issues is better than being talked to about your issues and views and not being taken into consideration your experience, living through the sitaution.

[Kelly]

In thinking of evaluation, I believe it encompasses a full analytical review of the practices and policies that are existing. If done effectively, with strong valid and reliable findings respective to relatable factors, then there would not only be a ‘proving’ of practice/policy effectiveness but also a restructuring and remodeling of the agency as whole. Ideally, major structural biases or malpractices would be highlighted necessitating potential role changes or changes in program directives from the top of the agency down. Evaluation can prompt major internal reviews that would hopefully result in more meaningful consistent change moving forward.

[Khadijah]

Yes that is true! Although I think a lot of evaluation usually happens ad-hoc – i.e. for a specific program – and is usually activated because of funder needs. But I agree that its impossible to separate out one specific program from the entire organization and evaluate it on its own terms – what’s needed is that whole-system review and a willingness to actually respond to the findings of that review. Have you done a full-scale evaluation of an entire organization – i.e. as either the organization being reviewed or as an evaluator?

[Lulu]

That’s such a great point, Khadijah!