[Keesha]

A major challenge faced by organizations and youth programs is a lack of funder engagement wherein, “funders do not spend
enough time at programs and therefore do not have a good understanding of the context and realities of the
youth sector”. Funders understand and evaluate a program primarily based on reports, which often seek to offer highlights and successful program results from a limited perspective. Furthermore, funders primarily engage with management and do not have a well-rounded understanding of the impact of work being done by front line staff, as well as the indirect impact on participants and lack an understanding of the realities when it comes to youth programming.

Programs should seek to engage funders in attending programming events or round tables, offering them the opportunity meet staff and participants and gain an understanding of how a program is truly successful or where it is lacking in and outside of the program goals. It is a way for both an organization and funder to engage in the reality of youth programming, and how both can seek to improve and lead participants to success.

[jiannaquickstad]

We are currently struggling with the challenge of ensuring that youth are being engaged meaningfully in the evaluation. There are areas where I think we are achieving this, however, a specific area where we are struggling is with forming an advisory committee. It is very difficult to make decisions regarding which youth/how many youth to have on the advisory committee. Would we love to include everyone? Of course. Is that feasible? Unfortunately, no. So that is our current grapple, trying to ensure that the youth who are engaged feel valued and like their contributions are important and meaningful even if they are not on the advisory committee.

[jiannaquickstad]

A few ways we could deal with this: 1) ask for volunteers (and hope everyone doesn’t volunteer?) 2) include youth who have been involved the longest, or 3) have them nominate each other. Open to other suggestions if anyone has any! Thanks!

[Gwen]

The challenges around capacity to analyse qualitative data are a significant challenge my organization is facing currently. We have all these stories, quotes, advice pieces for future participants, art, pictures, lists and lists of skills youth learned – but we don’t do anything with that data. We’re collecting it, and storing it, but we have no way to analyze it, or understand what it means. We have the information, but we don’t report on it or share it, because that capacity for really understanding how to analyze and interpret qualitative data is missing.

It is something we’re working on, we have a solid Knowledge Mobilization and Impact Measurement team now, and they’re working on building their own capacity for measuring and understanding qualitative data, and then will help the rest of the staff at our org build their capacities too – but it’s very much still a work in progress!!

Part of the issue too is that, because so much of our funding comes from the federal government, the reports we generate for funders are almost entirely focused on quantitative data. They don’t care about the actual impact of the program, they just want to know how many participants were reached, how many session hours were held, etc. Funders only care about the numbers, and we need that funding to keep coming in, so until very recently it was hard to justify growing our evaluation skill-sets to include qualitative data.

[April]

These challenges can be addressed by removing some of the barriers by offering more face-to-face communication, focus groups, and social inclusion. More focus needs to be placed on the possibilities that evaluation could offer youths for strategic learning instead of focusing on the outcomes.

[Agnes]

For youth programs, measuring outcomes can be challenging due to factors such as participant mobility, changing life circumstances, and the time and resources required to conduct follow up evaluations. To address this challenge, youth programs can consider implementing strategies such as tracking participants over time through follow-up surveys or interviews.

By overcoming these challenges, youth programs can better understand and communicate the long-term impact of their work on participants.

[Cameo]

Figure 11 shows many challenges doing evaluation that grassroots and mainstream organizations run into. In this graph you can see that 67% of respondents of grassroots orgs cited limited staff time as a challenge and 35% reported limited data collecting capacity. I relate to these both. I have a lot of facets to my job that I must manage and incorporating evaluation throughout the program is an additional time dedication that is difficult to balance. It’s important and my team does commit to completing ongoing reviews, however, the capacity struggle is real.

A possible solution to this could be having a set time block to review participant feedback and discuss ways to incorporate their suggestions. This “focus time” could be done collectively as a team, or individually before discussing together, depending on conflicting schedules and availability. Another strategy that comes to mind is breaking down large tasks relating to program evaluation and setting deadlines for yourself and your team to meet your goals. One more idea could be to invite additional stakeholders into the evaluation process– with more folks helping, the workload can be spread across more hands to lighten everyone’s load and incorporate a diverse array of perspectives into your evaluation.

[Lulu]

“While
many youth-serving organizations involve youth in data-collection as respondents”

– While I agree with this statement, it is actually quite difficult to drum up survey responses. For instance, our survey response rates have been quite low (even with incentives offered), which often times will not allow us to run meaningful statistical analyses due to the small sample size. I think one of the reasons might be that youth don’t fully understand the “importance” of program evaluation yet. This takes us to the second part of the statement “far fewer organizations involve youth in decision-making processes that shape
evaluation activities because of barriers such as supervisory requirements and
potential liabilities”. Maybe there’s a less “drastic” way to get youth involved in evaluation (e.g., workshops on eval methods) in order for them to better understand the meaning and the goals of this type of program.

[Marco]

Finding meaningful ways to involve youth in evaluation can be challenging given their other life commitments, such as work, school, personal life, extracurriculars, etc. Given that there is so much to gain by youth being meaningfully involved in evaluation activities, organizations need to prioritize making youth involvement as accessible and barrier-free as possible. Some ways to respond to this challenge is by scheduling meetings, focus groups, and sessions based on a schedule that works for them. Other ways include providing transportation reimbursements and honorariums. The evaluation activities that they are involved in need to be engaging, meaningful, and provide them with opportunities to build community and relationships with their peers whilst developing their skills and leadership abilities through the training opportunities provided and active participation in the evaluation process.

[Tara-Lynneet]

“Youth-sector organizations need and deserve a more fulsome model of evaluation, one that meets them where they are and builds on their strengths while recognizing their importance.”

I agree with this statement because evaluation is all about data and numbers opposed to the actual “person” behind the number or stat.