During a webinar that I recently hosted a participant asked if it was better to have an internal or external evaluator to conduct an evaluation plan.
This is a common question among nonprofit leaders, whether their program is new or established because at some point, they need to evaluate your program, whether it be for your team’s internal work or because a funder requires them to do so.
The short answer to the question:
It depends. 🙂
While there’s no hard and fast answer, here are three initial questions that will help you know whether you should hire an external evaluator.
- What are your evaluation questions? (What are the things that your stakeholders need and want to know about your program?)
- How long is the evaluation planned to take? (A few months, a year, more than a year?)
- Does your staff have the time and the necessary skills (i.e., the capacity) to perform the evaluation? (Given your evaluation questions and the length of the evaluation, is your current team able to take on the work?)
Once you have the answers to those questions, you’ll have a good sense of whether you can conduct the evaluation in house or if you will need support.
Let’s see how this applies to a nonprofit who is evaluating Queens, an after school program for Black high school girls that is in its pilot phase. The program manager (who has some experience with conducting program evaluations) knows that the success of Queens is contingent on 1) ensuring they’ve reached their target audience and 2) students are satisfied with the program. If they don’t get nail those two things, the program won’t work as intended. So, the program manager crafted a few evaluation questions:
- How many participants were present at the first session?
- How did participants hear about the program?
- How many participants completed the entire three-month program?
- How satisfied were the participants with the program?
In this scenario, the team decides that they don’t need an external evaluator and the program manager will also serve as the internal evaluator. Here are their reasons for that decision:
- The evaluation questions that the program manager has crafted can be answered using their existing systems (attendance records, pre and post surveys, etc.)
- The evaluation will be done over the course of three months and can be easily incorporated into the program manager’s role and responsibilities.
- The program manager has familiarity with conducting an evaluation and knows how to use data to interpret the results and make strategic decisions.
Ok, now let’s fast forward a few years and Queens is still going strong… thanks in large part to the decision to incorporate evaluation from the beginning. 😉
They’re ready to learn about their short-term program outcomes so they can understand what has changed in the lives of program participants. The original program manager is still in her role and has crafted the following evaluation questions:
- To what extent did our program change participants’ knowledge, attitude, or skills?
- Have participants benefitted from their participation in the program? If so, how?
- Did certain participants benefit more than others?
This time around, the Queens team decides to hire an external evaluator because:
- While their program manager has experience with evaluation, she doesn’t have the necessary technical skills to determine appropriate indicators and tools that will give them the data they need to answer their evaluation questions.
- The team estimates that the evaluation will take about a year to complete and no one has the capacity to fully take on the evaluation work, given their other responsibilities.
Now comes the fun part, which is finding the right evaluator. There are a host of questions that you should ask when hiring an evaluator. One of them should be:
Are we seeking someone who will add or build evaluation capacity?
In the case of adding capacity, the evaluator will do the evaluation for you. While they work with you to make sure everyone is aligned on the evaluation questions and methods and share the results of the evaluation, the evaluator does all of the actual evaluation work. On the other hand, building your team’s evaluation capacity means that the evaluator works with you throughout the evaluation process giving you the knowledge and skills to apply evaluation methods and approaches to your work, now and in the future.
One thing to keep in mind is that whether you have an internal or external evaluator, your team will need to be involved in the evaluation from start to finish. The evaluation cannot and should not be handed off completely to the evaluator. The entire purpose of an evaluation is to help your organization seek and use data to continuously assess and improve your work. And that only happens if all stakeholders are involved throughout the process.
Before you can perform an evaluation, you need to have a program logic model. I advocate for logic models so much because they serve many purposes, one of which is to help you create an evaluation plan. In the absence of a logic model, creating an evaluation plan is 10x harder because you don’t have the benefit of a roadmap that shows the outcomes and impact you planned for.
If you need help creating or modifying an existing program logic model, I encourage you to enroll in the Set Your Nonprofit Program Up for Success In Two Hours: A Logic Model Masterclass!