Frequently Asked Questions
Here are a few frequently asked questions that address common concerns, questions, or objections that clients may have before directly contacting us.
What is program evaluation?
In short, program evaluation is the examination of a program’s implementation and/or effectiveness. Program evaluation can take many forms, it can go by many names, and it isn’t restricted to assessing just “programs”—it can be used for analyzing strategies, initiatives, policies, and the like.
Program evaluation is often subdivided into formative
evaluation and summative evaluation:
-
Formative evaluation (sometimes referred to as process evaluation) entails periodic reporting on a program’s implementation and interim progress towards meeting goals and objectives. This type of evaluation is a powerful tool for engaging in continuous improvement.
-
Summative evaluation (sometimes referred to as outcome evaluation) entails reporting on the cumulative results of a program over a longer period, usually at the conclusion of the program itself or at the end of its funding cycle. When the design of a summative evaluation enables establishing causality/causation, it is characterized as an impact evaluation. That is, the evaluation is able to measure the program’s causal impact on participants, ruling out other explanations for the results.
Why does my program need to be evaluated?
Evaluation will help you determine to what extent your program is meeting its objectives and goals, how your program can be improved, what’s working and not working, and how to better serve your clients.
What is the difference between research and evaluation?
Research involves gathering and analyzing data to study a general phenomenon of interest. Research can be separate from or a precursor to evaluation. Once a phenomenon has been researched, a program can be implemented based on best practices revealed in that research and/or to address an issue uncovered in the research study.
When should I bring in an evaluator?
It’s best to bring in an evaluator when you decide to implement a program to address an identified need or issue. An evaluator will help you effectively design the program, including identify the outputs that will result from your program’s various activities and how those outputs connect to the program’s short-term, intermediate, and long-term outcomes.
What does the typical program evaluation process look like?
When Data Works evaluates a program, we meet with the program director and other relevant staff to get a thorough understanding of the program, its goals and objectives, and its reporting requirements. We work with program staff to create a logic model if one does not already exist. We then create, in collaboration with program staff, an evaluation plan that delineates all evaluation activities and their associated deliverables, along with a comprehensive timeline for data collection and reporting. Once the agreed-upon evaluation plan is in place, we follow it as closely as possible, with the understanding that revisions to the plan may be necessary along the way, depending upon the timing of data availability, changes in reporting requirements, shifts in program priorities, and the like. We maintain close contact with program personnel to ensure that our efforts align with their needs throughout. Our goal is to produce information that is timely, relevant, actionable, and meets all funding compliance requirements. We typically formulate our program evaluations to include ongoing gathering of data, ongoing reporting of program progress toward meeting goals and objectives, as well as a final study of program impact.
Our approach to program evaluation is highly flexible, personalized, and collaborative. We design each evaluation to fit the particularities of the program content and context. This means working within grant stipulations, giving program staff the information and tools they need to monitor program implementation and progress, and providing grantors and decision-makers with information about the program's effectiveness. Moreover, we design our evaluations around a feedback loop that ensures a two-way flow of information among evaluators, program personnel, grantors, and relevant decision-makers. We actively encourage stakeholder participation throughout the evaluation lifecycle, soliciting input and feedback on evaluation design, data collection and analysis, interpretation of results, reporting, and any other major activities associated with the evaluation.
How long do program evaluations typically take?
It depends. If an evaluator is with you from the beginning and assists with the program design, the evaluation can occur throughout the life of the program. It will include formative evaluations at set periods throughout the year to inform staff of the program’s implementation fidelity and progress towards objectives, as well as summative evaluations to provide information on the program’s progress towards meeting its goals.
Are program evaluations expensive?
For grant-funded programs, an evaluation is usually required. Consequently, a small percentage of the grant’s budget funds are typically allocated for external evaluation. If you are considering an evaluation for a program that’s not funded by a grant, you should certainly consider allotting funds for this activity. It is more expensive not to conduct an evaluation because you may be implementing a program that doesn’t work or performing costly activities that aren’t connected to or yielding outcomes that contribute to your ultimate goal. Additionally, program evaluations are often instrumental in helping organizations garner additional funding by demonstrating the impact the program is making, as well as highlighting the additional needs of the clients or community you are serving.
I have difficulty understanding data and statistics. How will your services benefit me?
We specialize in communicating information in a way that anyone can understand. We will train your staff and ensure they are comfortable with and fully understand their program information. We will work to empower your organization with the knowledge and tools it needs to analyze its impact.
.png)