Evaluation Methods and Evidence: from Measurement to Learning
Illustrative: Community conversations provide essential local perspectives that strengthen evidence, learning, and decision-making.
By Apricity Team
Across the aid and development sector, organizations are being asked to do more with less. Budget cuts, shifting donor priorities, and growing pressure to demonstrate value have forced difficult decisions about where resources are invested and how success is defined.
The instinct, understandably, has been to show impact — quickly, clearly, and with numbers. But showing results is not the same as learning from them.
Numbers can give the appearance of understanding without the substance of it. A program can hit its targets and still fall short for the communities it serves. It can produce clean data and still leave decision-makers no clearer about what to do next. As resources shrink and the stakes of making the wrong call rise, organizations need more than evidence that a program worked. They need evidence that helps them understand why it worked, for whom, under what conditions, and what should happen next.
Why rethinking evaluation design matters
Every evaluation encounters complexity — diverse communities, varied contexts, and evolving needs. Programs unfold differently across locations, engage and reach different people, and produce outcomes shaped by factors that no logframe fully anticipates.
And yet, while the realities that evaluations must navigate are broad, our approaches to them remain narrow. Standardized methods, fixed indicators, and predetermined questions are applied to realities that are anything but standard. The result is a persistent gap: between what evaluations measure and what actually matters; between the evidence that gets generated and the decisions this is meant to inform.
This is not a failure of intent. Most evaluators want to produce evidence that is useful. But good intentions are not enough when the methods we reach for do not fit the realities in front of us. The monitoring and evaluation field is grappling with this openly. Debates about adaptive management, participatory approaches, and the limits of logframes are not new — but they have taken on new urgency as resources shrink and the stakes of getting it wrong rise. At the same time, the tools available to evaluators have expanded: mixed-method designs, quasi-experimental approaches, and community-centered frameworks. The question is not whether better evaluation is possible. It is whether we are willing to do it.
Three assumptions that evaluations must overcome
If evaluation is to support learning rather than simply report results, it requires challenging some of the assumptions that continue to shape how evidence is generated and used.
The first assumption: contextual conditions are obstacles to good evaluation design. When programs are complex, resources are limited, or baselines are absent, the evaluator’s job is simply to work around these constraints as best they can. But context does not only limit what is possible — it actively shapes what is appropriate. Designing an evaluation means reading the realities in front of you and fitting methods to them, not the other way around. It also means staying open: when a design does not work for the conditions at hand, the response is to adapt, not to press on regardless.
The second assumption: headline figures tell the most important story. Once the data has been collected, interpretation can appear straightforward. But data without context is just noise. The same outcome in two different locations can mean two entirely different things. Understanding why patterns and inconsistencies emerge requires looking beyond averages and asking about how a program is experienced across different groups, places and circumstances.
The third: communities are recipients of evaluation, rather than participants in it. Their role is often limited to answering survey questions, rather than helping to shape what questions get asked. This assumption is both methodologically limiting and ethically problematic. The people closest to a program often understand it best. Leaving them out of the process is not neutral. It actively distorts the evidence base.
From measurement to learning
These assumptions are not abstract methodological debates. They shape the quality of the evidence organizations rely on to make decisions.
Drawing on Apricity’s recent work in Nepal and elsewhere, this blog series explores what happens when evaluators challenge these assumptions and approach evaluation as a process of learning rather than measurement alone.
We begin with design. Ben Jaques-Leslie examines what happens when evaluators resist the pull of familiar methods and instead ask a more important question: what is the most appropriate design this context actually allows? He argues that good evaluation design requires creativity, honesty about limitations, and a willingness to adapt when circumstances change.
Next, we turn to analysis. Sarah Staub and Ben Jaques-Leslie explore how meaningful insights often emerge not from averages but from the differences that averages conceal. Treating variation as a signal rather than noise shifts from a reporting exercise into a genuine learning process.
In our final article in this series, Sarah Staub argues that community perspectives are not supplementary to the evidence base — they are part of it. When program participants, implementers, and other stakeholders are engaged throughout the evaluation process, the result is not just more inclusive. It is more useful.
Measurement is the beginning, not the end
Each article focuses on a different stage of the evaluation process, but they share a common premise:
the purpose of evaluation is not simply to generate evidence. It is to generate evidence that helps organizations learn, adapt, and make better decisions.
In an environment where organizations are expected to demonstrate greater impact with fewer resources, evaluation cannot afford to be an exercise in measurement alone. Learning requires appropriate methods, thoughtful analysis, and a genuine commitment to understanding how programs are experienced by the people they are meant to serve.
That is the thread connecting the articles in this series. And it is why conversations about evaluation methods matter now more than ever.
New pieces in this series will be published every two weeks.
This series is written collaboratively by members of the Apricity team. Pieces throughout this series draw on experience gained through that work but reflect their own views and conclusions and do not necessarily represent the views of Apricity.
