By Ben Jaques-Leslie
Most evaluators work within a context — a sector, a funder type, a geography. Over time, that context starts to feel like the natural shape of the work — the way data gets collected, the questions that get asked, the standards that count as rigorous.
I have had the chance to work across a number of those contexts: state and federal government, academia, domestic nonprofits, and international development. That movement has been clarifying in unexpected ways. Most notably because
the differences reveal assumptions you didn’t know you were making.
These are observations, not research findings. I have not conducted a systematic study of domestic versus international evaluation practice. But some patterns have been striking enough to be worth sharing.
The first thing I have noticed is what does not change. Methods, for one. Quasi-experimental designs, matching approaches, difference-in-differences. These work the same way whether you are evaluating a food security program in Nepal or a workforce development initiative in a mid-sized American city. The underlying logic of causal inference does not care about geography. That has been a useful reminder: investing in methodological knowledge is worthwhile precisely because it is applicable across contexts.
The other constant is relational. Across every setting I have worked in, the quality of the work has depended on relationships with two groups in particular: the implementing partners or grantees who are closest to the program, and the participants whose lives the program is meant to affect. Their knowledge matters; as Philip Smith has recently shared, without this, our own biases go unchecked. Their perspectives also shape what questions are worth asking. And their rights — to privacy, to honest representation, to findings that are actually useful to them — are not optional considerations.
That relational orientation does not change with the funder or the distance. It is a commitment that fundamentally shapes the quality and integrity of the work, and one that I’ve relied on in each setting I’ve worked.
The more unexpected observation has to do with where rigor tends to show up. International development programs, at least the ones I have worked on, tend to carry substantial evaluation infrastructure. Structured surveys administered to hundreds of participants. Validated measurement tools for food security, dietary diversity, and women’s empowerment. Comparison groups, sampling frameworks, and formal reporting requirements. While working on a large-scale nutrition programming initiative across Nepal, Yemen, Guatemala, and Uganda, I saw the investment in measurement visible at every stage: in the depth of the survey instruments, in the volume of participant data collected, in the expectations around what an evaluation report needs to demonstrate.
My expectation, before working in this space, was that programs operating in low-resource, difficult-to-access settings would face the most pressure to cut corners on evidence. The reality has often been the opposite.
Domestic programs sometimes seem to have less appetite for formal evaluation. Surveys can be harder to get approved. Maybe the requirements placed on grantees are lighter or the infrastructure for measurement may be thinner.
I have a hypothesis about why: when funders are geographically and relationally close to the programs they support, the political cost of rigorous evaluation can feel higher. There are more relationships at stake if findings are unflattering. That proximity, which should in theory make accountability easier, can sometimes make it more complicated.
To be fair to both sides: domestic programs still evaluate, just less formally on average, and international programs are not paragons of rigor. Both vary enormously. And international development has its own accountability gaps — strong on measurement, sometimes weaker on whether any of it actually changes decisions.
The gap I notice most consistently across both contexts isn't data collection, reporting, or analysis — all three of those tend to happen. It's a deliberate, designed-in learning agenda that's usually missing. A set of questions that a program genuinely wants to understand, with a plan for how evidence will be used to answer them and a commitment to acting on what is found.
A learning agenda is not a wish list. It is a set of prioritized questions scoped to a realistic timeframe. Not everything a program might want to know, but what it most needs to understand in the next two years, given what decisions are actually on the table. That constraint is part of what makes it useful. Unbounded curiosity does not produce learning; focused questions do.
Evaluations answer some of those questions. Literature reviews can surface what is already known. Case studies can illuminate how something is working in practice. And data that already exists can often be repurposed to answer learning questions without any new data collection at all. This includes the administrative, implementation, and operational data that programs and governments collect in the course of doing their work, but also external sources like census data, market data, or other datasets collected for entirely different purposes. None of it was gathered to answer a research question; the insight comes from asking a new question of it. This is different from primary data collection, like surveys, which are designed specifically for evaluation or research. A learning agenda creates the frame; the choice of method follows from the question, not the other way around.
And what gets learned can change more than program content. It can change how a program is managed, how it operates, and how it is positioned strategically. A finding about who a program is actually reaching might not change the design at all; it might change who joins the next funding conversation. A finding about which components are hardest to implement might reshape staffing or sequencing before it ever touches the logic model.
That breadth is part of what makes learning agendas worth building. They connect evidence to decisions at multiple levels, not just to the next evaluation report.
Learning agendas do exist — in the federal government, they are formal, published, and taken seriously. I have worked on studies commissioned against one — a specific question, asked deliberately, answered with real data, briefed to leadership. Other questions on the same agenda were being worked in parallel by other people. What has never happened, in my experience, was the step after that: someone standing back to ask what the accumulated answers amounted to, and what should change as a result. The questions got answered. The agenda itself was never revisited.
I've seen the same thing happen with far less structure around it, in other settings. That is the version of this gap I see most consistently, and it is worth distinguishing from the others. Collecting data is not the same as learning from it. Producing a report is not the same as using it. And answering a question is not the same as anything changing because of it. That last step only happens when someone is positioned to act on what comes back, and when the answers are read together rather than one at a time. The investment that both domestic and international contexts tend to make in the former does not automatically produce the latter.
Illustrative: Ben Jaques-Leslie moved between sectors and geographies expecting different gaps in each. He found the same one, everywhere.
I am not arguing that everyone should move between sectors; there is real value in deep contextual knowledge built over time in one area. But the assumptions that any single context installs are worth examining. The standards that feel self-evidently rigorous in one setting may look quite less so from another angle. Practices that feel like unnecessary burdens in one context may turn out to be genuinely valuable. And the gaps — in learning, in participant voice, in honest use of evidence — tend to look the same regardless of where you are standing.
That, more than anything, is what moving across contexts has taught me: the problems are more similar than they appear, and the solutions are more portable than most people assume.
Ben Jaques-Leslie is part of the Apricity team. This piece draws on his own experience across multiple sectors and reflects his own views and conclusions and does not necessarily represent the views of Apricity.
