When You Can’t See the Program, How Do You Know It’s Working?
By Frank Salet
Most monitoring systems are built on a simple assumption: if you can’t see the program, you can’t measure it.
That assumption no longer holds in many places across development and humanitarian environments today, where conflict, access constraints, and rapidly changing conditions mean that direct observation is limited, risky, or impossible.
At the same time, the tools available to monitor these programs have expanded to include satellite imagery, geospatial analysis, mobile data collection, and community-based reporting, which now make it possible to observe programs without requiring physical access.
The question is no longer whether we can collect data. It is how we choose to use it. Drawing on Apricity’s recent evaluation work in Yemen, this blog series explores how technology-supported approaches are redefining what credible evidence looks like in fragile and conflict-affected contexts.
The Problem Isn’t a Lack of Data
Monitoring fails in fragile contexts not because data is missing, but because it still relies on direct observation rather than technology-enabled, triangulated evidence.
The issue is not the absence of data, but the way credibility is defined. In 2021, the World Bank argued that ‘the value of data for development is largely untapped’ and attributed this not to data scarcity, but instead to how data is governed, shared and used. Most monitoring systems still rely on direct observation as their primary source of evidence. Enumerators visit sites, evaluators verify outputs in person, and confidence comes from what can be seen directly.
When that is no longer possible, other forms of data are treated as secondary. Phone-based surveys are seen as less reliable. Third-party reports are approached cautiously. Satellite imagery is used as supporting material rather than as evidence in its own right. Community feedback is often treated as anecdotal.
As a result, even when multiple data sources exist, they are discounted because they do not fit the standard model of verification.
A Shift Toward Technology-Enabled Monitoring
A different approach requires a different premise. Instead of asking whether something can be directly observed, the focus shifts to whether enough independent signals exist to build confidence in what is happening.
Technology plays a central role in enabling this shift. Satellite imagery can show whether infrastructure exists or has changed over time. Geospatial analysis can map program data against context, such as population distribution, access to services, or environmental conditions. Mobile data collection can capture patterns across locations quickly. Community reporting and third-party monitoring add layers of context that formal systems often miss.
Each of these sources has limitations. But when triangulated, they can produce a more reliable and nuanced picture than any single method alone.
The goal is not to replace field observation, but to move away from treating it as the only credible form of evidence.
Satellite imagery and geospatial mapping are reshaping what credible evidence looks like in fragile and conflict-affected environments. Image: Frank Salet
What This Looks Like in Practice
In recent work in Yemen, direct access to many project sites was not possible due to security constraints. Rather than abandoning evaluation altogether, our approach shifted toward combining multiple technology-enabled data streams.
Host-national teams conducted surveys and site visits where feasible, while mobile contributors collected geo-referenced observations and photographs. Satellite imagery assessed visible infrastructure over time, and geospatial analysis mapped these data against contextual factors such as population distribution and service access. Through this approach, data alignment increased confidence in findings, while discrepancies revealed important gaps. Infrastructure that appeared functional in reports showed signs of deterioration in imagery or community feedback, prompting a more cautious interpretation of results.
Building evidence through technology-supported monitoring and evaluation requires more interpretation and judgment. It also provides a more transparent account of uncertainty and variation in program performance.
Rethinking What Counts as Evidence
In many fragile and hard-to-access environments, direct observation is no longer consistently feasible, and insisting on it as the benchmark can lead to incomplete or misleading conclusions.
Technology does not remove uncertainty. Indirect evidence can be ambiguous, and different data sources may conflict. There is also a risk of overconfidence if technological tools are used without sufficient context.
However, the alternative is not neutral. Monitoring systems that cannot adapt to access constraints risk producing less useful evidence at the very moment when programs need it most.
Why This Matters Now
Expectations for monitoring and evaluation are changing. Donors and implementers are asking not only whether activities were completed, but whether programs are working, where they need to adapt, and how decisions can be informed in real time.
In fragile environments, waiting for perfect data is not an option. Programs continue to operate, and decisions still need to be made.
Technology-enabled monitoring makes it possible to generate credible insights under these conditions, but only when used as part of a wider evidence approach. When combined with contextual knowledge and field-based understanding, these tools allow monitoring systems to identify gaps earlier, support adaptive management, and produce evidence that is useful for decision-making rather than simply fulfilling reporting requirements.
This requires a shift in mindset as much as a shift in tools. Indirect, technology-enabled evidence needs to be treated as a core part of the evidence base, not as a fallback.
What Comes Next
This shift raises a set of practical questions:
How can impact be measured when outcomes cannot be directly observed?
How can maps function as evidence rather than illustration?
What can satellite imagery verify reliably, and where are its limits?
These are not only technical questions. They are the starting points for important conversations about how monitoring and evaluation must work in complex environments.
In the next piece, we move from principle to practice: when direct access is not possible, how do you build evidence that is still credible and defensible? Drawing on recent evaluation work in Yemen, we explore what that looks like in practice.
Frank Salet is part of the Apricity team. This piece draws on experience gained through that work but reflects their own views and conclusions and does not necessarily represent the views of Apricity.
