Illustrative: In fragile settings, no one sees the whole picture. Evaluations are stronger when the people who hold different parts of it design them together.
By Sayra van den Berg and Sarah Staub
Evaluations in fragile settings often begin with a difficult question: how can we understand a place and its people well enough to ask the right questions, especially when access is limited and conditions change quickly? As external evaluators, we bring methodological experience, but also assumptions shaped by the contexts in which we have worked. Local partners bring knowledge of relationships, histories, and everyday realities that may be invisible to outsiders. Partnership is most valuable when both of those perspectives shape the evaluation from the outset.
The outsider problem
No evaluation begins from a neutral vantage point. As evaluators we arrive at the questions we ask, and the contexts we enter, already primed by prior experience. That experience helps us recognize patterns, but it can also lead us to overlook local histories, social relationships and ways of organizing daily life. We cannot reasonably be expected to develop a deep understanding of every social, cultural, and historical context in which we work. In fragile settings, this outsider problem sharpens. There is rarely the time, the access, or the conditions for an evaluation to correct itself as it goes, and the window to ask is often short. When there is no room to learn as you go or to return, asking the right questions, at the right time, and in respectful ways matters even more. That makes contextual knowledge essential before data collection begins.
Partnership is one way to close the distance between what we can see and what those closer to conditions on the ground are able to know. Together, an outsider’s perspective and local expertise complement and enrich each other. Equitable partnerships, where questions and how they are asked are decided jointly, ground evaluations in local knowledge. That makes them more credible in context. This is central to inclusive rigor, which no single evaluator viewpoint perspective is sufficient on its own achieves alone.
Beyond the performance of partnership
The commitment to practicing, rather than performing, partnership is easy to state and harder to keep. A partnership can all too easily be performed when partner organizations are named in proposals but handed a finished evaluation design to administer, with their involvement beginning only after the decisions that matter have already been made. In these cases, partnership risks becoming a box to check rather than an opportunity for meaningful collaboration.
Partnership is practiced sincerely when it begins early enough to shape the questions, the sampling, and who holds the clipboard.
Local partners hold knowledge that desk research and brief field visits cannot supply. Practicing partnership means recognizing that expertise and creating opportunities to jointly question assumptions, challenge proposed approaches, and meaningfully influence methodological decisions.
The importance of early partnership
In a recent nutrition program evaluation in a Ugandan refugee settlement, we partnered with a refugee-led organization from the start to carry out a mixed-methods evaluation. When we developed the evaluation survey instruments together, our own outsider lens led us, at times, to frame questions that were unintentionally insensitive to the realities of life in displacement. The evaluation had a particular interest in impacts on families,which meant the survey had to reflect what a family looks like in displacement, rather than an assumption carried in from outside. Adjusting our questions to ask how many children a respondent was responsible for rather than how many children a respondent had was one way in which we were able to adjust to the complexities of what family, care, and responsibility look like in contexts of fragility. Without our partner’s guidance and correction, we would have built the design around a narrow and inappropriate idea of what counts as a family.
In another evaluation, of a nutrition program in rural Nepal, we worked with a partner who brought gender expertise and established relationships in the areas where the evaluation took place. That partnership made it possible to work with female enumerators in a setting where gendered gaps in education would have otherwise made this unlikely, so that women were interviewed by women. Because the evaluation findings relied on women’s own experiences, this essential dimension of gender-sensitive evaluation design shaped whether respondents could speak openly.
When partnership does not mean shared power
Early involvement alone does not guarantee that partners can challenge an evaluation’s design. Funding, institutional authority, and control over deliverables can create power imbalances even when everyone intends to collaborate. If local colleagues do not feel able to disagree, an evaluation can lose precisely the insight the partnership was meant to provide.
During her doctoral research in rural Ghana, one of us saw how involvement can fall short of influence. Visiting researchers studying women’s work and household income had engaged local staff to refine interview questions and introduce participants. Yet their questions assumed that the previous day represented a typical workday and that women independently decided how to spend their earnings. In that community, market days, farming, religious observances, and social events shaped what each day of the week looked like, and decisions about income could also be negotiated within households, often with men having more influence over financial decisions. A one-day snapshot risked misrepresenting both work and financial decision-making.
The visiting team paid the local organization for facilitation and accommodation. We cannot know whether this financial relationship kept local staff from challenging the design, but it raised a question: did the partnership create enough room for disagreement? Once these realities were raised, the researchers revised their approach, at a cost in time and effort. Asking local partners how they would investigate the question before presenting a near-final method might have surfaced those assumptions earlier.
Trust is not simply access
Partnerships also shape whether people feel safe enough to speak. In separate doctoral research on a sensitive topic in Benin, trusted community and national partners helped establish introductions, but trust did not transfer by default. Participants chose the time and place of interviews. The researcher explained who she was and why she was asking, invited questions, and discussed anonymity before and after conversations. These practices did not remove risk or power differences, but they helped participants make informed choices about speaking. Furthermore, in fragile settings, we must also consider the risks faced by local partners, who may continue to face the consequences of their involvement long after external evaluators leave.
We try to hold ourselves to the same standard. That means leaving room for disagreement, and for changing the design as we learn, with the clients who commission us, with the organizations we evaluate, with our local partners, and with the communities whose experiences we depend on.
Partnerships are not methodological footnotes. They shape how we design and deliver evaluations. They influence which assumptions are challenged, which questions are asked, and which findings we can credibly reach. In fragile settings, a question worded inappropriately, asked insensitively, or timed poorly can be irreparable. Because there may be little chance to correct mistakes, sharing decision-making power is part of methodological rigor.
If partnership decides whose knowledge shapes an evaluation, the next question is what counts as success. Our next piece explores situated measurement and why definitions of success must reflect the realities of the places where programs operate.
Sayra van den Berg and Sarah Staub are part of the Apricity team. This piece draws on experience gained through their work at Apricity but reflects their own views and conclusions.
