Measuring what Matters
How Does That Work, Kirsten and Katharina?
Our Monitoring, Evaluation and Learning Manager Team answers the most important questions on the HRNS M&E system.
How HRNS uses Monitoring, Evaluation and Learning to improve smallholder projects
Hanns R. Neumann Stiftung (HRNS) uses Monitoring, Evaluation and Learning (MEL) to understand whether projects are creating meaningful change for smallholder coffee farming families. It allows us to track progress, reflect on results, improve project implementation and share evidence with partners and donors. In this interview, Kirsten Ehrich and Katharina Lima de Miranda, both Monitoring, Evaluation & Learning Managers at HRNS, explain how HRNS collects and uses data, why learning is central to impact measurement, and how new approaches are shaping the future of MEL at HRNS.
Why is impact measurement important for HRNS?
Kirsten: For HRNS, impact measurement is about responsibility, learning and better project steering. We want to understand how resources are used, what has been achieved and how our projects can become stronger over time. Monitoring, Evaluation and Learning (MEL) supports accountability towards partners and donors, but it also helps us make better decisions during implementation.
MEL gives us a structured way to ask whether we are on the right track. Are activities being implemented as planned? Are farmers and farming families reached? Are practices changing? And what can we learn from the results? This is important because our work takes place in complex rural contexts. Good data helps us move from assumptions to evidence.
What does HRNS mean by impact in smallholder farming?
Katharina: For us, impact starts with a simple distinction: implementing activities is not the same as creating change. A training, for example, is an important activity. The number of farmers trained tells us something about reach. But the more important question is whether farmers apply what they learned, whether this changes farm management, and whether these changes contribute to better income, food security or resilience.
This is why we look at the whole pathway from project activities to their direct results (outputs), changes in practices or behavior (outcomes), and longer-term effects on livelihoods (impact). We do not only want to know how much we did. We want to understand what changed, for whom and why. That also means looking carefully at the assumptions behind our projects. If we expect that training leads to improved practices, and that improved practices lead to better livelihoods, we need data to test whether this logic actually holds.
Which changes in smallholder livelihoods does HRNS measure?
Kirsten: We measure different dimensions of change. These include household income, food security, dietary diversity, coffee productivity, adoption of good agricultural practices, climate-smart practices and record keeping. We also look at farmer organizations, women's participation, youth participation and the role of young people in developing income-generating opportunities.
This reflects the way HRNS works. Smallholder livelihoods are complex. Productivity matters, but it is not the whole story. A farming family's resilience also depends on climate adaptation, access to knowledge, market opportunities, family decision-making, strong farmer organizations and the perspectives of women and youth. Our monitoring and evaluation system needs to capture this broader picture.
How does HRNS collect and analyze project data?
Katharina: We combine farmer surveys, project monitoring data and qualitative methods. Farmer surveys help us measure changes in areas such as income, yields, food security or adoption of practices. Monitoring data helps us track project activities, participation and reach. Qualitative methods, such as focus group discussions and interviews, help us understand why something works or why it does not.
Digital tools play an important role in this process. They help us collect data more systematically, check data quality and make results available for analysis and visualization. But data collection is only the first step. The real value comes when data is cleaned, analyzed, discussed with teams and used to improve project implementation.
How does HRNS ensure the quality of project data?
Kirsten: Reliable project data is essential for good decisions. This starts with clear indicators and well-designed data collection tools. It also means training the teams who collect data, checking for inconsistencies and creating routines for validation and follow-up.
We work closely with country teams because they understand the project context and can help interpret the data. Numbers alone are not enough. If something looks surprising in a dataset, the first step is often to ask the team: What is behind this? Was there a data quality issue, an implementation challenge, or a real change in the field? This dialogue is essential.
How does HRNS assess its contribution to change?
Katharina: We want our evidence to be credible, but we also need to work with the realities of project implementation. In rural development, impact is not always easy to prove. Farmers and farming communities are influenced by many factors beyond a project, such as climate events, market prices, household dynamics and other actors working in the same region.
This is why we choose our evaluation approaches carefully and are cautious with our claims. Sometimes an external evaluation with a comparison group is appropriate. In other cases, measurements at the start and end of a project (baseline and endline), combined with monitoring data and qualitative reflection, may be the more realistic option. The aim is not to overstate results, but to build a credible picture of what changed, what role HRNS plausibly played and what we can learn from it.
How does HRNS use evaluation findings to improve projects?
Kirsten: Evaluation findings only create value when they are used. This is why we are strengthening learning formats across HRNS. We want evaluation results to feed into conversations with project teams, country offices and partners. The key questions are: What do we learn from this? What should we continue? What should we adapt? And what should we do differently in the next project design?
This is also important for fundraising and proposal development. Evidence helps us show what has worked, but it also helps us be honest about challenges and improve future approaches. Learning from evaluations should not be a one-time exercise at the end of a project. It should be part of how we design, manage and adapt our work.
How is MEL evolving at HRNS?
Katharina: Monitoring, Reporting and Verification (MRV) is becoming increasingly important, especially in relation to carbon sequestration, regenerative agriculture and agroforestry. In practice, this means measuring changes such as carbon stored in trees, reporting the findings and verifying that the data and methods used are reliable. As HRNS works more on climate resilience and nature-based approaches, we need to understand environmental effects alongside social and economic change.
Social Return on Investment (SROI) is another area of growing importance. It can help us discuss the value created by a project in relation to the investment made. But we need to use it carefully. Not everything that matters can be expressed in monetary terms. Empowerment, gender equality, resilience or social capital are highly relevant, but they are complex and often long-term.
We are also exploring how AI-supported analysis can help us work more efficiently. This can include data quality checks, identifying patterns in large datasets, summarizing qualitative feedback or supporting dashboards. But AI does not replace professional judgment. Data protection, contextual understanding and critical review remain essential.
What is your ambition for MEL at HRNS?
Kirsten: I would like MEL to be seen as a practical support for better project work, not only as a reporting requirement. It should help teams understand what works, where adjustments are needed and how we can design stronger projects.
Katharina: For me, the ambition is that MEL becomes a shared learning practice across HRNS. Good data should not sit in reports. It should help us ask better questions, make better decisions and contribute to more resilient smallholder farming communities.
