A hand holds a young coffee plant in a nursery bag, with green leaves and other seedlings in the background.
A hand holds a young coffee plant in a nursery bag, with green leaves and other seedlings in the background.

Measuring what Matters

How Does That Work, Kirsten and Katharina?

Our Monitoring, Evaluation and Learning Manager Team answers the most important questions on the HRNS MEL system.

How HRNS uses Monitoring, Evaluation and Learning across its work with farming families, young people and nature

Hanns R. Neumann Stiftung (HRNS) uses Monitoring, Evaluation and Learning (MEL) to understand whether projects are creating meaningful change for smallholder coffee farming families, the environment, and youth in the communities. It allows us to track progress, reflect on results, improve project implementation and share evidence with partners and co-funders. In this interview, Kirsten Ehrich and Katharina Lima de Miranda, both Monitoring, Evaluation & Learning Managers at HRNS, explain how HRNS collects and uses data, why learning is central to impact measurement, and how new approaches are shaping the future of MEL at HRNS.

Why is impact measurement important for HRNS?

Kirsten: For HRNS, impact measurement is about responsibility, learning and better project steering. We want to understand how resources are used, what has been achieved and how our projects can become stronger over time. Monitoring, Evaluation and Learning (MEL) supports accountability towards partners and co-funders, but it also helps us make better decisions during implementation.

MEL gives us a structured way to ask whether we are on the right track. Are activities being implemented as planned? Are farmers and farming families reached? Are practices changing? Do we see the intended effects on farms, families, and nature? And what can we learn from the results? This is important because our work takes place in complex rural contexts. Good data helps us move from assumptions to evidence.

What does HRNS mean by impact in smallholder farming, youth programs or on nature & environment?

Katharina: For us, impact starts with a simple distinction: implementing activities is not the same as creating change. A training, for example, is an important activity. The number of farmers or youth trained tells us something about reach. But the more important question is whether trained participants apply what they learned, whether this changes farm management and entrepreneurial activities, and whether these changes contribute to better income, food security or resilience.

This is why we look at the whole pathway from project activities to their direct results (outputs), changes in practices or behavior (outcomes), and longer-term effects on livelihoods (impact). We do not only want to know how much we did. We want to understand what changed, for whom and why. That also means looking carefully at the assumptions behind our projects. If we expect that training leads to improved practices, and that improved practices lead to better livelihoods, better perspectives for youth, improved soils and water reservoirs, and a more intact ecosystem, we need data to test whether this logic actually holds.

Which changes in smallholder livelihoods, youth, and nature does HRNS measure?

Kirsten: We measure different dimensions of change. These include household income, food security, dietary diversity, coffee productivity, adoption of good agricultural practices, climate-smart practices and record keeping. For nature we track soil health, water holding capacity, ecosystem integrity, and in some projects even monitor indicator species via wildlife cameras. We also look at farmer organizations, women's participation, youth participation and the role of young people in developing income-generating opportunities.

This reflects the way HRNS works. Smallholder livelihoods, perspectives for rural youth and environmental protection are complex challenges. Farming productivity matters, but it is not the whole story. A farming family's resilience also depends on climate adaptation, protection of natural resources, access to knowledge, market opportunities, family decision-making, strong farmer organizations and the perspectives of women and youth. Our monitoring and evaluation system needs to capture this broader picture.

How does HRNS collect and analyze project data?

Katharina: We combine participant surveys, project monitoring data and qualitative methods. Participant surveys help us measure changes in areas such as entrepreneurial activity, income, yields, food security or adoption of practices. Monitoring data helps us track project activities, participation and reach. Qualitative methods, such as focus group discussions and interviews, help us understand why something works or why it does not.

Digital tools play an important role in this process. They help us collect data more systematically, check data quality and make results available for analysis and visualization. But data collection is only the first step. The real value comes when data is cleaned, analyzed, discussed with teams and used to improve project implementation.

Where HRNS supports other organizations through grants, we accept our grantees’ standard annual reports rather than prescribing a fixed MEL framework. Their projects are diverse, often innovative and rooted in local priorities, so we respect their own approaches to documenting progress and learning. This also keeps reporting proportionate and allows partners to focus their time and resources on their work with communities.

How does HRNS ensure the quality of project data?

Kirsten: Reliable project data is essential for good decisions. This starts with clear indicators and well-designed data collection tools and appropriate sampling. Who is included in a survey, and who is not reached, affects how we interpret the findings and to whom they apply. We aim to balance methodological rigor with the practical challenges of collecting data in different project contexts. It also means training the teams who collect data, checking for inconsistencies and creating routines for validation and follow-up.

We work closely with country teams because they understand the project context and can help interpret the data. Numbers alone are not enough. If something looks surprising in a dataset, the first step is often to ask the team: What is behind this? Was there a data quality issue, an implementation challenge, or a real change in the field? This dialogue is essential.

How does HRNS assess its contribution to change?

Katharina: We want our evidence to be credible, but we also need to work with the realities of project implementation. In rural development, impact is not always easy to prove. Farmers, youth and farming communities are influenced by many factors beyond a project, such as climate events, market prices, household dynamics and other actors working in the same region.

This is why we choose our evaluation approaches carefully and are cautious with our claims. Sometimes an external evaluation with a comparison group is appropriate. In other cases, measurements at the start and end of a project (baseline and endline), combined with monitoring data and qualitative reflection, may be the more realistic option. The aim is not to overstate results, but to build a credible picture of what changed, what role HRNS plausibly played and what we can learn from it.

How does HRNS use evaluation findings to improve projects?

Kirsten: Evaluation findings only create value when they are used. This is why we are strengthening learning formats across HRNS. We want evaluation results to feed into conversations with project teams, country offices, and partners. The key questions are: What do we learn from this? What should we continue? What should we adapt? And what should we do differently?

This is also important for fundraising and proposal development. Evidence helps us show what has worked, but it also helps us be honest about challenges and improve future approaches. Learning from evaluations should not be a one-time exercise at the end of a project. It should be part of how we design, manage, and adapt our work.

How is MEL evolving at HRNS?

Katharina: Monitoring, Reporting and Verification (MRV), supported by remote sensing, is becoming increasingly important, especially in relation to carbon sequestration, regenerative agriculture and agroforestry. Remote sensing can help us track changes for example in tree cover, open ground or vegetation health. As HRNS objectives combine farming family livelihoods with protecting nature and environment, we are increasingly looking at environmental effects alongside social and economic change.

We are also exploring how AI-supported analysis can help us work more efficiently, for example through data quality checks, identifying patterns in large datasets, summarizing qualitative feedback or supporting dashboards. These tools can support, but not replace, professional judgment. Data protection, contextual understanding and critical review remain essential.

What is your ambition for MEL at HRNS?

Kirsten:I would like MEL to be seen as a practical support for better project work, not only as a reporting requirement. It should help teams understand what works, where adjustments are needed and how we can design stronger projects.

Looking ahead, we also want to make MEL simpler and more consistent across projects. Together with country teams, we are defining shared priority indicators, calculation methods, survey questions and consistent sampling strategies, with local adaptations where needed and additional indicators where co-funders or innovative activities require them. We are also exploring opportunities to coordinate data collection with partners and use shared datasets where appropriate, reducing duplicate requests and the time required from participants, project teams and partners. This is intended to reduce preparation time and helps us track change across projects and over time—freeing up time to interpret findings with teams, young people and farmers, share lessons and adapt our projects.

Katharina: For me, the ambition is that MEL becomes a shared learning practice across HRNS. Good data should not sit in reports. It should help us ask better questions, make better decisions and contribute to more resilient smallholder farming communities, improve living conditions, expand opportunities for young people and protect nature.

Muddy Boots

Start reading our Muddy Boots Newsletter now.

Subscribe