Evaluations Promote a Culture of Error: Dr. Susanne Johanna Väth

As charities or networks in the field of development cooperation do we allow some space for introspection? And consider our scope, approach and impact! Do we know if we are relevant at all? To get to the bottom of these questions that often pop-up in our field of work, Siemens Stiftung commissioned CEval GmbH to carry out an external evaluation of the empowering people. Network. We spoke to Dr. Susanne Johanna Väth, Senior Project Manager at CEval – Center for Evaluation to understand all that goes behind to yield an effective assessment.

What was the evaluation methodology used for empowering people.Network?

Evaluating the impact of complex networks needs a complex approach. To evaluate the empowering people.Network (epN) we therefore, applied a contribution analysis. In the first step we developped together with Siemens Stiftung a ‘Theory of Change’ (ToC) to understand the mechanisms through which epN aims to create an impact. In the next step we reconstructed a detailled results model with theory-based hypotheses on the links between the activities and their intended impact. This layed the foundation for an analysis grid with empirically measurable indicators. This guided data analysis and structuring of results. We undertook a mixed-methods approach using different data sources, data collection instruments and analysis methods to validate findings. We carried out an online survey among all epN members and conducted expert interviews and a focus group discussion. We selected 8 epN member organizations for in-depth case studies. Overall, more than 150 individuals from different groups like epN member’s management, staff and stakeholders from the ecosystem shared information in the course of this evaluation.

How did you assess that it was best suited for the organization?

We at CEval follow a participatory approach to make sure that our work suits the needs of our clients. At the same time an initial review of available data and documents helps us inquire which evaluation design fits best. As this was the first evaluation of the epN, comprehensive baseline data was not yet available. Thus, an ex-post facto design had to be applied. Given this limitation and the complexity of the evaluation object, a contribution analysis was the best option to derive at plausible conclusions on the effect of the network.

© Evaluation of the empowering people. Network

Can you tell us some surprising/interesting elements that came up during the evaluation?

The evaluation brought forward very positive results. Beneficiaries highly appreciate that the epN accompanies social entrepreneurs over a longer period than other development initiatives. We found evidence that the diversity and regional distribution of the epN members are perceived as unique and allow global exchange and cooperation. It was interesting to find out that Siemens Stiftung contributes considerably to an enhanced credibility of the epN members as they see the membership as a kind of quality label.

With respect to learning, we came to the result that the dissemination of training contents and information services among local staff of the epN members offers potential that could be used even more intensively.

Why do you think it is important for social enterprises, charities, foundations to conduct evaluation?

Beyond transparency and accountability, evaluations are helpful to foster organisational learning. They do not only analyse relevance, effectiveness, efficiency, impact and sustainability of an intervention, but also provide evidence-based recommendations on how to improve the intervention itself and the result-orientation of the client. Evaluations allow assessing performance on clearly defined criteria. Therefore, they are often a precondition to apply for grants and enhance chances when applying for loans. Moreover, evaluations help to promote a culture of error. This means that they contribute to a mindset in which mistakes are not perceived as a personal or institutional failure, but as an opportunity for future development. In this regard, evaluations have the potential to trigger innovations.

The evaluation can be downloaded here.

Dr. Susanne Johanna Väth is a Senior Project Manager at CEval, with ten years of experience in the fields of Impact Evaluation and Development Economics. As a PhD Economist she has proven in-depth knowledge on quantitative and qualitative evaluation designs, data collection and analysis methods. Susanne has delivered successful (meta) evaluations and monitoring systems, providing backstopping services and conducting trainings. Her field research has predominantly been in African countries where she has covered a wide range of thematic areas including rural development, capacity development, education, inclusion, agricultural value chains, climate change adaptation, research cooperation and organisational development. Susanne has worked for international organisations like UNICEF, the World Bank, the European Commission, the German Federal Ministry of Education and Research to name a few.

0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *