Where’s the Research in Evaluation?

The GCRF is a strange beast—it’s both a conventional research fund focussed on ‘excellent science’ and a distinctly unconventional development fund, part of the UK Government’s ODA commitment and helping to achieve the UK AID Strategy.

it would be fair to say that this dual focus sometimes creates tension. Some people in the AID sector think the money should be spent on traditional development; some in the university sector think that the very best new thinking is in any case a ‘global good’.

I think it’s possible to have both. (I hesitate to say ‘have our cake and eat it’, since Boris Johnson applied this platitude to the UK’s departure from the European Union!).

In my work at UK Research and Innovation, helping to manage the GCRF, I’ve seen lots of examples of brilliant science combined with brilliant development—an Arts-led project on community responses to historical droughts in sub-Saharan Africa that has fed into the increased resilience in the present day; one on the cultural contexts of disease prevention in Southeast Asia that has improved health outcomes; or a study of the seismic safety of Kathmandu that has produced new knowledge about building structures in earthquakes and also created new protocols for the protection of heritage.

But as a researcher on the ‘Changing the Story’ project, I still feel the tension.

In the South African strand of the project, University of Pretoria co-investigators Professor Chaya Herman and Dr Charity Meki-Kombe have done fantastic work with the South African Holocaust and Genocide Foundation (SAHGF) to evaluate its efforts to develop leadership values amongst young people through ‘Change-Makers’. And their contribution to rolling out the programme across other African countries has been amazing too.

But is it research? What are the research questions—and what new insights can we gain?

Rolling out the Change Makers programme in Nigeria

Professor Herman posed these questions at our October project meeting in London, and there was a lively discussion with a range of views. And I’ve been reflecting on them ever since. What can the Changing the Story project add to new knowledge, in addition to evaluating and improving development practice? How can academia and development agencies work together and learn from each other?

For me, there are different ‘sizes’ of question that our practical work with heritage organisations striving to develop a human rights culture can help us address. First, and ‘biggest’, the way the SAHGF teaches about the Nazi genocide to help young people in South Africa, Rwanda, Nigeria and elsewhere think about their own traumatic histories and present-day challenges is a classic example of what Michael Rothberg has called ‘multidirectional memory’. A textured reading of the context, politics and practices of the SAHGF has persuaded me, however, that Rothberg needs to be heavily caveated. Heritage institutions respond to incentives and pressures from many stakeholders with different ‘investments’ in the histories being told, and rarely is the ‘multi-directional’ interaction of the different pasts entirely straightforward. Different parts of each story are emphasised in different contexts, in relation to cultural specificities and political realities. This means decisions are made about what to reveal and what to occlude, and therefore about editing rights and power. And that’s even before we begin to think about the inequalities of language and resource that create implicit or even explicit hierarchies.

Second, there is a set of research questions around ‘participatory arts’ in development. This is ‘smaller’ only in the sense that it is not a ‘meta’ enquiry in to the use (and abuse) of history but rather more tightly focussed on practice. Beyond the obvious questions—e.g. how is participation sustained? Can it be scaled up?—there are other research priorities. For example, what is participation in any given context? Is it the model that conforms to democratic (western…) values or a version that works with the grain of local cultures, even if this implies a trade-off with, say, gender equality or freedom of expression? Are there ‘universal norms’ of participation that everyone would recognise as such (and as positive)? How is power negotiated at the institutional, organisational and ‘event’ level, and how do language and the body relate to one another in different forms of participation? Do arts-based interventions incentivise social change or simply contain disaffection?

Third, while evaluation of a development programme aims to improve performance and outcomes, it can also be used for knowledge building if it can generate explanation why and how a programme works, under which conditions, how it is enacted in different contexts.

Rolling out the Change Makers programme in Nigeria

Of course, both the bigger and smaller research questions—and I realise that this distinction works less well than I had hoped!—should ideally loop directly back into development.

Perhaps this is the key intellectual challenge that working on a GCRF project poses. How can we mobilise the methods of Arts and Humanities research to address the urgent questions around culture, power and sustainability that—as we well know—define both development and development practice, and how can we extract from our focus on the local and the specific the kinds of ‘global understandings’ required to make a real difference at scale?

Stuart Taberner, Director of International and Interdisciplinary Research at UK Research and Innovation, and CI on the South Africa strand of Changing the Story

Chaya Herman, Professor at the Department of Education Management and Policy Studies at the University of Pretoria, and CI on the South Africa strand of Changing the Story