In this Year of Evaluation (2015) the Tavistock Institute of Human Relations (TIHR) is celebrating thirty-five years of evaluation activities.
A personal account of the emergence of evaluation at TIHR during this period by Principal Researcher / Consultant Frances Abraham.
What follows is a personal account of the emergence of evaluation at TIHR during this period, recalling my time in in evaluation research from 1983 to 2004. Reflective practice, participative action research, realist approaches, organisation and systems learning through discursive practices and using complexity theory have all influenced evaluation research at TIHR and continue to do so, in what is still an emergent field. During this Year of Evaluation, other accounts of evaluation will focus on different aspects of evaluation theory and practice over the coming year.
Evaluation research began at TIHR in 1980, with the evaluation of the Manpower Services’ Commission’s ‘Open Tech Programme’, led by Elliot Stern and Linden Hilgendorf. Both Elliot and Linden were already long-established staff members at TIHR and brought a wide range of knowledge and skills with them, including action research in organisational settings and more standard applied research methods, quantitative as well as qualitative. A stream of programme evaluations followed through the 1980s that led to the establishment in 1990 of the Evaluation Development and Review Unit at the Institute (EDRU).
Two theoretical and practice influences especially helped to shape the early approach to evaluation. First, there was TIHR general commitment to a systems perspective, involving working with all interests and levels of a system for change. Second, there was an embedded action research approach to working with clients in an iterative way, to engage them in learning from action and the consequences of action, refining ultimate objectives through joint reflection.
My own evaluation activities began in 1985 with Linden Hilgendorf, growing out of previous collaborative activity in action research with Elliot Stern and applied research with Linden. In 1986 Dione Hills and Liz Sommerlad joined TIHR, bringing a broader set of perspectives from research in community development and community education. Early evaluation research adapted TIHR’s action research methodologies to a stakeholder evaluation model. Many evaluations provided formative or developmental evaluation for stakeholders, sometimes alongside more summative evaluations for sponsors. The proposals for the four year 1986 Evaluations of the Self-Help Alliance and Evaluation of Demonstration Districts for Informal Carers explicitly included the following construction: that we would be finding out ‘what works for whom under what conditions’. Probably Ray Pawson and Ian Tilley were already explicating the realist evaluation position while we at TIHR were evolving our practice. These four year projects for the Department of Health, and associated follow-on projects in social care for DOH and others, provided support for the mixed model of formative/ developmental evaluation for stakeholder groups participating.
In the meantime the market for evaluation was also evolving, most notably in Europe, but also in the UK with explorations into information and communication technologies providing fertile ground for developing understanding of this emergent phenomenon throughout the 1990s. Evaluation studies for the ESRC of PICT (Programme of Information and Communication Technologies), and for HEFCE’S Joint Information Systems Committee (JISC) of their Electronic Libraries Programme would provide opportunities for developing our thinking and also our practice in the UK setting, building on evaluations undertaken in the EU context.
Elliot Stern as Director of EDRU had already begun to win work with the European Commission, which was considerably strengthened by the arrival of John Kelleher’s collaborative entrepreneurialism. Elliot and John’s activities in developing TIHR’s evaluation approach at the European level was also substantially supported by the recruitment by Elliot of Joe Cullen, bringing a more explicitly social science theory based approach to TIHR’s evaluation offer. A stay in EDRU by Carlos Frade foregrounded the way learning is socially constructed, in the tradition of Habermas and his followers, notably Lave and Wenger, but also including a wide range of other learning theorists. (These kinds of theoretical developments were also being partly reflected in the wider evaluation community following the publication of Guba and Lincoln’s ‘Fourth Generation Evaluation’, which sought to locate evaluation practice within a constructivist, rather than a positivist, scientific framework.)
These theories were applied by Elliot and John with Joe Cullen’s methodological contribution in a stream of ground-breaking evaluations focusing on learning around information and communication technologies and education, training and learning programmes. The work of Lave and Wenger in applying ideas of identity formation and negotiation involved in social learning were eagerly taken up by others in EDRU, including Liz Sommerlad and Camilla Child, together with colleagues in Europe, such as Dominique Danau at the erstwhile European Centre for Work and Society, among many others. Work-based learning initiatives, later involving also Jill Turbin and Kari Hadjivassiliou, intersected with an explosion of ICT applications, continued throughout the 1990s, requiring EDRU members to significantly expand theoretical perspectives to understand the iterative discursive patterns through which change was adopted.
The parallel development of Organisational Learning as a school in the organisational theory literature supported this new way of thinking. While in terms of implementation, the European Commission was a rather abstract client whose needs were difficult to address successfully, it did provide a space for this new approach to evaluation to emerge, alongside the realist approach, which was often practised in parallel. In European evaluations during the 1990s it was often possible to accommodate the emergent in evaluation approaches through designing in opportunities for ‘concertation’ and recalibration between participants, providing for continual mutual adjustment and reshaping of outcomes.
From the early 1990s, TIHR’s organisational consultancy staff were beginning to take into account the implications of complexity theory, following collaboration between Eric Miller and Ralph Stacey at the School for Complexity Management at the University of Hertfordshire (the impact of which is outlined here). This influence was strengthened by the arrival of Eliat Aram from the Hertfordshire Complexity School in 2005 and, since 2010, TIHR’s second Director. The idea of change occurring through the shadow organisation is especially well-adapted to the evaluation of social programme sponsored by third parties, such as UK Government departments and the EU, providing opportunities for local organizations to adapt to the emergent in relation to the communities and markets they serve as well as changes in technology. While theory-based evaluation strategies, rather than more experimental/ counterfactual models, including ‘theory of change’ approaches are being increasingly embraced by evaluators and evaluation clients, the applications of complexity theory within evaluation markets is only just beginning to be addressed, with a current initiative by the ESRC to establish a centre for the evaluation of complexity, particularly as applied to work in the environmental/energy sector.
We can foresee that the application of complexity thinking to evaluation will be difficult for some sponsors to accept, with its challenge to the strong desire for certainty that putting money into a particular activity will inevitably lead to specified outputs and final outcomes. On the other hand, externally sponsored programmes across a wide range of institutions, organisations and actors of different kinds, can provide the conditions for change happening through the shadow organisation that Stacey sees as essential to the local interaction which can achieve it. With its emphasis on discursive practices and conversational routines, the ideas embodied in complexity theory provide a satisfying understanding of the way change occurs and fits well with the evaluation practices at TIHR which have been developed over the past thirty-five years and are being elaborated in different ways today by a new set of practitioners.
Find out more about Frances Abraham