Effective practice: the past, present and future of probation research

An article by Peter Raynor.

From early optimism to ‘nothing works’

Since researchers first began looking seriously at probation and its effects, the pendulum has swung between optimism and pessimism. This article aims to track and explain these movements, and to argue that we now know enough to enter a new period of realism, in which the question is not so much whether probation can be effective (we know it can), but how, with what support and in what policy contexts that effectiveness can be made real.

The origins of probation itself go back to the nineteenth century, but probation as we know it, as a public service provided within the criminal justice system, is largely a product of the twentieth century, and research on the impact of probation began in the 1950s. At that time, criminologists made largely positive statements: Manuel Lopez-Rey, the head of the United Nations Social Defence Section, wrote ‘If I were asked which, among the modern methods for the treatment of offenders is the most promising, without hesitation I would say: Probation’ (Lopez-Rey 1957). For the criminologist Max Grünhut (1952) the essential elements of probation were ‘conditional suspension of punishment, and personal care and supervision by a court welfare officer’.

During the 1960s probation, like other forms of social work, expanded in Europe, the United States and many other countries, alongside the general increase in state-provided welfare services. Its effectiveness was largely taken for granted, and this was still more or less the situation when I joined the Probation Service in England in 1970. However, within a very few years this era of optimism came to an end: serious research on both sides of the Atlantic began to raise questions about whether probation was doing any good at all. The major American review reported (and somewhat exaggerated) by Robert Martinson in 1974 gave us ‘very little reason to hope that we have in fact found a sure way of reducing recidivism through rehabilitation’, and a series of studies in the UK contributed to similarly pessimistic conclusions. For example, the UK Government’s ‘IMPACT’ study published in 1976 showed that probationers receiving intensive services on smaller caseloads did no better (actually slightly, but not significantly worse) than those supervised on normal caseloads. These and similar studies came to be summed up as ‘nothing works’, which remained the widespread orthodox view until the 1990s. It was, of course, popular with some politicians who were looking for reasons to reduce public expenditure.

Using social science to evaluate probation

At this point we need to think about social science and evaluation methods. Social-scientific service evaluation depends largely on three processes, which can be described as understanding, measurement and comparison. In our field, we need to understand what people are doing and how they are trying to do it; we need to measure effects, and crucially we need to compare those effects with what happens to other similar people receiving different services or inputs, or none. Early studies of probation, such as that by Leon Radzinowicz in 1958, reached optimistic conclusions by measuring outcomes but making no relevant comparisons with the results of other sentences. If such comparisons were included probation did not do so well, and first offenders actually reoffended more on probation than if they were fined. The ‘nothing works’ researchers knew about the need for measurement and relevant comparisons, but did not adequately understand or describe the work actually done by probation officers. They left it as what Jim Bonta in Canada has called the ‘black box’ of supervision. Measuring outputs without understanding inputs leaves open the possibility that there is a mixture of good and bad practice, which means that any good effects from the good practice are likely to be cancelled out by the bad practice, so that researchers will find no overall positive impact – and this is what they found.  Detailed study of what practitioners were actually doing, and of the results of different practices, did not become generally available in criminal justice until the 1990s and they led in due course to a new era of optimism and to attempts in many countries to implement ‘what works’.

From programmes to skills and implementation

The foundations of this new optimism were provided by meta-analysis, largely in Canada and the USA, of the methods used in a variety of programmes for offenders (the best known of these is probably that published by Don Andrews and colleagues in 1990). These researchers summarise the characteristics of effective practice as being based on risk, need and responsivity or RNR: briefly, we should aim to provide more intensive services to higher-risk offenders where they will make more difference; we need to address the needs and deficits in people’s lives and resources which make them likely to offend; and we need to do this in a way which fits with offenders’ own motivations, life goals and ways of learning. These broad principles, and a number of others, have stood up well in research since then. In practice, the most immediate effect was the development and implementation of group programmes in which practitioners were guided by manuals, which allowed researchers for the first time to have a clear picture of intended inputs; however, many programmes turned out less effective in large-scale practice than they were in pilots (see Raynor and Robinson 2009). Reasons for this included insensitive and over-directive management of change, and lack of attention to the skills of practitioners, which were still not well understood.

Since the first decade of this century we have started to fill this gap. Studies of practitioner skills in Canada, Australia, the USA and Britain have analysed officers’ contacts with probationers and found that the more skilled practitioners produce consistently lower rates of re-offending, and that officers can be trained to improve skills. In addition, studies of the management and implementation of ‘what works’ in criminal justice agencies have informed developments like the Correctional Programs Assessment Inventory, which can help to assess whether practice is being managed and supported in the right way. Overall, we can now confidently say that probation is effective if you employ the right people, train them in the right skills, carry on supporting them and training them, and then let them get on with their work without constant interference.  (Why did it take us so long to learn this?)

‘Post-truth’ and after

For the future, three problems stand out: we still do not make enough use of what ex-offenders themselves can tell us about how they stopped (‘desistance’); we still need to understand better exactly how officers’ contacts with offenders can help them see themselves and their lives differently; and we need to pay more attention to communication with policy-makers. For example, recent decisions which have had a very bad effect on probation in Britain were made without any evidence at all, and supported by false claims, in the political style we have come to call ‘post-truth’. We need to continue to insist that evidence is fundamental to effective practice.

For further reading see McNeill, Raynor and Trotter (2010) Offender Supervision, Routledge, and Ugwudike, Raynor and Annison (forthcoming) Evidence-Based Skills in Criminal Justice, Policy Press, to be published Autumn 2017.

Click here for the presentation from Peter Raynor.