What to evaluate
New initiatives, special programmes and new ways of working are the prime targets for evaluation because we need to be clear that the new ways are more effective than the old. However, there is also a need to evaluate existing practice with the same rigour. Such evaluations do not need to be on a large scale, and there is great value in small-scale evaluations that have clear objectives and appropriate methods. For instance, individuals who work with offenders evaluating their own one-to-one work can highlight important issues. It is often possible to make small modifications to existing data collection or reporting processes to provide the basis for more systematic evaluation of working methods.
The reduction of offending is generally the ultimate aim of work with offenders, but this is usually addressed by focussing on one or more aspects or issues that are related to offending, such as substance abuse or antisocial attitudes. In order to better understand the relationship between such factors and changes in offending behaviour it is important that the immediate objective(s) of such work is assessed. These are often known as intermediate outcomes. Without knowledge of the achievement of the programme objectives it will not be possible to explain how a change in offending has been achieved.
Evaluation can be undertaken about any aspect of practice, and can be done at an individual case level as well as with groups and in relation to special programmes. For effective practice to generate knowledge it is important that such evaluation is informed by ethical and methodological rigour. The key is to know the ‘what’ – the precise aspect of practice being evaluated, and the ‘why’ – the theoretical reasoning that explains why it is expected to be effective.
It is also important to consider the evaluation of evaluation methods and instruments – often referred to as validation. Much evaluation of work with offenders has been undertaken in the Western world, but little of it has been replicated in other contexts, though there is an increase in ways of working crossing geographical boundaries. There is considerable scope for re-testing previously used methods and instruments, as well as developing new ones. When taking a way of working from one context to another, such as from one country to another, it is important to evaluate the work in the new context to ensure that it retains its effectiveness, such as the transfer of the SEED model from England to Romania as described in workstream 2 of the STREAM project.