Clarifying objectives

This stage is crucial to the success of the evaluation. The literature review undertakenin Workstream 3 of this project identified a range of reasons for developing work with offenders, any or all of which could be the focus for an evaluation.
Clarity at this stage improves the utility of the evaluation. This clarity can be attained by the evaluator and the user addressing a series of questions about the nature of the evaluation.
- Who is the evaluation for?
- What mechanism is in place for receiving the evaluation and acting upon it?
- What sort of decisions will be based on the outcome of the evaluation?
- What sort of information will best assist that decision?
- How important is this particular evaluation?
- Are there time constraints on the evaluation?
- What resources are available to conduct it?
Another set of questions concerns the practice that is being evaluated.

- What is already known about this practice?
- What is the theory that underpins this practice?
- What are the objectives of the practice?
- What do we want to know from this evaluation?
This is the stage at which to clarify what is known about the area of practice: some knowledge of the relevant literature can guide the evaluation into fruitful areas.
Clear programme and practice aims are important here, whether they are about work with an individual offender or the content of a groupwork programme for specifically targeted offenders.
Theories that underpin the practice, both explicit and implicit, should be explored. Often the theories of why a particular way of working will be effective have not been articulated clearly, many of them being based on practitioner ‘theories in action’. The evaluator has an important role to play here in asking the questions ‘why?’ and ‘how?’ to aid the articulation of those theories so that they can be assessed in the evaluation. If there are preconceptions about the practice it is important that these are brought to the fore so that the evaluation can be designed to enable these preconceptions to be disproved, if invalid. It is well established that research and evaluation cannot prove a theory, no matter how often affirmative results are produced. It is much easier to disprove a theory. It is also important to remember that any one piece of evaluation is unlikely to provide ‘the’ answer or the ultimate proof. A more realistic aim is to enlighten and add to the knowledge that currently exists.
Clearly identifying the objectives of the evaluation is the most critical stage of the evaluation process – the stage from which the rest of the process flows. It should not be skimped. This is the stage at which, even for small-scale evaluation, time spent in clarifying ideas will reap rewards in the process and outcome of the evaluation.

- It is important to understand the context (literature, theory) of the evaluation and the practice being evaluated.
- It is valuable to specify in writing the evaluation purpose and questions, even if only for private future reference.
- It is crucial that the aims of the programme or practice being evaluated are clearly spelled out.
- It is imperative to sit down and work out what the evaluation is all about before trying to decide which method would be the best.
This preliminary work is more formally described as clarifying the conceptual framework for the evaluation. Clarifying the objectives of an evaluation is the stage at which evaluators and the commissioners of evaluation work out

- WHAT they want to know, and what they really do not know
- WHY they want to know it, e.g. how the information would be used if it were available
- WHO they want to tell, and to what end
- WHEN they need to results available to inform decision making.
Careful thought to these questions will help to answer the questions HOW, and WHERE to do the evaluation, aspects which are covered in the methodological sections of this site.

Who? Why? When?

The Table here presents a framework to consider the who? the why? and the when? of evaluation in relation to three probation examples: a strategic evaluation, a more operational one and an individual offender.
This is not to suggest that evaluation at one level is not of interest or value to the audience at another level, rather it is to say that the primary interest of the audiences will vary.

What to evaluate?