Designing your evaluation

Clarifying the objectives of the evaluation is important. Data collection and analysis does not occur in a vacuum – there is always a reason for undertaking such work. It is important to make aims explicit before setting out on the exercise, to ensure that the right sort of information is collected for the analysis. An evaluation does not have to be a grand, all encompassing exercise. It is something that can be undertaken in small stages, collecting data to provide insights and raise questions that can be addressed in another stage.
- Evaluation does not have to be a grand scale exercise, it can be low key.
- There are a variety of ways in which the topic of interest can be illuminated and evaluated.
- Existing sources of statistical information can frequently throw light on a topic – always check out what already exists before going off to collect more.
- The question initially addressed is likely to change in the early stages.
Once the aims and objectives of the evaluation are clear, it is possible to think about the sort of design that would best meet these objectives. Without clear aims and objectives it is likely that the proverbial hammer could be chosen to crack a proverbial nut, with the consequent waste of resources. It is at this design stage that methods are chosen that are appropriate to the questions and feasible for the resources and timescale available.

The notion of design covers a range of aspects: the overall approach (e.g. experimental, quasi-experimental, case study), the methods of data collection (e.g. interview, questionnaire/form, observation), sample sizes and procedures, means of processing data and analysing it. All aspects of design are informed by the need for methodological rigour, particularly reliability and validity – two concepts central to the accuracy of the evaluation.
This is the stage at which decisions are made about what sort of evaluation design and methods are

- most appropriate to address the objectives of the evaluation, and
- feasible within the timescale and resources available for the work
This is inevitably a question of compromise, and is a process that can often be eased by looking to what already exists, in terms of design and instruments as well as evidence. Using tried and tested instruments rather than designing from scratch can reduce the time and resources needed for a project as well as creating data that is comparable with other studies.
The three stages of Clarifying Objectives, Types of evaluation design, and Sampling are addressed on separate pages.
Evaluation guide