Direct observation is frequently the only way of addressing certain issues, for instance behaviour in a waiting area following refurbishment, but it can be very time consuming. It is a particularly useful method in the early stages of a piece of work where the evaluator knows little about the topic being evaluated. Direct observation with an evaluator’s ‘eye’ (focus on specific aspects) can generate very useful leads about the questions to address using other methods. In social situations it can register what people do, how they interact with things and each other, what they say to one another and how they say it.
Sometimes evaluation requires direct observation and assessment by an independent evaluator. A good example here is the concept of ‘programme integrity’, the assessment of whether a programme has been delivered as designed and according to written documentation. Direct observation by someone who understands the programme, but is impartial to the process of delivery, is a good way of making that assessment, and is why accredited programmes are videoed and checked routinely. This is both for quality assurance and because it can be used in evaluation.
A valuable use of observational methods is to assess behavioural change. We can use a range of instruments to assess attitudes and change, and self-report methods, but these are all subject to problems in relation to the extent to which they reflect actual behaviour. Observing the offending behaviour of offenders who have participated in a programme is, to say the least, unlikely, and we would not advocate trying, but there are certain types of offending that are linked to certain types of behaviour which may be observable in a probation setting. Examples that come to mind are sex offending and domestic violence. There is a body of work that demonstrates some of the behaviours, observable by supervising officers, which can provide a better (more valid) indication of change, and there is scope for further development of this technique. The Prison Service evaluation of the sex offender treatment programme successfully employed observational techniques.
One of the main criticisms of observation as a technique is that people alter their behaviour when they know they are being observed. There are two possible ways to overcome this, both of which have drawbacks. The first technique is not to tell people that they are being observed: so called ‘covert’ observation. There are practical limits to the circumstances in which it is possible to observe people without their knowledge, the most obvious possibility is the use of video recording. Even where it is practically possible to undertake covert observation, there are significant ethical dilemmas involved as it conflicts with the principle of autonomy – that participants have the right to refuse to participate.
The other way often used to overcome so called ‘observer effect’ is to create a situation in which people forget that they are being observed. The easiest way to do this is where the observer has a legitimate role in the proceedings, called ‘participant observation’. The biggest difficulty for the observer in these circumstances is one of remaining a detached observer. However, it can be undertaken very successfully. A classic example is Views from the Boys (Parker 1974), where the researcher spent time with ‘down town adolescents’, the better to understand their lives.
There can be a fine line between the participant observer and the reflective practitioner: a service delivery staff member who purposely reflects on practice with individuals to develop more effective practice. Delivery of a quality service requires reflection on practice to enable improvement, however, and thus the same ethical issues will not arise. Focused observation can be a useful evaluation tool for the staff and can realistically be built into evaluation. For instance, West Yorkshire Probation Service developed a simple yet effective technique whereby the group leaders observed and assessed offence accounts made during the course of a groupwork programme: once during the early sessions and again at the end.
There is a range of techniques which can be employed to improve the focus of the observer and to record the events. The reader wishing to learn more about observational approaches should read Robson (1993), chapter 8, which covers them in some detail.