Each of the various methods of collecting data for evaluation is essentially a systematic way of asking questions to obtain accurate answers that can address your issues. Questions are asked of people or documents, and can be asked by direct contact or indirectly through another means (e.g. paper).
The methods of collecting data are:
questionnaires and forms
(also look at focus groups
- observation, including video recording
- document or content analysis
The process of probation practice routinely generates substantial amounts of data, both written and computerised. Its creation for use in operation and performance measurement can mean that such data is reliable and comprehensive, and much of it could be used in evaluation. Data extracted from information systems about individual cases or reports can form the basis of a database for an evaluation.
Whatever mechanism is chosen for the collection of data for the evaluation, it is important that regular checks are made that the material is being created, and that urgent steps are taken to fill gaps or to change processes that prove to be faulty. For instance, instruments designed to be completed at the last session of a programme will be missing for any offender who does not attend that last session. There must be a mechanism in place to chase missing information.
It is very easy to underestimate the amount of time and effort that is required in the collection and processing of data, particularly in plugging gaps in the data and checking anomalies. It is usually better to design a procedure whereby data is processed and checked routinely throughout the duration of the data collection of the evaluation, rather than waiting until the end when time can be limited and memories fallible.