Reporting and Sharing
This stage of the evaluation process is often neglected, frequently not being considered in any detail until after the data have been analysed. While reporting is generally considered to be an end point to an evaluation, it is often helpful to build some aspect of reporting into earlier stages. This needs to be considered carefully and balanced against the need for the practice to be ‘uncontaminated’ by the evaluation, but interim reporting can be a valuable means of maintaining participant motivation for the evaluation process. Some evaluation designs, for example process evaluation or formative action research, have regular reporting and feedback as an integral part of the evaluation process.
Academic researchers are required to publish their work in books and journals, but in-service evaluation is quite different in this respect. There is a specific audience awaiting the results and it is important that these results are presented in a way that is useful to that audience. This may require a different format to the traditional ‘academic’ report, such as a simple written summary report or an ‘in person’ presentation to a group. This need for results to be presented in an immediate and accessible way is a strength of the evaluative approach, but at the same time means that the results of an exercise are likely to reach a relatively small number of people.
This is particularly true within the probation services, where much useful evaluation about probation practice is not published outside the local service in which it occurs. This point was made by Chapman and Hough (1998 p.107) and is attested by the wealth of material provided for the survey of evaluations undertaken for this publication, which had not previously been widely available. NPRIE provides a forum for some sharing of this material, frequently on an informal basis, though more and more services are now making copies of evaluation reports available to other services through the ACOP Bulletin. In conjunction with the publication of this handbook, the three main probation service libraries have now agreed to establish and maintain a collection of evaluation reports (see 3.6.3). However, there is a need for evaluators of probation practice to consider other routes for wider publication, such as practice journals, and a need for probation managers to encourage and facilitate evaluators to do this.
At the same time, it is important to consider issues of ‘ownership’ of the material, and the rights of different parties to see and possibly require changes before wider publication. Where the evaluator is externally contracted to undertake evaluation these issues should be included in the terms of the contract. Where the evaluator is an employee of the organisation for which the evaluation is undertaken these issues should be negotiated during the planning stages of the evaluation, and agreed in writing. In some circumstances contracts of employment will include reference to publication. See Chapman and Hough (1998 p. 106) on the right to publish.
Another important consideration in relation to publication is commitments made to participants in the early stages of the work. Where participants were promised copies of a report, or the right to comment on drafts before publication, these should be budgeted for in both time and cost, and promises kept. Where assurances of confidentiality have been given reports should be double-checked to ensure that individuals cannot be identified from the material presented.
The reporting stage is the point at which all the work on the evaluation comes together and the evaluator’s reflections, understanding and interpretation of the results are presented to others. Just as with all other stages of the evaluation process, this one too should be guided by consideration of the original questions and purposes of the evaluation. The purpose of reporting is to explain the work to others, present your interpretation of the findings, and appropriately locate them within the framework of existing evidence and policy.