Using qualitative and evaluative performance information well

Our general observations

We saw good examples of public organisations measuring their progress on impacts and outcomes in their annual reports. Some public organisations presented meaningful quantitative measures in the context of informative trend information.

Other public organisations used case studies to evaluate and give insight into their outcomes and impacts. Qualitative information was most useful where quantitative information didn't have much relevance because of the nature of the work.

It is important for public organisations to explain clearly and succinctly why they selected the type of information used and the processes for ensuring that it’s robust. Combining and providing a mix of quantitative and qualitative performance information can often provide a rich account of performance. 

Transport Accident Investigation Commission 

The Transport Accident Investigation Commission's 2020/21 Annual Report uses case studies as an effective way of explaining, assessing, and presenting how it has performed, particularly in areas where the factors influencing an outcome are complex and there are no relevant or appropriate quantitative forms of measurement.

What we liked:

  • Case studies used to demonstrate impact.
  • Case studies have a clear and logical structure.
  • Clear description under each section.
  • Clear explanation of why case studies are the most appropriate for assessing performance, e.g. challenges in direct measurement, complexities of the transport system, and time to see changes in outcomes. (See page 20 of the annual report.)

Image from the Transport Accident Investigation Commission’s 2020/21 Annual Report showing a case study that describes an inquiry into engine abnormalities in Trent engines. It described what the Commission did and the difference it made.

Transport Accident Investigation Commission 2021-2025 statement of intent

The Transport Accident Investigation Commission uses case studies as an effective way to demonstrate performance. The Commission is also intending to improve how it assesses its performance by supplementing its use of case studies with an appropriate mix of quantitative measures of performance to provide a wider view.

What we liked:

  • In its statement of intent, the Commission is looking to improve its performance reporting framework and use of case studies by adding quantitative measures.

Image from the Transport Accident Investigation Commission 2021-2025 statement of intent that describes how it will improve its performance reporting framework and use of case studies by adding quantitative measures.

Ministry of Justice

The Ministry of Justice’s 2020/21 Annual Report includes a good example of evaluating and reporting on the impact of information campaigns.

What we liked:

  • Reporting on the evaluation of a key information programme, including against important characteristics.

Image from the Ministry’s of Justice’s 2020/21 Annual Report that describes how the Ministry of Justice commissioned an independent survey to assess the impact of its information programme.