Part 3: Elements of the performance story

3.1
In this Part, we describe the six aspects of the performance story that we focus on during our audits, why we consider these aspects important, and what our expectations are.

3.2
When assessing public entities' forecast performance information, we focus on those aspects of the performance story that we consider are central to providing the reader with a clear understanding of the public entity's purpose, the goods and services it delivers, and whether it is successful.

3.3
The six aspects that we pay particular attention to reflect the guidance that we have issued to public entities. The six aspects are:

• strategic context;
• specifying and presenting the outcomes framework;
• main measures and targets for impacts and outcomes;
• linking impacts and outcomes to outputs;
• specifying outputs and output classes; and
• performance measures and targets for outputs.

Strategic context

3.4
To help the reader understand the forecast performance information reported by a public entity, it is important that it sets out a clear overview of the strategic context within which it operates. This should include:

• the nature and scope of the public entity's functions; and
• its operating environment, the challenges this presents, and its immediate priorities.

3.5
The nature and scope of a public entity's functions should explain the reason for the public entity's existence, what it exists to achieve, and the duties, powers, and functions that it can use. This information will remain relatively fixed over time, because it is fundamental to the public entity's strategic positioning and reflects its governing legislation and statutory requirements.

3.6
Information that describes the dynamic context in which a public entity is operating is needed to complement the more stable information on the nature and scope of its functions. This should include information about changes to the operating environment, such as the economic outlook, immediate priorities (including those set by the Government), and any other major challenges or risks. It is important that the public entity describes clearly for the reader how it expects these issues to affect it and how it is planning to respond.

3.7
Presenting information on the public entity's strategic context should allow the reader to understand why the public entity is seeking to contribute to the impacts and outcomes it has identified, and its priorities.

Specifying and presenting the outcomes framework

3.8
An SOI must clearly identify the impacts, outcomes, or objectives that the public entity aims to contribute to or influence. The strategic context should have already made it clear to the reader why the reported impacts and outcomes are relevant to the public entity. Outputs explain how the public entity intends to intervene to help bring about the outcomes.

3.9
Outcome statements must describe the main desired effects that the public entity is seeking to achieve. An outcome statement should refer to the state or condition of society, the community, economy, or environment. It should also include the desired change in that state or condition.⁶

3.10
Every public entity is part of wider public sector efforts. It can be helpful for public entities to refer to the work of other contributing public entities in achieving outcomes. This is a useful way for the reader to see how a particular public entity's work relates to the work done by other parts of the public sector.

3.11
In presenting their outcomes frameworks, public entities need to be clear about the point at which they can reasonably aim to have an influence on whether an outcome is achieved. It is at this point that we consider that main measures should be identified. Including higher-level or sector-wide outcomes can add useful context for the reader. However, trying to demonstrate the contribution made to more remote outcomes is difficult, can increase the cost of data collection, and is of questionable value to the reader.

3.12
The number of layers in a public entity's performance story will vary depending on the nature of its business, the outputs it delivers, and the outcomes it has identified. It is important that there are enough layers for the reader to understand how providing a particular good or service ultimately contributes to achieving a specified outcome. Too many layers can mean that measurable outcomes should be set at a level closer to the activities of the public entity. A gap in the explanation can indicate that an intermediate step is needed.

Main measures and targets for impacts and outcomes

3.13
Main measures and targets for impacts and/or outcomes are essential for the reader to know whether a public entity is making progress. There should be few of these measures, enabling readers to see the "big picture" of the public entity's strategic intent without being overwhelmed by detail. Main measures should be relevant to the impact or outcome being assessed and not just focus on matters that are easy to measure.

3.14
Impact and outcome targets must cover at least the full period of the SOI (a minimum of three years). Where a change in outcomes occurs only after a long period, it may be more appropriate to identify main measures closer to the point of service delivery, where change will be easier to see during the period of the SOI. Longer-term changes in outcomes can then be provided as contextual information.

3.15
Impacts and outcomes are about what a public entity is trying to influence rather than what it can control. Therefore, it may be appropriate to express targets as a general direction of change, such as an increase against a baseline, rather than in precise terms or levels. However, when a public entity does this, it needs to clearly indicate the relative change desired against a baseline or trend.

3.16
Public entities need to include comparative information to help the reader understand the targets the public entity has set for its main measures and allow the reader to judge their appropriateness. Comparative information can be:

• past and future targets;
• past results;
• comparisons with other organisations, countries, regions, or national averages; or
• other forms of benchmarking.

Linking impacts and outcomes to outputs

3.17
The relationship between outputs and impacts or outcomes should be credible and clearly explained in both the SOI and the Information Supporting the Estimates. This helps the reader to understand how the public entity considers that the goods and services it delivers will result in improved impacts and outcomes. It also allows readers to understand the alignment between the public entity's medium-term goals and its annual performance plan.

3.18
Presenting outputs in the context of outcomes is more difficult for government departments, now that the forecast SSP is no longer reported in the SOI. Instead, the forecast SSP information is reported in the Information Supporting the Estimates. However, we consider that all public entities can do more to use their accountability documents to demonstrate the links between outputs, impacts, and outcomes. The SOI, in particular, allows more freedom and flexibility for public entities to best present the important links between these key components of the performance story.

Specifying outputs and output classes

3.19
The Public Finance Act 1989 and Crown Entities Act 2004 define outputs as the goods or services that are supplied by a public entity or Crown entity.

3.20
The forecast SSP should include the outputs for which the public entity is accountable. These are the goods and services provided to third parties, whether delivered directly by the public entity or by third parties. All significant goods and services should be identified and reported on, to ensure that readers get a comprehensive, balanced, and proportionate picture of the public entity's services.

3.21
The definitions do not include goods and services that are produced for purchase or consumption solely within a group of public entities or Crown entities.⁷ Therefore, the outputs identified by a public entity should exclude other elements (including inputs, activities, processes, impacts, or outcomes), except where these are useful or necessary to provide context. If an entity refers to other elements of performance within the forecast SSP, the other elements should be identified as such and not reported as if they were outputs.

3.22
Using output classes allows the public entity to group similar services. This enables the reader to make sense of the public entity's outputs without being overwhelmed with detail. The underlying rationale for the output classes should be clear and justifiable.

3.23
Public entities should use consistent output classes, and output titles and descriptions, throughout their forecast information, including in their financial information. This links services and performance levels with their associated costs, so that readers are better able to assess the public entity's cost-effectiveness.

3.24
Output information should be supported by contextual information telling the reader why the services are delivered. The reader should be able to understand the relevance of the output performance measures in the context of the effect that successful service delivery should have on achieving the desired outcomes.

Performance measures and targets for outputs

3.25
Performance measures for outputs need to be meaningful. They need to measure the aspects of service that represent good performance, are within the control of the reporting public entity, and reflect the challenges, priorities, or other focus of management for that service. In other words, performance measures need to be relevant, controllable, valid, verifiable, unbiased, complete and balanced, understandable, and comparable.

3.26
To provide a balanced and rounded view of service performance, performance measures for outputs usually need to cover several dimensions – for example, quality, quantity, and timeliness.

3.27
Although outside the scope of this discussion paper, all performance measures should be supported by detailed definitions and appropriate systems to collect the relevant data. Internal controls should include documenting of these systems and provide scope to reduce the volume of technical information presented in the forecast.

3.28
Performance measures for outputs need to have well-specified targets with comparative information. This lets the reader gauge the public entity's intended level of performance for the period, and later on to compare actual performance against those plans. Targets should be reasonable and reflect the priorities of the public entity, and its resources, decisions, and past performance.

3.29
SOIs (including the forecast SSPs for Crown entities) provide scope for reporting longer-tem historical trend information that gives readers context for the targets set. At a minimum, targets must be included for the forthcoming year. Longer-term targets help readers understand the direction of intended performance. Comparative information from other public entities, regions, or countries can provide useful benchmarks.

3.30
Generally, data that records demand-driven events relates to matters outside the reporting public entity's control. Demand-driven data can provide context, and be useful when it relates to real measures of performance – for example, to understand the effect on the quality or cost of services from a change in the volumes being handled.

3.31
Demand-driven data – for example, the number of applications received, number of prisoners, or number of students enrolled in compulsory education programmes – is not a substitute for "true" service performance measures and targets, or results. Demand-driven data is not often a true measure of output performance, because it does not tell the user about how well the service is being provided.

6: As defined in section 2 of the Public Finance Act 1989. Outcomes are not defined in the Crown Entities Act 2004.

7: Section 2 of the Public Finance Act has a definition of outputs that means the goods or services supplied by a department, Crown entity, Office of Parliament, or other person or body. The definition includes goods and services that a department, Crown entity, Office of Parliament, or other person or body has agreed or contracted to supply on a contingent basis, but that have not been supplied.