Part 5: The performance framework

5.1
In this Part, we discuss our observations of the first of the three main individual content areas of the LTCCP – the performance framework. We discuss asset management planning in Part 6 and the financial management framework in Part 7.

The Local Government Act 2002 and our expectations about the performance framework and performance information

5.2
The Act gives local authorities a framework and powers to decide the activities they carry out and how they will carry them out. In providing this framework, the Act also provides that local authorities are accountable to their communities. The LTCCP is a specific basis for this accountability.

5.3
A core element of achieving accountability, which in turn drives asset management and the financial forecast, is the required statement in the LTCCP of:

… the intended levels of service provision for the group of activities, including the performance targets and other measures by which actual levels of service provision may meaningfully be assessed.¹

5.4
The performance framework should reflect the local authority’s intended direction overall and for groups of activities. It should also provide clarity about what the local authority plans to achieve in the short term and an indication of its plans and performance in the longer term. It is the main mechanism for the local authority to inform its community about:

• the range, quality, quantity, and effect of the services it intends to provide; and
• the choices made by the local authority for its services.

5.5
How these levels of service have been developed, expressed, and related to other parts of the LTCCP has been of central interest to us when auditing the LTCCP, as the performance framework is one of the three matters on which the auditor is specifically required to report under sections 84(4)(c) and 94(1)(c) of the Act. These sections require the LTCCP to contain a report from the local authority’s auditor on:

… the extent to which the forecast information and performance measures provide an appropriate framework for the meaningful assessment of the actual levels of service provision.

5.6
Reporting on the forecast information and measures has required us to consider whether a local authority’s LTCCP provides a meaningful performance framework for its groups of activities that:

• sets out the rationale for the local authority’s involvement in each activity, the local authority’s objectives, and any significant risks or negative effects that need to be managed in the course of delivering the activity;
• sets levels of service measures that provide a useful way of understanding how the service delivers on the local authority’s rationale for performance involvement;
• provides information about the planned levels of service for the next three years and outlines planned levels of service for the subsequent seven years; and
• sets reasonable targets and other measures to allow the actual levels of service provision (as reported at the end of each financial year in the annual report) to be meaningfully assessed.

5.7
We identified 11 attributes of useful performance reporting and associated processes for the LTCCP. We expected performance reporting to:

• be approached in a systematic, transparent way throughout the local authority;
• present important information consistently across services;
• have a logical flow, including a sustainability context;
• relate to management decision-making;
• align with business and asset information;
• be based on sound information on resources and the infrastructure asset base;
• result in reasonable targets and reliable assumptions;
• identify risks and potential significant negative effects;
• be complete and balanced, with levels of service clearly identified;
• be relevant and meaningful to users and the community; and
• use performance measures that are accurate, neutral, and comparable.

5.8
Several elements are required for a local authority’s performance planning, reporting, and management framework to perform its intended functions in the areas of decision-making and accountability. Figure 16 outlines these elements and shows the flow of the Act’s requirements. Together, these elements form a feedback loop that allows local authorities to manage current service delivery and to plan for future delivery. Therefore, it is important that all the elements are in place, of good quality, and well-linked to provide a strong framework.

Figure 16
Feedback flow of performance in the Local Government Act 2002

Observations from our LTCCP work

5.9
Our observations of the performance framework are drawn from issues raised with us by auditors.

5.10
There are no specific standards or guidance that we can rely on when considering whether a local authority has complied with the Act’s requirements for performance frameworks and information. Therefore, to reach conclusions about the extent to which a meaningful performance framework had been set out in the LTCCP, we used our own work and knowledge of local authorities to apply a "size and scale" approach.²

5.11
As a result, our conclusions about the meaningfulness of performance frameworks and information were based on the practice generally adhered to by relevant peer local authorities. The limitation of this approach is that the practice as a whole may not be as is intended by the Act – a limitation that we consider is relevant to performance frameworks and information in the LTCCP.

5.12
Two local authorities received non-standard audit opinions³ as a result of issues directly related to performance information:

• Gisborne District Council’s LTCCP Statement of Proposal and final LTCCP received a qualified audit opinion because the documents did not clearly set out the council’s desired levels of service. Consequently, the LTCCP documents did not adequately link forecast expenditure to what the council was trying to achieve through that expenditure. This affected our view on the reasonableness of the underlying information and the meaningfulness of the performance measures.
• Stratford District Council’s LTCCP Statement of Proposal received a nonstandard audit opinion because it inconsistently outlined the desired levels of service and the associated performance measures. It did not, therefore, present a complete forecast and performance framework to the community. The council chose to review the information before releasing the final LTCCP. As a result of the improvements made after that review, we were able to issue an unqualified audit opinion on the final LTCCP.

5.13
While these two councils were the only ones to receive non-standard audit opinions for issues relating to performance frameworks and information, in our view, this was the area of greatest deficiency associated with the 2006-16 LTCCP Statements of Proposal.

5.14
Although we accept that best practice is developing as the local government sector learns more about preparing LTCCPs, a determined effort is needed to improve the ability of many local authorities to undertake planning that allows them to address the long-term needs of their communities.

5.15
Overall, the analysis from our LTCCP audits suggests that the performance framework is a major area requiring development for many local authorities. The attributes of effective performance reporting that we anticipated were often not reflected in the LTCCP or in the underlying information and assumptions.

Common weaknesses in performance frameworks

5.16
During our LTCCP audits, we noted weaknesses and issues throughout the elements and processes used by local authorities for their performance planning, reporting, and management frameworks. In our view, along with performance frameworks, noted above, the performance information processes used by most local authorities require significant development to achieve the intentions of the Act.

5.17
Common weaknesses that arose during our LTCCP audits related to:

• a lack of a logical flow in performance reporting;
• levels of service, and performance measures and targets; and
• outcomes monitoring.

Logical flow in performance reporting

5.18
The relevance and appropriateness of selected levels of service performance measures and targets can only be assessed if the local authority sets out the rationale for its activities clearly and logically, describing how its assessment of well-being and the community’s outcomes provide the basis for the selection and direction of its activities. The selection of levels of service and performance measures and targets should reflect this rationale, the local authority’s main strategies and objectives, and any associated risks and negative effects. Figure 17 shows the logical flow of the Act’s performance requirements.

Figure 17
Logical flow in the Local Government Act’s performance requirements

5.19
We found that many LTCCPs struggled to demonstrate the overall alignment of the performance framework. In about 25% of LTCCPs, there was a weak link between the rationale for measures and levels of service information. This was because the reasons that certain services were being provided were not clearly stated. In more than 30% of LTCCPs, activities were only weakly linked to performance measures and outcomes, leading to confusion about how the activities would be measured and how they would contribute to furthering outcomes and/or well-being.

Levels of service and performance measures and targets

5.20
Information about levels of service and performance measures and targets is less meaningful when the logical flow of information is not complete or clear. Therefore, the logical flow weaknesses noted in paragraph 5.19 compounded issues about levels of service and performance measures and targets.

5.21
Almost 25% of LTCCPs lacked clarity about definitions of levels of service within the information on each group of activities. We were particularly surprised to find that about 20% of local authorities did not have clearly defined levels of service in their underlying asset management planning information (see Part 6).

5.22
Without sound asset information to give effect to intended levels of service, it is difficult to assess and plan for the asset capacity needed and the associated financial costs. It is also difficult to measure achievement of those levels of service.

5.23
This lack of information about levels of service affected the extent to which local authorities were able to identify accurate, neutral, comparable, and reasonably based best-estimate performance targets.

5.24
More than 65% of local authorities had performance measure shortcomings to varying extents, meaning that the intentions of the groups of activities were unclear and there was often no identifiable measure to assess achievement. These shortcomings most commonly related to:

• the relevance of measures to the rationale for the activity;
• the context of the environment and risks within which the activity operated; and
• the reasonableness of the 10-year performance targets (that is, whether these were reasonably based best estimates).

5.25
We expected that the intended achievements resulting from a service or activity and the associated risks and potential negative effects would influence the selection of relevant performance measures.

5.26
Clause 2(1)(c) of Schedule 10 of the Act requires the LTCCP to, for each group of activities of the local authority, outline any significant negative effects that any activity within the group of activities may have on the social, economic, environmental, or cultural well-being of the local community. This requires the local authority to consider potential significant negative effects on community well-being and to disclose crucial assumptions. We observed that about 30% of LTCCPs either did not include information on negative effects or had poor coverage of negative effects.

5.27
The risks and negative effects that were identified in contextual information for groups of activities did not appear to have resulted in performance measures and targets for managing such risks. For example, contextual information identified that resource consent requirements for taking water were being frequently breached or that there was not enough water available to meet demand at peak periods, but there was no discussion about how water supply issues were being managed.

5.28
A common approach used to measure the performance of levels of service information was to state that the levels of service, and, therefore, the performance measures and targets, were assumed to be constant for the duration of the LTCCP. More than 30% of local authorities had constant, or near constant, levels of service for the 10 years of their LTCCP. However, when considered in conjunction with the information on other groups of activities, it often appeared that local authorities had failed to integrate the effect of asset or financial decisions with their associated effects on levels of service. This meant that, while asset and/ or financial information would be indicating significant change, levels of service information would inappropriately remain constant.

5.29
In 2006, this issue arose particularly around activity areas such as water services and building consents. Local authorities had not included the intended improvements in drinking water quality or supply within their levels of service and had not provided for changes affecting building consent services.

5.30
Many local authorities are struggling to meet the new requirements of the Building Act 2004 amendments, which raises a concern that local authorities are in a "double jeopardy" situation. The Building Act 2004 (among others) requires local authorities to process applications within certain time constraints. However, in many instances, the annual reports of local authorities show that these deadlines are not being met. The dilemma arises, therefore, whether a local authority’s LTCCP should show a reasonable estimate based on past performance and effectively plan to breach the Building Act or whether it should show an unreasonable target that reflects its legislative obligations.

5.31
In our view, a sustained historic failure to meet statutory requirements indicates the need to invest greater effort into a service. A local authority should advise within the activity information in the LTCCP that it has not previously met its statutory obligations and outline the steps it is taking to remedy this. In suggesting that local authorities outline such steps, we appreciate that reasons for failure to meet statutory requirements may have long-term resourcing effects (for example, to increase funding to deal with new statutory requirements) or may not be resolvable in the short term (for example, where there are skills and expertise shortages).

Outcomes monitoring

5.32
In addition to inadequate performance measures and targets, supporting information on the achievement of outcomes was also frequently incomplete. Our analysis of the LTCCP audits found that nearly 30% of local authorities did not have complete outcomes monitoring. Again this shows the lack of a direct link to measuring specific community outcomes. These outcomes often did not have specific targets and measures. The monitoring arrangements for community outcomes that were included were often expressed very generally.

5.33
We were aware that the framework and measures for monitoring outcomes were in the very early stages of development because of the different arrangements for local authorities for identifying outcomes in their first LTCCP in 2003 or 2004.⁴ Local authorities were required to include the information for the first time in the 2006-16 LTCCP. As a result, we expect outcomes monitoring information to show improvement when local authorities publish their annual and community outcomes reports.

5.34
Monitoring outcomes is a critical component of the performance framework, as it allows local authorities to demonstrate the extent to which their activities are furthering community outcomes and/or well-being. Information disclosed in the LTCCP needs to communicate how the identified outcome is being achieved, with reference to the current state of the outcome and the measures used to assess change in the state of the outcome over time.

5.35
In more than 10% of LTCCPs, community outcomes were not clearly linked to the local authority’s strategy. This created confusion about how the local authority’s strategic vision integrated with the community vision, which is articulated through the community outcomes.

5.36
We noted a peculiarity in outcomes monitoring by two local authorities. They had disclosed activities and tasks under each identified outcome as the means intended to demonstrate how the outcome is being furthered, rather than using indicators that might allow the state of the outcome to be assessed. It does not seem likely to us that an activity- based or task-based approach to outcomes monitoring would fulfil the expectations of the Act. This is because the specified activities do not measure the outcome, so there is a weakness in the link. It raises the question as to whether the local authority is adequately monitoring the community outcomes.

5.37
We consider that the better practice is to include evaluative indicators, rather than use activity reporting to measure outcomes. However, we could not see that an activity reporting approach was precluded by the Act. Therefore, we could only advise the two local authorities taking this approach of our view on the matter.

5.38
Every three years, local authorities are required to report on their monitoring of the achievement of community outcomes to show the attainment of community well-being and the community outcomes planned. To show how the local authority is contributing to community outcomes, it is required to report annually on how activities it carries out result in the furthering of community outcomes.

5.39
The community outcomes and well-being identified by the local authority should flow into the intended levels of service and the performance measures and targets to allow for meaningful assessment. Annual reporting should explain any variance between intended and actual levels of service, and how the levels of service link to activities that help to advance community outcomes and well-being.

Our conclusions

5.40
We are particularly concerned that levels of service identified by local authorities appear to be poorly defined. Levels of service are vital in their own right, and they also underpin asset management and financial planning.

5.41
The performance framework and information should form a feedback loop to fulfil its intended function. However, in our view, the performance framework and the information that comprise this feedback loop were the areas of greatest deficiency in the 2006-16 LTCCPs.

5.42
SOLGM has set up a working party to improve this area. We hope to provide input to this timely and vital initiative.

1: Schedule 10, clause 2(a).

2: We identified peer local authorities by grouping them according to size and scale, and common factors such as population, rates to median income, population to area, full-time equivalent staff, debt to equity, and other local authority income.

3: For a full explanation of non-standard audit reports, see article 1.5 in Local government: Results of the 2005/06 audits, ISBN 0-478-18186-8.

4: See section 279(2) of the Act.