Part 9: Improving service performance reporting

Central government: Results of the 2011/12 audits.

9.1
In this Part, we report on improvements in the quality of performance reporting as we continue to phase in the Auditor-General's revised auditing standard for auditing service performance reports.³² Our efforts will continue as we apply the revised auditing standard to more public entities in 2012/13. In particular, we discuss:

the importance of performance information and reporting;
how we are phasing in our revised auditing standard;
results of audit reports for 2011/12 using the revised auditing standard;
trends in entities' quality of performance reporting; and
our future work to phase in the revised auditing standard.

Why performance information and reporting is important

9.2
Performance reporting is an integral part of our parliamentary accountability system, enabling Parliament to hold ministers and central government entities to account. Parliament is required to make decisions about the funds to be voted through the annual Budget process and to review the performance of the entities that have used those funds. Parliament and the public rely on accountability documents to assess the performance of public entities and how effectively they use public funds.

9.3
If public entities are clear on why they exist, what changes they are seeking to influence, and how their services contribute to those desired changes, then they can build a sound framework for planning, measuring, managing, and reporting their performance. When public entities have good performance systems for running their organisations and making informed decisions, they should more easily produce external accountability reports that are relevant, reliable, and in keeping with legislation and generally accepted accounting practice. Within these reports, we expect public entities to:

clearly explain what they are trying to achieve;
clearly explain the services they are accountable for providing;
present a framework that shows the cause-and-effect relationships and assumptions underpinning the spending of taxpayer funds on public services to achieve public benefit;
report targets and results through a well-rounded suite of appropriate performance measures; and
report externally in a way that is integrated with the information they use to manage their business.

9.4
We continue to witness significant improvements in the quality of government departments' and Crown entities' performance reports.

Phasing in our revised auditing standard and accompanying support

9.5
We are phasing in a revised standard for auditing service performance reports. Our revised standard requires our auditors to modify their audit opinion if the performance information in the annual report does not, in their opinion, fairly reflect performance for the year. Under the previous auditing standard, there was very little scope for appointed auditors to modify their audit opinion if the content of the performance report was inadequate.

9.6
Before implementing our revised auditing standard, we have endeavoured to prompt and help public entities to raise the standard of their performance reporting through our Office's support initiatives, combined with regular evaluation and feedback to entities from appointed auditors. To help make the improvement programme manageable and to provide a concentrated effort on our support initiatives, we phased in the application of the revised standard.

9.7
In 2010/11, 28 of the larger and more complex entities were audited using the revised standard.

9.8
In 2011/12, we applied the revised standard to the audit of a further 34 (mainly medium-size) departments and Crown entities.

9.9
We will apply the revised standard to our audits of the remaining (generally smaller) departments and Crown entities in 2012/13.

9.10
We provided entities with advance notice about the revised auditing standard to allow time and to provide support for them to up-skill and improve their performance information and reporting. In the lead-up to applying the revised standard to our audits, we placed greater emphasis on performance information in the audit, provided clear signals and specific advice to entities about continual improvement in this area, and provided guidance and other support to help entities improve.

9.11
We have supported public entities by:

giving feedback on performance documents;
holding one-on-one conversations about business objectives and how to reflect these in performance information systems;
providing guidance and advice, seminars, and workshops; and
giving other forms of support, such as online resources to support an emerging community of practice.

Audit reports for 2011/12

9.12
We issued just one modified opinion for poor performance reporting for those entities we audited using the revised standard in 2011/12. We modified our opinion for NZFSC because of the absence of results data (see Figure 2 and paragraphs 5.22 and 7.7). However, the quality of NZFSC's performance reporting framework has been of a high standard for several years.

9.13
In Figure 13, we list the 28 departments and Crown entities that were first audited using the revised auditing standard in 2010/11 and the 34 departments and Crown entities that were audited using the revised auditing standard for the first time in 2011/12.

Figure 13
Entities audited using the revised auditing standard for service performance reporting

Entities audited using the revised auditing standard for the first time in 2010/11 (first phase – Category A)	Entities audited using the revised auditing standard for the first time in 2011/12 (second phase – Category B)
Government departments	Government departments
Department of Conservation Department of Corrections Department of Labour Inland Revenue Department Land Information New Zealand Ministry of Agriculture and Forestry/Ministry for Primary Industries Ministry of Economic Development Ministry of Education Ministry for the Environment Ministry of Fisheries Ministry of Foreign Affairs and Trade Ministry of Health Ministry of Justice Ministry of Social Development New Zealand Customs Service New Zealand Defence Force New Zealand Police Te Puni Kōkiri The Treasury	Crown Law Office Department of Building and Housing Department of Internal Affairs Department of the Prime Minister and Cabinet Education Review Office Ministry of Culture and Heritage Ministry of Defence Ministry of Pacific Island Affairs Ministry of Science and Innovation Ministry of Transport Parliamentary Service State Services Commission Statistics New Zealand
Crown entities	Crown entities
Accident Compensation Corporation Housing New Zealand Corporation Legal Services Agency New Zealand Fire Service Commission New Zealand Lotteries Commission New Zealand Trade and Enterprise New Zealand Transport Agency Public Trust Tertiary Education Commission	Arts Council of New Zealand Charities Commission Civil Aviation Authority Commerce Commission Earthquake Commission Government Superannuation Fund Authority Guardians of New Zealand Superannuation Health Research Council of New Zealand Maritime New Zealand Museum of New Zealand Te Papa Tongarewa New Zealand Blood Service New Zealand Film Commission New Zealand Qualifications Authority New Zealand Symphony Orchestra New Zealand Tourism Board New Zealand Venture Investment Fund Limited Radio New Zealand Sport and Recreation New Zealand Te Māngai Pāho (Maori Broadcasting Commission) Te Taura Whiri i te Reo Māori (Māori Language Commission) Television New Zealand

9.14
During the last year, appointed auditors maintained contact with the medium-size "second phase" entities, offered advice on critical deficiencies and where to make improvements, monitored progress, and reviewed draft performance frameworks and accountability documents. Entities in this group have responded well and continued to make significant improvements to their performance frameworks, choice of performance measures, and presentation of those frameworks and measures in their annual reports.

Assessing the quality of performance reporting

9.15
As well as attesting to an entity's service performance in their audit opinions, appointed auditors assign a grade for service performance information and its associated systems and controls (see Appendix 1). The grade for the service performance aspect can be "Poor", "Needs improvement", "Good", or "Very good". (See Figure 27, Appendix 1, for explanations of these grades.)

Entities audited using the revised audit standard for the first time in 2010/11 (first phase – Category A entities)

9.16
In Figures 14 and 15, we present a time-series of the grades for performance reporting that our appointed auditors have assigned to 24 government departments and Crown entities since 2008/09. These 24 entities are those to which we applied our revised auditing standard from 2010/11 and for which we issued a performance information grade for all four years.³³

9.17
Figure 14 presents results by grade value and highlights a significant reduction in "Poor" and "Needs improvement" grades during the four years, with a related rise in "Good" grades and the emergence of entities attaining "Very good" grades.

Figure 14
Performance reporting grades by grade value (Category A entities), 2008/09 to 2011/12

9.18
Figure 15 highlights the movement in the distribution of grades from year to year. The shape of the grade clusters shows a distinct move to the right (that is, improvement) during the last four years.

9.19
Both graphs show that the largest improvement took place during 2010/11, which was the year in which we first applied our revised auditing standard to these 24 entities. Since then, entities have continued to improve, with grades skewing more towards the higher end of the scale.

Figure 15
Performance reporting grades by year (Category A entities), 2008/09 to 2011/12

9.20
This year is the first time we have identified "Very good" performance reporting within this larger group of entities (Inland Revenue and Ministry for the Environment). Those receiving a "Good" grade have more than trebled during the past four years. The incidence of the "Needs improvement" grade has reduced to less than a third of what it was, and the "Poor" grades disappeared during the year that we first applied our revised auditing standard to these entities (that is, 2010/11).

Entities audited using the revised auditing standard for the first time in 2011/12 (second phase – Category B entities)

9.21
In Figures 16 and 17, we present a four-year time-series of the grades for performance reporting for those entities to which we applied our revised auditing standard for the first time in 2011/12. This group comprises 30 (mainly) medium-size government departments and Crown entities for which we have issued a performance information grade for all four years.³⁴

9.22
Figure 16 presents results by grade value and highlights a considerable reduction in "Poor" and "Needs improvement" grades over the four years, with a related rise in "Good" grades and one entity achieving a "Very good" grade.

Figure 16
Performance reporting grades by grade value (Category B entities), 2008/09 to 2011/12

9.23
Figure 17 highlights the movement in the distribution of grades from year to year. As with the group of larger entities (Category A), the grade clusters for Category B entities also show a distinct move to the right during the last four years, signalling considerable improvement.

9.24
The grades set out in Figures 16 and 17 show little improvement to 2009/10 but a marked improvement beginning in 2010/11, with improvement continuing at similar pace in 2011/12. This group of entities responded, in 2010/11, with improved performance reporting one year before we first applied our revised auditing standard to them and continued the pace of change during the year we first applied the revised standard (2011/12). The grades are now clearly skewed towards the higher end of the scale.

Figure 17
Performance reporting grades by year (Category B entities), 2008/09 to 2011/12

9.25
One entity within this group has achieved a "Very good" grade (New Zealand Blood Service). Those entities receiving a "Good" grade have more than doubled during the past four years. The incidence of the "Needs improvement" grade is less than half of what it was in 2008/09, and the "Poor" grades among this group of entities disappeared early on.

9.26
The extent of genuine improvement may be understated by the graphs, because we have seen examples of improved performance reporting that has not resulted in an increased grade. In the short term, we encourage entities and the users of performance reports to focus on how the entities have responded to our specific recommendations for improvement and how they are resolving the current deficiencies in their reporting frameworks and practices.

9.27
In the medium term, we expect to see reporting improvements reflected in grade increases, and that is what we are seeing now. As we have phased in our revised auditing standard during the past two years, we have made a concerted effort to support entities, and many entities have made a concerted effort to improve the quality of their performance reports. These improvements are evident in the significant grade increases issued for performance reporting in 2010/11 and 2011/12.

Future intentions

9.28
In 2012, we ran workshops to support the third group of entities (Category C) to which we will apply our revised auditing standard in 2012/13. Our aim was to help this group of entities to improve their performance information frameworks, systems, and reports as they prepared their 2012-15 Statements of Intent, which they will report against in 2012/13. We also helped establish a community of practice for this group and provided them with a secure website where they could access tools, guidance, and advice as well as share ideas and documents.

9.29
In 2013, appointed auditors will concentrate efforts on monitoring the progress of this third group as these entities prepare their 2012/13 annual reports, which we will then audit using our revised auditing standard.

32: Controller and Auditor-General (March 2011), The Auditor-General's Auditing Standards, Section 28: AG-4 (Revised): The audit of service performance reports, available at www.oag.govt.nz.

33: We applied our revised auditing standard to 26 entities from this group in 2011/12, two less than in 2010/11, because of the disestablishment of the Ministry of Fisheries and the Legal Services Agency in the previous year. We issued grades to 24 of the 26 remaining entities because of the disestablishment of the Department of Labour and the Ministry of Economic Development at the 30 June 2012 year-end.

34: We applied our revised auditing standard to 34 entities from this group in 2011/12. We have issued grades to 30 of these 34 entities. We did not issue grades for three disestablished entities (Charities Commission, Department of Building and Housing, and Ministry of Science and Innovation). The New Zealand Symphony Orchestra is excluded from our figures because of its later balance sheet date.

page top