Part 2: Our output classes, performance measures, and targets (including Statement of service performance)
Our output classes and the outputs within those classes are:
- Audit and assurance services
- Supporting accountability to Parliament
- Performance audits and inquiries
Audit and assurance services
The main purpose of an annual audit is to provide independent assurance about the fair disclosure of the financial information – and, in many instances, non-financial information – within annual reports. An audit involves a range of procedures, tests, and management and governance enquiries to support our opinion.
In carrying out annual audits, auditors consider the legislative mandate, and may raise recommendations for improvements in matters of effectiveness and efficiency, compliance with statutory obligations, waste, probity, and financial prudence in management letters to the governors and managers of public entities. We also use our annual audits to gather information and knowledge about public entities to assist us in advising Parliament and other stakeholders and to help determine the work we do in our performance audits, inquiries, long-term plan audits of local authorities, good practice guides, and discussion papers.
A foundation for trust in the public sector is that public entities respond to their statutory public reporting obligations and to findings resulting from audits. Public entities' approach to public reporting and their responses to audit recommendations demonstrate their commitment to effective, efficient, and accountable service delivery.
In 2010/11, annual audits and other assurance services accounted for 88% of our total expenditure. The major portion of this output class relates to annual audits of public entities that are required by statute to be carried out.
There are two main products from an annual audit:
- the audit report; and
- the management report,
- plus, for some public entities:
- there is also a financial review report.
The audit report is addressed to the readers of the financial statements and, where applicable, the performance information. It provides the auditor's independent opinion (the audit opinion) on whether the financial statements (and, where applicable, the performance information) fairly reflect the public entity's financial performance and position (and, where applicable, service performance information). If the financial statements fairly reflect the public entity's financial performance and position (and, where applicable, the service performance information), the auditor issues an audit report with an unmodified opinion. However, if the auditor identifies a material3 error or omission in the financial statements or performance information, the auditor issues an audit report with a modified opinion. In addition to the audit opinion, the auditor can also draw readers' attention to other matters, some of which could be highlighted within the financial statements (and, where applicable, the service performance information) and others of which are not.
The management report is addressed to the governing body or the senior management of public entities. It sets out any significant issues identified by the auditor during the audit and provides recommendations for improving the public entity's controls, systems, and processes.
If public entities are subject to financial review by a select committee, we also report the results of annual audits to responsible Ministers and select committees. The report includes a grading for public entities, based on our assessment of their management control environment, and financial and service performance (where required) systems and controls.
Key processes supporting annual audits
Delivery of annual audits is supported by several key processes, including:
- appointing auditors and monitoring audit fees;
- setting the Auditor-General's auditing standards;
- maintaining auditor independence; and
- quality assurance.
Appointing auditors and monitoring audit fees
The Auditor-General appoints auditors to carry out the annual audits of public entities. These auditors are appointed from a group of about 50 audit service providers. The group includes Audit New Zealand and private sector accounting firms, which range from the four major chartered accounting firms to sole practitioners. Most audits are allocated directly to an auditor, but from time to time an auditor is appointed to carry out an audit after a competitive tender.
Because we mainly use an allocation approach, we monitor audit fees at the point of negotiation between the appointed auditor and the public entity. We also provide a comparative analysis to help resolve any concerns about proposed audit fees. Our objective in monitoring fees is to ensure that fees are based on realistic hours (that is, hours that reflect the nature and extent of work required), an appropriate audit team mix, and charge-out rates that are in line with market rates. We aim for fees that are fair to the public entities and also provide a fair return to the auditors for the work required by them to meet the Auditor-General's auditing standards.
During 2010/11, the Auditor-General appointed or reappointed auditors (except where a change of auditor was made within the same audit service provider) to conduct audits of 14 schools and 300 other public entities and their subsidiaries.
Setting the Auditor-General's auditing standards
Section 23 of the Public Audit Act 2001 requires the auditing standards of the Auditor-General to be published, by way of a report to the House of Representatives, "at least once every 3 years". The Auditor-General's Auditing Standardswere most recently published in March 2011. These standards are available on our website (www.oag.govt.nz).
Section 23 also requires that our annual report include a description of any significant changes made to the standards during the year. As part of the March 2011 re-publication, the auditing standards were significantly updated so that they are now aligned to the format and requirements of the New Zealand Equivalents to International Standards on Auditing (ISA (NZ) Auditing Standards). The ISA (NZ) auditing standards were progressively introduced over the last two years in New Zealand and now fully apply to all audits carried out by members of the New Zealand Institute of Chartered Accountants.
In addition to the process to align the Auditor-General's auditing standards with ISA (NZ) Auditing Standards, we developed two new standards covering the work of the Office other than annual audits,4 because the scope of ISA (NZ) auditing standards now more clearly applies to annual audits only.
Maintaining auditor independence
High independence standards are set for both the Auditor-General's employees and appointed auditors from private sector accounting firms. Compliance with these standards is monitored regularly. Any threats to auditor independence that were identified during the year were subject to mitigation actions that the Auditor-General considered appropriate to either eliminate the threats or reduce them to an acceptable level.
Quality assurance
We carry out quality assurance reviews of appointed auditors to ensure that they have complied with the relevant professional standards, as well as the Auditor-General's own published auditing standards. We aim to review the performance of each of our appointed auditors at least once every three years. There are five levels of quality assurance rating, assessed using the reviewers' overall judgement of the quality of the audit work carried out. The five levels are "excellent", "very good", "good", "satisfactory", and "re-review".
We also carry out reviews of each firm's quality control systems that are designed to provide assurance about the organisational factors that affect audit quality. We aim to complete these reviews at least once every three years in conjunction with our quality assurance reviews of appointed auditors.
Main impact measures and standards
We report a range of impact measures and standards that are designed to help gauge the impact of our annual audit and other audits and assurance services.
The measures of intended impacts of the appropriation for this output class are as shown in Figure 4.
Figure 4
Actual performance against impact measures and standards for output class: Audit and assurance services
| 2010/11 forecast main impact measures and standards | Actual |
|---|---|
| The number of public entities' audited annual reports issued within the statutory time frame is improved (or at least maintained), measured against the previous two years. The actual result for 2010/11 was: Total audits due for completion in year: 3,902 Percentage on time: 87%* |
Percentage of audits completed on time for the five years from 2007 to 2011  |
| The number of public entities' audited annual reports containing modified** opinions is reduced (or at least maintained), measured against the previous two years. The actual result for 2010/11 was: 111 which was 2.8% of the total opinions issued. |
Percentage of audited financial reports that contain modified audit opinions for the five years from 2007 to 2011  |
* We have assumed in this figure that all 202 school audits exempted from the statutory deadline would have been completed on time had it not been for the major earthquake on 22 February 2011.
** According to the International Standard on Auditing (New Zealand) ISA (NZ) 705, a modified opinion includes a qualified opinion (an except-for opinion under the previous Auditing Standard, AS-702: The Audit Report on an Attest Audit), an adverse opinion, or a disclaimer of opinion.
| 2010/11 forecast main impact measures and standards | Actual |
|---|---|
| Public entities' acceptance of audit service providers' management report recommendations is improved (or at least maintained), measured against the previous two years. (Note – In prior years this has been assessed against only Audit New Zealand's management report recommendations.) The actual result for 2010/11 was: 72% accepted; 12% rejected; 16% noted , under consideration or not responded to |
Percentage of management report recommendations accepted by public entities for the five years from 2007 to 2011  |
| Central government entities' management control environment, financial information, and service performance* information systems and controls are improved (or at least maintained), measured against the previous two years.** The actual result for: 2009/10 grades for management control environment: Very good 50% Good 42% Needs improvement 7% Poor 1% 2009/10 Grades for financial information systems and controls: Very good 44% Good 50% Needs improvement 5% Poor 1% 2009/10 Grades for service performance information and associated systems and controls: Very good 1% Good 31% Needs improvement 58% Poor 10% |
Grades for management control environment (MCE) for 2008, 2009, and 2010 |
Grades for financial information systems and controls (FISC) for 2008, 2009, and 2010 |
|
Grades for service performance information and associated systems and controls (SPIASC) for 2009, and 2010 SPIASC was first graded in 2008/09. The 2008/09 results did not include district health boards (DHBs), which were all graded "needs improvement/poor", but the 2009/10 results did include DHBs that were graded either "needs improvement" or "poor". |
* Service performance information and associated systems and controls was graded for the first time under our new framework as part of the 2008/09 audits or with audit work carried out in 2009/10.
** Graphs show the year-end period for which the audit is carried out. 2010/11 year audits and associated grades are completed during 2011/12.
Overall comments on the impact of our annual audits and other audits and assurance services
We are generally pleased with the trends in our impact measures and standards as follows:
- a high percentage of audited financial statements being completed within statutory deadlines despite the disruption caused by the recent earthquakes, to both public entities and auditors alike;
- a low percentage of modified audit opinions being maintained despite a small increase compared with the previous two years;
- a high percentage of acceptance by public entities of our recommendations for improvement being maintained despite a small reduction compared with the previous two years; and
- a high percentage of "good" and "very good" grades assigned to public entities' systems and controls being maintained for MCF and FISC and an improving percentage for SPIASC.
Performance against output delivery measures and standards
We also report a range of output measures and standards (see figure 5) that are designed to assess our own performance in carrying out annual audits and other audits and assurance services.
Figure 5
Actual performance against output delivery measures and standards for output class: Audit and assurance services
| 2010/11 measures and standards of output delivery (and forecast for 2011/12 and 2012/13) | Actual |
|---|---|
| Less than 30% of the outstanding audit reports at 30 June 2011 are because of inaction on our part. We achieved our forecast standard with an actual result of: Total arrears at 30 June 2011*: 319 (compared with 313 at 30 June 2010) Arrears due to inaction on our part: 20% |
Percentage of outstanding audit reports at 30 June because of our inaction for the five years from 2007 to 2011 |
| All management reports are issued within six weeks of issuing the audit report. We did not achieve our forecast standard with an actual result of 94% issued within six weeks. |
Percentage of management reports issued within six weeks for the five years from 2007 to 2011 |
| No outstanding LTP (previously referred to as LTCCP) opinions at 30 June of the year in which LTPs are to be adopted by local authorities are because of inaction on our part. All LTP management reports are issued within six weeks of issuing the LTP opinion. Not applicable. |
Long-term plan audits are carried out on a three-yearly basis. The last LTP audits were carried out in 2008/09. The next opinions will be issued in 2011/12. In 2008/09 there were no outstanding audit opinions at 30 June as a result of inaction on our part. 85% of the management reports were issued within six weeks. |
| Client satisfaction survey results show that, overall, 75% of respondents are satisfied with the quality of audit work (including the expertise of staff and the quality of the entities' relationships with their audit service provider) We achieved our forecast standard with an actual result of 90%. |
Percentage of clients satisfied with the quality of audit work for the five years from 2007 to 2011  |
| Quality assurance reviews for all appointed auditors are completed during a three-year period. Of the auditors reviewed in any given year, 95% achieve a result of satisfactory or better. We achieved our forecast standard with an actual result of 100%. |
Percentage of auditors achieving a grade of satisfactory or better from quality assurance reviews for the five years from 2007 to 2011  |
| An annual independent review of our processes confirms the probity and objectivity of the methods and systems we use to allocate and tender audits, and monitor the reasonableness of audit fees. The review was carried out in 2010/11 and we achieved our forecast standard with the confirmation being provided. |
An annual review was carried out each year and the confirmation provided. Sir David Gascoigne's report is on pages 35–38. |
| The Officers of Parliament Committee accepts any significant proposals for an appropriation increase in audit fees and expenses. Not applicable. |
2008, 2010, and 2011 – No significant proposal was made for an appropriation increase in audit fees and expenses. 2009 – A request for an increase in appropriation of $50,000 for audits of smaller entities was not accepted by the Officers of Parliament Committee. |
* We have excluded from this figure all uncompleted school audits at 30 June 2011 that were exempted from the statutory deadline following the major earthquake on 22 February 2011.
Overall comments on our performance in carrying out annual audits and other audits and assurance services
We are generally pleased with the trends in our output measures and standards as follows:
- There was a minor increase in the number of audits in arrears at 30 June, which wasn't unexpected given the disruption caused by the recent earthquakes, while the school audit arrears figures have been adjusted to exclude uncompleted audits for exempt schools. The earthquakes have affected the ability of our audit service providers to complete other audits and the figures have not been adjusted to reflect this disruption.
- There was a significant decline in the percentage of audit arrears being due to inaction on our part, which reflects an increased focus by auditors in this area.
- There was a slight reduction in the timeliness of issuing management reports to public entities following the completion of our audits, against an ambitious target of 100%.
- There was an increase in an already high level of client satisfaction as expressed by a sample of public entities.
- There was maintenance of a high percentage of appointed auditors receiving a quality assurance grade of "satisfactory" or better.
- There was confirmation from an independent review that our processes for audit appointments and audit fee monitoring have been carried out with due probity and objectivity.
During the year, we focused on the following developments:
- The Ministry of Economic Development and External Reporting Board (XRB) changes, with the XRB taking over responsibility for setting financial reporting standards from the New Zealand Institute of Chartered Accountants as from 1 July 2011. The XRB has indicated that, in future, financial reporting standards will be developed using a multi-standards approach. We support this change in standard-setting responsibility and change in approach because we consider that these changes are likely to better meet the needs of users of public entity reports.
- Strengthening of auditor regulation as, in future, all auditors of issuers will be subject to regulation by the Financial Markets Authority (FMA). However, to protect the independence of the Auditor-General, audits of issuers carried out under the Public Audit Act 2001 are not subject to the new regulations. Nevertheless, the Auditor-General intends to implement processes which are equivalent to the new regulations, including inviting the FMA to carry out quality reviews of issuer audits carried out on behalf of the Auditor-General.
- Continuing to phase in the Auditor-General's revised auditing standard on reporting performance information. The 2010 audits of local authorities were carried out in accordance with the new standard, and the 2011 audits of major government departments and Crown entities are being carried out in accordance with the new standard. The new standard requires the auditor to increase focus on the appropriateness of the information presented by each public entity rather than focus primarily on verification of the information presented. The Auditor-General considers that such an approach is likely to increase the relevance and understandability of the information presented by public entities, and there is evidence that this is already occurring.
Independent review of audit allocation and fee setting and monitoring
- Our independent reviewer has again assessed our audit allocation and fee-setting and monitoring systems (the full report is included at pages 35–38). The reviewer concluded that the processes for allocating audits in the public sector and for setting fees in the financial year to 30 June 2011 have been carried out with due probity and objectivity.
- During 2010/11, we started a comprehensive review of the allocation of audits among the Auditor-General's audit service providers, including Audit New Zealand. We expect that this will result in some audits being reallocated between audit service providers. If a change is proposed, we will consult with those public entities affected.
Audit fees
- There continues to be some pressure on audit fees. During 2010/11, fees have again been affected by entity-specific issues, in addition to some pressure on average charge-out rates as a result of auditors using a more senior team mix coupled with the return of some salary-driven pressure as the market for accountants and auditors begins to free up again.
- Figure 6 summarises the movements in audit fees from 2008/9 to 2010/11 based on those audit fees that had been agreed at the time when the analysis was prepared. It shows the fee movement by sector and how much is due to movements in audit hours and how much is due to the average hourly cost of carrying out audits.
Figure 6
Analysis of movements in audit fees
| 2009/10 to 2010/11 | 2008/09 to 2009/10 | |||||||
|---|---|---|---|---|---|---|---|---|
| Number of entities | Increase in total fee | Due to hours | Due to charge-out rate | Number of entities | Increase in total fee | Due to hours | Due to charge-out rate | |
| Central government | 350 | 4.2% | 2.7% | 1.5% | 310 | 4.6% | 4.3% | 0.3% |
| Local government | 370 | 3.2% | 1.6% | 1.6% | 271 | 6.1% | 6.3% | (0.2%) |
| Schools | 2438 | 4.2% | 1.0% | 3.2% | 2446 | 7.6% | 4.8% | 2.8% |
| Total* | 3158 | 3.9% | 2.0% | 1.9% | 3027 | 5.8% | 5.1% | 0.7% |
* During the year, the Auckland "supercity" restructure resulted in the disestablishment of eight local authorities and a number of council-controlled organisations, and the establishment of Auckland Council and a number of new council-controlled organisations. Audits for a 16-month period were required for the disestablishing entities, and the first-year audits for the new entities are expected to be significantly more complex than normal. The fee movements in Figure 6 include the abnormal fees that result from those changes. If the entities affected by the changes were removed from the table for the 2010/11 year, the total fee movement for local government would be 4.4% (3.4% due to hours and 1.0% due to rates), and total fee movement would be 4.3% (2.6% due to hours and 1.7% due to rates).
Figure 7
Financial performance of output class: Audit and assurance services
| 2010/11 Actual $000 |
2010/11 Supp. Estimates $000 |
2009/10 Actual $000 |
2008/09 Actual $000 |
|
|---|---|---|---|---|
| Income | ||||
| Crown | 150 | 150 | 150 | 150 |
| Other | 65,194 | 64,797 | 63,939 | 60,483 |
| Expenditure | (65,043) | (64,947) | (62,736) | (60,602) |
| Surplus/(Deficit) | 301 | 0 | 1,353 | 31 |
Supporting accountability to Parliament
This output class includes two outputs:
- Services to Parliament – providing advice and assistance to select committees and other stakeholders; and
- Controller function – carrying out the Controller function.
Services to Parliament
Through our services to Parliament, we provide advice and assistance to select committees, Ministers, and individual members of Parliament, as well as to central agencies and other public sector representative groups, to assist them in their work to improve the performance and accountability of public entities.
The main ways in which this advice and assistance occurs is through:
- reports and advice to select committees to assist their financial reviews of government departments and Offices of Parliament, State-owned enterprises, and some Crown entities;
- reports and advice to select committees to assist their examination of the Estimates of Appropriations; and
- reports to responsible Ministers on the results of the annual audits.
- reporting to Parliament and other constituencies on matters arising from our annual audits (including at least two reports to Parliament on the results of our audits in central and local government);
- responding to requests and participating in working parties on matters related to financial management and accountability with other stakeholders, including government departments, central agencies, local authorities, professional bodies, sector organisations, and other public entities;
- working with Auditors-General in other countries to encourage, promote, and advance capability and co-operation in the field of public audit;
- supporting good governance in the Pacific through our role as Secretariat of the Pacific Association of Supreme Audit Institutions (PASAI), and the Auditor-General's representation of the PASAI region on the governing board of the International Organisation of Supreme Audit Institutions (INTOSAI);
- being executing agent for the Pacific Regional Audit Initiative (funded by the Asian Development Bank, with co-financing from the Japan Special Fund and the Government of Australia);
- being a member of various INTOSAI committees and working groups, including the Professional Standards Steering Committee (whose meeting we hosted in Wellington in June 2011), and chairing two working groups tasked with developing guidance for supreme audit institutions; and
- hosting international delegations, often at the request of Parliament or the Ministry of Foreign Affairs and Trade, and providing opportunities to exchange information and build networks internationally.
The Controller function
The Controller function of the Controller and Auditor-General provides independent assurance to Parliament that the expenses and capital expenditure of government departments and Offices of Parliament have been incurred for purposes that are lawful and within the scope, amount, and period of the appropriation or other authority.
The OAG and appointed auditors carry out standard procedures to give effect to the Controller function in keeping with the Auditor-General's auditing standards and a Memorandum of Understanding with the Treasury. This involves reviewing monthly reports provided by the Treasury, and advising the Treasury of any issues arising and the action to be taken.
Each year, we report to Parliament on the significant issues arising from the operation of the Controller function.
Figure 8
Supporting accountability to Parliament – main impact measures and standards
| 2010–13 main impact measures and standards | Actual |
|---|---|
| Select committees confirm that the Office's advice assists them in Estimates of Appropriation and financial review examinations. The actual result for 2010/11 was 86%. |
Percentage of select committee members who confirmed that our advice assisted them in Estimates of Appropriations and financial review examinations for the five years from 2007 to 2011  |
| Expenses and capital expenditure of departments and Offices of Parliament are incurred for purposes that are lawful and within the scope, amount, and period of the appropriation or other authority. | The operation of the monthly Controller process and the appropriation audit were carried out to ensure that this measure as achieved each year for the five years from 2007 to 2011. |
Figure 9
Services to Parliament – output measures and standards
| 2010/11 measures and standards of output delivery (and forecast for 2011/12 and 2012/13) | Actual |
|---|---|
| Reports and advice are given to select committees and Ministers at least two days before an examination, unless otherwise agreed. We achieved our forecast standard with an actual result of 100%. |
Percentage of reports and advice given to select committees and Ministers at least two days before an examination, unless otherwise agreed, for the five years from 2007 to 2011  |
| An internal review of a sample of financial review, Estimates, and Ministerial reports confirms that they meet relevant standards and procedures, including that reports are consistent in their framework and approach and are peer reviewed in draft. (The nature, extent, and frequency of the quality assurance review are based on risk. The review is carried out during a three-year period.) Not applicable. |
A review undertaken in June 2011 confirmed that the appropriate systems are in place and they operate effectively 2009 and 2010 – there was no internal review in these years. 2008 – confirmed internal review of a sample of reports. |
At least 85% of select committee members we survey rate the advice they receive from us as 4 or better on a scale of 1 to 5 for:
|
Percentage of select committee members who rated our advice as 4 or better on a scale of 1 to 5 for quality and usefulness for the five years from 2007 to 2011  |
| At least 85% of other stakeholders we survey rate the advice they receive from us as 4 or better on a scale of 1 to 5 for relevance and usefulness. We did not achieve our forecast standard with an actual result of 75%. |
Percentage of other stakeholders who rated our advice as 4 or better on a scale of 1 to 5 for relevance and usefulness for the five years from 2007 to 2011  |
We reported on the results of our 2009/10 annual audits to Parliament, presenting our central government results in two volumes in December 2010 and March 2011. For local government, we reported on the results of our annual audits in April 2011 and on the results of our Long-Term Council Community Plan audits in August 2010.
Our results for this output class show that our work continues to play an important part in supporting accountability to Parliament. This is pleasing, given the continuing high demand for advice to select committees. Our stakeholder feedback report notes that the feedback received and the ratings show that the Office is continuing to perform at a high level, with stakeholders valuing the professional and impartial advice and guidance that they receive from the Office.
Select committee members continued to rate the quality and usefulness of our advice at 86% – similar to the ratings we have had for the previous three years. The same percentage of select committee members surveyed also agree that the Office's advice assists in their Estimates of Appropriation and financial review examinations. However, this is a decrease from 100% in the previous three years. We also saw a decline in the rating of other stakeholders for the relevance and usefulness of our advice. The small number of stakeholders we survey does lead to significant shifts in percentage terms from year to year. For each of these two instances, all but one stakeholder rated us 4 or better on a scale of one to five.
The feedback from our stakeholder interviews shows that the Office is continuing to perform at a high level. We received useful improvement suggestions from our stakeholders, the main themes of which were to continue to offer more insight and advice from what we learn through our annual and discretionary work to drive performance improvement in the public sector.
This feedback aligns well with our strategic initiative over the next three years to focus on good analysis and reporting of sector information. We will be carrying out work to better identify, categorise, and manage the flow of information from annual audits into our services to Parliament and carrying out performance audits and inquiries. We want to communicate to Parliament and our other stakeholders what we learn from our annual and discretionary work, so they can get the best from what we know and respond to issues and concerns as they see them.
We will also take account of our stakeholder feedback in the review we are doing in the first half of 2011/12. We are reviewing our various advisory products to select committees to ensure that they remain relevant and useful, and achieve their purpose of supporting accountability to Parliament.
Figure 10
The Controller function – output measures and standards
| 2010/11 measures and standards of output delivery (and forecast for 2011/12 and 2012/13) | Actual |
|---|---|
| Monthly statements provided by the Treasury are reviewed for the period September to June inclusive.* Advice of issues arising and action to be taken is provided to the Treasury and appointed auditors within five working days of receipt of the statement. We achieved our forecast standard. |
2010/11 – all monthly procedures have been followed and agreed time frames achieved. 2006-10 – all monthly procedures have been followed and agreed time frames achieved. |
| Internal quality assurance is undertaken to gain assurance that our policies, procedures, and standards in relation to the Controller function have been applied appropriately. (The nature, extent, and frequency of the quality assurance review are based on risk. The review is carried out during a three-year period.) Not applicable. |
No internal review was undertaken in 2010/11. 2008/09 – An internal review was carried out in May 2009, which confirmed that the control work carried out was consistent with the Memorandum of Understanding and that the monthly processes operated effectively. There was considerable improvement in the appropriation audit approach and documentation to demonstrate compliance with the Auditor-General's auditing standard AG-2. |
* This action is based on the requirement in section 65Y of the Public Finance Act 1989 for the Treasury to submit reports to the Auditor-General following the preparation (under section 31A) of financial statements of the Government "after the end of each month (except for the first 2 months and the last month) in each financial year".
Figure 11
Financial performance of output class: Supporting accountability to Parliament
| 2010/11 Actual $000 |
2010/11 Supp. Estimates $000 |
2009/10 Actual $000 |
2008/09 Actual $000 |
|
|---|---|---|---|---|
| Income | ||||
| Crown | 2,460 | 2,460 | 2,460 | 2,460 |
| Other | 0 | 0 | 0 | 7 |
| Expenditure | (2,442) | (2,460) | (2,342) | (2,457) |
| Surplus/(Deficit) | 18 | 0 | 118 | 10 |
Performance audits and inquiries
The Public Audit Act 2001 provides the Auditor-General with discretion to carry out performance audits to look at:
- the extent to which activities are carried out effectively and efficiently;
- compliance with statutory obligations;
- any acts or omissions to determine whether waste has resulted or may result; and
- any act or omission showing or appearing to show a lack of probity or financial prudence by a public entity or its members, office holders, or employees.
Each year, we usually publish 19 to 21 reports on performance audits, inquiries, and good practice guides.
The Act also gives the Auditor-General the ability to inquire into a public entity's use of its resources.
Our inquiry work is largely reactive to issues of public concern. Each year, we usually receive:
- 200 to 300 requests for inquiries; and
- 50 to 100 enquiries under the Local Authorities (Members' Interests) Act 1968.
The Auditor-General must also respond to requests for approvals in relation to pecuniary interest questions regulated by the Local Authorities (Members' Interests) Act.
Our performance audit and inquiry work allows the Auditor-General to consider and provide advice about the above matters in greater depth than is appropriate within the statutory scope of an annual audit.
Figure 12
Performance audits and inquiries – main impact measures and standards
| 2010-13 main impact measures and standards | Actual |
|---|---|
| Entities accept or respond to the recommendations made in our performance audits, as assessed by internal review of performance audits published in the previous year. The results are presented to the Officers of Parliament Committee in our annual follow-up report. | 2010/11 – In April 2011, we reported to Parliament on how well five public entities have implemented our recommendations. The entities had generally accepted our recommendations, but progress in implementing them had been faster for some recommendations than others. For those entities, and for other public entities, communicating results is an ongoing challenge. A selection of our performance audit reports were reviewed each year from 2006 to 2010, and the results were presented to the Officers of Parliament Committee. The reviews concluded that our recommendations had been accepted by the relevant public entities and either had been implemented or were being implemented. |
| Entities take action in response to concerns identified in inquiry reports, as assessed by follow-up on a sample of significant and major inquiries undertaken in the previous year. | 2011 – We followed up on three inquiries. In all cases, the entity accepted our comments and took action to address the concerns. 2010 – We followed up on four of our 13 inquiries within these categories from the previous year that contained recommendations or suggestions for action. There was a high acceptance of our views and comments, which were contributing to significant change. 2009 – We followed up on four of the 11 inquiries within these categories from the previous year that contained recommendations or suggestions for action. In all instances, we were satisfied with the action taken. |
Performance audits
A performance audit is a significant and in-depth audit covering issues of effectiveness and efficiency. It provides Parliament with assurance about specific issues or programmes and how well these are managed by the relevant public entity or entities. We also do other studies that may result in published good practice guidance on topical issues of public sector accountability and performance to assist public entities to better manage these issues.
Figure 13
Performance audits – output measures and standards
| Measures and standards of output delivery for 2010/11 (and forecast for 2011/12 and 2012/13) | Actual |
|---|---|
| We complete 19 to 21 reports on matters arising from performance audits and other studies, and inquiries. We achieved our forecast standard. The actual result for 2010/11 was 26. |
Completed reports on matters arising from performance audits and other studies, and inquiries from 2007 to 2011 |
| Select committees and other stakeholders are satisfied with the proposed work programme of performance audits (as indicated by feedback on our draft annual work programme). We achieved our forecast standard |
2011 – Feedback received from select committees and other stakeholders mainly supported our proposals and approach, with a specific suggestion made for future years being the performance of the New Zealand Police in road safety enforcement. 2007 to 2010 – Feedback received from select committees and other stakeholders mainly supported our proposals and approach. |
At least 85% of the stakeholders that we seek feedback from rate our performance audit reports (relevant to their sector or interest) as 4 or better on a scale of 1 to 5 for:
We did not achieve our forecast standard for usefulness, with an actual result of 75%. |
Percentage of select committee, local government, and other stakeholders who are satisfied with the quality and usefulness of our performance audit reports for the five years from 2007 to 2011  |
| Our performance audit methodology reflects good practice for undertaking such audits, as assessed every second year by the Australian National Audit Office. We achieved our forecast standard. |
2010/11 – The Australian National Audit Office (ANAO) reviewed two performance audits and found that they met the majority of the review criteria and were generally conducted in keeping with key OAG processes and practices. 2008/09 – The ANAO reviewed two performance audits and confirmed areas in which the quality of our reports is strong and areas for us to improve. 2006/07 – The ANAO reviewed two performance audits and endorsed the quality of the reports. |
| Each year, independent reviews of two performance audits are undertaken. These reviews confirm the quality of the reports in terms of the presentation of administrative and management context, report structure, presentation, and format (including use of graphics and statistics), and the reasonableness of the methodology used and the resulting conclusions and recommendations. We achieved our forecast standard. |
2010/11 – Two of our reports were independently reviewed by a panel of reviewers used by the Australasian Council of Auditors-General (ACAG) and two New Zealand reviewers. Overall, the reviewers' ratings showed that one report was rated higher than reports from other audit offices that the ACAG panel had reviewed over the past three years, and higher than the average for our reports reviewed over the past three years. The other report was rated below the average for reports from other offices and below the average score for our reports. 2007 to 2010 – Independent reviews of two performance audits confirmed the quality of reports, and provided feedback on areas for us to improve. |
| Internal quality assurance reviews on selected performance audit reports confirm that reports are prepared in keeping with the performance audit methodology. (The nature, extent, and frequency of the quality assurance review are based on risk. The review is carried during a three-year period.) We achieved our forecast standard. |
Internal review in 2010/11 confirmed that appropriate systems and controls are in place and that reports are prepared in keeping with the performance audit methodology. There was no internal review in 2008/09 or 2009/10. Internal review in 2007/08 confirmed that appropriate systems and controls are in place and that reports are prepared in keeping with the performance audit methodology. |
Reports on performance audits, other studies, and major inquiries completed during 2010/11
In 2010/11, we completed 26 reports on matters arising from performance audits, other studies, and major inquiries. Each of the reports that we complete is unique and requires a different level of time and resource, so the number of reports that we produce can be expected to fluctuate from year to year.
The 26 reports we completed during 2010/11 cover a broad range of issues and different types of report, including audit and inquiry reports, reports promoting good practice and learning from experience, a checklist for use by school boards of trustees and principals, and guidance to members of local authorities. We presented reports in two of our areas of key focus:
- Auckland – on the transition to the amalgamated Auckland Council; and
- performance reporting – on examples of better practice in forecasting and on how cost-effectiveness can be demonstrated in annual reporting.
The reports are listed in Figure 14. Appendix 1 on pages 103–114 summarises each of these reports. Copies of published reports are available on our website: www.oag.govt.nz.
Figure 14
Performance audits, other studies, and major inquiries completed in 2010/11
| Performance audits and other studies from our 2010/11 annual plan | |
| 1 | Spending on supplies and services by district health boards: Learning from examples (published in September 2010) |
| 2 | Matters arising from Auckland Council's planning document (published in December 2010) |
| 3 | Public entities' progress in implementing the Auditor-General's recommendations (published in April 2011) |
| 4 | Inland Revenue Department: Making it easy to comply (published in June 2011) |
| 5 | Central government: Cost-effectiveness and improving annual reports (published in June 2011) |
| 6 | Final audits of Auckland's dissolved councils, and managing leaky home liabilities (published in June 2011) |
| Performance audits and other studies from previous annual plans | |
| 7 | Inland Revenue Department: Managing child support debt (published in July 2010) |
| 8 | District health boards: Availability and accessibility of after-hours services (published in September 2010) |
| 9 | New Zealand Transport Agency: Information and planning for maintaining and renewing the state highway network (published in September 2010) |
| 10 | Department of Internal Affairs: Administration of two grant schemes (published in November 2010) |
| 11 | Defence Major Projects Report (published in November 2010 by the Ministry of Defence and New Zealand Defence Force – overview and opinion by the Auditor-General) |
| 12 | Sport and Recreation New Zealand: Improving how it measures its performance (published in December 2010) |
| 13 | Ministry of Social Development: Managing the recovery of debt (published in April 2011) |
| 14 | Progress in delivering publicly funded scheduled services to patients (published in June 2011) |
| Major inquiries and other studies not in our 2010/11 annual plan or previous annual plans | |
| 15 | Inquiry into New Zealand Defence Force payments to officers seconded to the United Nations (published in July 2010) |
| 16 | Inquiry into the Plumbers, Gasfitters, and Drainlayers Board (published in August 2010) |
| 17 | Matters arising from the 2009-19 long-term council community plans (published in August 2010) |
| 18 | Effectiveness of the Get Checked diabetes programme (published in September 2010) |
| 19 | Guidance for members of local authorities about the local authorities (Members' Interests) Act 1968 (published in October 2010) |
| 20 | Inquiry into payments to chief executives of dissolving local authorities in Auckland (published in October 2010) |
| 21 | How the Department of Internal Affairs manages spending that could give personal benefit to Ministers (published in December 2010) |
| 22 | Provision of billboard for Len Brown's Mayoral Campaign (published in February 2011) |
| 23 | Effective reporting on students' progress and achievement (published in February 2011) |
| 24 | Central government: Case studies in reporting forecast performance information (published in February 2011) |
| 25 | District Health Boards: Learning from 2010-13 Statements of Intent (published in February 2011) |
| 26 | Review of the Northland Events Centre (published in May 2011) |
Progress with completion of our 2010/11 annual plan
On pages 65-70 of our Annual Plan 2010/11, we listed the 19 performance audits and other studies that we proposed to start in 2010/11. Our actual work programme varies from that planned in response to changing priorities, such as urgent work on new inquiries, and changes in government policy or entity circumstances affecting the timing or relevance of audits. As at 30 June 2011, we had completed five of the 19 proposed performance audits and other studies. Another 11 were at various stages of completion and scheduled to be published in 2011/12. We have deferred the other three proposed performance audits. Figure 17 shows details of the progress we have made in completing the proposed work programme set out in our 2010/11 annual plan.
Figure 15
Progress with completion of our 2010/11 annual plan
| Performance audit or study | Status | References and notes | |
|---|---|---|---|
| 1 | ONTRACK follow-up audit: Maintaining and renewing the rail network | Under way | Scheduled to be published in 2011/12 |
| 2 | Freshwater quality in New Zealand: Effectiveness of management responses | Completed after 30/6/11 | Managing freshwater quality: Challenges for regional councils (published in September 2011) |
| 3 | New Zealand Transport Agency: State highway maintenance – Part 2 | Completed after 30/6/11 | New Zealand Transport Agency: Delivering maintenance and renewal work on the state highway network (published in September 2011) |
| 4 | Transpower: Managing the national grid | Completed after 30/6/11 | Transpower New Zealand Limited: Managing risks to transmission assets (published in October 2011) |
| 5 | Housing New Zealand Corporation: Maintenance of state housing follow-up | Deferred | To consider ongoing relevance of this audit, given changes in the sector |
| 6 | Accident Compensation Corporation: Management of funds | Deferred | To consider ongoing relevance of this audit, given changes in the sector |
| 7 | New Zealand Customs Service: Effectiveness of the customs revenue assurance strategy and programme | Completed after 30/6/11 | New Zealand Customs Service: Providing assurance about revenue (published in July 2011) |
| 8 | Quality assurance and administration of internally assessed standards for the National Certificate of Educational Achievement | Under way | Scheduled to be published in 2011/12 |
| 9 | Ministry of Health and district health boards: Provision of home-based support services for older people | Completed after 30/6/11 | Home-based support services for older people (published in July 2011) |
| 10 | The role of teacher registration and initial teacher education course approval processes in ensuring the quality of teaching in New Zealand schools | Under way | Scheduled to be published in 2011/12 |
| 11 | Management of care and protection services for children and young people in care | Deferred | To consider ongoing relevance of this audit, given changes in the sector |
| 12 | District Health Boards: Contract management practice | Completed in 2010/11 | Spending on supplies and services by district health boards: Learning from examples (published in September 2010) |
| 13 | Review of reported performance information | Completed in 2010/11 Completed after 30/6/11 |
Central government: Cost-effectiveness and improving annual reports (published in June 2011) Local government: Improving the usefulness of annual reports (published in September 2011) |
| 14 | The effectiveness of responses to Māori housing needs and demand | Completed after 30/6/11 | Government planning and support for housing on Māori land (published in September 2011) |
| 15 | The demand for water – Auckland region | Completed | Planning to meet the forecast demand for drinking water in Auckland (published in August 2011) |
| 16 | Matters arising from our audit of the Auckland planning document | Completed in 2010/11 | Matters arising from Auckland Council's planning document (published in December 2010) |
| 17 | Observations on the audits of Auckland councils being dissolved. | Completed in 2010/11 | Final audits of Auckland's dissolved councils, and managing leaky home liabilities (published in June 2011) |
| 18 | The Treasury: Administration of the Retail Deposit Guarantee Scheme | Completed after 30/6/11 | The Treasury: Implementing and managing the Crown Retail Deposit Guarantee Scheme (published in October 2011) |
| 19 | Inland Revenue Department: The voluntary compliance strategy. | Completed in 2010/11 | Inland Revenue Department: Making it easy to comply (published in June 2011) |
There are four other pieces of work from previous annual plans that we have decided to defer pending further consideration of changes in the respective sectors and whether our proposed audits remain relevant and valuable. These are work on:
- Department of Corrections: Managing prisoner employment;
- Department of Building and Housing: Effective management of tenancy services;
- Tertiary Education Commission: Monitoring of tertiary education institutions; and
- Ministry of Education: Effective management of the Crown's financial interests in integrated schools.
We have also decided to cancel a proposed performance audit of social marketing campaigns as our preliminary inquiries found that that there were no significant issues warranting a performance audit of this topic.
The relevance and usefulness of our work programme
We consulted with Parliament and other stakeholders to ensure that our programme of proposed performance audits and other studies would be relevant and useful to Parliament, public entities, and the public. The feedback we received mainly supported the proposals we made, with a specific suggestion made for future years to review the performance of the New Zealand Police in road safety enforcement.
The strength of our methodology
Every second year, our performance audit methodology is assessed by the Australian National Audit Office (ANAO). The ANAO reviewed two of our performance audits in 2010/11 and found that they met most of the review criteria and were generally conducted in keeping with key OAG processes and practices. The review team identified clear reporting, strong analysis and evidence collection, and sound planning as our strengths. They also identified that we could improve the structure and content of the Auditor-General's overview, presentation of the scope of audits, and how we conduct multi-entity audits. We are encouraged by the generally positive findings from the ANAO's review, and will consider and adopt useful suggestions as part of continuing improvements to our performance audit work.
We also conduct internal quality assurance reviews on selected performance audit reports to confirm that reports are prepared in keeping with our performance audit methodology. The internal review in 2010/11 confirmed that appropriate systems and controls are in place to mitigate identified potential risks in conducting performance audits, and that reports are prepared in keeping with our performance audit methodology.
The quality and usefulness of our reports
Each year, we seek the views of a small sample of select committee chairpersons, deputy chairpersons, and other stakeholders on the quality and usefulness of the performance audit reports that we have published. For 2010/11, we are pleased that 88% of the stakeholders we surveyed rated the quality of our performance audits highly, an increase from last year and above our target of 85%. The proportion of stakeholders rating the usefulness of our performance audit reports highly has dropped to 75% in 2010/11 from 100% the year before, falling below our target of 85%. One person thought that we could carry out wider benchmarking of public sector agencies against the private sector.
Overall, the proportion of stakeholders rating both the quality and usefulness of our performance audit reports highly has fluctuated during the last five years. The extent of the fluctuation is partly a reflection of the relatively small number of stakeholders that we survey each year, which means that one or two rating changes can affect the percentages significantly. We are encouraged that the trend during the last four years is that stakeholders continue to rate the quality and usefulness of our performance audit reports highly, and our aim is to consistently meet or exceed our targets.
Each year, we also have two of our performance audit reports independently reviewed to assess the quality of the reports in terms of: the presentation of administrative and management context; report structure, presentation, and format (including use of graphics and statistics); and the reasonableness of the methodology used and the resulting conclusions and recommendations. For 2010/11, two of our reports were independently reviewed by a panel of reviewers used by the Australasian Council of Auditors-General (ACAG) and a New Zealand reviewer.
Overall, the reviewers' ratings showed that our progress report on the Civil Aviation Authority was rated higher than reports from other audit offices that the ACAG panel had reviewed over the past three years, and higher than the average for our reports reviewed over the past three years. The other report was rated slightly below the average for reports from other offices and below the average for our reports.
Many of the reviewers commented on the overall quality and value that the reports represented. They also noted that, overall, the reports were well presented and easy to understand. Other strengths noted were that:
- the purpose of the reports and their scope was clear;
- the findings of the reports were clearly tied in to the evidence that was presented, and good analysis was used to present conclusions; and
- the reports used appropriate methodologies to answer the audit questions.
The reviewers noted the following areas where we could improve our reports:
- the degree of seriousness and risk around some of the findings could be made clearer and we could better explain the scale of problems that we identify;
- our reports could include cost-benefit analysis applied to the recommendations; and
- we could refer more to best practice overseas.
Overall, we were pleased by the positive and useful feedback provided from the reviews, which we will incorporate as part of continuing improvements to our performance audit work. We are committed to adding value through insightful assessment of the effectiveness and efficiency of the activities we report on and to continuously strengthening the depth and relevance of our performance audits and other studies.
Inquiries
The Auditor-General has the discretion to inquire into a public entity's use of resources. The Auditor-General can carry out inquiries on her own initiative and when correspondence from the public draws attention to potential issues. A few such issues lead to major inquiries. We also administer the Local Authorities (Members' Interests) Act 1968, which governs the financial interests of members of local authorities.
Each year, we usually receive:
- 200 to 300 requests for inquiries; and
- 50 to 100 enquiries under the Local Authorities (Members' Interests) Act.
- We received slightly fewer requests for inquiries during 2010/11 – 196 new requests for inquiries and 89 requests on Members' Interests Act matters. We had carried forward a total of 23 requests from the previous year, and ended the year carrying forward 18 requests. In terms of workflow, we usually have about 20 inquiry matters open at any point in time.
Figure 16
Number and type of inquiries completed
| 2010/11 | 2009/10 | 2008/09 | 2007/08 | |
|---|---|---|---|---|
| Major | 3 | 5 | 2 | 0 |
| Significant | 10 | 16 | 11 | 11 |
| Routine | 144 | 182 | 99 | 115 |
| No inquiry | 41 | 66 | 142 | 116 |
| LAMIA* | 91 | 80 | 53 | 110 |
| Total | 289 | 349 | 307 | 354 |
* Local Authorities (Members' Interests) Act 1968.
Our performance
- As signalled in last year's annual report, the number of larger and more complex inquiries that we have dealt with in recent years had placed some strain on our resources and affected our timeliness. In particular, when we began the year, we were still working on some longstanding major and significant inquiries. We completed the two longstanding major inquiries in July and August 2010 – Inquiry into New Zealand Defence Force payments to officers seconded to the United Nations and Inquiry into the Plumbers, Gasfitters, and Drainlayers Board – and concluded the various significant inquiries later in the year. We maintained the flow of routine work throughout the year, and have been able to complete new inquiries within our timeliness targets. By 30 June 2011, we had largely "caught up", and the volume of work at present is manageable.
- We hope to improve our performance further in the coming year. To this end, we have made a number of changes to our internal processes to streamline our work, and have revised our public information on what we do and do not look at in inquiries to explain more clearly to people when it is appropriate to come to us and what to expect.
Figure 17
Inquiries – output measures and standards
| Measures and standards of output delivery for 2010/11 (and forecast for 2011/12 and 2012/13) | Actual |
|---|---|
80% of our findings on inquiries are reported to the relevant parties within:
|
Percentage of findings on routine inquiries reported to relevant parties within three months for the five years from 2007 to 2011  Percentage of findings on significant inquiries reported to the relevant parties within six months for the four years from 2008 to 2011  Results for major inquiries: 2008 – No major inquiries were carried out. 2009 – Two major inquiries; both reported within 13 months. 2010 – Five major inquiries; four reported within 12 months. 2011 – Three major inquiries; one reported within 12 months. |
| For enquiries under the Local Authorities (Members' Interests) Act 1968, we complete 80% of enquiries within 30 working days. We achieved our forecast target with an actual result of 88%. We completed 80 of 91 requests within the target time frame. |
Percentage of enquiries under the Local Authorities (Members' Interests) Act 1968 completed within 30 working days for the five years from 2007 to 2011  |
| Responses to requests for inquiries, and our administering of the Local Authorities (Members' Interests) Act 1968 requests, are in accordance with relevant policies, procedures, and standards, as confirmed by internal quality assurance review. (The nature, extent, and frequency of the quality assurance review are determined based on risk. The review is carried out during a three-year period.) |
2010/11 - Reviews were completed for both the general inquiries and Members' Interests inquiries systems, and confirmed that requests are undertaken in accordance with relevant policies, procedures, and standards. No internal quality assurance review was done in 2009/10 No internal quality assurance review was done in 2008/09. |
We completed and presented to Parliament the following reports on major inquiries during 2010/11:
- Inquiry into the New Zealand Defence Force payments to officers seconded to the United Nations (July 2010);
- Inquiry into the Plumbers, Gasfitters, and Drainlayers Board (August 2010); and
- How the Department of Internal Affairs manages spending that could give personal benefit to Ministers (December 2010).
In each case, we identified significant problems. We intend to follow up on each of these reports to assess progress with remedying the problems we identified, once the relevant agencies have had a reasonable opportunity to make changes.
We also completed and published our findings on several significant local government issues that attracted attention:
- Inquiry into payments to chief executives of dissolving local authorities in Auckland (October 2010);
- Provision of billboard for Len Brown's Mayoral Campaign (February 2011); and
- Review of the Northland Events Centre project (May 2011).
The Local Authorities (Members' Interests) Act 1968
In late 2010, our work relating to the Members' Interests Act was dominated by the local authority elections (held in October 2010). Much of the increase in the number of issues we dealt with was caused by election-related queries, because the Act contains some complex rules that limit eligibility to take office. We also contributed a segment on the Act to the training for new members that Local Government New Zealand provided around the country.
In our last annual report, we recorded that we had written a letter, jointly with the president of Local Government New Zealand, to the Minister of Local Government setting out our concerns with the Act and asking for it to be reviewed. We are pleased to note that the Minister has recently initiated a full review and published a discussion paper on options for reform, which was released on 24 August 2011. We will co-operate closely with the Department of Internal Affairs as the review progresses.
Figure 18
Financial performance of output class: Performance audits and inquiries
| 2010/11 Actual $000 |
2010/11 Supp. Estimates $000 |
2009/10 Actual $000 |
2008/09 Actual $000 |
|
|---|---|---|---|---|
| Income | ||||
| Crown | 6,587 | 6,587 | 6,587 | 6,587 |
| Other | 0 | 0 | 0 | 17 |
| Expenditure | (5,991) | (6,587) | (6,044) | (6,159) |
| Surplus/(Deficit) | 596 | 0 | 543 | 445 |
3: "Material" as defined in the Auditor-General’s Statement on International Standard on Auditing (New Zealand) 320: Materiality in Planning and Performing an Audit, "means, for the purposes of forming an opinion on a public entity’s financial and non-financial information, a statement, omission, fact, or item of such a nature or amount that its disclosure, or the method of treating it, given full consideration of the circumstances applying at the time the financial and non-financial information is completed, has the potential to influence likely users of the financial and non-financial information in making decisions or assessments about the stewardship and allocation of resources, and the performance of the public entity."
4 : Including performance audits, inquiries, and other auditing services.
page top