Part 22: Independent review of the work of Audit New Zealand

How the review was carried out

22.1
When KDC asked the Auditor-General to begin this inquiry, it was clear that any examination of what happened at KDC and with the wastewater project also needed to examine the role played by KDC's auditors. Understanding this aspect was also vital from the Auditor-General's perspective, given that she is the responsible statutory officer who appoints providers to carry out audit work on her behalf.

22.2
All organisations review aspects of their own work from time to time to support ongoing improvement. The challenge for this part of the inquiry was to ensure that the review of audit work was carried out independently, so that it would be publicly credible.

22.3
To that end, the terms of reference for the inquiry set out that the Auditor-General would appoint an independent person to carry out the review of audit work. We commissioned Mr Neil Cherry for this work.¹ Mr Cherry worked for Audit New Zealand at an early stage in his career (as have most auditors with public sector audit experience). We therefore asked Mr Des Pearson, the former Auditor-General of Victoria, Australia, to act as a peer reviewer for Mr Cherry.

22.4
Appendix 6 includes the terms of reference for the review. The terms of reference required the review to take a comprehensive approach, by looking at 10 years of annual audits of KDC and three audits of KDC's LTPs.

22.5
Mr Cherry describes how he carried out the review in his report, which is published in full as Appendix 6. In summary, the review:

• gathered background information on KDC and the issues of concern to the inquiry;
• for each audit being reviewed, obtained the relevant accounting and auditing standards, internal methodology documents, and other guidance that was applicable at the time;
• reviewed all the audit files in detail and interviewed relevant staff;
• prepared a draft report;
• sought comment from Audit New Zealand and the relevant individual staff members;
• revised the draft report in light of the comments received; and
• circulated a final draft for further comment from Audit New Zealand, individual staff members, and Mr Pearson as peer reviewer.

22.6
Mr Cherry provided the penultimate draft report to the inquiry team on 1 November 2013, so that we could prepare to incorporate it into this final report. He provided us with his final report on 15 November 2013.

What the review found

22.7
The full report is published as Appendix 6. It is a substantial and technical document, so we summarise it briefly here.

The purpose of an audit

22.8
The review emphasises the need to understand the role of an auditor compared with the responsibilities of the entity being audited. The entity is responsible for complying with legislative and accountability requirements. It must maintain financial and other records so that it can accurately meet its reporting obligations. The auditor's role is to provide an independent opinion on whether the information reported by the entity is reasonably reliable.

22.9
Local authorities are required to have specific information in their annual reports and LTPs audited. The purpose of the audit is to enhance the degree of confidence that readers can have in the audited information.

22.10
In technical language, the auditor's objective is to obtain reasonable assurance that the audited information:

• is free from any material misstatement; and
• is presented in a manner consistent with generally accepted accounting practice.

22.11
The auditor then provides an opinion on whether the information meets those standards.

22.12
"Reasonable assurance" does not mean that the auditor has tested every transaction or disclosure, or looked at every aspect of the local authority's operations and activities. Auditors assess whether particular activities, projects, or transactions have a direct material effect on the information being audited and plan their work accordingly.

Audit systems and methodologies

22.13
Overall, the review concluded that the audit methodologies and approaches developed by Audit New Zealand and/or the OAG provided a reasonable and appropriate basis for an auditor to plan and perform audit engagements. At all times, the audit methodology aligned with the requirements of the applicable auditing and assurance standards.

22.14
The review also noted that there were three separate independent quality reviews of Audit New Zealand's systems and methodology during the 10 years under review. Two were carried out by the New Zealand Institute of Chartered Accountants, and one was carried out by the Australasian Council of Auditors General. These reviews were similar in scope to those carried out on other chartered accounting practices. None of those reviews identified any significant issues or deficiencies.

Audits carried out between 2003 and 2005

22.15
Between 2003 and 2005, the auditor carried out an annual assessment of the management of the wastewater project, to help plan the audit work. Mr Cherry's review considered that this work was too narrow to be a complete assessment of the project. The review also considered that this work did not give enough attention to understanding the funding and financial implications and the Council's overall governance and risk management. However, the review accepted that this limitation did not affect how the audit was planned and performed, given that the project was still being developed during those years. The auditor was still in a position to properly express an opinion on the information in the annual report.

22.16
Mr Cherry's review criticises the sufficiency of documentation in the audit files for these years. Auditors are required to document the evidence and basis for any professional judgements they make during their audit work. The review found that, in many cases, there was not enough documentary evidence in the file to show the testing that had been done and to support the judgements that had been made.

22.17
In particular, the review noted that there was not enough documentation to explain how or why the auditor assessed the management control environment as good or effective during these years. The review concluded that the auditor's consideration of the financial management environment had been too narrow and relied too much on discussions with management during these years.

22.18
Overall, the review concluded that the audits carried out during these years satisfied the audit objective: the auditor had obtained reasonable assurance that the audited information met the required standards.

Audits carried out between 2006 and 2009

22.19
Mr Cherry's review found that, between 2006 and 2009, problems continued with the adequacy of documentation in the audit files to support the work done and judgements reached. Problems also continued with the assessment of the management control environment within KDC.

22.20
Given what was happening with the wastewater project during these years, the project should have been a major focus for the auditor. However, the auditor relied on the assessment of the wastewater project that had been carried out in 2005. The review found no evidence that the auditor updated their understanding of the project. The files show that the auditor did not identify any significant audit risk associated with the wastewater project during these years. The auditor did not adequately assess the project's potential effect on KDC or the audit engagement. Nor did the auditor give any independent consideration to the accounting treatment for the project.

22.21
The review concluded that the auditor inappropriately relied on KDC's management as a primary source of audit evidence and did not independently corroborate the information management provided.

22.22
As a result, the planning and assessment of risk that was done did not provide a sound basis for planning and performing the audits during this period. There was also limited testing of, and limited consideration given to, the wastewater project during the actual audit work. Some test procedures were performed incompletely or inadequately.

22.23
The review concluded that, for the 2009 Annual Report, the result was that the financial statements materially misstated information and that an incorrect audit opinion was issued.

22.24
The review also found significant deficiencies in the auditor's work on rates during these years. The review concluded that the auditor should have detected many of the irregularities covered by the Validation Bill through normal test procedures. The review comments that the primary reasons for the deficiencies were:

• A lack of understanding of the requirements of the [Local Government (Rating) Act 2002];

• A lack of understanding of the objective or purposes of the audit test procedure; and

• A failure to adequately identify the risks and significance of non-compliance.

These lapses meant that the auditor was not in a position to apply sufficient professional judgment when performing the audit test procedures, or identifying potential material non-compliance, or irregularities, should they be present. The deficiencies noted in the auditor's work were not identified or addressed through the engagement quality review processes.²

22.25
The review notes the highly subjective nature of the auditor's work on legislative compliance and that the individual auditors had felt that they had little guidance to help them determine when they had done enough work. It records that Audit New Zealand has guidance in its manual that complies with professional requirements, but accepts that this is an area that needs more attention.

22.26
Overall, the review concludes that the audits during these years were substandard in some respects and questions whether they satisfied the audit objective. This conclusion affects four annual audits and two audits of LTPs.

Audits carried out between 2010 and 2012

22.27
Mr Cherry's review found that the quality of audit work improved between 2010 and 2012. The sufficiency and quality of documentation in the audit files increased, showing the additional effort that was going into the KDC audit as problems came to light. The review found only minor deficiencies in audit quality during these years. These deficiencies were unlikely to affect the auditor's conclusions.

22.28
During the 2010 audit, KDC accepted legal advice that there were problems with the relevant targeted rate that needed to be addressed. KDC disclosed this in its annual report, so the audit report did not comment on it further. The 2011 Annual Report included extensive disclosure on a wide range of issues, including the restatement of financial information to address the misstatement in the 2009 financial statements.

22.29
Overall, the review concluded that the audit work during these years satisfied the audit objective. The review notes that considerable resources were devoted to KDC's audit during these years, as problems came to light.

The Auditor-General's response

22.30
Mr Cherry's independent review found that audit quality varied significantly during these 10 years and was inadequate in some cases. This is obviously not satisfactory. The Auditor-General has an overall responsibility to ensure that all audits carried out in her name reach an appropriate standard of quality. In this Section, we summarise the steps that the Auditor-General has taken or is taking to address these concerns about audit quality within Audit New Zealand.

Changes that have already been made

22.31
As noted in Part 1, the Auditor-General removed Audit New Zealand from the KDC audit in late 2012 and appointed a new audit service provider. This change was made once the OAG's internal quality assurance report on the audit work had been finalised and as soon as a transition to a new auditor was practicable. In practice, it was not practicable to change audit service providers until Audit New Zealand had completed the audits for the 2012 Annual Report and 2012–22 LTP.

22.32
The OAG maintains a quality assurance programme that periodically assesses samples of audit files for all appointed auditors. That programme of work has been advanced to ensure that all appointed auditors working in the local government sector will have had their work on local authority audits assessed by June 2014.

22.33
The OAG and Audit New Zealand have reviewed Audit New Zealand's methodology for considering rates during local authority audits. For annual reports prepared in 2013, the OAG required all appointed auditors of local authorities to complete a quality assurance questionnaire to enable the OAG to assess rating practice in the sector and to ensure that local authority auditors were applying a consistent standard to their work. The OAG is currently analysing the results of this work and will report on it separately.

Other steps being taken

22.34
The Auditor-General is also taking some additional steps to ensure that Parliament, the public, and public entities can rely on the quality of audit work from Audit New Zealand. Given the questions raised by this independent review, she and Audit New Zealand have agreed that it needs to demonstrate that it is able to maintain a consistent and appropriate standard of audit quality.

22.35
The Auditor-General appreciates that Audit New Zealand, during the last five years in particular, has been working systematically to improve audit quality and achieve consistency throughout its portfolio. The Auditor-General endorses those steps, but considers that more needs to be done to demonstrate the necessary consistency.

22.36
Therefore, the Auditor-General has asked Audit New Zealand to provide her with assurance that it has reached the objective of consistent audit quality throughout its portfolio. Audit New Zealand will develop and implement an improvement programme, which will be overseen by two independent advisers to the Auditor-General.

22.37
Until this regime is able to satisfy the Auditor-General that Audit New Zealand's work is consistently maintaining an appropriate standard, she will not allocate any new audit engagements to Audit New Zealand, nor allocate any existing Audit New Zealand audits to other providers.

22.38
The Auditor-General emphasises that the concerns raised by this independent review do not suggest widespread or systemic problems that would justify removing audits from Audit New Zealand. Furthermore, she appreciates the professionalism and hard work of the Audit New Zealand leadership team and staff. However, the question mark over its work at present means that it is not appropriate to add to its portfolio until it has demonstrated that it can achieve consistency of audit quality.

22.39
To give some practical context to this decision, during the last five years, the number of new public entities created each year that require auditors to be appointed has ranged from 40 to 80.³ Audit New Zealand would usually expect to be appointed to carry out a reasonable number of these new audits.

1: Mr Cherry is the current chairman of the New Zealand Auditing and Assurance Standards Board, a board member at the External Reporting Board, and a board member of the Australian Auditing and Assurance Standards Board. He carried out this review in his personal capacity.

2: Paragraph 43.10 of Mr Cherry's report.

3: The overall portfolio of public entities requiring auditors usually sits at about 4000 entities. Many new entities are subsidiaries or other controlled entities of existing public entities. Others result from restructuring within the public sector.