Appendix 4: Report on the quality of annual audits
High-quality audit work is important. It enables us to provide assurance over public spending and help improve financial management and reporting.
This report focuses on our system of quality management for annual audits carried out on behalf of the Auditor-General.
The way we operate
The Auditor-General appoints auditors to carry out annual audits of public organisations. These auditors are appointed from Audit New Zealand (the Auditor-General's in-house audit service provider) and 22 private sector audit firms. There are about 155 auditors with the authority to audit and issue audit reports for the public organisations they have been appointed to audit.
Because of the way we operate, the key elements that need to operate effectively are:
- the appointment of auditors considered to be independent to carry out audits;
- the Auditor-General's auditing standards that auditors are required to apply;2 and
- the quality of the work that auditors perform.
Our quality management system
Professional and ethical standards3 require us to operate a system of quality management to provide reasonable assurance that we comply with standards and applicable legal and regulatory requirements, and issue appropriate reports from our work. The system of quality management is based on seven elements:
- governance and leadership;
- our risk assessment processes;
- ethical requirements;
- performing our audit work;
- resources;
- information and communication; and
- monitoring quality and remediating deficiencies.4
We report below on the processes, policies, and procedures that support each element of audit quality, as it applies to annual audits carried out on behalf of the Auditor-General. We also include some quality indicators, which measure performance.
Governance and leadership
Our governance and reporting structure contributes to audit quality
The Auditor-General is ultimately responsible for the system of quality management for all audits carried out on his behalf.
Audit quality is governed through the Office's Audit Performance and Quality Governance Committee. The role of the committee is to monitor audit delivery and quality. Membership of the committee includes the Auditor-General and Deputy Auditor-General. The committee's remit includes:
- monitoring the strategic and operational risks associated with audit quality;
- monitoring the operating effectiveness and efficiency of the quality framework against the audit quality indicators;
- monitoring the findings of internal and external reviews of audit quality; and
- monitoring progress in addressing the findings and recommendations made in internal and external reviews.
The committee met five times during 2023/24.
We also have an Audit and Risk Committee that provides independent advice to the Auditor-General. There is a separate report from that committee included at Appendix 1.
Our values underpin audit quality
The Auditor-General relies on all staff acting with integrity, so that there will be trust and confidence in the work that we do. Integrity is about consistently adhering to strong and moral ethical principles.
The Auditor-General is the Office's Integrity Officer because of the importance of integrity in our work. The Integrity Officer leads integrity work, which includes:
- increasing awareness of integrity matters across the organisation;
- ensuring integrity is regularly on the agenda at senior leadership level;
- supporting ongoing work to progress and implement the matters identified in the Office's Integrity Strategy; and
- identifying and ensuring alignment of progress on integrity matters with other work under way across the Office.
Our integrity framework has the building blocks a public organisation should have in place for a strong and effective integrity system and is consistent with the good practice material we have published for use by public organisations.5
We measure and report progress internally to staff.
We obtain independent views about audit quality
As well as our internal monitoring of audit quality, we obtain external independent views about audit quality. The Financial Markets Authority reviews audit files of public organisations operating in capital markets, including some public organisations audited by private sector audit firms. Every four years, we invite the Financial Markets Authority to carry out a review, so the next review is expected to occur in 2026.
The Financial Markets Authority reviewed both the system of quality control and a sample of audit files. Although its report identified low-rated findings about the system of quality control, and low and medium rated findings for the sample of audit files, none of the findings would change the audit reports that were issued.
The New Zealand Institute of Chartered Accountants Practice Review reviewed a sample of audit files. Its report noted it was evident that Audit New Zealand applies high quality standards in its audits. The report identified some findings, none of which would change the audit reports that were issued.
We have considered the findings in each report to identify improvements that can be made to our system of quality management.
Our risk assessment processes
Our risk management processes for audit quality operate at two levels; the strategic level and the system of quality management level.
Strategic risks affect the Office achieving its strategic objectives. Audit failure that leads to a loss of trust and confidence is one of our strategic risks. Such a failure could occur if an audit report with an incorrect audit opinion was issued.
The system of quality management risks relate to our quality objectives for each of the seven elements outlined above.
We have systems and processes designed to mitigate both strategic and quality management risks. We have identified policies and procedures to improve or enhance our risk mitigations. We will develop a work programme to address these.
Ethical requirements
Our policies, procedures, and methods promote an ethical workplace
Independence is fundamental to our ability to act with integrity, be objective, and maintain an attitude of professional scepticism. Professional and ethical standards require auditors to be independent of the organisation they are auditing. The Auditor-General's auditing standards, which incorporate these professional and ethical standards set a high standard for independence, both of mind and in appearance.
The Auditor-General's auditing standard on independence applies to all staff, including Audit New Zealand, and the private sector audit firms that carry out public sector audits. The standard is based on the requirements of the New Zealand standard issued by the External Reporting Board, to the extent there is not a conflict with the Auditor-General's legislated mandate and responsibilities. The Auditor-General's standard goes further and restricts the work auditors can carry out for an organisation they audit to only work of an assurance nature.
We monitor compliance with audit independence requirements
We monitor compliance with the Auditor-General's auditing standard on independence in several ways.
For staff, including Audit New Zealand, the work that can be done is limited by the Public Audit Act 2001, and the independence of those involved in annual audits is closely monitored, including as part of our quality assurance review program for annual audits.
For private sector audit firms, we monitor the other services they carry out for public organisations they audit on behalf of the Auditor-General. We also pre-approve or decline work they propose carrying out where independence may be questioned, and we consider independence as part of our quality assurance review program for annual audits.
In 2023/24, 46 proposals for work other than the audit were referred for approval (51 in 2022/23). Of these, 45 were approved and one was declined. There was one instance where we identified a threat to independence and took action to manage and mitigate the threat.
For 2023/24 we were satisfied that independence standards were upheld.
We monitor how long key audit staff audit the same organisation
The Auditor-General's auditing standards limit the number of years that key audit staff can carry out the same annual audit. This is to safeguard against the threat to independence that may arise from auditing an organisation for a long time. The standard specifies the length of time that key audit staff can be assigned to the annual audit, before being rotated off the audit.
For 2023/24 we complied with the Auditor-General's standard.
Resources
All audits are allocated, either to senior staff in Audit New Zealand or to partners in private sector audit firms. Therefore, recruiting, hiring, retaining, and promoting qualified audit staff is performed by audit service providers. Our expectation is that audit work is completed by staff with the right skills and experience. We monitor the skills of audit teams as part of our quality assurance reviews.
For 2023/24 we were generally satisfied with the skills and experience of staff allocated to audits, based on our quality assurance reviews.
Performing our audit work
We establish, maintain, and communicate audit expectations
The Office of the Auditor-General requires all audit service providers to have their own audit methodology and to apply the professional quality standards. This typically results in appropriate levels of review of audit files. Auditors carry out audits based on the Auditor-General's auditing standards and requirements and guidance provided through an audit brief. We updated two Auditor-General Statements for improvements identified from our quality reviews.
We communicate the mandatory quality requirements we place on firms that carry out audits or other engagements on behalf of the Auditor-General.6
We require auditors to consult about matters that could affect an audit report
The Auditor-General's auditing standards require auditors to consult on specific matters that could result in a non-standard audit report. The Office of the Auditor-General has an Opinions Review Committee that meets, as required, to determine the modifications to be included in audit opinions and other matters to be highlighted in audit reports. For 2023/24, the committee met on 44 occasions (45 in 2022/23).
We analyse the nature of matters considered by the committee and communicate to auditors so that they can maintain an awareness for these in their audit work. In 2023/24, potential probity matters were considered during two meetings (eight in 2022/23) and uncertainties in estimates relating to financial statements were considered during three meetings (10 in 2022/23).
To support and ensure consistency of our audit reports on council consultation documents and long-term plans, we reviewed the proposed audit report and key judgements supporting these. In 2023/24, we reviewed 26 audit reports on council consultation documents and long-term plans.
We require auditors to inform us of issues discovered in audited information
Sometimes during an audit an auditor discovers, or is made aware of, an error or misstatement in the prior year's financial statements or performance information. If known at the time, this would have resulted in changes to the financial statements or performance information, or a qualification of the opinion in the auditor's report.
When material errors are identified, common practice, in line with auditing and accounting standards, is to retrospectively correct for misstatement in the first set of financial statements authorised after the error is discovered.
Prior period errors by financial year
| Year | Prior period errors included in annual reports | As a percentage of the total number of reports issued in the year |
|---|---|---|
| 2023/24 | 52 | 1.6% |
| 2022/23 | 49 | 1.8% |
| 2021/22 | 33 | 1.2% |
Most of the errors were in the financial statements of schools, local authorities, council-controlled organisations, and State-owned enterprises, and related to incorrect classification or measurement of financial instruments, errors in recognising or derecognising property, plant and equipment, and incorrect classification of financial instruments.
The number and impact of errors or misstatements can signal potential problems with the audit. We follow up errors with auditors to understand why the error wasn't identified during the audit, and whether improvements can be made in future audits.
We also communicate the findings of our analysis to our auditors.
We require an engagement quality review to occur for particular audits
We have a standard that sets out when an engagement quality review is required. Our auditors need to consider a wide range of quality risk factors as part of assessing when to appoint an engagement quality reviewer to respond to the risks.
An engagement quality review provides an objective evaluation of significant judgements made by auditors and the conclusions reached before the auditor signs the audit opinion. The engagement quality reviewer's evaluation of significant judgements takes into account professional standards and applicable legal and regulatory requirements. The engagement quality reviewer is not part of the engagement team.
We assess compliance with the engagement quality review as part of our quality assurance reviews, including evidence that the review met the requirements of the standard. For 2023/24, based on our quality reviews, we were satisfied that all audits that required it had an effective engagement quality review.
Monitoring quality and remediating deficiencies
Frequency of our quality reviews
Monitoring compliance with the Auditor-General's auditing standards is a key element of our system of quality. Our quality reviews are in addition to audit service providers monitoring their own system of quality and complying with professional ethical standards.
Our quality reviews of annual audits are designed to determine whether audit engagements comply with the Auditor-General's auditing standards, relevant regulatory and legal requirements, and our policies.
Our monitoring covers all auditors appointed to carry out audits. The frequency of reviews is based on risk.7
We choose the audit files we want to review in accordance with our quality assurance policy, which considers the size and complexity of the audit. We also perform quality reviews of some audit files before the audit report is issued, depending on risk factors.
Quality reviews performed by financial year
| Year | Number of appointed auditors who had a quality review | Number of instances where the audit file was reviewed after the corresponding audit report was issued | Number of instances where the audit file was reviewed before the corresponding audit report was issued |
|---|---|---|---|
| 2023/24 | 51 | 63 | 13 |
| 2022/23 | 31 | 47 | 11 |
| 2021/22 | 34 | 79 | 6 |
In applying a risk-based focus, the frequency of our reviews for auditors where no risk factors are present can be more than five years. We have 20 (16%) auditors where their last quality review was more than five years ago.
Quality review results of audit file inspections
We grade each audit file we review.
Percentage of audit files reviewed that met our quality expectations
| Year | Met our quality expectations | Did not meet our quality expectations |
|---|---|---|
| 2023/24 | 84% | 16% |
| 2022/23 | 74% | 26% |
| 2021/22 | 84% | 16% |
Our results are similar to those reported by the New Zealand Financial Markets Authority and similar to international averages.8
Where audit files do not meet our quality expectations, we assess whether deficiencies call into question the appropriateness of the opinion included in the audit report. In all situations we were satisfied that the audit opinion was appropriate.
How we determine the cause of quality review findings
We ask audit service providers to perform root-cause analysis for the more significant findings from our quality reviews. Such analysis provides a deeper understanding of improvements that are needed, including improvements to audit methodologies. By gaining a deeper understanding of the causes of quality deficiencies identified, targeted changes and follow-up actions can be developed.
We monitor the interventions that audit firms are implementing to help prevent these recurring.
How we assess timely and effective remediation of quality review findings
We require our auditors to remediate the more significant deficiencies, and where necessary, make changes to the audit approach for subsequent audits.
For our 2023/24 quality assurance work, auditors are in the process of responding to our quality review findings and remediating the more significant deficiencies. Our follow-up work to assess remediation will occur as part of our 2024/25 quality review programme.
We perform a follow-up review
As well as assessing auditors' remediation of the more significant deficiencies, we carry out a follow-up quality assurance review once the auditor's quality improvement programme has been completed. This usually occurs within 18 months, to give sufficient time to the auditor and their firm to make changes and have them reflected in future audits. If it is important that particular changes are made before the next audit, we confirm those changes before the next audit report is issued.
At 30 June 2024, there are 14 auditors who did not meet our quality expectations. For 11 of these auditors the review was carried out during the past 18 months and a follow-up review was not due.
Sometimes there are significant findings from follow-up reviews. If this occurs, and the auditors continue to carry out audits, more support is provided to the auditors. The other three auditors are in this position.
Statement of the effectiveness of the Office's system of quality management
In keeping with the New Zealand External Reporting Board's standard on quality management, we carried out an evaluation of the overall system of quality management at 15 December 2023. We concluded that, except for matters relating to an identified deficiency that has a severe but not pervasive effect on the design, implementation, and operation of the system of quality management, the system of quality management provides us with reasonable assurance that the objectives of the system of quality management are being achieved.
A key contributor to our conclusion related to the status of root-cause analysis of the more significant deficiencies from quality assurance reviews of audit engagements. Although we were satisfied that audit reports contained the correct audit opinion, we were not satisfied that the root-cause analysis and any necessary remediation to the system of quality management had occurred. The root-cause analysis has now been completed.
2: The Public Audit Act 2001 requires the Auditor-General to set auditing standards for carrying out audits. These are referred to as the Auditor-General's auditing standards. These standards incorporate the New Zealand auditing standards and include professional and ethical standards specific to independence and audit quality.
3: The relevant Professional and Ethical standard is PES 3: Quality Management for Firms that Perform Audits and Reviews of Financial Statements, or Other Assurance or Related Service Engagements issued by the New Zealand Auditing and Assurance Board of the External Reporting Board.
4: PES 3 has eight components. The component Acceptance and Continuance of Client Relationships and Specific Engagements in not directly relevant to the Office as we cannot refuse to be the auditor of a public entity. We include requirements on our audit services providers to advise us of issues which may ordinarily lead them to decline performing an audit so that we can manage the situation.
5: See Putting integrity at the core of how public organisations operate: An integrity framework for the public sector at oag.parliament.nz.
6: The quality requirements are set out in one of the Auditor-General's auditing standards, AG PES 3: Quality Management for Firms that Perform Audits or Other Engagements on Behalf of the Auditor-General.
7: Our policy is to perform a quality review:
- for all new appointed auditors after their first year of appointment;
- once within a three-year period for each appointed auditor, unless the risk profile supports extending that period (with the agreement of the Assistant Auditor-General, Audit Quality) to a frequency up to once every five years; and
- when the overall assessment grade assigned to an appointed auditor requires another quality review.
8: New Zealand Financial Markets Authority, Audit Quality Monitoring Report 1 July 2022 – 30 June 2023, page 6, reports 84% compliance of audit files reviewed in 2022/23 and 72% in 2022. The international average of non-compliant files is 26%.