Annex 1: Performance indicator summary

Detailed assessment of the New Zealand Office of the Auditor-General against International Standards.

This annex provides a summary table of the SAI performance indicators. The table also records the criteria within each dimension that have been met and not met. For each indicator, the table specifies the scoring assigned along with a brief explanation for the scoring.

Indicator	Domain	Dimensions	Criteria Results Summary	Score and Explanation
A. Independence and Legal Framework
SAI-1	Independence of the SAI	Overall score: 3 As a written constitution is not in place Dimension 1 cannot score higher than one. The independence arrangements established through the Public Audit Act 2001 are considered to be as strong as possible with this constitutional context and no changes to the current arrangement have been identified as necessary.
SAI-1		Appropriate and effective constitutional framework Financial independence / autonomy Organizational independence / autonomy Independence of the Head of SAI and its Officials	Criteria c, e, g are met, criteria a, b, and d are not met. Criterion f is not applicable. Criteria a to g are met. Criteria a to g are met. Criteria a to f are met.	1 4 4 4
SAI-2	Mandate of the SAI	Overall score: 3 This score is impacted by the requirement for a written constitution in order to score full marks in dimension 3. Other than this matter, the scope of the mandate and ability to carry it out are very broad and strong.
SAI-2	Mandate of the SAI	Sufficiently broad mandate Access to information Right and obligation to report	Criterion a, and c to i are met, criterion b is not applicable. Criteria a to d are met. Criteria b to g are met criterion a is not met.	4 4 2
B. Internal Governance and Ethics
SAI-3	Strategic Planning Cycle	Overall score: 3 There is a robust strategic planning cycle. It can be improved by establishing clearer links between goals and outcomes and annual planning outputs, providing clarity on how annual work is prioritised to meet longer term goals and outcomes and how risks to achieving the goals and outcomes are mitigated.
SAI-3	Strategic Planning Cycle	Content of the Strategic Plan Content of the Annual Plan/Operational Plan Organizational Planning Process Monitoring and Performance Reporting	Criteria a to c, e and f are met, criterion d is not met and criterion g is not applicable. Criteria a and b, d, f and g are met, criteria c and e are not met. All criteria are met Criteria a to d, and f and g are met, criterion e is not met.	3 3 4 3
SAI-4	Organizational Control Environment	Overall score: 3 Although much of the organisational control environment is robust, there is a need for improvement in risk management, internal audit confirmation of internal controls and a broadening of the quality assurance system to include all Office products.
SAI-4	Organizational Control Environment	Internal Control Environment – Ethics, Integrity and Organizational Structure System of internal control Quality Control System Quality Assurance System	Criteria a to j are met, criteria k and l are not met. Criteria a, b, d, f, h and i are met, criteria c, e, and g are not met and criterion j is not applicable. Criteria a to e are met Criteria a to h are met, criteria i and j are not met.	3 2 4 2
SAI-5	Outsourced Audits	Overall score: 4 All of the criteria of the framework are met.
SAI-5	Outsourced Audits	Process for Selection of Contracted Auditor Quality Control of Outsourced Audits Quality Assurance of Outsourced Audits	Criteria a to g are met Criteria a to e are met Criteria a to g are met	4 4 4
SAI-6	Leadership and Internal Communication	Overall score: 4 All of the criteria of the framework are met.
SAI-6	Leadership and Internal Communication	Leadership Internal Communication	Criteria a to h are met Criteria a to f are met	4 4
SAI-7	Overall Audit Planning and Follow-Up	Overall score: 3 The overall planning process is robust. The Annual Plan includes all of the recommended content items apart from an explicit consideration at the overall level of any risks to delivery of the financial audit programme.
SAI-7	Overall Audit Planning and Follow-Up	Overall Audit Planning Process Overall Audit Plan Content Existence of Effective Follow-up Mechanisms	Criteria a to g are met Criteria a to d are met and criterion e is not met Not rated	4 3 Not rated
C. Audit Quality and Reporting
SAI-8	Audit Coverage	Overall score: 3 Financial and performance audits are addressed by the Office to the highest level and coverage is to a high standard. Compliance audits are not approached on a systematic basis.
		Financial Audit Coverage Performance Audit Coverage Compliance Audit Coverage	100% met Criteria a to h are met Assessed as a score of 1 as there is no systematic assessment of risk and materiality	4 4 1
SAI-9	Financial Audit Standards and Quality Management	Overall score: 4 All requirement of financial audit and quality control standards are met.
SAI-9	Financial Audit Standards and Quality Management	Financial Audit Standards and policies Financial Audit Team Management and Skills Quality Control in Financial Audit	Criteria a to v are met Criteria a to k are met Criteria a to e are met	4 4 4
SAI-10	Financial Audit Process	Overall score: 3 Although sufficient criteria are met to obtain the highest overall score in each area, as an independent external review has not been completed since 2012 the maximum possible score is reduced.
SAI-10	Financial Audit Process	Planning Financial Audits Implementing Financial Audits Evaluating Audit Evidence, Concluding and Reporting in Financial Audits	Criterion a not applicable, criteria b to h and k are met, criteria i and j not met Criteria a, c, d(ii), f and g are not met, criteria b and d(i) and d(iv) and e are met, criterion d(iii) is not applicable Criteria a to i are met and criteria j and k are not applicable	3 3 3
SAI-11	Financial Audit Results	Overall score: 3 Financial audits are completed and reported in a timely manner. The Office has no mandate to make audit reports publicly available. Reporting recommendations are followed up at the entity level based on best practice but findings are not systematically monitored and reported on publicly.
SAI-11	Financial Audit Results	Timely Submission of Financial Audit Results Timely Publication of Financial Audit Results SAI Follow-up on Implementation of Financial Audit Observations and Recommendations	Score 4 as more than 80% achieved Criteria is not applicable Criteria a and c are not met and criteria b and d are met	4 Not applicable 2
SAI-12	Performance Audit Standards and Quality Management	Overall score: 4 There is only one aspect of audit planning that ISSAIs recommend that is not contained in the Performance Audit Manual.
SAI-12	Performance Audit Standards and Quality Management	Performance Audit Standards and Policies Performance Audit Team Management and Skills Quality Control in Performance Audit	Criteria a, b and d to u are met, criterion c is not met Criteria a to m are met Criteria a to f are met.	3 4 4
SAI-13	Performance Audit Process	Overall score: 2 In order to score at the highest in this SAI an independent review of the performance audit practice is required and this has not occurred. In addition there are aspects of the planning reporting that are not completed or not explicitly evidenced.
SAI-13	Performance Audit Process	Planning Performance Audits Implementing Performance Audits Reporting on Performance Audits	Criteria a, c, e, i, k and m are met, criteria b, d, f to h, j and l are not met Criteria a to k are met Criteria b to f, h, i, l and m are met, criteria a, g and j are not met and criterion k is not applicable.	2 3 2
SAI-14	Performance Audit Results	Overall score 4 All criteria are met at the highest level.
SAI-14	Performance Audit Results	Timely Submission of Performance Audit Reports Timely Publication of Performance Audit Reports SAI Follow-up on Implementation of Performance Audit Observations and Recommendations	Score 4 as all reports are tabled within 30 days Score 4 as all reports are publicly available within 15 days of tabling Criteria a to f are met.	4 4 4
SAI-15	Compliance Audit Standards and Quality Management	Overall score: 3 Sufficient standards and policies and quality control procedures are in place that are applied to compliance audit to fully meet the criteria. However, as there is no systematic approach to training staff for these audits the score for the team management and skills dimension is substantially reduced.
SAI-15	Compliance Audit Standards and Quality Management	Compliance Audit Standards and Policies Compliance Audit Team Management and Skills Quality Control in Compliance Audit	Criteria a to o and q are met, criteria p and r are not met Criteria c, g, h and p are met, criteria a, b, d, e, i to o are not met and criterion f is not applicable Criteria f to k are met. NB: There are no criteria a to e.	4 1 4
SAI-16	Compliance Audit Process	Overall score: 2 There are weaknesses in planning and implementing compliance audits, however all the criteria for evaluating evidence, concluding and reporting are met. However, the overarching requirements for an independent review of the Compliance Audit practice has not been met.
SAI-16	Compliance Audit Process	Planning Compliance Audits Implementing Compliance Audits Evaluating Asudit Evidence, Concluding and Reporting in Compliance Audits	Criteria a, d, f, h, j and k were met, criteria b, c, e, g and i were not met Criteria a and e were met, criterion d was not met and criteria b and c were not applicable to the sample selected so not rated. Criteria a to j were met	2 2 3
SAI-17	Compliance Audit Results	Overall score: Not rated There is no system in place to measure timeliness of completion of compliance audits nor follow-up of observations or recommendations. There is no requirement to publish compliance audit results.
SAI-17	Compliance Audit Results	Timely Submission of Compliance Audit Results Timely Publication of Compliance Audit Results SAI Follow-up on Implementation of Compliance Audit Observations and Recommendations	Not rated Not applicable Criteria a to d are not met	Not rated Not applicable 0
SAI-18	Control and Judgement of Rendered Accounts (for Court Model SAIs)	Not applicable	Not applicable	Not applicable
SAI-19 (optional)	Public Sector Audit Standards and Quality Management	Public Sector Audit Standards and Policies Public Sector Audit Team Management and Skills Quality Control in Public Sector Audit	Not assessed	Not assessed
SAI-20 (optional)	Public Sector Audit Process	Planning Public Sector Audits Implementing Public Sector Audits Reporting on Public Sector Audits	Not assessed	Not assessed
SAI-21 (optional)	Public Sector Audit Results	Timely Submission of Public Sector Audit Results Timely Publication of Public Sector Audit Results SAI Follow-up on Implementation of Public Sector Audit Observations and Recommendations	Not assessed	Not assessed
D. Financial Management, Assets and Support Services
SAI-22	Financial Management, Assets and Support Services	Overall score: 3 Overall financial and internal control system are robust. There is a lack of an overarching physical infrastructure strategy, although as physical infrastructure is leased lease renewals provide a framework for reassessing needs. While administrative service roles are appropriately filled there is not a systematic periodic review programme in place to evaluate these services.
SAI-22	Financial Management, Assets and Support Services	Financial Management Planning and Effective Use of Assets and Infrastructure Administrative Support Services	Criteria a to k met Criterion a not met, criteria b to e met Criteria a to c met, criterion d not met	4 3 3
E. Human Resources and Training
SAI-23	Human Resource Management	Overall score: 3 All of the recommended human resources related functions, policies and processes are in place. However, this is currently a lack of an overall Human Resources strategy.
SAI-23	Human Resource Management	Human Resources Function Human Resources Strategy Human Resources Recruitment Remuneration, Promotion and Staff Welfare	Criteria a to g are met Criteria a to g are not met Criteria a to g are met Criteria a to h are met	4 0 4 4
SAI-24	Professional Development and Training	Overall score: 3 There is no overall human resources strategy and there is less structure to training for OAG and non-professional staff than financial audit staff. Professional development and training of financial and performance auditors is largely robust. However, there is insufficient training in the specific skills needed to ensure quality standards are consistently maintained in compliance audits.
SAI-24	Professional Development and Training	Plans and Processes for Professional Development and Training Financial Audit Professional Development and Training Performance Audit Professional Development and Training Compliance Audit Professional Development and Training	Criteria a, c and d are met, criteria b, f and g are not met Criteria a to d are met Criteria a to d are met Criterion a is met and criteria b to d are not met	2 4 4 1
F. Communication and Stakeholder Management
SAI-25	Communication with the Legislature, Executive and Judiciary	Overall score: 4 There are robust policies and processes in place communicate audit findings with the Legislature, the Executive and other parties. The overall Communications Strategy is robust. However, effective monitoring of the current strategy is still being established.
SAI-25	Communication with the Legislature, Executive and Judiciary	Communications Strategy Good Practices Regarding Communication with the Legislature Good practices Regarding communication with the Executive Good practices Regarding communication with the Judiciary, prosecuting and investigating agencies	Criteria a to e and g are met, criterion f is not met Criteria a to h are met Criteria a to d are met Criteria a and b are met, criterion c is not rated and criteria d and e are not applicable.	3 4 4 NA
SAI-26	Communication with the Media, Citizens and Civil Society Organizations	Overall score: 4 The framework expects that a SAI should conduct press conferences and press releases. The Office does not do this as standard practice and is of the view that communication goals are achieved by other means. Opportunities to work with citizens and civil society groups are in place, initiatives to hear and understand their opinions are developing.
SAI-26		Good Practices Regarding Communication with the Media Good Practices Regarding Communication with Citizens and Civil Society Organisations	Criterion criteria a to f are met Criteria a to h are met	4 4