Annex 1: Performance indicator summary

Detailed assessment of the New Zealand Office of the Auditor-General against International Standards.

This annex provides a summary table of the SAI performance indicators. The table also records the criteria within each dimension that have been met and not met. For each indicator, the table specifies the scoring assigned along with a brief explanation for the scoring.

IndicatorDomainDimensionsCriteria Results SummaryScore and Explanation
A. Independence and Legal Framework
SAI-1 Independence of the SAI Overall score: 3

As a written constitution is not in place Dimension 1 cannot score higher than one. The independence arrangements established through the Public Audit Act 2001 are considered to be as strong as possible with this constitutional context and no changes to the current arrangement have been identified as necessary.
 
  1. Appropriate and effective constitutional framework
  2. Financial independence / autonomy
  3. Organizational independence / autonomy
  4. Independence of the Head of SAI and its Officials
  1. Criteria c, e, g are met, criteria a, b, and d are not met. Criterion f is not applicable.
  2. Criteria a to g are met.
  3. Criteria a to g are met.
  4. Criteria a to f are met.
  1. 1
  2. 4
  3. 4
  4. 4
SAI-2 Mandate of the SAI Overall score: 3

This score is impacted by the requirement for a written constitution in order to score full marks in dimension 3. Other than this matter, the scope of the mandate and ability to carry it out are very broad and strong.
  1. Sufficiently broad mandate
  2. Access to information
  3. Right and obligation to report
  1. Criterion a, and c to i are met, criterion b is not applicable.
  2. Criteria a to d are met.
  3. Criteria b to g are met criterion a is not met.
  1. 4
  2. 4
  3. 2
B. Internal Governance and Ethics
SAI-3 Strategic Planning Cycle Overall score: 3

There is a robust strategic planning cycle. It can be improved by establishing clearer links between goals and outcomes and annual planning outputs, providing clarity on how annual work is prioritised to meet longer term goals and outcomes and how risks to achieving the goals and outcomes are mitigated.
  1. Content of the Strategic Plan
  2. Content of the Annual Plan/Operational Plan
  3. Organizational Planning Process
  4. Monitoring and Performance Reporting
  1. Criteria a to c, e and f are met, criterion d is not met and criterion g is not applicable.
  2. Criteria a and b, d, f and g are met, criteria c and e are not met.
  3. All criteria are met
  4. Criteria a to d, and f and g are met, criterion e is not met.
  1. 3
  2. 3
  3. 4
  4. 3
SAI-4 Organizational Control Environment Overall score: 3

Although much of the organisational control environment is robust, there is a need for improvement in risk management, internal audit confirmation of internal controls and a broadening of the quality assurance system to include all Office products.
  1. Internal Control Environment – Ethics, Integrity and Organizational Structure
  2. System of internal control
  3. Quality Control System
  4. Quality Assurance System
  1. Criteria a to j are met, criteria k and l are not met.
  2. Criteria a, b, d, f, h and i are met, criteria c, e, and g are not met and criterion j is not applicable.
  3. Criteria a to e are met
  4. Criteria a to h are met, criteria i and j are not met.
  1. 3
  2. 2
  3. 4
  4. 2
SAI-5 Outsourced Audits Overall score: 4
All of the criteria of the framework are met.
  1. Process for Selection of Contracted Auditor
  2. Quality Control of Outsourced Audits
  3. Quality Assurance of Outsourced Audits
  1. Criteria a to g are met
  2. Criteria a to e are met
  3. Criteria a to g are met
  1. 4
  2. 4
  3. 4
SAI-6 Leadership and Internal Communication Overall score: 4

All of the criteria of the framework are met.
  1. Leadership
  2. Internal Communication
  1. Criteria a to h are met
  2. Criteria a to f are met
  1. 4
  2. 4
SAI-7 Overall Audit Planning and Follow-Up Overall score: 3

The overall planning process is robust. The Annual Plan includes all of the recommended content items apart from an explicit consideration at the overall level of any risks to delivery of the financial audit programme.
  1. Overall Audit Planning Process
  2. Overall Audit Plan Content
  3. Existence of Effective Follow-up Mechanisms
  1. Criteria a to g are met
  2. Criteria a to d are met and criterion e is not met
  3. Not rated
  1. 4
  2. 3
  3. Not rated
C. Audit Quality and Reporting
SAI-8 Audit Coverage Overall score: 3

Financial and performance audits are addressed by the Office to the highest level and coverage is to a high standard. Compliance audits are not approached on a systematic basis.
   
  1. Financial Audit Coverage
  2. Performance Audit Coverage
  3. Compliance Audit Coverage
  1. 100% met
  2. Criteria a to h are met
  3. Assessed as a score of 1 as there is no systematic assessment of risk and materiality
  1. 4
  2. 4
  3. 1
SAI-9 Financial Audit Standards and Quality Management Overall score: 4

All requirement of financial audit and quality control standards are met.
  1. Financial Audit Standards and policies
  2. Financial Audit Team Management and Skills
  3. Quality Control in Financial Audit
  1. Criteria a to v are met
  2. Criteria a to k are met
  3. Criteria a to e are met
  1. 4
  2. 4
  3. 4
SAI-10 Financial Audit Process Overall score: 3

Although sufficient criteria are met to obtain the highest overall score in each area, as an independent external review has not been completed since 2012 the maximum possible score is reduced.
  1. Planning Financial Audits
  2. Implementing Financial Audits
  3. Evaluating Audit Evidence, Concluding and Reporting in Financial Audits
  1. Criterion a not applicable, criteria b to h and k are met, criteria i and j not met
  2. Criteria a, c, d(ii), f and g are not met, criteria b and d(i) and d(iv) and e are met, criterion d(iii) is not applicable
  3. Criteria a to i are met and criteria j and k are not applicable
  1. 3
  2. 3
  3. 3
SAI-11 Financial Audit Results Overall score: 3

Financial audits are completed and reported in a timely manner. The Office has no mandate to make audit reports publicly available. Reporting recommendations are followed up at the entity level based on best practice but findings are not systematically monitored and reported on publicly.
  1. Timely Submission of Financial Audit Results
  2. Timely Publication of Financial Audit Results
  3. SAI Follow-up on Implementation of Financial Audit Observations and Recommendations
  1. Score 4 as more than 80% achieved
  2. Criteria is not applicable
  3. Criteria a and c are not met and criteria b and d are met
  1. 4
  2. Not applicable
  3. 2
SAI-12 Performance Audit Standards and Quality Management Overall score: 4

There is only one aspect of audit planning that ISSAIs recommend that is not contained in the Performance Audit Manual.
  1. Performance Audit Standards and Policies
  2. Performance Audit Team Management and Skills
  3. Quality Control in Performance Audit
  1. Criteria a, b and d to u are met, criterion c is not met
  2. Criteria a to m are met
  3. Criteria a to f are met.
  1. 3
  2. 4
  3. 4
SAI-13 Performance Audit Process Overall score: 2

In order to score at the highest in this SAI an independent review of the performance audit practice is required and this has not occurred. In addition there are aspects of the planning reporting that are not completed or not explicitly evidenced.
  1. Planning Performance Audits
  2. Implementing Performance Audits
  3. Reporting on Performance Audits
  1. Criteria a, c, e, i, k and m are met, criteria b, d, f to h, j and l are not met
  2. Criteria a to k are met
  3. Criteria b to f, h, i, l and m are met, criteria a, g and j are not met and criterion k is not applicable.
  1. 2
  2. 3
  3. 2
SAI-14 Performance Audit Results Overall score 4

All criteria are met at the highest level.
  1. Timely Submission of Performance Audit Reports
  2. Timely Publication of Performance Audit Reports
  3. SAI Follow-up on Implementation of Performance Audit Observations and Recommendations
  1. Score 4 as all reports are tabled within 30 days
  2. Score 4 as all reports are publicly available within 15 days of tabling
  3. Criteria a to f are met.
  1. 4
  2. 4
  3. 4
SAI-15 Compliance Audit Standards and Quality Management Overall score: 3

Sufficient standards and policies and quality control procedures are in place that are applied to compliance audit to fully meet the criteria. However, as there is no systematic approach to training staff for these audits the score for the team management and skills dimension is substantially reduced.
  1. Compliance Audit Standards and Policies
  2. Compliance Audit Team Management and Skills
  3. Quality Control in Compliance Audit
  1. Criteria a to o and q are met, criteria p and r are not met
  2. Criteria c, g, h and p are met, criteria a, b, d, e, i to o are not met and criterion f is not applicable
  3. Criteria f to k are met.
    NB: There are no criteria a to e.
  1. 4
  2. 1
  3. 4
SAI-16 Compliance Audit Process Overall score: 2

There are weaknesses in planning and implementing compliance audits, however all the criteria for evaluating evidence, concluding and reporting are met. However, the overarching requirements for an independent review of the Compliance Audit practice has not been met.
  1. Planning Compliance Audits
  2. Implementing Compliance Audits
  3. Evaluating Asudit Evidence, Concluding and Reporting in Compliance Audits
  1. Criteria a, d, f, h, j and k were met, criteria b, c, e, g and i were not met
  2. Criteria a and e were met, criterion d was not met and criteria b and c were not applicable to the sample selected so not rated.
  3. Criteria a to j were met
  1. 2
  2. 2
  3. 3
SAI-17 Compliance Audit Results Overall score: Not rated

There is no system in place to measure timeliness of completion of compliance audits nor follow-up of observations or recommendations. There is no requirement to publish compliance audit results.
  1. Timely Submission of Compliance Audit Results
  2. Timely Publication of Compliance Audit Results
  3. SAI Follow-up on Implementation of Compliance Audit Observations and Recommendations
  1. Not rated
  2. Not applicable
  3. Criteria a to d are not met
  1. Not rated
  2. Not applicable
  3. 0
SAI-18 Control and Judgement of Rendered Accounts (for Court Model SAIs) Not applicable Not applicable Not applicable
SAI-19
(optional)
Public Sector Audit Standards and Quality Management
  1. Public Sector Audit Standards and Policies
  2. Public Sector Audit Team Management and Skills
  3. Quality Control in Public Sector Audit
Not assessed Not assessed
SAI-20
(optional)
Public Sector Audit Process
  1. Planning Public Sector Audits
  2. Implementing Public Sector Audits
  3. Reporting on Public Sector Audits
Not assessed Not assessed
SAI-21
(optional)
Public Sector Audit Results
  1. Timely Submission of Public Sector Audit Results
  2. Timely Publication of Public Sector Audit Results
  3. SAI Follow-up on Implementation of Public Sector Audit Observations and Recommendations
Not assessed Not assessed
D. Financial Management, Assets and Support Services
SAI-22 Financial Management, Assets and Support Services Overall score: 3

Overall financial and internal control system are robust. There is a lack of an overarching physical infrastructure strategy, although as physical infrastructure is leased lease renewals provide a framework for reassessing needs. While administrative service roles are appropriately filled there is not a systematic periodic review programme in place to evaluate these services.
  1. Financial Management
  2. Planning and Effective Use of Assets and Infrastructure
  3. Administrative Support Services
  1. Criteria a to k met
  2. Criterion a not met, criteria b to e met
  3. Criteria a to c met, criterion d not met
  1. 4
  2. 3
  3. 3
E. Human Resources and Training
SAI-23 Human Resource Management Overall score: 3

All of the recommended human resources related functions, policies and processes are in place. However, this is currently a lack of an overall Human Resources strategy.
  1. Human Resources Function
  2. Human Resources Strategy
  3. Human Resources Recruitment
  4. Remuneration, Promotion and Staff Welfare
  1. Criteria a to g are met
  2. Criteria a to g are not met
  3. Criteria a to g are met
  4. Criteria a to h are met
  1. 4
  2. 0
  3. 4
  4. 4
SAI-24 Professional Development and Training Overall score: 3

There is no overall human resources strategy and there is less structure to training for OAG and non-professional staff than financial audit staff. Professional development and training of financial and performance auditors is largely robust. However, there is insufficient training in the specific skills needed to ensure quality standards are consistently maintained in compliance audits.
  1. Plans and Processes for Professional Development and Training
  2. Financial Audit Professional Development and Training
  3. Performance Audit Professional Development and Training
  4. Compliance Audit Professional Development and Training
  1. Criteria a, c and d are met, criteria b, f and g are not met
  2. Criteria a to d are met
  3. Criteria a to d are met
  4. Criterion a is met and criteria b to d are not met
  1. 2
  2. 4
  3. 4
  4. 1
F. Communication and Stakeholder Management
SAI-25 Communication with the Legislature, Executive and Judiciary Overall score: 4

There are robust policies and processes in place communicate audit findings with the Legislature, the Executive and other parties. The overall Communications Strategy is robust. However, effective monitoring of the current strategy is still being established.
  1. Communications Strategy
  2. Good Practices Regarding Communication with the Legislature
  3. Good practices Regarding communication with the Executive
  4. Good practices Regarding communication with the Judiciary, prosecuting and investigating agencies
  1. Criteria a to e and g are met, criterion f is not met
  2. Criteria a to h are met
  3. Criteria a to d are met
  4. Criteria a and b are met, criterion c is not rated and criteria d and e are not applicable.
  1. 3
  2. 4
  3. 4
  4. NA
SAI-26 Communication with the Media, Citizens and Civil Society Organizations Overall score: 4

The framework expects that a SAI should conduct press conferences and press releases. The Office does not do this as standard practice and is of the view that communication goals are achieved by other means. Opportunities to work with citizens and civil society groups are in place, initiatives to hear and understand their opinions are developing.
  1. Good Practices Regarding Communication with the Media
  2. Good Practices Regarding Communication with Citizens and Civil Society Organisations
  1. Criterion criteria a to f are met
  2. Criteria a to h are met
  1. 4
  2.  4