Part 3: Assessing entities' environment, systems, and controls

3.01
In this Part, we report on our 2006/07 assessments of the environment, systems, and controls of government departments, Crown entities (excluding school boards of trustees and tertiary education institutions), and State-owned enterprises.

Background

3.02
As part of the annual financial statements audit, our auditors examine, assess, and grade central government entities’ environment, systems, and controls for managing and reporting financial and service performance information. We report these assessments to the entity, the responsible Ministers, and the relevant select committees.

3.03
Our examination of an entity’s environment, systems, and controls is in the context of the auditor’s work in forming an opinion on the financial and service performance statements. The purpose of commenting on these aspects is to highlight areas for improvement the audit identified, and the grades assigned directly represent the recommendations for improvement as at the end of the financial year.

3.04
We introduced a new assessment framework in the 2006/07 financial year to improve the transparency, usefulness, and understandability of our reporting.¹ It replaced the framework we had used for the previous 13 years.

3.05
We applied our new approach in the following sectors for 2006/07:

• government departments;
• Crown entities, excluding school boards of trustees and tertiary education institutions; and
• State-owned enterprises.

The areas we examine

3.06
We assess and report on three areas:

• management control environment;
• financial information systems and controls; and
• service performance information and associated systems and controls.

3.07
The management control environment is the foundation of the control environment, and the areas that our audit may consider are:

• clarity of strategic planning;
• communication and enforcement of integrity and ethical values;
• participation by those charged with governance (for example, the involvement and influence of the audit committee and the board, or equivalent);
• risk assessment and management;
• legislative compliance arrangements;
• key entity-level control policies and procedures;
• assignment of authority and responsibility; and
• information systems and communication (including information technology planning and decision-making).

3.08
Financial information systems and controls are the systems and controls (including application-level computer controls) over financial performance and financial reporting. Examples of areas that our audit may consider are:

• appropriateness of information provided and reported;
• presentation of financial information;
• reliability of systems for collecting and reporting information;
• control activity (including process-level policies and procedures); and
• monitoring of information.

3.09
Service performance information and associated systems and controls refers to the quality of the service performance measures selected for reporting against, and the systems and controls (including application-level computer controls) over service performance reporting. Examples of areas that our audit may consider are:

• appropriateness of information provided and reported;
• presentation of information in the Statement of Service Performance;
• reliability of systems for collecting and reporting information;
• control activity (including process-level policies and procedures); and
• monitoring of information.

Our grading system

3.10
Auditors base the grades that they assign in their assessments on deficiencies observed through the audit, and on the associated recommendations for improvement. Auditors’ conclusions on deficiencies (that is, the gap between “actual practice” and “how practice should be”), and the associated recommendations for improvement, are based on their assessment of how far what the entity does is short of “good practice”. “Good practice” is based on auditors’ professional expertise and judgement, taking into account what is deemed appropriate for each entity, given its size, nature, and complexity. Our grading scale is shown in Figure 1.

Figure 1
Grading scale for assessment of environment, systems, and controls

Interpretation of results

3.11
Our auditors’ approach and the standards they apply reflect the unique circumstances of each entity in each financial year. Entities vary greatly in size and organisational structure, and sometimes undergo restructuring. Grades for a particular entity may fluctuate from year to year. Some of the factors that may contribute to such fluctuations include changes in the operating environment, standards, best practice expectations, and auditor emphasis. For these reasons, we advise caution when comparing grades between years and between different entities.

3.12
How an entity responds to the auditor’s recommendations for improvement, as they arise, is more important than the grade change from year to year. Because of the factors that may cause fluctuations, a downward shift in grade, for example, may not indicate deterioration - it may just be that the entity has not kept pace with good practice expectations for similar entities between one year and the next. Consequently, the long-term trend in grade movement is a more useful indication of progress than year-to-year grade changes.

3.13
In future years, we intend to further analyse our assessments, to provide more information on the main areas the entities involved need to improve. We will also provide comparative information and trend analysis. We also expect to make ongoing refinements to our assessment approach.

The results for 2006/07

3.14
We assessed the environment, systems, and controls in each of the entities we audited. We graded both the management control environment and the financial information systems and controls. For those entities required to prepare a Statement of Service Performance, we did not grade but provided comments on improvements they could make to their service performance information and controls.

3.15
We reported the results to the entity (the chief executive and the Board where relevant), the responsible Minister, and the select committee that conducts the entity's financial review.

3.16
Figure 2 shows a summary of the grades for the management control environment and financial information systems and controls.

3.17
We have not provided comparisons with the previous year because the assessment approaches are too dissimilar to be compared.

3.18
We allowed for a transitional period before we started providing gradings of service performance information and associated systems and controls. In 2006/07, we introduced a greater emphasis on the appropriateness of service performance information. In doing so, we expected the shortcomings identified in our reviews of service performance reporting to affect entities’ grades more significantly than they have to date. Our transitional approach allows entities time to adjust to this change of emphasis, and make the necessary improvements.

3.19
We will take the same approach in 2007/08, namely not grading service performance information and associated systems and controls but providing comments on where improvements can be made. Part 5 explains the reasons for this.

3.20
There are two main points to note from the summary of 2006/07 results.

• Many government departments and district health boards (DHBs) received a “needs improvement” grade for either the management control environment or financial information systems and controls. Of particular note is that 21% of departments and 24% of DHBs need to improve their management control environment, 33% of DHBs need to improve their financial information systems and controls, and 14% of DHBs needed to improve both. This is concerning, and we would expect the entities involved to take action to address the recommendations for improvement that the respective auditors have made.
• The results were better for Crown Research Institutes, other Crown entities, and State-owned enterprises, with only a small percentage needing to improve in either of these areas.

Figure 2
Summary of grades for 2006/07

Notes:

1. Areas covered in our assessment framework are:
   • MCE - Management control environment; and
   • FISC - Financial information systems and controls.
2. Ratings used are:
   • VG - Very good;
   • G - Good;
   • NI - Needs improvement; and
   • P - Poor.
3. The entities included in the summary are those referred to under the relevant categories in the Financial Statements of the Government of New Zealand for the year ended 30 June 2007 at pages 102 and 103. Government departments exclude Offices of Parliament, the Government Communications Security Bureau, and the Security Intelligence Service. School boards of trustees and tertiary education institutions are not included in Other Crown entities. Air New Zealand Limited has been included as if it were a State-owned enterprise. Terralink New Zealand Limited (in liquidation) and Electricity Corporation of New Zealand Limited have been excluded from State-owned enterprises.
4. The summary includes only one grade per entity, and uses the grades for the primary parts of the entities involved. For a small number of entities, and where we deem appropriate on a case-by-case basis, we report separate grades to cover different parts of the entities’ operations (for example, where there is a semiautonomous body operating within the entity). In 2006/07, the grades for the different parts differed in the case of only one of these entities.

1: For more information on the differences between the new and old frameworks, see our report Central government: Results of the 2005/06 audits, parliamentary paper B.29[07a], pages 25-29.