Part 2: Assessing entities' environment, systems, and controls
2.1
In this Part, we report on our 2007/08 assessments of the environment, systems, and controls of government departments, Crown entities (excluding school boards of trustees and tertiary education institutions), and State-owned enterprises.
Background
2.2
As part of the annual financial audit, our auditors examine, assess, and grade central government entities’ environment, systems, and controls for managing and reporting financial and service performance information. We report these assessments to the entity, the responsible Ministers, and the relevant select committees.
2.3
Our auditors examine an entity’s environment, systems, and controls in the context of their work in forming an opinion on the financial and service performance statements. The purpose of commenting on these aspects is to highlight the areas for improvement that the audit identified. The grades assigned directly represent the recommendations for improvement as at the end of the financial year.
2.4
We introduced a new assessment framework in 2006/07 to improve the transparency, usefulness, and clarity of our reporting. It replaced the framework that we had used for the previous 13 years.
2.5
We apply our assessment framework to:
- government departments;
- Crown entities, excluding school boards of trustees and tertiary education institutions; and
- State-owned enterprises.
The areas we examine
2.6
We assess and report on three areas:
- management control environment;
- financial information systems and controls; and
- service performance information and associated systems and controls.
2.7
The management control environment is the foundation of the control environment. The areas that our audit may consider are the public entities’:
- clarity of strategic planning;
- communication and enforcement of integrity and ethical values;
- participation of people with governance responsibilities;
- overall legislative compliance arrangements;
- major control policies and procedures;
- risk identification, assessment, and management practices;
- information systems and communication to support the implementation and maintenance of financial and service delivery intentions and controls;
- monitoring of policies and processes;
- commitment to competence;
- organisational structure and assignment of authority and responsibility; and
- management philosophy and operating style, and whether they emphasise effectiveness and efficiency.
2.8
Financial information systems and controls are the systems and controls (including application-level computer controls) over financial performance and financial reporting.
2.9
Service performance information and associated systems and controls refer to the quality of the service performance measures selected for reporting against, and the systems and controls (including application-level computer controls) over service performance reporting.
2.10
Examples of areas that our audit may consider under both the financial and service performance systems and controls aspects are:
- appropriateness of information provided and reported;
- presentation of information in the statement of service performance (SSP);
- reliability of systems;
- control activity (including process-level policies and procedures); and
- monitoring of information.
Our grading system
2.11
Auditors base the grades that they assign on deficiencies observed through the audit (that is, the gap between “actual practice” and “how practice should be”), and on the associated recommendations for improvement. Auditors base their conclusions on deficiencies, and the associated recommendations for improvement, on their assessment of how far what the entity does is short of “good practice”. “Good practice” is based on auditors’ professional expertise and judgement, taking into account what is considered appropriate for each entity, given its size, nature, and complexity. Figure 1 shows our grading scale.
Figure 1
Grading scale for assessment of environment, systems, and controls
| Grade | Explanation of grade |
|---|---|
| Very good | No improvements are necessary. |
| Good | Improvements would be beneficial and the entity should address these. |
| Needs improvement | Improvements are necessary and should be addressed at the earliest reasonable opportunity. |
| Poor | Major improvements are required, to which the entity should give urgent attention. |
Interpretation of results
2.12
Our auditors’ approach and the standards they apply reflect the circumstances of each entity in each financial year. Entities vary greatly in size and organisational structure, and sometimes undergo restructuring. Grades for a particular entity may fluctuate from year to year. Some of the factors that may cause fluctuations include changes in the operating environment, standards, good practice expectations, and auditor emphasis. For these reasons, we advise caution when comparing grades between years and between different entities.
2.13
How an entity responds to the auditor’s recommendations for improvement is more important than the grade change from year to year. A downward shift in grade, for example, may not indicate deterioration – it may just be that the entity has not kept pace with good practice expectations for similar entities between one year and the next. Consequently, the long-term trend in grade movement is a more useful indication of progress than year-to-year grade changes.
The results from 2006/07
2.14
We noted in last year’s article that we intended to analyse our assessments further, to highlight important issues and trends underlying our assessments and grades.
2.15
This further analysis on last year’s results indicates that there is a wide variety of factors affecting individual entities, and the auditors’ assessment and grading. There were, however, some common themes.
2.16
In the district health board sector, we reported last year that 24% of entities needed to improve their management control environment, and that 33% needed to improve their financial information systems and controls. Our analysis of factors behind these grades shows that the single most commonly occurring issue is poor procedures and practices for procurement. This was also a significant issue for those district health boards that we graded as good.
2.17
Under our framework, this means that improvements in procurement are necessary for a significant number of district health boards and should be addressed at the earliest reasonable opportunity. For a significant number of other district health boards, improvements in procurement would be beneficial and should be addressed (see Part 5, paragraphs 5.34-5.49).
2.18
For the government department sector, we reported last year that 21% needed to improve their management control environment, and 16% needed to improve their financial information systems and controls. From our analysis, there were fewer dominant individual themes within this sector. This partly reflects the diverse nature of government departments.
2.19
The quality of procurement procedures and practices was also a theme within the management control environment for the government department sector, but the most frequently occurring issue was legislative compliance. For this issue, we routinely reported that agencies did not have a process for continuous review of, or positive assertion over, their ongoing compliance with relevant legislation.
The results for 2007/08
2.20
We assessed the environment, systems, and controls in each of the entities we audited. We graded both the management control environment and the financial information systems and controls. For those entities required to prepare a SSP, we did not grade their service performance information and controls but provided comments on improvements they could make.
2.21
We reported the results to the entity (the chief executive and the Board where relevant), the responsible Minister, and the select committee that conducts the entity’s financial review.
2.22
We have allowed for a transitional period before we start grading service performance information and associated systems and controls, and so have not graded this area in 2006/07 or 2007/08. Since 2006/07, we have placed a greater emphasis on the appropriateness of service performance information. In doing so, we expected the shortcomings identified in our reviews of service performance reporting to affect entities’ grades more significantly than they have to date. Our transitional approach allows entities time to adjust to this change of emphasis, and make the necessary improvements.
2.23
We intend to begin grading service performance information and associated systems and controls in 2008/09. This will reflect both:
- how appropriately entities specify the measures they will report future performance against (with an emphasis on the development of the entity’s relevant accountability plan for 2009/10 and beyond – for example, its Statement of Intent); and
- entities’ reporting of their performance for 2008/09.
2.24
Figure 2 shows a summary of the grades, by type of entity, for the management control environment and financial information system controls. As in 2006/07, we graded no entities as poor either for the management control environment or for the financial information systems and controls.
Figure 2
Summary of grades by type of entity for 2007/08, compared with 2006/07
| Summary of grades for 2007/08 | Number of entities | Grades received for MCE (%) | Grades received for FISC (%) | ||||
|---|---|---|---|---|---|---|---|
| VG | G | NI | VG | G | NI | ||
| Government departments | 39 | 28 | 59 | 13 | 23 | 62 | 15 |
| District Health Boards | 21 | 5 | 71 | 24 | 0 | 67 | 33 |
| Crown Research Institutes | 9 | 67 | 33 | 0 | 11 | 89 | 0 |
| Other Crown entities | 65 | 58 | 37 | 5 | 51 | 46 | 3 |
| State-owned enterprises | 17 | 76 | 12 | 12 | 29 | 71 | 0 |
| Summary of grades for 2006/07 | Number of entities | Grades received for MCE (%) | Grades received for FISC (%) | ||||
|---|---|---|---|---|---|---|---|
| VG | G | NI | VG | G | NI | ||
| Government departments | 38 | 13 | 66 | 21 | 18 | 66 | 16 |
| District Health Boards | 21 | 0 | 76 | 24 | 0 | 67 | 33 |
| Crown Research Institutes | 9 | 56 | 44 | 0 | 11 | 89 | 0 |
| Other Crown entities | 65 | 53 | 42 | 5 | 32 | 63 | 5 |
| State-owned enterprises | 18 | 66 | 28 | 6 | 17 | 78 | 5 |
Areas covered in our assessment framework are:
- MCE – Management control environment;
- FISC – Financial information systems and controls; and
- SPIASC – Service performance information and associated systems and controls (which was assessed but not graded in the 2006/07 financial year).
Ratings used are VG – Very good; G – Good; NI – Needs improvement; and P – Poor.
The entities included in the above analysis are those referred to under the relevant categories in the Financial Statements of the Government of New Zealand for the Year Ended 30 June 2008 at pages 167 and 168. Government departments exclude Offices of Parliament, the Government Communications Security Bureau, and the Security Intelligence Service. School boards of trustees and tertiary education institutions are not included in the above analysis for other Crown entities. Air New Zealand Limited has been included as if it were a State-owned enterprise. The Electricity Corporation of New Zealand Limited and Terralink New Zealand Limited (in liquidation) have been excluded from the analysis for State-owned enterprises.
The summary includes only one grade per entity, and uses the grades of primary parts of the entities involved. For a small number of entities, and where we deem appropriate on a case-by-case basis, we report separate grades to cover different parts of the entities’ operations (for example, where there is a semi-autonomous body operating within the entity).
2.25
Taking all entities that we assessed and graded as a whole, there has been a marginal improvement in grades since 2006/07. The proportion of very good grades has increased in 2007/08 for both the management control environment and for financial information systems and controls. This reflects that entities are improving their grades from good to very good, as the proportion of needs improvement grades is similar in both years for both aspects. This is shown in Figures 3 and 4.
Figure 3
Management control environment for 2007/08, compared with 2006/07
Figure 4
Financial information systems and controls for 2007/08, compared with 2006/07
2.26
There are different features and trends within the individual sectors underlying this overall improvement, which we discuss below.
2.27
We noted in last year’s article that a significant number of district health boards and government departments were graded as needing improvement in both the management control environment and the financial information systems and controls. This remains the case in 2007/08.
2.28
For government departments, the 2007/08 grades show some improvement from 2006/07. In 2007/08, for both the management control environment and the financial information systems and controls, a smaller percentage of government departments were graded as needing improvement and a larger percentage were graded as very good.
2.29
It remains of concern that 13% and 15% of government departments need to improve their management control environment and their financial information systems and controls, respectively.
2.30
In the case of the district health boards (except for one district health board that achieved a very good grade for its management control environment), the 2007/08 results show that the sector did not manage to improve its grades from 2006/07.
2.31
Nearly a quarter (24%) of district health boards still need to improve their management control environment, and one-third of district health boards need to improve their financial information systems and controls. This lack of improvement is unsatisfactory.
2.32
In the State-owned enterprise sector, the percentage of entities graded as very good for both the management control environment and the financial information systems and controls has increased.
2.33
However, in this sector, the percentage of entities graded as needing improvement in the management control environment has doubled. Of State-owned enterprises, 12% − a significant proportion − are now graded as needing improvement in this area.
2.34
We expect entities to take appropriate action to address the matters raised by our auditors and to achieve the recommended areas for improvement.
2.35
We analysed the issues raised in 2006/07, and the actions taken by agencies to address them during 2007/08. This analysis indicates that most of the issues we raised were addressed, and the agencies are to be commended for this. We now intend to focus on the areas where our recommendations for improvement were not addressed.