Part 5: Governing and monitoring the home-repair programme

5.1
Governance and monitoring are important components in effective management and control. Good governance requires effective monitoring of a programme's performance. Good controls require effective risk identification and management.

5.2
In this Part, we assess how the home-repair programme has been governed and managed. We look at:

• programme governance;
• operational controls;
• risk management and internal audit; and
• key performance indicators.

Summary of our findings

5.3
The home-repair programme is governed by a group consisting of senior EQC and Fletcher Construction staff. The group meets regularly and its decisions are clearly recorded. The group has identified that it needs further clarity on the authority given to it by EQC. We support that happening.

5.4
The introduction of key systems, controls, and support functions needed to keep pace with the evolving scale, nature, and complexity of the programme. They did not. Some systems, controls, and support functions needed to be in place earlier.

5.5
EQC has an understanding of the main risks to the operation of the home-repair programme and has investigated some aspects of operation in detail through two internal audits.

5.6
Performance metrics reported at a governance level have evolved during the home-repair programme and were under review when we were finalising this report. In our view, EQC's main measure of the home-repair programme's value for money is of limited use.

Programme governance

The home-repair programme is governed by a group consisting of senior EQC and Fletcher Construction staff. The group meets regularly, and its decisions are clearly recorded. The group has identified that it needs further clarity on the authority given to it by EQC.

5.7
EQC has recognised that it could improve the governance of the home-repair programme.

5.8
The PCG is the home-repair programme's governance body. The PCG's monthly meetings are formally carried out, have clear action points, and are well minuted.

5.9
As part of common practice, governance bodies carry out evaluations of their effectiveness. During its February and March 2013 meetings, the PCG discussed the need for the "PCG forum to be more effective in providing strategic leadership and direction for the Canterbury Home Repair Programme".

5.10
The minutes of the February 2013 meeting also state that "clarification on what authority has been given to PCG from EQC was needed in order to establish true functionality and purpose".

5.11
The PCG has recognised that clarifying its objectives will allow confirmation of the new terms of reference and resetting of key performance indicators.

Operational controls

The introduction of key systems, controls, and support functions needed to keep pace with the evolving scale, nature, and complexity of the programme. They did not. Some systems, controls, and support functions needed to be in place earlier.

Controlling the repair process

5.12
Although the circumstances behind the home-repair programme certainly meant action was required, the introduction of systems, controls, and support functions needed to keep pace with the scale, nature, and complexity of the programme. They did not. In our view, they needed to be in place earlier, because more than $500 million was spent during the first 18 months of the home-repair programme.

5.13
At the time of the first major Canterbury earthquake, there were no processes, procedures, and controls for setting up and running a home-repair programme. The processes, procedures (primarily a set of 17 Standard Operating Procedures, or SOPs), and controls had to be created "from scratch".

5.14
The SOPs are meant to ensure consistency between hubs. They cover topics such as scoping, contractor management, pricing, approvals, and variations to scopes of work. Fletcher Construction has also produced best trade-practice guides, including one for painting and decorating.

5.15
There have been, and there still are, inconsistencies in practice between repair hubs. We observed inconsistences in practice across the repair files that we reviewed and from our discussions with hub staff. Inconsistencies create the potential for inequitable treatment of people, but they can also be a source of innovation.

5.16
As well as the set of SOPs, other systems, controls, and support functions have had to be created or enhanced to match the complexity, risks, and nature of the home-repair programme. These include:

• detailed risk-based data analysis;
• fraud prevention and identification systems;
• programme-specific surveys; and
• complaints and mediation systems.

5.17
Mechanisms for detailed risk-based data analysis, integrated complaints management, mediation, and fraud prevention and identification, appropriate to the nature and scale of the home-repair programme, were finalised about one and a half years after the programme started. While these controls and support functions were not fully in place, there was a higher risk of programme weaknesses not being picked up and acted on.

5.18
EQC started regularly obtaining satisfaction feedback specific to the home-repair programme in February 2013.

Strengthening controls over the repair process

5.19
EQC's internal and contracted audit work and our annual financial audit work¹⁶ have identified the need for EQC to put in place further controls on aspects of its home-repair activities and to strengthen some of the controls already in place.

5.20
The overall state and effect of the evolving controls over the home-repair programme were summarised in a May 2012 internal audit report:

Fundamentally both [EQC and Fletcher Construction] lack effective quality control of processes that validate that functions are being performed consistently throughout the managed repairs network. This impacts costs and also the customer experience through the process.

5.21
The 2012 internal audit work also found "pockets of excellence" at every hub audited. These were practices that individuals in those hubs had put in place to create efficiencies, improve effectiveness, or enhance the control environment.

5.22
In May 2013, there was a follow-up to the 2012 internal audit. This showed that some progress had been made in addressing consistency matters, but there were still further improvements required.

Important tools and controls

5.23
Based on our analysis and observations during our audit work, we have identified potentially important tools and controls for managing a home-repair programme. These are described in Figure 16, including the reasons why they are important.

Figure 16
Tools and controls for managing a home-repair programme

Risk management and internal audit

EQC has an understanding of the risks to the operation of the home-repair programme and has investigated some aspects of operation in detail through two internal audits. EQC has operated the programme with a high level of awareness of the risks to EQC.

5.24
An important finding of the May 2012 internal audit was a lack of effective independent review by EQC of Fletcher Construction's quantity surveyor cost approvals and of invoices for home-repair work. The 2013 follow-up internal audit found a lack of reconciliation of financial information between EQC's and Fletcher Construction's systems.

5.25
Both internal audits found a degree of inconsistency in practices between repair hubs.

Risk management

5.26
Although EQC has never operated a large repair programme, it has, over several years, identified risks associated with settling claims by repairing damaged houses. This includes lessons from the small home-repair programme it put in place after a 2003 earthquake in Fiordland.

5.27
In our view, these lessons and risks should be considered by the managers of any future repair programme. The risks with a large-scale repair programme, identified by EQC before 2010, include:

• conflicts of interest if the people who manage the repair project are also directly involved in repair work;
• paying margins to project management providers that are outside of market conditions;
• further damage occurring before repairs are paid for;
• inadequate consideration of economic conditions in the area where the work is being done when assessing the cost of repairs;
• repairs being made to damage that was not caused by the earthquake;
• inadequate quality control or quality assurance checks of the repair work;
• much repair work involving redecoration (a type of repair work with which there can be a high level of homeowner dissatisfaction); and
• having a range of costly liabilities (for losses as a result of defective repairs and for obligations under the Health and Safety in Employment Act 1992, the Building Act 2004, and the Resource Management Act 1993).

Risks with the home-repair programme

5.28
EQC identified the main risks with the home-repair programme and has assessed the effect of those risks should they occur. We discuss how EQC has managed these risks elsewhere in this report. The main risks include:

• substandard repair work (see paragraphs 3.26-3.43);
• failure to provide adequate safety arrangements (see paragraphs 3.44-3.58);
• failure to manage contractor and staff well-being (EQC has identified a range of health and wellness activities to reduce this risk);
• failure to retain the necessary people and skills (EQC uses standard organisational health practices to reduce this risk, which are outlined in its annual reports);
• vulnerable people not being appropriately looked after (see paragraphs 3.79-3.82 and 3.86-3.97);
• a lack of streamlined end-to-end repair processes (see paragraphs 5.12-5.23);
• sub-optimal procurement of building materials (see paragraphs 6.18-6.26); and
• failure to manage fraud and other forms of exploitation (see paragraphs 6.27-6.35).

5.29
EQC and Fletcher Construction have held joint home-repair programme risk-identification workshops from 4 November 2011. EQC began to analyse in detail the contractor and repair data to inform risk identification from about July 2012.

Key performance indicators

Performance metrics are sent to the PCG to help govern the home-repair programme. These metrics have evolved during the programme and were being reviewed when we were finalising this report. In our view, EQC's main measure of the home-repair programme's value for money is of limited use.

5.30
Some metrics reported to the PCG are not consistently reported (for example, the results of quality audits of completed repairs), some metrics have no targets in the reported material (for example, the number of houses to be repaired each month), and there is no comparative data reported for some metrics (for example, the number and types of complaints received).

5.31
EQC has set targets for finishing all repairs in the home-repair programme and for overall claims-handling costs as a percentage of total costs (this measure is wider than the home-repair programme).

5.32
"Value for money" metrics are identified in monthly reports to the PCG. These are percentage differences between:

• the "original budget" (the assessed damage) and the "adjusted budget" (the approved work scope); and
• the "adjusted budget" (the approved work scope) and the final invoiced cost.

5.33
We commented in Part 3 on the reasons for variance between assessed damage and the approved work scope budgets. The reasons are a combination of matters outside EQC's control, such as multiple earthquakes and hidden damage, and matters within EQC's control, such as the quality of damage assessments.

5.34
Because of the range of matters contributing to percentage increases, including many outside of EQC's control, EQC instead focuses on the increase in costs between the adjusted budget (the approved work scope) and the final invoiced cost as the main measure of its performance. So far, the cost increase is less than 2%.

5.35
In our view, this measure of the home-repair programme's value for money is of limited use. It does not cover all the main elements of value for money in the circumstances, and EQC adjusts the approved scope of work as more information is made available. This means that large differences between the approved scope and the final cost are unlikely.

5.36
When we were finalising this report, EQC was reviewing the key performance indicators for the home-repair programme. We encourage EQC to also consider the relationship between the key performance indicators and the objectives of the home-repair programme.

16: The need to strengthen controls was noted in our Office's briefings to the Finance and Expenditure Committee about EQC, available at www.parliament.nz (in the Advice section, Committee documents).