Part 2: Overall findings
2.1
In this Part, we outline:
- some of the characteristics of, and changes in, the environment that EQC has been working in since 2013;
- EQC's improvement activities since our 2013 report;
- our assessment of the performance of the programme against the dimensions of quality, timeliness, cost, and quantity; and
- our overall assessment of the performance of the programme.
A challenging, changing, and complex operating environment
2.2
No repair programmes are directly comparable with EQC's programme. This limits EQC's options for making informed direct comparisons about its performance relative to other organisations.
2.3
The environment that EQC has been operating in since our 2013 report has continued to be challenging, changing, and complex.
2.4
Within this environment, EQC has continued to operate and improve the programme, while successfully obtaining reinsurance. We described the importance of ongoing reinsurance cover in our 2013 report. Since that report, EQC has continued to obtain reinsurance while working to improve the programme.
2.5
In short, the direct cost of the programme to taxpayers would be higher without reinsurance, and the wider New Zealand insurance industry could have been affected. Reinsurers have regularly looked at EQC's performance as part of deciding whether to continue to provide reinsurance.
2.6
EQC has had to work with several decisions and events since 2013. EQC has adapted to and accommodated these decisions and events, including:
- additional guidance released by MBIE on assessing, repairing, or rebuilding multi-unit dwellings, and site ground improvement;
- WorkSafe New Zealand (WorkSafe) concluding its investigation into the management of asbestos in the programme;
- the Royal Society of New Zealand and the Office of the Prime Minister's Chief Science Advisor review of scientific evidence about the risks of asbestos exposure to residents of houses undergoing renovation and repair work; and
- MBIE finalising its reports about 13 EQC home repairs as part of its Canterbury earthquake damage and repair work, and assessing repairs to 101 homes.
The Earthquake Commission's progress since our 2013 report
2.7
Figure 1 shows the main improvement activities that EQC has carried out since our 2013 report. EQC has improved all the areas of activity that we made recommendations for in our 2013 report.
2.8
These improvements have helped EQC to:
- continue to manage overall repair cost inflation well;
- promote good health and safety practices;
- give some customers more certainty about repairs;
- have a better understanding of what it needs to do to become a more customer-focused organisation;
- have a better understanding of the quality of repairs delivered in the programme; and
- know what is required to deliver a similar programme better if it needs to in the future.
Effective management of repair costs but mixed customer experiences
2.9
We have considered EQC's improvements since our 2013 report in determining our overall assessment of the programme's performance against EQC's goals. These were that all repairs are completed properly, safely, as quickly as practicable, and in a manner that provides value for money in the circumstances.
2.10
EQC did not anticipate the scale of multiple events and circumstances that it has faced. It was forced to work in a reactive manner from a "standing start". EQC had to draw from the limited workforce available in Canterbury. EQC also needed to clearly identify earthquake damage and then repair that damage.
2.11
EQC set up the programme quickly and made ongoing improvements to its management of the programme. We provide our assessment of the performance of the programme in Figure 2 in terms of quality, cost, timeliness, and quantity since it started.
2.12
In Figure 2, we break down:
- quality into technical repair quality, customer interactions, surveyed customer satisfaction with quality, and health and safety;
- cost into repair cost and programme management cost; and
- timeliness into performance against the time frames set by EQC for the programme, and the overall time frame in which the programme has been delivered.
Figure 1
Main improvement activities since 2013 against the dimensions of quality, cost, timeliness, and quantity
| Quality | |||
|---|---|---|---|
| Technical quality of repairs | Quality of customer interactions | Surveyed customer satisfaction with quality | Management of health and safety |
| Increased attendance by the quality assurance team at repair sign-off visits. Completed a rationalisation of repair hubs. Strengthened contractor management. |
Commissioned a report on the quality of EQC's end-to-end customer interactions, which advised that "the journey towards a customer focused organisational model and culture is a strategic necessity". | Maintained high levels of surveyed customer satisfaction with repair quality, immediately after a repair has been completed. | Continued leadership of good health and safety practices in construction work. |
| Cost | |
|---|---|
| Project management cost | Repair work cost |
| More complete and consistent reporting of programme performance. Signed a contract variation with Fletcher Construction. |
Maintained ongoing good management of repair cost inflation. As at 30 June 2014, EQC estimates that the cost of a repair is on average 14.2% higher than it was in February 2011. In comparison, the Canterbury inflation rate for purchasing a new house, which will have been affected by similar cost pressures and industry cost structures as home repair work, was 30.9% during roughly the same period. EQC told us that its underlying labour rates for repair work have not increased since April 2014. The Canterbury inflation rate for the cost of a new house increased by 3.96% in 2014/15. |
| Timeliness | |
|---|---|
| Timeliness against targets | Overall timeliness of repair completion |
| Introduced a certainty initiative to give customers more certainty about when their homes would be repaired. | Increased focus on outstanding repairs and the issues delaying completion of repairs. |
| Quantity |
|---|
| Volume of repairs completed |
| Completed 26,156 primary substantive repairs from July 2013 to June 2015. |
Figure 2
Performance of the programme since it started against the dimensions of quality, cost, timeliness, and quantity
| Quality | |||
|---|---|---|---|
| Technical quality of repairs | Quality of customer interactions | Surveyed customer satisfaction with quality | Management of health and safety |
| According to the survey carried out immediately after repairs have been completed, customer satisfaction with quality remains high, generally in the range of 80%-90%. The programme has put a lot of emphasis and effort into supporting good health and safety practices. These efforts have been successful, even though there have been some risks with asbestos management (as investigated by WorkSafe). EQC has identified that about 8%-10% of repairs have needed additional work after the original work has been completed. Some repair work has not met the requirements of the Building Code, with some problems with the programme's quality controls in some instances. EQC does not have formal processes for learning from complaints. EQC has obtained advice on how to improve its customer service. The advice has identified that "the journey towards a customer focused organisational model and culture is a strategic necessity". |
|||
| Cost | |
|---|---|
| Project management cost | Repair work cost |
| Programme management costs have increased in absolute and proportional terms. About $340 million had been spent on project management to the end of June 2015. Repair cost inflation has been well managed. |
|
| Timeliness | |
|---|---|
| Timeliness against targets | Overall timeliness of repair completion |
| EQC's original target for completing all repairs was December 2015. It brought this date forward to December 2014 but has not met this target and some other targets. Some homeowners have not received the level of certainty they were expecting and have had to wait long periods of time for information. EQC has not completed repairs for vulnerable people significantly sooner than for other customers. However, it has, on average, issued work orders to begin repair work sooner for those customers. Efforts have been made to work with vulnerable people in the programme. During the programme, EQC introduced an initiative to give customers more certainty about when their homes would be repaired. |
|
| Quantity |
|---|
| Volume of repairs completed |
| The programme had, as at 30 June 2015, practically completed 66,252 repairs. The programme has also completed 65,642 emergency repairs and 19,499 clean heat installations (based on information from Fletcher Construction as at 1 October 2015). |
Our overall assessment
2.13
For several reasons, it is difficult to reach an overall conclusion on the performance of the programme in terms of efficiency (whether results have been maximised for the cost), effectiveness (whether the programme has delivered the expected results), and economy (whether the cost has been reasonable). These reasons include:
- the changing and complex circumstances in which the programme is operating;
- the lack of directly comparable benchmarks from equivalent repair programmes;
- the trade-offs that have to be made in the programme, including between timing and cost;
- some uncertainty about the quality of repairs achieved in the programme;
- stakeholders' differing experiences of the programme; and
- broadly defined programme goals – to properly complete all repairs, safely, as quickly as practicable, and in a manner that provides value for money in the circumstances.
2.14
Two aspects of the performance of the programme are particularly notable. The first is EQC's effective management of repair cost inflation – meaning that repair costs appear to be economic in the circumstances. The second is EQC's mixed performance in terms of customer interactions and experience – meaning that the programme has not been fully effective in the circumstances for some customers, including some vulnerable people.
2.15
Some people who are still waiting for repairs to be completed may not be able to live in their homes. Their frustration with the performance of the programme is understandable. Others who have faced long periods of uncertainty about the status of repair work or who have needed more work done after the original repair work may also be frustrated with performance.
2.16
It is important that EQC does not lose sight of the ongoing importance of effective service delivery and learning from customers' complaints.