Part 3: Repair quality

3.1
In this Part, we describe EQC's progress against our 2013 recommendation about repair quality.

3.2
We recommended that EQC continue to improve its approach to auditing repairs in the programme so that it is well informed about the scale and type of repair quality risks, can mitigate those risks where possible, and can match the resourcing of its quality assurance processes to the significance of those risks.

3.3
The Appendix provides further information on what we found and recommended in 2013 about EQC's management of repair quality.

3.4
It is important to note that repair quality issues range from minor, such as touch-ups to internal painting, to less minor, such as poor sub-floor work where piles have not been connected to bearers.

Summary

We found it difficult to assess EQC's overall performance in managing repair quality, even though EQC has improved its understanding of repair quality. EQC's survey of customer satisfaction immediately after repairs have been completed shows high levels of customer satisfaction with the quality of repairs. However, some of the programme's repair work has not met the requirements of the Building Code. There is a level of rework in the programme, but there are no definitive directly comparable benchmarks to compare this against. EQC does not have a formal process for learning from complaints. EQC has continued to support good health and safety practices.

3.5
One of the main objectives of the programme was to repair homes to a consistent quality. EQC has continued to improve its approach to auditing repairs in the programme. EQC is better informed about the scale and type of risks to the quality of repairs. It has taken steps to mitigate those risks, but there have been some problems with quality control for some repairs within the programme.

3.6
When assessing quality, several factors must be considered. These include whether the repairs were effective from a customer and a technical point of view, whether interactions with customers were good during the repair, and whether good health and safety practices were used.

3.7
We found it difficult to assess EQC's overall performance in managing repair quality, even though EQC has improved its understanding of repair quality.

3.8
Forming a conclusion on the overall quality of repairs is difficult. On one hand, there are problems with the quality of some repairs. On the other hand, many thousands of people are residing in repaired houses. Homeowners' perceptions of the quality of repairs depend heavily on their individual circumstances and experiences.

3.9
EQC has continued to support good health and safety practices. This is reflected in low numbers of serious harm incidents and low numbers of injuries for every million hours worked, compared with a construction industry benchmark. This is a considerable achievement.

Improvements and continued activities

Improvements

3.10
Since 2013, EQC and Fletcher EQR (EQR) ⁴ have between them:

• introduced a quality assurance process that involves EQC visiting a large sample of homes when completed repairs are signed off;
• introduced a guideline requiring that work is not to be signed off with known defects or remedial work needed;
• introduced "circuit breaker" meetings to progress difficult claims with complex repair issues;
• introduced a Claim Record Book for each dwelling repair that requires project completion certificates to be issued only after it is confirmed that all minor defects have been rectified (EQC could not tell us how many project completion certificates had been issued);
• strengthened contractor management through a tiered approach, with contractors being allocated to a tier based on their performance, more regular reviews of contractor performance, and rationalising of contractors;
• completed a rationalisation of repair hubs with the aim of achieving more consistent repair processes and documentation over a reduced number of operating locations;
• transferred control of sampling and testing for asbestos from contractors to EQR; and
• looked for building industry benchmarks for repair quality to compare its performance against.

3.11
EQC's quality assurance work includes monitoring remediation work, investigating complaints, quality assurance inspections, and attending the "sign-off" stage of a repair.

3.12
Aspects of EQC's strengthening of its quality assurance work were in response to some issues with the quality and supervision of repair work in the programme.

3.13
When we did our follow-up work, EQC was continuing to assess the adequacy of its quality assurance processes with the aim of reporting on this to EQC's Board in October 2015. EQC has recognised that inconsistent practices, inconsistent definitions of quality assurance, and a lack of documentation are impediments to achieving adequate quality assurance processes. ⁵

Continued activities

3.14
Since 2013, EQC has continued to:

• maintain an effective focus on health and safety;
• survey customer satisfaction with repair quality, and other matters, immediately after repairs have been completed;
• use the standard industry concept of practical completion in relationships with contractors; ⁶
• review contractor performance; and
• investigate issues such as fraud through its Claims Review Team.

3.15
EQC's support of good health and safety practices has been successful. EQC told us that there have been 38 million hours of work without a fatality. We comment further in paragraphs 3.25-3.27 about the low levels of injury within the programme.

Activities not performed

Recommendations from the Ministerial review of EQC's customer satisfaction survey

3.16
EQC has not addressed some recommendations from a Ministerial review of the customer satisfaction survey that EQC conducts immediately after repairs have been completed. KPMG carried out the review, ⁷ to which we had input.

3.17
EQC continues to conduct this survey. The Ministerial review suggested that a different organisation conducting the survey would strengthen perceptions of independence.

3.18
EQC decided not to have an independent third party conduct the survey because EQC wanted to preserve the opportunity for its call centre agents to assist customers with any queries and address any issues raised during calls. EQC also achieved cost savings by conducting the survey itself.

3.19
However, not having an independent third party conduct the survey still runs the risk of a perception of a lack of independence. EQC told us that, in its view, outsourcing the survey would not lead to improved customer outcomes, and the benefits of EQC conducting the survey outweighed the benefits of outsourcing the survey.

3.20
EQC has also not addressed a recommendation in the Ministerial review that it document the purpose and methodology of each of the two customer satisfaction surveys it uses, including the customer satisfaction survey conducted immediately after repairs have been completed.

Recording of repair quality information

3.21
We referred to the introduction of Claim Record Books in paragraph 3.10. These contain hard copy templates of forms to be completed at various stages during the repair. They are intended to serve as records of the repair work and support consistent repair management processes. In our view, information in the Claim Record Books could be collated to provide information about the quality of repair work.

3.22
EQR told us that it scans forms from the Claim Record Books into its systems to enable monitoring. However, EQR does not collate important information from the scanned forms, including:

• the reasons for missing homeowners' signatures – a possible indicator of whether the sign-off process is being properly followed;
• the number of practical completion certificates issued – a record by the person responsible for the repair that a repair has been completed;
• the number of invoices from repair contractors that have not been paid because the necessary documents have not been provided (such as a PS3, Memorandum of Building, inspection records, and others) – a possible indication that some of the repair process steps were not followed; and
• the proportion of completed repairs that have the necessary documents on file (such as a PS3, Memorandum of Building, inspection records, and others) – a possible indication that some of the repair process steps were not followed and a possible indication of the quality of record-keeping.

3.23
Project management of the programme was expected to involve maintaining full, complete, and accurate records. EQC told us that EQR started finalisation processes in 2014 to check that the required paperwork – such as that described in paragraphs 3.21-3.22 – is available, anomalies identified, and further investigation completed. As at late October 2015, most of these processes are yet to be carried out for repair work that needed the oversight of the technical repair hub. EQC told us that, in early November 2015, there were about 3600 claims overseen by the technical repair hub that still needed these processes to be finalised.

3.24
The monthly audits of a proportion of repairs against repair work standards in the programme that we described in our 2013 report ended in June 2014. EQR did this auditing, which EQC considered to be a limited quality review process. EQC's quality assurance work has been strengthened since then.

Forming an overall conclusion on the quality of repairs is difficult

Health and safety practice

3.25
EQC has placed much emphasis on health and safety in the programme. An indicator of this is the 12-month rolling average of the number of reported injuries for every million hours worked in the programme. From March 2014 to March 2015, this rate has been declining and generally below the target rate of no more than six injuries reported for every million hours worked. This compares favourably with industry norms.

3.26
One indicator of these industry norms is the 2014 results from the Business Leaders' Health & Safety Forum's Benchmarking Report. ⁸ This found that, for the members of that forum participating in the benchmarking in 2014, there was an average of about 46 injuries for every million hours worked in the construction sector.

3.27
From July 2013 to March 2015, there have been only 11 serious harm incidents reported in the programme. In that period, about 26,000 repairs were completed. The number of reported injuries for every million hours worked is low compared to the volume of work carried out in the programme.

3.28
Asbestos management is an area of risk. WorkSafe launched an investigation in response to allegations about the adequacy of EQC's and EQR's systems for identifying and managing asbestos during the initial stages of the rebuild.

3.29
WorkSafe concluded in October 2014 that, during the programme, considerable improvements have been made in the way contractors manage asbestos. WorkSafe found that the level of asbestos likely to have been released was very low, as was the risk to workers. WorkSafe also found that the risk to residents is likely to have been even lower. ⁹

3.30
An April 2015 report on behalf of the Royal Society of New Zealand and the Office of the Prime Minister's Chief Science Advisor looked at asbestos exposure in New Zealand. ¹⁰ It found that:

… remediation activities such as those that have taken place in Canterbury are unlikely to result in any significant increase in risk to homeowners and occupants of damaged houses, unless they were performing the work themselves, without taking proper precautions such as wetting the surfaces and using a respirator.

Repair quality

Surveyed customer satisfaction

3.31
There is a high level of surveyed customer satisfaction with the quality of repairs. Figure 3 shows the percentage of customers who rated their satisfaction with the quality of repairs immediately after the repairs had been completed as being "satisfied" or "very satisfied" (using a four-week rolling average).

Figure 3
Percentage of surveyed Earthquake Commission customers satisfied with the quality of repairs on completion, July 2013 to April 2015

Source: Graph based on information provided by EQC.

3.32
Using the survey question on satisfaction with the quality of repairs alone as a measure for the quality of repairs is narrow and does not provide a full view of repair quality. This is because the measure:

• is only one of several satisfaction measures surveyed at the same time;
• is experiential in nature (and this is only one type of quality measure) and is limited to the work that homeowners can see (for example, cosmetic repairs rather than foundation repairs);
• is conducted before some defects have had time to appear;
• runs the risk of a lack of independence because EQC conducts the survey and EQC has not taken the steps the Ministerial review of the survey recommended to address this perception risk; and
• relies on there being a direct link between repair quality and customer satisfaction.

3.33
In EQC's view, most repairs in the programme are cosmetic, with defects being typically immediately obvious – that is, within a week of the repairs being completed. Because of this, EQC does not believe the survey question about the quality of repairs is too narrow.

3.34
The link between actual repair quality and customer satisfaction is not exact. For example, a 2014 Building Research Association of New Zealand (BRANZ) survey of new homeowners found that the proportion of respondents who were at least fairly satisfied with the overall quality of the work was about the same as the proportion of those who had to call back their builder to fix defects. This means that customer satisfaction is not a direct measure of the actual quality of repairs, depending on how quality is defined.

3.35
Another example is the MBIE report we discuss in paragraphs 3.50-3.60. About two-thirds of the participants in the MBIE report indicated that they were satisfied with their repairs even though slightly more than half of the repairs covered by the survey did not comply with the Building Code or potentially had minor defects.

3.36
The BRANZ surveys also found that defects reported by homeowners mainly related to finishes rather than to weathertightness and durability issues. This is because those issues are not immediately visible to homeowners. ¹¹

3.37
In our view, EQC's customer satisfaction survey measures the perceived quality of repairs, not the actual quality of repairs. All of the measures in the customer satisfaction survey are "experiential" – that is, based on observation and experience.

3.38
Carrying out the customer satisfaction survey within a week of repairs being completed means that repair quality issues could arise after this time.

Unanticipated additional work rate

3.39
We attempted to determine the normal industry rate of rework and unanticipated additional work for repairs to residential dwellings. We looked at publicly available information from surveys and research reports, and met with informed industry participants. The comments we received and the information we obtained did not present a consistent view or a view that could be directly compared with EQC's repair work.

3.40
However, it was clear that some level of rework and unanticipated additional work is normal in the residential construction industry. There is also some level of non-compliance in the industry with the requirements of the Building Code on first inspection by a building consent authority.

3.41
The indicators we found include those from the 2010 Research New Zealand survey, those used in BRANZ's annual surveys of new homeowners, and BRANZ's site inspections of 225 homes under construction. ¹² None of these indicators was directly comparable with EQC's repair work.

3.42
However, EQC has used the results of a 2010 Research New Zealand survey ¹³ as an internal benchmark to measure its own estimated remedial repair rate against.

3.43
In comparing its performance with the findings of the 2010 Research New Zealand survey, EQC told its Minister that "Approximately 8% of the homes within the programme have required some remedial action, a rate that is lower than comparable building programmes."

3.44
In our view, this is not an appropriate comparison. We have shared our detailed analysis with EQC, explaining why we came to this conclusion.

3.45
Figure 4 shows that some repairs with defects or deferred work were signed off between November 2013 and April 2015. This is for about 20% of the site visits made by EQC's quality assurance team. Only two-thirds of repairs get signed off at the first completion inspection without defects or deferred works.

Figure 4
Levels of sign-off of repairs during quality assurance team visits, November 2013 to April 2015

Source: Graph based on information provided by EQC.

3.46
EQC estimates that about 8%-10% of homes repaired in the programme have needed some aspect of the repair work to be remedied. We refer to this work in this report as "unanticipated additional work". We have examined the methodology and documentation EQC has used to determine this rate. There is a logic to EQC's analysis, but it is clear that the figure is an estimate.

3.47
EQC estimates that the quality of workmanship is a contributing factor in about 70% of the unanticipated additional work cases by volume (30% by value). These costs are potentially recoverable from the contractor. Other contributing factors are failed repair strategy, ¹⁴ damage omitted on scopes of work, missed work included on scopes of work, and new damage. Figure 5 shows the value of unanticipated additional work performed during the programme by cause, between June 2014 and June 2015. EQC told us in July 2015 that the current average cost of unanticipated additional work at the time of our work was about $6,500 for each repair.

Figure 5
Unanticipated additional work by cumulative value and cause, June 2014 to June 2015

Source: Graph based on information provided by EQC.

3.48
EQC has a potential liability under the Building Act 2004 for implied warranties and under the Consumer Guarantees Act 1993 for the next one to 10 years.

3.49
In proportional terms, the cost of unanticipated additional work carried out to date is very small (less than 1.2% of total repair costs). Although there is some degree of uncertainty about the absolute cost of unanticipated additional work during the full life of the programme, the estimated cost is also very small in proportional terms (less than 1.4% of total repair costs).

Ministry of Business, Innovation and Employment's repair survey

3.50
A further indicator of repair quality is the findings of MBIE's work looking at:

• 13 complaints about EQC's repair work; and
• 101 repairs in Christchurch, most of which (74) are EQC repairs.

3.51
It is important to note that MBIE's findings are not statistically representative of repairs throughout the programme.

3.52
MBIE's first piece of work provides an indication of the types of risks that EQC needs to be alert to. These include non-compliance with the Building Code for reasons such as poor-quality sub-floor repair work (for example, using inappropriate packing material, not securing piles to bearers, or using inappropriate foundation fillers), and repair strategies that are inconsistent with MBIE guidelines.

3.53
MBIE's second piece of work provides some indicative information about the quality of certain repairs that have been carried out under the building consent exemption in the Building Act. ¹⁵

3.54
MBIE released its report on 19 August 2015. The report looked at completed structural repairs that were exempt from requiring a building consent under Schedule 1 of the Building Act. The sample of homes with exempt structural repairs is considered to be a relatively small proportion of the total number of homes covered by the programme.

3.55
Twenty-six of the 74 homes in the sample repaired by EQC (selected by MBIE from a list of 2182 homes provided by EQC against MBIE criteria) were found to be non-compliant with the structural and/or durability provisions of the Building Code.

3.56
Repairs to a further 23 homes were found to potentially have minor defects, although it is not clear how many of these 23 homes were EQC repairs.

3.57
MBIE did not identify any potentially lethal risks, and it considers that remedying the non-compliant repairs would be relatively easy for most homes. MBIE found that non-compliance issues were essentially restricted to "jack and pack" ¹⁶ repairs and crack repairs of perimeter concreted foundations in homes with suspended timber floors.

3.58
MBIE did not find any compliance issues with more technically complex repair methods, such as levelling concrete slab floors by grout injection. The Building Act's exemptions from requiring a building consent were generally applied appropriately.

3.59
EQC considers that the value of MBIE's findings is limited because no control group was established, the sample size is small and not statistically valid, and customers agreed to participate, which introduces bias.

3.60
However, EQC is intending to recheck the repair files of 3600 homes (as at 28 August 2015) as a result of MBIE's findings.

Quality controls and the risk of substandard quality work

3.61
There has been a lot of publicity about the role of contractors and licensed building practitioners in the issues discussed in the MBIE report. For some of the repairs covered by MBIE's report, it is clear that some contractors' controls on workmanship, some of EQR's controls on contractor performance and work supervision, and/or some of EQC's quality assurance controls did not prevent the quality issues from arising. In this sense, there have been some problems with the controls over some of these repairs. We describe some of the controls in paragraphs 3.66-3.68.

3.62
Inadequate quality control was one of the risks of a home-repair programme that EQC was aware of before 2010.

3.63
EQC identified that substandard repair work was one of the programme's main risks. EQC management has acknowledged that there will be potential quality risks with the repairs carried out early in the programme. EQC knew in October 2013 that the programme's controls for ensuring compliance with the Building Code were not being consistently followed.

3.64
EQC has strengthened its repair quality controls. Figure 6 shows the cumulative number of completed repairs and when some of the quality controls were introduced.

Figure 6
Cumulative number of completed repairs and timing of the introduction of selected quality controls, November 2010 to April 2015

Source: Graph based on information provided by EQC.

3.65
Several controls were in place when we did our work.

3.66
One of the controls in place from the beginning of the programme was using licensed building practitioners for some repair work. Licensed building practitioners are required to meet the standards of competence for licensing, such as ensuring that building work is of an appropriate standard. This control has the potential to ensure that the contractor will complete any additional work and can be subject to a formal complaints process.

3.67
The controls that EQR was responsible for included:

• Contractor induction and management processes. This control was strengthened over time. Criminal, credit, and conflict-of-interest checks became a routine part of the contractor accreditation process from around May 2013. This control has the potential to stop poor-quality contractors from entering the programme and to remove contractors doing poor-quality work from the programme. EQC has recognised that this control may not have been effective in managing the performance of sub-contractors and was not consistently applied to repair hubs until at least late 2013.
• Supervision of contractors by contract supervisors. This control was in place from around the start of the programme. This control enables the potential monitoring of work in progress and inspection of completed repair work. EQC has identified that contract supervisors "had too many files to meet performance expectations".
• Site sign-off process. This control enables the potential inspection of completed repair work. EQC has acknowledged that the sign-off process from engineers was not operating effectively for some sub-floor work managed through the technical repair hub.

3.68
The controls that EQC was responsible for included:

• EQC staff attendance at a sample of joint on-site sign-offs. This control was introduced from August 2014. Members of the EQC quality assurance team now attend a sample of joint on-site sign-offs. That team has been in place since March 2013.
• Introduction of a claim completion guideline. This control was introduced in May 2014. The control requires that repair work is not to be signed off as completed with known defects or additional work required.
• EQC's investigations team. This team primarily carries out investigations related to fraud. From time to time, these investigations discover quality issues. EQC told us that the team has carried out 151 contractor audits since mid-2013.

3.69
As we noted in our 2013 report, when EQC was procuring a provider of project management services, EQC wanted the provider to accept primary responsibility for substandard repair work. ¹⁷ None of the bidders would accept primary liability.

3.70
Although contractors and EQR have responsibilities in the programme, EQC continues to be ultimately responsible for the programme. EQC will seek remediation from third parties, including contractors and EQR, depending on whatever warranties or other legal recourse options are available.

3.71
EQC needs to continue to be alert to risks to the quality of repairs. These risks include non-compliance with the Building Code, repair strategies that are inconsistent with MBIE guidelines, and poor-quality sub-floor repair work. EQC has acknowledged the importance of having systems and processes to effectively manage any future unanticipated additional work.

---

4: Fletcher EQR is a business unit of Fletcher Construction that project manages repairs in the programme on EQC's behalf.

5: EQC Board paper (15 July 2015), Canterbury Home Repair Programme Quality Assurance Overview.

6: Work is considered "practically complete" when houses can be used for their intended purpose without material inconvenience, and the repairs are complete except for minor defects and minor omissions that are still to be completed.

7: KPMG for the State Services Commission (December 2013), Independent Review of the Earthquake Commission's Customer Satisfaction Survey.

8: Business Leaders Health & Safety Forum (April 2015), Benchmarking Report for the period 1 January 2014 to 31 December 2014 – including three year trends for 2012 to 2014, page 14. This report is available at www.zeroharm.org.nz.

9: WorkSafe New Zealand's 21 October 2014 press release, Asbestos investigation completed. Available at the WorkSafe section of www.business.govt.nz.

10: Asbestos exposure in New Zealand: Review of the scientific evidence of non-occupational risks. Available at www.pmcsa.org.nz.

11: These findings were reported in the June/July 2015 edition of Build, pages 84-85.

12: The results of the site inspections were reported in the June/July 2015 edition of Build, pages 84-85.

13: Research New Zealand (May 2010), Residential Consumers' Experiences of Commissioning Building Work. A Survey of Homeowners who Obtained Building Consents in 2005. This survey covered 752 building projects exceeding $50,000 in value carried out between 2005 and 2010 for which building consent was obtained in 2005.

14: EQC told us that the state of the housing stock, including the level of deferred maintenance, has an effect on deciding which repair strategies are appropriate.

15: MBIE's report is focused on new building work that is exempt from requiring a building consent. MBIE considered whether this exempted work has been finished to a standard compliant with the Building Code. The sample of 101 repairs was selected from information provided by EQC, Housing New Zealand Corporation, other insurance companies, and repairs completed by people after opting out of the programme.

16: This is a method used to make floors level. Jacks are used to raise a house to a level. Where this reveals gaps between piles and bearers, material is placed on top of the piles in those gaps – referred to as packing – to support the levelled house. The jacks are then removed. There are expectations about when it is appropriate to use this method, the nature of the material used for packing, and how that material is to be secured.

17: Paragraphs 3.35-3.37.