Part 3: Overseeing the roll-out

3.1
In this Part, we discuss how effectively Crown Fibre has overseen the commercial partners' performance in terms of time, cost, and quality.

Summary of our findings

3.2
Crown Fibre has managed the performance of the commercial partners effectively. Crown Fibre has good oversight of their performance against time and cost expectations. It also has a testing programme to assure the quality of the commercial partners' work. Crown Fibre also has oversight over roll-out performance through its membership on the partnerships' governance bodies for this work.

3.3
Overall, Crown Fibre:

• can influence performance by using contractual mechanisms when performance is lacking;
• uses its insight into roll-out performance effectively to identify and manage risks that threaten the success of the roll-out;
• makes sure that roll-out information gets to the right places; and
• regularly reviews its own effectiveness and prioritises resources appropriately to ensure that it remains useful to the roll-out.

Crown Fibre monitors commercial partners' performance effectively and keeps the roll-out on schedule

3.4
Assurance about performance to time, cost, and quality is important. It measures how well parties are meeting their contracted obligations.

3.5
We expected Crown Fibre to effectively monitor its commercial partners' performance relative to expectations of time, cost, and quality. We also looked at whether Crown Fibre had adequate assurance that the network was being built to the right quality.

3.6
We expected to see roll-out performance information getting to the right places within Crown Fibre and to others that need it.

3.7
We also expected performance levers to be available and used where necessary to motivate and influence commercial partners to fix performance issues.

3.8
Crown Fibre has different ways of understanding and assessing the commercial partners' performance regarding time, cost, and quality. Crown Fibre used various performance levers and influences appropriately to get performance back on schedule.

Crown Fibre has effective oversight of its commercial partners' performance

3.9
Crown Fibre has good oversight, at both a managerial and a governance level, of the commercial partners' performance in building the network to time and cost expectations. Crown Fibre's management team works directly with the commercial partners where performance is unsatisfactory or to help the commercial partners face challenges.

3.10
Crown Fibre oversees actual roll-out progress by collecting standardised information from the commercial partners on a monthly basis. Regular communication with the commercial partners allows this information and any issues that have arisen to be discussed. This reporting arrangement allows Crown Fibre to test and compare information and assess performance.

3.11
Crown Fibre governance can look beyond immediate challenges faced by the commercial partners to understand their effect on the whole roll-out. Crown Fibre can then set a strategic or tactical direction to deal with specific issues at an appropriate level.

3.12
Crown Fibre receives good-quality information, so it has oversight and can exert influence. Crown Fibre monitors time and cost performance through reporting from the commercial partners. This reporting uses an agreed format and clear definitions. Crown Fibre uses its own information to validate its commercial partners' reporting. Crown Fibre also checks the commercial partners' reporting for early warning signs of issues, at either commercial partner or roll-out level.

Crown Fibre runs a User Acceptance Testing programme

3.13
Crown Fibre designed and runs a User Acceptance Testing programme to assure itself of network quality. User Acceptance Testing includes specific processes for Crown Fibre's inspection and testing of the network parts as the commercial partners complete them. The commercial partners confirm the readiness of parts by "handing them over" to Crown Fibre for User Acceptance Testing. The commercial partners also give pertinent fibre layout documents and other information to Crown Fibre for it to inspect. Confirming the built parts ready to be tested through User Acceptance Testing is called "handover".

3.14
We consider that the User Acceptance Testing programme and related processes cover the essential elements we expected. User Acceptance Testing is designed to assess work against appropriate standards. User Acceptance Testing complements, but is independent of, the commercial partners' own Quality Assurance programme and processes. Errors or faults are identified and recorded, and corrective action is evidenced and scrutinised.

3.15
The processes we saw on paper were demonstrated to us when we attended a User Acceptance Testing inspection. We observed the professional interaction between the User Acceptance Testing tester and the commercial partner's Quality Assurance representative and the way testing is completed, agreed, and recorded. This interaction complemented our review of the documentation and reinforced our view that User Acceptance Testing covers the right aspects to provide assurance of quality.

3.16
User Acceptance Testing involves physically inspecting fibre that goes to hub points (traditionally called "exchanges"), and fibre from hub points to the roadside junction boxes (called "fibre access points") where street fibre can be connected to premises. The process also involves electronically testing the performance of the fibre. Figure 4 shows a typical underground fibre access point.

Figure 4
Example of a typical fibre access point, Lower Hutt

3.17
Crown Fibre runs a "three sets of eyes" assurance model:

• inspection by the commercial partner's site foreman as the building work is done;
• the commercial partner's own quality assurance process, when the building work is completed; and
• Crown Fibre's User Acceptance Testing, when the completed part is handed over.

3.18
User Acceptance Testing includes following up on faults. Faults might be detected by a commercial partner's quality assurance checks or by Crown Fibre's User Acceptance Testing. User Acceptance Testing is not signed off until the commercial partners show Crown Fibre that faults are fixed.

3.19
Crown Fibre employs five contractors to run User Acceptance Testing throughout New Zealand. They inspect and test according to set User Acceptance Testing procedures. User Acceptance Testing is independent of commercial partners' quality assurance, but it does look at the method and results of each quality assurance programme, with the option of cross-referencing results.

3.20
User Acceptance Testing takes place on site with the commercial partners' quality assurance people present. The two parties affirm and agree User Acceptance Testing results. This helps prevent later differences about testing and results, and ensures that faults can be fixed promptly.

3.21
We observed a User Acceptance Testing inspection. The people involved in the User Acceptance Testing who we spoke to showed their knowledge and experience, and passion for the network.

3.22
The size of the User Acceptance Testing sample varies. User Acceptance Testing inspects and tests a percentage of the handover, with the discretion to increase or decrease it, depending on past results, risks, or issues. Which handover parts get tested can be randomly chosen or selected deliberately. We consider this flexibility important. Commercial partners get variable periods of notice about which parts of handovers are selected. This brings an element of surprise to testing.

3.23
Timely User Acceptance Testing can be difficult to achieve. The rate of User Acceptance Testing handovers has not been as consistent as Crown Fibre expected. Under annual targets, a commercial partner can opt to complete the same build step throughout the parts of the network it is building before proceeding to the next step. This has resulted in network parts being handed over less regularly, when done in bulk. This has usually occurred approaching target deadlines and has resulted in backlogs of User Acceptance Testing.

3.24
People at Crown Fibre told us that bulk handover caused an uneven flow of User Acceptance Testing work. User Acceptance Testing people sometimes did not have enough work to use their full capacity. At other times, heavy workloads would put them under significant strain. In our view, smaller, more regular targets might have encouraged steadier handover rates. We encourage Crown Fibre and its commercial partners to keep trying to make User Acceptance Testing handovers more regular.

Representation on the commercial partners' governance bodies gives Crown Fibre oversight

3.25
Crown Fibre's representation on local fibre company boards and the Chorus Project Steering Committee allows it to directly oversee the commercial partners' performance. Crown Fibre people on these boards told us that their position gives them a better picture of the roll-out's performance than reporting and testing alone. This representation also helps Crown Fibre work directly with commercial partners to address performance issues. It gives Crown Fibre insight into the risks and issues faced by the commercial partners and how the commercial partners are addressing them.

3.26
The skills needed to govern performance and to manage performance are different. Some commercial partners expressed concern that discussions at the partnership governance level sometimes descended into detail and hampered strategic discussion. They thought this was because of the partnership governance structure. Crown Fibre's management representatives were being asked to operate at a governance level, and some commercial partners felt that it was "difficult for [Crown Fibre management] to decide which hat they were wearing".

Crown Fibre management gives its board good information

3.27
The board of Crown Fibre receives good quality and timely reporting about the roll-out. We reviewed board reporting with various levels of detail. The reports we looked at gave the board clear and relevant information and commentary. Board members we spoke to agreed. They thought that the quality of information helped them do their job.

Crown Fibre keeps Ministers and central agencies well informed

3.28
The Minister for Communications and the Minister for State-Owned Enterprises get monthly reports from Crown Fibre. Crown Fibre also provides regular information to its monitoring agency (the Treasury) and to the Ministry of Business, Innovation and Employment.

3.29
The Ministry of Business, Innovation and Employment told us that it used independent data analysis to get assurance that the reporting it got from Crown Fibre (some supplied by the commercial partners) was accurate. Initially, some work was needed to clarify and align definitions. The Ministry considers the information to be reliable and accurate.

3.30
Reporting also includes information on the performance of connections, including those to priority users and the public. This information helps the Ministers to understand the experiences of those who opt to connect. The Government does not fund residential connections, but commercial partners report to Crown Fibre on the timeliness of such connections. This performance is measured against Service Level Agreements that commercial partners have with internet service providers (see Part 5).

Commercial partners' performance was managed through performance levers

3.31
Crown Fibre has used performance levers thoughtfully when the commercial partners' performance has fallen short of expectations. In our view, this approach has been largely effective in prompting the commercial partners to address performance issues.

3.32
If one party raises the possibility of applying performance levers, this will always draw out differences of perspective between the parties. Performance levers would rarely be mentioned unless one party questioned another's performance. In our view, when performance is not up to expectations, the possibility of applying performance levers is always implicit. Crown Fibre and the commercial partners both understood this.

3.33
Some commercial partners did not meet targets early in the roll-out. Crown Fibre discussed performance levers with them strategically, to get the commercial partners in agreement and committed to fixing the issues that caused the shortfall.

3.34
Some of the commercial partners felt that Crown Fibre was too quick to raise the prospect of using performance levers. One commercial partner told us it felt that Crown Fibre was pointing to the contract "from day one" in any discussion of targets. Another commercial partner felt that Crown Fibre made little, if any, effort to understand why problems meeting targets were arising or what pressures on the commercial partner might be causing them.

3.35
In our view, a strong commitment to keeping to contracted terms is appropriate. These discussions happened when agreed targets were not being met. In some instances, performance levers were applicable but not enforced – for example, when one commercial partner was not meeting its own targets but the roll-out was on target because other commercial partners were over-performing. Crown Fibre did not enforce payment of the penalties that were applicable, encouraging the commercial partner to concentrate on fixing the causes of the non-performance. However, Crown Fibre kept the option of later enforcement open. Crown Fibre showed it meant business, but it was flexible enough to find less punitive ways to encourage the commercial partner to address the performance issues.

Liquidated damages are redirected to enhance the network

3.36
In our view, Crown Fibre's application of liquidated damages is creative and appropriate. Liquidated damages for commercial partners materially breaching contracted requirements are a financial penalty payable to Crown Fibre. Throughout the whole roll-out to date, liquidated damages have been applied on only two occasions. These were applied to two commercial partners on different occasions where the existence of, and potential for enforcing, performance levers alone did not influence the desired turnarounds.

3.37
When Crown Fibre enforced penalties in these two instances, rather than retaining the payments, it directed that the payments be reinvested in parts of the network that were additional to what those commercial partners had been contracted to build. This resulted in enhancements to the network.

3.38
This approach meant that the two commercial partners would build and eventually own a better network than the contract initially required. However, the commercial partners did not necessarily see it this way. One told us that, irrespective of the outcome, "It still cost us financially." In our view, Crown Fibre's approach resulted in enhancements to the network sooner than might otherwise have occurred.

Crown Fibre manages risks well

3.39
Crown Fibre can proactively manage risks to the roll-out if it understands the internal and external threats to the roll-out's success. We expected to see a risk-management framework setting out how Crown Fibre should use its oversight to identify and manage risks to the roll-out. We also expected Crown Fibre to follow this framework when managing risk.

3.40
Crown Fibre has a risk-management framework. Crown Fibre follows this framework and uses it effectively to identify threats to the roll-out. Crown Fibre works proactively, including working with the commercial partners, to reduce the likelihood or effect of these risks.

Crown Fibre uses an effective risk-management framework

3.41
In our view, Crown Fibre has an effective risk-management framework. We reviewed documents that showed us that Crown Fibre followed the framework, identified risks, and dealt with them effectively. We saw how:

• Crown Fibre regularly reviews, measures, and prioritises known risks;
• Crown Fibre assigns ownership of risks and management responsibilities;
• specific risks are managed day-to-day;
• the right people are made aware of risks and how they are managed; and
• new or emerging risks entered the risk-management process.

3.42
In our view, Crown Fibre manages and oversees risk effectively, whether it is specific to a commercial partner or more systemic and far-reaching. Crown Fibre's people apply thought and insight to understanding and managing risks, taking account of environments and context.

3.43
Crown Fibre is serious about risks to health and safety. Crown Fibre people told us that they see health and safety as a critical risk to the roll-out. Health and safety risks, controls, and incident management are a standing item on Crown Fibre's board meeting agendas.

3.44
The roll-out is largely a civil works project operating in busy and dangerous urban traffic. We saw this in Auckland, where traffic was moving close to the roll-out worksite. We saw a site foreman taking ownership of visitors' health and safety. He checked that we had registered ourselves and that we understood the rules we had to follow.

3.45
We also saw accountability for health and safety in action when we attended a User Acceptance Testing inspection. We were told clearly what was expected of us as visitors, what to watch for, and what to do if an incident occurred. We were encouraged to ask questions.

3.46
We were told that it took some time for everyone on site to understand their health and safety responsibilities. Crown Fibre listened to feedback and put in place health and safety procedures that work well.

Crown Fibre reviews its strategies regularly

3.47
Crown Fibre's role is likely to change as the roll-out progresses and priorities change. We expected Crown Fibre's role to be reviewed regularly for relevance and currency. We expected this review to cover its past performance and strategy. We also expected the review to assess how Crown Fibre could best use its resources to play its strategic role most effectively.

3.48
Crown Fibre measures its own performance against strategic goals and reviews these goals regularly to consider its continued relevance. It considers its future role as the roll-out progresses and how to succeed in it.

Crown Fibre reviews its performance against strategic goals and its goals for relevance

3.49
Crown Fibre regularly reviews its performance against strategic targets and reports on its own performance targets. Crown Fibre also considers its future relevance and strategic direction. It reviews its main strategic focus areas for continued relevance. These areas include commercial partner-specific strategies and wider aspects of the roll-out, such as contract management.

3.50
Senior management and staff participate in workshops to produce each review. The review is reported to Crown Fibre's board for members' input, direction, and endorsement.

3.51
In our view, this regular review of strategy is effective and an important way of making sure that Crown Fibre contributes the best it can to the roll-out.

Crown Fibre regularly reviews its strategic direction, capability, and capacity

3.52
In our view, Crown Fibre has the right capabilities and capacity to continue performing its useful role in the roll-out effectively. This applies at both management and governance levels.

3.53
Crown Fibre decided early to employ staff who could deal with commercial partners on an equal footing. Crown Fibre also decided to pay them similarly to the private sector.

3.54
Crown Fibre also considers how to position itself to fulfil its role in the future. Crown Fibre assesses the need for changes to business structures, numbers of full-time equivalent staff, and staff capability as its role changes. Crown Fibre's senior managers told us that a measure of Crown Fibre's success would be its eventual need to scale back to a purely monitoring role.

3.55
Crown Fibre's role has not been scaled back yet. Crown Fibre has evolved from focusing on selecting and contracting commercial partners to focusing on the roll-out. However, managing the performance of the commercial partners has not been easy. Crown Fibre has remained more closely involved for longer than it had expected. Crown Fibre told us that it still needs to manage and influence the commercial partners' performance to ensure that the roll-out stays on schedule.