Part 4: Contracts are not yet fully collaborative

4.1
In this Part, we look at the collaborative intent of the Network Outcomes Contract model. We discuss:

• how the formal contract groups are beginning to work together effectively;
• some processes that undermine the model’s collaborative intent; and
• that the Agency does not measure and report on its own performance.

4.2
The Network Outcomes Contract model emphasises a strong collaborative and partnering relationship between the Agency and suppliers to achieve the contract outcomes.

4.3
Figure 14 shows the 10 crucial elements, as described by the contract, which underpin this relationship.

Figure 14
Elements of a collaborative relationship

Source: New Zealand Transport Agency.

4.4
These behavioural elements are expected to be applied to processes that support the contracts. Agency staff and contract teams need to work together to give advice and deliver maintenance work. The parties depend on each other. For example, the Agency approves renewals, within the limits agreed at tender, which influences the amount of routine maintenance work the contract team needs to do. The contract team decides on the routine maintenance work, in line with its maintenance management plan, which can influence how often renewals are needed.

4.5
On average, the contract teams that receive “outstanding” on their overall performance assessment tend to score highly on the health of the relationship surveys. These surveys collect responses from suppliers, subcontractors, and members of the Contract Management Teams and Contract Boards (including Agency staff).

4.6
We expected the Agency to manage the contracts effectively, by maintaining effective relationships with suppliers. This would contribute to maintaining state highways effectively and efficiently.

Summary of findings

4.7
The Contract Management Teams and the Contract Boards have become more effective over time. They provide the formal channels for managing the relationship between the Agency and suppliers.

4.8
Relationships between the Agency’s regional staff and the contract teams were mostly effective. However, the Agency needs to continue to focus on ensuring that it has the right capability and capacity to support effective relationships.

4.9
Some processes have undermined the collaborative intent of the contracts. We recommend that the Agency review the roles and delegations of the parties involved in the contracts so that they are clearly understood and support collaborative processes between the Agency and suppliers.

4.10
Other factors that can influence the relationship include whether the supplier has adopted a stewardship approach to the network (see paragraph 4.34), whether they follow the required plans, and the pricing for the contracts (see paragraph 2.69 to 2.71).

4.11
To support changes in behaviour that match the collaborative intent of the model, the Agency was to monitor and assess its performance and report this to the Contract Boards. This has not happened. We recommend that the Agency measure and report its performance in supporting the contract teams so it can identify opportunities to improve and be held to account for its performance.

Formal contract groups are beginning to work effectively

4.12
The formal contract groups have become more effective over time, leading to better leadership and guidance of the contract teams. Two groups are responsible for managing and governing the contracts:

• the Contract Management Teams, which are responsible for the day-to-day management and leadership of the contract teams; and
• the Contract Boards, which are responsible for providing joint governance of the contracts.

4.13
The Contract Management Teams meet monthly to review each contract team’s performance, risks, progress against delivering the work programme, and the results of audits. We received feedback that the Contract Management Teams are reasonably effective. They are improving their ability to remain focused on leading and guiding the contract rather than get too much into the operational detail.

4.14
The Contract Boards meet every four months to review reports by the relevant Contract Management Teams. The Contract Boards consider and review the performance of the contract teams, risks, the relationship between Agency and supplier staff, and any other issues.

4.15
Generally, people considered that the Agency merging some Contract Boards increased the value of the Contract Boards. This enabled them to focus on themes and promote sharing of experiences and perspectives between the contract teams, rather than get into the operational detail of each contract.

4.16
The increased effectiveness of the formal contract groups means that they can provide better leadership and guidance to the contract teams. However, there are opportunities for them to improve.

4.17
We expect the Agency to consider the roles and responsibilities of these groups as part of its response to Recommendation 3 (see paragraph 4.35). This would help the Agency to identify any opportunities to improve the groups’ effectiveness so that they better support the collaborative intent of the contracts.

Need to manage the risk to capability and capacity

4.18
Outside of the formal contract groups, Agency and supplier staff maintain regular contact to resolve any issues or to progress any needed work. In practice, the Agency and suppliers rely heavily on the people employed as Maintenance Contract Managers and Contract Managers as the primary points of contact for each organisation to maintain effective relationships.

4.19
In the contracts we looked at, relationships between Agency regional staff and suppliers were mostly effective. Behaviours such as openness, trust, and understanding each other’s motivations support an effective relationship.

4.20
However, we are aware of instances where this relationship has not been effective. For example, one of the parties felt that the other was not acting in an honest and fair way. In this instance, the Agency and supplier intervened to resolve the issue.

4.21
To date, the Agency has done limited succession planning to ensure that it has people with the right skills and who can demonstrate the relationship elements described in Figure 14 in the main contract management roles. This is a risk to the Agency’s ability to maintain effective relationships and retain institutional knowledge.

Some processes undermine the model’s collaborative intent

4.22
Suppliers and Agency staff told us that some processes constrain the ability for them to collaborate. In our view, this has also contributed to a perception from suppliers that they did not see the collaborative intent of the contract reflected in decisions that are ultimately made outside of the core contract team.

4.23
There is an inherent tension between the Agency wanting ownership of the state highways and national consistency, and it working in partnership with suppliers to do what is best for the state highway network. Over time, the Agency has asserted more control over the contracts and its regional staff, at the cost of collaboration with suppliers.

4.24
We saw this in how each contract team’s work programme is decided and the process for assessing the contract team’s performance. As discussed in paragraphs 2.45 to 2.67, the Agency has benefited from changes to these processes. These benefits include getting greater understanding of the need for proposed renewals, a more nationally consistent process to identify and approve those renewals, and being able to compare and contrast the contract teams’ performance. However, some aspects of these changes have also undermined the collaborative intent of the Network Outcomes Contract model.

4.25
The Agency has centralised funding decisions, which has led to reduced delegation of decision-making to the Agency’s regional teams (see paragraphs 2.45 to 2.55). We acknowledge that delays in getting funding approvals can be a result of the contract teams not providing relevant data or not properly identifying renewals. However, the reduced delegation of decision-making authority to regional teams has also contributed to delays in getting funding approvals.

4.26
When these delays occur, they can significantly affect:

• collaboration between the Agency and suppliers;
• the contract teams securing the necessary resources to deliver the work; and
• the ability of contract teams to identify and realise potential efficiencies, such as combining work with the Agency’s other programmes.

4.27
The Agency is making changes to its annual planning process, including changing to a three-year plan to allow more flexibility for contract teams and the Agency’s regional staff to move work within funding brackets. This might help reduce some of the issues we have identified.

The performance assessment process can cause frustration

4.28
The Agency needs to assess performance consistently so that all contract teams are treated fairly. Equally, because the performance assessment had relied on contract teams auditing themselves and reporting their performance, they are obliged to provide the Agency with reliable and relevant information.

4.29
In 2015/16, the Agency introduced a moderation process to help it apply self-assessment scores consistently to all contracts (see paragraph 2.63). However, we identified several issues with the Agency’s moderation process that have led to frustration, risking undermining the collaborative intent of the contracts. These issues include:

• a perception that Contract Boards are not adequately involved when changes are made to the performance scores, despite its governance role over the contracts – in particular, where a Contract Board has accepted a contract team’s mitigating circumstances that the Planning and Performance Team subsequently does not accept;
• accepting performance that does not appear to meet the intent of the performance indicators – for example, the Agency allowed one contract to include spending on accommodation providers and local tearooms as part of meeting the key performance indicator to award a minimum amount of work to subcontractors;¹¹
• providing limited feedback to contract teams and a lack of transparency of changes to individual scores for each reporting period; and
• a lack of timeliness in receiving the moderated results means contract teams have little opportunity to make changes before the next reporting period.

4.30
Suppliers and Agency regional staff told us that they received minimal feedback from the Planning and Performance Team about why their contract team’s performance scores had changed or how they could improve their performance results.

4.31
Contract teams also told us that they spend significant time and effort in collating and reporting on the current performance measures and indicators. Some suppliers and Agency staff questioned whether the effort in reporting against the performance measures and indicators is worthwhile.

4.32
As part of the latest round of contracts, the Agency has decreased the number of performance measures and indicators, which the Agency expects will reduce the time and effort the contract teams need to collate and report on their performance. For example, changes to the key performance indicators mean that the Agency can measure them, rather than relying on the contract teams for the relevant information (see paragraph 5.23).

4.33
These issues with the performance assessment process have led to tension and frustration. For example, some suppliers and Agency regional staff said they felt that the Planning and Performance Team was looking for reasons to reduce the contract team’s performance score and not taking local circumstances into consideration.

4.34
Most of the issues we identified with the processes come from the Agency asserting more control over the contracts than it originally intended, potentially undermining the collaborative intent of the contracts. The Agency told us that it initially expected a high level of network stewardship from suppliers. This was reflected in the commitments made by suppliers in their tenders. However, this expectation has not been met in the delivery of some contracts, leading to the Agency asserting more control to ensure that its expectations are met.

4.35
In our view, now that the Agency and suppliers better understand what the contracts involve, the Agency needs to reconsider the roles and delegations of people and groups involved in the contracts so it gets the right balance between national consistency and working in partnership with suppliers. This will also help the contract teams reduce delays and achieve better programming of work.

4.36
The Agency agreed that there needs to be more empowerment but that this needs to be balanced with national consistency. The Agency said that forming the Maintenance Contracts Governance Group enables decisions by the Contract Boards to be more efficiently elevated to the right decision-maker.

4.37
The Agency told us that it has also established, through the Industry Liaison Management-Maintenance, a cross-industry working group to improve the way that suppliers and the Agency collaborate. To date, issues planned to be addressed include:

• providing training to Agency and supplier staff to give people the skills to collaborate;
• changing the way Contract Boards operate; and
• on an ongoing basis, measuring collaboration as part of assessing the relationship between Agency and supplier staff.

The Agency has not measured and reported its own performance under the model

4.38
The performance measures and indicators for the contracts focus on the supplier’s performance, through its contract teams. However, there are no performance measures and indicators focused on the Agency’s performance. Having such measures would allow the Agency to identify opportunities to improve its own performance. It would also provide transparency to suppliers and help hold the Agency to account.

4.39
Under the first round of contracts, to assess the performance of suppliers and the Agency equally, the Agency’s performance was to be assessed and reported to the Contract Boards. This would support changes in behaviour that match the collaborative intent of the Network Outcomes Contract model.

4.40
To date, this has not happened. Some suppliers told us that they feel that the current performance measures and indicators do not recognise the obligations of the Agency to support the contracts. We also heard that the influence of the Agency’s actions on the contract’s performance is not recognised.

4.41
In 2018, two reviews highlighted that the performance measures and indicators did not focus on the Agency. One review stated that the next round of contracts represented an opportunity to make a step change in the maturity of the relationship and the model by measuring and reporting the performance of
the Agency.

4.42
We agree that the conclusion of the current round of contracts provides the Agency with an opportunity to prepare an approach to measuring and reporting its own performance under the model. Implementing this in the second round of contracts would help the Agency to learn from its performance and make improvements or adjustments.

4.43
The Agency told us that it intended to use the collaboration measure in paragraph 4.37 to establish how effectively it currently supports the contracts. It then intends to prepare a framework to monitor its performance, with the Maintenance Contracts Governance Group reviewing the Agency’s performance annually.

11: The intent of the key performance indicator was to support a healthy road maintenance subcontractor market.