Part 5: Lessons are yet to be fully learned and applied

5.1
In this Part, we look at how the Agency identifies, shares, and applies lessons. We also discuss the latest round of contracts.

5.2
Effectively identifying and applying lessons learned from the Network Outcomes Contract model can help the Agency to continuously improve how it operates and the results it achieves.

5.3
We expected to see evidence that the Agency identifies, shares, and applies lessons from managing the contracts.

Summary of findings

5.4
The Agency has learned some lessons and applied some, including to the latest round of contracts. However, this has been limited because the Agency has not assessed the benefits from introducing the Network Outcomes Contract model or measured its own performance.

Some lessons have been learned and applied

5.5
The Agency has learned and applied some lessons to improve the effectiveness and efficiency of maintaining state highways. It is also making changes that could help it more systematically apply lessons.

5.6
The Agency uses reviews, such as the Contract Management Reviews and benchmarking of the performance results, to learn lessons. Some of the lessons that have been identified include:

• ensuring that suppliers will have access to Agency personnel when they need it;
• improving succession planning for the Agency and suppliers;
• better tracking and recording of a contract team’s non-compliance with contract plans; and
• reviewing the incentives for preventative maintenance.

5.7
The contract documents also encourage suppliers and their contract teams to learn lessons. For example, post-construction design assessments check that roads have been built to an acceptable standard.

5.8
We saw examples of contract teams doing reviews after completing renewals work and after responding to incidents, such as after ex-Cyclone Fehi hit the West Coast in early 2018. Doing these reviews supports continuous improvement in how the contract teams maintain state highways.

5.9
Both the Agency and suppliers with multiple contracts have established forums that allow their staff to learn and share lessons. The Contract Boards also help to share lessons between contracts run by the same supplier.

5.10
The Agency has set up other groups, such as the Industry Liaison Management-Maintenance, to learn and share lessons with suppliers. The Agency is working with the main suppliers in the Industry Liaison Management-Maintenance to establish six cross-industry work streams to identify and implement improvements in:

• safety;
• people;
• sustainability;
• systems;
• innovation; and
• collaboration.

5.11
The Agency can use information from these groups to make any necessary changes, including issuing clarifications on how to interpret the requirements of the contract or guidance.

5.12
The Agency uses the contract teams’ annual performance workshops and Contract Board meetings to provide specific feedback to suppliers and contract teams. For example, the Planning and Performance Team presented on themes from the Contract Management Reviews at the Hawke’s Bay contract team’s 2017/18 annual performance workshop. This feedback from the Agency can help to focus contract teams on what they need to do to improve their performance.

5.13
In response to lessons learned, the Agency has clarified and changed the standard conditions of contracts and changed its processes, such as the annual plan process. The contract teams have also applied lessons learned, including innovations and changes in response to the Contract Management Reviews. For example, the North Canterbury contract team introduced a new tool to track the main tasks in each period and who is responsible for completing them.

5.14
The Agency’s newly created Maintenance Contracts Governance Group is responsible for implementing recommendations and improvement from reviews and audits. The Maintenance Contracts Governance Group is expected to help the Agency apply lessons more systematically.

Room to improve how lessons are learned and shared

5.15
Although the Agency has done a lot of work to apply specific lessons, in our view, it needs to take a more comprehensive view of how the Network Outcomes Contract model is operating. It could also share lessons and innovations more effectively.

5.16
The Agency’s ability to identify lessons has been limited because it does not:

• measure and report on the benefits and outcomes from the Network Outcomes Contract model (see Part 2); and
• measure and report its own performance under the Network Outcomes Contract model (see Part 4).

5.17
We have made recommendations for the Agency to address these limitations. Doing so should provide the Agency with valuable insights into its performance and the improvements it could make.

5.18
We asked Agency staff and contract teams about how well lessons were identified, shared, and applied. Generally, people thought that they had several ways to identify and share lessons and that they found them useful. However, they identified some weaknesses, including:

• a lack of opportunity to meet staff from the Agency and other contract teams and share lessons learned;
• innovations could be shared better between the contract teams; and
• the Agency could share lessons more with suppliers, and the Industry Advisory Group could be more effective in this role.

5.19
In our view, the Agency could usefully investigate further opportunities to enable contract teams and Agency staff to share ideas, problems, and challenges with each other.

5.20
The Agency agreed that the groups available to share lessons could be more effective. It has recently recruited five Principal Network Managers throughout the country. Their role is to recognise and promote best practice among all Agency staff responsible for managing state highways.

Changes made to the latest round of contracts

5.21
In 2018/19, the Agency made changes during the latest round of contracts, including documentation and specifications, reviewing the performance measures and indicators, and updating guidance.

5.22
The Agency has used the results of reviews, such as the contract management review, to prepare for the latest round of contracts. It also ran various workshops with Agency staff, the Industry Advisory Group, and local authorities to get their views on what was and was not working.

5.23
The changes the Agency made to the latest round of contracts include:

• clarifying the responsibilities of key roles and the Contract Boards;
• changing the performance measures and indicators, including operational performance measures, key result areas, and key performance indicators, by reducing their overall number and introducing prerequisites. Changes to the key performance indicators also mean that the Agency can measure them internally, with the intention of reducing the burden on contract teams to provide the relevant information;
• increasing the performance assessment periods for each financial year from three to four and scoring performance only once each year; and
• changing how it calculates increases in contract terms.

5.24
These changes should reduce some of the issues with the first round of contracts. However, the Agency and the industry agree that there are still issues in the latest round of contracts. These include the auditing of the operational performance measures and the amount of administration the contracts require.

5.25
In our view, the Agency needs to monitor these changes closely to ensure that they do not worsen existing issues (such as the timeliness of receiving performance assessment feedback) and affect the collaborative intent of the Network Outcomes Contract model.

5.26
To date, the Agency has completed tenders for Marlborough and Taranaki using the new contract. The Agency told us that it intends to review how the latest contracts are working. This should allow the Agency to consider whether the changes to the contracts are helping to reduce some of the issues with the first round of the contracts, to identify any unintended consequences, and to make any necessary changes.