Part 5: Monitoring performance and improving the cost-effectiveness of maintenance work

5.1
Our second report concluded that NZTA regularly monitored the performance of its contractors.

5.2
However, NZTA needed to be more consistent in carrying out its performance monitoring and reviews. It also needed to respond better to contractor performance issues. NZTA did not systematically or consistently assess information at a national level or benchmark contractor performance.

New performance monitoring framework

5.3
A new performance monitoring framework exists under the Network Outcomes Contracts. NZTA considers that this will strengthen monitoring, allow it to apply a more consistent approach to all contracts, and allow it to benchmark performance. Figure 6 shows the framework, which is made up of:

• operational performance measures;
• key result areas and key performance indicators;
• contract service outcomes;
• performance-based at-risk payments; and
• tenure rewards.

5.4
Operational performance measures reflect NZTA’s expectation of the contractor’s service, performance, management, and capability. There are about 140 operational performance measures, which cover construction quality, asset condition, night-time condition, and management activities. The primary supplier measures operational performance measures through a monthly self-compliance audit process, which is overseen by NZTA’s Maintenance Contract Manager.

5.5
Key result areas are the outcomes sought at the strategic level. They help NZTA to achieve the Government’s strategic objectives, NZTA’s strategic priorities, and compliance with the Land Transport Act 1998 and the Resource Management Act 1991. Key performance indicators specify the key result areas in more detail.

5.6
The key result areas cover:

• safety;
• customer satisfaction;
• sustainability;
• assurance and value;
• network performance; and
• the health of the relationship between NZTA and the primary supplier.

Figure 6
Performance monitoring framework under Network Outcomes Contracts

Source: Redrawn from a figure supplied by NZTA.

5.7
Contract incentives and rewards underpin the performance management system. These include:

• An at-risk component of 10% of the primary supplier’s tendered base lump sum amount is assessed monthly. Any amount withheld is deducted from the primary supplier’s monthly lump sum payment.
• A “key result area reward” payment (up to $200,000 each year) is assessed annually. It is based on an overall assessment of a primary supplier’s performance against the key result areas.
• Contract terms can be extended or reduced depending on performance.

5.8
We consider that the revised performance framework and contract incentives are an important development. They have the potential to improve the provision of maintenance and renewals works by:

• shifting the focus of the contract from what services the primary supplier must provide to the outcomes they must achieve;
• enhancing NZTA’s ability to focus on the outcomes that matter, such as customer service, safety, quality, network availability, and reliability;
• delivering better performance by consistently applying the at-risk, key result area reward, and contract extension incentives;
• making it easier for NZTA and the primary supplier to measure, discuss, and improve performance; and
• allowing NZTA to benchmark performance for all contracts and use that information to manage the network better.

Monitoring primary supplier performance

5.9
NZTA now adopts a three-tier approach to monitoring performance.

5.10
NZTA’s Maintenance Contract Manager and Contract Management Team hold monthly performance meetings with the primary supplier to review a monthly performance report created by the primary supplier. There is one Maintenance Contract Manager for each Network Outcomes Contract.

5.11
Each Network Outcomes Contract has a Contract Management Board, which is made up of two representatives from NZTA and two representatives from the primary supplier. These Boards review contract progress, review the annual key result area achievement result, and recommend the appropriate tenure implications and reward achievements to NZTA’s Value Assurance Committee. The Boards also have a role in resolving conflicts between NZTA and the supplier.

5.12
The Value Assurance Committee has delegations for state highway matters. It consolidates and considers performance results annually and determines the appropriate key result area reward payment and any tenure implications.

5.13
Figure 7 shows the main elements of the performance monitoring framework.

Figure 7
Main elements of the New Zealand Transport Agency’s performance monitoring framework

Source: Redrawn from a table supplied by NZTA.

5.14
NZTA designed the performance framework so that each role, at each level, could have maximum oversight of the areas they are accountable for. However, NZTA will need to ensure that this delegated performance monitoring happens in practice. It will need assurance that accurate and consistent reporting to contractors is being done, and that contractors are carrying out any required performance improvements.

5.15
NZTA has systems for responding to any problems with primary supplier performance. As well as the mechanisms set out in paragraphs 5.9-5.14 and Figure 7, Network Outcomes Contracts provide for arranging informal meetings to discuss any emerging issues, including performance issues.

5.16
Network Outcomes Contracts adopt a strong collaborative and partnering framework for the relationship between NZTA and the primary supplier. For example, the contracts establish a series of “Key Elements” underpinning the relationship. These include “Honesty in all dealings”, “An environment where each party communicates freely in an open manner on all issues”, “An environment of mutual trust to be developed”, and “All issues to be considered with fairness to the parties involved”.

Benchmarking

5.17
NZTA is placing increased emphasis on benchmarking. A Performance Management Team is responsible for monitoring, benchmarking, and reporting on the performance of Network Outcomes Contracts.

5.18
This team analyses the factors influencing the cost of maintenance and renewals work, and the balance between the costs of preventative and reactive maintenance. It also compares, contrasts, and analyses the performance of primary suppliers and benchmark performance.

5.19
NZTA is preparing a framework to estimate and measure the value derived from Network Outcomes Contracts compared to the previous contract regime.

5.20
NZTA is also preparing a Performance Framework Tool to provide a nationally consistent measurement system for supplier performance. This tool will:

• record audit results and performance information from primary suppliers;
• identify and report non-compliances and poor performance to the Maintenance Contract Managers;
• complete accurate and consistent calculations of financial penalties, incentives, and tenure extensions; and
• allow for analysis and reporting of consolidated performance results for all 23 contracts.¹²

5.21
In summary, NZTA has, or is putting in place, a variety of tools to monitor and benchmark primary supplier performance. The application of these tools, combined with operational performance measures and key result areas, should enable NZTA to compare and contrast the relative performance of each of its maintenance and operations suppliers and identify opportunities for improvement, including getting better value for money and better performance outcomes.

12: NZTA expects to deploy the system in 2014. The system will be completed in January 2015, ready for “go live” in February 2015.

page top