Part 4: How the agreements were working
4.1
In this Part, we report on how well the agreements were working when we examined them at the end of 2006. For the agreements to be functioning well, we expected to find:
- effective governance of the agreements;
- effective management of risks on entering into the agreements and effective ongoing management of risks;
- effective communication and reporting between Transit and the district councils; and
- effective contractual arrangements underpinning the collaborative agreements.
4.2
We did not conduct a financial audit of all contractual arrangements underpinning the collaborative agreements. We looked at whether the main contractual arrangements were operating effectively by examining whether:
- roles and responsibilities had been clearly defined in contracts and were functioning as intended;
- risks had been clearly allocated through contracts;
- contracts had been awarded for an appropriate length of time;
- there were adequate arrangements for monitoring and reporting contract performance; and
- there had been any major problems with contract variations and payments.
The delegation from Transit to Rotorua District Council
4.3
The delegation to Rotorua District Council was working well, with little ongoing management input from Transit.
Governance
4.4
Governance was working well, although the informality of arrangements and infrequent reviews meant that there was a risk of governance issues not being picked up early.
4.5
Governance of the delegation was mainly through quarterly meetings between the council's district engineer and Transit's area manager. No minutes were produced for these meetings. Transit's regional manager raised issues with the council's district engineer and the council's highway manager as they arose, rather than through any regular formal governance forum. The regional manager was very much on the sidelines of the delegation. Transit told us that bringing the council into the chain between Transit as the ultimate asset manager for state highways and the state highway maintenance contractors lengthened it and that Transit was more distant from the decision-making process. The council told us that governance was being strengthened by the council establishing a stronger relationship with Transit's regional office in Tauranga.
4.6
The formal agreement provides for the delegation to be reviewed every three years. The last review was in 2000 and concluded that there was reasonable confidence that performance expectations had been met and the main requirements of the delegation complied with.
| Recommendation 1 |
|---|
| We recommend that Transit and Rotorua District Council review the operation of the delegation in the next year, as it was last reviewed in 2000. |
| Recommendation 2 |
|---|
| We recommend that Transit and Rotorua District Council consider, as part of the review of the delegation in the next year, how arrangements for governance of the delegation might be further strengthened. |
| Recommendation 3 |
|---|
| We recommend that Transit and Rotorua District Council review the delegation at three-yearly intervals, as specified in the delegation agreement, to ensure that it continues to operate cost-effectively for both parties. |
4.7
Under the delegation agreement, Transit paid the council an administration fee. The fee was to cover the council's state highway management costs (mainly the costs of employing the highway manager). It was made up of a percentage of annual expenditure on the state highways plus a charge for each kilometre of highway. The council told us that, because expenditure on the state highways fluctuated from year to year, there was sometimes a small surplus or deficit to the council at the end of any one year. This balanced out over a number of years. The council reported state highway management costs in its Long-Term Council Community Plan, and the plan for 2006-2016 shows that costs and the administration fee are expected to broadly balance over time.
Risk management
4.8
Transit had not delegated responsibility to the council for planning and access control for state highways to avoid potential conflicts of interest Transit thought might arise. Transit believed that the council could have a conflict in balancing ratepayer interests with protecting the state highway network.
4.9
The council's highway manager told us that there was potential for conflicts of interest to arise for him in public meetings (for example, when planning consents were discussed). He always made it clear that, although he was not a Transit employee, he was giving Transit's view. He also said that the council had managed this risk by separating responsibility for state highways and locals roads between himself and the local roads manager.
4.10
The Transit regional manager told us that a potential risk, particularly if important staff within the council changed, was that the council did not manage the delegation in accordance with Transit's policy and procedures. He said that this risk was managed through contract management reviews by Transit. Under the delegation, the council is also required to notify Transit of any changes in important staff that would affect its ability to exercise the delegation.
Communication and reporting
4.11
Within the council, the highway manager had regular contact with councillors. The councillor that we spoke to valued the highway manager's excellent knowledge of district roading matters and how he used it in managing state highway activities. The highway manager also had regular contact with the council's local roads manager on specific projects.
4.12
The highway manager's main contact with Transit was through the Transit regional office. He had quarterly meetings at the Transit regional office, with other communication driven by issues as they arose. The highway manager told us that he was in frequent contact with staff in the Transit regional office about planning consents, capital projects, and other matters.
4.13
The highway manager also communicated regularly with the public and the New Zealand Police, and liaised with local roading organisations (such as "Drivewise", a road safety trust), who believed that the highway manager kept them well informed about roading issues.
4.14
The highway manager told us that he reported regularly to Transit on state highway contracts, projects, and finances using Transit's standard reporting system, which he had a direct link to. He monitored standard Transit key performance measures for state highway condition. The highway manager and the local roading manager also reported to the council's Works Committee every six weeks.
Contractual arrangements for state highway professional services
4.15
The council administered nine separate or combined state highway maintenance contracts and one state highway professional services contract. We focused on the contractual arrangement with the state highway professional services contractor because professional services support effective management of roading assets and include monitoring the operations of maintenance contractors. We did not examine the maintenance contracts themselves. Contractual arrangements managed by the council for state highway professional services appeared to be functioning well.
Roles and responsibilities
4.16
The state highway professional services contract was a traditional contract with defined roles and responsibilities. However, a representative for the professional services contractor told us that, in practice, a partnership arrangement was operating between the contractor and the council that went beyond the state highways contract. The representative said that there was a good understanding between the contractor and the council based on mutual respect and an open, honest relationship. The representative also said that the contractor and the council jointly worked together and took pride in maintaining the state highways.
Allocation of risks
4.17
The contract was for a set sum. Some known risks were allocated to partners through the contract, but the contractor carried the risk of staying within the set sum.
Length of contract
4.18
The contract was for an initial term of three years and could be extended by two periods of one year. The contractor told us that the term of the contract was not a problem, although a longer contract would provide more certainty and reduce re-tendering costs.
Performance monitoring and reporting
4.19
Adequate performance monitoring and reporting arrangements were in place. The representative for the contractor told us that the council's highway manager evaluated the performance of the contractor every month using Transit's standard performance assessment system. Performance was discussed at weekly liaison meetings between the contractor and the council, and action was taken where needed. The representative also told us that Transit reviewed the contractor every few years for compliance with Transit's policies and procedures.
Variations and payments
4.20
There was a clear process for approving contract variations. Variations to the scope of the state highways professional services contract went to Transit for approval. Once variations had been approved by Transit, they were also presented to the council's Works Committee for approval.
4.21
The contractor told us that payment claims were submitted to the council's highway manager, and the process ran smoothly.
The agreement between Transit and Marlborough District Council
4.22
Transit's agreement with Marlborough District Council was working well.
Governance
4.23
Governance was working well. The manager of Transit's Marlborough Roads office and the council's assets and services manager had formal responsibility for governance.
4.24
A procedural audit of the Marlborough Roads office by Land Transport New Zealand in late 2004 concluded that, overall, Marlborough Roads was well managed, with appropriate governance structures in place.
4.25
The agreement provides for Transit and the council to split the administration costs of running the Marlborough Roads office. Transit covers 40% of the administration costs, and the council pays Transit a management fee to cover the remaining 60%.
4.26
The Marlborough Roads office analyses timesheets annually to ensure that the split of administration costs between Transit and the council remains fair and reasonable. The procedural audit by Land Transport New Zealand in late 2004 reviewed the timesheet analysis for the year ended 30 June 2004 and found that the split of administration costs was still reasonable.
4.27
During the five years of the initial agreement, the management fee paid by the council to Transit remained the same. It was increased when the agreement was renewed from 1 July 2005 to cover increased staffing, administrative, and rental costs since 2000.
Risk management
4.28
When setting up the agreement, Transit and the council jointly identified and managed risks. There was no formal risk management plan for the ongoing operation of the agreement, although the agreement itself provides a level of protection for both parties, including conditions for terminating the agreement. The people from Transit and the council that we spoke to were also alert to potential ongoing risks, and the risks were being managed.
4.29
A loss of council support through changing membership was recognised as a risk. There was strong ongoing commitment to the agreement from the council staff and councillors that we spoke to.
4.30
Potential conflicts of interest were recognised as a risk. As an ex-council employee, the manager of Marlborough Roads was very aware of the need to retain independence to avoid potential conflicts of interest where Transit and the council had different views on issues.
4.31
Staff turnover that led to a loss of local expertise within Marlborough Roads was also recognised as a risk. The council had already identified a potential replacement for an important staff member of Marlborough Roads who might retire in the near future and was taking steps to equip the potential replacement with the necessary skills and experience for the Marlborough Roads role. The council recognised that Transit would be responsible for appointing any replacement and was not bound to consult the council.
| Recommendation 4 |
|---|
| We recommend that Transit (Marlborough Roads) and Marlborough District Council prepare a succession plan for the Marlborough Roads office that covers how the capability of the organisation will be maintained in the future as staff change. |
Communication and reporting
4.32
Communication between Marlborough Roads staff and the council's staff and councillors was considered to be excellent. Having the Marlborough Roads office close to the council was helpful. The council told us that there had been few formal meetings between the council and Marlborough Roads in recent months, but informal contact continued almost daily and formal meetings were held as required. The procedural audit by Land Transport New Zealand in late 2004 concluded that the relationship between the parties was co-operative and was based on open and honest communication, both formal and informal.
4.33
Communication between the Marlborough Roads office and the other roading stakeholder groups that we spoke to was also considered to be good in most cases. The New Zealand Automobile Association and the New Zealand Police believed that communication with them was useful and that they were listened to. The chairman of the regional branch of the New Zealand Road Transport Association (which represents road haulage companies) said that he would like more regular round-table discussions involving road user groups.
4.34
The Marlborough Roads office produced regular reports for the council covering finances, operations, communications (including complaints), and safety. These reported issues on an exception basis. The council had to approve variations from the council's roading budget, and the council told us that the Marlborough Roads office was good at staying within budget. Marlborough Roads also produced an annual report and used standard Land Transport New Zealand and Transit indicators to report to the council and Transit on the condition of the district roading network. The council was very happy with the level of information that it received.
4.35
The manager of the Marlborough Roads office told us that the council's performance indicators for Marlborough Roads were not well defined in the agreement, and we agree with this assessment. One of the feasibility project workstreams leading up to the establishment of the Marlborough Roads office recommended that specific performance targets be developed to ensure that service expectations were clear.
| Recommendation 5 |
|---|
| We recommend that Transit (Marlborough Roads office) and Marlborough District Council devise more specific targets for the council's service expectations from Marlborough Roads and build these into their reporting process to provide a clearer picture of ongoing performance under the agreement. |
Contractual arrangements for joint professional services
4.36
Transit's Marlborough Roads office contracted separately for professional services and physical works on the district roading network. There was a single professional services contract that covered state highways and local roads, and we focused our examination on the contractual arrangement between Marlborough Roads and the professional services contractor. As noted earlier, professional services support effective management of roading assets and include monitoring the operations of maintenance contractors. We did not examine contractual arrangements between Marlborough Roads and the maintenance contractors. The contractual arrangement for joint professional services appeared to be functioning well.
Roles and responsibilities
4.37
The professional services contract defined the main roles and responsibilities of the professional services contractor. They were to ensure that optimal levels of service and investment in the network were achieved and to audit the maintenance contractor's systems to ensure that they demonstrated fulfilment of their contractual requirements. The professional services contractor told us that, over time, the relationship with Marlborough Roads had become more of a partnership that went beyond the roles and responsibilities defined in the contract. The professional services contractor had effectively become part of Marlborough Roads and was able to respond flexibility to requirements - for example, by stepping in to provide customer support if Marlborough Roads staff were not available.
Allocation of risks
4.38
The professional services contractor carried the risk of staying within the sum agreed for the professional services contract. The risks that Transit's Marlborough Roads office retained as the client were specified in the contract.
Length of contract
4.39
The professional services contract was for an initial term of three years and could be extended by two periods of one year. The initial contract for joint professional services ran for five years and at the end this term was extended by another year to provide continuity for the new contractor who took over the southern area maintenance contract. The professional services contract for services from 1 July 2007 was to be re-tendered.
4.40
The professional services contractor told us that a longer term contract would be useful to fit in with the contractor's responsibility for long-term management of the network over the next 10 to 20 years.
Performance monitoring and reporting
4.41
Performance monitoring and reporting arrangements were adequate. Performance of the professional services contractor was assessed against the overall desired outcome of providing a safe and efficient roading network. The professional services contractor was required to report monthly to Marlborough Roads using a roading network scorecard, which tracked eight network performance criteria. Marlborough Roads' representatives also held regular meetings with the professional services contractor to review performance. No performance issues were brought to our attention.
4.42
The opportunity to be a preferred supplier for the Marlborough Roads office acted as an incentive for the professional services contractor to score well. There had been no formal disputes with Marlborough Roads.
Variations and payments
4.43
The professional services contractor told us that there had been no formal variations to the contract. The contractor had not asked for extra payments for performing additional activities under the partnership approach that had evolved. In the contractor's opinion, the relationship with Marlborough Roads would have become dysfunctional if the contractor had sought variations to the contract for these activities.
4.44
Payment arrangements appeared to be working well. Marlborough Roads invoiced the council monthly for the management fee to cover the office's administrative costs, payments claimed by contractors, and the costs of other functions that it managed such as parking. The professional services contractor checked payment claims made by maintenance contractors.
The joint contract let by Transit and Western Bay of Plenty District Council
4.45
The Western Bay of Plenty joint contract had been independently reviewed twice to assess how well it was working and the progress in achieving the specified outcomes. The first review was conducted in May 2004 and the second in August 2006. We have drawn on these independent reviews and our own interviews and examination of documents in making our assessment of the contract. The contract and arrangements for managing it were working better over time. Arrangements for governance of the contract, joint working within the consortium of contractors, and monitoring and reporting contract performance were improving as they were refined.
Governance
4.46
Governance had improved over time and was working well. The August 2006 independent review of the contract concluded that there was appropriate representation from the council and Transit on both the Joint Client Panel and the Management Board, which are responsible for overseeing performance and implementation of the contract respectively. It noted that the Management Board was working effectively in its role of governance, addressing the interests of both parties. However, the Transit regional office was not represented on the Management Board, and its integration into the contract management arrangements had not been totally successful. The review recommended that action was required to improve the Transit regional office's "buy-in" to the contract.
4.47
We agree with this assessment. Representatives from the council, Transit, and the contractor, including the contract superintendent who administered the contract on behalf of the council and Transit, told us that the Joint Client Panel and the Management Board were working well. Governance roles and responsibilities had recently been clarified, and operation of the Joint Client Panel had improved. The chairman of the Management Board and the superintendent now attended meetings of the Joint Client Panel, and the Joint Client Panel received a two-page summary report of Management Board meetings. A new Transit regional asset manager had also been appointed, and more regular meetings were now taking place between the regional asset manager and the contractor.
Risk management
4.48
Risks outside of those covered by the contract were not being formally managed through a risk management plan, but the superintendent and representatives of the council and contractor that we spoke to were aware of them and they were being managed informally.
4.49
One risk was that any changes in personnel might lead to a lack of ongoing commitment to the contract. The council told us that new councillors were briefed on the philosophy and operation of the contract.
4.50
Failure to build and maintain co-operative relationships was also recognised as a risk. The contract manager for the consortium of contractors (known as In3roads) told us that the less adversarial and more co-operative approach embodied in the contract required a different mindset. It had taken about three years to set up joint working arrangements among the contractors making up In3roads, and working arrangements within In3roads had improved. One of the In3roads contractors told us that the contract would operate more effectively if a formal alliance was in place.
4.51
The independence of the superintendent being compromised was raised as a potential risk in the August 2006 review of the contract. The review noted that having the superintendent's office in the same location as the contractor's may promote a more open and positive relationship and greater administrative efficiency but that there was also a risk that this could be seen by others as compromising the superintendent's independence. The review recommended that consideration be given as to how to address any negative perceptions of the independence of the superintendent's role. In response, the Management Board said that they would keep perceptions of the independence of the superintendent's role under review. We found that the superintendent was alert to the need to maintain independence and balance in performing his role.
Communication and reporting
4.52
Communication across the parties to the contract was good. The August 2006 independent review concluded that there was a high standard of information to the Management Board and that sound relationships existed across all parties at all levels. Our interviews with the superintendent and representatives from the council and the contractor indicated that regular and useful meetings were taking place and that communication was good.
Contractual arrangements for the performance-based contract
4.53
Contractual arrangements for the performance-based contract were working well and were being refined over time.
Roles and responsibilities
4.54
The Principals Agreement and the contract defined roles and responsibilities. Our interviews with the superintendent and representatives from the council and the contractor showed that roles and responsibilities were generally well understood.
Allocation of risks
4.55
An appendix to the contract allocates risks between the parties to the contract. As the contract is for a set sum, the contractor is responsible for covering the additional costs of dealing with any uncertain events or circumstances affecting performance of the services, unless specified otherwise in the contract.
Length of contract
4.56
The In3roads contract manager told us that the 10-year contract term had allowed useful lead-in time to put systems and processes in place to support the road maintenance services. The August 2006 independent review report noted that In3roads were to be commended for continually improving their systems. The council also told us that the contract remained flexible enough to accommodate changes.
Performance monitoring and reporting
4.57
Performance monitoring and reporting arrangements were being refined over time to be more effective.
4.58
The contract included a large number of performance measures some of which were split between local roads and state highways. Levels of service for local roads were based on the council's 1998 Road Asset Management Plan and 2002/03 District Roading Programme. The measures comprised:
- management performance measures covering the contractor's reporting and communications, quality systems, management plans, and the delivery of professional services;
- key performance measures covering the overall condition of the roading assets and the overall safety of the network; and
- operational performance measures covering road users' expectations about the district roading network's day-to-day serviceability.
4.59
Compliance with the measures by the contractor was monitored through a jointly administered independent data collection contract and through observation and audit by the council and Transit as they saw fit. Performance was regularly reported to the Management Board, Joint Client Panel, and the council's Operations Committee. The Joint Client Panel and Management Board also received reports on potential improvement initiatives that were required to be explored and developed under the contract. Procedures for handling poor performance and disputes were set out in the contract. There had been no serious disputes.
4.60
The accuracy of reporting was being refined over time. In the past, the Management Board had expressed concerns about the accuracy of the data on which reported performance was based. The August 2006 independent review concluded that compliance with the key performance measures was being accurately reported.
4.61
For the operational performance measures, the August 2006 independent review found that in the past there had been problems with assessing compliance with the measures and that how compliance was reported could be improved. The review recommended that the council and Transit conduct random targeted audits of compliance, that In3roads develop a means of demonstrating compliance with operational performance measures for state highways, and that In3roads consider improving the means by which it reported compliance.
4.62
In3roads was taking steps to improve the clarity and completeness of reporting. In3roads was developing a balanced scorecard to show the health of the contract through a series of "traffic light" indicators and supporting information that covered indicators of people and knowledge, network quality, management of risks, value for money, stakeholder satisfaction, and financial health. The Joint Client Panel had asked the superintendent to consider whether other contract well-being indicators that traced features such as morale, teamwork, communication, and technical competency could be developed.
4.63
The number and relevance of measures was also being reviewed. The August 2006 independent review found that the number of measures, particularly for the council's roading assets, meant that the Management Board was distracted by too much detail from time to time. Representatives from the contractor had also told the review team that some of the measures were stifling innovation. The review recommended that the Management Board review the effectiveness of the measures in demonstrating outcomes.
4.64
The Management Board was looking at reducing the number of key performance measures for the contract. The parties to the contract met on a number of occasions in the latter half of 2006 to discuss measures that ought to be changed in their and road users' interests. Reasons for changing the measures included:
- The level of service provided was not meeting road users' expectations.
- The measure was not driving In3roads to carry out work in areas of perceived need.
- The measure was having no influence on In3roads' decision-making, and the data gathering and reporting was a wasted expense.
- Data collected at different times were showing such large swings as to lack credibility.
- Contract requirements were no longer aligned with changing state highway standards.
4.65
Proposed improvements to levels of service and revisions of performance measures had been identified, some of which required additional input from, or incurred additional costs for, In3roads. Areas where other contract requirements could be relaxed to produce enough savings to offset the cost of the proposed improvements without having any detrimental effect were being identified at the time of our audit.
Variations and payments
4.66
The contract superintendent told us that about 120 variations had been negotiated to the contract since it was awarded. The August 2006 independent review examined the system for processing variations and found that the system was sound, although the tracking and reporting of total costs of completed and committed variations needed to be improved.
4.67
Arrangements for payments were set out in the contract. Essentially, the set sum was spread across the 10-year term of the contract, and the amount due for each year was paid to the contractor by equal monthly instalments. The In3roads contract manager told us that the payment process worked smoothly.