Part 2: Summary of changes to the selected defence acquisition projects
2.1
In this Part, we provide:
- our overall findings about how costs and time frames have changed during the acquisition process for each of the 10 selected projects;
- high-level analysis of those changes for each project; and
- a summary of the extent of the changes for each project.
2.2
We looked at cost and time frame changes between the Approval to Commence and
Approval to Commit points in the acquisition process. We also looked at changes between
the Approval to Commit point and the latest forecasts that the Ministry reported to the
Committee in December 2007. For one project managed by the NZDF, we looked at
changes between the Approval to Commit point and the latest forecasts the NZDF provided
to us at the end of 2007 and in April 2008.
2.3
One of the main objectives of our original audit was to provide Parliament with a standard set of information that recorded the progress of each of the 10 selected acquisition projects from
the Approval to Commence point, including the reasons for changes to costs, time frames,
and essential user requirements. The information we present here is limited to a high-level
analysis because of the difficulties we had in sourcing that standard set of information. Part 3
explains these difficulties in more detail.
2.4
We were able to compile enough information to describe each project’s purpose and history,
and to produce summary tables on the changes to costs and time frames since the project
started. However, the tables contain many assumptions because consistent and comparable
information was not always available. The summary tables and descriptions of each project’s
purpose and history are in the Appendix.
2.5
To provide an analysis of each project’s progress against the costs and time frames set at
the Approval to Commence point, we had to assume:
- an Approval to Commence point for those projects that started before the CMF was first implemented in May 2004, by identifying the relevant historical approval; and
- some dates for project time frames, because the dates were sometimes imprecise or loosely inferred rather than clearly specified in project approval documents. If dates were mentioned in other documents but not in project approval documents, we have used those. Where we have made assumptions, we have been conservative. For example, if only the year for delivery was indicated, we have assumed that delivery would be by the end of the year.
2.6
For projects that started before the CMF was implemented, we have not always used the
same Approval to Commence figures that the Ministry did in its report to the Committee in
December 2007. We note that the Ministry used figures from LTDPs and reported them as
Approval to Commence figures. We also note that the Ministry was inconsistent in how it
used LTDP figures:
- The Ministry did not always use the project cost and time frame figures from the first LTDP the project was described in. For example, for the Medium Utility Helicopter project, we use the $400 million to $550 million range from the Cabinet Approval to Commence (which cited the 2003 LTDP). In its report to the Committee, the Ministry used the $400 million to $550 million range from the 2004 LTDP, which also included the Training/Light Utility Helicopter.
- The Ministry did not always use cost and time frame figures from the same LTDP. For example, for the Approval to Commence point for the Light Operational Vehicle project, the Ministry used the 2002 LTDP for costs and the 2004 LTDP for time frames.
2.7
Wherever possible, we have identified the project-specific Cabinet approval that equates to
the Approval to Commence point, and used this as the source of approved figures. To be
consistent, we also cite estimated cost and time frame information from the same approval
document. We were not always able to do this, because time frames were sometimes not
specified at the Approval to Commence point. In those situations, we have used LTDP
figures or information from other sources. Identifying the project-approval equivalent to the
Approval to Commence point was not always easy, and was one of the difficulties we had in
conducting our original audit. We discuss this difficulty further in paragraphs 3.11-3.13.
2.8
In the summary tables of information in the Appendix, we provide references to the sources
of our information. We also show where our figures come from a different source – and differ
– from those the Ministry reported to the Committee in December 2007.
Our overall findings
2.9
During our audit, we were mindful of the uncertainty in the defence agencies’ estimates
before the Approval to Commit point. The defence agencies do not indicate the degree of
uncertainty inherent in their estimates, but our work confirms that the costs approved at the
Approval to Commit point are sometimes significantly higher than those estimated at the
Approval to Commence point.
Changes between the Approval to Commence and Approval to Commit points
2.10
Overall, for most of the 10 projects covered by our audit, estimated costs and time frames
between the Approval to Commence and Approval to Commit points had increased – in
some cases, significantly.
2.11
The defence agencies advised us that changes between the Approval to Commence and
Approval to Commit points are unavoidable because:
- the defence agencies can only make general enquiries of potential suppliers before the Approval to Commence point, so the estimate presented to Cabinet then is, at best, an “intelligent guess";
- the precision of estimates at the Approval to Commence point will vary depending on whether the capability is an established piece of equipment or whether design work is needed; and
- projects are almost always paid for in overseas currency, so prices are subject to foreign exchange fluctuations. To cover this uncertainty, in all of the submissions to Cabinet for Approval to Commit, the project cost is adjusted to accommodate the estimated cost of buying foreign currency in the future.
2.12
The CMF indicates that, before the Approval to Commence point, the defence agencies can
consult with suppliers to get information to support the development of robust cost estimates.
If this is not the defence agencies’ practice, then the CMF needs to be amended to reflect
actual practice. We understand that the Ministry will soon be making major changes to the
CMF to incorporate the requirements of the Treasury’s Capital Asset Management Review,
and the Ministry’s own examination of the level of accuracy that can be expected in cost
estimates at various stages of the acquisition process.
Changes from the Approval to Commit point to the defence agencies’ forecasts at the end of 2007
2.13
Between the Approval to Commit point and the latest forecasts, no further cost increases
were forecast or had occurred for most of the projects, but most had experienced further
delays. The delays were smaller than those between the Approval to Commence and
Approval to Commit points.
2.14
We expected few changes to costs and time frames after the Approval to Commit point. The
defence agencies advised us that the forecast costs and time frames at the Approval to
Commit point are much more accurate because the Ministry has been able to get costed
proposals from potential suppliers and is about to award a firm fixed-price contract.
High-level analysis of cost and time frame changes for each project
2.15
Figure 1 provides for each project:
- a summary of the extent of changes to project costs and time frames between the Approval to Commence point and the latest forecasts at December 2007 (and April 2008 for the Improvised Explosive Device Disposal project managed by the NZDF); and
- some commentary on the reasons for the cost and time frame changes.
2.16
The sources we used included:
- cost and time frame estimates for projects in LTDPs;
- Cabinet papers seeking Approval to Commence and Approval to Commit;
- project progress reports, evaluations, and contract documents from the Ministry’s acquisition files; and
- the project cost and time frame information reported by the Ministry to the Committee in
December 2007, and by the NZDF to us during the audit.
Figure 1
Summary of cost and time frame changes for each of the selected defence acquisition projects
| Medium Utility Helicopter | ||
|---|---|---|
| Purpose: | To replace 14 Iroquois helicopters with eight NH90 Medium Utility Helicopters. |  |
| Changes between the Approval to Commence and Approval to Commit points The cost approved at the Approval to Commit point ($771.7 million) was $221.7 million more than the upper limit of the estimated range at the Approval to Commence point ($550 million). Between the approval points, the date of introduction into service increased by 42 months. Changes between the Approval to Commit point and the end of 2007 Since the Approval to Commit point, there have been no further changes to costs or time frames. Overall, the changes for the Medium Utility Helicopter project came from the difference between initial estimates and the more accurate costs from the tender process. The lengthy tender process also delayed the project. |
||
| Multi-Role Vessel and Patrol Vessels (Project Protector) | ||
|---|---|---|
| Purpose: | To replace HMNZS Canterbury with a Multi-Role Vessel and introduce a mix of offshore and inshore patrol capabilities through the purchase of two Offshore Patrol Vessels and four Inshore Patrol Vessels. |    |
| Changes between the Approval to Commence and Approval to Commit points The cost approved at the Approval to Commit point ($499.7 million) was $0.3 million less than the Cabinet-directed cap at the Approval to Commence point ($500 million). The estimated delivery date for the Multi-Role Vessel increased by 12 months (no timing was given for the delivery of the Patrol Vessels at the Approval to Commence point). Changes between the Approval to Commit point and the end of 2007 Between the Approval to Commit point and the end of 2007, there was no reported change in cost. However, the Treasury understands that the defence agencies, after the Ministry's December 2007 forecast, have found that their project commitments now exceed the project's budget. Since the Approval to Commit point, the Multi-Role Vessel's delivery was delayed by another six months. The last Patrol Vessel is forecast to be delivered 10 months later than the date defined at the Approval to Commit point. Overall, there have been delays in design, construction, and testing of the vessels. The defence agencies have advised us that Ministers have been informed of the project's cost and time situation. |
||
| P-3 Systems Upgrade | ||
|---|---|---|
| Purpose: | To upgrade the mission, and communications and navigation systems, of six P-3 Orion aircraft. |  |
| Changes between the Approval to Commence and Approval to Commit points The cost approved at the Approval to Commit point ($373.1 million) was $91.1 million more than the upper limit of the estimated range at the Approval to Commence point ($282 million). The estimated delivery date for the last aircraft increased by 18 months. Changes between the Approval to Commit point and the end of 2007 Since the Approval to Commit point, the forecast of cost has decreased by $0.1 million, but the delivery date for the last aircraft has increased by another three months. Overall, the increase in the estimated cost and the delays are because of technical issues with the prototype aircraft and increased specifications. |
||
| C-130 Life Extension | ||
|---|---|---|
| Purpose: | To upgrade and extend the life of five C-130 Hercules aircraft to 2017, improving aircraft availability and reliability. |  |
| Changes between the Approval to Commence and Approval to Commit points The cost approved at the Approval to Commit point ($233.7 million) was $18.3 million less than the estimate at the Approval to Commence point ($252 million). The estimated delivery date for the last aircraft increased by 15 months. Changes between the Approval to Commit point and the end of 2007 Since the Approval to Commit point, the forecast of cost has decreased by $0.7 million, but the delivery date of the last aircraft has increased by another six months. Overall, the project’s estimated cost decreased as the Ministry received more information about the actual cost of capabilities, even though capabilities have been added. The increases in the delivery time frames were the result of the contractor and the Ministry encountering technical difficulties with the modifications to the first aircraft. |
||
| Boeing 757 Acquisition and Modification | ||
|---|---|---|
| Purpose: | To acquire and modify two Boeing 757-200 aircraft to replace two ageing Boeing 727s. Modifications to the 757 aircraft were required to meet the NZDF's capability requirements. |  |
| Changes between the Approval to Commence and Approval to Commit points The cost approved at the Approval to Commit point ($220.6 million) was $21.3 million more than the estimate at the Approval to Commence point ($199.3 million). The estimated delivery date for the modified aircraft increased by 36 months. Changes between the Approval to Commit point and the end of 2007 Since the Approval to Commit point, there has been no change in the forecast of cost, but the delivery date has increased by another 12 months. Overall, the aircraft were purchased at a cost greater than that estimated, but were acquired on schedule. There was an increase in the estimated cost and delivery date for the modified aircraft. There have been issues agreeing contractual terms and subsequent changes to the requirements of the modification programme (for example, the addition of Antarctic operations required the installation of software and navigation equipment). |
||
| Training/Light Utility Helicopter | ||
|---|---|---|
| Purpose: | To replace five Sioux helicopters with five Agusta-Westland A109 Training/Light Utility helicopters and a flight simulator. |  |
| The Training/Light Utility Helicopter contract was still being negotiated during our audit, and the Approval to Commit point came after the Ministry's December 2007 report to the Committee. However, the first estimate of cost and time proved unrealistic, with an increase of $99 million between the 2002 and 2006 LTDPs. The defence agencies advised that the estimate in the 2002 LTDP was based on an "off-the-shelf" civil helicopter. The 2006 LTDP estimate was informed by technical changes to the Medium Utility Helicopter project that meant an "off-the-shelf" civil helicopter would no longer meet the NZDF's training needs. It became necessary to seek a militarised helicopter at additional cost. Cabinet noted that the $110 million figure was an estimate, subject to confirmation after further discussions with manufacturers and suppliers. The cost approved at the Approval to Commit point in April 2008 ($139.3 million) was $29.3 million more than the estimated cost at the Approval to Commence point ($110 million), using the 2006 rather than 2002 estimate. The estimated delivery time frame for the helicopters has increased by 36 months between the Approval to Commence and the Approval to Commit points. |
||
| Light Operational Vehicle | ||
|---|---|---|
| Purpose: | To acquire a fleet of Light Operational Vehicles to replace the Army's Land Rovers, which were overdue for replacement. Our analysis focused on the military vehicles purchased as part of the project. |  |
| Changes between the Approval to Commence and Approval to Commit points We have a different interpretation of what the Approval to Commence point is than the Ministry, so there are some differences between our figures and those reported by the Ministry to the Committee in December 2007. We use the July 1999 Cabinet approval figure ($56.1 million) for the Approval to Commence point. The Ministry uses the range ($60 to $110 million) from the 2002 LTDP because it considers the 1999 approval related to a project that had been cancelled. We could not verify this through the Ministry's documentation. Based on our interpretation, we consider that the cost approved at the Approval to Commit point ($93.3 million) was $37.2 million more than the estimate at the Approval to Commence point ($56.1 million). The estimated delivery date for the vehicles increased by 69 months. Changes between the Approval to Commit point and delivery Between the Approval to Commit point and the vehicles' delivery, there was no change in cost, and delivery was six months earlier than forecast. Overall, the main reasons for the cost and time changes were a failed initial tender process, subsequent changes to the numbers and types of vehicle required as the Army refined its requirements, and Special Operations Vehicles being added to the project to close a mobility gap for the Special Forces. |
||
| Medium Range Anti-Armour Weapon | ||
|---|---|---|
| Purpose: | To acquire the Javelin "fire and forget" Medium Range Anti-Armour Weapon to negate threats from tanks and armoured vehicles at a distance of several kilometres. |  |
| Changes between the Approval to Commence and Approval to Commit points The cost approved at the Approval to Commit point ($23.9 million) was $2.4 million more than the estimate at the Approval to Commence point ($21.5 million). The estimated delivery date for the weapon increased by 24 months. Changes between the Approval to Commit point and delivery Between the Approval to Commit point and the weapon's delivery, there was no change in cost, while delivery was delayed by another 16 months. The delay since the Approval to Commit point was because of a manufacturing defect affecting the worldwide supply of missiles for the weapon. |
||
| Improvised Explosive Device Disposal | ||
|---|---|---|
| Purpose: | To enhance the NZDF's existing Improvised Explosive Device Disposal (IEDD) capability and develop a credible chemical, biological, radiological, and conventional IEDD capability. Both capabilities cover detection, identification, field evaluation, rendering safe recovery, and final disposal. |  |
| Changes between the Approval to Commence and Approval to Commit points The cost approved at the Approval to Commit point ($21.6 million) was $1.9 million more than the estimate at the Approval to Commence point ($19.7 million). It was not possible to calculate any change in delivery date because there was no indication of this in the Cabinet Approval to Commence. Changes between the Approval to Commit point and the end of 2007 While timing is still within that defined at the Approval to Commit point (2007/08 financial year), the NZDF told us that it is likely that it will need to seek additional funds to complete the project. As at April 2008, the NZDF projected that another $0.75 million would be required, although that would be confirmed after the completion of a tender round. The cost changes are the result of increased construction costs for IEDD units. Although the project is on schedule, the NZDF has advised that a lack of suitably trained and qualified personnel means that it has difficulty maintaining staff levels. |
||
| Very Low Level Air Defence Alerting and Cueing System | ||
|---|---|---|
| Purpose: | To acquire the Alerting and Cueing System needed to make completely operational the Very Low Level Air Defence (VLLAD) missiles and launchers that were acquired and delivered through a separate acquisition. The Alerting and Cueing System detects, identifies, and warns of approaching aircraft then assists with the aiming of the VLLAD weapons. |  |
| Changes between the Approval to Commence and Approval to Commit points The cost approved at the Approval to Commit point ($13.7 million) was $5.7 million more than the estimate at the Approval to Commence point ($8 million). The estimated delivery date for the system increased by 39 months. Changes between the Approval to Commit point and the end of 2007 Between the Approval to Commit point and the end of 2007 there was no reported change in cost. However, in April 2008 the Cabinet External Relations and Defence Committee approved the addition of $0.55 million to the budget to complete the project. The delivery date increased by nine months since the Approval to Commit. The significant cost and time changes are largely the result of there not being an "off-the-shelf" Alerting and Cueing System at the time of first estimates, increases in the first estimates of cost and time once such a system was found, and then the need to purchase additional equipment to make the system fully operational after it was delivered. |
||
Source: All images from the Ministry of Defence, except for Training/Light Utility Helicopter (which is from Agusta-Westland) and the Improvised Explosive Device Disposal (which is from the New Zealand Army).
Summary of cost and time frame changes for each project
2.17
We used the information on cost and time frame changes for each project to look at the
pattern of changes for all of the projects. In particular, we looked at how much the costs and
time frames had changed:
- between the Approval to Commence and the Approval to Commit points; and
- between the Approval to Commit point and the defence agencies’ forecasts at the end of
2007.
Changes between the Approval to Commence and Approval to Commit points
2.18
Figure 2 shows the cost and time frame changes for each of the selected acquisition projects
between the Approval to Commence and Approval to Commit points.
2.19
Except for Project Protector and the C-130 Life Extension project, estimated costs increased
between the Approval to Commence and Approval to Commit points. The increases ranged
from $1.9 million for the Improvised Explosive Device Disposal project to $221.7 million for
the Medium Utility Helicopter project. The decreases were $0.3 million for Project Protector
and $18.3 million for the C-130 Life Extension project.
2.20
Except for the Improvised Explosive Device Disposal project, where there was no indication
of timing at the Approval to Commence point, the projects experienced delays between the
Approval to Commence and the Approval to Commit points. Delays ranged from 12 months
for the Multi-Role Vessel component of Project Protector to 69 months for the Light Operational Vehicle.
Figure 2
Cost and time frame changes between the Approval to Commence and Approval to Commit points for each of the
selected defence acquisition projects
| Project | Change in costs | Change in time (months) |
|
|---|---|---|---|
| $m | % | ||
| Medium Utility Helicopter | +221.7 | +40 | +42 |
| Multi-Role Vessel and Patrol Vessels (Project Protector) |
0.3 | -0.06 | +12 (for the Multi-Role Vessel. No timing was given in the Cabinet Approval to Commence for the Patrol Vessels) |
| P-3 Systems Upgrade | +91.1 | +32 | +18 |
| C-130 Life Extension | -18.3 | -7 | +15 |
| Boeing 757 Acquisition and Modification | +21.3 | +11 | +36 |
| Training/Light Utility Helicopter | +29.3 | +27 | +36 |
| Light Operational Vehicle | +37.2 | +66 | +69 |
| Medium Range Anti-Armour Weapon | +2.4 | +11 | +24 |
| Improvised Explosive Device Disposal | +1.9 | +10 | Not determined as no timing was given in the Cabinet Approval to Commence |
| Very Low Level Air Defence Alerting and Cueing System |
+5.7 | +71 | +39 |
Changes from the Approval to Commit point to the defence agencies’ forecasts at the end of 2007
2.21
Figure 3 shows the cost and time frame changes for each of the selected projects between
the Approval to Commit point and the defence agencies’ forecasts at the end of 2007 (except
for the Training/Light Utility Helicopter project, because it had not reached the Approval to
Commit point at the end of 2007). Figure 3 shows that each project was reported to be either on, or slightly under, the budget defined at the Approval to Commit point, but most had
experienced further delays. The extent of the delays was generally less than that between
the Approval to Commence and Approval to Commit points. According to the Ministry, there
has been little movement in costs because of its practice of entering into firm fixed-price
contracts, where the cost risk is borne by the supplier. However, the Ministry says that it has
only a limited ability to affect the timelines of suppliers. The Ministry’s influence is limited to
withholding milestone payments and threatening liquidated damages claims.
2.22
As at the end of 2007, against the cost and time parameters set at the Approval to Commit point:
- one project was delivered on budget and early (Light Operational Vehicle);
- two projects were under budget but late (P-3 Systems Upgrade and C-130 Life Extension);
- two projects were on budget and on time (Medium Utility Helicopter and Improvised Explosive Device Disposal); and
- four projects were on budget but running late (Project Protector, Boeing 757 Acquisition and Modification, Medium Range Anti-Armour Weapon, and Very Low Level Air Defence Alerting and Cueing System).
2.23
However, we understand from the Ministry, the NZDF, and the Treasury that Project
Protector and the Improvised Explosive Device Disposal projects are likely to require
additional funds before they can be completed. The Very Low Level Air Defence Alerting and
Cueing System project has received an additional $0.55 million for it to be completed, but
this is not reflected in Figure 3. Cabinet approval for the additional funds came after the
Ministry reported its forecast to the Committee.
Figure 3
Cost and time frame changes between the Approval to Commit point and the defence agencies’ forecasts at the end
of 2007 for each of the selected defence acquisition projects
| Project | Change in costs | Change in time (months) | |
|---|---|---|---|
| $m | % | ||
| Medium Utility Helicopter | 0 | 0 | 0 |
| Multi-Role Vessel and Patrol Vessels (Project Protector) | 0 | 0 | +6 months for the Multi-Role Vessel and +10 months for the Patrol Vessels |
| P-3 Systems Upgrade | -0.1 | -0.03 | +3 |
| C-130 Life Extension | -0.7 | -0.3 | +6 |
| Boeing 757 Acquisition and Modification | 0 | 0 | +12 |
| Training/Light Utility Helicopter | Not applicable because the contract had yet to be finalised as at the end of 2007 | ||
| Light Operational Vehicle | 0 | 0 | -6 |
| Medium Range Anti-Armour Weapon | 0 | 0 | +16 |
| Improvised Explosive Device Disposal | 0 | 0 | 0 |
| Very Low Level Air Defence Alerting and Cueing System | 0 | 0 | +9 |