Part 4: Operating Gateway reviews in New Zealand

4.1
In this Part, we:

• consider how well Gateway reviews have been operating in New Zealand;
• examine whether the right projects have Gateway reviews at the right time; and
• consider how well Gateway reviews fit with other types of project monitoring and assurance.

Our expectations

4.2
We expected to see:

• Gateway reviews following a set methodology;
• effective processes to make sure that qualifying projects have Gateway reviews at the right time; and
• a clear understanding of how Gateway reviews fit with other monitoring and assurance functions.

Summary of our findings

4.3
Gateway reviews have been operating well in New Zealand. They have been carried out in accordance with the official Gateway methodology. This has been achieved through training and support and helps make sure that projects get the benefits associated with Gateway reviews.

4.4
Most projects are having Gateway reviews as required. The Treasury is aware that a few projects may be missed but, in general, it has effective processes to make sure that qualifying projects are identified and kept track of as they progress.

4.5
There is not a clear understanding of how Gateway reviews fit with other monitoring and assurance activities, in particular those activities run by different parts of "the Corporate Centre".² As a result, these activities may not be as effective and efficient as they could be and may not deliver the best outcomes for projects.

Gateway reviews follow a set methodology

4.6
Gateway reviews are planned and operated in a way that helps to ensure that they follow the Gateway methodology. There is some flexibility in the way that each review is carried out to best meet the needs of each individual project, but the general approach has to be followed and the main features included. These features include the confidentiality of the reports and the advisory nature of the reviews.

4.7
The following processes have been put in place to help review teams follow the methodology:

• all Gateway reviewers have to attend training before they can take part in a review;
• review teams always include some experienced reviewers who have been through the process several times;
• there is a planning day held for each review, which is attended by the project sponsor, the review team, and someone from the Gateway unit, when the review scope and approach are confirmed;
• review teams are provided with guidance and templates; and
• someone from the Gateway unit also attends at times during the review week, to provide advice on the Gateway review process and its application.

4.8
The reviewers we spoke to had a good understanding of the Gateway approach. Some of them had taken part in reviews where new reviewers had started to explore areas that are out of scope of Gateway reviews, such as policy decisions. This was picked up and resolved by the rest of the team.

4.9
The methodology has to be followed in order to use the Gateway brand, which is owned by the United Kingdom Government. This means that any other jurisdiction that uses Gateway reviews has limited scope to change how the reviews are applied. A few small changes have been made to fit the New Zealand context (for example, changes in terminology in supporting documents). Generally, however, the reviews here are carried out in the same way as they are in the United Kingdom and Australia.

4.10
By following the Gateway methodology, reviews are more likely to deliver the benefits associated with Gateway reviews.

4.11
The Treasury has processes to make sure that most qualifying projects have the right Gateway review at the right time. We found consistency between the agencies most likely to have high-risk projects and the agencies that have Gateway reviews. This shows that the Treasury's processes are largely effective.

4.12
Agencies with qualifying projects are supposed to submit a risk profile assessment to the Treasury, but this does not always happen. This might be intentional or it might reflect a lack of awareness. The Gateway unit looks at information from various sources to identify potentially qualifying projects for which an assessment has not been submitted. These information sources include:

• other parts of the Treasury, such as the people who monitor capital expenditure and review business cases;
• other central agencies with a monitoring role; and
• the online register of government tenders (GETS), which the Gateway unit monitors regularly.

4.13
Sometimes people connected with an agency will alert the Gateway unit about projects that need to complete a risk profile assessment. The Gateway unit can then ask an agency to comply. The Treasury is confident that more than 90% of all qualifying projects have Gateway reviews.

4.14
The Treasury is aware of one or two agencies that actively avoid engagement with the Treasury's monitoring and review processes, including Gateway reviews. This is being dealt with at an organisational level. Some other agencies engage with Gateway reviews but only when they are prompted. The Treasury has found that, when an agency has been through a few reviews, it is much more likely to engage proactively with Gateway reviews.

4.15
We looked at which agencies have had Gateway reviews and compared this with the agencies that the Treasury has identified as investment-intensive. We expected to see a reasonable level of consistency between them. Figure 3 shows how many projects from each agency have had Gateway reviews. Figure 4 shows the current list of the most investment-intensive agencies (categorised by the Treasury as "Tier 1" and "Tier 2").

4.16
Figures 3 and 4 show that most of the investment-intensive agencies have had Gateway reviews, as expected. Those that have not had reviews might own or control a high value of assets but might not have carried out any large or high-risk projects since Gateway reviews were introduced. Figures 3 and 4 support the Treasury's assertion that most eligible projects have Gateway reviews.

Figure 3
Number of projects that have included Gateway reviews, by agency

Source: Our analysis of Treasury data.

Figure 4
Investment-intensive agencies

Source: The Treasury; http://www.treasury.govt.nz/statesector/investment-intensive-agencies.

4.17
Once a project has had its first Gateway review, the Treasury makes sure that subsequent reviews happen at the right time. Some project sponsors contact the Gateway unit when their project is ready for the next review. For others, the Gateway unit keeps track of each project and contacts the project sponsor when the next Gateway review is due. If a project has been delayed then the next review might be postponed. A few projects have been cancelled or reduced in scope and therefore no longer required Gateway reviews.

* Ōtākaro Limited is responsible for projects that previously came under the Canterbury Earthquake Recovery Authority.

4.18
For most projects, Gateway reviews, like many other types of intervention, can be most useful early on, before the project has committed to a particular solution. This is when ideas can be tested and there is more scope to modify the project to help it succeed.

Improvements have been made to Gateway reviews over time

4.19
The Gateway unit uses up-to-date information to make changes to Gateway reviews. In particular, the Gateway unit keeps in close contact with the United Kingdom Major Projects Authority – the agency with responsibility for the Gateway brand. This relationship led to the introduction of delivery confidence ratings at the start of 2014, a change that had already been made in some other jurisdictions. New Zealand also takes part in an Australasian Gateway forum, where Gateway units from different jurisdictions can share ideas for improving Gateway reviews.

4.20
The Gateway unit also uses feedback from project sponsors and reviewers to make minor adjustments to the way it operates Gateway reviews.

It is not clear how Gateway reviews fit with other monitoring and assurance activities

4.21
We found that there was not a clear understanding of how Gateway reviews fit with other types of monitoring and assurance activities that apply to most large projects. In particular, we found that roles and responsibilities in the Corporate Centre for monitoring and assurance of major projects were not clear. The Treasury confirmed this and told us that recent attempts to reach agreement on these roles and responsibilities had not been successful.

4.22
Our audit did not look in depth at these other types of monitoring and assurance activities or how well all these activities work together to get the best outcomes for projects. However, we did expect to find a clear understanding of how Gateway reviews fit with those other activities.

4.23
Without this understanding, monitoring and assurance activities can be less effective and efficient. There is the potential for overlap if different reviews look at the same things. In some cases, this could lead to conflicting advice, especially if the purpose of each review is not clear. There is also the potential for reviews to completely miss some aspects of a project.

Gateway reviews and Independent Quality Assurance

4.24
One particular type of review that also applies to most projects eligible for Gateway reviews is Independent Quality Assurance (IQA). An IQA review is carried out by an external provider; this is usually a consulting firm. Some people we spoke to did not understand the difference between an IQA review and a Gateway review. The main differences between them are shown in Figure 5.

Figure 5
Differences between Gateway reviews and Independent Quality Assurance reviews

4.25
There are some similarities between Gateway reviews and IQA reviews. For example, both tend to use interviews and document reviews. Some duplication in the matters they look at is also possible. However, agencies have some discretion to direct the scope or focus of an IQA review and so minimise any duplication.

4.26
We note that, in the United Kingdom, where Gateway reviews were designed, IQA reviews are not in common use. It is not clear how much effort went into ensuring that Gateway reviews did not duplicate IQA reviews when Gateway reviews were introduced in New Zealand. However, the State Services Commission was responsible for both introducing Gateway reviews and helping agencies to plan IQA reviews, and was well placed to ensure a clear distinction. Documentation that we looked at included a table showing the differences between Gateway reviews and IQA reviews.

2: "The Corporate Centre" is a term used to describe public entities that have some oversight role in large government projects and programmes. The Corporate Centre includes the Treasury (investment assurance), the State Services Commission (capability assurance), the Government Chief Information Officer (information and communication technology assurance), and the Department of the Prime Minister and Cabinet (co-ordinating and supporting government priorities). The Corporate Centre also includes the public entities that have lead responsibilities throughout government for procurement and property (Ministry of Business, Innovation and Employment) and shared ICT capabilities (Department of Internal Affairs).