Part 5: Capability and capacity for Gateway reviews

5.1
In this Part, we consider whether agencies, the Treasury, and review teams have the capability and capacity to carry out their respective roles in Gateway reviews effectively.

Our expectations

5.2
We expected to see:

• agencies able to take part in reviews and implement the resulting recommendations;
• appropriate resourcing to support Gateway reviews;
• a sufficient pool of suitably qualified and experienced Gateway reviewers; and
• processes to make sure that the best team is selected for each Gateway review.

Summary of our findings

5.3
In our view, agencies have the capability and capacity to pay for and participate in Gateway reviews. The $75,000 cost of each Gateway review is reasonable, especially when considered as a percentage of the overall cost of each project. For the review itself, there is some burden on the agency, but this is limited. Project sponsors told us they did not find Gateway reviews too onerous.

5.4
The Treasury has adequate resources to run Gateway reviews and does this well. However, it has limited capacity for additional work, such as making improvements.

5.5
Having the right review team is critical to the success of Gateway reviews. There is a large pool of reviewers available, which helps make sure that the best team is put together for each review. This can be challenging, but the Treasury puts a lot of effort into doing this well.

5.6
Review team independence is also important and has been shown to bring benefits to projects.

Agencies

The cost of Gateway reviews is reasonable

5.7
In our view, the cost of Gateway reviews is reasonable, especially when compared to the overall budget of the types of project that have Gateway reviews. Each review costs the agency $75,000. For a project of $50 million, five Gateway reviews will cost 0.75% of the budget. Most projects going through Gateway reviews have much larger budgets, and few of them have all five reviews.

5.8
Agencies with more experience of Gateway reviews are more likely to build the costs into their budgets. Not planning for Gateway review costs will inevitably make the costs harder to accommodate.

Gateway reviews are not too onerous for agencies

5.9
We consider that, on balance, Gateway reviews are not too onerous for agencies. Any type of project review will place demands on the agency and project team involved. Project teams are often under a lot of pressure already, and reviews require people to give up their time to provide documents and attend interviews.

5.10
Gateway reviews are designed to reduce the burden on agencies and project teams. For example, the Gateway unit selects and contracts the review team and makes all arrangements for the team's travel and accommodation. Most people involved from within the agency will be required to attend only one interview of an hour's duration. Although review teams need to see documentation, this would comprise key project documents that we would expect to be complete and easily available as a matter of course for any project.

5.11
The burden of a Gateway review is greater on the project sponsor, who will have to engage more frequently with the review team, including attending the daily debriefs. Project sponsors told us that they did not find this burden too much and accepted the need to allow for Gateway reviews as part of doing a major public sector project. Feedback received by the Treasury shows that most project sponsors agree that one week is a suitable duration for a review. Project sponsors have commented that one week is a good balance between giving the review team enough time to understand the project while also allowing the team to report back its findings quickly.

5.12
The burden of a Gateway review does not stop at the end of the review week. After receiving the Gateway report, the project sponsor will have recommendations to consider and implement. We discuss how well agencies implement Gateway recommendations in Part 6. However, we have not seen any evidence that agencies lack the capacity and capability to implement recommendations made during Gateway reviews.

The Treasury

The Treasury has enough resourcing to run Gateway reviews but little capacity for improvement

5.13
The Gateway unit has adequate capability and capacity to run Gateway reviews, but no spare capacity. The unit currently has two full-time staff members and one part-time staff member and supports 25-30 reviews each year. Over the last few years, the number of reviews has increased, but the number of staff in the Gateway unit has decreased. This means that there is no spare capacity for adding value through activities such as researching and implementing improvements.

5.14
The Gateway unit has been run effectively and efficiently by the same person for several years. However, this has meant that the unit is heavily dependent on that one person's knowledge and experience.

Gateway review teams

There are enough reviewers available to carry out Gateway reviews to a high standard

5.15
There is a pool of nearly 700 reviewers from New Zealand, Australia, and the United Kingdom available for carrying out Gateway reviews in New Zealand. Having a large pool of reviewers makes it easier to find the best team of available reviewers for each review.

5.16
Review team members come from various backgrounds. Each team is put together to get the best mix of skills and experience for each project. For example, the review team for a construction project that is about to sign a contract might include people with experience in construction, contract management, and project management.

5.17
Currently, about a third of the review pool is Australian, with a few from the United Kingdom. Most review teams include at least one Australian. Overseas reviewers may have expertise not available in New Zealand. They are often used as review team leaders because they typically have more experience, particularly for large or complex projects. Overseas reviewers also have fewer conflicts of interest with New Zealand agencies.

5.18
The review team is critical to the effectiveness of a Gateway review. When a team does not have the right skills and experience, or does not work well together, the advice for the project sponsor is likely to be of less value.

New Zealand is still dependent on reviewers from overseas

5.19
New Zealand still uses a lot of reviewers from overseas. The United Kingdom and Australia have been running Gateway reviews for longer than New Zealand and have more projects that have been through Gateway reviews. This means that reviewers from those countries often have a lot more experience than New Zealand reviewers. When Gateway reviews were first introduced, most reviewers came from Australia and some from the United Kingdom, because there was little experience in New Zealand. Over time, more New Zealanders have become reviewers, and there is less reliance on reviewers from overseas.

5.20
Reviewers can come from the private or the public sector. For some projects, reviewers from the private sector have the most suitable skills and experience. Review teams usually include at least one reviewer from the public sector, because they understand the system that public sector projects operate in.

5.21
Figure 6 shows how the mix of reviewers taking part in reviews has changed since Gateway reviews were introduced in New Zealand.

Figure 6
New Zealand Gateway reviewers by country and sector

Source: The Treasury.

The Treasury trains and assesses reviewers

5.22
The Gateway unit makes sure that reviewers are prepared and suitable to work in and lead a Gateway review team. An application process makes sure that all new reviewers have the right skills and experience. New reviewers have to attend a workshop about Gateway reviews before they can take part in a review. The workshop explains how the reviews work and what reviewers should be looking for.

5.23
After each review, the review team leader gives feedback to the Gateway unit on the team members. This feedback can identify people with the potential to become team leaders, or those who are not suitable for Gateway reviews. Review team members also provide feedback on the team leader. Feedback from project sponsors is also used.

5.24
Gateway reviews are carried out by a team of four reviewers, one of whom is the review team leader. To become a review team leader, a reviewer usually has to complete 8-10 reviews with consistently good feedback. They are then invited to a workshop for review team leaders.

The Treasury understands how to put a high-performing review team together

5.25
The Gateway unit puts a lot of effort into getting the best team for each review. Our interviews, together with feedback provided to the Gateway unit, confirmed that this is usually done well.

5.26
Gateway reviews are high-level and intensive, and are based on interviews. Review team members therefore need the right mix of technical and professional skills, combined with suitable interpersonal skills. Technical and professional skills are needed to give high-quality advice quickly.

5.27
People with the right skills are often ruled out of a review team by a conflict of interest. This can be a particular problem in specialist areas in New Zealand, where only a few people have expertise in areas such as education funding systems, heritage construction, or urban renewal.

5.28
The right interpersonal skills are important, because reviewers have to work as a team and communicate effectively with the agency.

5.29
The Gateway unit also considers the mentoring and development needs of review team members. A new team leader will always have an experienced team leader in their team. Each review will include only one new reviewer. Experienced reviewers can be placed with an experienced leader, who can assess whether they are ready to take on the role of team leader.

5.30
Although there is a large pool of reviewers, taking all these matters into consideration means that it can sometimes be difficult to find the right mix of available people for each review. We heard about some instances where a reviewer was not suitable, and this affected how well the team was able to work with the project sponsor, and the quality of advice they provided.

Review team independence is important

5.31
Review team members are independent of the project being reviewed and the agency managing the project. They are contracted directly by the Treasury, not the agency, and are restricted from providing any services to the project after the review. This independence means that review teams can offer impartial advice.

5.32
One project we looked at had been fast-tracked, and the project sponsor had been concerned that he might have missed something important. The review team's independence brought a more objective view, and this was able to reassure the project sponsor that he had not missed anything critical.

5.33
For another project that was making slow progress, the independent review team was able to see that the agency was working in silos when people within the agency could not see this themselves. Having this view expressed by an independent team meant that it was more easily accepted by the agency, which then committed to resolve the problem.