Auditor-General's overview
Government agencies manage significant assets that are critical for providing public services. Every year, those agencies invest millions of dollars in projects and programmes to acquire or develop new assets, so that they can continue to provide the public services New Zealanders need. The public needs to have confidence that these projects and programmes are using money in the right way and on the right things, and that the benefits that the agencies have undertaken to deliver will be realised.
Gateway reviews are a peer review process designed to provide independent and timely advice to the people responsible for major projects and programmes. The reviews take place at key stages in the project or programme's life cycle. Gateway reviews make recommendations to help projects and programmes succeed. They do not pass or fail a project and, despite their name, are not designed to stop a project or programme that is not going well. There are 25-30 reviews carried out each year with a total annual cost of about $2 million.
Since their introduction in 2008, there have been over 180 reviews of more than 80 projects and programmes involving about 40 agencies and with a total value of about $45 billion. Projects and programmes that have had Gateway reviews include the Transmission Gully project, a patient information system for South Island district health boards, the refurbishment of Government House, and the Inland Revenue Department's Business Transformation programme. These are some examples of important asset developments for New Zealand's future.
My staff looked at whether Gateway reviews have been designed and implemented to deliver benefits to major projects and programmes in the public sector.
In my view, Gateway reviews have delivered benefits to individual projects and programmes. Project sponsors have reported that they find Gateway reviews beneficial. My staff saw specific examples where a Gateway review had provided advice that helped a project reach an important milestone. For example, a Gateway review was able to help a project reach agreement between stakeholders who were in conflict. Without this agreement, the project would not have been able to progress.
For many projects and programmes, specific benefits, including cost savings, are harder to identify. There are many other influences on the success of projects and programmes, and the impact of Gateway reviews can be difficult to isolate. A cost-benefit analysis of Gateway reviews in the United Kingdom found that projects that had reviews experienced cost savings of 2-4% of the total project costs. This type of analysis is very complex and has not been replicated in New Zealand.
Gateway reviews can be effective only if they are applied to projects and programmes where they can make a difference. All projects and programmes considered to be high-risk, based on common causes of project failure, are required to have Gateway reviews. The Treasury has effective processes to identify high-risk projects and programmes and make sure that they do actually have Gateway reviews at the right times.
Having the right review team is also critical to the success of Gateway reviews. There is a large pool of reviewers to choose from, and the Treasury puts a lot of effort into getting a review team with the best mix of skills and experience for each project or programme. This is usually done well.
When review team members come from government agencies themselves, they can take what they have learned from a review and apply it to their own agency. Opportunities for this can be limited, however, as it is hard for senior public servants to dedicate a week to a Gateway review, and currently only about 20% of reviewers come from the New Zealand public service.
Gateway reviews can have a wider impact than just the projects and programmes that are reviewed. Some lessons from Gateway reviews are already shared, but I consider that this could be done more effectively. I would like to see the Treasury share more examples of good practice and find ways to help make sure that lessons are put into practice in other projects and programmes. In this way, such reviews could have a broader beneficial effect for the public sector as a whole.
When Gateway reviews were introduced to New Zealand, they had already been running successfully overseas, and it was reasonable to assume that they would benefit projects and programmes here. However, there was not a strong case to show how Gateway reviews would specifically meet the needs of New Zealand projects and programmes.
Gateway reviews do have limitations. They are a "one size fits all" approach and are unlikely to be the best way of helping all projects and programmes at all times.
The Treasury is aware of the limitations of Gateway reviews and is planning to introduce new types of investment reviews to run as well as, or instead of, Gateway reviews. I would expect that, in considering the types of investment reviews it offers, the Treasury would ensure that it takes into account the limitations of Gateway reviews, the specific needs of projects and programmes in New Zealand, and the information and assurance needs of major stakeholders.
I thank the Treasury for co-operating with our audit. I also thank other people we spoke to, including agency staff, Gateway reviewers, and people from other parts of the assurance system, who all kindly shared their experiences of Gateway reviews.
Lyn Provost
Controller and Auditor-General
15 September 2016