Appendix 1: Key findings

Kānoa – Regional Economic Development and Investment Unit rapidly developed advice for Ministers about what funds could be repurposed

Despite challenging circumstances, Kānoa – Regional Economic Development and Investment Unit (Kānoa-RDU) worked well to prepare timely advice for Cabinet and Regional Economic Development Ministers³ to repurpose funding from Provincial Growth Fund (PGF) projects to the PGF reset. This work was successfully completed in about six weeks (see Appendix 2 for the timeline).

Kānoa-RDU prepared advice for repurposing funding based on an urgent Ministerial directive. This included identifying where funds could be repurposed and exploring options for adjusting existing funding commitments to new objectives.

Kānoa-RDU categorised existing PGF projects based on their level of progress. It considered factors such as whether the project was viable under market conditions caused by Covid-19 (see Appendix 2 for more detail). Kānoa-RDU then discussed the status of projects with people who had funding approved or existing contracts. Kānoa-RDU used the information it collected to provide Cabinet with a longlist of projects that could have their funding repurposed.

Kānoa-RDU engaged effectively with people who had projects funded or existing contracts that were varied. Although this was done rapidly, we saw that care was taken to engage with people and it was appropriately documented. Kānoa-RDU acknowledged that communication was not always perfect, but said that its general communications approach served them well.

In our view, Kānoa-RDU’s work to provide advice on funds to repurpose and make available for the PGF reset was well managed.

Kānoa – Regional Economic Development and Investment Unit's performance of its due diligence of applicants has improved

Due diligence validates aspects of funding applications and provides a level of assurance about the risks to an investment. The due diligence checks that we looked at are those more able to be factually verified separately from the more judgement-based assessment of an application’s strengths and weaknesses. For the new projects in our sample, we looked at Kānoa-RDU’s due diligence checks of applicants (for example, Companies Office checks, insolvency register checks, and Serious Fraud Office checks).

In our 2020 report Managing the Provincial Growth Fund, we said that we expected due diligence processes to take place before funding commitments were made or project applications approved. We also said that processes needed improving. At that time, we found that there was not enough evidence about how, when, and where Kānoa-RDU addressed questions raised during due diligence processes. We suggested that Kānoa-RDU systematically track matters raised that needed to be followed up. In our report, we noted that Kānoa-RDU had started work to respond to recommendations that we and others had made about due diligence processes.

For the projects that we looked at as part of our work on the PGF reset, we found that due diligence had been carried out and documented as appropriate. Where issues were raised, it was clear how these were to be addressed. This is an improvement since our 2020 report.

We expected to see more attention to risk identification and management

Because PGF reset projects needed to progress quickly, we expected a strong focus on risk identification and management. This means we expected to see evidence of:

• proactive identification and assessment of potential risks to the specific objectives of the PGF reset;
• Kānoa-RDU setting out how it will treat those risks; and
• regular monitoring, reviewing and reporting on those risks.

We saw some evidence that risks related to the implementation of the PGF reset were identified and managed. We also saw some risks identified and managed at an individual project level. However, we did not see evidence that enough consideration was given to identifying, assessing, or managing potential risks to the PGF reset’s overall objectives. When the PGF reset objectives were introduced, the potential for new or emerging risks to those objectives should have been identified more proactively so that potential consequences, such as waste and delay, could have been better understood and planned for. This might have included, for example, considering the potential for supply chain risks that could affect a project starting within six months of the PGF reset (which could affect whether jobs could be created immediately). We also did not see documented evidence that risks to the PGF reset’s objectives were highlighted clearly to Ministers.

We cannot be confident that project applications were adequately assessed against objectives

We expected decision-makers to understand and consistently apply criteria for funding PGF reset projects. In our view, the objectives of the PGF reset and the investment principles created a framework of broad criteria for assessing project applications and making decisions that would have been complicated to apply (see Appendix 2 for a description of the PGF reset objectives and investment principles). Despite this, we expected the framework to be applied consistently and that documentation about project applications would include clear records of how Kānoa-RDU assessed whether the projects’ claimed benefits were considered reasonable and achievable in the time frame indicated.

Documentation from the six projects we looked at in detail (see Appendix 3) showed that the assessment of applications, or consideration of whether applications met all objectives, was inconsistent. There was a specific assessment process for Tier 1 renovation project applications (see Appendix 2 for a description of the tiers). Of the other project applications, we saw variation in their assessment against objectives ranging from all PGF reset objectives (as required by Cabinet), to only the three new objectives, or the original five PGF objectives.

We also looked at how applicants’ claims for their projects were verified. This included seeking evidence of assessors considering whether a project’s expected benefits were in line with the PGF reset objectives (such as jobs created and other stated benefits to regional economies). We would have expected explicit documentary evidence of how these claims were scrutinised rather than staff only noting that the claimed benefits were assessed as reasonable and progressing the application to decision-makers. We found that these assessments were not well documented for most of the projects we looked at. Given that the key purpose of the PGF reset was to reprioritise funding to projects that would contribute to achieving its objectives, failing to document the verification of claims made by applicants could undermine the public and Parliament’s confidence in the robustness of the decision-making process. We described a similar finding in our 2020 report on the PGF.

In our view, better guidance was required to assist assessors in ensuring that assessments of applications cover all objectives, including verifying the benefits claimed by applicants, and documenting how the claimed benefits were verified.

The evidence we saw does not give us confidence that assessors routinely and adequately considered whether projects would deliver against the PGF reset’s objectives or if they appropriately factored this into their advice to decision-makers. Given funding was repurposed from previously approved projects, we would expect to see sufficient assurance provided to decision-makers that new projects would meet the objectives of the PGF reset.

More attention to monitoring and evaluation of the Provincial Growth Fund is required

We expected that Kānoa-RDU would be well prepared to monitor and regularly report on the PGF reset’s progress towards its objectives. However, it is not clear to us how Kānoa-RDU is doing this or planning to do this.

We did note that some efforts have been made to monitor and report on projects funded by the PGF reset. These included:

• The individual projects that we looked at were required to report progress against expected deliverables, and we saw evidence of this happening.
• Progress reports for Kānoa-RDU’s wider work programme, including PGF reset projects, were provided to the Cabinet Economic Development Committee and the Minister of Regional Economic Development.
• Some information on the status of all projects across all funds managed by Kānoa-RDU is available on the Grow Regions website (on an Excel spreadsheet that was last updated on 31 March 2023 at the time of writing).
• Brief reports were provided to Ministers and media releases were published on the Grow Regions website about individual completed projects funded by the PGF reset.

However, despite these efforts, we saw no evidence of clear reporting on how well the PGF reset was achieving its objectives or how the PGF reset’s overall success or value for money could be determined. There was a lack of clear data available and a lack of clarity about what constitutes success, including evidence of targets for the achievement of objectives. This should have been anticipated and been built into the design of the PGF reset. For example, Kānoa-RDU reported “jobs created”, and told us that data on job creation is point-in-time employment, based on information collected from funding recipients. In our view, point-in-time employment recorded at different times helps demonstrate how many people are employed at a given time but does not provide clear information on the overall quantum of employment that the PGF reset investments have created. The current information makes it difficult to determine whether the actual number of reported jobs created was achieving the objective.

In our view, the lack of clear data and definitions of success is in part a result of the decision to use the PGF reset objectives and investment principles as a framework of criteria for decision-making. In our 2020 report on the PGF, we said that the way the fund was designed and set up, including the use of broad criteria, created challenges for its implementation. In the case of the PGF reset, it also created challenges for providing clear reporting on how well objectives (such as immediate job creation, as discussed above) are being achieved.

We also did not see evidence of planning for an evaluation of the outcomes of the PGF reset.⁴ Kānoa-RDU told us that it might, in the future, consider a specific evaluation of the PGF reset. However, it said it would be difficult to establish how the PGF reset was affecting broader regional outcomes when other programmes of funding were also providing significant fiscal stimulus in the regions at the time. We acknowledge this and other challenges that need to be overcome to evaluate the contribution of individual projects towards regional outcomes, including as part of large funding programmes such as the PGF reset. However, in our view, information that is not part of project monitoring could be collected and analysed to better understand the impact these projects are having on communities and regions.

Kānoa-RDU told us it has recently commissioned four impact studies of the Regional Economic Development investment approach to date. We have not reviewed the proposed scope for these studies, but Kānoa-RDU has told us that it will build on the early evaluation of the PGF and help to guide ongoing investment decisions and future evaluations.

In our view, Kānoa-RDU should include the three new objectives of the PGF reset (immediate job creation, projects under way within six months, and visibility to communities) in a wider separate evaluation of the PGF at an appropriate time, noting that the full impact of the PGF (including the PGF reset) projects might not emerge for several years. This will help to show the PGF reset’s progress towards achieving its objectives and to assess its value for money.

Demonstration of important processes and record-keeping was incomplete

We have noted throughout our key findings that there has been a lack of evidence in some parts of Kānoa-RDU’s work that we looked at, as well as a lack of demonstration of some important processes.

Some of these evidence gaps related to the “design” stage of the PGF reset, including a lack of evidence of proactive identification and ongoing management of risks to all objectives (discussed above). There were other gaps related to projects funded by the PGF reset. Because there was a lack of documentation of, for example, aspects of project assessment for the specific projects we looked at, we needed to ask people who were directly involved in those projects. Additional evidence was able to be provided for some aspects, but not for others.

We accept that the amount of time passed since PGF reset funding decisions were made meant that people were less likely to remember details. Creating appropriate records at the time would have mitigated this.

3: Regional Economic Development Ministers are Ministers with delegation to make decisions to fund PGF and PGF reset projects. At the time of the PGF reset, these were four Ministers with five portfolios: the Minister of Finance, Minister of Regional Economic Development, Minister of Economic Development, Minister of Transport, and Minister of Trade and Export Growth.

4: An evaluation of the PGF by Allen + Clarke dated June 2022 did not include projects funded by the PGF reset. It also did not focus on all emergent outcomes related to the PGF objectives, such as environmental sustainability. The evaluators said that more time is needed to fully assess the PGF’s contribution to regional gross domestic product. We agree with the recommendation that Kānoa-RDU should start preparing to evaluate the PGF’s long-term impact.