Part 4: Risk assessment and monitoring

4.1
In this Part, we discuss our findings on NZTE’s:

• risk assessment of grant
• applicants; and
• monitoring of grants.

Risk assessment

Overall finding from 2004 report

4.2
In 2004, NZTE did not have a consistent approach to assessing risk for its grant programmes. The GSF programme had a comprehensive approach to risk assessment, but the other grant programmes varied. This included the SIF programme, where there was no formal risk profiling of grant applicants.

Our expectations

4.3
We expected NZTE to have:

• set up a risk-based approach to help assess grant applicants; and
• comprehensive guidance for assessing the risk profile of applicants for grants.

Our findings

Programme-level risk profiling

4.4
NZTE has designed and introduced detailed risk profiles for all grant programmes as part of the business process improvement project. These risk profiles aim to identify all risks for each grant programme (such as operational, financial, reputation, and political) to ensure that the business processes and documentation cover those risks. The profiles assign risk ratings to the probability and effect of each identified risk, outline mitigation options, and link to the processes or documentation that address each risk. In our view, this is a rigorous and comprehensive way to mitigate the risks associated with grant programmes.

Identified operational and reputation risk with the EDG-MD programme

4.5
We identified an operational risk with the EDG-MD programme that might also have implications for NZTE’s reputation. The 2007 Budget increased funding for this programme from $36.3 million for 2006/07 to $51.4 million for 2007/08. This increase in EDG-MD programme funding was not accompanied by an increase in operational funding for grant administration.

4.6
The EDG-MD programme is now the largest grant programme NZTE administers, and there has been a substantial increase in applicants to it. This has created a significant backlog with processing applications. To reduce this backlog, NZTE altered its application and assessment processes in late 2007. Previously, the EDG-MD grant process required applicants to specify the activities for which they sought market development funding, and administrators would assess the eligibility of each activity before approving grants. Applicants are now given a detailed schedule of eligible activities and costs for EDG-MD funding, and their eligibility is not assessed until claims are lodged.

4.7
In our view, while this process change suitably reduces the time taken to process applications, it could transfer the backlog to the claims stage for EDG-MD recipients in 2008. NZTE should monitor the risk of a backlog to processing claims and, if necessary, consider appropriate measures to counter its effect.

Risk assessment of grant applicants from our audit sample

4.8
The GSF, SIF, and EDG-MD programmes have processes in place for assessing the risk applicants pose. These included routine background checks of the legal status of applicants, as well as scrutiny of financial background and credit checks.

4.9
For the GSF programme, members of an assessment panel each complete an assessment sheet to assign risk ratings to a range of variables. An overall risk rating score is then produced to guide the panel with its assessment decision, or to seek further information from applicants. This system is comprehensive and appears to work well.

4.10
Individual assessors complete risk assessments for the EDG-MD and SIF programmes to build a recipient risk profile. The higher the risk rating, the more analysis or scrutiny of applications is expected.

4.11
In our view, guidance for staff on assessing risks and interpreting their potential effect could be improved. Except for the GSF panel, whose approach allows risk rating scores to be compared, we were unable to assess the consistency of risk assessments. We note that this finding was also made by a recent NZTE internal audit that reviewed the grant programmes we examined in 2004. NZTE management’s response to this internal audit finding noted that a risk management framework has recently been created that will be used for refining risk assessment tools in individual grant programmes.

Monitoring

Overall finding from 2004 report

4.12
In 2004, the monitoring of grant recipients and the collection of monitoring information was inconsistent in all the grant programmes we examined. This included cases where required monitoring reports from grant recipients were either not received or only partially completed. In some cases, we were unable to tell whether contact had been maintained with recipients after a grant had been approved.

Our expectations

4.13
We expected:

• all grant programmes to have been reviewed to ensure that monitoring information is collected consistently;
• monitoring activities to be tailored to take account of the risk profile of grant recipients; and
• grant recipients to submit completion reports before final funding instalments are paid.

Our findings

Framework requirements for monitoring and reporting

4.14
NZTE has reviewed reporting and monitoring requirements as part of its wider business process improvement review. The grant administration framework states that the main objective of monitoring is to identify risks and barriers to progress and to ensure that funding is spent for approved purposes. Monitoring must be active and documented. The recommended recording method is using Pivotal. For reporting, the framework stipulates that grant recipients must submit progress and final reports for funded projects. Reporting should show progress against pre-determined project milestones.

4.15
The operating guidelines for the three grant programmes we audited all reiterate the framework’s basic requirements for monitoring and reporting. All three grant programmes require monitoring to identify risks and barriers to progress of funded projects. Furthermore, grant monitoring must be active. Individual grant contracts set out specific reporting requirements. These normally require GSF and SIF grant recipients to submit either quarterly or monthly progress reports. EDGMD recipients were required to submit progress reports at months six and 10 of the funding year.

Monitoring and reporting of grants in our audit sample

4.16
All 10 of the GSF grants in our sample met the reporting requirements. There was only one case where a final evaluation report (of a funded project) was not received until NZTE staff took follow-up action.

4.17
Half of the GSF grants we examined were actively monitored, as defined by the operating guidelines. For the other half of our sample, there was evidence of some monitoring on file or in Pivotal, but contact frequency was less than the monthly stipulation in the operating guidelines. We note, however, that later revisions of the operating guidelines for the GSF programme have removed the requirement for monthly contact. This is because NZTE is introducing individual client management plans to define specific monitoring requirements for each GSF recipient.

4.18
There was active monitoring of all the grants we sampled from the EDG-MD programme. This was evident from good monitoring notes recorded in Pivotal. Two grants out of the sample of 10 were missing progress reports from the grant files. Given that the electronic client management system is currently under review, NZTE could consider introducing an alert system to prompt administrators to remind grant recipients to submit progress reports.

4.19
Reporting requirements were only partly met in our sample of SIF grants. In all four SIF grants we examined, actual practice varied from the reporting requirements agreed in the funding contracts. The operating guidelines do provide NZTE with discretion to waive specific deliverable or milestone requirements (such as providing a progress report). However, we did not see documented evidence of waivers being used. There were also some significant delays between contractual deadlines for progress reports to be submitted and follow-up by grant administration staff.

4.20
It was also difficult to assess whether the grants in our SIF sample had been actively monitored, as defined by their operating guidelines. The guidelines require at least monthly monitoring contact, with records kept of minutes of progress meetings or email contact. Although we found some evidence of monitoring contact with grant recipients, this was usually substantially less often than monthly and not always about the specific grant. NZTE could make greater use of Pivotal to record regular monitoring activity of SIF grants. This could include ensuring that the grant administration framework’s recommendation that Pivotal contain enough information for any NZTE staff member to learn enough about a recipient to hold a constructive meeting with them is met.

Withholding final grant instalment payments if recipient reporting requirements are not met

4.21
In 2004, we recommended that a proportion of grants be withheld until the grant recipient provides a final report when a funded project is completed. Provision to do this is now built into the operating guidelines of all three grant programmes we audited.