Part 5: Reviewing and evaluating grant administration
5.1
In this Part, we discuss those aspects of reviewing and evaluating that the Department has done well. It has:
Summary of our findings
5.2
The Department has performed or commissioned many reviews of its grant administration processes and systems. They include its Business Transformation Project and an evaluation of COGS. The interdependence of the various improvement initiatives has been recognised, and the work has been well aligned and logically planned.
Business Transformation Project
5.3
The Business Transformation Project includes:
- a project to phase in an outcomes framework for the Lottery Grants scheme;
- Project Pai Ake (to find a grant administration database and business system to replace Grants Online); and
- the Business Process Review and contingent projects, including:
- work on value for money;
- website access for applicants;
- risk assessment; and
- a review of forms and documentation.
Outcomes framework for the Lottery Grants scheme
5.4
The Department has been phasing in the Lottery Grants Board's outcomes framework, with Lottery committees adopting "outcomes-focused decision-making" in line with implementing the new grant management system (see paragraph 5.9).13 This requires the committees to base their funding decisions on changes or improvements applicants seek to make that are most likely to benefit the community.
5.5
The Department has noted the need to provide support and training to help applicants to articulate outcomes clearly, in measurable terms that will enable the Department to analyse and evaluate outcomes data in due course.
5.6
We are pleased that the Department is implementing a framework to drive decision-making to achieve specific outcomes, and to measure the benefits of its funding. It is important for the Department and the Lottery Grants Board to track the effectiveness of its funding, for accountability purposes and to target funding wisely in the future. We encourage the Department to consider implementing a similar framework for COGS.
Project Pai Ake – establishing the new grant administration system
5.7
Project Pai Ake is a business initiative to develop and implement the capability to support the Department's grant administration process and management business (as outlined in the Business Process Manual), and community advisory and development activity.
5.8
The associated business case noted that replacing the Department's current electronic grant administration systems, including Grants Online, was needed to avoid continued inefficiency and high costs (because of the inflexibility of the system and its inability to comply with legal and best practice requirements). The benefits of replacing the system include reducing administration costs over time, promoting best practice in the processes used, and compliance with the appropriate financial, audit, and internal control standards.
5.9
The business case for Project Pai Ake was approved in February 2010. It called for $1.5 million of capital expenditure and $283,000 of operating expenditure, for a commercial product that could be tailored to the Department's needs.14 The Department selected a preferred vendor in July 2010, and expects to complete contract negotiations by the end of 2010. The Department's original aim of full implementation by April 2011 will be confirmed or otherwise when these negotiations are completed.
5.10
If the new grant administration system is effectively implemented, it should result in the availability of much better and more readily available information about who has previously received grants, for what purpose, and what the result (or outcome) of those grants has been.
5.11
Appendix 4 sets out more detailed information about the proposed functionality of the Department's new grant administration system.
The Business Process Review
5.12
The Department conducted a comprehensive review of its business processes from 2007 to 2009. The first part of the process was a Better Funding Practice project, which established the Business Process Manual. It also provided the framework for reviewing all of the steps in the grant administration process, and aligning that process with operational policy. The work also informed the design of the new business system, to ensure that it would include the capability and functions to support the Business Process Manual.
5.13
The Department has continued to work on improving the grant administration process, and on related projects, to be incorporated into the new business system. These include the risk assessment tool (see paragraph 3.41) and a "forms and applications project". The "forms and applications project" aims to review, analyse, and standardise all forms and documentation for grant recipients, enable information to be retrieved for monitoring, reporting, and analysis, and ensure that the forms and documents are easy to use.
5.14
In our view, the proposed forms are clear, provide good guidance to applicants, and are well designed to capture data useful for reporting. However, we encourage the Department to test thoroughly how understandable the forms are to smaller community groups, particularly immigrant groups who may struggle with English that is not as plainly written as it could be.
Evaluation of COGS is under way
5.15
An "impact evaluation" of COGS was completed in June 2010. The objectives of the evaluation included assessing the effectiveness of changes from a review in 2003/04 and evaluating how well COGS is achieving the desired outcomes in the community. The Department expects that the results of the evaluation will be made public in December 2010, after consultation with the National COGS Committee. We were told that the Minister for the Community and Voluntary Sector will present a report of the findings to Cabinet.
5.16
Results of the surveys of Local Distribution Committee members and of staff conducted as part of the evaluation reflected some of the matters we noted during our audit.
13: Beginning with the Lottery National Community Committee, most Lottery committees will be aligned with the framework. Exceptions are Lottery Individuals with Disabilities (to be partly aligned), Lottery Pacific Provider Development Fund (disestablished in 2010), the Significant Projects Fund (not holding a funding round in 2010 or 2011), and the Minister's Discretionary Fund (which is not bound by Lottery Grants Board policy).
14: Department of Internal Affairs (2010), Statement of Intent 2010-13, Wellington, page 3.
page top