Part 2: Programmes funded from departmental appropriations
2.1
In this Part, we:
- describe the Kaitātaki-a-Rohe programme and Māori Business Facilitation Service; and
- present our findings from the grants we examined in each of these programmes.
The Kaitātaki-a-Rohe programme
2.2
The Kaitātaki-a-Rohe programme directly supports Māori community
development workers (known as Kaitātaki-a-Rohe) within communities. The
workers:
- are selected from within, work with, and are accountable to, their communities;
- work on community-directed development initiatives; and
- are jointly supported by Te Puni Kōkiri and the community.
2.3
The Kaitātaki-a-Rohe programme awards contracts that typically run for a
two-year period. Te Puni Kōkiri provides funding to an entity of $75,000 for
each Kaitātaki-a-Rohe each year. This funding helps cover salary costs, car hire,
telephone rental, computer leasing, and other administrative costs.
Our audit sample
2.4
We selected a sample of 11 grants, which accounted for 14% of the grants under
this programme for 2004/05 to 2005/06. Figure 3 summarises our audit sample.
Figure 3
Our sample of Kaitātaki-a-Rohe grants
| 2004/05-2005/06 | |
|---|---|
| Number of grants examined | 11 |
| Percentage of all grants in the programme | 14% |
| Total value of grants examined | $1.9m |
| Percentage of all grants in the programme | 20% |
| Range in value of individual grants examined | $150,000-$225,000 |
Ministerial criteria for the Kaitātaki-a-Rohe programme
2.5
The Kaitātaki-a-Rohe programme started in July 2002. It built on Te Puni Kōkiri’s
existing development programmes. The Ministerial criteria for the programme is
to provide human resources for a two- or three-year period to support:
- community-owned and community-led initiatives responding to local issues; and
- targeted initiatives responding directly to the needs of whānau, hapū, iwi, and Māori.
2.6
All the grants in our sample complied with the Ministerial criteria.
The approval process
2.7
Applications for Kaitātaki-a-Rohe funding are submitted to the relevant regional
office of Te Puni Kōkiri. Kaiwhakarite (fieldworkers) in the regional office assess the
suitability of the applications.
2.8
Applications are supposed to include a job description for the Kaitātaki-a-Rohe
position, and details of the planned recruitment process. In the sample of grant
files we examined, three of the 11 did not have the job description or details of
the recruitment process on file.
2.9
The Regional Director then reviews the Kaiwhakarite’s assessment and
recommendations. Applications assessed after November 2005 might also get
reviewed and assessed by a Contracts Advisor in the regional office. This role
was introduced by Te Puni Kōkiri from November 2005 to perform contract
management and provide an extra level of quality assurance with application
assessments and reviews.
2.10
Applications recommended by the regional office for funding are then submitted
to the national office for Te Puni Kōkiri’s Executive Committee to consider.1
2.11
Appendix 1 summarises the approval process for the Kaitātaki-a-Rohe and
Capacity Building programmes.
2.12
Te Puni Kōkiri’s Operations Manual sets out the criteria and steps staff should use
to assess Kaitātaki-a-Rohe applications. Some of the grants in our sample were
awarded before the Operations Manual was introduced in November 2005, and
we took that into account when we carried out our analysis. Figure 4 summarises
the criteria and assessment steps, and examples of key elements they should
cover.
Figure 4
Assessment criteria for Kaitātaki-a-Rohe applications
| Assessment criteria/step | Key elements to be assessed |
|---|---|
| Legal status | Checks on status of applicant organisation, including, for example, checks of:
|
| Financial management capability | Assessment of financial capability of applicant, including examination of such things as:
|
| Governance and management capability | Assessment of a clear separation between governance and management. Includes assessment of such details as:
|
| Funding history | Examination of previous investment and funding in the applicant organisation to avoid duplication and also to help assess performance ability of the applicant. |
| Conflicts of interest | Identification of any potential conflicts of interest for either Te Puni Kōkiri employees or members of the applicant organisation. |
| Proposal assessment | Assessment of proposed projects to ensure that:
|
| Regional assessment | Regional Directors review proposal assessments, including checking that they support the programme’s aims. From November 2005, a Contracts Advisor in regional offices might also perform a quality assurance function, and prepare an appraisal summary of proposals. |
| National office strategic assessment | National office staff review the proposal and regional office recommendation, checking that all relevant documentation has been obtained. A strategic appraisal should be done to ensure national consistency of the application process, and alignment with Te Puni Kōkiri’s strategies. |
Source: Te Puni Kōkiri’s Operations Manual.
2.13
Figure 5 shows the results of our examination of Kaitātaki-a-Rohe grant files
against these assessment criteria and steps.
2.14
We have used “satisfactory” to indicate where there was enough evidence on
file to determine that the criteria had been met, and “unsatisfactory” to indicate
where such evidence was lacking.
Figure 5
Kaitātaki-a-Rohe applications: adequacy of the documentation to support Te Puni
Kōkiri’s assessments against the programme criteria
| Programme criteria | Number of files | |
|---|---|---|
| Satisfactory | Unsatisfactory | |
| Legal status | 11 | 0 |
| Financial management capability | 1 | 10 |
| Governance and management capability | 5 | 6 |
| Funding history | 2 | 9 |
| Conflicts of interest | 0 | 11 |
| Proposal assessment | 11 | 0 |
| Regional assessment* | 8 | 1 |
| National office strategic assessment | 0 | 11 |
*Applicable to only nine files.
2.15
In the sample of grants we examined, there was not enough evidence on file to
thoroughly assess all the funding criteria required about the capability of the
organisation and the application it submitted.
2.16
The following areas were of specific concern:
- For most of the grants we reviewed, there was no evidence in the file of a financial assessment, although all the files included copies of the most recent audited financial statements.
- There was little evidence of the funding history of applicants. Where there were details of previous funding, there was no information to indicate whether previous funding was managed well, whether budgets were met, whether objectives were achieved, or whether projects were completed on time.
- There was no evidence to indicate that conflicts of interest had been considered. Te Puni Kōkiri relies on the Kaiwhakarite’s and Regional Director’s knowledge of the local community to ensure that any conflicts are noted and managed, but there was little supporting evidence in the files.
- There was no evidence of the national office strategic assessment in the regional files, but the Executive Committee had approved all the applications that we reviewed. Where the Executive Committee initially rejected an application, it was returned to the regional office for further work and resubmission. We found no evidence of applications that did not eventually result in funding.
| Recommendation 1 |
|---|
| We recommend that Te Puni Kōkiri create a template to record all the information required to thoroughly assess an organisation’s capability to administer the funding for Kaitātaki-a-Rohe positions. |
Documentation
2.17
All the grants we examined were supported by an application, a completed
assessment form, a signed contract, and clear payment schedules.
Monitoring the Kaitātaki-a-Rohe programme
Progress reporting
2.18
Payments were spread across the period of the contract, usually in quarterly
instalments. There was an expectation that the payments would be made once
a formal quarterly report was received. Quarterly reports were assessed by a
Kaiwhakarite.
2.19
We identified some weaknesses with these progress reports in six of the 11 files
that we reviewed:
- It was unclear whether a Kaiwhakarite had assessed the adequacy of the progress report.
- Progress reports did not report clearly on all the objectives of the role, or reported on objectives that were not part of the contract.
- There was little evidence that actual expenditure on salary, car hire, and computer leasing was monitored.
2.20
In one of these six files there were no quarterly progress reports.
Monitoring visits
2.21
There was documented evidence of regular monitoring for about half of the
sample of files we examined. For the other files, there was no documented
evidence to indicate that Te Puni Kōkiri staff had carried out regular monitoring
visits.
| Recommendation 2 |
|---|
| We recommend that Te Puni Kōkiri monitor progress toward the outcomes for Kaitātaki-a-Rohe defined in the contract, document the regular monitoring visits, and clearly describe actual progress against milestones. |
Evaluating the effectiveness of the Kaitātaki-a-Rohe programme
2.22
In our sample of grant files, most of the projects involved were due to finish
in 2007. Where they were already complete (for 4 out of 11 files), we found no
evidence that individual projects had been evaluated.
| Recommendation 3 |
|---|
| We recommend that Te Puni Kōkiri set up a system to evaluate the effectiveness of individual Kaitātaki-a-Rohe grants, ensuring that evaluations are appropriate for the size and complexity of projects, and assess whether project objectives have been met. |
2.23
Te Puni Kōkiri told us that it evaluated the Kaitātaki-a-Rohe programme in
June 2004, and that the evaluation provided information on the programme’s
achievements and information useful to policy development. We understand that
the programme will be evaluated again when the individual projects are complete.
Māori Business Facilitation Service
2.24
The Māori Business Facilitation Service is a business-mentoring service that
provides advice, guidance, and business facilitation services essential to
building new or existing Māori businesses to increase Māori participation in the
commercial sector. There were 12 organisations contracted to Te Puni Kōkiri during
the period from July 2004 to June 2006 to provide this service throughout the
country. Regionally-based Accredited Business Mentors (business mentors) deliver
the service free to individuals and businesses.
2.25
This service started in 2000. Te Puni Kōkiri carried out a formal tender process in
2003 to re-tender the business mentor contracts for the following three years. The successful contractors received an annual retainer of between $40,000 and
$80,000 that was based on the geographical area and the market they covered. They received additional commission payments of between $250 and $900 as
individual businesses reached milestones within the business mentoring process
(for example, completing a business plan).
2.26
As we mentioned in Part 1, Te Puni Kōkiri does not consider the Māori Business
Facilitation Service to be a grant programme. We chose to examine it with the
other programmes because it funds a service which is free to the recipient, and it
receives significant funding for business mentoring support. In addition, business
mentors apply by submitting a tender for funding to deliver the service, Te Puni
Kōkiri monitors the delivery of the service, and the programme is evaluated. In
effect, the programme is administered in a similar way to Te Puni Kōkiri’s other
grant programmes.
Our audit sample
2.27
We selected a sample of half the contracts in effect between 1 July 2004 and
30 June 2006. This covered more than 75% of the funding provided during that
period. Figure 6 summarises our audit sample.
Figure 6
Our sample of Māori Business Facilitation Service contracts
| 2004/05-2005/06 | |
|---|---|
| Number of contracts examined | 6 |
| Percentage of all contracts in the programme | 50% |
| Total value of contracts examined | $3.3m |
| Percentage of all contracts in the programme | 77% |
| Range in value of individual contracts examined | $346,000-$1m |
Ministerial criteria for the Māori Business Facilitation Service
2.28
Cabinet agreed that the Māori Business Facilitation Service would assist small to
medium-sized Māori businesses with a comprehensive range of business services. These services were:
- pre-commercial facilitation;
- facilitating access to grants/finance;
- post-commercial assistance; and
- promoting an enterprise culture by assisting Māori business networks.2
2.29
Cabinet noted particularly that:
- the Māori Business Facilitation Service tendering process for potential providers will include contractual requirements that ensure that they can provide a quality service for Māori women; and
- the Māori Business Facilitation Service Management Team will:
- develop a database on the participation rate of Māori women within the Māori Business Facilitation Service;
- promote and engage Māori women in regional focus groups; and
- develop strategies to increase networking opportunities for Māori business women.
2.30
The Māori Business Facilitation Service was structured around a business
framework split into eight phases, from assessing the business idea through to
business expansion.
2.31
The 2003 Request for Proposal document included details of the range of
business services to be provided, and requested detailed information about the
skills and expertise of the proposed business mentors. It did not include any
specific requirements to provide a service for Māori women, although our review
of the service delivery and statistics provided by Te Puni Kōkiri indicated that
Māori women were involved in almost half of the businesses that received the
mentoring service. In addition, Te Puni Kōkiri told us that more than half of its
account managers for the Māori Business Facilitation Service were Māori women.
The approval process
2.32
The 2003 Request for Proposal included details of how Te Puni Kōkiri would
evaluate the tenders to select contractors to provide a comprehensive service to
new and existing Māori businesses throughout the country.
2.33
Te Puni Kōkiri assessed the tenders by reviewing the supporting documentation
submitted with the tenders and interviewing those who submitted tenders. To
award the funding, Te Puni Kōkiri then prepared contracts with the 12 who were
successful in their tender bid.
2.34
We reviewed the tenders for six of the 12 contracts, and compared the content
with the requirements of the Request for Proposal.
2.35
All the successful tenders complied with the information requirements. However,
the information requested focused on the skills and expertise expected, rather
than how the Māori Business Facilitation Service objectives were to be delivered. This made it difficult to link delivery of the contract back to meeting the objectives
of the Māori Business Facilitation Service. In our view, monitoring and evaluation
would be easier if contractors had clearly set out how they would meet the
objectives of the Māori Business Facilitation Service.
| Recommendation 4 |
|---|
| We recommend that Te Puni Kōkiri ensure that future Requests for Proposals require those who submitted tenders to set out how they will meet the objectives of the Māori Business Facilitation Service. |
Documentation
2.36
Electronic records for the Māori Business Facilitation Service were stored in a
database. The database recorded the progress of each business involved in the
mentoring programme. Business mentors in the regions and staff in the national
office could access the database, and it was an effective way to share information.
2.37
Reports from the database were also used to support the payment of
commissions. Copies of the reports were held in paper files, together with copies
of the contract.
2.38
We found some errors in the administration of these contracts and the associated
payments, including:
- invoices that did not add up correctly were authorised for payment;
- errors on invoices were noted after authorisation and it was unclear whether they had been resolved;
- commission payments were authorised that were not supported by the required reports from Te Puni Kōkiri’s database; and
- commissions were claimed and paid for services that were not specified in the contract (although they were relevant to business mentoring).
| Recommendation 5 |
|---|
| We recommend that Te Puni Kōkiri review its administrative arrangements for the Māori Business Facilitation Service, and ensure that staff are aware of their responsibilities for checking that invoices are correct and comply with the requirements of contracts. |
Monitoring the Māori Business Facilitation Service
2.39
Te Puni Kōkiri account managers are responsible for contracts and monitor the
Māori Business Facilitation Service on an ongoing basis, including signing off
that business mentors have completed each milestone satisfactorily. Account
managers also deliver the first three phases of the business framework, and
then refer the businesses on to the business mentors. There is regular contact
between account managers and business mentors, but much of the monitoring
undertaken by Te Puni Kōkiri is not documented. Te Puni Kōkiri showed us annual
and six-monthly contract reviews, but did not have documentation to support the
ongoing weekly and monthly liaison with business mentors.
2.40
For example, one of the performance measures was meeting a client satisfaction
level of 95%. The business mentor’s quarterly reports included confirmation that they had reached the required satisfaction level. This was supported by the
account manager’s knowledge of the service provided, rather than a formal client
survey, so there was no supporting documentation.
2.41
Evidence of contract management was limited for the six contracts we examined. We noted that:
- One contractor had delivered more than the contract maximum on some elements of the contract, and was paid an additional $99,000 during the two years of the contract. The contract variation to support this additional funding was retrospective, and was signed after the contract term was complete and all the payments had been made. We were told by staff that the administration of the contract variation began in May 2006 when the contract still had a month to run. Documentation to demonstrate the ongoing management of this contractor’s performance was limited.
- In 2004/05, there was under-expenditure (that is, the contractors delivered fewer services and were paid less money by Te Puni Kōkiri than expected) of $306,000 in total for five contracts, and $238,000 in total for four contracts in 2005/06.
- Three contractors delivered less than 55% of the business services required of them in 2004/05 (the first year of the contract). Te Puni Kōkiri staff told us that the performance of these contractors was managed, but there was limited documentation to support this.
- In the second year of these contracts (2005/06), the performance of one contractor improved to deliver 65% of the contract, and another contractor improved to deliver 54% of their contract. The third contractor’s performance was worse in 2005/06, with 38% of the contract delivered.
- We noted that Te Puni Kōkiri continued to make the retainer payments throughout the contract period (in total, $300,000 during the two-year period) for these three particular contractors.
2.42
Te Puni Kōkiri’s view is that the performance noted does not equate to poor
performance because it does not consider all the contract indicators to be key
performance criteria.
| Recommendation 6 |
|---|
| We recommend that Te Puni Kōkiri regularly monitor the performance of Māori Business Facilitation Service contractors, document the monitoring, and identify where contract variation needs to be considered if the demand for mentoring services is greater or less than expected. |
1: Executive Committee membership might vary occasionally, but usually comprises all Regional Directors, the Deputy Secretary – Relationships and Information, and representatives from some national office business units.
2: Implementation of the Māori Business Facilitation Service, Cabinet Committee on Closing the Gaps (GAP (00) 43 dated 3 July 2000).
3: Implementation of the Māori Business Facilitation Service, Cabinet Committee on Closing the Gaps (GAP (00) 43 dated 3 July 2000).
page top