Part 5: Funding, prioritisation, and value for money

Ministry of Education: Supporting professional development for teachers.

5.1
In this Part, we outline our expectations of the Ministry's funding, prioritisation of resources, and value-for-money considerations when administering teacher professional development contracts. We then set out our findings, conclusions, and recommendations for each of these areas.

Our expectations

5.2
We expected the Ministry to promote the efficient funding and delivery of professional development for teachers through:

  • a funding strategy or plan that:
    • aligns funding streams with professional development needs;
    • prioritises professional development so that resources are directed towards the areas of greatest priority and need;
    • encourages appropriate participation in professional development initiatives;
  • the monitoring of funding and services to determine whether it is targeting priority areas and reaching those most in need of professional development; and
  • contract management practices for awarding and monitoring contracts for the professional development of teachers that include:
    • considering value for money when awarding contracts;
    • using efficiency-related performance measures when monitoring contracts.

Summary of our findings

5.3
We did not see strong evidence of a coherent funding strategy that consistently prioritises the supply of professional development funding to the areas of greatest need, although we acknowledge that the Ministry's Schooling Improvement initiatives are specifically targeted at high-needs schools. Neither did we observe regular reviews of the efficiency of providers, or the consistent use of efficiency-related performance measures for the professional development contracts we looked at.

5.4 We identified that:

  • the Ministry did not have a systematic approach to setting priorities across the full range of professional development initiatives over which the Ministry has some control or influence;
  • the Ministry has only limited documented evidence of reviewing providers' value for money when renewing contracts, even though such reviews are a requirement of the Ministry's contracting management guidelines;
  • some milestone reporting appears to have potentially high compliance costs relative to the benefits gained from the reports; and
  • the Ministry makes only limited use of claw-backs (the recovery of funds for undelivered services) where this is permitted in contracts with providers of professional development for teachers, although in some cases substitute services were provided.

5.5
However, we have observed that the Ministry has made recent efforts to improve its work to support the efficiency of professional development initiatives for teachers. Examples include reviews of the information that the Ministry collects from providers, comparisons of some programme costs for each student, and internal discussions about the coherence and alignment within and across initiatives. We support these efforts and encourage the Ministry to do more of them across the range of its professional development initiatives for teachers.

Funding

Strategy and planning

5.6
Beyond the strategic information in its Statement of Intent, we did not see evidence of a strategy or plan covering the whole of the Ministry’s teacher professional development funding activities in one document, although strategy documents exist for some initiatives.

5.7
Without clear direction from the Ministry, it is difficult for providers to plan their services, and for schools to plan and prioritise their professional development needs. Without assurance that initiatives will be offered in the long term, schools may over-commit to initiatives given the uncertainty about when the initiatives will be available in the future.

Funding levels

5.8
The Ministry has conducted some analysis identifying the cost for each teacher of specific professional development initiatives1 that have been shown to be effective in raising student outcomes. The cost of these initiatives ranged from $1,624 to $4,300 for each teacher. The Ministry estimates that if each of "approximately 50,000" teachers were to access one programme a year at a value of $3,500, the annual cost for these teachers would be about $175 million.2

5.9
We support the Ministry analysing the cost per teacher to help inform decisions about the adequacy of the overall level of funding allocated to professional development. We acknowledge, however, that there are many assumptions that need to be made and tested before definitive conclusions can be reached on the appropriateness of current funding levels relative to the quoted $175 million annual cost. This includes assessing the overall level of need for professional development services.

5.10
A challenge the Ministry faces is the need to ensure that the professional development initiatives that it funds will not rely on ongoing resourcing directly from the Ministry. That is, it needs to ensure that the initiatives are sustainable. By sustainable, we mean that teaching practice learnt as part of a professional development initiative continues to be implemented after the Ministry’s funding of the initiative has stopped. As far back as 2005, the Ministry stated that the responsibility for teacher professional development support and funding for its Schooling Improvement programmes is a shared arrangement between the Ministry and schools.

5.11
We consider it is important that the Ministry identify and include all of its spending on professional development for teachers when considering the adequacy of professional development funding levels.

Recommendation 7
We recommend that the Ministry of Education include all if its spending on professional development for teachers when deciding the priority of initiatives to fund, and when considering the adequacy of professional development funding.

5.12
We would consider this recommendation to be implemented if we saw publicly reported documents showing the Ministry considering all of its spending on professional development for teachers when deciding the priority of initiatives to fund, and when considering the adequacy of professional development funding.

Financial incentives

5.13
The Ministry looks for changes in its approach to support professional development for teachers that might present opportunities to alter the mixture of funding arrangements and associated incentives. For example, the Ministry has done some initial work comparing various funding mechanisms to determine whether they support national policy directions and equity among schools. This comparison also identified possible outcomes and risks of the various funding mechanisms, and opportunities to provide incentives or build partnerships among schools.

5.14
Apart from the requirements for renewing a teacher’s practising certificate and salary effects through increased levels of qualification, there are few requirements or direct financial incentives within the current system for individual teachers to participate in professional development.

Prioritisation

5.15
ERO has stated:

Ultimately it is the Crown’s responsibility to ensure that the differing needs and priorities of all those with an interest in in-service training are brought together.3

5.16
This is the Ministry’s responsibility on behalf of the Crown.

5.17
The Ministry decides which areas of professional development to fund for teachers, including which providers to fund.

5.18
Providers decide, within any criteria set by the Ministry, who to provide teacher professional development services to and, to an extent, what specific services to provide and where they are to be provided.

5.19
Schools decide which professional development initiatives they need and which initiatives they will purchase with operational funding.

Ministry-level prioritisation

5.20
Over time the Ministry has considered a variety of factors when making decisions about the priority of various professional development initiatives. These factors have included government priorities, cross-government strategies, Ministry analysis of student achievement and other data, ERO’s findings, national initiatives, and research findings.

5.21
At the time of our audit, the Ministry was working on a systematic approach to prioritising its allocation of funding for the professional development of teachers across various initiatives. The Ministry has advised the Minister that the approach will be based on transparent and evidence-based decisions to prioritise initiatives with strategic importance and effect on the education system.

5.22
In a 2006 internal document,4 the Ministry identified what it believes are important areas of focus to provide successful learning outcomes for students. The areas are:

  • effective teaching in the foundation areas of literacy and numeracy, and development of key competencies through the curriculum learning areas;
  • building teacher capability in assessment and inquiry based approach that enable effective teaching and learning across schooling; and
  • other areas where international and national data has identified the need to improve the quality of teaching and learning outcomes.

5.23
In the same document, the Ministry identified some gaps in its information about the effectiveness of its current arrangements in promoting effective professional development for teachers. We encourage the Ministry to work on reducing these information gaps given the strategic importance of this information.

Provider-level prioritisation

5.24
The Ministry has advised the Minister that providers vary in their “approaches to the identification of, and planning for, locally determined professional development needs”. It has also identified that the effectiveness of professional development can be improved by the “development of mechanisms that enable schools and providers to respond more flexibly to professional development needs”.5

5.25
We identified a range of methods used by the providers we spoke with to determine how they allocate their professional development resources across topics, schools, and teachers. The factors that influenced their allocation decisions included:

  • the Ministry’s contractual requirements;
  • school performance data;
  • school direct requests for services;
  • information and discussions with Ministry regional offices;
  • meetings and correspondence with schools;
  • reviews of teacher recruitment advertisements in the Education Gazette, as an indicator of individual schools’ needs;
  • working with other providers to share knowledge;
  • geographic clustering of schools;
  • school commitment and resourcing; and
  • surveys of teachers’ professional development needs.

5.26
We saw evidence of the Ministry telling at least one provider in 2004 that it needed to be more active in setting priorities for who would receive its services rather than relying on advertising widely and responding to potential recipients who "put their hand up first".

5.27
Some providers of professional development said they have traditionally used demographic data and National Certificate of Educational Achievement (NCEA) results to prioritise among schools when planning where to provide professional development services for teachers.

5.28
We saw evidence of the Ministry’s national office staff , in at least one year, having met with its local and regional office staff and with a School Support Services provider to discuss priorities for allocating funded hours across subject areas, before negotiating the contract with the provider.

School-level prioritisation

5.29
The Ministry set out its expectations of school-level professional development priorities in a 2000 newsletter6 to schools. This stated:

Planning for professional development is informed by the New Zealand Curriculum, school and teacher self-review, principal and teacher appraisals, and analysis of quality assessment information. The school identifies professional development needs, draws up a timeline for meeting these, and evaluates and costs possible professional development options both within and outside the school. As well as particular local needs, the school will need to take account of national priorities, such as implementation of curriculum statements or other government initiatives and the professional development being offered in support of these.

5.30
We have not seen a more recent statement of the Ministry’s expectations of schools for planning and setting priorities for professional development but we understand that the 2000 expectations have not been withdrawn.

5.31
The people we spoke with during our audit indicated that schools differ in how they decide on professional development for teachers. We were told that principals play a critical part in the allocation of professional development funding within schools, and that some schools have professional development committees.

5.32
We were also told that there is a potential for schools to work together more at a regional level.

5.33
School leaders make decisions about how much operational funding they will use for teacher professional development. We were told during our audit fieldwork that a school’s professional development budget may, in some circumstances, be used for other purposes.

5.34
We were also told that schools’ selection of professional development can sometimes be influenced by the availability and central funding of various professional development initiatives more than by identified school needs. ERO is intending to look at school-level decision-making as part of the work we described in paragraph 1.12.

5.35
We have not drawn conclusions on the issues covered in paragraphs 5.33 and 5.34 because school decision-making was outside the scope of our audit.

5.36
Because schools are not required to report the amount of operational funding they spend specifically on professional development, it is difficult for the Ministry to track school spending in this area. However, the Ministry has identified "schools’ relatively low investment in professional development"7 as an issue.

5.37
The Ministry reviewed schools’ operational funding in 2006 and found that "decisions need to be made at both the national level and school level, about whether existing resources are being used as effectively as possible and how to reprioritise resources to get better results".8

5.38
We were told that ERO is doing a national evaluation of professional development and learning. This evaluation will include information about responsiveness to local needs.

Recommendation 8
We recommend that the Ministry of Education reduce the risk of over-commitment by schools to, or waste in the provision of, professional development initiatives that can occur when schools participate in too many or too many similar Ministry-funded initiatives.

5.39
The Ministry has said that it believes there is a significant workload (and therefore cost) involved in implementing this recommendation for all professional development initiatives, given what it might gain from the work. However, the Ministry has indicated that there is merit in developing a system to record information on the schools participating in selected initiatives (a system that Ministry staff and professional development providers could access). The examples of selected initiatives given by the Ministry are Schooling Improvement, e-learning clusters, and Extending High Standards Across Schools.

5.40
We would consider this recommendation to be implemented if we saw evidence of the Ministry designing, monitoring, and managing the application process for initiatives to reduce the risk of over-commitment by schools to professional development initiatives or waste in the provision of professional development initiatives.

Value for money

5.41
The Ministry’s contract management guidelines require contract managers to consider value for money throughout the contracting and procurement process, including when renewing contracts. However, what constitutes value for money for any given professional development initiative is not defined. We have defined value for money as using resources effectively, economically, and without waste, with due regard for the total costs and benefits of an arrangement and its contribution to the outcomes the entity is trying to achieve.

5.42
We have seen evidence that the Ministry has measured the cost of some of its initiatives for each teacher or each student. Such analysis has been limited and has not considered whether the money could have been used more effectively elsewhere (in other schools or other initiatives), or whether greater benefits could have been achieved through a different investment.

5.43
In a wider review looking at $181 million of contracted initiatives, the Ministry identified many teacher professional development initiatives that have good outcomes, are aligned with government objectives, and provide value for money. This included School Support Services.

5.44
One of the challenges the Ministry faces is how to obtain value for money when contracting with other public entities. This is because in some instances factors beyond those related to the cost and quality of outputs, in particular the government’s ownership interest in the entity, may also need to be taken into account.

Obtaining value for money

Contestable contracts

5.45
The Ministry funds and manages a range of public and private providers of professional development services through contestable contracting arrangements. The use of contestable funding is a key mechanism to support contracts delivering value for money.

5.46
A 2006 review9 of the Ministry’s contestable funding, carried out by the Ministry with sector involvement, described the responsibility for managing contestable funds within the Ministry as diffuse and disconnected. A number of operational issues associated with contestable funding have also been identified in a Ministry review of schools’ operational funding.10 These included:

  • difficulty for schools in getting information on initiatives;
  • inter-relationships with other funding streams;
  • compliance costs (associated with winning a contract and reporting against it); and
  • sustainability of a professional development initiative (given funding is for a defined purpose for a defined period).

5.47
Issues with contestable funding for professional development for teachers that were raised with us during our fieldwork were consistent with some of the issues identified in the Ministry’s review of contestable funding.

5.48
The risk of diff use and disconnected funding responsibilities within the Ministry is that some schools may receive more than their appropriate share of professional development funding to the detriment of other schools.

5.49
Changes to the Ministry’s contestable funding process suggested by the contestable funding review included:

  • ... the establishment of a central repository with information about all funds;
  • combining funds where the outcomes sought are similar; and
  • where possible and desirable, alignment of application processes and reporting and accountability requirements.11

5.50
We support the Ministry making these changes to its process.

5.51
The Ministry told us that it has made some progress with respect to the application process for contestable funding pools. In 2007, the Ministry began work on streamlining the application process, which involved restructuring the contestable funding information on the Ministry’s website and developing common application forms for all contestable programmes.

5.52
At the time of our audit, the Ministry was developing a business system to receive applications online and store them in a central database.

Recommendation 9
We recommend that the Ministry of Education include value-for-money considerations when purchasing new, or evaluating existing, professional development initiatives for teachers.

5.53
In its response to the draft of this report, the Ministry noted that it will be asking its staff to review how value-for-money considerations are handled, to review guidelines and templates and their use, and to train Ministry staff .

5.54
We would consider this recommendation to be implemented if we saw documents showing how value for money was considered when the Ministry funded new professional development initiatives, and when it evaluated or commissioned evaluations of professional development initiatives.

Non-contestable contracts

5.55
With non-contestable contracting, competitive tendering processes cannot be used to help ensure a fair market rate. In such situations, we expect the Ministry to pursue other ways of achieving value for money, such as benchmark pricing.

5.56
We have seen evidence of a Ministry review to inform pricing for non-contestable professional development services. This review set benchmarks for "reasonable" personnel and non-personnel costs of School Support Services offered from tertiary education providers. The review was in draft form when we completed this audit, but its purpose is to inform the Ministry’s future School Support Services contract negotiations. We encourage the Ministry to periodically review arrangements and benchmark prices within and across initiatives.

5.57
In 2003, the Ministry began issuing its non-contestable teacher professional development contracts with some tertiary education providers on a rolling three-year cycle. This arrangement provides a greater degree of certainty to these providers compared to an annual contract (for example, for staffing and planning purposes), although the specific allocation of resources across priorities within the contract is negotiated annually.

Monitoring and value for money

5.58
The Ministry has implemented a contracting model involving relatively detailed outputs, clear milestone reporting arrangements, and feedback to providers. This is for both contestable and non-contestable contracts. This approach enables the Ministry to ensure that providers are allocating resources and delivering services in proportion to the priorities and target areas specified within contracts.

5.59
The contract files that we reviewed were readily available and complete, except for the risk information we discussed in Part 4. However, we did not find strong evidence that the Ministry consistently uses efficiency-related performance measures as part of its monitoring of providers’ performance in delivering professional development contracts.

5.60
Milestone reporting and monitoring have transaction costs for the Ministry and providers. This information should be used as efficiently as possible. We observed large and detailed milestone reports being provided to the Ministry for some contracts for professional development for teachers. The longest report we saw was for a large multi-million dollar contract − the report was more than 600 pages in length. Other service providers we spoke with have found reporting requirements to be lengthy and time-consuming, with questionable added value.

5.61
Although, to an extent, contractual requirements influence the volume of reporting material, Ministry and provider judgements about what is appropriate to meet those requirements also influence that volume. The Ministry gave us a copy of the guidelines it has issued to providers regarding an appropriate length and expected content of milestone reports, and the intended audience of the reports. The Ministry also told us that it would support shorter but more useful reports.

5.62
We encourage all parties to carefully consider the relative costs and benefits associated with producing and assessing large documents for milestone reporting. As well as being costly to administer for both provider and funder, such documents also have the potential to “bury” the important messages being conveyed. Large documents also have the potential to focus attention on detailed issues and not the overall progress towards an outcome. This is a point that a former chief executive of the Ministry had previously raised about a specific teacher professional development contract during his tenure as chief executive.

5.63
Large documents also make it difficult to synthesise issues raised in the monitoring information that are common to all providers and initiatives.

5.64
We were encouraged by some evidence of the Ministry having commissioned analysis across the key milestone reporting documents of School Support Services as part of an "ongoing, developmental process that seeks to inform and strengthen" School Support Service milestone reporting.12 The analysis identified the types and quality of evidence in the milestone reports, how the evidence is used by School Support Service providers, and whether providers have improved students’ outcomes.

5.65
As a result of the Ministry-commissioned analysis, the Ministry considers that the following actions will improve the effectiveness of School Support Services. The actions are:

  • continued professional support for providers in developing applied research expertise;
  • increased opportunities for collaboration between providers; and
  • aligned formats for reporting data to enable more credible comparative analysis.

5.66
In our view, these actions also have the potential to get better value for money for the Ministry.

Recommendation 10
We recommend that the Ministry of Education continue to work with the contracted providers of professional development initiatives for teachers to ensure that contract monitoring reports are useful and do not create inappropriate compliance costs for providers or the Ministry.

5.67
In its response to the draft of this report, the Ministry told us that it is "trialling a new contract reporting framework [for School Support Services] in 2008 which will be evaluated in terms of compliance costs and the usefulness of information for improvement and accountability".

5.68
We would consider the recommendation to be implemented if we saw documented evidence of discussions between the Ministry and providers about how to address issues with the usefulness and compliance costs of reporting requirements in contracts.

Claw-back provisions

5.69
A claw-back provision is one mechanism the Ministry can use to help ensure that it gets value for money. Contracts with claw-back provisions allow the Ministry to recover funds if services are not delivered in the required period.

5.70
We saw variable enforcement of claw-back provisions and associated decision-making and we did not see clear criteria for the Ministry’s staff to use in deciding when to invoke claw-back provisions.

5.71
In our view, claw-back provisions need to be applied consistently to be effective in holding the provider accountable for delivering the services agreed to in the contract.

Recommendation 11
We recommend that the Ministry of Education prepare clear guidance for staff about using the provisions in contracts to recover funds for undelivered services from the providers of professional development initiatives for teachers, and ensure that the guidance is followed.

5.72
We would consider this recommendation to be implemented if we saw documents providing clear guidance for staff on using the contract provisions for recovering funds.

1: The initiatives are: Numeracy Development Project; Te Kotahitanga; Strengthening Education in Mangere and Otara; Assessment for Learning; and initiatives within the Literacy Strategy.

2: Ministry of Education (2007), Education Report: Professional Development Provision Progress Update.

3: Education Review Office (2000), In-Service Training for Teachers in New Zealand Schools.

4: Ministry of Education (2006), Education Report: Centrally Funded In-Service Professional Development Provision.

5: Ministry of Education (2006), Education Report: Centrally Funded In-Service Professional Development Provision.

6: Ministry of Education (2000), Sharpening the Focus, Issue 5.

7: Ministry of Education (2007), Learning Policy Frameworks Project Plan: Professional Learning and Development Provision.

8: Ministry of Education (2006), Review of Schools’ Operational Funding.

9: Ministry of Education (2006), Education Report: Review of Contestable Funding.

10: Ministry of Education (2006), Review of Schools’ Operational Funding.

11: Ministry of Education (2006), Education Report: Review of Contestable Funding.

12: Gorinski, R. (2007), Analysis of Selected Output Areas from 2006 School Support Services’ Milestone 3 Reports.

page top