Part 5: Using information about the Strategy's progress

5.1
In this Part, we comment on:

• how the Ministry has used the information it has about the Strategy's progress;
• the gaps in what the Ministry has done; and
• what the Ministry needs to do to improve its use of monitoring and evaluation information.

Using the information collected

5.2
The Ministry told us that it takes an evidence-based approach and incorporates the information it has collected into its policy-making and operational decision-making systems. The Ministry had difficulty in clearly showing us how it used the information to monitor the Strategy's progress and recommend whether to maintain the current course or make changes. The Ministry was most easily able to show that information had been used where the response involved financial expenditure. For example:

• The Ministry monitors primary health care expenditure against forecasts, which provides information about the use of the subsidies and initiatives.
• The Ministry explained that it had introduced a mechanism for general practices to get permission for proposed increases to their advertised fees. The reason for the approvals system is to maintain the goal of lower-cost services.
• The Zero Fees for Under 6s subsidy was introduced after identifying that the number of general practices offering free care to children less than six years old had dropped to unsatisfactory levels.
• The Very Low Cost Access payments were introduced to ensure that patients' pay less when enrolled with participating PHOs.

5.3
The Ministry provided us with limited examples of reports analysing and responding to independent evaluation reports. For example, the original Care Plus Programme was modified using information gained from the programme's evaluation and advice provided to the Minister by the National Health Committee about meeting the needs of people with chronic conditions.

5.4
When the Ministry publishes evaluation reports on it website, it sometimes includes a brief statement of the Ministry's early reaction to the report. The statement may be part of a media release or included on the webpage. A good example is a newsletter called Primary Mental Health Care in New Zealand (November 2007) that, among other things, responds to interim evaluation results and links the results to current and planned work.

5.5
The Ministry was able to show how the last report listed in Figure 4 was incorporated into a document called Primary Health Care Strategy: Key Directions for the Information Environment (August 2007). This document sets out how to create an improved information environment that supports the Strategy.

Gaps in showing how information has been used

5.6 There are limited examples that show how the Ministry has fed what it has learned from its monitoring efforts into policy and operational decision-making.

Problems with capability

5.7
In 2007, the Ministry observed:

… limited formal experience and access to capability in relation to sector oriented performance improvement, change management, organisational development and learning, and performance analysis. The Ministry has been, therefore, unable to consistently facilitate and support performance improvement with DHBs in relation to the [Strategy], including fostering DHB links and dissemination of learning.¹

5.8
The Ministry has advised us that it now has enough capability to address these matters. We have not audited the steps the Ministry has taken to improve its capability.

What the Ministry needs to do

5.9
The Ministry should summarise in its reporting how it has used, or intends to use, the information it collects to maintain or improve progress towards the Strategy's goals. It would be useful to publish the Ministry's response to evaluation reports on the Ministry's website alongside those reports. This would serve two purposes. It would tell the public about what the Ministry intends to do, and it would assure participants in evaluations that the Ministry has considered the findings and made decisions about them.

5.10
To be useful, the Ministry's reports need to be suitable for the differing needs of DHBs, PHOs, Parliament, and the public. The Ministry's reports (and reports prepared for the Ministry) would be more accessible if they were written in plain English, or included plain English summaries of technical and densely written information.

1: Ministry of Health (2007), Health Report Ref. No.: 20070693: Review of the Primary Health Care Strategy Monitoring Environment, Wellington.