4 A discussion about quality

4.1 Output quality measures used by DHBs

The Consistency Panel observed that the SOIs contain few measures of the quality of outputs (services), particularly direct measures of quality (as opposed, for instance, to impact measures that may indirectly indicate service quality).

More than half of the DHBs appear to have five or fewer performance measures relating directly to service quality.⁵

Table 3 sets out some commonly used measures of quality, and Table 4 sets out some measures that are not commonly used but clearly relate to the quality aspect of performance.

Table 3: Commonly used measures of quality

Table 4: Other measures of quality

4.2 Explicit identification and explanation of measures of a service's quality

It is not always clear which dimension of performance a measure is intended to show. About six of the DHBs do explicitly identify the relevant dimension of performance in at least some cases. However, even in these cases it is not necessarily clear how the measure shows a particular dimension of performance.

Giving supporting explanations about the intention and scope of the performance measures, where they are not self-evident, would add value to the performance information.

For example, Taranaki DHB has a measure on the percentage of total acute admissions that were treated as day-stay cases. It comments that a decrease in the percentage of these could indicate fewer inappropriate admissions. This comment clarifies the rationale for the measure and target.

4.3 Certification and accreditation

Lakes DHB uses a number of measures relating to certification and accreditation (for example, that medical practitioners hold relevant and current practising certificates and that hospitals are accredited).

It is for the sector to decide on the appropriateness of including these measures in the forecast SSP. You may wish to consider it, so we bring this to your attention.

4.4 Other insights into the quality of service delivery

We acknowledge that DHBs present a number of valuable measures in their SOIs, which form part of the picture of the overall quality of the health services. These include:

• measures that directly measure other dimensions of performance but can also be regarded as sub-components of the overall quality of a service – for example, the timeliness of cancer treatment (timeliness dimension) and the number of mental health clients with current relapse prevention plans (quantity dimension) are also likely to be integral to the overall quality of the service;
• coverage/uptake measures – for example, screening rates to detect breast cancer can be an indicator of the quality of health promotion services; and
• impact measures – for example, the rate of ambulatory sensitive admissions to hospital can be an indicator of the quality of health promotion services and primary health sector care.

These measures are often useful proxy measures of the quality of output delivery. Nonetheless, the Crown Entities Act requires output measures in (forecast) SSPs. Therefore, it is important that, where practicable, DHBs report using relevant and reliable measures of service quality (as distinct from impact measures or service quantity, timeliness, or uptake measures).

4.5 Conclusion

In conclusion, it is important that there be a discussion at sector level, and that DHBs and auditors discuss output quality performance measures. The conversations could include:

• What measures do DHBs see as measuring service quality? Are they direct measures of service quality? What do DHBs consider themselves accountable for?
• What is the relationship with internal performance management of the quality of services?
• What measures are other DHBs using that could be relevant to them?
• Would there be merit in the forecast SSP explicitly labelling those measures that relate to the quality dimension of performance as "measures of quality"?
• Within the context of key risks, new initiatives/services, and business-as- usual, do the planned measures give a reasonable and proportionate picture of output quality for significant services?
• How, or should, basic certification and accreditation requirements of medical practitioners and facilities be reflected in the forecast SSP?
• What, if any, supporting context is needed to ensure that the reasons for using particular measures is understood by the reader?

5: We recorded in a spreadsheet performance measures from the SOIs that may relate to the quality dimension of output performance. The spreadsheet is subjective and inconsistent – for example, if a DHB identified a measure as a measure of the quality dimension of performance but it was not clear to us how the measure was demonstrating quality, we included it in the spreadsheet for that DHB. We did not, however, consistently include it in the spreadsheet for any other DHBs using that measure. Bearing that in mind, the spreadsheet is available for discussion purposes from the auditor.