4 A discussion about quality

District Health Boards: Learning from 2010-2013 Statements of Intent.
Quality performance measures
Quality is an important dimension of output performance. SOIs should include measures of the quality or standard expected of outputs, where practicable.

4.1 Output quality measures used by DHBs

The Consistency Panel observed that the SOIs contain few measures of the quality of outputs (services), particularly direct measures of quality (as opposed, for instance, to impact measures that may indirectly indicate service quality).

More than half of the DHBs appear to have five or fewer performance measures relating directly to service quality.5

Table 3 sets out some commonly used measures of quality, and Table 4 sets out some measures that are not commonly used but clearly relate to the quality aspect of performance.

Table 3: Commonly used measures of quality

Acute re-admissions to hospital (18 DHBs)
30 day mortality rate (10 DHBs)
  • Rate of mortality within 30 days of discharge from hospital (the parameters of this measure may vary between DHBs)
Elective Services Performance Indicators Compliance (9 DHBs) (DHBs could consider explicitly identifying which ESPIs measure the quality of service provided)
Patient/consumer complaints (6 DHBs)
  • Complaints - older people
    • aged residential care services
    • high level of satisfaction with NASC services, measured by increase in positive feedback/decrease in complaints
    • respite care services
    • day services
  • Complaints – respite care services
  • Support services (NASC) – number of complaints
  • Complaints – hospital services
    • number of complaints per patient contact
    • complaints closed within 20 working days
    • Health and Disability Commission complaints that result in a finding of breach of the Code
  • Resolution of complaints
Patient/consumer satisfaction (6 DHBs)
  • Satisfaction with hospital services
  • Percentage of women rating their post natal length of stay as "just right"
  • Satisfaction with DHB facilities
  • Whanau/family satisfaction with palliative care services

Table 4: Other measures of quality

Hospital services:
  • Reduction in central-line-associated bacteraemia
  • Surgical site infections
  • Hospital-acquired blood stream infections per 1000 bed days
  • Proportion of patients with hospital-acquired pressure injuries
  • Decubitus ulcers
  • Planned day-surgery cases that stay one or more nights
  • Patient falls
  • Audit score for hospital responsiveness to family violence, child, and partner abuse
  • Compliance with World Health Organisation hand-hygiene guidelines
  • Improved Health of the Nation Outcome Scales (HoNOS) scores for people discharged from mental health inpatient services
Support services:
  • Percentage of patients in the palliative service who die in the place of their choosing

4.2 Explicit identification and explanation of measures of a service's quality

It is not always clear which dimension of performance a measure is intended to show. About six of the DHBs do explicitly identify the relevant dimension of performance in at least some cases. However, even in these cases it is not necessarily clear how the measure shows a particular dimension of performance.

Giving supporting explanations about the intention and scope of the performance measures, where they are not self-evident, would add value to the performance information.

For example, Taranaki DHB has a measure on the percentage of total acute admissions that were treated as day-stay cases. It comments that a decrease in the percentage of these could indicate fewer inappropriate admissions. This comment clarifies the rationale for the measure and target.

4.3 Certification and accreditation

Lakes DHB uses a number of measures relating to certification and accreditation (for example, that medical practitioners hold relevant and current practising certificates and that hospitals are accredited).

It is for the sector to decide on the appropriateness of including these measures in the forecast SSP. You may wish to consider it, so we bring this to your attention.

4.4 Other insights into the quality of service delivery

We acknowledge that DHBs present a number of valuable measures in their SOIs, which form part of the picture of the overall quality of the health services. These include:

  • measures that directly measure other dimensions of performance but can also be regarded as sub-components of the overall quality of a service – for example, the timeliness of cancer treatment (timeliness dimension) and the number of mental health clients with current relapse prevention plans (quantity dimension) are also likely to be integral to the overall quality of the service;
  • coverage/uptake measures – for example, screening rates to detect breast cancer can be an indicator of the quality of health promotion services; and
  • impact measures – for example, the rate of ambulatory sensitive admissions to hospital can be an indicator of the quality of health promotion services and primary health sector care.

These measures are often useful proxy measures of the quality of output delivery. Nonetheless, the Crown Entities Act requires output measures in (forecast) SSPs. Therefore, it is important that, where practicable, DHBs report using relevant and reliable measures of service quality (as distinct from impact measures or service quantity, timeliness, or uptake measures).

4.5 Conclusion

In conclusion, it is important that there be a discussion at sector level, and that DHBs and auditors discuss output quality performance measures. The conversations could include:

  • What measures do DHBs see as measuring service quality? Are they direct measures of service quality? What do DHBs consider themselves accountable for?
  • What is the relationship with internal performance management of the quality of services?
  • What measures are other DHBs using that could be relevant to them?
  • Would there be merit in the forecast SSP explicitly labelling those measures that relate to the quality dimension of performance as "measures of quality"?
  • Within the context of key risks, new initiatives/services, and business-as- usual, do the planned measures give a reasonable and proportionate picture of output quality for significant services?
  • How, or should, basic certification and accreditation requirements of medical practitioners and facilities be reflected in the forecast SSP?
  • What, if any, supporting context is needed to ensure that the reasons for using particular measures is understood by the reader?

5: We recorded in a spreadsheet performance measures from the SOIs that may relate to the quality dimension of output performance. The spreadsheet is subjective and inconsistent – for example, if a DHB identified a measure as a measure of the quality dimension of performance but it was not clear to us how the measure was demonstrating quality, we included it in the spreadsheet for that DHB. We did not, however, consistently include it in the spreadsheet for any other DHBs using that measure. Bearing that in mind, the spreadsheet is available for discussion purposes from the auditor.

page top