Auditor-General's overview

There are about 34,000 people living in 715 certified rest homes throughout the country. Older people who live in rest homes are some of the most vulnerable in our society, so it is important to have effective arrangements for checking the quality and safety of rest home services. The effectiveness and efficiency of such arrangements was the focus of a performance audit by my staff.

By law, rest homes have to provide residents with care that meets the Health and Disability Services Standards (the Standards). To provide residential care services for older people, rest homes must be certified by the Director-General of Health and, to remain certified, rest homes must be audited to check whether they meet the many criteria set out in the Standards.

The Ministry of Health (the Ministry) is responsible for the auditing and certification of rest homes. In my view, since its introduction in October 2002, certification of rest homes has not provided adequate assurance that rest homes have met the criteria in the Standards, and the Ministry did not respond quickly enough to address weaknesses and risks in the arrangements that it has known about since 2004.

The Ministry is actively trying to address shortcomings in the effectiveness of auditing and certification arrangements. For example, it has a certification improvement project and wider work programme that have been well managed, and the project has so far met most of its milestones. Communication between all those involved in overseeing rest homes has improved. The Ministry has also begun to manage risks more systematically.

However, more work remains to be done and it is still too early to tell whether the efforts to make the current arrangements work as intended will make a difference or whether certification is fundamentally unable to do what the legislation envisaged.

Auditing by designated auditing agencies has been inconsistent and sometimes of poor quality

Audits of rest homes can never eliminate the risk of poor care. Audits can only establish whether, at a particular point in time, rest homes have the systems and processes in place to minimise that risk.

The Ministry uses eight designated auditing agencies (DAAs) to carry out audits of rest homes.

The Ministry has known since 2004 that auditing by DAAs is inconsistent and sometimes of a poor quality. Notwithstanding its recent efforts, and evidence that DAAs are improving some aspects of their work, the Ministry did not respond to these problems quickly enough or with enough effect.

There are examples from 2008 and 2009 where DAAs have failed to find or report instances where rest homes have not met the criteria in the Standards. Serious failures in the care of residents have been identified later by other regulatory bodies. The frequency of these events may have been low, but they are significant because the failings are serious.

Progress reporting is a mechanism that is supposed to ensure that rest homes take action to fix problems identified by DAAs. Progress reporting is not always effective and is not leading to sustained improvements. Our file reviews showed that DAAs mostly rely on rest homes to report on their own progress and rarely make follow-up visits to verify that action has been taken. Some rest homes are repeatedly failing to meet the same or closely related criteria in the Standards, and some DAAs are behind in submitting progress reports to the Ministry.

Until its current programme of work, the Ministry's quality assurance of DAAs largely consisted of an evaluation of DAA audit reports (many of which were not properly completed). Except in times of crisis, the Ministry has given little feedback to DAAs on their performance, and it has never removed a DAA's designation despite evidence of sustained poor performance.

In my view, the Ministry must strengthen how it oversees the work of DAAs and how it deals with poor performance by DAAs. The Ministry's current programme of work has begun to address many of the weaknesses in auditing and certification. For example, the quality assurance system that the Ministry uses to oversee the work of DAAs has improved. There is now more stringent evaluation of DAA reports, closer scrutiny of DAAs known to be performing poorly, and the Ministry has begun to observe audits by DAAs. Communication between the Ministry and the aged care sector, including DHBs, has also improved. This work should have started sooner.

The Ministry has made efforts in the last two years to identify and address the problems with the current certification arrangements. Further, the need to improve the skill level and capacity of the HealthCERT team was recognised and restructuring began in 2008. A supporting information technology platform was put in place in 2008, enabling an accelerated work plan to continue throughout 2009.

Monitoring by district health boards

Rest homes are also monitored by DHBs. Most rest homes have a contract with their local DHB – the Age Related Residential Care Services Agreement (the age-related care contract). DHBs are required by law to monitor the delivery and performance of services by rest homes that they hold an age-related care contract with. Although the age-related care contract is the same throughout the country, individual DHBs interpret and monitor the contract differently. I encourage DHBs to achieve consistency in this matter.

Most (65%) DHBs do not consider certification to be reliable. Fourteen DHBs carry out their own auditing of rest homes (usually through their shared service agency), which largely duplicates the auditing carried out by DAAs. This diverts scarce resources from other monitoring work that could focus more on improving the quality of care in those rest homes where the risk to rest home residents is greatest.

Monitoring of rest homes by DHBs has not been well co-ordinated with the work of the Ministry. Occasionally, DHB auditors and DAA auditors have audited a rest home within days of each other, or on the same day. The results of the audits are often quite different.

Variable risk management

Certification relies on audits. Auditor independence is integral to the audit. We identified various threats to auditor independence inherent in the system. In our view, the Ministry could have responded more quickly to risks identified in the certification arrangements.

The Ministry now has a risk register, and has plans for managing several of the most significant risks, including:

• the risk of conflicts of interest;
• the risk that rest homes might select the cheapest or most lenient DAA;
• the risk that commercial pressures might influence an auditor's independence;
• the risk that DAAs might interpret and audit the Standards differently; and
• the risk that auditors might have inadequate skills and expertise.

Rest homes can choose which DAA will audit them, and most of the auditors who work for DAAs are freelance contractors. My staff found evidence of DAA auditors offering and providing rest homes with services in addition to auditing. Until 2009, the Ministry had not closely scrutinised the pricing and other business practices of DAAs. It now has limited knowledge of audit fees that different DAAs charge rest homes.

Certification audits are arranged well in advance, which means that the audit team may not see the rest home as it usually operates. The audit team (usually two people) is expected to check more than 200 criteria for a certification audit.

Some criteria will take longer than others to check. For example, DAAs check that the records kept are legible. DAAs also check that rest home residents are actively involved in the planning of each stage of service provision, which requires interviewing rest home residents and sometimes their families.

Most audits are carried out with only two days on site, which leaves DAAs little time to check each criterion thoroughly. A DAA could lose the rest home's business if the price for the audit were too high, which creates a commercial incentive to carry out audits quickly.

Ongoing effectiveness of certification

Rest home operators, DAAs, and DHBs agree that the introduction of the Standards and certification have raised standards in rest homes. However, there is evidence that the rate of improvement has slowed, and some rest homes consistently receive poor ratings for the same or closely related criteria. In addition, rest homes throughout the sector are often given poor ratings for some Standards - for example, the medicine management standard.

Overall findings

I am encouraged by the work the Ministry has done with DAAs and DHBs this year. However, it is too early to judge whether the changes being made will make the auditing, certification, and monitoring of rest homes more effective and efficient.

My Office will do more work in 2011 to look at whether the changes the Ministry is now making have improved the effectiveness of the overall certification process. More fundamental changes to the design of the auditing, certification, and monitoring arrangements may yet be needed. I recommend that the Ministry consider whether other arrangements would be more effective and reliable.

I thank the staff of the Ministry, DAAs, DHBs and their shared service agencies, rest home providers, and the organisations that provide advocacy services for the elderly, for helping my staff with this performance audit.

Lyn Provost
Controller and Auditor-General

15 December 2009