Part 6: Monitoring by district health boards

Effectiveness of arrangements to check the standard of services provided by rest homes.

6.1
In this Part, we discuss:

the different methods that DHBs use to monitor the services provided to elderly people whose care in rest homes the DHBs fund;
the duplication between DHB audits and certification audits by DAAs;
reducing reliance on routine contract auditing as a quality assurance tool; and
communication and co-ordination with the Ministry.

Our overall findings

6.2
Although the age-related care contract is the same throughout the country, DHBs interpret and monitor the contract differently. Most DHBs do not feel that they can rely on the findings of certification audits. They carry out their own auditing, which duplicates the auditing by DAAs.

6.3
Communication and co-ordination between DHBs and the Ministry has improved significantly in 2009. DHBs now have access to the Ministry's online database of certification reports, and have worked with the Ministry to improve the processes for dealing with complaints. They have also worked with the Ministry, and others, on a pilot project to replace surveillance audits with unannounced (or "spot") audits.

6.4
Some DHBs have begun to change the way they monitor rest homes, by providing rest homes with more targeted help and assistance. There are early signs that this approach can be effective. Greater communication and co-ordination among DHBs would help to share the lessons learned about the most effective means of monitoring rest homes.

Methods that district health boards use to monitor rest homes

DHBs use a range of methods to monitor the age-related care contract, and these methods vary from one DHB to the next.

6.5
The age-related care contract that DHBs have with rest homes is agreed at a national level every year. It covers rest home, dementia, and hospital-level care provided in a residential care setting. Although the age-related care contract between DHBs and rest homes is the same throughout the country, the way in which DHBs monitor the contract varies.

6.6
DHBs have a wide range of activities available to them to monitor contracts, including routine auditing, informal visits, liaison with community groups that provide advocacy services for older people (such as Age Concern), consulting DHB clinical staff who provide services to people living in rest homes (such as general practitioners and geriatricians), monitoring of complaints and concerns to identify any trends, and drawing on the knowledge of their Needs Assessment Co-ordination Services agency.

6.7
DHBs do not have many staff available for monitoring contracts with rest homes. They employ on average one full-time equivalent member of staff for monitoring and, on average, the ratio of DHB monitoring staff to rest homes is around 1:25. There is a wide variation among DHBs, with the ratio ranging from 1:7 to 1:80.

6.8
In most cases, DHBs commission their SSA to audit rest homes against the contract. There are four SSAs that provide services to DHBs:

South Island Shared Service Agency Limited, owned by the South Island DHBs (Southland, Otago, West Coast. South Canterbury, Canterbury, West Coast and Nelson Marlborough);
Technical Advisory Services, owned by the central region DHBs (Capital and Coast, Hutt Valley, Wairarapa, MidCentral, Whanganui, and Hawke's Bay);
HealthShare, owned by the midland DHBs (Taranaki, Lakes, Waikato, Tairawhiti, and Bay of Plenty). This agency also provides audit services to the Northern DHBs (Auckland, Waitemata, Counties Manukau, and Northland); and
Northern DHB Support Agency. Although this SSA does not carry out audits, it carries out research and analysis on behalf of northern DHBs. It is owned by Auckland, Counties Manukau, and Waitemata DHBs, and provides services to Northland DHB.

6.9
SSAs support DHBs' funding of health and disability services by carrying out research, analysis, and audits on the DHBs' behalf. They use a combination of employees and contractors to conduct audits and may also use clinical expertise available from DHBs (such as geriatricians and pharmacists) to assist audits when this is necessary. Some of the contractors who work for SSAs also carry out certification audits on contract to DAAs.

6.10
DHBs conduct regular reviews of SSAs to make sure that their work is of a satisfactory quality. DHBs have received largely positive reports about the work of SSAs, although issues of timeliness, internal auditing within the SSAs, management of contractors, and auditor competency have been raised during these reviews.

6.11
DHBs are largely satisfied with the work of their SSAs and, in general, find that audit reports from SSAs accurately reflect the status of the rest homes. One DHB commented:

The quality of audits from the shared services agencies is comprehensive and gives a clear picture of providers' ability to provide a required standard of care at an individual provider level.

6.12
SSAs do not have third-party accreditation, although they will be able to apply for accreditation under the same arrangements the Ministry is putting in place for DAAs.

Duplication with certification audits by designated auditing agencies

Most DHBs use routine contract audits of rest homes to monitor rest home compliance with the age-related care contract. These routine contract audits largely duplicate the audits that are carried out by DAAs for certification purposes.

6.13
Fourteen of the 21 DHBs carry out routine contract audits that check whether rest homes are meeting the requirements of the age-related care contract. The number and frequency of audits by DHBs varies. Some DHBs have a three-yearly cycle of audits. Other DHBs prioritise audits according to factors that include:

information they have about a rest home from complaints or other sources;
when the rest home was last audited and what that audit found; and
variations that have been made to the contract.

6.14
In 2004, an SSA carried out some analysis to identify duplication, overlaps, and gaps between the Standards and the age-related care contract. The analysis found a strong overlap between the approach used for certification audits and the approach used for routine contract audits. Because there is a high degree of overlap between the Standards and the age-related care contract, the routine contract audits cover many of the same aspects of care as the audits conducted by DAAs for certification. In the SSA's analysis, only 26% of the items covered by the routine contract audit were not covered during a certification audit.

Clip-on audits

6.15
To reduce duplication, the central region DHBs (Capital and Coast, Hutt Valley, Wairarapa, MidCentral, Whanganui, and Hawke's Bay) pay a fee to DAAs to extend the audits they do for certification to include the aspects of the age-related care contract not covered by the Standards. This is called a "clip-on" or "tag-on" audit. Clip-on audits reduce the auditing burden on rest homes because their compliance with the Standards and the age-related care contract are checked at the same time.

6.16
Clip-on audits are cheaper for DHBs than routine contract audits. Clip-on audits cost DHBs around $450. The cost of routine contract audits ranges from $3,500 to $7,500 (more expensive audits include the monitoring of the rest home's progress in implementing recommendations and requirements arising from the previous audit report).

6.17
As we noted in Part 3, DHBs have agreed that certification audits could help with the DHBs' monitoring of rest homes. The confidence of DHBs in DAA auditing needs to increase first. If all DHBs had confidence in the auditing by DAAs and used clip-on audits, significant sums could be redeployed to other means of assuring the quality of care provided in rest homes. For example, in 2007/08 DHBs (or their SSAs) carried out 118 routine contract audits. If the DHBs had felt able to use clip-on audits instead, collectively they would have saved around $600,000.

Lack of confidence in audits by designated auditing agencies

6.18
Despite the lower cost of paying DAAs, most DHBs continue with their own routine contract audits because they do not have confidence in the consistency and reliability of DAA auditing. The lack of confidence in DAA auditing is widespread among DHBs. Only four DHBs out of the 20 that responded to our survey consider certification to be reliable in assuring the safety of residents. Only six responded that the regime is reliable in providing assurance about the quality of care that rest homes provide. Many DHBs told us that they find inconsistencies between DAA audits and the audits carried out by SSAs.

6.19
One of the DHBs that uses clip-on audits is carrying out much of its own auditing in 2009 because of its concerns that clip-on audits are not reliable enough to monitor the age-related care contract. This DHB commissioned nine contract audits in the 2008/09 financial year, which were additional to its scheduled nine clip-on audits. The DHB plans to carry out a similar level of contract auditing in 2009/10.

6.20
In the course of their monitoring work, DHBs say that they find failures in care that were not detected by DAA auditors (see Figures 9 and 10). They are also aware, through complaints and through information they get from staff in hospitals, that the standard of care provided by a rest home is not always reflected in DAA audit reports. Figure 8 includes some of the responses from DHBs to our survey.

Figure 8
Survey responses – district health boards' concerns about the rigour of certification audits

"In [DHB] experience DAA audits generally rate provider service provision, quality and safety as being higher than audits performed by the DHB's audit agency. For example, in the 2007 year two providers who had recently undergone DAA audits and received good audit reports were found soon after by the DHB's auditors to be seriously defective in terms of the quality and safety of service provision." "We have seen instances where a provider received three years' certification and then needed significant attention and help through an issues-based audit or in one extreme case in 2008, through a Temporary Manager. In other cases, providers were said by HealthCERT to have few or no issues outstanding and yet the providers needed a comprehensive service assessment and follow-up from a clinician such as a Registered Nurse experienced in aged care to make changes to ensure resident safety and service quality. This is an area of significant concern for the DHB. When the certification and contractual audits have very different findings against the same criteria it can be a source of tension, mistrust and other difficulties between the provider and the Planning and Funding team, and affects the credibility of both Planning and Funding and DAAs." "Of the DAA audits done for certification we received about half of them at the DHB. In general, quality of care delivery issues are not picked up. At times complaints received regarding care arrive around the same time as the audit is done. At times we have found complaints investigations have led to the DHB undertaking a special audit and having to appoint a Temporary Manager when the provider holds a recent certification or has had a recent certification or surveillance audit that hasn't identified significant issues, or hasn't identified the same issues." "We have recently had two instances where providers have received certification audits with no concerns identified. Subsequent spot audits were undertaken by [the Ministry] following complaints which identified serious quality and safety issues in both cases." "A DAA report for one facility resulted in three-year certification, only to require a temporary manager within three months of certification."

6.21
Failures in the care of the elderly in rest homes that have not been picked up by DAA audits are often found during DHB-commissioned issues-based audits. DHBs ask their SSAs to carry out an issues-based audit when risks in a particular rest home have been identified, or when serious complaints have been made. These audits are normally unannounced and they focus on the specific issues or risks that have come to the DHB's attention. Where necessary, clinical and managerial experts, such as specialist geriatricians, general practitioners, and accountants (where there are governance and financial risks) are used and the audit teams can be larger than those used in either routine DHB contract audits or DAA certification audits. Figure 9 provides an example of an issues-based audit.

Figure 9
Example of an issues-based audit

In late 2008, a resident at a rest home had, in the course of two weeks, become ill and died. The family complained to the company operating the home that their relative had not received the medical intervention he needed, his room was dirty and messy, and a nurse was rude and derogatory to family members who were concerned at the man's declining health. The company operating the rest home carried out an investigation and most of the allegations were accepted. A plan was prepared to introduce a number of procedures to ensure that there was no repetition of poor care. The District Health Board had been informed of this incident and commissioned its SSA to carry out an unannounced audit of the rest home. What the SSA did The audit was carried out six months after the company's improvement plans were to have been implemented. It was an unannounced audit. A lead auditor and an auditor with clinical skills and experience spent two days reviewing the rest home. This included: Examining 10 files of residents who had been recently admitted – they checked if long-term plans had been completed in time and that they were evaluated and reviewed, liaison with GPs, if the residents had been regularly weighed and if unexpected weight loss had been referred to the GP. The review found gaps in care planning, clinical records, and monitoring. Reviewing all staff files. They found that most staff had not had their performance reviewed and most staff files did not have records of training. Checking staff rosters from a two-month period. The home complied with its obligations. Reviewing 13 internal audits and found weaknesses in the documentation in the audits. Inspecting the building, including the kitchen, which was found to be clean and well kept. What the SSA audit team concluded The audit found that, of the 10 planned improvements, three had been introduced. In addition, the audit found that there had been a lack of regular review of clinical practices at the rest home. There was a lack of consistent documentation recording the provision of the care provided to residents who had become unwell. This created a risk of inaccurate reporting and the risk of gaps existing in the provision of clinical services, exposing residents to possible harm. Soon after this audit, the clinical services manager at the home was suspended. Responsible management actions The company operating the rest home in this example had behaved responsibly. It had commissioned an investigation into this case where poor care was suspected, but staff of the rest home did not implement the improvement plan. The failings were then identified by the SSA. The rest home had been audited several times by the DAA, and the problems had not been identified.

6.22
Because issues-based audits concentrate on particular parts of the age-related care contract, auditors are able to inspect those aspects of the care given in rest homes in more depth and detail than DAAs do. This may explain some of the inconsistencies between DAA audit reports and those by SSAs after issues-based audits.

6.23
Some of the inconsistency may also be because of the different levels of detail between the Standards and the specifications of the age-related care contract. The age-related care contract is more specific than the Standards. For example, with staffing levels, Standard 2.7.3 states that the organisation (in this instance, the rest home) should ensure "the appointment of appropriate service providers [care staff] to safely meet the needs of consumers". The age-related care contract is much more specific, setting out the minimum number of staff required depending on the number of residents and their level of need.

6.24
The age-related care contract is also more specific about:

the content of admission agreements;
the content, timing, and amount of staff education;
having policies for aspects of care such as personal grooming and hygiene, wound care, continence, and management of challenging behaviour; and
the provision of dressing and continence supplies.

6.25
The contradictory nature of findings in audits of the same rest homes conducted by different organisations is a serious concern. Examples such as those given in this report cannot be explained by differences in the way that specifications and standards in the age-related care contract and those required for certification are written.

Inconsistent interpretations

6.26
There is also evidence of inconsistency in the way in which the age-related care contract is interpreted by different DHBs. Some companies that have rest homes in different DHBs find that different DHBs interpret the contract differently and place greater emphasis on different aspects of the contract. For example, one rest home operator, which has the same admission agreement in all its rest homes throughout the country, had the experience that some DHBs found the admission agreement did not fulfil the contract while other DHBs found that it did.

6.27
Consistency is important, not just for rest homes to know what is expected of them, but also to provide assurance to the public that they or their relatives will receive the same standard of care regardless of where they live.

Recommendation 7
We recommend that district health boards work together to ensure that they and their shared service agencies are interpreting the Age Related Residential Care Services Agreement consistently.

6.28
Greater sharing of experience, knowledge, and lessons learned between the different DHBs and their SSAs would help to improve consistency and provide the assurance the public needs when making choices about where they or their relatives should live once they can no longer live independently. Although the Health of Older People General Managers of DHBs meet quarterly, we consider that there is scope for more shared learning, especially between the SSAs, which do not communicate regularly.

Recommendation 8
We recommend that district health boards share information relevant to improving the safety and quality of services provided by rest homes quickly and freely with other agencies working in the rest home sector.

6.29
In 2009, DHBs agreed in principle to reduce contract auditing and work more closely with DAAs. This agreement is conditional on evidence that DAAs are providing more consistent and reliable information as a result of the Ministry's improvement project and wider programme of initiatives. DHBs would also like to have a role in the appointment of auditors.

6.30
It is inefficient for different public entities to spend their resources duplicating each other's efforts. In our view, DHBs should stop auditing rest homes as soon as they have confidence in the efficacy of auditing by DAAs. Using the information from certification audits (as well as clip-on audits) as one of the means to monitor the age-related care contract will release resources so that DHBs can target their monitoring of rest homes to where the risk to the safety and quality of residents is greatest.

Recommendation 9
We recommend that, once auditing by designated auditing agencies is effective and reliable, district health boards stop routine contract auditing and use their resources to work with those rest homes where improvements are needed most.

Moving away from auditing as a monitoring tool

Auditing is one means of monitoring the age-related care contract and some DHBs are beginning to shift their monitoring to focus more on quality improvement.

6.31
Some DHBs are beginning to question the effectiveness of relying on audits to monitor the age-related care contract. The information that audits provide is limited to a period of a few days (during business hours) and is collated after the rest home has had several months (or longer) to prepare for the audits (see Figure 10).

Figure 10
Survey responses – district health boards' concerns about the usefulness of audits

"It is becoming increasingly concerning that neither Contract auditing and/or certification audits are adequate mechanisms to monitor the quality of services provided or the outcomes for clients. They are "snap-shots" undertaken with the Provider having had long lead in times to ensure that on the day the necessary steps have been taken to meet the requirements of either audit." "On a couple of occasions we have found that neither type of audit has produced evidence of problems that our monitoring and intelligence has indicated were present and that the passage of time has confirmed." "Because the certification audits are scheduled and notified audits there is a risk of gaming to a higher quality of care for that occasion. The audit process is based on trust that the providers won't enhance their service during the audit, but anecdotally we hear this happens often e.g. by having a higher number of [registered nurses] on site than there are normally etc." "There are many aspects of ‘real-time' care an audit cannot assess as the practical nature of these are beyond the scope of the … certification audit in terms of time and attention to detail e.g. reviewing paper policy and procedures rather than watching care being provided – this is an inherent problem of audit methodology reinforced by the anecdotal finding that the more notice a provider has for an audit the better the outcome of the audit – whereas an audit of ‘actual practice' may highlight more quality issues."

6.32
Some DHBs have moved away from audit-based monitoring to provide more clinical support to rest homes where opportunities for improvement are identified. For example, Waitemata DHB does not carry out routine contract audits or clip-on audits. Instead, the DHB contracts an Aged Residential Care Support and Quality Advisor who visits all the rest homes every six months to provide them with support to meet the level and quality of services required in the contract. If the Aged Residential Care Support and Quality Advisor becomes aware of poor practice or the DHB receives complaints about a rest home, Waitemata DHB will arrange for an issues-based audit of the rest home. Some providers told us that this DHB provides the most effective monitoring.

6.33
Another DHB employs a contractor to check the compliance of rest homes against the age-related care contract when it has received complaints or has concerns about the care provided by the rest home. These assessments are used in place of issues-based audits. In some cases where problems are identified, the contractor, with the knowledge of the DHB, has offered their services to the rest home to help remedy the faults. A rest home operator that we spoke to about these arrangements said that they felt compelled to accept and pay (more than $10,000) for the services of the contractor, even though they believed that such an arrangement would seem to represent a conflict of interest.

6.34
In our view, such an arrangement impairs the independence of the contractor. It may also lead to the perception that the contractor has an incentive to find problems, because the rest home will then hire them to provide advice on how to remedy those problems. DHBs need to ensure that there are rules in place to limit any perception that contractors might have a financial incentive to identify problems when checking the compliance of rest homes against the age-related care contract.

Communication and co-ordination with the Ministry of Health

Communication and co-ordination between DHBs and the Ministry has been poor. There are signs that communication and co-ordination are improving.

6.35
As we have described, the auditing of rest homes for certification is similar to the audits many DHBs commission to monitor the age-related care contract. The monitoring of rest homes could be more efficient if there was more collaboration between the work of the Ministry, DAAs, and DHBs and SSAs. The auditing by DHBs is not well co-ordinated with that carried out by DAAs. There have been occasions when DHB auditors and DAA auditors have audited a rest home within days of each other. In one of the files we reviewed, the DHB auditors and the DAA auditors arrived at a rest home on the same day to carry out separate audits.

6.36
Recently, with the implementation of the Ministry's improvement project, there is better co-ordination between the different organisations involved in regulating rest homes, most notably with the development of a new approach to surveillance audits (see Figures 11 and 12).

Figure 11
Survey responses – communication between district health boards and the Ministry of Health

Some DHBs still find communication poor "HealthCERT will notify us when there are serious concerns regarding a provider, but not as a matter of routine. As a general rule, information sharing is very limited if at all. We believe there are opportunities for all auditing agencies to work more closely together for the benefit of residents, providers and funders." "As DHBs are not informed of the results of DAA audits, and what issues influence a certification decision, it is not always clear on what basis a certification decision has been made. Also it is not always clear how the length of the certification period is determined, and how it relates to the quality issues identified." "We would find closer communication and collaboration between DAAs and HealthCERT and the DHB to be of significant value. At present neither HealthCERT nor DAAs contact the DHB to discuss any issue we might have with a facility prior to undertaking an audit. This is not a difficult undertaking and would provide a large value add to the process. Additionally, it would be useful for the DHB to have a copy of any audits undertaken by these parties to form part of their overall understanding of the quality of service being delivered by a facility." Others find that it is improving "This year, HealthCERT has communicated regularly with [the DHB] on complaints received, and on certification matters. There is evidence that where HealthCERT and the DHB work collaboratively on a provider's quality issues, and when this is aligned with timely communications from [the Health and Disability Commissioner], the agencies gain a comprehensive picture of a provider's quality issues. We [are then] able to target our responses in a more effective manner." "It would be beneficial to have greater dialogue/interaction between DHBs and HealthCERT, however we acknowledge the activity taking place at a national level in this regard."

Figure 12
Example of collaboration between the Ministry of Health, district health boards, and designated auditing agencies

The spot audit pilot project The Ministry-led spot audit pilot project is part of the Ministry's rest home audits improvement project. The spot audit pilot project aims to improve public confidence in the care provided by rest homes by introducing unannounced (or "spot") audits and also reducing duplication between DHB and DAA audits. The Ministry has been working with DHBs, DAAs, consumer advocates, and rest home providers to put this project in place. The spot audits will replace the current surveillance audits, carried out by DAAs as a condition of certification. Spot audits will have a clinical focus and will also include consideration of DHB contractual requirements. DHBs will have the opportunity to be involved in the audit process. They will be able to share information with the DAA about a rest home before the DAA carries out the audit, be involved in following up any issues found during the audit, and receive the final audit report. Because of the DHB's involvement with the audit before it takes place, the chances of spot audits being carried out at the same time as DHB audits should be reduced. Twenty-three spot audits are being piloted and will result in an evaluation report that will inform the national roll-out of spot audits, planned for January 2010. As at November 2009, 14 spot audits had been completed.

page top