Part 4: Is the information from "Get Checked" being used to improve diabetes services?

Ministry of Health and district health boards: Effectiveness of the "Get Checked" diabetes programme.

4.1
In this Part, we report on whether:

  • the information being collected from the programme was being entered into the diabetes registers;
  • the information in the registers was being used to promote improvements in diabetes services and carry out clinical audit; and
  • LDTs and DHBs were using the information in their diabetes registers to coordinate and plan diabetes care services in their districts.

Is the information from the programme being collected and entered into the diabetes registers?

4.2
An important goal in the 1997 Diabetes Strategy was to:

Establish information systems to collect data to enable identification of people with diabetes and to monitor the care provided.1

4.3
Diabetes 2000 introduced a requirement for diabetes registers to be maintained that include an agreed national minimum dataset of information, which is updated following an annual review. Diabetes 2000 also considered that it was most logical for primary care organisations (now PHOs) to set up and update the main registers for diabetes and the health status of people with diabetes.

4.4
As noted in this report, the DHBs have funding arrangements with a number of different organisations to administer the diabetes programme. We refer to these organisations as programme administrators.

4.5
One of the roles of a programme administrator is to enter the information collected from the programme into the diabetes register. We expected that the information collected from the annual check:

  • would be enough to monitor patient health status; and
  • would be correctly entered in the diabetes register.

Is enough information collected?

4.6
The annual check service specification (see paragraph 2.24) requires:

  • the programme administrator to record the data specified in the minimum dataset in its diabetes register; and
  • aggregated data to be collected and reported to LDTs.

4.7
We set out in Appendix 2 the contents of the minimum dataset and the aggregated data that must be sent to the LDTs through the programme.

Is the information from “Get Checked” being used to improve diabetes services?

4.8
During our discussions with the DHBs, PHOs, and programme administrators that we visited, staff agreed that the minimum dataset was the right information to monitor patient health status and identify risks and complications. However, some of the people we interviewed expressed concern about using the aggregated data to assess the effectiveness of the programme at both a district and national level. This is discussed further in paragraphs 5.19 to 5.35.

Is the information correctly entered in the diabetes register?

4.9
Generally the GP or diabetes nurse enters the data in the patient management system (although some general practices still submit manual forms for entry by programme administrators), and then the information is sent electronically to the programme administrator. The programme administrator records the information in the diabetes register and arranges for the DHB or HealthPAC2 to be invoiced for the cost of the check. Invoices are sent in batches and are usually sent monthly.

4.10
We were not able to audit general practices’ data entry (because of our mandate, the number of GPs and practice nurses, and patient privacy issues), but we did ask programme administrators about the systems that they had in place, to ensure that the data entered in their register was accurate. We found that some programme administrators had encountered problems in setting up and maintaining accurate registers and that, while some of the problems had been resolved, others continued.

4.11
Software for a diabetes register or database was not supplied when the programme was introduced. Programme administrators needed to develop or purchase software themselves. This had resulted in a variety of systems being set up with varying degrees of success.

4.12
South Link Health Inc and the Wellington Regional Diabetes Trust had wellestablished data collection systems.

4.13
On the other hand, smaller programme administrators have had difficulty establishing data collection systems because they have not had the required IT expertise. For example, a group of smaller programme administrators in the Auckland DHB obtained funding from the Auckland LDT to fund a “Get Checked” project manager for 2005 to assist the programme administrators with their IT systems. The programme administrators’ problems included:

  • a lack of IT skills, which meant that the administrators were not able to fix problems when they occurred – for example, when information thought to have been sent to their server had not been received because general practices had not corrected data entry errors or had not filled templates in correctly;
  • GPs using different patient management systems, making it difficult to get consistent results;
  • data not being able to be sent to the Health Link server because of errors;
  • the Health Link server not being able to communicate electronically with HealthPAC; and
  • payment for the completed annual checks taking up to three months, and general practices having difficulty reconciling the checks that they had done to the payments.

4.14
The Hawke’s Bay DHB had only recently set up an electronic system that PHOs had confidence in. The DHB originally contracted out the management of the diabetes register. However, an audit of the register by the DHB identified problems with the records kept. At the beginning of 2003, the DHB entered into an interim 15-month funding arrangement with three independent practitioner associations to collect the data from their GPs. Problems continued, and the DHB assumed responsibility for collecting the data from March 2004, when PHOs were set up. The data was collected manually until an electronic database was finalised in February 2005. IT problems were still being experienced by some PHOs in the Hawke’s Bay at the time of our audit. One of the three PHOs that we visited advised us that the number of free checks reported as having been completed by their GPs was more than that reported by the DHB in the diabetes register. To resolve this issue, the DHB and the PHO had jointly appointed an IT liaison person to identify the problem.

4.15
Another PHO that we visited, that had its nine practices on the same patient management system, was still having difficulty identifying the numbers of patients that had attended for a free annual check. We were told that these difficulties resulted from reconciling its data with other Chronic Care Management data.

4.16
As payment invoices are generated on the basis of the data received from GPs, we were looking for assurance that the programme administrators were checking that the data received from GPs was accurate. We established that:

  • Capital & Coast DHB had commissioned an independent auditor to audit whether its programme administrator (the Wellington Regional Diabetes Trust) was achieving the funding arrangement requirements, including those relating to data quality. The audit noted that occasional checks of data entry were carried out, but the Trust’s staff members were unsure of the DHB’s expectations regarding appropriate audit and quality assurance processes required under the funding arrangement.
  • Some data validity checks were being done by a few programme administrators – for example, checking for valid National Health Index numbers, and business rules being applied at the front end of data capture to ensure that only one claim was submitted for each 12-month period. However, we do not consider that these checks are enough to establish that payments were legitimate. We also consider that the data was not being adequately audited.

4.17
Some LDTs had problems getting reliable information. For example, the groups performing the LDT function for the Counties Manukau DHB had expressed concerns about the quality of the data they had received in the past and had worked to improve this.

4.18
The Auckland LDT had also engaged an IT analyst during 2004 to help the smaller programme administrators to examine and improve their “Get Checked” reporting systems. The Auckland LDT noted in its 2004 annual report that the programme administrators would need ongoing support. The DHB and the past chairperson of the LDT were still not confident that the information being received from programme administrators was reliable when we visited in November 2006.

4.19
The Auckland DHB notes in its 2006 Annual Report that:

ADHB Diabetes Centre undertakes retinal screening. Their records indicate that there were 8590 visits for retinal photography from 7208 different individuals in the past 24 months. Approximately 90% of these patients are from ADHB (+/- 6500).

The Get Checked report on the other hand, noted that 2801 patients had undergone retinal screening within the past 2 years.

The vast difference between these numbers reflects the difficulties that primary care clinicians have with entering “Get Checked” data and also the numbers of diabetics undergoing routine care whose names are not submitted through “Get Checked”. 3

4.20
The Ministry has recognised that there have been problems with the existing software for collecting diabetes register information: it is difficult to use and maintain, it cannot be readily used to share information, and it cannot link to enrolment and other data in future.

4.21
Since March 2006, the Ministry has supported upgrading the software, including developing regional IT hubs. The Ministry considers that the new software will deliver most benefit if programme administrators choose to use a “hosted server” at a regional level.

Conclusion

4.22
After some initial difficulties, the information from the programme was being entered in the diabetes register in all the DHBs that we visited. However, we have concerns about whether the data recorded by some programme administrators is complete and accurate, because of IT system and usage problems. A variety of factors contributed to the problems. These included general practices needing to improve IT skills, programme administrators having to receive data from a variety of patient management systems, and programme administrators needing to improve IT skills. The upgraded system and development of regional IT hubs should help to resolve some of these issues.

4.23
We are also concerned, especially given the IT problems noted, that there is inadequate auditing of the payments made on the basis of the data entered in the diabetes register.

Recommendation 7
We recommend that district health boards ensure that the information in their diabetes registers is accurate and updated, and work with programme administrators to identify, clarify, and resolve current problems affecting data quality.
Recommendation 8
We recommend that district health boards ensure that enough audit processes are in place to verify that payments are being made for genuine annual checks, and that they work with their programme administrators to achieve this.

Is the information in the diabetes registers being used to promote improvements in diabetes services and carry out clinical audit?

4.24
We expected that:

  • programme administrators would monitor the results for PHOs and GPs, and report a summary of the results and analysis to PHOs and GPs to allow them to benchmark their results;
  • programme administrators and PHOs would use the benchmarking information to identify educational and training opportunities; and
  • programme administrators and PHOs would carry out clinical audits.

Is monitoring and feedback carried out?

4.25
The use of the data in the diabetes registers and reporting back to PHOs and individual GPs varied significantly between the programme administrators that we visited. For example, South Link Health Inc did a detailed annual report which analysed data by GP, practice, the PHO, the DHB, and South Link Health Inc (which covers five DHBs). The feedback provided by South Link Health Inc included:

  • tables that showed numbers and percentages of patients’ results broken down by demographic factors, treatments, process measures, and outcomes for each year from 2000;
  • a graph that showed the percentage of non-smokers, past smokers, and smokers by GP, practice, the PHO, the DHB, and South Link Health Inc;
  • cohort analysis that followed the same groups of patients for the past four years (which again was done at the practitioner, general practice, PHO, DHB, and South Link Health Inc levels);
  • tables that compared improvements in glycaemic control over time for patients with an HbA1c greater than 8% and 9%, by GP, practice, PHO, and DHB; and
  • a printout for each GP that gave details of each patient that the GP had enrolled in the programme, and which highlighted where the GP might target efforts to improve patient health.

4.26
The Wellington Regional Diabetes Trust produced six-monthly provider reports that gave details of the number of annual checks done and the results of the checks. The reports also highlighted where GPs should consider prescribing medication to reduce the risk of kidney and cardiovascular complications.

4.27
Counties Manukau DHB also sent monthly reports to PHOs covering the results of those people diagnosed with diabetes who had been included in the programme through its Chronic Care Management programme. The reports gave the average results for each item in the minimum dataset by ethnicity (Māori, Pacific Island peoples, and all). It showed the first visit, the latest data, and the results at one year. The DHB also sent a report to all PHOs showing comparative results for the programme for PHOs in the DHB against the minimum dataset. Counties Manukau DHB also generated monthly reports for PHOs on patients who were overdue for an annual check and those who had not attended for their annual check.

4.28
General practices in the Hawke’s Bay had not had feedback until about June 2006 because of problems with the IT systems (see paragraph 4.14). Under the new arrangement, in which the Hawke’s Bay DHB administered the diabetes register and programme data, the Hawkes Bay PHO advised us that it had limited access to the monthly data recorded in the diabetes register and needed access to the actual performance and outcomes, not just the numbers of checks being done. The PHO noted that software problems and reliable access to good data were the biggest problems.

4.29
In the past, Auckland PHO Limited had used another programme administrator to manage its diabetes register, and had not been able to access the results at a GP level. The PHO had taken over managing the diabetes register itself this year using the new Ministry IT system, and was looking forward to being able to analyse the data and feed results back to individual GPs.

Do primary health organisations and programme administrators use the information to identify educational and training opportunities?

4.30
Some programme administrators and PHOs that we visited also provided feedback to GPs through continuing medical education sessions. For example, Total Healthcare Otara fed its results back to GPs and practice nurses in quarterly continuing education sessions. It analysed by general practice the attendance rates, trends in the HbA1c levels, and use of treatment plans. It chose to take this approach in recognition that diabetes management involves both diabetes nurses, practice managers, and GPs. Practice managers in the Counties Manukau DHB were also sent monthly reports with summary data about their patients in the Chronic Care Management Diabetes programme.

Are clinical audits carried out?

4.31
None of the programme administrators, PHOs, or DHBs that we visited used the information in diabetes registers to conduct regular clinical audits to provide assurance that general practices are providing diabetes care in line with the evidence-based best practice guidelines and national referral guidelines.

4.32
One programme administrator, South Link Health Inc, was supporting its GPs to carry out their own clinical audits. GPs hold the information, which was available to the DHBs and PHOs if requested. Some PHOs told us that they were keen to offer a clinical audit to GPs once they had access to better information from the programme.

Conclusion

4.33
The programme administrators that we visited were not monitoring data from the programme and feeding results back to PHOs and GPs on a consistent basis. While some programme administrators did it very well, others need to ensure that they do it on a regular basis.

4.34
None of the programme administrators, PHOs or DHBs that we visited were doing regular clinical audits using the information in diabetes registers. Although we acknowledge that there are difficulties with the data in some areas, this should not preclude programme administrators, PHOs, and DHBs using the base data that they do have available.

Recommendation 9
We recommend that district health boards work with programme administrators to ensure that the data from the “Get Checked” programme is thoroughly analysed and the results regularly reported back to general practices to improve diabetes care.
Recommendation 10
We recommend that district health boards work with primary health organisations and programme administrators to ensure that adequate clinical audit is carried out to provide assurance that general practices are providing diabetes care in line with the evidence-based best practice guidelines and national referral guidelines.

Are local diabetes teams and district health boards using the information in the diabetes registers to co-ordinate and plan diabetes services in their districts?

4.35
The first LDTs were set up in mid-2000 as part of the implementation plan in Diabetes 2000. At that time, an LDT service specification was included as part of their funding arrangement with LDTs (see paragraph 2.32). In summary, the terms of reference for LDTs are to identify the health needs of people with diabetes, their family/whānau, and their communities; to monitor the use of resources related to diabetes; and to recommend any improvements deemed necessary.

4.36
LDTs were set up and operating differently in the DHBs that we visited. Over time, some had changed how they operated. The LDT representatives that we spoke were dedicated in their commitment to improving diabetes services.

4.37
In 2003, the Counties Manukau DHB established a Diabetes/Cardiovascular Advisory Group in place of its LDT.4 This group provided advice to Counties Manukau DHB and to the broader Counties Manukau health sector on issues to do with designing and implementing health interventions for diabetes and cardiovascular disease. This group was responsible for preparing the annual report for the Ministry (as required under the programme), and produced quarterly reports to the DHB Board. It:

  • was working with the PHOs to resolve IT issues and improve the timeliness and quality of the data;
  • had completed a pricing review and recommended an increase to $60 for each annual review that had been approved; and
  • was to work with the social marketing team to develop strategies to improve the take-up of the programme.

4.38
There was a close relationship between the Hawke’s Bay DHB and its LDT. The role of the Hawke’s Bay LDT was different from that of other regions, in that the DHB collected the data from the free annual check. The Hawke’s Bay DHB prepared the LDT’s annual report, and the LDT was asked to comment on the report.

4.39
The Auckland LDT was in transition at the time of our audit. Diabetes Auckland’s contract as the LDT had finished in July 2006, and the new provider signed the contract at the end of April 2007.

4.40
The Tairawhiti LDT was not meeting at the time of our audit and the Tairawhiti DHB was considering the future of the LDT in its district. The DHB considered that there was a great deal of pressure for health professionals to attend committees and groups in its areas, and it was looking at how the various groups could be realigned or combined to take the pressure offthe health professionals.

4.41
The Capital & Coast DHB considered that the work that its LDT did and the data from the programme were an important means of informing the DHB’s future direction in diabetes care and management.

4.42
The Otago LDT had been established in 2001, and the contract for administering it had changed hands in 2005. The LDT contracted a project co-ordinator to support it in its role. There was a DHB representative on the LDT, but the LDT operated largely at arms length to the DHB, and formally reported to the DHB through an annual report.

4.43
LDTs and DHBs should be using the information in the diabetes register to coordinate and plan services in their districts. We expected that:

  • LDT annual reports review the effectiveness of diabetes healthcare services and recommend ways of improving service quality.
  • The DHB or a committee of the DHB receive the LDT annual reports and give them due consideration.

Do local diabetes team annual reports review the effectiveness of diabetes services and recommend improvements?

4.44
The main output of the LDT is an annual report that covers the preceding calendar year, and which must be provided to the DHB and the Ministry by 14 February each year. The annual report should review the effectiveness of diabetes healthcare services and recommend ways of improving service quality. The LDT service specification requires the report to:

  • specifically review the provision and management of diabetes services for all people with diabetes, but especially high-risk groups such as Māori and Pacific Island peoples;
  • estimate the incidence and prevalence of diabetes in the population by ethnicity;
  • include analysis of primary care data and other clinical information reported to the LDT by diabetes service providers;
  • review DHB accountability indicators and targets for the previous year by ethnicity, and recommend accountability indicator targets for the DHB for the coming year;
  • review progress toward the achievement of other targets and goals, both locally and nationally;
  • identify new service initiatives or changes implemented during the year and, if possible, review their effectiveness;
  • highlight services that are working well and identify problems and priorities; and
  • provide recommendations for ongoing service quality improvement.

4.45
We obtained copies of the LDT annual reports for five of the six districts that we visited. The Tairawhiti district’s LDT had not produced an annual report. For the five LDTs that did produce reports, we reviewed the reports to establish whether the above requirements were being met. We focused on the most recent report available at the time of our visit, which was the report to 31 December 2005.

4.46
Overall, we found that, although two LDTs had gone a significant way to meeting the service specification requirements, none met the requirements in full. The remaining three LDTs still had some way to go.

4.47
We found that all five of the LDTs with annual reports had reported on the incidence and prevalence of diabetes (using information supplied by the Ministry), and had reviewed the DHB accountability indicators and targets for the preceding year by ethnicity. In the Tairawhiti district, where the LDT had not produced a report, the DHB had done this analysis for its district as well. However, the depth of analysis of the primary care data varied.

4.48
For the requirement that LDTs review the provision and management of diabetes services for all people with diabetes, especially high-risk groups such as Māori and Pacific Island peoples, we found that only two out of the five LDTs considered this issue in their annual reports.

4.49
For “other clinical information”, LDTs are required to collect information from specialist diabetes services (both hospital and non-hospital based). Only two LDTs included information on specialist diabetes services in their reports. However, this information was descriptive, and contained very little robust analysis of the supply and demand for specialist diabetes services in their districts. One diabetes specialist from secondary diabetes care services considered that this was a key issue – that, while a reasonable picture was being established of those patients seen in primary care, there was a gap for those patients being seen in secondary care (the patients with the more complex conditions). The diabetes specialist considered that it would be good to have data collected in a similar way in secondary care to complete the diabetes picture. This would also allow comparisons between secondary care units throughout the country.

4.50
One LDT report identified the new services introduced during the year, but the review of the effectiveness of these new measures was limited.

4.51
Only one LDT report identified “issues” where further resources were required or the quality of services needed to be improved. The report also recommended service improvements to the DHB.

4.52
One diabetes specialist from secondary diabetes care services advised us that LDTs did not circulate the programme data to all interested parties, and noted that diabetes specialists in secondary care did not get a chance to comment on the data. He also noted that LDTs needed access to someone with statistical or epidemiological expertise to ensure that programme data were robustly analysed.

Do district health boards give local diabetes team reports due consideration?

4.53
The LDT service specification requires that the LDT annual report and any supporting documents be presented to a meeting of the DHB, or appropriate subcommittee, and that any recommendations accepted be included in the DHB’s District Annual Plan.

4.54
We found that the relationship between the LDTs and their respective DHBs varied significantly between the DHBs that we audited. The annual reports of the LDTs received different levels of consideration by the DHBs. Of the five LDTs that produced an annual report:

  • one LDT’s recommendations had been incorporated by the DHB into the DHB’s District Annual Plan;
  • one LDT had been replaced by a Diabetes/Cardiovascular Advisory Group which reported regularly to the DHB;
  • one LDT reported on its annual report as part of quarterly reporting to the Community and Public Health Advisory Committee5 of the DHB, and told us that the committee was very interested in the report;
  • one LDT had actively sought the DHB’s consideration of its most recent report and had referred the recommendations to the DHB’s management, who recommended to the DHB that they should not be actioned. The LDT was not told this, and subsequently read about it in the DHB’s newsletter after our visit in October 2006; and
  • one LDT had not had any response from the DHB and told us that the DHB was preparing a cardiovascular disease/diabetes strategy, but the LDT had not had any input into its development.

4.55
One of the main concerns of the LDT chairpersons we interviewed was that the LDTs could only report. No-one had to listen to the LDTs, and they had no power to require action on any concerns that they had. One chairperson thought that there should be another loop where the data from the LDT reports was collected nationally.

4.56
The Health Funding Authority set up a National Diabetes Working Group in October 1999, just before LDTs were set up. Its role was to oversee the development of a disease management approach to diabetes in New Zealand, from primary prevention through to tertiary treatment, drawing on the best available evidence. One of the functions of the National Diabetes Working Group was to review the annual reports of LDTs and assist with identifying priority areas and effective initiatives. The National Diabetes Working Group was required to produce an annual report to the Ministry on diabetes in New Zealand, with recommendations for improving disease management approaches to diabetes and diabetes prevention. The National Diabetes Working Group was disbanded during 2003/04.

4.57
An Expert Advisory Group for Diabetes and Cardiovascular Disease was set up early in 2006. It is working on a quality improvement plan for diabetes and cardiovascular disease. Any national co-ordination role that this group might have is yet to be determined.

4.58
In the six DHBs that we visited, we found three examples where LDTs and DHBs working together had improved service provision – Hawke’s Bay, Auckland, and Counties Manukau DHBs.

4.59
Since June 2005, the Hawke’s Bay DHB and its LDT have been working to resolve the shortage of podiatry services. A podiatry subcommittee was set up to recommend a solution to the problem. The subcommittee assessed various models (for example, whether to provide a mobile podiatry service or a fixed service) to identify the best option. The selected scheme provided a list of providers for the patient to choose from and the patient was to be funded for up to four visits a year. The scheme was expected to cost $150,000 for the first year.

4.60
The Auckland LDT obtained funding to appoint an IT analyst to work with the smaller Auckland PHOs to examine and improve their “Get Checked” data. The appointment was made during 2005 for a period of 12 months.

4.61
In 2002, the Counties Manukau LDT highlighted the low take-up and the long waiting time for the DHB’s retinal screening6 service. This resulted in the LDT reviewing the service and community-based retinal screening services being implemented. The LDT also addressed the high number of people that were not attending their scheduled appointment, which increased the efficiency of the DHB’s current service.

Conclusion

4.62
LDTs were collecting and reporting the information in the diabetes registers. However, this is only part of the information required to evaluate and plan diabetes services in their districts. None of the LDTs that we spoke to were receiving information on the numbers of patients attending specialist diabetes services or on the services’ capacity to deliver.

4.63
In addition, four LDTs were finding it difficult to get DHBs to listen to their recommendations. With the disbanding of the National Diabetes Working Group during the 2003/04 year, there is now no national analysis of, or “voice” for, the recommendations put forward by these teams.

Recommendation 11
We recommend that district health boards work with local diabetes teams to carry out a more robust analysis of supply and demand for diabetes services at both the primary and secondary care levels, so that any shortages in services provided at both the primary and secondary care levels can be identified.
Recommendation 12
We recommend that the Ministry of Health and district health boards review the role of the local diabetes teams to establish how these teams are best able to adequately fulfil the role of providing advice on the effectiveness of healthcare services for people with diabetes.
Recommendation 13
We recommend that the Ministry of Health and district health boards consider how to improve the adoption of the local diabetes teams’ recommendations.

1: Page 19.

2: The Health Payments, Agreements and Compliance Unit in the Ministry of Health, which supplies services to health funders.

3: Page 59.

4: For the purposes of our report, we have included this group as an LDT in our discussion on LDTs in paragraphs 4.46 to 4.53, because its functions included those of an LDT.

5: DHBs are required to establish a Community and Public Health Advisory Committee under the New Zealand Public Health and Disability Act 2000. It is a statutory advisory committee and may comprise both board members and members of the public. The committee provides a key means for community voices to be heard.

6: Eye screening using retinal photography, done every two years for people with diabetes who do not have retinopathy (a disease in the retina of the eye).

page top