Part 4: The Pivotal Role of the Case Manager
Introduction
4.1
In paragraph 2.15 on page 30, we concluded that the following activities potentially have the greatest effect on case manager accuracy:
- case managers’ caseloads;
- checks on case managers’ performance (the 5+5 checks);
- team coaches’ support and training of case managers; and
- case manager performance assessments.
4.2
In our view, it is important that the Ministry understands the effect that these activities have on accuracy.
4.3
The Ministry has information about the volumes for each of these activities, and reports this information internally, although we found that some comparative volume information could be more systematically collected and circulated. However, the Ministry does not have information on the effects that these activities have upon accuracy.
4.4
This lack of information on effect extends beyond these activities – such as to policy development, which did not, for instance, consider the effect on accuracy when creating the new Domestic Purposes Benefit Programme in 2002. However, WINZ and the Ministry’s Sector Policy Group are working closely together (under the Future Directions Programme) to simplify the benefit system. This will have an effect on accuracy.
4.5
In this part, we explore some of the potential implications for accuracy of each of the four activities.
Case Managers’ Caseloads
4.6
High caseloads can create pressures on case managers that may lead to lower levels of accuracy. This effect can be mitigated, such as by improving case managers’ skills, or improving support systems to enable them to cope better. However, there is a limit to the volume of cases that an individual case manager can handle without putting accuracy at risk.
4.7
Decisions on caseloads are made at three levels:
- national;
- regional; and
- service centre.
National Level Resource Allocation
4.8
The Ministry distributes funding for Service Delivery in a standard manner across regions according to caseload. It uses a “pure” model for the distribution, in that most of the funding is allocated according to the number of clients12 that each region has on its register. There are no substantial adjustments for other factors related to accuracy, and some of the adjustments that are made are also related to pure caseload (e.g. ratio of team coaches to case managers).
4.9
The Ministry believes that it is best to have almost all funding distributed to Regional Commissioners and for them to manage within their allocation. Therefore, only a very small amount – less than 0.5% – is retained by National Office as a ‘bidding fund’ that would potentially be available to deal with short-term pressures in particular regions.
4.10
We concluded that regional variations could place pressure on accuracy in a number of ways that affected caseloads. For example, a region could have a disproportionate number of beneficiaries with complex needs requiring assessments for different benefit types that can be particularly time-consuming for case managers.
4.11
Using the number of applications that a region receives for Special Needs Grants and Advances as an indicator of the complexity13 of beneficiaries’ needs, we found that there are large differences between regions (see Figure 7 on the opposite page). The lowest proportion of applications for a Special Needs Grant was only 52% of the number of working-age beneficiaries, and the highest proportion of applications was 136%.
Figure 7
Applications for Additional Assistance 1999 to 2002 as a Proportion of Working-age Beneficiaries
| |
Lowest Regional Ratio % |
Highest Regional Ratio % |
|---|---|---|
| Special Needs Grants | 52 | 136 |
| Advances | 41 | 91 |
4.12
We concluded that the average complexity of benefits influences, but does not uniquely determine, a region’s accuracy rate. Because there are a number of factors that influence overall accuracy, it is possible that a region can have a complex population of beneficiaries and still record high accuracy figures by intensifying other activities. However, because the Ministry does not know the effect on accuracy of individual activities, it would be difficult to determine whether and how a region’s accuracy figure has been achieved.
Caseload Decisions at Regional Level
4.13
Within the national funding allocated to them, Regional Commissioners are free to set caseloads for particular service centres as they see fit – they are not expected to stick rigidly with the national caseloads on which the funding was based. The main constraint on Regional Commissioners is the expectation that they will meet KPI targets – so their decisions on where to place resources need to be made with this in mind.
4.14
For example, Regional Commissioners may vary:
- the degree of specialisation in their service centres – Figure 8 on the next page illustrates the two extremes of specialisation and non-specialisation; and
- the proportion of investment in team coaches (who support and train case managers – see paragraphs 4.36-4.44) – which may need to be greater than the proportion indicated in the national model – for example, in an area that has relatively high staff turnover and high proportions of relatively new staff.
Figure 8
Options for Managing Caseloads – To Specialise or Not?
4.15
In the Wellington Region we found that some of the savings from having a high turnover and relatively high numbers of new staff starting had been used to establish a new post at regional level to concentrate on risk management – partly in order to address having high numbers of new and relatively inexperienced staff.
Caseload Decisions at Service Centre Level
4.16
Each service centre manager can also flexibly deploy staff. Variations from the regional pattern must be approved by the Regional Operations Manager to ensure that any changes are consistent with the region’s goals.
4.17
Variations at service centre level are usually adjustments to caseloads to take account of case manager experience – for example, new case managers are often given lighter caseloads until they attain the necessary level of skill. Types of work allocated may also be varied – for example, a member of staff who is considered to be efficient, accurate, and conscientious in decision-making and recording might deal with a disproportionate number of the “once-only” changes to circumstances that beneficiaries report to the service centre.
Conclusions
4.18
The Ministry’s arrangements for allocating resources and devolving responsibility for managing them are consistent with giving managers appropriate responsibility for what they control and making them accountable for results.
4.19
There is limited sharing of information between regions. Benefit administration is likely to be improved if reliable assessments of the benefits and risks to accuracy of different approaches to improving accuracy were to be routinely shared with other regions rather than remaining as lessons held within particular regions.
4.20
Because accuracy results are only reported annually to the national level, any region that adopts an approach that turns out to alter accuracy will not come to national management’s attention in a timely way. In the meantime, if the impact is to reduce accuracy, beneficiaries may be disadvantaged for a considerable length of time.
4.21
Regions are resourced according to the number of clients. Those regions with a greater than average number of beneficiaries with complex circumstances could be disadvantaged, because the additional work arising from dealing with these beneficiaries could leave them less able than other regions to meet their accuracy target.
| Recommendation 3 |
|---|
| We recommend that – The Ministry continues to promote sharing of information among regions and with National Office on approaches to staff deployment and managing caseloads. |
“5+5 Checks” on Case Managers’ Performance
4.22
Case managers’ pay and promotion are linked to the achievement of individual targets that are in turn linked to those set at regional level.
4.23
One of the ways in which the Ministry monitors case managers’ individual performance on accuracy is through “5+5 checks”. These involve the team coach checking the accuracy of five applications and five reviews completed by the case manager within the last month. The team coach has discretion to undertake additional checks to ensure that the five applications and five reviews originally chosen are not unrepresentative of the case manager’s work.
4.24
For a new case manager, the team coach checks every application and review until the case manager reaches a prescribed level of proficiency, after which the 5+5 checks are applied. If the case manager’s accuracy deteriorates at any point, the number of checks is increased until the service centre manager is satisfied that the problem has been rectified and the case manager has returned to the prescribed level of proficiency.
4.25
As a minimum, the 5+5 checks follow those required for the Accuracy Reporting Programme (“ARP” – see Part Five), and in some regions also involve wider checks to cover the general work practices of case managers.
4.26
Our technical referee for this audit identified some significant inference risks when 5+5 checks are used to assess the performance of individual case managers. The risks derive from the very small sample taken of each case manager’s work. The problem is manageable but, in our view, does need to be addressed.
How Do 5+5 Checks Assist Accuracy?
4.27
5+5 checks assist accuracy in two main ways:
- by providing the information required to measure individual case manager accuracy and to identify any need for improvements; and
- by providing the same information for the same purposes at service centre level to enable managers to monitor and improve the accuracy of processing within the centre or within groups of case managers.
4.28
Because 5+5 checks are continuous, they enable the service centre manager to monitor performance throughout the year. By identifying trends in 5+5 check results, the manager can identify training needs or raise awareness of processing aspects among case managers and their team coaches.
4.29
At a large service centre, the number of 5+5 checks might be about 150 a month. The same centre would probably be asked to check only four or five applications or reviews for the Accuracy Reporting Programme.
4.30
The Ministry does not use 5+5 checks for national reporting, because they are less objective and less consistent than the checks made under the ARP. Not all regions review the results between service centres, and there is no review of the results between regions or between similar service centres in different regions.
4.31
After the bulk of our field work was carried out, the Ministry informed us that it has initiated a procedure of compiling Team Coach data (5+5 checks) from time to time to provide context or elaboration to nationally compiled data and to inform other quality improvement mechanisms such as Helpline or Quality Control.
4.32
If ARP and 5+5 data could be validly “combined”, a better picture of performance at regional level could be obtained. One method by which the ARP and 5+5 sampling systems might be linked involves what is called ‘double sampling’. In essence, the method involves using a special sample to determine the nature of the relationship between measurements of accuracy derived from ARP and 5+5 approaches and using this relationship to infer regional accuracy information from 5+5 data.
Conclusions
4.33
KPIs are a central feature of the Ministry’s culture. Case managers have clear, measurable targets to aim for in their day-to-day work.
4.34
The 5+5 checks provide valuable information on accuracy in service centres, and are an effective means of achieving and maintaining accuracy. In those regions that look at the results across service centres, they also provide useful comparative information.
4.35
The lack of more high-level comparative analysis of the results – particularly between service centres dealing with similar beneficiary profiles and problems – is a missed opportunity to compare performance and to transfer lessons.
Role of Team Coaches
4.36
The role of team coaches is wider and more supportive than just checking case managers’ work. For example, a case manager can book a time with the team coach to discuss a specific problem. Team coaches also identify shortcomings through the 5+5 checks, and work one-on-one with the case manager to help improve their performance.
4.37
Team coaches can also manage training of groups at service centre level (held on most Wednesdays throughout the Ministry). This includes training to bring in new Ministry-wide developments as well as issues specific to the service centre – which means that the team coaches generally have a good understanding of Ministry-wide changes.
4.38
All the team coaches we spoke to saw their role as fostering a culture of continuous improvement in their service centres. Team coaches are generally experienced case managers, so they understand the pressures and difficulties that case managers face and have the authority and experience to gain respect.
4.39
Staff throughout the Ministry – case managers, service centre managers, and Regional Commissioners – acknowledge the importance of team coaches in achieving and maintaining accuracy. Case managers we spoke to praised the concept of team coaches, and said they valued the support and guidance they provide.
How Many Team Coaches Are Required?
4.40
The Ministry’s national funding model provides for a ratio of one team coach to every 25 case managers – a ratio that was set in 1995 after a review of the numbers needed to maintain targeted performance. In 2000 the Ministry re-allocated funds to ensure that the ratio did not drop below this level.
4.41
Regional Commissioners can alter this ratio as they see fit. Some regions have reduced the number of team coaches and increased the number of case managers – enabling them to marginally reduce the caseload per case manager. Other regions have done the reverse – increased the number of team coaches, funded by a reduction in case managers and higher caseloads.
4.42
In addition, the role that a team coach performs can be varied – such as to only undertake checking or training, or to take on other administrative tasks.
4.43
We found that, before 2002, the Ministry did not seek to collect information on different practices in relation to numbers and the role of team coaches. Experience during 2002 in some regions suggested that varying the number of team coaches could be counterproductive and this information was shared among all regions.
Conclusion
4.44
Team coaches provide important and responsive support and training to the case managers. We consider that not sharing information on how regions use team coaches is (as with the 5+5 checks) a missed opportunity to compare performance and to transfer lessons.
| Recommendation 6 |
|---|
| We recommend that – The Ministry continues to promote sharing of information among regions and with National Office on approaches to the use of team coaches. |
Case Manager Performance Assessment
4.45
The KPI for processing accuracy appears throughout the Ministry – right down to individual case managers’ performance agreements. Each case manager is also expected to meet a large number of other performance standards as part of their performance agreement. The majority of these other performance standards are directed at the employment element of the case manager’s duties – for instance, the number of long-term unemployed people the case manager has placed in work.
4.46
We found variation between regions in how performance pay is related to accuracy. Some regions see an accuracy standard as a compulsory target – a case manager must achieve or better it before any bonus will be paid. The aim is to ensure that, no matter which case manager handles a beneficiary’s case, a minimum level of accuracy can be expected.
4.47
Other regions assess performance more broadly across all the performance standards. A case manager might have a low accuracy result (below the 90% level at which the Ministry defines a case manager as competent) but still receive a bonus if they have performed at a high level in respect of employment targets.
Conclusion
4.48
The Ministry has a systematic approach to linking individual performance of its case managers to its KPI for processing accuracy. However, the way some regions assess performance creates a risk that staff will not see accuracy as a high priority, potentially leading to an inconsistent level of accuracy between case managers and regions. We do note that the Ministry audits performance assessments from time to time and we encourage this to continue.
12: This is greater than the number of beneficiaries, because it includes people not on a benefit whom the Ministry is helping to find work.
13: We acknowledge that this is a “rough” measure. To be more precise, it would be necessary to separate out regional differences in eligibility from regional differences in the interpretation of eligibility.
page top