Part 3: Improving and assuring the quality of case management interactions
3.1
In our 2014 report, we found that the systems and processes ACC used in case management did not support the delivery of a consistently fair service that focused on the needs of claimants.
3.2
We also found that ACC did not consistently use its own tool for reviewing and assuring the quality of case managers' engagements with claimants. The quality assessments of those engagements often lacked enough input from claimants.
3.3
In our 2014 report, we recommended that ACC:
… strengthen the overall approach and tools that it uses to guide, monitor, review, and manage the quality of its case management services – to ensure that people consistently receive treatment and rehabilitation services of the highest practicable standard and fairness in line with the requirements of the Code of ACC Claimants' Rights.
3.4
In this Part, we discuss how ACC has responded to this recommendation.
Summary of findings
3.5
NGCM promotes claimant- and outcome-focused decisions that factor in the claimant's needs. Through NGCM, there is clear potential to improve claimants' experiences.
3.6
Although ACC has been implementing the NGCM approach since 2017, it has not fully embedded the approach yet. ACC's early evaluation indicates that claimant experience and recovery outcomes are improving. However, it is too early yet to assess whether NGCM's benefits will be fully realised.
3.7
ACC told us that it has planned three internal assurance reviews that focus on how it has implemented and embedded NGCM. We strongly encourage ACC to fully evaluate NGCM once it is embedded and to report the results of that evaluation publicly. This would provide ACC and the public with greater assurance that NGCM has improved ACC's capacity and capability to deliver high-quality case management services.
3.8
ACC's new quality assurance framework has the appropriate elements for ACC to understand the quality of case managers' engagements and performance. These include assessing the case manager against ACC's broad expectations of the way they should interact with claimants, as well as how they perform their role.
3.9
We understand that regular and timely claimant feedback and metric-based performance data also inform these assessments – for example, how often a case manager fails to gather the appropriate claimant authority forms. Any person-to-person interaction, email, or task (or combination) that engages with claimants or affects their experience can be used to inform these assessments.
3.10
The results of the assessment are discussed with the case manager to identify performance improvements and coaching needs. In this way, the framework helps build staff capability to deliver the expected quality of case management.
3.11
However, the framework is new and yet to be fully embedded in all of ACC's case management services. It is not yet possible to know whether the framework will be fully effective. We encourage ACC to continue embedding the framework and use feedback to ensure continuous improvement.
3.12
ACC told us that it has improved how it collects customer feedback and derives insights. It also told us that it actively turns those insights into actions that can lead to improvements in its services – for example, by collecting feedback at more regular points on the claimant's pathway to recovery.
3.13
However, ACC acknowledges that it still needs to do more to become truly claimant-centred. It told us that building a "culture of feedback" was a critical part of this.
Supporting more efficiency in case management
3.14
An important aspect of NGCM is its emphasis on claimants who have greater needs. This includes more meaningful conversations with claimants so that ACC better understands the full range of their needs. This should lead to more effective decisions and the best practicable outcomes.
3.15
To enable case managers to focus on the needs of claimants, ACC has reallocated some administrative tasks. It has also enabled claimants with less complex needs to have more control over their recovery. Both of these improvements enable staff to focus more of their time on those who need it most.
3.16
ACC has automated parts of how it assesses an initial application and identifies the appropriate case management approach. The system now uses business rules to instantly and consistently determine the acceptability of claims that meet specific criteria as soon as they are lodged.
3.17
The system uses a statistical model to determine the probability that ACC will accept a claim and is informed by data from 12 million anonymised lodged claims.
3.18
ACC is also currently replacing its payment system. The new payment system will reduce the amount of manual inputs required by ACC staff. The system links directly to the Inland Revenue Department, which means that, in most cases, ACC will no longer have to ask employers for claimants' earnings information or have to continually follow this up when the information has not been provided.
3.19
In March 2018, ACC introduced a digital service called MyACC. Claimants can access MyACC through their desktop or mobile devices to manage aspects of their claim online. Claimants can now apply for entitlements online, including weekly compensation. They can also check information about their claim and entitlements.
Measures to assess improvements in claimant experience
3.20
We have looked at ACC's measures to see whether we are able to assess how NGCM has affected claimants' experiences. The two main sources of information are:11
- claimant experience and rates of complaint that differentiate NGCM from the previous system; and
- publicly reported indicators, including a measure of claimants' trust and confidence in ACC (the Client (Claimant) Net Trust Score) and claimants' views of whether they consider that ACC focuses on achieving the best possible outcomes for claimants.
Claimant experience and complaints
3.21
ACC provided us with three measures that it considers help it measure claimant experience under NGCM. These show some improvement in claimant experience. Figures 3 compares claimant experience results under NGCM with the overall result.12 Figure 4 compares results under the NGCM and the previous case management model.
Figure 3
Comparing claimant experience results under NGCM and overall results across all cases
| NGCM result | Overall result | |
|---|---|---|
| Claimant satisfaction* (quarter to 30 June 2020) | 84% | 80% |
| Client (Claimant) Net Trust Score** (year to 30 June 2020) | +37 | +31 |
* Satisfaction measures the extent to which claimants are satisfied with their dealings with ACC.
** Client (Claimant) Net Trust Score is a measurement of the extent to which claimants have trust and confidence in ACC. Individuals rate trust and confidence in ACC on a 0-10 scale. The net trust score is the proportion of respondents with high trust and confidence (scores of 9-10) less the proportion of respondents with low trust and confidence (score 0-6). Scores can range from -100 to +100. Scores below zero indicate a higher proportion of respondents with low trust and confidence. Source: Accident Compensation Corporation.
Figure 4
Ratio of NGCM complaints monitored through the key customer feedback channel to active claims and of the previous case management model
| NGCM (measured in the March 2020 quarter) | Previous case management model | |
|---|---|---|
| Ratio of complaints to active entitlement claims | 1.3 : 1000 | 4.0 : 1000 |
Source: Accident Compensation Corporation.
3.22
Information in both Figure 3 and Figure 4 indicates that claimants managed under NGCM appear to be more satisfied with the new approach.
3.23
It is important to note that these results are only early indications of the potential for improved claimant experience with the NGCM approach. Most of this information was gathered when the new approach was still being rolled out.
3.24
ACC told us that, for other performance measures, it is difficult to determine which information on individual claimants should be directly attributed to NGCM or the previous model. This is because many claimants moved from the previous model to NGCM during their recovery. This applies particularly to claimants with a longer-term recovery. ACC expects to be in a better position to assess the changes from the end of 2020.
Publicly reported indicators
3.25
Figure 5 includes the results of two of ACC's publicly reported indicators: Client (Claimant) Net Trust Score and another measure on claimant perception of ACC's focus on their outcomes.
3.26
ACC publicly reports the overall Net Trust Score for all claimants. The scores reported for the previous two years were reasonably consistent, with an improvement in the score in 2019/20.13
3.27
Figure 5 shows the Client (Claimant) Net Trust Score and an additional measure about claimant experience that ACC reported publicly only from September 2019.
Figure 5
Claimant experience measures that are publicly reported
| Measure | 2017/18 | 2018/19 | 2019/20 |
|---|---|---|---|
| Client (Claimant) Net Trust Score | +25.0 | +24.0 | +31.0 |
| (The percentage of claimant respondents who felt that) ACC is focused on the best possible outcomes for clients (claimants) given their situation. | 77% | 77% | 79% |
Source: Adapted from the Accident Compensation Corporation's publicly available reporting and from information provided by the Accident Compensation Corporation.
Assessing this information
3.28
ACC has been encouraged by the results showing improvements in claimant experience under NGCM so far. ACC told us that these results cover the early stages of rolling out NGCM and its testing and learning about the approach in Launch Pad. ACC told us that the results provide strong evidence of improvement and give it high confidence in NGCM.
3.29
Measures of claimants' experiences and their rehabilitation outcomes will be important indicators of NGCM's success.
3.30
Although we acknowledge that these results are promising, we are mindful that there is not yet enough data to support a robust comparison between NGCM and the previous case management approach.
3.31
ACC needs to continue to closely monitor all of these performance measures and reflect on what they indicate about NGCM's effectiveness.
3.32
We strongly encourage ACC to comprehensively evaluate NGCM once it is fully embedded. The indicators of customer experience and the views of claimants experiencing NGCM will be a critical part of this evaluation.
3.33
Making the results of the evaluation and other measures of performance readily accessible to the public will also provide transparency on how well NGCM is meeting its objectives and give a better view of the value for money of ACC's investment in the new approach.
Assessing case management quality and building case management capability
3.34
ACC told us that its new Customer Experience Quality (CXQ) framework will provide assurance that claimants experience consistent and high-quality interactions with case managers. ACC created the framework only recently, and it is still being embedded in ACC's case management services.
3.35
CXQ views quality from the claimant's perspective. It centres on four behavioural responsibilities that case managers are expected to demonstrate when working with claimants: "Care", "Can", "Know", and "Understand". ACC considers that applying the framework empowers its case managers to explore and understand each claimant's individual needs and respond in the most effective way.
3.36
CXQ is part of an overarching Quality and Performance framework broadly described as "Ways of Working". The Quality and Performance framework brings together CXQ assessments with information on other elements of case managers' performance.
3.37
CXQ assessments are combined with broader claimant feedback, work performance metrics, and exception reporting to inform a wider assessment of a case manager's overall performance and development needs.
3.38
Team leaders review different engagements with claimants (such as in person, on the phone, or by email) and provide feedback to case managers on the quality of the engagements.
3.39
Coaching conversations with staff reinforce positive aspects of interacting with claimants and identify opportunities to develop the capability of case managers and make improvements. Coaching conversations are held in different ways and at different times, both formally and informally.
3.40
Case managers can also be paired with Practice Mentors to help develop their capability. Practice Mentors offer practical advice to help case managers consider situations from different angles and make good decisions by being better informed.
3.41
The Performance Team gathers and aggregates information from CXQ and the Quality Performance framework to identify broader insights to inform organisational improvements.
3.42
ACC's leadership team receives regular reporting on both aggregated and specific case management information. Information for this reporting is often sourced as recently as the previous evening. This provides the leadership team with up-to-date information about service quality, case management performance, claimant and staff feedback, and what is being done to address issues and make improvements.
Using claimant feedback to derive insight and direct it for action
3.43
Focusing on the perspectives of claimants is one of the main ways that ACC is moving to a more claimant-centred approach.14 This has been described as building a "culture of feedback". Claimants now have a wider and more immediate range of options to provide feedback on their experiences. Through its Customer Feedback Strategy, ACC aims to improve its ability to more readily seek, hear, and act on customer and claimant feedback.
3.44
A new technology platform called "Heartbeat" plays a lead role in helping ACC improve how it works with and supports claimants. Heartbeat solicits feedback from claimants (and providers) at important points on the claimant's pathway (called "moments of truth"). It seeks this feedback through surveys sent by emails and text messages.
3.45
Feedback from Heartbeat is the main way that ACC accesses claimants' perspectives on the quality of case management services they receive. ACC also uses this information to improve the quality of case management at an organisational level.
3.46
ACC considers that the immediacy of channels such as Heartbeat is critical to identifying and addressing existing or potential issues early. Staff can be alerted to potential issues and "check in" with the claimant to resolve them early.
3.47
ACC demonstrates a willingness to learn from and reinforce good practices identified through Heartbeat. For example, instances of positive feedback in Heartbeat are shared at ACC's leadership meetings, where the leadership team can consider how to apply good practice more widely. ACC leaders can send personalised emails to acknowledge staff who have been praised in feedback received through Heartbeat.
3.48
The Customer Insights and Experience team oversees the feedback channels. The team uses the information from feedback channels and other performance data to:
- detect signs of potential dissatisfaction for timely intervention with individual claimants; and
- provide data that the team can use to identify systemic themes.
3.49
The Customer Insights and Experience team have done a lot of work to understand what drives two aspects of claimant experience that ACC measures:
- Satisfaction – a measure of the extent to which claimant respondents are satisfied with their dealings with ACC; and
- Client (Claimant) Net Trust Score – a measure of the extent to which claimant respondents have trust and confidence in ACC.15
3.50
Understanding these drivers enables the Customer Insights and Experience team to help identify improvements.
Short-loop and long-loop feedback
3.51
ACC categorises feedback so it can deal quickly and directly with straightforward issues and appropriately consider more complex insights that point to opportunities for systemic improvement. The feedback is categorised as being either "short loop" or "long loop".
3.52
Broadly speaking, short-loop feedback involves feedback that can be addressed, or improvements that can be made, quickly and easily. Short-loop feedback might refer to the needs of a particular claimant, the activities of a particular team or location, or a small process change.
3.53
Short-loop feedback might be a concern a claimant has expressed about the management of their case. This specific feedback is directed to the staff member who last interacted with the claimant. So that the claimant's feedback does not "fall through the cracks", that staff member retains ownership of the issue until it is resolved.
3.54
Long-loop feedback generally involves feedback that has a broader application or impact or involves more extensive changes.
3.55
Long-loop feedback is directed to a process referred to as "continuous delivery". Dedicated teams summarise feedback and insights and funnel these into "service imperatives".
3.56
Service imperatives are high-level aspects of service (including case management) that are assessed as having particular value to the claimant. Proposed improvements are investigated and then prioritised for progression.
3.57
The Customer Insights and Experience team also monitors the rate at which changes are implemented. This is also a strong focus for ACC's leadership team.
Embedding a day-to-day focus on learning
3.58
Staff in local teams gather every morning to learn from each other and discuss how they can improve the quality of service. These meetings are referred to as "Buzz meetings".
3.59
Buzz meetings are held at every level of service delivery, including at leadership level.
3.60
Buzz meetings are the main way that teams share and learn from each other's experiences and ideas, as well as discuss communication from ACC leadership. Positive feedback from claimants and good practice is also shared. ACC told us that these meetings have helped staff adjust to the new focus on "claimant-centred" case management.
11: We have not audited the methodology or information related to the measures and results as part of this follow-up work.
12: Where we refer to an overall result, this includes information from claimants under NGCM and claimants under the previous model.
13: This 2019/20 information is not currently publicly available.
14: In paragraphs 4.13-4.19, we also discuss how ACC is getting feedback from claimants with complex needs.
15: We discussed this measure earlier in this Part (see paragraphs 3.25-3.27 and Figure 3).