Part 5: Using complaints and other comments to keep improving services
5.1
In this Part, we look at how the Ministry analyses complaints and other comments to inform efforts to continuously improve services.
Summary of our findings
5.2
Work and Income and StudyLink analyse complaints and other comments to help improve services. There are some good examples of this.
5.3
However, the Ministry has no Ministry-wide analysis that could lead to improvements throughout the system. Often improvements are focused on providing training or coaching to staff. Analysing complaints strategically to identify any systemic problems would help the Ministry.
A "one-MSD" approach
5.4
The Ministry is looking to change to a single operating model, which it describes as "one-MSD". The Ministry points out that its current operating model has resulted in duplicated resources and inconsistent processes, creating inefficiencies and barriers to delivering ideal results. The Ministry wants to move to an integrated operating model that is centred on people and accessible, to help the Ministry better meet people's needs and reduce the number of times that people need to interact with the Ministry.
5.5
The Ministry has a project under way to identify how complaints could be better managed under a "one-MSD" approach. We took this into account as we looked at how the Ministry uses complaints information and other comments to help make better decisions.
5.6
We consider that an integrated approach to complaints would help the Ministry to see any inconsistencies in good practices as well as identify areas for service improvement.
5.7
While moving towards an integrated approach, the Ministry needs to consider what its business needs are in relation to an information management system for complaints management. The goal should be for any information management system to support rather than drive a Ministry-wide approach to complaints.
Analysing complaints and other comments to help improve services
5.8
The Ministry as a whole does not analyse or monitor trends in complaints. However, some sections carry out their own analysis. For example, Work and Income managers are expected to regularly monitor trends in complaints so that improvements can be made. Complaints in HIYA can be searched and reported by the number of complaints, how many are resolved, how many come through a service centre, and type. Reports can be prepared at local, regional, and national offices.
5.9
Although reports can be prepared at any office, Work and Income regional offices are responsible and accountable for monitoring and analysing trends. All regional offices have analysts whose role involves analysing regional complaints information. We saw examples of some of the regional reports. As mentioned in paragraph 4.7, the reporting process is still being put into practice and these reports vary in detail.
5.10
StudyLink's Report Writing Team is responsible for monitoring and analysing all complaints about StudyLink.
5.11
Senior Services staff carry out some analysis at the National Office. However, they do not have enough data to carry out detailed analysis.
Numbers and types of complaints
5.12
The Australian National Audit Office states that:
Complaints information can provide an agency with a picture of weaknesses in existing programs, policies, and service and program delivery. As such, regular and accurate reporting on complaints is necessary to prompt an agency to monitor emerging trends and respond accordingly.3
5.13
We saw examples of Work and Income and StudyLink staff monitoring the number of formally recorded complaints.
5.14
Senior Services has no formal process for monitoring the number of complaints it receives. Some Local Services Managers keep registers so that they can see the number of complaints they receive. Complaints to Senior Services are perceived to be "generally rare". However, without a formal way to record this information, it is difficult to confirm that they are rare.
5.15
Senior leaders' monitoring and overseeing the number of complaints that all sections receive would allow the Ministry to see clearly how many complaints it receives and resolves, and how well the Ministry responds to complaints.
5.16
Sections analyse the types of complaint they receive in different ways and to varying extents.
5.17
The most common types of complaint for Work and Income (see Figure 7) are about:
- interpersonal skills/staff attitude (30% in 2013);
- action taken by a staff member that caused hardship or distress (29% in 2013);
- information provided (18% in 2013); and
- timeliness (15% in 2013).
Figure 7
Types of complaint recorded by Work and Income, 2010 to 2013
Source: Ministry of Social Development.
Note: Only the five most numerous types of complaint are shown.
5.18
Senior Services has no formal process for recording complaints. The information we gathered indicates that the most common types of complaint that Senior Services receives are the same as for Work and Income.
5.19
Staff told us about a problem with the Senior Services national work queue. This system is designed to help staff process work faster so that people's problems and applications can be dealt with faster. However, some staff perceive that the work queue is causing people to wait longer and the number of complaints to increase. Other staff consider that the system is working as intended. It is difficult to know how widespread the problem is when Senior Services does not record this information systematically.
5.20
A strong theme that came through in our interviews was that some of the Ministry's staff perceived that frontline staff often communicated poorly, usually about decisions about entitlements. This poor communicating leads to complaints or reviews of decisions.
5.21
Some of the factors that contributed to complaints about communicating poorly included:
- the way the message was delivered or conveyed;
- a skill deficit – for example, staff being unable to explain complex matters; and
- staff lacking time to explain complex matters.
5.22
Staff said that some people's strong sense of entitlement or wanting confirmation from a more senior member of staff could contribute to complaints about poor communication.
Linking information about complaints with other data
5.23
The Ministry carries out several client satisfaction surveys. The Work and Income survey asks about "drivers of satisfaction". The State Services Commission has identified the main factors that drive satisfaction with public services. They are:
- the service experience met your expectations;
- staff were competent;
- you were treated fairly;
- you feel your individual circumstances were taken into account;
- staff kept their promises – they did what they said they would do; and
- it is an example of good value for tax dollars spent.
5.24
Work and Income surveys about 18,000 clients each year. Those surveyed include working-age beneficiaries and Senior Services' clients. StudyLink carries out a satisfaction survey of about 3000 clients each year.
5.25
The Work and Income surveys show that overall client satisfaction has remained stable at about 85% since 2009. Satisfaction with the contact centre is marginally higher than with frontline services. The survey records how dissatisfied people are and analyses the reasons for the dissatisfaction by contact centre and frontline service.
5.26
The StudyLink survey also shows that, overall, client satisfaction has remained at about 85% since 2010. The survey also measures satisfaction with different channels for contacting StudyLink and analyses reasons for overall dissatisfaction.
5.27
As part of analysing complaints, some sections compare information about complaints with the results of client satisfaction surveys. Managers find this information valuable. Examples of this include:
- For all Work and Income contact centre sites, the most common complaint is about the information provided to clients. Client satisfaction results for the contact centre showed that the contact centre was below its performance standard for questions about staff knowledge, confidence, and people feeling that they had been told everything that they needed to know.
- StudyLink's student satisfaction survey results identified that the website was difficult to use. The analysis of complaints identified that the website's messages were unclear.
5.28
Managers can compare client satisfaction survey results with results of reviews of decisions. We were told that the analysing the data helps managers to understand the quality of their service. It also helps give the Ministry a picture of how it is doing overall.
5.29
Senior Services also monitors the Kiwis Count survey results to gain a broader understanding of client satisfaction.4
Using information about complaints and other comments to improve services
5.30
We saw several examples of the Ministry using information from complaints and other analysis to improve services. These examples were local, regional, and national. Some local and regional examples included:
- preparing materials to help train and coach staff who provided poor service;
- improving information for staff; and
- changing the design of the office to make people more comfortable.
5.31
As one manager stated: "If [we] stuff up, [we] own it, fix it, learn from it, and move on." Of those surveyed for our audit, 61% felt that it had been worthwhile to raise their problem with the Ministry.
5.32
We saw system-level improvements that the Ministry made in response to complaints and other comments.
Appointment times for Work and Income
5.33
Work and Income received complaints about delays and waiting times for appointments. The client satisfaction survey highlighted this. The survey identified that waiting times affected satisfaction rates.
5.34
Work and Income Service Centres now have a tool called "Queue Manager" that monitors how long people have been waiting for their appointment. People with booked appointments are assigned, where appropriate, to the next available case manager when they arrive. People who have walked in without a booked appointment wait until the next available appointment.
5.35
The tool monitors how long people have to wait and shows how long a case manager has been with a person. The tool allows managers to see trends and peak times so that they can plan and adjust staffing for those times.
StudyLink website
5.36
StudyLink prefers people to use its website. However, the client satisfaction survey and complaints identified that the website was difficult to use (see paragraph 5.27). StudyLink began a project to improve the website. The changes, including a new search engine, went live before Christmas 2013 and further changes are expected. StudyLink convened focus group meetings with students while improving the website to check that changes would meet students' needs.
Improving how to analyse complaints to reveal any systemic problems
5.37
As noted in paragraph 5.30, managers will often provide training or coaching to staff in response to a failure to deliver satisfactory service. In some instances, providing training to staff can improve services.
5.38
However, as the Australian National Audit Office points out, although some complaints deal with occasional breakdowns in service delivery, other complaints are systemic in nature and can suggest that policies or procedures need to be changed. In our view, the Ministry needs to be able to analyse complaints information from throughout the Ministry to identify any systemic problems that need to be addressed.
5.39
We saw several examples of what the Ministry calls "service failure" and a respected occupational psychologist calls "failure demand".5 In one example, staff were found to have failed to provide adequate service on 11 separate occasions. Another example resulted in "unjustifiable service failure" on 16 separate occasions. In both instances, staff members were found to be at fault. Feedback was given to staff who provided inadequate service. The examples were used as internal training tools.
5.40
The number of service failures in the examples we saw suggests to us that it is not just individual staff at fault. Providing further training alone will not address problems that might be systemic.
5.41
As one person noted in one of the Ministry's client satisfaction surveys:
They haven't been able to give me information or they have given me unreliable or contradictory information. There is poor communication between the people on the phone and the people that make the decisions.
5.42
The occupational psychologist notes that, sometimes, staff performance accounts for a small proportion of the responsibility for a breakdown in service. The rest of the responsibility might lie with the system.
5.43
The following example shows how more in-depth analysis can be carried out. It shows how staff performance forms only part of overall service quality. The latest quarterly report available for Work and Income's satisfaction survey in 2013 showed that satisfaction with staff members is consistently higher than satisfaction with the quality of the services delivered. The survey suggested that staff service was only one aspect of total service delivery. It was likely that other factors, such as Work and Income entitlement policies or financial help received, influence people's perceptions of the quality of service delivery.
5.44
The survey identified that waiting times for appointments appear to strongly influence overall satisfaction (see paragraph 5.33). Figure 8 shows the survey results for quality of service delivery for Work and Income, by quarter, from 2009 to 2013.
Figure 8
Clients' satisfaction with the quality of service delivery of Work and Income, 2009 to 2013
Source: Ministry of Social Development.
Initiatives under way in the Ministry
5.45
Initiatives that the Ministry has under way as part of "one-MSD" include:
- the simplification programme;
- the "building blue" initiative;
- the central data hub; and
- a Ministry-wide approach to complaints.
The simplification programme
5.46
The simplification programme aims to simplify the Ministry's processes for interacting with people to be more efficient and focused on people, and to leave frontline staff free to deal with people. The programme will explore changes to the Social Security Act 1964 (where needed) to achieve this.
Building blue initiative
5.47
Some staff see the Ministry as being too concerned with avoiding risks, highly prescriptive, and having a "command and control" culture. The "building blue" initiative is a culture change programme intended to move the Ministry towards a more constructive culture and leadership style.
Central data hub
5.48
The Ministry is looking to modernise its information and communications technology platform and establish a data hub. The aim is to bring together information about a person from different sections of the Ministry that deliver services to the public. This should enable the Ministry to share more information between agencies, to develop the capability to analyse consolidated data, and to learn what works best for the people to whom it delivers services.
| Recommendation 1 We recommend that the Ministry of Social Development prepare an approach that enables it to collect information about complaints from throughout the Ministry, monitor and report this information to the Ministry's Leadership Team consistently, and use the information to improve its systems and how it delivers services. |
| Recommendation 2 We recommend that the Ministry of Social Development improve the clarity of standards and response times for responding to complaints, and how well it keeps people informed of progress and outcomes. |
3: Australian National Audit Office (2014), Management of Complaints and Other Feedback: Australian Taxation Office, Canberra.
4: Kiwis Count is a survey run by the State Services Commission that measures citizens' satisfaction with public services.
5: Failure demand is the failure to do something right that causes the client to make further demands on the organisation. See Seddon, J. (2005), Freedom from command and control: a better way to make the work work… the Toyota system for service organisations, Vanguard Education Ltd, Buckingham, page 26.