Part 4: Achieving the expected benefits and learning from projects
4.1
In this Part, we discuss:
Summary of findings
4.2
There are indications that customers and the Council are receiving benefits from the online services programme. However, not all of the outcomes in the customer-friendly services focus area are being tracked and reported to the subgroup. This means that the subgroup does not have a complete view of how the online services programme and other customer-focused initiatives are helping the Council to provide customer-friendly services.
4.3
The Council has not made as much progress as it had planned in achieving benefits from the Smart Forms project. There is also limited information about whether the Council is getting the expected non-financial benefits from the Identity Management project – for example, whether the project has made it easier for customers to access the Council's online services.
4.4
The Council should improve its tracking of the expected non-financial benefits from projects to give it a better view of whether projects are achieving all of their expected benefits, including for customers.
4.5
Although lessons from projects have been identified and applied to other projects, there is a risk that this will not happen consistently for every project.
What has been achieved so far?
The online services programme is delivering improvements for customers
4.6
By the end of June 2017, the Council had "digitised" 22% of its most common transactions. This was below its goal of 27% of transactions by this date because the Council was focused on the launch of another Council-wide project, New Core. However, although the Council has not made as much progress as planned, it is now easier for people to access services online. The Council expects that, with the completion of New Core, more progress can be made, especially with back-end processes.
4.7
Customer feedback about smart forms has been positive. The Licensing and Compliance Services unit surveys people at three different points during the online application process. Staff from this unit told us that they receive positive feedback about the smart forms through the surveys and feedback given to frontline staff.
4.8
From 10 April 2017 to 30 July 2017, there have been over 30,000 log-ins using Identity Management to access the Council's online services. However, based on the reporting that goes to the subgroup, it is not clear whether the expected non-financial benefits from the Identity Management project are being achieved (see paragraph 4.25).
Financial benefits from projects
4.9
Overall, projects are delivering financial benefits to the Council. Figure 9 shows the estimated net financial benefits from approved projects that are part of the customer-friendly services focus area. From 2017 to 2019, the Council estimates a total benefit of about $9.3 million.
Figure 9
Estimated net financial benefits to Auckland Council from approved projects under the customer-friendly services focus area, 2017-19
| Type of financial benefit | 2017 $ | 2018 $ | 2019 $ | Total $ |
|---|---|---|---|---|
| Cost reduction | 144,308 | 1,101,209 | 1,455,090 | 2,700,607 |
| Increase in revenue | 291,878 | 718,256 | 1,273,964 | 2,284,098 |
| Cost avoidance | 955,895 | 1,628,086 | 1,754,346 | 4,338,327 |
| Total | 1,392,081 | 3,447,551 | 4,483,400 | 9,323,032 |
Source: Office of the Auditor-General, using information from Auckland Council.
4.10
More than 70% of applications need to be completed online for the Council to get financial benefits from the Smart Forms project. The Council planned to reach this target by July 2017. It has not done so.
4.11
The percentage of applications completed online initially increased from December 2016 to February 2017 (see Figure 10). Since then, an average of 25% of applications have been completed online. Different types of applications have significantly different online application rates. For July 2017, the highest rate was 30% and the lowest was 11%.
Figure 10
Percentage of smart form applications completed online each month, from December 2016 to July 2017
Source: Office of the Auditor-General, using information from Auckland Council.
4.12
Representatives from the business unit and the online services programme meet regularly to identify how they can increase the percentage of applications completed online.
4.13
The Identity Management project does not have any direct financial benefits (see paragraph 4.22) but instead enables Council services to be accessed online.
Other benefits from the online services programme
4.14
Council staff said that the online services programme was delivering benefits that were not in its business case, such as:
- better information on the activities of business units;
- helping to change the Council's culture so that staff can help the people who need it most;
- better capability to extract and analyse data to support the advice given to customers, third-party stakeholders, and elected officials;
- improved reporting to better identify issues in business units; and
- having an overview of all the Council's services.
The Council has made changes to help get the expected benefits
4.15
The Council has made changes to help it understand whether the expected benefits from projects are being achieved. For example, the Council has a new review process to ensure that it is achieving the benefits outlined in project business cases. This review looks at benefits gained from a completed project and identifies any opportunities to achieve further benefits. The first review looked at an online booking system for Council-managed venues, including seeking feedback from users. The Council plans further reviews, including on the Smart Forms project.
4.16
Council staff said that achieving and measuring benefits was an area of focus for the Council. We encourage the Council to continue to improve its understanding of whether the expected benefits are being achieved.
Improving the tracking and monitoring of benefits
Reporting progress on performance measures
4.17
The online services programme's business case did not specify any performance measures that were separate from the performance measures of the customer-friendly services focus area. Instead, the business case linked the programme's achievements to the organisational strategy outcomes and associated performance measures. For this reason, we looked at the reporting of the performance measures under the customer-friendly services focus area.
4.18
The Council has set up systems to track its progress towards putting 70% of the Council's most common transactions online. This is the only performance measure for the customer-friendly services focus area that is regularly tracked and reported to the subgroup.
4.19
Two of the other performance measures (increasing the Council's customer experience score and decreasing the processing time for consents) are reported in quarterly performance reviews to the Council's Executive Leadership Team. We did not see any reporting for the last two performance measures (improving the Council's customer effort score and decreasing the cost to serve).
4.20
In our view, the Council needs to report progress towards achieving all of the performance measures of the customer-friendly services focus area. This would provide a more complete view of the effects of the online services programme and other initiatives in achieving the Council's aims under the focus area, including making it easier for customers to access and use services.
Reporting of non-financial benefits can be improved
4.21
Expected benefits for projects in the online services programme are identified in their business cases. The expected financial benefit for the Smart Forms project was $135,000 in gross savings if 70% of applications were submitted online. Expected non-financial benefits included a better customer experience and allowing staff to focus on higher-value work.
4.22
The Identity Management project business case did not specify any financial benefits. Instead, financial benefits were expected to be realised by other Council services being made possible by a log-in service and by making online services easier for customers to access. However, as part of approving the business case, the Investment Group asked about setting targets related to use.
4.23
After the projects were launched, metrics were set up to track benefits. For the Smart Forms project, the main metric is the percentage of applications filed online. Realised benefits are then reported to the subgroup and the Investment Group, which holds overall responsibility. The online services programme also provides informal reporting on benefits – for example, through email updates to members of the subgroup.
4.24
Some Council staff thought that the reporting on project benefits was good overall. However, we found that there have been some problems. For example, it was discovered that reporting on the use of Identity Management was inaccurate, and reliable reporting began only in April 2017 – four months after the project was launched.
4.25
We did not see any reporting to the subgroup on the expected non-financial benefits from the Identity Management or Smart Forms projects, such as reporting on whether people find it easier to access the Council's online services now they can use a single log-in account.
4.26
Some Council staff said that benefits for people using the Council's services need to be better defined and measured. For example, it would be useful to assess whether the Council has made the process of applying for services easier and better. The need to measure benefits to customers, and non-financial benefits generally, is a challenge for many organisations.
4.27
The Council told us that it is working to measure the non-financial benefits of projects. Reporting on the non-financial benefits of projects would give the subgroup and other governance groups a better picture of whether projects are realising all of their expected benefits for the Council, its staff, and the public.
| Recommendation 3 We recommend that Auckland Council review its monitoring and reporting on progress towards achieving the financial and the non-financial benefits from projects and how projects contribute towards achieving all of the objectives of the customer-friendly services focus area. |
Lessons are being applied
4.28
Throughout its work on the online services programme, the Council has identified and applied lessons from the Identity Management and Smart Forms projects. Lessons are informally shared through the online services programme's quick daily meetings, informal project manager lunches, and technical meetings with other parts of the Council. One of the project managers said that there is not a day where there is not an "Aha!" moment from the daily meetings.
4.29
There is also a process for formally recording lessons. At the end of the Identity Management and Smart Forms projects, there were formal post-project reviews and workshops on what went well and what could been done better. Some of the lessons identified from these included:
- the importance of regular communications and updates;
- working with the responsible business unit during the early development stages;
- including customer testing earlier; and
- needing a clearer minimum viable product definition so that the project team knew what to prioritise.
4.30
The Council is applying these lessons to other projects. The Smart Forms project's business case identified the opportunity to use its lessons on technical delivery and customer insights to reduce the delivery risk of the Consenting Made Easy programme. Council staff told us that this is now happening.
4.31
The Council intends to use lessons from the Identity Management project for the My Account project.
4.32
However, Council staff told us that sharing and applying lessons can depend on who is involved. This could lead to the lessons being applied inconsistently. We suggest that the Council look at ways to ensure that lessons are applied to relevant projects, regardless of who is involved.