Part 4: Systems for monitoring benefits performance are reliable
4.1
In this Part, we discuss how:
- Inland Revenue monitors progress towards realising benefits;
- internal reports inform decision-making; and
- Inland Revenue is well placed for the future.
4.2
Reliable information enables decision-makers to understand the current performance of the programme or project and re-direct activities or adapt their practices as needed. High-quality information also provides Parliament and the public with assurance that public money is being spent responsibly and that the programme or project remains on track to achieve the intended outcomes. Importantly, making reliable information about performance publicly available provides transparency, which increases trust and confidence in the work an organisation is doing.
4.3
We expected Inland Revenue to produce reliable information about how the programme was performing. This information would direct activities and give monitoring agencies and the Crown assurance that the programme is progressing towards its intended outcomes.
4.4
We expected that Inland Revenue would get this information by regularly and consistently monitoring its benefit measures and regularly providing the information to decision-makers in a useful way.
Summary of findings
4.5
We have confidence that Inland Revenue produces reliable information about the benefits the programme achieves and measures, monitors, and reports progress that is consistent with the requirements of external monitoring agencies.
4.6
Inland Revenue has set baselines to assess progress against and uses high-quality enterprise data to inform its measures. It collects information at regular intervals and has processes to ensure the quality of that information.
4.7
Inland Revenue has made reporting on the programme publicly available. For example, summaries of external reviews, benefit reporting, reports to ministers, and updated business cases are available on Inland Revenue’s website. Inland Revenue decided early that it would keep the public well informed about the programme.
4.8
In our view, based on the progress Inland Revenue has made in achieving the programme’s targets and its continual focus on benefits management, it is well placed to continue managing, monitoring, and reporting on programme benefits.
Inland Revenue monitors progress consistently and well
4.9
Inland Revenue monitors benefits well using defined and reviewed benefit measures and baselines. The benefit monitoring arrangements have evolved during the programme, adapting to Inland Revenue’s internal and external feedback. By monitoring benefits, Inland Revenue can see how the programme is progressing towards its intended outcomes.
4.10
Inland Revenue provides an assessment of the programme’s progress to Parliament and to central agencies that are responsible for monitoring significant Crown investments, such as the Treasury. It reports the monitoring information it collects on the programme to the Treasury and Cabinet twice a year.
4.11
Inland Revenue also reports this information internally to its Performance and Investment Committee twice a year. It also produces monthly reports for the Ministers of Finance and Revenue that describe progress towards the programme’s intended outcomes.
4.12
The 2015 business case defined the expected benefits, benefit targets, and baselines to assess progress against. Using this information, Inland Revenue produced the initial programme and Stage 1 benefit plans. These benefit plans described the timeline for when it expects the benefits of the programme to be realised, how it would measure the benefits, and when it would report the results.
4.13
The benefit plan for Stage 1 sets out how Inland Revenue would measure and report benefits. At this stage, it measured the programme against the overall expected benefits of the programme.
4.14
These arrangements were appropriate early in the programme because the new technology platform’s capabilities and how customers would respond to the new ways of working were unknown. The expected benefits for Stage 1 were small compared to the benefits of the overall programme. However, Inland Revenue refined and improved the programme’s monitoring arrangements to account for greater certainty and new information over time.
4.15
In 2017, as the programme progressed and more information became available, Inland Revenue developed “indicators of progress”, which set annual targets with “lead and lag indicators” to measure and monitor the programme’s progress towards achieving its intended outcomes.
4.16
These indicators of progress, which were developed in agreement with the Treasury and approved by the Ministers of Finance and Revenue, replaced the benefit targets and monitoring arrangements established in the 2015 business case.
4.17
The lead and lag indicators show the Treasury how benefits are progressing and whether it needs to escalate issues if progress on achieving benefits falls behind schedule. These indicators of progress were introduced and communicated after Stage 1 of the programme was implemented in 2017. Introducing these indicators reflected the cumulative effect of the changes introduced throughout the multi-year programme. Inland Revenue staff understood these monitoring arrangements, including the new indicators, well.
4.18
As well as introducing the annual lead and lag indicators, Inland Revenue refined its forecasting of the administration savings benefit. It did this at the request of the Treasury. These forecasts used additional information generated from further programme planning to create additional indicators that it reports quarterly.
4.19
The purpose of this was to help provide more assurance that the programme will achieve its administration savings benefit.
4.20
Where benefits are difficult to measure quantitatively, Inland Revenue produces regular case studies to demonstrate progress. Inland Revenue has a schedule for completing case studies and has consistently complied with that schedule. Although the schedule for case studies was determined at the same time as the lead and lag indicators, additional case studies have been produced as needed and when information becomes available. At the time of our audit, Inland Revenue had produced 19 case studies. Together, these case studies help demonstrate that Inland Revenue is achieving benefits.
4.21
These case studies show the benefits that Inland Revenue is achieving for the duration of the programme. They are focused on how the changes from the programme support achieving benefits and highlight instances that did not go to plan.
4.22
For example, in the case study about payday reporting,19 Inland Revenue reported that its efforts to support transitioning to payday reporting did not go as planned. Inland Revenue enabled voluntary payday reporting for the year leading up to it being compulsory on 1 April 2019 but did not get the uptake in voluntary compliance it had expected. However, it considered that this voluntary period enabled it to test its systems and processes.
4.23
In our view, the monitoring arrangements in Figure 7 help to provide more certainty to Inland Revenue, the Treasury, and Cabinet about when benefits are expected to be realised than earlier information provided .
Figure 7
The programme’s measures and targets, agreed with the Treasury in 2017
| Outcome – Easier for customers | |||||||
Investment objectives
|
|||||||
| Indicators | 2017/18 | 2018/19 | 2019/20 | 2020/21 | 2021/22 | 2022/23 | 2023/24 |
| Lead indicators | |||||||
| Digital uptake by customers | 26% | 54% | 72% | 75% | 78% | 82% | 85% |
| Percentage of customers who find it easy to comply | 82% | 85% | 87% | 88% | 89% | 90% | 90% |
| Reduction in compliance time for SME customers (hours each year) | 3 | 8 | 13 | 15 | 16 | 17 | 18 |
| System availability for customer-facing e-channels | 99.2% | 99.3% | 99.3% | 99.4% | 99.4% | 99.5% | 99.5% |
| Lag indicators | |||||||
| Customer outcomes achieved from information sharing and security of information | This will be measured through a series of case studies | ||||||
| Cumulative reduction in compliance costs for SMEs | $30m | $160m | $370m | $590m | $820m | $1,070m | $1,330m |
| Cumulative additional Crown revenue to Government | $90m | $280m | $570m | $1,110m | $1,860m | $2,880m | |
| Outcome – Reduced time and cost to implement policy | |||||||
Investment objectives
|
|||||||
| Indicators | 2017/18 | 2018/19 | 2019/20 | 2020/21 | 2021/22 | 2022/23 | 2023/24 |
| Lead indicator | |||||||
| Reduction in the time and cost to implement policy | This will be measured through a series of case studies. | ||||||
| Lag indicator | |||||||
| Increased revenue system resilience as assessed by Inland Revenue | Low | Low | Partial | Partial | High | High | High |
| Outcome – Inland Revenue is more efficient | |||||||
Investment objective
|
|||||||
| Indicators | 2017/18 | 2018/19 | 2019/20 | 2020/21 | 2021/22 | 2022/23 | 2023/24 |
| Lead indicator | |||||||
| Digital uptake by customers | 26% | 54% | 72% | 75% | 78% | 82% | 85% |
| Lag indicators | |||||||
| Annual reduction in Inland Revenue’s administrative costs | $5m | $10m | $80m | $100m | $100m | $100m | $100m |
| Cumulative reduction in Inland Revenue’s administrative costs | $5m | $15m | $95m | $195m | $295m | $395m | $495m |
Source: Adapted from the Treasury (2017), Thresholds for monitoring Inland Revenue’s Business Transformation benefits and outcomes, Wellington, page 3. These figures include inflation, depreciation, and capital charge.
4.24
Inland Revenue has changed the way it monitors its progress towards achieving benefits. As the programme progresses and more information becomes available, new opportunities for monitoring the programme’s benefits will become available.
4.25
We encourage Inland Revenue to identify opportunities to build on and improve the way it measures and monitors its progress in achieving the programme’s intended outcomes. This will help with benefit planning and provide additional assurance to Parliament and the public about the level of benefits the programme is achieving.
We have confidence in the monitoring information
4.26
We have confidence in the information Inland Revenue produces about realising the programme’s benefits. Inland Revenue has a clear understanding of how benefits can be attributed to the programme. It has clear records of the definitions and data sources used to measure the benefits and quality control checks to ensure the information’s accuracy.
4.27
Inland Revenue’s benefits management strategy sets out its approach to attributing benefits to the programme. The strategy notes that other projects existing alongside the programme may result in improved outcomes for Inland Revenue. The benefits from these other projects should not be included when assessing the programme’s benefits.
4.28
As well as the information on attributing benefits outlined in its benefits management strategy, Inland Revenue has issued specific guidance on benefits attribution for the benefit area “increase Crown revenue”. The guidance provides specific examples of what can and cannot be attributed to an increase in Crown revenue as a result of the changes enabled by the programme. Inland Revenue also tasked its Benefits Realisation Co-ordination Group with ensuring that benefits are not counted twice.
4.29
The Treasury’s guide Managing benefits from projects and programmes: Guide for practitioners includes a section on attributing benefits. The guide identifies value or driver modelling as a method to support this.
4.30
The guide states that, if a programme benefit is “a reduction in operating costs”, an entity could identify the “driver” for the reduction. If it is related to staff effort and overhead costs, the associated benefit can be isolated and attributed to the programme.
4.31
Inland Revenue has followed this guidance. For example, in Stage 1 of the programme, one of the programme-funded initiatives was to provide more digital channels to customers to complete their tax obligations. As a result of this initiative, Inland Revenue attributed $2.4 million to the administration savings benefit because more customers were using online forms and saving paper and postage costs. The driver for this change was the additional digital channels created from Stage 1.
4.32
As discussed in paragraphs 2.36-2.41, Inland Revenue chose, wherever possible, to use enterprise performance measures to monitor the programme’s benefits. We were told that it did this so that information produced about the programme’s benefits is the same as the information that informs its annual report. This information is subject to the business controls that are tested during the annual audit to check accuracy and correct input.
4.33
We found that Inland Revenue gains assurance in its enterprise performance measures by:
- having clear records of the definitions and data sources used to measure the benefits; and
- having the information checked and then signed off by a different person carrying out quarterly internal quality assurance checks.
4.34
The purpose of these checks is to ensure the accuracy of all of its performance results and financial information. The programme’s governors told us that they felt that these regular checks gave them confidence that the information is reliable. In Inland Revenue’s 2018/19 annual audit, our appointed auditor’s opinion was that Inland Revenue’s performance information presented fairly, in all material respects, what had been achieved.
Internal reports inform decision-making
4.35
Inland Revenue produces regular internal reports about benefits management. We found that the reporting was high quality and that the information reported was relevant and consistent.
4.36
Inland Revenue’s decision-makers told us that they were satisfied with the information they received about the programme. This information is used to set direction and improve the programme’s activities and benefits planning.
4.37
In 2015, Inland Revenue committed to reporting to the Business Performance Board (now replaced by the Performance and Investment Committee) every six months on its progress on realising benefits. This aligns with the programme updates for Cabinet and is an expectation from the Treasury’s benefits guide.
4.38
As well as these regular governance updates, Inland Revenue produces reports about the programme’s benefits, including release readiness assessments and the results from case studies produced about the programme’s benefits.
4.39
We found that Inland Revenue uses the information generated about benefits to get a better understanding of when benefits are expected and to make adjustments as needed. For example, when the programme reported that it had reached its target for reducing departmental administrative costs in 2018/19, Inland Revenue moved to extract cost savings from future operating budgets.
4.40
Similarly, the programme has reported internally that it was unlikely to achieve the targets associated with Release 3 in 2019/20. Inland Revenue found that customers did not appear to have fully adjusted to the changes from previous releases. For that reason, Inland Revenue expects the public’s adjustment to the changes in Release 3 to be slower than anticipated.
4.41
As a result, Inland Revenue has prepared more detailed forecasts (known as “glide paths”) that outline when and in what areas it expects the estimated reduction in administrative costs to be achieved. These glide paths take into account the slower than anticipated uptake and the resources needed to achieve this administration cost savings.
4.42
Although the updated estimates demonstrate a slower than originally anticipated achievement of benefits in 2019/20, Inland Revenue estimates that the overall level of administrative savings to be achieved by 2023/24 will be greater than initially estimated.
4.43
Inland Revenue has provided these new estimates to the Treasury, as required by the programme’s monitoring arrangements. The November 2019 update to Cabinet noted that “[i]nformation provided by Inland Revenue of glide-path targets is useful to track this progress.”20
4.44
We consider the work Inland Revenue has done to refine its understanding about when benefits are expected provides decision-makers with more understanding of what needs to be done to realise benefits.
External reports are of good quality
4.45
Having defined and regular external reporting requirements supports benefits management by providing an opportunity for stakeholders to reflect on a programme’s achievement, strengths, and weaknesses. It also supports general engagement with stakeholders.
4.46
We expected the programme to have clear expectations for when Inland Revenue would report information about benefits externally. We also expected that the reported information would be useful and relevant to various audiences and that material changes to the programme would be clearly communicated.
4.47
The 2015 business case and benefits management plans clearly outline external reporting expectations. Inland Revenue produces monthly programme updates for the Ministers of Finance and Revenue, and reports to Cabinet twice a year.
4.48
Inland Revenue consistently produces these updates and makes them publicly available on its website. The updates provide useful information about how the programme is progressing against its expected benefits and overall financial performance. The updates also identify key risks to the programme and outline any material changes.
4.49
As well as providing Ministers and Cabinet with assurance that the programme is progressing towards its intended outcomes, Inland Revenue also provides insight into specific service benefits that customers receive.
4.50
For example, Inland Revenue reported its progress towards the benefit “easier for customers”, using an agreed measure “reduction in customer tax compliance burden ([per annum] hours per SME)” in the July 2018 update to Cabinet. Inland Revenue reported that there had been an average 10-hour reduction in the time small and medium enterprises took to complete their tax obligations. The target for this benefit was a three-hour reduction.
4.51
The reports provided to Ministers and Cabinet also identify the programme’s key risks, including:
- managing customer fatigue as a result of the significant amount of change that has already been made and will continue to be made;
- supporting customers to keep compliance costs down;
- maintaining services to customers; and
- keeping the old revenue system running while the programme is being completed.
4.52
The updates also include additional commentary from central agencies. For example, in the July 2019 update, central agencies advised that they were working with Inland Revenue to improve visibility of the operational metrics used to forecast the realisation of the administration savings benefit.
4.53
In the November 2019 update, central agencies advised that Inland Revenue had successfully met its administration savings targets to date and that the forecast for administration savings showed an increase in the later years. In our view, making these reports publicly available is an effective way to keep the public informed on the progress of benefits and supports transparency.
4.54
As well as the monthly reporting to Ministers and the twice-yearly reporting to Cabinet, Inland Revenue produces addenda for programme business cases. The addenda outline the material changes to the programme since Cabinet approved it. They have the same format as the 2015 business case and provide updates on the programme’s progress against its expected benefits. Making these updates publicly available keeps stakeholders informed about high-level updates to the programme.
4.55
Over time, Inland Revenue has adjusted the way it reports some of its non-financial benefits externally. The October 2019 Gateway review recommended that Inland Revenue improve its reporting of benefits by making its progress towards its non-administration savings outcomes more visible.
4.56
As a result, Inland Revenue now provides a percentage assessment of how much progress it has made towards achieving the non-financial benefits “improving agility so that policy changes can be made in a timely and cost-effective manner” and “minimising the risk of protracted system outages and intermittent systems failure”.21
4.57
In our view, this change improves the programme’s reporting by providing easy-to-understand graphs that demonstrate Inland Revenue’s progress in some of the main benefit areas.
4.58
These examples of external reporting of the programme provide Cabinet and the public with consistent and relevant information to understand the programme’s health and assess progress with realising its benefits.
Inland Revenue is well placed for the future
4.59
As at September 2019,22 Inland Revenue reported that it has achieved all but one of its targets against the programme’s lead and lag indicators, which are the agreed indicators of progress related to benefit areas. This was communicated in the programme’s reporting and Inland Revenue’s 2019 Programme business case addendum (see Figure 8).
Figure 8
Inland Revenue’s progress against the lead and lag indicators of the Business Transformation programme
| Outcome: Easier for customers | |||||||||
Investment objective:
|
|||||||||
| Indicators | 2017/18 | 2018/19 | 2019/20 | 2020/21 | 2021/22 | 2022/23 | 2023/24 | ||
| Target | Actual | Target | Actual | ||||||
| Lead indicators | |||||||||
| Digital uptake by customers | 26% | 83% | 54% | 89% | 72% | 75% | 78% | 82% | 85% |
| Percentage of customers who find it easy to comply | 82% | 82% | 85% | 80% | 87% | 88% | 89% | 90% | 90% |
| Reduction in compliance time for SME customers (hours each year) | 3 | 10 | 8 | 9 | 13 | 15 | 16 | 17 | 18 |
| System availability for customer-facing e-channels | 99.2% | 98.9% | 99.3% | 99.4% | 99.3% | 99.4% | 99.4% | 99.5% | 99.5% |
| Lag indicators | |||||||||
| Customer outcomes achieved from information sharing and security of information | This will be measured through a series of case studies | ||||||||
| Cumulative reduction in compliance costs for SMEs | $30m | $80m | $160m | $280m | $370m | $590m | $820m | $1,070m | $1,330m |
| Cumulative additional Crown revenue to Government | $90m | $90m | $280m | $570m | $1,110m | $1,860m | $2,880m | ||
| Outcome: Reduced time and cost to implement policy | |||||||||
Investment objective:
|
|||||||||
| Indicators | 2017/18 | 2018/19 | 2019/20 | 2020/21 | 2021/22 | 2022/23 | 2023/24 | ||
| Target | Actual | Target | Actual | ||||||
| Lead indicators | |||||||||
| Reduction in the time and cost to implement policy | This will be measured through a series of case studies | ||||||||
| Lag indicator | |||||||||
| Increased revenue system resilience as assessed by Inland Revenue | Low | Partial | Low | Partial | Partial | Partial | High | High | High |
| Outcome: Inland Revenue is more efficient | |||||||||
| Investment objective: Improving productivity and reducing the cost of providing Inland Revenue’s services |
|||||||||
| Indicators | 2017/18 | 2018/19 | 2019/20 | 2020/21 | 2021/22 | 2022/23 | 2023/24 | ||
| Target | Actual | Target | Actual | ||||||
| Lead indicators | |||||||||
| Digital uptake by customers | 26% | 83% | 54% | 89% | 72% | 75% | 78% | 82% | 85% |
| Lag indicators | |||||||||
| Annual reduction in Inland Revenue’s administrative costs | $5m | $12m | $10m | $48m | $80m | $100m | $100m | $100m | $100m |
| Cumulative reduction in Inland Revenue’s administrative costs | $5m | $12m | $15m | $60m | $95m | $195m | $295m | $395m | $495m |
Note: Inland Revenue uses proxy measures (based on the lead and lag indicators in Figure 8) and case studies to measure the benefit area “increase Crown revenue” because of the difficulty of direct attribution.
Source: Based on Inland Revenue (2019), Programme business case addendum: Business Transformation programme – Implementing New Zealand’s future revenue system, Wellington. These figures include inflation, depreciation, and capital charge.
4.60
These indicators demonstrate several significant changes, such as:
- on average, small to medium businesses are spending less time meeting their tax obligations; and
- Inland Revenue has removed $60 million from future costs.
4.61
For 2018/19, Inland Revenue had met all but one of its lead and lag indicators. Inland Revenue did not meet the expected target for the lead indicator for the benefit area “percentage of customers who find it easy to comply”. This indicator is measured using the “customer satisfaction and perception survey”. The target for 2018/19 was 85%, but Inland Revenue achieved 80%.
4.62
We were told that this is because Release 3 resulted in many changes to the way customers were expected to interact with the revenue system. As a result of these changes, some customers’ experiences have not been positive.
4.63
One of the major changes was that employers were required to provide information on employees every pay day rather than monthly. Inland Revenue reports in the November 2019 update to Cabinet that it expects this to improve in the next year as these changes are embedded.
4.64
Although the programme is well under way, there are significant benefits to be delivered in the next few years. We expect Inland Revenue to continue to report on the realisation of benefits as the programme progresses.
4.65
As a result of the programme, the way Inland Revenue carries out its tax administration has changed significantly. It expects these changes to help drive the realisation of the programme’s benefits. For example, Inland Revenue was able to automatically issue income tax assessments for the tax year ending 31 March 2019. As a result, 1.3 million people received refunds worth a total of $572 million, and 271,000 people have a total of $95 million to pay.
4.66
Inland Revenue now also requires employers to provide information each pay day. This means that Inland Revenue has real-time information about income tax and ensures that people pay the right amount of tax throughout the year. This reduces the need for over-payments and under-payments to Inland Revenue.
4.67
Inland Revenue is on track to realising the programme’s intended benefits. However, the gap between the benefits achieved to date and benefits sought by 2023/24 is significant, and there is some uncertainty about the realisation of those benefits.
4.68
As a result of Covid-19, the Minister of Revenue reported to Cabinet in July 2020 that the final release of the programme is being re-planned. This is to reflect the effect of supporting the Government’s response to Covid-19 on Inland Revenue’s capacity, the challenges of staff needing to work from home for an extended period of time, and the financial effects of Covid-19 on all New Zealanders.
4.69
At the time of writing this report, the potential implications of re-planning the final stage and the potential effects on the benefits of the programme are still being worked through. Cabinet expects Inland Revenue to provide this information in November 2020.
4.70
We encourage Inland Revenue to focus on monitoring risks to the programme, including those outlined in paragraph 4.51, so that progress to date is not undermined. The Covid-19 lockdown and its potential effects is an extreme example of a challenge that can arise during a programme of this significance and length.
4.71
In our view, Inland Revenue has made good progress towards realising the programme’s benefits and is well placed to manage, measure, and monitor those benefits in the future. This is because of the encouraging progress it has made, the deliberate focus and effort it has put into benefits management, and the quality of programme monitoring and reporting.
4.72
However, we will not be able to say with certainty that the significant investment in the programme represents value for money until the benefits from the completed programme are measured.
19: This is the requirement for employers to file an employment information form every time they pay their employees. For more information, see ird.govt.nz.
20: Cabinet paper (2019), Inland Revenue’s November 2019 transformation update, page 12.
21: Inland Revenue (2019), Programme business case addendum: Business Transformation programme – Implementing New Zealand’s future revenue system, Wellington, page 20.
22: Inland Revenue will report information on the achievement of benefits in 2019/20 in November 2020.