Part 3: What has been delivered and what to focus on to realise benefits
3.1
In this Part, we outline:
- what the Vision 2015 Programme has delivered;
- what the Vision 2015 Programme has yet to deliver;
- the cost of the Vision 2015 Programme;
- the time taken to deliver the Vision 2015 Programme; and
- what Immigration New Zealand needs to focus on to realise all the intended benefits.
3.2
Although the Vision 2015 Programme has made progress in delivering efficiencies to the visa processing service and created a foundation to realise benefits, more needs to be done to ensure that these benefits are fully realised.
What the Vision 2015 Programme has delivered
3.3
Broadly, the Vision 2015 Programme has delivered:
- Immigration Online, which allows people to apply for visitor, student, and work visas online;
- electronic visas (eVisas), which remove the need for physical visa labels on passports;
- the ability for people to check the status of their visa online and to communicate with Immigration New Zealand through their online account;
- a new biometric identity management system;
- a risk-based triage and a verification model that triages visa applications according to risk levels; and
- an online system that allows medical assessments to be recorded, viewed, and stored securely.
3.4
The Vision 2015 Programme has, for the first time, introduced standard operating procedures for processing visitor, student, and work visa applications. This brought consistency, which has helped Immigration New Zealand achieve some process efficiencies, improve the quality of its decision-making, and improve customer experience.
3.5
In Appendix 4 we outline, in chronological order, the changes and improvements that were rolled out during the Vision 2015 Programme.
Benefits realised so far
3.6
The programme has produced some positive effects on Immigration New Zealand's visa processing efficiency. Our analysis showed that, although the average processing time7 has increased in some cases, the variation in how long processing takes has reduced for some visa applications. For example, the variation in the time taken to process visitor visa applications in offshore offices has dropped over time.
3.7
In Figure 4, our analysis showed the average time to approve a visitor visa in offshore offices has decreased over time despite a significant increase in the number of applications received.
Figure 4
Average time taken to approve a visitor visa application in offshore offices
| Duration | Number of approved visitor visa applications | Average number of days taken to approve a visitor visa application |
|---|---|---|
| July 2013 to June 2014 | 167,279 | 7.6 |
| July 2014 to June 2015 | 195,267 | 6.5 |
| July 2015 to June 2016 | 258,714 | 5.9 |
Note: Between July 2013 and June 2016, Immigration New Zealand received about 1.9 million visa applications. Visitor visa applications made up 45% of all visa applications. Nearly 79% of visitor visa applications were processed in offshore offices.
Source: Our analysis of data from Immigration New Zealand.
3.8
Immigration New Zealand had to hire more visa processing staff to meet the continued increase in visa applications. However, improvements brought by the Vision 2015 Programme resulted in a smaller increase in staffing levels than Immigration New Zealand estimated. Our own analysis and Immigration New Zealand's forecasting suggested that Immigration New Zealand had needed roughly 250 to 300 fewer full-time equivalent staff8 than before the improvements were made (Figure 5).
Figure 5
Predicted number of staff required to process visa applications without the Vision 2015 Programme changes, compared with actual number of staff following the changes
Note: The calculations are based on the number of customers, rather than visa applications, per full-time equivalent staff. Many visa applications have more than one person associated with each one and there are some visa applicants who have more than one visa application.
Source: Our analysis of data from Immigration New Zealand.
3.9
In many countries, customers (such as visa applicants and licensed immigration advisers) are now able to electronically submit visa applications and medical doctors can put medical certificates online. People can now check the status of their visa online, and education institutions can check the validity of student visas online.
3.10
Before the Vision 2015 Programme, all medical certificates were received in the mail. eMedical, the electronic system that lets visa applicants and medical doctors put medical certificates online, has significantly reduced the reliance on paper medical certificates. Up to 30 September 2016, only 1.3% of medical certificates had been received in the mail. One of the benefits of eMedical is that processing medical certificates is mostly automated using Immigration New Zealand's business rules. This should reduce visa processing times. A specialist Health Assessment Team was set up to process all medical certificates not processed automatically. Having specialist in-house capability should improve the quality of decisions about medical assessments. Up to 30 September 2016, 71% of medical certificates had been automatically processed by eMedical.
3.11
The number of online visa applications has been increasing (Figure 6).
Figure 6
Visa applications received online, by year and by quarter
Notes:
The third quarter of 2014 includes only the months of August and September. Online applications were not yet available in July 2014.
There is no online application process for Resident visas.
Source: Immigration New Zealand.
3.12
Immigration New Zealand believed that the integrity of the visa processing system might have been compromised without the changes delivered through the Vision 2015 Programme.
Estimated savings
3.13
As at 30 June 2016, Immigration New Zealand's data showed that the Vision 2015 Programme had delivered $10.2 million in savings over 2014/15 and 2015/16 (Figure 7). Although the savings were less than expected because of the increased number of visa applications, Immigration New Zealand is on track to meet its estimated annual savings as at September/October 2014 (see Figure 3) of $12.3 million in 2018/19. If Immigration New Zealand had not invested in the Vision 2015 Programme, it is likely that it would have faced greater costs to process the increasing number of visa applications.
3.14
The estimated savings exclude the benefits associated with the extra $28.4 million approved under the Vision 2015 Programme extension business case (see paragraph 2.33). This work will be completed outside the Vision 2015 Programme, and so its related benefits are not tracked as part of the Vision 2015 Programme.
Figure 7
Breakdown of estimated savings from the Vision 2015 Programme
| 2014/15 | 2015/16 | |||||
|---|---|---|---|---|---|---|
| Estimated in the Vision 2015 Programme business case | Estimated in the Vision 2015 Programme benefit plan | Estimated by Immigration New Zealand as actually saved | Estimated in the Vision 2015 Programme business case | Estimated in the Vision 2015 Programme benefit plan | Estimated by Immigration New Zealand as actually saved | |
| Discontinuation of Ministry of Foreign Affairs and Trade's visa services | $3.6 million | $3.3 million | $3.3 million | $4.2 million | $3.3 million | $3.3 million |
| Process efficiencies* | $2.4 million | $0.17 million | - | $6 million | $0.71 million | $0.96 million |
| Visa Services operational efficiencies** | - | - | $2.2 million | - | - | $0.44 million |
| Total annual savings | $6 million | $3.47 million | $5.5 million | $10.2 million | $4.01 million | $4.7 million |
Note: We have not audited the savings because they are estimated figures. The Vision 2015 Programme benefit plan sets out the Vision 2015 Programme's planned benefits.
* These are time and effort savings as a result of improvements in processing visas, such as the automation of some processes. It includes hiring fewer staff.
** These are related to the costs of managing visa processing operations, such as changes in Immigration New Zealand offices' management structures and direct savings from having to handle less paper (for example, less postage and courier fees).
Source: Immigration New Zealand.
What the Vision 2015 Programme has yet to deliver
3.15
The Vision 2015 Programme concluded in June 2016. Immigration New Zealand delivered all capabilities outlined in the Immigration Global Management System and Vision 2015 Programme business cases except one – the core visa processing system was not fully replaced. Immigration New Zealand made a decision to keep parts of the core visa processing system and plans are in place to maintain it for the foreseeable future. We discussed this decision in paragraphs 2.23 to 2.25.
Benefits yet to be realised
3.16
At the time of our audit, Immigration New Zealand had not started monitoring and reporting the Vision 2015 Programme's service benefits. Only the financial benefits were measured and reported. Although financial benefits are important, they are only part of what the Vision 2015 Programme has delivered.
3.17
Immigration New Zealand has identified and described the Vision 2015 Programme's intended benefits at a high level. These are tracked back to the Vision 2015 Programme's business case. Immigration New Zealand also identified key performance indicators for each of the 29 projects in the Vision 2015 Programme. However, we have not seen how these key performance indicators are brought together to show whether the Vision 2015 Programme's intended benefits will be met (we outlined the intended benefits in paragraph 2.27).
3.18
As a result, we could not tell to what extent the Vision 2015 Programme's overall service benefits and wider outcomes have been achieved.
Cost of the Vision 2015 Programme
3.19
The Vision 2015 Programme was delivered within budget. As the Vision 2015 Programme progressed, costs were tracked and reported. The final cost of the Vision 2015 Programme was $119.3 million.
3.20
The cost included the Cabinet-approved investment of $108.4 million and additional approved adjustments of $10.9 million.
3.21
The Vision 2015 Programme invested in quality assurance such as Gateway reviews and independent quality assurance. The overall cost of quality assurance was about 2.25%9 of the cost of the Vision 2015 Programme. In our view, this compares favourably with the cost of quality assurance for other major projects.
Time taken to deliver the Vision 2015 Programme
3.22
The Vision 2015 Programme's time frame was extended by six months.
3.23
Immigration New Zealand took a pragmatic and focused approach to ensure that it met most of the Vision 2015 Programme's time frames. However, some events outside its control caused delays. For example, Immigration New Zealand had to move from the Department of Labour to the newly formed MBIE. This led to some rework of the new ICT system to ensure compatibility with MBIE's infrastructure.
3.24
Further work to ensure that the business was ready to change caused delays. For example, the roll-out of the identity management system was postponed so that more detailed business operational readiness testing could be done.
3.25
Although it is important to meet deadlines, it should not be done at the expense of quality. By delaying the roll-out of the identity management system, Immigration New Zealand could ensure that the business was ready to receive the new system.
What Immigration New Zealand needs to focus on to realise all the intended benefits
3.26
Immigration New Zealand completed the Vision 2015 Programme broadly on time, within budget, and in a challenging environment (we described this in paragraph 2.21).
3.27
However, changes are still being made and embedded.
3.28
The Vision 2015 Programme never intended to fully deliver its benefits at the time of our audit. However, the Vision 2015 Programme has provided a good foundation for these benefits to be realised. It is crucial that Immigration New Zealand ensures that the changes made are sustainable so that the full benefits of the Vision 2015 Programme can be realised.
3.29
The Vision 2015 Programme's benefits rely on two key factors:
- most visa applications being submitted online; and
- an effective triage approach for visa applications.
3.30
An effective triage system would mean low-risk applications take less time to process, allowing resources to be reallocated to more high-risk and complex applications. This would mean online visa applications are processed more efficiently at any of Immigration New Zealand's offices.
3.31
Immigration New Zealand needs to maintain a balanced and strategic focus when rolling out new changes and making sure these changes are put in place. Since the Vision 2015 Programme concluded, there has been a transfer of knowledge and skills to Immigration New Zealand's key staff. New governance arrangements are in place to oversee future improvements.
3.32
However, there are risks to the successful delivery of the remaining benefits. Below we discuss what Immigration New Zealand needs to focus on to ensure that the Vision 2015 Programme delivers the intended benefits.
Put in place a strong performance framework
3.33
During our audit, there was no integrated performance management and reporting framework.
3.34
Reporting teams did not work together and the use of information to guide decisions was fragmented. Visa Services did not have an agreed set of measures that could be analysed and applied throughout the visa processing service. This meant that it was difficult for Immigration New Zealand's leadership to make well-informed, strategic decisions.
3.35
To ensure that the full benefits are realised, Immigration New Zealand needs to collect the right performance information consistently across the visa processing service, monitor and report the performance measures, and use this performance information to target improvements. The performance information needs to include improving the consistency and quality of visa decisions so that visa applicants are treated consistently regardless of where their application is processed. Immigration New Zealand also needs to allocate the right capability and resources to do this.
3.36
Immigration New Zealand needs to strengthen how it collects, prioritises, and responds to feedback from frontline staff. Tools and processes to gather staff feedback have not been consistent or effective during the change process. Staff reported feeling their feedback appeared to "go into a black hole" and it was not clear to them whether anything changed as a result.
| Recommendation 2 |
| To fully realise the benefits from the Vision 2015 Programme, we recommend that Immigration New Zealand strengthen the collection and management of feedback from frontline staff on the information and communications technology system and process changes introduced by the Vision 2015 Programme, to ensure that further improvements are made to the visa application process. |
3.37
Immigration New Zealand needs to better understand the detailed benefits and effects of the changes rolled out in different parts of the world. Immigration New Zealand works in a global environment and it needs to understand the local context of each of its offshore offices when applying changes in different parts of the world. For example, the benefits of online services may not be realised in the Pacific region because of limited access to the Internet.
3.38
Immigration New Zealand needs to strengthen its understanding of the benefits to external stakeholders and to the wider economy.
3.39
Immigration New Zealand has created a performance management and reporting framework as one of the 29 projects that are part of the Vision 2015 Programme. However, the performance management and reporting framework has yet to be fully introduced to all Immigration New Zealand's offices.
| Recommendation 3 |
| To fully realise the benefits from the Vision 2015 Programme, we recommend that Immigration New Zealand prioritise and resource a performance management and reporting framework that measures and reports on the Vision 2015 Programme's financial and service benefits. |
Consolidate improvements made so far
3.40
It is important that Immigration New Zealand takes time to consolidate the improvements made so far so it can to realise all the intended benefits. We found Immigration New Zealand to be used to change. However, there is a risk that staff could suffer from change fatigue.
3.41
Change is still happening, and there will be more improvements and upgrades to the new systems. There are also wider changes in MBIE that will affect Immigration New Zealand.
3.42
Staff need time to familiarise themselves with the new systems and new business processes to ensure that changes are effective. Many frontline staff we spoke to were busy and dealing with the added pressure of an increasing number of visa applications. Immigration New Zealand needs to support staff appropriately and allow time for changes to set in.
3.43
Process improvements, supported by technology improvements, would bring consistency and efficiencies. However, staff need to be using the new systems and following the new business processes to realise these benefits.
3.44
Immigration New Zealand informs us that a team has been set up to review all the changes introduced through the Vision 2015 Programme and the plans to put in place the changes. This team is expected to ensure that Immigration New Zealand has a clear road map for the range of property, management, and process efficiency benefits that Immigration New Zealand can expect to achieve over the next three years. This work is under way and is scheduled to be completed by the end of March 2017.
Accountability for realising benefits
3.45
Immigration New Zealand and the wider MBIE must be fully accountable and responsible for delivering benefits.
3.46
It is important that there is strong leadership and senior support to help accelerate critical decisions, resolve resourcing issues, set and manage realistic expectations, and add impetus to a project. Having engaged and active sponsorship from the Chief Executive can help ensure that there is continued focus on realising benefits that are in line with MBIE's goals.
3.47
It is important that accountabilities are clear to those responsible for delivering and putting in place change.
| Recommendation 4 |
| To fully realise the intended benefits from the Vision 2015 Programme, we recommend that Immigration New Zealand ensure that it and the wider Ministry of Business, Innovation and Employment share the responsibility and accountability for realising the full benefits of the Vision 2015 Programme. |
7: The median processing time is the mid-point value of the time it takes to process a type of visa. We have used the median value because this calculates the time taken "normally" to process a visa. The median processing time is not skewed by large or small values.
8: These exclude Immigration New Zealand's policy staff.
9: This includes costs for external assurance, the Ministry's internal audit, independent board members, and the programme's assurance manager.