Part 9: Lessons from the six projects

Realising benefits from six public sector technology projects.

9.1
In this Part, we discuss lessons from the six projects that may be relevant for other ICT-enabled projects in the public sector.

Good benefits realisation in practice

9.2
We researched models of managing benefits realisation. We then created our own model, drawing, in particular, on the work of the New South Wales Department of Finance and Services. Our contractor then reviewed and amended this model. The amended model is shown in Figure 2. We used the model in Figure 2 to help us identify elements of good practice in the six projects and to develop the six themes that we discuss in this Part.

9.3
A critical feature of the model is the continuous process of planning, reviewing, reporting, and updating of the benefits being and to be realised.

9.4
We have identified six themes and lessons from the projects:

  • understanding the environment and making the most of circumstances;
  • using a business-led, flexible, and agile approach;
  • having strong support from leaders and senior managers;
  • working effectively with the right people, including end users;
  • using the right technology tools; and
  • monitoring and understanding the benefits.

9.5
It is important to consider four things when looking at the common themes and lessons:

  • having elements of good benefits realisation practice should not be interpreted as meaning all aspects of a project were best practice – none of the projects showed good practice in all respects;
  • good benefits realisation practice for a given case study was good practice in the context of that case study, but may not be good practice in all contexts – good practice cannot necessarily be "cut and paste" from one project to another;
  • good benefits realisation practice should not come at the expense of other aspects of managing successful projects, such as working within the available time and resources – good practice is not good practice when delivered at all cost; and
  • some of the good benefits realisation practice is closely linked to wider good management practices and may not be specific to the technology.

Figure 2
Using information and communication technology to realise benefits

Figure 2 - Using information and communication technology to realise benefits.

Source: School of Government, Victoria University of Wellington.

Understand the environment and make the most of circumstances

9.6
From the six projects we have drawn two factors that show an understanding of the environment and making the most of the circumstances. These are:

  • identifying increasing or future demand for services as an impetus for change; and
  • recognising extreme or special circumstances of a given situation, including the impetus of limited time.

9.7
These factors relate to the benefits realisation management strategy stage of the process outlined in Figure 2.

9.8
Acting strategically is about an organisation fitting its services to the environment in which it works and the changes forecast to that environment. Fitting with the environment and being able to capitalise on the opportunities that the environment presents were a feature of some of the six projects.

9.9
Fitting with the environment included using limited time as an opportunity, not a constraint. This was particularly a feature of the Christchurch Earthquake Support System, where services had to be delivered within a few days of the 22 February 2011 earthquake.

9.10
Another feature of the Christchurch earthquake project was the unique circumstances leading to Cabinet agreeing to special privacy legislation. This legislation was a strong enabler of the project, and might not have happened in other circumstances. The Ministry of Social Development's identifying the need for special cross-agency information sharing arrangements, and getting these put in place, were important contributors to realising benefits.

9.11
In SmartGate and Landonline, Customs and LINZ foresaw large increases in demand for their services. These increases could not be met if the services continued to be delivered in the same way. Both public entities understood that the changing nature of the demand for their services would make their services unsustainable. This led to a clear business purpose and pre-emptive action, using ICT as an enabler, to prevent a crisis.

Be business-led, flexible, and agile

9.12
The six projects show that a business-led, flexible and agile approach:

  • has a focused business purpose;
  • has people with detailed knowledge of the business involved;
  • does not try to solve everything at the same time;
  • follows an iterative or pilot approach; and
  • uses current technology where it makes sense to and does not reinvent solutions.

9.13
These factors relate to the benefits realisation management strategy and benefit planning stages outlined in Figure 2.

9.14
Technology is not an end in itself. To effectively realise benefits using ICT, it is important that a clear business purpose guides how we use technology. Often, this business purpose may mean that people do things differently.

9.15
A clear business purpose was a notable practice in three of the projects:

  • InfoConnect's clear business purpose was to give road users accurate, timely, and relevant traffic information. Giving third parties access to the information to distribute it has achieved this. This has meant that the purpose was achieved without NZTA having to get into non-core business (developing software applications) and having to directly manage all the risks that would entail.
  • SmartGate's clear business purpose was to give effect to the Australian and New Zealand Prime Ministers' commitment to make the border between their countries more efficient and make it easier for Australians and New Zealanders to move between the two countries.
  • The business purpose of the Student Loan System was complex. The first purpose was to make more real-time integrated information available to borrowers, specifically the current balance of their loan. A second, and more transformational purpose, was to redesign Inland Revenue's systems. This was to make them more adaptable to policy changes, able to incorporate changes more quickly, and be more automated for customers. This second purpose is yet to be met.

9.16
Because ICT-enabled projects have a business focus, it is important to involve people who know a lot about the business. This involvement was notable in the Student Loan System and Landonline projects. These projects covered complex business practices that required specialised business knowledge, including detailed knowledge of the history of business practices. In the case of Landonline, subject matter experts were seconded to the project teams. This reduced the time needed to consult with other parts of the business.

9.17
Having a flexible and agile approach is perhaps the most critical aspect of good practice that we saw. This is because it is at the heart of the continuous nature of managing benefits realisation shown in Figure 2. This is the dynamic process of planning, reviewing, reporting, and updating the benefits being and to be realised. At the heart of this are the business benefits being sought.

9.18
There is no need to solve everything at once. Often, this is important because of the long time that some of the projects can take to set up, given their complexity, and the changes to technology and the demands on the project that can happen during that time. In certain situations, taking a flexible and iterative approach can be good way to manage risks, reducing the amount of unnecessary work and rework that might otherwise result.

9.19
Elements of a flexible and iterative approach to the work were a notable feature of three of the projects:

  • The InfoConnect initiative was piloted before being rolled out and the work was phased.
  • In effect, SmartGate was piloted. Customs borrowed a SmartGate device from Australia before committing to the technology. This allowed Customs to test and investigate the device, and allowed Customs to use the SmartGate technology quickly. There were only nine months between Cabinet's endorsement of Customs' SmartGate plans and the first SmartGate device going into use in Auckland.
  • The approach used for the 111 Deaf Text Service has given the Police a technology platform that can be used for mass text communication with a defined community. For example, the platform was used to communicate with the Police and emergency service staff in Christchurch after the 22 February 2011 earthquake. In effect, the 111 Deaf Text Service piloted the technology, first with the culturally deaf community and then with the wider hearing-impaired community.

9.20
Working on new technology or solving technology problems already solved by someone else is not always cost-effective and can result in unnecessary risks.

9.21
The InfoConnect, SmartGate, and 111 Deaf Text Service projects involved entities performing a national and/or international scan of how other entities had addressed the service challenges that they faced. With SmartGate, this resulted in adopting "off the shelf" technology. With the 111 Deaf Text Service, it meant using a technology that other police forces had not used for this purpose.

Have strong leadership and senior support

9.22
Strong leadership and support from main stakeholders, such as Ministers and senior managers:

  • are critical to effectively realise benefits in ICT-enabled projects;
  • can help to accelerate critical decisions, resolve resource blockages, set and manage realistic expectations, and add impetus to a project; and
  • are particularly important for the reporting stages, when information about the project is given to people outside the project for them to make decisions this includes the business case and benefits planning and reporting stages outlined in Figure 2.

9.23
Some of the projects had strong support from politicians. With the Christchurch Earthquake Employment Support System, the Minister of Social Development reported daily to the media how many employers and employees had received payments. This gave the Minister a direct stake in the project.

9.24
The New Zealand and Australian Prime Ministers made a commitment to SmartGate. Because the Prime Minister and Cabinet prioritised SmartGate, it was a priority for the chief executive and Customs. The project team could rely on having the resources they needed from the organisation to complete the project on time.

9.25
With Landonline, the chief executive's active and engaged sponsorship was important for communicating the importance of the project.

Work effectively with the right people, including end users

9.26
The six projects show that successful project teams work effectively with the right people. In particular, they:

  • pay a lot of attention to users' experiences; and
  • have strong and collaborative relationships within the project team, with vendors, and with principal stakeholders, including other public entities involved.

9.27
These factors relate to all the stages of the benefits realisation process outlined in Figure 2.

9.28
The complexities of the environments in which public entities work mean that many parties may have an interest in their business. This includes end users, the providers of technology, and politicians. Achieving benefits through technology can depend on all of these people.

9.29
In planning and setting up the 111 Deaf Text System, the Police worked closely with the culturally deaf community. This means that the Police better understand what cultural deafness means in a practical sense. They have adjusted operational practices, not just the provision of 111 services, to the needs of this specific community.

9.30
Strong support from external stakeholders was a feature of the Landonline project. Strongly committed stakeholder groups, including law societies and surveyors, contributed to understanding better what users required. Landonline project teams included paid stakeholder representatives.

9.31
Because business benefits are derived when people, enabled by technology, do things differently, having strong collaboration and commitment among those involved within a public entity is important. Strong collaboration and commitment were notable in:

  • the Christchurch Earthquake Employment Support System, with staff being co-located and doing the initial design for the system during a weekend; and
  • InfoConnect, where NZTA collaborated successfully with third parties that had the expertise, time, and funding to provide effective solutions.

Use the right technology tools

9.32
These six projects show that it is important to use the right technology tools for a given set of circumstances. The tools we have highlighted are:

  • agile methodologies;
  • making information open; and
  • open-source technology tools.

9.33
These factors primarily relate to the benefits realisation management strategy and benefits planning stages outlined in Figure 2.

9.34
Many agencies and individuals faced extreme challenges after the large Canterbury earthquakes. This included the Ministry of Social Development, which the Government wanted to distribute financial support to employers and employees immediately after the 22 February 2011 Christchurch earthquake. The purpose of providing this support was to reduce uncertainty about jobs and businesses in Christchurch, and to help people to pay their bills. Therefore, haste was important.

9.35
In response to this challenge, Ministry of Social Development staff designed and built the online Earthquake Employment Support System over a weekend. This was possible because of their choice of rapid development methodologies for system development. The Kanban method and the Ruby on Rails open-source web development framework were used and $53 million in payments were made in the first week of the system being in place.

9.36
The InfoConnect initiative used open-source software tools. This kept costs down and brought support from application developers. Having the support of application developers was critical, as these developers were relied on to distribute NZTA's information.

9.37
NZTA made its road user data largely open. It was freely available to application developers who registered with NZTA. Registration gave the agency some control over misuse of the data. But it did not stop developers adding further value to the data. Making data available in this way had the added benefit of improving data quality through feedback from users.

Monitor and understand the benefits

9.38
The factors we have drawn from the six projects that demonstrate effective monitoring and understanding of the benefits are:

  • clear articulation of the benefits; and
  • routine monitoring of the benefits being realised.

9.39
These factors primarily relate to the benefits planning and benefits monitoring stages outlined in Figure 2.

9.40
Documentation that specifically planned, reported, or catalogued the benefits being realised was not a strong feature in the projects. We consider such documentation to be good benefits realisation practice because it is a critical part of monitoring and evaluating the benefits of a given project.

9.41
Four projects did not follow the benefits realisation cycle outlined in Figure 2 in terms of the continuous process of documentation and review of benefits. This is clearly an area for improvement in managing benefits realisation in public sector ICT-enabled projects.

9.42
Inland Revenue clearly stated the benefits to be achieved as part of its Student Loan System project and monitoring of the project at a programme level focused on benefits realisation. This helped to identify that the initial project would not realise the intended benefits and an informed decision was made to redesign the project to deliver a more realistic set of outcomes. Inland Revenue's decision to redesign the project was sensible in the context of the benefits yet to be realised and resources used.

9.43
Monitoring of benefits was a feature of Landonline. In practice, this was achieved by picking elements of benefits realisation management from the methodologies used in the project. A pragmatic approach was required because of inconsistent language and expectations.

Consider the themes identified

9.44
The themes that we have identified should not be considered as all-encompassing, mutually exclusive, or unique to the process of realising benefits.

9.45
In our view, the themes are practical and useful and should be considered carefully when planning to realise benefits using ICT.

page top