Part 7: Mechanisms used to improve the response during 2020

Co-ordination of the all-of-government response to the Covid-19 pandemic in 2020.

7.1
In this Part, we provide an overview of key mechanisms used to identify and implement improvements to the all-of-government response in 2020. We discuss how:

7.2
We expected that reasonable steps would be taken to identify issues and risks to the all-of-government response and to continually improve it.

7.3
Given the complexities and rate of change in the operating environment, we did not expect that systems would always work smoothly. However, as the response evolved, we expected to see processes established to support sharing of lessons and more wider and consistent application of good practices.

Summary of findings

7.4
It is clear that officials made significant ongoing efforts to understand how well the all-of-government response was working in 2020 and improve it.

7.5
However, there was no standard process or overarching plan for ensuring that continuous improvements to the all-of-government response were carried out. Many approaches to identifying issues and making improvements were used, but how well they were developed and applied varied.

7.6
Some of these approaches were not implemented in a regular or integrated way, not carried out as intended, or relied on people rather than good processes. There were also disruptions such as the August 2020 outbreak and challenges created by high staff turnover.

7.7
Formal risk management and system assurance functions were introduced from December 2020. This provided more confidence that issues would be addressed.

Scenario modelling, simulations, and trials were used to test and improve the response

7.8
Using international and domestic public health information, the Ministry of Health and the all-of-government response carried out modelling to see how many Covid-19 cases there could be if different approaches were taken. This modelling and the simulation exercises held in March 2020 informed the elimination approach and other mitigating actions.

7.9
Officials worked proactively to plan and test the operational response in 2020. This included planned stress-testing of contact tracing in June 2020 and testing a resurgence response days just before the August 2020 outbreak. DPMC also organised testing to see whether the system was ready for a summer resurgence.

7.10
Officials updated the Covid-19 Tracer App to include manual entries, Bluetooth, and National Health Index numbers. We saw evidence that algorithm-based tools were in development in November 2020 to help rapid decision-making on super-spreader events and incoming travel risks.

7.11
Trials of Bluetooth Covid-19 contact tracing cards as a wearable alternative to QR code scanning were carried out. In late 2020, a prototype co-designed with Te Arawa iwi was trialled in Rotorua, but a national roll-out was found to be unviable.

7.12
The Institute of Environmental Science and Research ran an eight-week pilot from November 2020 to test wastewater for traces of Covid-19 at MIQ hotels and in a sample of towns. This testing continued in selected locations, including where possible outbreaks were suspected.

Officials learned from other countries’ experiences

7.13
From Covid-19’s earliest emergence, international data, research, and information about other countries’ experiences were reported to senior officials and other decision-makers. This continued throughout the response.

7.14
We saw examples of a Covid-19 weekly monitoring report to Ministers that included information about the situations in other countries and approaches they were taking. Overseas trends and developments were also part of strategic insights papers for the Covid-19 Group’s leadership team.

7.15
In June 2020, the Quin discussed a paper by the National Crisis Management Centre about how other countries had removed restrictions. In July 2020, the Covid-19 Group sourced documents from Victoria, Australia on lessons from its outbreak and subsequent lockdown. We saw a Cabinet paper on resurgence planning that used this information.

7.16
Some Ministers and senior officials, including the Chief Executive of DPMC and the Director-General of Health, discussed approaches to the Covid-19 pandemic with foreign counterparts and other officials. New Zealand also participated in regular information exchanges through a global network of chief science advisors and other international groups.

Officials used a wide range of data to inform the response

7.17
Officials regularly tracked quantitative information on a range of measures. This included the number of Covid-19 tests, positive cases, potential breaches of lockdown, and QR scans. Efforts were made to source additional relevant data.

7.18
Officials used dashboards to report key information. We were provided with samples of reporting from all-of-government workstreams that identified top issues and used traffic light colour coding to show the status and progress of activities and measures.

7.19
Real-time data was collected and projections were available to help inform policy, resourcing, and public information decisions. For example, in June 2020, an urgent two-day review was carried out when the number of people entering New Zealand threatened the capacity of MIQ facilities and staffing levels. Supply and demand figures, along with international arrivals and public health information, continued to inform border settings in 2020.

7.20
We saw evidence that, in early August 2020, officials were concerned about research that indicated a significant reduction in the public’s compliance with Covid-19 health measures. This led to the development of a Be Ready advertising campaign. In November 2020, in response to low Covid-19 Tracer App usage, the Make Summer Unstoppable campaign encouraged New Zealanders to keep up precautionary health behaviours.

7.21
The all-of-government communications team actively monitored social media posts for misinformation and disinformation,70 and took corrective steps. These included responding to posts and providing clarifications through daily public messages written for Ministers, officials, and the media.

7.22
We also saw that the Covid-19 Group developed ways to monitor risks to its own operations. By December 2020, it had adopted decision-making tools to help assess and address risks to its role leading the all-of-government response.

7.23
These risks included deteriorating relationships with key agencies and corporate business disruption (for example, from Covid-19 Group staff being in isolation or an earthquake making its work systems inaccessible).

7.24
The Covid-19 Group identified mitigations and reported its internal risk assessment to its leadership team and to DPMC’s Risk and Assurance Committee.

Internal and cross-agency engagement identified and resolved problems

7.25
We consistently heard that meetings (both internal and with other response agencies) were critical for identifying and helping to implement improvements. Getting the right people together on a regular basis was seen as one of the most effective and efficient ways for sharing information, raising issues, identifying priorities, challenging assumptions, and finding solutions.

7.26
However, interactions were not always smooth. Ongoing effort was needed to improve communication and working relationships. High staff turnover and frequent changes to operating arrangements made this challenging.

7.27
We heard that, in 2021, the Deputy Chief Executives of the Covid-19 Group and the Ministry of Health were in daily contact, even during periods of no community transmission. Deputy chief executives responsible for Covid-19 policy still met up to three times a week.

7.28
Some of the normal hierarchies and conventions were put aside during the height of the all-of-government response. Ministers spoke directly with officials who were experts in their field, rather than communicating through line management. Advice was sometimes provided through these exchanges, rather than through formal papers.

7.29
Although this approach was efficient, it also meant that decision-makers did not always have access to full analysis or documentation.

7.30
We heard that the all-of-government response brought together agencies that might not ordinarily work together, such as the Ministry of Health and the Ministry of Business, Innovation and Employment. These relationships are seen as having continued value.71

7.31
In November 2020, Ministers discussed the need for Covid-19 advice to be better integrated. However, we were told that having a DPMC liaison officer in the Ministry of Health and a staff member from the Ministry of Health in the Covid-19 Group helped improve mutual understanding and relationships.

7.32
We also heard about the value of the Covid-19 Group, the Ministry of Health, and the Ministry of Business, Innovation and Employment each having a private secretary in the office of the Minister for Covid-19 Response. We were told that they all worked together closely and provided a useful communication channel both within and between agencies.

Officials made efforts to understand and reflect the public’s views

7.33
To keep the public’s trust and confidence in the all-of-government response, officials used different methods to understand people’s needs and experiences at different Alert Levels. However, we did not see evidence of an overarching plan to carry out these activities or to co-ordinate various efforts between agencies during 2020.

7.34
Surveys were conducted to help understand the impact of the Covid-19 pandemic on people’s lives and to inform response settings. These included the Ministry of Health’s Covid-19 Health and Well-being Survey,72 and research commissioned in March and June 2020 to help Ministry of Health and National Crisis Management Centre officials understand how the public was responding to the Government’s Covid-19 communications.73

7.35
The Covid-19 Group worked with the Ministry of Health to analyse the best way to maintain the public’s trust and confidence and improve people’s compliance. The Ministry led a behavioural insights project and shared its findings at a November 2020 workshop attended by policy staff from the Covid-19 Group.

7.36
Other attempts to seek public views included the Operations Command Centre developing a Good Ideas website to bring together the public’s views on how to improve the all-of-government response.

7.37
For some Covid-19 response plans, the Ministry of Health invited views from a range of people. In June 2020, the Ministry used an online tool to seek feedback from key stakeholder groups to help inform an action plan to respond to the Independent review of Covid-19 clusters in aged residential care facilities. In the same month, the Ministry also used the tool to consult on its Covid-19 psychosocial and mental wellbeing recovery plan. It published a summary of feedback in October and a revised plan in December 2020.

7.38
However, we also understand that some plans had to be revised because they did not adequately reflect stakeholder feedback. This included the Ministry of Health’s Updated Covid-19 Māori health response action plan.

7.39
Some people we interviewed said that they were aware of shortcomings in how the all-of-government response engaged with communities in 2020. Quin meeting minutes show that, from March 2020, leadership team members were aware of iwi concerns and identified actions to address them.

7.40
However, some of these efforts were not fully effective. In May 2020, for example, the Quin discussed iwi concerns about contact tracing that had not been satisfied and that iwi had not been given the opportunity to contribute to tangi guidelines. In November 2020, the Covid-19 Group Leadership Team was made aware that iwi had concerns about not being consulted early enough in any work to change Alert Level boundaries for a future resurgence.

7.41
A Caring for Communities workstream was developed that had several responsibilities, including to give welfare support to vulnerable or hard-to-reach people, establishing community liaison and outreach, engaging with iwi, and gathering intelligence and insights.74 Initially, the Chief Executive of Te Arawhiti led this workstream, which was run through the Operations Command Centre. A governance group chaired by the Chief Executive of the Ministry of Social Development was set up to oversee the work.

7.42
We saw evidence that this workstream filled a critical gap in co-ordinating welfare support and was retained as the all-of-government response evolved.75

7.43
A private sector workstream was also set up through the Operations Command Centre. Its work included a private/public partnership project which helped procure and manage critical supplies to essential workforces. It also helped businesses connect with others that were supporting vulnerable communities.

7.44
Provision was made for an ongoing business sector liaison once the all-of-government response transitioned to DPMC in July 2020. We discuss developments to better include communities and businesses in Part 8.

A range of review and reflection activities supported improvements

7.45
Many people we interviewed stated that reviews were one of the main ways to influence change and improve the all-of-government response. For the second rapid review, the chairperson of ODESC asked to be briefed on findings as they emerged. These reviews were referred to in requests to Cabinet to strengthen the structures and resourcing of the Covid-19 response system.

7.46
Many other reviews of aspects of the Covid-19 response were carried out, intended to be carried out, or identified as being needed. Reviews often reacted to specific issues of concern to the public, such as the supply of personal protective equipment, the performance of contact tracing, and the process for exemptions to managed isolation.

7.47
We saw a continuous improvement plan that NEMA produced in April 2020. This high-level plan outlined how NCMC North was expected to manage continuous improvements.

7.48
We also understand that short internal reviews (known as hot washes) were held in the National Crisis Management Centre. NEMA told us that feedback was also collected from National Crisis Management Centre staff on any operational issues found during their shift.

7.49
NEMA said that this information was used to make tactical improvements where appropriate. These changes were communicated as and when needed at staff briefings.

7.50
We heard that workshops and team debriefs were held in other parts of the response but that these tended to be ad hoc rather than systematic. We did not see evidence that the outcomes of these discussions were consistently documented, shared, and acted on.

7.51
We saw evidence that the all-of-government response looked for ways to better achieve its objectives. Over months, the Quin discussed options for the most appropriate operating model and how best to transition to it. The Covid-19 Group Leadership Team regularly reflected on ways to improve its own operations.

7.52
From October 2020, the Covid-19 Group’s Leadership Team identified that valuable perspectives and knowledge were being lost because many seconded staff were leaving. The leadership team was increasingly concerned that “things were falling through the cracks” and agreed to introduce exit interviews as one solution.

7.53
Efforts had previously been made to capture insights from departing staff. An NCMC North document from June 2020 referred to an exit survey that included questions on what worked well and what could be improved.

7.54
We also saw reference to other reflection and review exercises from different parts of the response. In June 2020, DPMC’s Chief Executive informed the Hazard Risk Board that a process was in development for a Covid-19 “lessons learned” exercise. This did not occur, but DPMC told us that other sources (rapid reviews and hot washes) had provided information.

7.55
In July 2020, there was also a two-day national civil defence emergency management debrief of the Covid-19 response. Attendees discussed the bespoke response structure, inter-agency co-ordination, reporting, and planning.

7.56
This event informed an independent assessment of the civil defence emergency management response to the Covid-19 pandemic for NEMA from January to June 2020. The assessment made 29 recommendations and was provided to NEMA in December 2020. The Ministry of Health also provided us with details of various reviews it carried out in 2020.

7.57
During the August 2020 outbreak, the chairperson of ODESC formed a Red Team to challenge and strengthen the response’s decision-making.76 We understand that this was the first time during 2020 that a Red Team had been used in relation to the Covid-19 pandemic.77

7.58
Every day for four weeks, the Red Team developed critical questions for the National Response Leadership Team. These questions were both operational and strategic in focus, such as “What consideration has been given to building hospital capacity, particularly in areas where resources are stretched, to deal with both business-as-usual and Covid-19 incursion demand?” and “Are you confident that the lessons identified from this response are being translated into system learning?”. In our view, many of these questions have enduring value.

More systematic approaches to making improvements and managing risks were introduced

7.59
Towards the end of 2020 a number of initiatives to encourage continuous improvement were implemented:

  • The Covid-19 Chief Executives’ Board incorporated a permanent Red Team function.
  • A continuous improvement function was formally assigned to the Covid-19 Group as part of its system assurance role. Part of this involves testing the National Resurgence Response Plan through regular table-top exercising and continuously updating it to reflect lessons from the Covid-19 response.
  • NEMA became responsible for setting up a continuous improvement framework and an operational system for learning lessons. It also became responsible for the National Exercise Programme.

7.60
We were told that NEMA had set up a small continuous improvement team by February 2021 and was seeking to build its capacity. NEMA secured additional funding to deliver improvement initiatives through Budget 2021, including stronger hazard management expertise. We heard that NEMA established a continuous improvement unit in January 2022.

7.61
We expect the Covid-19 Group and NEMA to work closely together in carrying out their respective responsibilities for supporting continuous improvements and also promoting risk reduction and readiness.

Centralised risk management was introduced

7.62
Part of the Covid-19 Group’s assurance role included system-wide risk management.

7.63
In August 2020, the Covid-19 Group requested agencies’ risk registers to help develop a framework for monitoring and managing Covid-related risks throughout the system. However, this work was stalled by the August 2020 outbreak, when the Group had to re-prioritise focus on the response.

7.64
By December 2020, the Covid-19 Group had progressed a draft framework to manage system risks for the new Covid-19 Chief Executives’ Board to use. The framework identified 14 key risk areas.

7.65
Most of these were potential issues that could not be resolved by one agency or workstream (known as cross-cutting or horizontal risks). Examples include a flawed national strategy, not having enough staff to support the response system, losing public confidence, and a poorly performing economy. The other risks were potential issues that a particular workstream was managing but that would have wide-reaching impacts if the risk eventuated (vertical risks). Examples were the effectiveness of MIQ and the vaccine roll-out.

7.66
By May 2021, the Covid-19 Group had started using the framework to update the Board on key risk areas.

Parliamentary scrutiny of the response enhanced transparency

7.67
The Leader of the Opposition chaired the Epidemic Response Committee. Between 31 March and 26 May 2020, it met three times each week to scrutinise what the Government had done and planned to do in response to the Covid-19 pandemic.

7.68
The work of the Committee increased transparency and public awareness of the response. The Committee heard from officials, Ministers, and the public on a range of topics about how the response was being managed. Meetings were live-streamed and had high public interest (more than three million views on Parliament’s Facebook and Vimeo channels).78 The Epidemic Response Committee also examined two pieces of legislation and suggested amendments, informed by public submissions.79

7.69
From late May 2020, after Parliament resumed sitting, each agency with Covid-19 responsibilities was accountable through normal departmental select committee arrangements.

7.70
In a July 2020 review, Parliament’s Standing Orders Committee encouraged Parliament to keep some successful elements of the Epidemic Response Committee model. These elements included its more conversational style of scrutiny and more frequent appearances by Ministers.

70: Misinformation means false information presented as fact, either intentionally or unintentionally. Disinformation means false information deliberately spread to mislead people.

71: DPMC has expressed this view. See Governance and Administration Committee (2021), Hansard transcript: 2019/20 Annual Review of the Department of the Prime Minister and Cabinet, at parliament.nz. Many people we interviewed reflected on the opportunity to come together as part of the all-of-government response. A few people told us about “taking off agency hats” to do what was needed for the collective government response.

72: This survey ran from March to September 2020. It involved calling participants daily to ask questions about the clarity of Alert Level rules, their mental health, and their ability to pay basic living costs.

73: TRA (2020), Covid-19 – Response to communications. Baseline measure: 30-31 March, Clemenger BBDO and Ministry of Health and TRA (2020), Covid-19 – Response to Government communications, DPMC. See research at covid19.govt.nz.

74: This workstream supported the welfare activities carried out by 16 regional Civil Defence Emergency Management Groups, co-ordinated by NEMA. We saw a list that identified priority communities. It included people with addictions, recent migrants, refugees, gang families, homeless people, disabled people, ethnic people, and children in need of protection.

75: We were told that this group continued to operate during the August 2021 resurgence, supported by national and regional welfare co-ordination groups.

76: As stated in paragraph 2.32, a Red Team can be set up to do a semi-independent real-time review of activities to assure ODESC that the full range of actions is being considered for a response.

77: The National Security System handbook outlines best practices for running a Red Team, including advice about appropriate frequency. It states that running a Red Team “should not be a one-off activity and should be conducted to allow undetected vulnerability to be identified and addressed”, while not becoming too disruptive.

78: These 3 million views were estimated to account for 10%-20% of total viewership of the Committee across all media (including through frequent replay of its proceedings in news media during lockdown). See Hellyer, G (February 2021), “Assessing Parliament’s response to the Covid-19 pandemic”, Policy quarterly 17:1, at ojs.victoria.ac.nz/pq.

79: These were the Covid-19 Response (Further Management Measures) Legislation Bill and the Immigration (Covid-19 Response) Amendment Bill.