Part 4: Vehicle inspections will need to change
4.1
In this Part, we describe:
- the intended objectives and outcomes from regulating vehicle inspectors and inspecting organisations, including how these are measured and reported; and
- why vehicle inspections will need to change.
4.2
We expected NZTA to have clearly described what it wants to achieve from its regulation of vehicle inspectors and inspecting organisations and to know how well it is doing that. In particular, we expected NZTA to:
- have clear objectives and outcomes for its regulation of vehicle inspectors and inspecting organisations;
- regularly assess its effectiveness;
- report its performance to the public; and
- make improvements where needed.
Objectives and outcomes could be clearer
The links between the regulation of vehicle inspectors and inspecting organisations and broader outcomes are not clear
4.3
Throughout this audit, the people we spoke to were committed to improving road safety and saw the current warrant and certificate of fitness system as an important contributor to that. We agree there is a link between ensuring vehicle safety and making our roads safer, although we did not see this link clearly described in any of NZTA's or the Ministry of Transport's strategic documents or other publicly available information.
4.4
In addition, we did not see clear regulatory objectives for the vehicle inspection system linked to measurable impacts or well-defined compliance outcomes, measures, or targets (such as an increase in the level of compliance, identified through site reviews).
4.5
Understanding the extent to which warrant and certificate of fitness inspections contribute to road safety is not easy. Data indicates that vehicle faults contribute to less than 4% of serious crashes in New Zealand. At the same time, vehicle age, design, and safety features can have a greater influence on the severity of crashes.
4.6
Without a good understanding of the relationship between vehicle inspections and road safety, it is difficult for NZTA to make informed policy and investment choices about where to focus its efforts (for example, whether investing in more site reviews will make a substantial contribution to improved road safety outcomes or whether it is best to focus on other parts of the system, like road design), or for NZTA to understand how its performance affects road safety outcomes.
4.7
Nevertheless, NZTA is accountable to the public and needs to be able to explain why vehicle inspections are needed, particularly to vehicle owners who have to pay for inspections and can be penalised if they do not.
| Recommendation 2 |
|---|
| We recommend that the New Zealand Transport Agency Waka Kotahi more clearly describe how vehicle inspections, and its regulation of vehicle inspectors and inspecting organisations, contribute to vehicle safety and the objectives and outcomes it wants to achieve. |
Current performance measures focus on activity, not impact
4.8
NZTA has measures to track and report on its performance in regulating vehicle inspectors and inspecting organisations (see Figure 5). In our view, these measures do not give a sufficient overview of performance.
Figure 5
How NZTA measures and reports on its regulatory performance
| Performance measure | Where is it reported? |
|---|---|
| Number of compliance activities completed | Annual report (as an annual figure) Safer Vehicles monthly report Road Safety Regulation monthly report |
| Percentage of non-compliance found during site reviews | Road Safety Regulation monthly report |
| Percentage breakdown of responses to non-compliance | Road Safety Regulation monthly report |
| Proportion of non-compliance actions for vehicle inspecting organisations, vehicle certifiers, and vehicle inspectors that are progressed within acceptable time frames | Annual report (as an annual figure) Safer Vehicles monthly report (year to date and monthly) Road Safety Regulation monthly report (monthly result) |
| Number of non-compliance cases completed | Safer Vehicles monthly report Road Safety Regulation monthly report (shows new cases, active cases, and closed cases by month) |
4.9
The Safer Vehicles team tracks its own performance using various indicators. Monthly reporting for the team focuses on activity, such as the number of site reviews and the number of non-compliance cases completed.
4.10
The monthly report includes two indicators that are part of NZTA's overall performance framework. The first of these is the indicator included in NZTA's annual report about the proportion of non-compliance actions progressed within acceptable time frames. This measure provides accountability for NZTA's timeliness in responding to non-compliance, which was one of the major issues contributing to the regulatory failure.
4.11
The other indicator is the number of compliance activities completed in the year to date.13 In 2023/24, NZTA completed over 5200 such reviews, significantly exceeding its annual target of 3500. We understand this target is based on the number of reviews needed each year to ensure that all vehicle inspectors and inspecting organisations are reviewed at least three-yearly. The total number of reviews completed includes overdue reviews.
4.12
Although this indicator tells us that NZTA is meeting its activity target, there are no indicators that tell us what impact this activity is having on vehicle safety. As a result, it is more difficult for NZTA to determine whether it is over- or under-investing in this.
4.13
A separate monthly report to NZTA's leadership, which covers all aspects of road safety regulation, includes additional performance information about the Safer Vehicles team. As well as a monthly breakdown of performance for the two indicators described above, this report shows a percentage breakdown of the levels of compliance and non-compliance found during site reviews for vehicle inspectors and inspecting organisations. However, there is no target for the overall level of compliance, or the rate of improvement, so it is difficult to determine whether a result represents good or poor performance.
4.14
As discussed in Part 3, other reporting about the regulation of vehicle inspectors and inspecting organisations is also available. Examples we saw included the results of quality assurance reviews and reporting of progress made on the backlog of site reviews.
4.15
Current reporting has value as a management tool because it shows whether the Safer Vehicles team is getting through the volume of work it is required to complete for the year and whether its work meets time and quality expectations. Managers need to know when expectations are not being met so they can then intervene as needed.
4.16
What the reporting does not show is how effectively NZTA's regulation of vehicle inspectors and inspecting organisations contributes to positive road safety outcomes.
Improvements could be more targeted if there was better information
4.17
We were told throughout our audit that NZTA has significantly improved its regulation of vehicle inspectors and inspecting organisations since the regulatory failure and that the basics of a good regulatory system are now in place.
4.18
We saw that NZTA was committed to continue making improvements to increase efficiency and enhance regulatory practices. Examples of recent improvements included:
- trialling a digital version of site review documents, to save time for Certification Officers; and
- creating a new non-technical role to carry out assessments of quality management systems, so that Certification Officers have more time to assess vehicle inspectors (see paragraph 3.22).
4.19
However, improvements could make a greater difference if NZTA had a clearer understanding of the relative effectiveness of the range of interventions available.
4.20
Better information could also support more efficient ways of working. For example, NZTA uses some analysis to identify risk (see paragraph 3.29), but with deeper insights about which vehicle inspectors and inspecting organisations are more at risk of non-compliance, NZTA could take a more targeted compliance approach and focus its resources more effectively. This could also reduce the burden on more compliant vehicle inspectors and inspecting organisations.
4.21
Another potential improvement could be if a pattern of non-compliance points to an aspect of vehicle inspection where greater education or focus is needed, this could be addressed centrally rather than on a case-by-case basis.
4.22
In our view, further development of data and analytics capability would support NZTA to have the information needed to make these types of improvements, including deeper insights about performance to identify where its work could be more efficient.
| Recommendation 3 |
|---|
We recommend that the New Zealand Transport Agency Waka Kotahi further develop its data and analytics capability to:
|
The public could be better informed about performance
4.23
There are opportunities for NZTA to be more accountable to the public for its performance and to give the public more confidence in the integrity of the system for issuing warrants and certificates of fitness.
4.24
NZTA provides some information about its regulation of vehicle inspectors and inspecting organisations on its website, as well as the indicators in its annual report (see Figure 5). More information, targeted to different audiences (for example, the public or the vehicle inspection industry), would provide better transparency and accountability for the quality and effectiveness of its regulatory work.
4.25
Information currently available on NZTA's website includes the VIRM and the appointment requirements. NZTA's newsletter for vehicle inspectors can be accessed through the vehicle inspection portal. NZTA also publishes a general regulatory newsletter, but this has limited information about vehicle inspectors and inspecting organisations. This newsletter sometimes covers the prosecution of a vehicle inspector but does not include information about how well the monitoring function overall is performing.
4.26
NZTA has published the inquiry and review that were completed after the regulatory failure, as well as the follow-up review in 2021 (see paragraph 2.17). This provides some transparency, but could be improved by the addition of more recent updates about the progress it is making with previous recommendations. It is important that the public see NZTA learning from its past and adjusting its approach based on these reviews and what it finds through its regulatory activities.
4.27
We encourage NZTA to consider what information the public would find useful and how best to communicate it. This might vary for different groups. For example, the vehicle inspection industry will have different information needs to vehicle owners.
| Recommendation 4 |
|---|
| We recommend that the New Zealand Transport Agency Waka Kotahi regularly publish information about the impact of its regulation of vehicle inspectors and inspecting organisations and about the progress it has made with recommendations from internal and external reviews. |
The vehicle inspections system will need to adapt as new technologies emerge
4.28
Improving the current approach for regulating vehicle inspectors and inspecting organisations is important but, in our view, NZTA needs to consider what long-term changes are also needed.
4.29
Warrants and certificates of fitness are currently issued based on a physical inspection of a vehicle's mechanical safety features and systems. Some vehicles now have safety features, including automated features, that use more advanced technology (for example, blind spot monitoring and autonomous emergency braking). These cannot be checked with a physical inspection and are not looked at when a warrant or certificate of fitness is issued.
4.30
Some vehicles need an exemption because they do not meet the prescriptive requirements in the Rules. For example, a vehicle previously needed an exemption if it used cameras instead of mirrors for side and rear views. Cameras allow a driver see to the side or behind, but the Rule required mirrors. NZTA had to make case-by-case exemptions for these vehicles to get a warrant or certificate of fitness until the Rule was updated.
4.31
Each example like this requires its own exemption, and we were told there are currently hundreds of exemptions in place. This is not efficient and risks compliance being overlooked or misunderstood because of the resulting complexity.
4.32
The Ministry of Transport's August 2022 long-term insights briefing about automated vehicles noted that the scale and nature of future advances, and how they will be received by consumers, was uncertain.14 There could be safety benefits from more automation, but the risks were not yet well understood.
4.33
Some advanced safety features use software that can be updated remotely. The functioning and security of these systems, and the need to keep software updated, are examples of risks that need to be considered.
4.34
The insights briefing recognised that New Zealand does not currently have systems to regulate automated vehicles and that the standards for vehicle inspections will need to be overhauled to include an emphasis on vehicle software and sensor systems. At a minimum, regulation will need to be fit for purpose for automated vehicles and those with non-observable safety features.
4.35
Although it is likely that physical inspection of vehicles will continue, different skill sets will be needed to test electronic and software-driven systems. This will affect both the vehicle inspection industry and its regulators.
4.36
We understand that policy work to reform the vehicle regulatory system, including vehicle inspections, is expected to start in 2025.
4.37
As this work progresses, we encourage organisations to consider how best to address the challenges identified in our audit related to the complexity and prescription of the Rules. This could support consistency and make it easier for vehicle inspectors and inspecting organisations to understand and comply with requirements.
| Recommendation 5 |
|---|
We recommend that the New Zealand Transport Agency Waka Kotahi and the Ministry of Transport progress work to ensure that vehicle inspection requirements are:
|
13: This refers mostly to site reviews.
14: Ministry of Transport (2022), He whakamahuki i matapaeroa: Long-term Insights Briefing, at transport.govt.nz.