Part 2: Civil Aviation Authority's response to the Coroner's recommendations

Responses to the Coroner's recommendations on the June 2003 Air Adventures crash.

2.1
The CAA was responsible for responding to 24 recommendations from the Coroner.

2.2
In this Part, we discuss our assessment of whether the CAA properly considered the Coroner’s recommendations, took timely action based on that consideration, and reported its progress accurately. In making our assessment we:

examined the process that the CAA had used to decide on its response to the recommendations;
checked, where action had not been taken in response to recommendations because the CAA considered New Zealand’s current practice to be in line with international standards or already allowed for what the Coroner recommended, that this was correct;
verified that the decisions the CAA had made about the recommendations, and the actions that it had taken, were documented and supported by appropriate evidence;
considered the timeliness of the actions taken;
confirmed, where work still had to be done, that a plan was in place to complete the work; and
examined the process used by the CAA’s executive management team (the Executive) and the Board to sign off the response to each of the recommendations, to ensure that the process was robust.

Our expectations

2.3
In assessing whether the CAA properly considered, took timely action on, and accurately reported its responses to the Coroner’s recommendations, we expected as a matter of good practice the CAA to have:

assigned responsibility to an individual or team qualified to consider what action needed to be taken;
monitored and reviewed the action taken to ensure that it was sufficient, appropriate, and timely; and
ensured that the appropriate authority signed off its acceptance of the decisions made and action taken.

Summary of our findings

2.4
In our view, the process used to examine each recommendation, and the range of information used by the CAA in forming its conclusions, provides evidence that the CAA had properly considered each of the Coroner’s recommendations and responded in a timely manner.

2.5
Overall, we found that:

responsibility for the recommendations was assigned to qualified personnel;
the Executive monitored progress;
the Board signed off the Executive’s responses to the Coroner’s recommendations; and
decisions made about the response to each of the recommendations were documented and supported by appropriate evidence.

2.6
Having reviewed the way in which the various issues had been considered, and the actions that had been taken, we have confidence in the CAA’s decision-making process.

Assigning responsibility to qualified personnel

2.7
The Executive assessed the recommendations and assigned them to operational groups. The General Manager of each of the operational groups made a further assessment to determine which recommendations would be handled directly, and which would be assigned to a small project team headed by the retired Deputy Director of Civil Aviation. The retired Deputy Director was appointed head of the project team because of his experience and knowledge of the aviation industry and the CAA.

2.8
The project team comprised the retired Deputy Director and a staff member from each of the operational groups: the General Aviation Group, the Airlines Group, and the Personnel Licensing and Aviation Services Group (PLAS). Other members were co-opted when required.

2.9
A Steering Group, which comprised the Executive, was set up to oversee the process.

2.10
The project team was set up at the end of July 2006. Terms of reference were prepared to set out the scope of the team’s work. The terms of reference required the team to:

assess the assigned Coroner’s recommendations, formulate a CAA policy position, and where required decide on consequent action(s);
submit the proposed actions to the Steering Group for approval;
draft plans for implementing the approved actions and consult with all stakeholders as necessary during the development of the plans;
submit the plans to the Steering Group for approval;
if necessary, modify draft plans to incorporate Steering Group requirements; and
hand over final plans to the General Manager of the Group identified as responsible for implementing them.

2.11
Twenty-one of the recommendations were assigned to the project team for research, assessment, and a final judgement about whether they could or should be implemented (in part or in full). The other three recommendations remained with the General Manager PLAS because they belonged more appropriately with this Group.

Monitoring of progress

2.12
An Action Tracking Sheet was set up to monitor the progress in implementing the CAA’s (and the Ministry’s) responses to the Coroner’s recommendations. For each of the Coroner’s recommendations, the tracking sheet detailed the agency responsible for it, the action that the agency was taking, and the progress that had been made. The tracking sheet was updated periodically (on a monthly or bimonthly basis) and was available on both the CAA’s and the Ministry’s websites. The CAA and the Ministry set target dates for completing proposed actions. Where a proposed action included more than one significant task, they set proposed dates for completing each task. The milestones that had been achieved since the previous update were also included in the tracking sheet.

2.13
We reviewed the timeliness of the CAA’s completion of proposed actions against the timeframes published in the tracking sheets. We noted only two instances where the target dates were either not achieved by the target time or not achieved within two months of the target time. One instance was a recommendation (paragraph 586 of the Coroner’s report) that required the CAA to consider implementing a system that allowed consumers to gauge the safety record of an operator. The second was a recommendation (paragraph 598 of the Coroner’s report) that required the CAA to monitor the individual pilot, separately from monitoring the operator, from a competency and safety perspective.

2.14
These items were still incomplete at the end of our audit. Both have been included in the CAA’s business planning process. We will see what progress has been made when we return to the CAA later this year.

2.15
The project team reported to the Steering Group on 14 September, 5 October, and 31 October 2006. The project team’s reports were included as an Appendix to the tracking sheets posted on the CAA’s and Ministry’s websites.

Signing off the responses to the Coroner’s recommendations

2.16
The project team gave its assessment of each of the Coroner’s recommendations to the Executive in January 2007.

2.17
In response to the assessment, further action continued during the next seven months. In August 2007, the Executive prepared a paper for the Board that summarised the work of the project team and the Executive’s conclusions on each of the recommendations. The report also detailed the changes that had been made to respond to the recommendations (in some cases the corrective action was still in progress), and the reasons why no action had been taken for some recommendations.

2.18
The Board considered and endorsed the report at its meeting on 31 August 2007, subject to a few changes. The final version - Civil Aviation Authority Report on the Evaluation and Assessment of Coroner’s Recommendations (the evaluation and assessment report) - was completed and approved by the Board at its October 2007 meeting.

2.19
Of the 24 recommendations, as at October 2007, the CAA:

accepted and completed, or was still taking action to complete, 11;
considered that nine were addressed through the Act, the current aviation rules, or were in line with international standards; and
had not accepted four because the CAA had carried out alternative actions instead.

Documentation and supporting evidence

2.20
In Figures 1 to 3, we have paraphrased the evaluation and assessment report approved by the Board in October 2007. We have grouped the recommendations based on the type of response by the CAA (accepting the recommendation and taking action, considering it already dealt with, or taking alternative action). The full version of the report is on the CAA’s website (www.caa.govt.nz).

2.21
We have audited the information supporting the report. We can confirm that the evaluation and assessment report correctly reports the evaluation process used, the information considered, the conclusions reached, and action taken by the CAA.

2.22
Responding to the Coroner’s recommendations was a significant piece of work for the CAA. The Coroner’s recommendations covered improvements to aviation rules, pilot training and testing requirements, and pilot monitoring (including the effectiveness of the surveillance system). The recommendations required a range of responses that varied in complexity. Therefore, the time and resources required to respond also varied, from major system changes to considering the adequacy of existing aviation rules. In some cases, the CAA’s response was a specific and once-only action, and in other cases the CAA’s response has become part of its ongoing business.

Figure 1
The CAA’s response to the 11 Coroner’s recommendations that it accepted

Coroner’s recommendation	That the CAA examines the requirement as to reporting of occurrences to ensure understanding and consistency of application. Paragraph in the Coroner’s report: 555
The CAA’s response	The CAA reviewed how it engaged with the aviation sector to improve awareness of: the obligation to report occurrences; the mechanisms available for reporting occurrences; and the need to report occurrences to build a reliable statistical data base. The CAA has implemented the recommendation.
Action taken by the CAA	The CAA will continue with efforts to increase awareness in the aviation sector of the need to report occurrences, and the methods for doing so.

Coroner’s recommendation	That the CAA give close consideration to Dr Sharples’ submissions and sources in considering outcomes from this inquest with particular reference to mandatory reporting of colleagues where aviation practice falls below acceptable professional standards. Paragraph in the Coroner’s report: 581
The CAA’s response	The CAA considered Dr Sharples’ submission in the context of existing CAA surveillance policy. The CAA concluded that: mechanisms are in place to report safety concerns or risks with the performance and practice of others in the sector to the CAA; and how the CAA uses the information reported to it is an important issue.
Action taken by the CAA	The CAA has: reviewed and revised its policies on the use of information reported by industry participants; and restructured to enable more effective and transparent relationships between its investigatory and safety information functions.

Coroner’s recommendation	That the CAA urgently review and upgrade single-pilot IFR training and testing requirements, including night flying and flying in adverse meteorological conditions. Paragraph in the Coroner’s report: 559
The CAA’s response	The CAA considered the practices of other civil aviation authorities, and the weather conditions associated with comparable rules in those countries. The CAA determined that the existing rules could be improved by raising the requirements for single-pilot Instrument Flight Rules and night operations.
Action taken by the CAA	The CAA has upgraded training requirements for single-pilot Instrument Flight Rules and night operations. The CAA has completed proposed rule amendments. Instrument Flight Rules and night flight examination requirements were being reviewed.

Coroner’s recommendation	That the CAA urgently review single pilot IFR processes, requirements and best practices, including the use of coupled approaches and the identification and assistance to pilots who demonstrate any difficulty in IFR procedures. Paragraph in the Coroner’s report: 564
The CAA’s response	The CAA reviewed New Zealand Civil Aviation Rules Parts 61.37 (c) and 61.807. The CAA: agreed with the recommendation to review single-pilot Instrument Flight Rules processes, requirements, and best practice, including the use of coupled approaches; but considered that identifying and helping pilots who show difficulty in Instrument Flight Rules Procedures was already addressed by the New Zealand flight training system.
Action taken by the CAA	The CAA: published an advisory circular in February 2006; upgraded minimum flight experience and training requirements for pilots of small aeroplanes in commercial air operations; and was reviewing whether the Rules should require currency for single pilot operations in the single-pilot role.

Coroner’s recommendation	That there is a mandatory requirement for a mechanism of passenger complaint for passengers on commercial flights in the GA sector. Paragraph in the Coroner’s report: 578
The CAA’s response	The CAA reviewed existing mechanisms for passengers on commercial flights in the General Aviation Sector to make complaints. The CAA concluded that: existing systems provide a mechanism for complaints, but they could be made easier for passengers to access and understand; passengers make complaints when they are aware of potential risks, issues, or concerns; and the CAA website could be more user friendly.
Action taken by the CAA	The CAA: will amend the rules to make passenger safety briefing cards (including details on how to lodge a complaint on safety concerns) mandatory; will investigate ways to inform the public about the 0507 4 SAFETY telephone number; and has revised and updated its website.

Coroner’s recommendation	That the CAA adopt a lower threshold than was apparent from the evidence at this inquest with respect to the activities of Air Adventures and Mr Bannerman, to toleration of deviation from the Rules that affect the safety of passengers. Paragraph in the Coroner’s report: 582
The CAA’s response	The CAA reviewed its Surveillance Policy to determine whether it was clear about the use of available regulatory tools and the circumstances in which they could be applied. The CAA agreed that the threshold referred to by the Coroner must be clearly recognised by all CAA staff, and that staff must have clear guidance in identifying the regulatory tools to be used.
Action taken by the CAA	The CAA: revised and reissued its Surveillance Policy (which now includes guidance on regulatory tools available to enforce compliance); and is improving induction and ongoing training.

Coroner’s recommendation	That the CAA gives consideration to implementing a system whereby consumers can gauge the safety record of an operator. Paragraph in the Coroner’s report: 586
The CAA’s response	The CAA considered that the recommendation should be subject to further policy evaluation before such a system could be implemented. The CAA agreed with the Coroner’s recommendation. Subject to the relevant policy analysis, an amendment could be sought to the Act to enable an Operator Safety Rating System to be introduced.
Action taken by the CAA	The CAA will: do policy work on developing an Operator Safety Rating System; and provide an information pack to the public that outlines the safety issues and risks associated with different types of aviation activity.

Coroner’s recommendation	That the CAA implements a process of monitoring the individual pilot, separate from monitoring the operator, from a competency and safety perspective. Paragraph in the Coroner’s report: 598
The CAA’s response	The CAA assessed this recommendation in the context of the Act and associated rules. The CAA database allows for monitoring pilots. When specific concerns about a pilot’s behaviour are registered with the CAA, the system has been shown to be responsive.
Action taken by the CAA	The CAA will do more policy work on the safety benefits of monitoring individual pilots (as well as monitoring operators and/or organisations).

Coroner’s recommendation	That the CAA further consider and improve the effectiveness of its surveillance system for operators and pilots in the GA sector and give consideration to the merits of the information being made available to the operator. Paragraph in the Coroner’s report: 605
The CAA’s response	The CAA agreed with the recommendation. In 2004, it started to review its audit and intervention processes. The CAA concluded that the Surveillance Review Project and the Risk Assessment and Intervention Project would address the recommendation.
Action taken by the CAA	The CAA will: continue to develop its Surveillance and Risk Assessment and Intervention processes; and provide an updated risk profile to each operator or organisation audited.

Coroner’s recommendation	That the CAA require operators with three or less aircraft operating from two or less bases to have a simple form of organisational management system which reflects the reality of the operation and reflects safe practices. Paragraph in the Coroner’s report: 607
The CAA’s response	The CAA assessed the requirements in the Rules. The CAA accepted the Coroner’s recommendation and has amended its operator certification and surveillance procedures.
Action taken by the CAA	The CAA is strengthening its certification procedures to ensure that an operator’s management system appropriately reflects the complexity of their business.

Coroner’s recommendation	That the CAA reviews the [Minimum Equipment List] MEL process to ensure the adequacy of the process to require safe, up-to-date and operable instruments for flights with fee-paying passengers. Paragraph in the Coroner’s report: 612
The CAA’s response	The CAA reviewed its Minimum Equipment List process and the relevant rules governing Minimum Equipment Lists. The CAA considered that the process was robust and appropriate. However, it conceded that the rule requiring 100% operability in the absence of a Minimum Equipment List is impracticable for day-to-day flight operations.
Action taken by the CAA	The CAA will consider whether amending the rules to require a Minimum Equipment List in all Instrument Flight situations is justified.

Figure 2
The CAA’s response to the nine Coroner’s recommendations that it considered were already covered

Coroner’s recommendation	That the CAA reviews the adequacy of existing Rules as to: Minimum Decision Altitude requirements for single-pilot IFR; and Minimum visibility requirements for making an instrument approach. Paragraph in the Coroner’s report: 552
The CAA’s response	The CAA compared New Zealand’s minimum requirements to international standards - in particular, the Joint Aviation Authorities Joint Aviation Requirements. The CAA concluded that the existing standards were in line with international standards and practice.
Action taken by the CAA	The CAA will review the Rules to ensure that the intention of the Rules is reflected in the wording.

Coroner’s recommendation	That the CAA in conjunction with the Airways Corporation consider the adequacy of compulsory reporting of certain categories of Incident including where safety has been apparently compromised by the actions of the pilot of an aircraft. Paragraph in the Coroner’s report: 557
The CAA’s response	The CAA, in consultation with the Airways Corporation of New Zealand: considered the effectiveness of the current Rules; and reviewed two specific occurrences involving Air Adventures to assess whether the occurrences should have been reported. They concluded that the existing Rules for incident notification were effective, and that the two cases identified in the Coroner’s report did not suggest need for amendment.
Action taken by the CAA	No further action.

Coroner’s recommendation	That the CAA considers the adequacy of Rule 61.37(c) in relation to instrument approach and use of autopilot. Paragraph in the Coroner’s report: 561
The CAA’s response	The CAA reviewed Civil Aviation Rule 61.37(c). The CAA concluded that the rule was adequate, and that it would be inappropriate to link the rule with instrument approaches and the use of the auto-pilot.
Action taken by the CAA	No further action.

Coroner’s recommendation	That the CAA review the adequacy of Rules for non-certified GPS systems in relation to instrument landings to ensure a pilot-in-command cannot use the GPS system in instrument approaches and consider amending the Rules to require non-certified GPS systems to be disengaged before a pilot commences an instrument approach. Paragraph in the Coroner’s report: 567
The CAA’s response	The CAA reviewed the rules regulating the use of non-certified Global Positioning Systems in instrument landings. They concluded that the rules were adequate and already prohibited the use of non-certified Global Positioning Systems equipment in instrument landings.
Action taken by the CAA	No further action.

Coroner’s recommendation	That consideration be given to the CAA Rules being amended with respect to GA operators and pilots operating in the GA sector, to introduce a system of “demerit points” to take account on a cumulative basis (with appropriate time limitation periods) of any history of non-compliance with the Rules by the operator or, as the case may be, the pilot. Paragraph in the Coroner’s report: 583
The CAA’s response	The CAA reviewed the provisions in the Civil Aviation Act, and in particular the provisions of section 10. The CAA concluded that: the Act enabled it to assess participants’ ‘fit and proper person’ status and that this became part of their record of performance; and a change in the Act to introduce demerit points was not warranted because of the relatively low number of participants who would be issued with them.
Action taken by the CAA	No further action.

Coroner’s recommendation	That the CAA strengthens the criteria for requiring dual pilots for flights with fee-paying passengers. Paragraph in the Coroner’s report: 590
The CAA’s response	The CAA evaluated the current requirements for flight crewing, the safety risks associated with different types of operation, and the practical issues that arise from the Coroner’s recommendation. The CAA concluded that New Zealand conforms with international practice.
Action taken by the CAA	No further action.

Coroner’s recommendation	To monitor more effectively the safety of the General Aviation sector that the CAA obtain independent verification of all flight test results as they happen. Paragraph in the Coroner’s report: 600
The CAA’s response	The CAA assessed the recommendation in accordance with the division of responsibilities specified in the Civil Aviation Act, under which the Director monitors the exercise of the privileges and responsibilities of flight examiners. The CAA concluded that existing regulatory tools for monitoring the performance of Flight examiners and Instructors were satisfactory.
Action taken by the CAA	No further action.

Coroner’s recommendation	That the CAA be empowered to investigate the financial viability of an operator’s business, where the CAA has reasonable belief that the safety of the operation could be compromised. Paragraph in the Coroner’s report: 609
The CAA’s response	The CAA reviewed the powers vested in the Director under the Civil Aviation Act. The CAA concluded that the Act provides the Director with the power to investigate the financial records of an operator in the interests of aviation safety.
Action taken by the CAA	No further action.

Coroner’s recommendation	That autopilots be subject to regular functional tests to ensure their reliability for all purposes. Paragraph in the Coroner’s report: 614
The CAA’s response	The CAA reviewed existing rule requirements, relevant aircraft Flight Manuals, and Airworthiness Directives that apply to auto-pilot systems. The rules do not require the auto-pilot to have a coupled approach mode and this is in line with international practice. Auto-pilots are required to be maintained in accordance with the manufacturer’s instructions. The rules specify routine procedures for rectifying defects noted by the pilot.
Action taken by the CAA	No further action.

Figure 3
The CAA’s response to the four recommendations that it took alternative action on

Coroner’s recommendation	That the CAA Act and/or the Rules be amended to allow for all Occurrence Reports, ARCs and safety information relevant to a pilot and/or an operator, to be made available to the Flight Examiner of that pilot or any Instructor conducting type rating training carrying out competency checks. Paragraph in the Coroner’s report: 595
The CAA’s response	The CAA reviewed the existing flight training system and processes for carrying out competency checks, including issues of consistency with Flight examiners and Instructors, and the effectiveness of guidance information provided to Flight examiners and Instructors. The Flight examiner’s role is to apply objective competency standards at the time of the test. The CAA surveyed the opinions of practising Flight examiners. There was little support for the recommendation on the grounds of safety benefit, attendant risks to voluntary occurrence reporting, reduced objectivity, predetermination, privacy concerns, and practical implementation difficulties.
Action taken by the CAA	The CAA does not intend to adopt the recommendation. However, the CAA has introduced a comprehensive set of Flight Test Standards Guides for Flight examiners and Instructors, and conducts biennial Flight examiner and Instructor standardisation seminars for industry.

Coroner's recommendation	That the CAA take steps to encourage, where possible, Flight Crew Competency Checks for an individual pilot over a period to be carried out by different Flight Examiners. Paragraph in the Coroner's report: 602
The CAA’s response	The CAA assessed current systems for Flight Crew Competency Checks and sought comment from practising Flight Examiners on the benefits and usefulness (or otherwise) of requiring tests to be performed by different Flight Examiners.
Action taken by the CAA	The CAA considered that the step taken to provide additional flight test information and to improve standardisation by regular instructor/examiner seminars addresses the issue.

Coroner’s recommendation	That the CAA Rules require all aircraft providing passenger air transport services, be fitted with appropriate safety warning devices and other up-to-date instruments, regardless of the age of the aircraft. Paragraph in the Coroner’s report: 611
The CAA’s response	The CAA assessed the requirements of the current Rules that apply to passenger air transport services. The CAA concluded that, given the requirements in section 33 of the Act about rule-making, the recommendation was too broad in scope to be practical.
Action taken by the CAA	A rule requiring the fitting of terrain awareness and warning systems in aircraft operating under Instrument Flight Rules is intended to be in place by mid-2008.

Coroner’s recommendation	(i) That the CAA implement measures to enforce the provisions of Rule 91.7(a) with respect to IFR flights; and (ii) that the CAA consider Rules as to any pilot-in-command or co-pilot having a cell phone in his or her possession in the cockpit of an aircraft during flight. Paragraph in the Coroner’s report: 616
The CAA’s response	The CAA reviewed: the requirements of the Act and the rules about using cell phones on aircraft; safety risks associated with pilots not being able to access cell phones to communicate in certain circumstances; and existing educational and guidance information about using cell phones on aircraft. The CAA considered whether pilots should be banned from having cell phones in the cockpit during a flight, and concluded that such a ban would be counter to safety.
Action taken by the CAA	The CAA will continue to educate pilots on the current rule. It will enhance the enforcement of its provisions by drawing them to the attention of passengers through briefing cards. The briefing cards will also set out how to direct a complaint to the CAA. Otherwise, no further action will be taken.

page top