Appendix 3: Civil Aviation Authority's response to our recommendations

Responses to the Coroner's recommendations on the June 2003 Air Adventures crash.

The table describes the CAA’s response, as at 6 December 2007, to the 10 recommendations we made in our 2005 report.

Our recommendation	CAA’s response
We recommend that the CAA continue to establish measures to better assess the effectiveness of its safety interventions.	The Risk Assessment & Intervention Project, focusing primarily at the operator/participant level, is in the implementation phase. A research project with the objective of providing an indication of the safety benefit provided by the full range of higher level interventions, including for example training and education, inspection and monitoring, and enforcement, has yet to be scoped and carried out.
We recommend that the CAA improve its analysis of industry information by: including more analysis of the information in the Aviation Safety Report and the Aviation Safety Summary Report to support further action, and to improve the timeliness of these reports; and improving analysis of accident and incident data (for example, by identifying further opportunities – such as the CAA’s joint study of pilot-caused and controller-caused airspace incidents), from which the CAA will draft recommendations for safety intervention mechanisms.	A project is underway to define the needs of the three operational groups and to investigate the ability of the Safety Analysis Unit to provide the necessary information and interpretation. The CAA is reviewing the mechanisms and capability to enhance its ability to identify safety improvement opportunities and make recommendations for safety initiatives and interventions.
We recommend that the CAA further develop the tools it uses to assess the risk associated with individual operators. For example: For the non-compliance index to be more effective, CAA inspector need to correctly record all instances of non-compliance, as well as the actual audit hours spent with each operator. Operators need to be further encouraged to advise the CAA of instances of non-compliance. For the quality index score to be more consistent, it should be supported by the information in the routine audit report, and reasons for significant changes should be explained. For client risk assessments to be more useful to the surveillance process, the CAA needs to re-assess their function. These assessments identify changes to a company’s operation, but not necessarily changes to risk. We recommend that this tool be used to highlight any changes in the company’s operations for inspectors, who would then be responsible for assessing the effect of those changes on the risk of an individual operator.	This has been actioned by the CAA through staff directives and training. The new audit report format produced as part of the Surveillance Project makes it less likely that non-compliance findings are unrecorded in the audit report. Internal audits of the operational groups for the past year show a significant improvement in recording non-compliance. The Quality Index has been embedded in the risk profiling system that has been developed as part of both the Surveillance Review Project and the Risk Assessment and Intervention Projects. So far as the operator or participant is concerned, it has effectively been replaced by what is now known as the ‘Risk Profile’, which is now recorded in the Audit Report. The management of the Risk Profiles is under ongoing action by CAA management and staff. This is a core focus of the Risk Assessment and Intervention Project and linked directly to the Surveillance Review Project. Both these projects are now in the implementation phase. Also, the CAA Policy Unit is working on a policy paper to better define policy for the use of safety information within the CAA. The outcome of this work is intended, in part, to improve the handling and use of such information within the organisation. This should improve the accuracy of risk assessments, and enable more effective intervention at the operator or participant level.
We recommend that the CAA use better indicators of the financial status of operators when assessing operator risk, both at certification and during surveillance.	A review of available academic and aviation literature has not established a firm link between organisational safety performance and financial condition. Therefore, the CAA will retain the current financial risk parameter in the risk profiling system that examines a client’s financial status with respect to the CAA. Further, the Director of Civil Aviation has the power under the Civil Aviation Act to require detailed financial data from a client for certification or monitoring.
We recommend that the CAA ensure that its inspectors follow the policies procedures set down for certification.	The Director of Civil Aviation has issued a directive to staff that CAA are to adhere to approved policy and procedures. In addition, the Certification Project, which is intended to improve the consistency and effectiveness of certification, is nearing completion. The improvement, maintenance, and updating of CAA policies and procedures is an ongoing process. The CAA Internal Audit Unit (the Professional Standards Unit) reports that all CAA Groups and Units are satisfying this responsibility. CAA Document Control Procedures require all internal documents to be reviewed each year to ensure that they remain current.
We recommend that the CAA continue with its review of its surveillance function. In undertaking this review and designing a new approach, the CAA should: ensure that the audit process directs resources at the highest-risk operators; direct appropriate activities and interventions at high-risk Safety Target Groups; give priority to the sampling project (a sampling methodology will allow inspectors to make informed decisions on the work necessary to cover the assessed risk); assess where reliance can be placed on operator’s own quality and risk management systems, so that audits can be targeted at higher-risk areas; ensure that the depth and frequency of surveillance is adjusted to reflect operator and operation risk; and develop guidelines to indicate when instances of non-compliance should be referred to the CAA’s Law Enforcement Unit for further action.	The Risk Assessment and Intervention Project and the Surveillance Review Project have provided the tools to both direct interventions for high risk operators and to conduct surveillance on-site. The General Aviation Group is now moving from a time-based surveillance programme to one directed by the operator risk profile. The Risk Assessment and Intervention Project and the Surveillance Review Project have provided tools to assist inspectors when planning and conducting an audit. In addition, as noted above, the Risk Assessment and Intervention project provides the information for conducting risk-targeted oversight and interventions. The new Risk Assessment and Intervention Project provides the tools to allow this to occur, with the automated process providing an assessment of the operator’s overall safety risk. However, a ‘post implementation’ review of the risk profiling system is planned for May 2008 and one of the issues that will be considered then is whether an explicit assessment of an operator’s quality or safety management system should be added as one of the parameters on which the assessment is made. The Risk Assessment and Intervention Project and the Surveillance Review Project have provided tools to direct interventions at highest risk operators and to conduct surveillance on-site. The General Aviation Group is now moving from a time-based surveillance programme to one directed by the operator risk profile. The CAA’s Surveillance Policy was updated in December 2006 to provide guidance on when instances of non-compliance should be referred to the CAA’s Law Enforcement Unit for further action.
We recommend that CAA inspectors issue a Finding Notice for all identified instances of non-compliance and non-conformance.	The Surveillance Review Project, which is currently in its implementation phase, improves the requirement for findings notices to be raised. Improved manager review has also been introduced as part of the changed business processes.
We recommend that CAA establish a system that ensures that operators take quick and effective corrective action when inspectors tell them to do so. This system should include re-assignment of responsibility for that function when an inspector leaves the CAA.	The Surveillance Review Project has established the business process for ensuring that operators take quick and effective corrective action when inspectors tell them to do so, and for reassigning responsibility when an inspector leaves the CAA.
We recommend that CAA inspectors ensure that they record all time spent on the surveillance function. Continuing to do otherwise will affect the accuracy of the CAA’s risk analysis tools, and its ability to produce accurate business cases.	The Director of Civil Aviation has issued a directive to staff to ensure that all time spent on surveillance activities is recorded. This item is routinely audited during CAA internal audits.
We recommend that the CAA: ensure sufficient investment in training CAA staff so that they develop and maintain the appropriate skills to carry out their functions; review its staffing levels when the current review of the surveillance function has been completed, to ensure that it has sufficient resources to undertake this function (both the review of the surveillance function and the review of staffing levels need to take account of the potential pressures or “surges” put on inspectors as a result of unanticipated requests for certifications); ensure that the operational groups comply with the CAA’s generic policies and procedures (particularly relating to Quality Assurance); promote consistent standards of quality and practices throughout the operational groups by ensuring that they address internal audit Finding Notices; and ensure that the internal audit section is appropriately staffed to enable the CAA’s operations and inspectors to be audited on a more regular basis.	A training database has been developed as a tool to manage the training programme and the level of investment made. A policy and some procedures are being developed. The CAA has been subject to a number of reviews in recent years, and in July 2007 it restructured to improve its cohesiveness and effectiveness. The final staff appointment resulting from the restructure will occur early in 2008. Following the restructure, the Director is considering the need for additional resources in operational areas. This work remains linked, at least partially, to implementation of the Risk Assessment and Intervention Project and the Surveillance Project, which will not be fully in place until June 2008. It will be difficult to fully assess the resource implications of the projects until after June 2008. Procedures have been revised to ensure that the operational groups comply with the CAA’s generic policies and procedures. The issue of consistency in standards and practices has been resolved. The review of open internal findings is carried out fortnightly by the Professional Standards Group and regularly reported to the CAA Audit and Risk Management Committee. The current staff level for the internal audit section is considered sufficient to perform the required role.

page top