Part 4: New certification and surveillance systems

4.1
In this Part, we discuss:

• the design of the new certification and surveillance systems;
• our assessment of whether it is likely to address the recommendations in our 2005 report; and
• the expectations we will use when we audit the CAA later this year.

Background

4.2
Our 1997 audit identified some serious problems with the standard of safety audits of civil aviation operators carried out by CAA safety inspectors.

4.3
Our audit in 2000 established that, while the CAA had made progress, we still had concerns.

4.4
The CAA introduced a new organisational structure in May 2000 and most recently in 2007. In 2000, the safety audit unit (a unit within the Safety Certification Group) was amalgamated into the operational groups. In addition, small aircraft operators were required to gain certification by February 2003, which effectively changed the surveillance approach taken by the CAA safety inspectors towards these operators.

4.5
Our audit in 2005 covered both the certification and surveillance functions, to assess whether:

• the certification (or entry) function ensured that prospective operators understood and were capable of complying with the Civil Aviation Act, the Civil Aviation Rules, and the conditions of their aviation document(s); and
• an effective surveillance function was operating, to ensure that an acceptable level of civil aviation safety was maintained.

4.6
While the certification process used by the Airline Group (which covers planes carrying 10 or more passengers) was generally sound, the process used by the General Aviation Group (which covers smaller planes, agricultural operators, and helicopters) was not as good. Six out of the 11 operators that we reviewed for the General Aviation sector appeared to have been certified without their understanding, or being able to comply with, their own expositions¹ and the Civil Aviation Rules.

4.7
We also still had significant concerns with the surveillance (safety audit) function. The areas that we had particular concerns about were:

• the effectiveness of the risk analysis and risk assessment • processes;
• ensuring that the risk analysis informed the surveillance process; and
• ensuring that operators, or groups of operators, that were assessed as “high risk” were appropriately targeted in both the depth and frequency of the surveillance carried out.

4.8
Our report in 2005 contained 10 recommendations (see Appendix 2).

Civil Aviation Authority’s response

4.9
In 2004, the CAA launched two major projects - the Surveillance Review Project and the Risk Assessment and Intervention Project. The objective of these two projects was to improve both the effectiveness and efficiency of the surveillance process.

4.10
Improvements in the effectiveness of the surveillance process would be achieved by providing managers with an up-to-date risk profile of each operator (based on relevant risk indicators) to enable them to target their surveillance activities. The project was also designed to provide managers with guidance on the most appropriate regulatory intervention for that risk profile.

4.11
Improvements in efficiency were to be achieved by automating the process and also by ensuring that staff carried out their assigned role – that is, that auditors audited, managers managed, and administrators carried out the administrative tasks.

4.12
A further project, the Certification project, began in 2005 to improve the processes for initial certifications and renewals.

4.13
Appendix 3 sets out the actions that the CAA has told us that it has taken (as at 6 December 2007) to address the recommendations in our 2005 report. Our next audit will examine the action that the CAA has said it has taken, and the effectiveness of the new systems that it has established.

Intended audit approach for the new systems

4.14
The new certification and surveillance systems should address our 2005 recommendations, if the system works in practice as it has been described to us.

4.15
In our next audit, we will examine whether:

• the surveillance staff follow the policies and procedures set out for certification and surveillance functions;
• during the certification process, the surveillance staff check the exposition to ensure that it addresses the aviation rules;
• the initial risk profile is reflected in the surveillance plan or audit modules and electronic checklists;
• the surveillance staff actually carry out the level of surveillance identified using the system, and that reasons for changes to the levels of sampling are clearly documented and properly approved;
• the surveillance staff consider changes to the risk assessment and the scope of the surveillance audit before they carry out a surveillance audit;
• the risk assessment and surveillance plan is refined as a result of a surveillance audit;
• the surveillance staff take appropriate follow-up action to address the findings of the surveillance audit;
• the surveillance staff take action when they are notified of a change in an operator’s risk profile;
• the surveillance plan results in a higher frequency of visits for high risk operators; and
• the surveillance plan results in all the rules being covered during the period of the licence.

4.16
We will also check that:

• guidelines have been prepared to indicate when instances of non-compliance should be referred to the CAA’s Law Enforcement Unit for further action; and
• finding notices are being issued for all identified instances of non-compliance and non-conformance, and that operators are taking timely corrective action in response to those finding notices.

1: An exposition is a set of manuals containing information about an operator’s (or operation’s) general policies, duties, operational control policy, procedures, and the responsibilities of personnel. The exposition is the principal means of showing that the management and control systems required under the Civil Aviation Rules are in place. Part 119 requires these manuals to include the instructions, procedures, and information necessary to permit the personnel concerned to perform their duties and responsibilities with an acceptable degree of safety. The information that must be addressed in the exposition depends on the scope of the operation.