Part 4: The certification function

• General Aviation sector
• Airline sector

4.1
The main purpose of certification is to ensure that prospective operators understand and are capable of complying with the Act and the CARs. It therefore restricts entry into the civil aviation system to those operators who will operate safely, and keeps potentially unsafe operators out. In this Part, we report on how well the certification process is being used to do this in both the General Aviation and Airline sectors.

4.2
We reviewed samples of certifications for each sector, to establish whether the process used was an effective “gatekeeper” in keeping unsafe operators from entering the civil aviation system. We consider that an effective certification process would ensure that operators have a good understanding of what is required in terms of their own expositions, as well as a good understanding of the Act and CARs requirements. We expected to find few instances of operators subsequently identified, through CAA surveillance activities, as failing to maintain the level of compliance and conformance necessary for certification.

General Aviation sector

4.3
Since our December 2000 audit, operators of aircraft with 2 or more engines who were previously operating under a Transitional Air Operator Certificate, were required to gain Part 119/135 certification by the end of February 2001. Single-engine, fixed-wing, and helicopter operators were required to gain their certification by the end of February 2003.

4.4
We reviewed a sample of 11 certifications (7% of the total number of General Aviation certifications). The behaviours demonstrated (as shown in the examples below) by 6 of these operators within 12 months of certification suggested that they had been certificated without understanding, or being able to comply with, their own expositions or the CARs.

The CAA’s compliance with its policies and procedures

4.5
Our review found that the CAA’s stated policies and procedures were not always followed. The procedures require:

• inspectors to review the exposition to ensure that it complies with the CARs;
• an entry inspection to confirm that the management systems detailed in the exposition are in place;
• an initial 6-month period during which a spot check is undertaken; and
• a compliance inspection near the end of the initial 6-month period to confirm that the operator can demonstrate compliance with their documented systems and procedures, and to establish whether these are adequate for the nature and size of the operation.

4.6
In one case, an operator was given a 6-month certificate without the exposition being fully checked to ensure that it complied with the Act and the CARs as required by the CAA’s policy and procedures. This was done on the basis that the operator purchased an exposition that was CARs-compliant. The Safety Audit & Entry Inspection Report observed–

The operator was found to be in compliance with all of the relevant rules and ready for certification under the new rules. Although the Exposition has not yet been critiqued in the detail required, it is generic to the degree that we have confidence that it shows rule compliance under Part 119/135. A detailed critique of the Exposition will be provided separately and it will detail those items that were not checked on this audit. The Company Base at … will be inspected as part of the compliance inspection.

It is proposed that a Part 119/135 certificate be issued for six months with a spot check to be held after approximately three months, and a compliance inspection be conducted at the five-month point to check that the procedures that are detailed in the IEX are being used.

4.7
The practice of buying a “generic” exposition from a supplier, which passes the CAA inspector’s critique, can lead to problems if the operator’s actual policy and procedures are different to, and not brought into line with, the generic exposition. In such cases, the operator will be non-conforming with their exposition and possibly non-compliant with the CARs.

4.8
For example, the report on the compliance inspection, (conducted 6 months after the entry inspection) on the operator with a generic exposition (paragraph 4.6) noted–

It is confirmed from the logbook inspection that the recording of maintenance as specified in the operator’s exposition and the rules has not been carried out… procedures should be introduced through the company’s quality assurance system to ensure the appropriate detail is contained in the logbooks.

4.9
Although we accept that, in this instance, the maintenance had been done, we were concerned that the inspectors, having noted during the compliance inspection that this operator had failed to conform with its own exposition, issued the operator with a long-term (5-year) certificate. Our understanding is that certification requires the achievement of a certain level of competency, and that, during compliance inspections, operators are required to demonstrate that management systems detailed in the exposition are in place and functioning as specified. We do not believe that was demonstrated in this instance. Moreover, we note that, in a subsequent safety audit conducted a year after the compliance inspection, the audit report observed–

Exposition procedures were not always being followed and the Company appeared to have put insufficient effort into reviewing and amending the manual to ensure that it remained a true reflection of the operation.

4.10
While CAA policy and procedures require an entry inspection and a compliance inspection, we noted one instance where these inspections were combined. The operator was certificated for 5 years, provided that a spot check was done within 3-6 months of certification, but we found no evidence that the spot check was done. It is therefore not surprising that, within 12 months of certification, the inspector noted problems with the operator being ignorant of many of the CARs requirements and its own exposition (see paragraph 4.15 – “the first operator”).

4.11
The CAA has advised us that this practice occurred occasionally where an already certified operator subsequently changed its name after completing the full initial entry process. While we acknowledge:

• that the operator was previously operating as a trading arm of another operator, and was seeking certification to enable them to operate independently; and
• that the exposition the operator was intending to use was virtually the same as that used in the operating arm,

the initial review of the exposition highlighted 44 corrections required to the exposition, and further corrections were required after the entry/compliance inspection. In our opinion, the issues with the exposition should have meant that combining the entry and compliance inspections was not justified in this instance.

Is the CAA’s certification process an effective “gatekeeper” for the General Aviation sector?

4.12
In order for the process to act as an effective “gatekeeper”, the General Aviation Group needs to be more rigorous in its assessment of the operator’s ability to comply with the Act and the CARs. Out of a sample of 11 operator certifications that we reviewed, we noted that CAA inspectors had identified significant problems with 6 operators who, within 12 months of being re-issued with a certificate for up to 5 years, were found to be not conforming with their expositions.

4.13
The problems identified from our sample were such that they brought into question the quality of the certification undertaken by inspectors. Our review raised questions as to whether operators had fully understood the purpose and requirements of their exposition and certificate, and whether certification had adequately “tested” the operators.

4.14
For example:

• For the first operator, within 12 months of certification, the routine audit report noted –

  …the Company is in ignorance of many of the requirements of the Rules and its own exposition procedures… extra spot checking will be scheduled over the coming year to confirm that the issues raised in this report are corrected and compliance is improved.

This operator also had a special purpose audit within 20 months of certification, because of concerns relating to the “risk” posed by the operator. Our expectation was that, having undergone certification, there should not have been such a significant change in behaviour over a 20-month period that the operator’s risk increased to the extent of warranting a special purpose audit.

• For the second operator, we noted from the routine audit report (12 months after certification) that–

  Exposition procedures were not always being followed and the Company appeared to have put insufficient effort into reviewing and amending the manual to ensure that it remained a true reflection of the operation. However, in spite of this, it was pleasing to see that there was a genuine desire to operate appropriately and safely, and there was no evidence to suggest that any unsafe practises were condoned or exercised by senior persons within the company.

• For the third operator, the report from the routine audit noted–

  The Auditor found that this system [quality assurance/management system] fell well sort (sic) of the compliance standard, whilst staff were trying hard to come to grips with a change in its operation and legislative requirements there can and was no excuse for the poor performance of this system…the system currently in place cannot and will not be able to keep the Company in a compliant state.

  From the number of Findings issued for Maintenance deficiencies it is evident that there are some major lapses with the Maintenance Control and this is common when one person carries out all duties.

This operator’s QI scores (59% for flight operations and 64% for maintenance) did not achieve the required 65% “pass” mark.

• For the fourth operator, the routine audit report (12 months after certification) stated–

  It was extremely disappointing to the auditor to find such a large degree of non-compliance, the problem is twofold the operator has to take responsibility for their own exposition and make sure that amendments are approved before inclusion into the exposition, they also need to ensure that the exposition reflects its own operation and that procedures contained within are those that the operator use …Confusion or mis-communication has led the operator into an unacceptable level of non-compliance.

• For the fifth operator, the routine audit report stated–

  The Exposition does not adequately describe the current organisation. Consequently, the operator does not comply with elements of the computer Exposition because those sections of the Exposition do not describe current Company practice.

  The Company has in place a number of systems/processes which are in compliance with applicable legislation but which are at variance to their Exposition …Particular items of concern have been identified as currency of Airworthiness Directives and the full completion of Logbooks.

• For the sixth operator, there were enough concerns about the organisation within 7 months of certification to require a special purpose audit. Resulting from that audit, 26 Finding Notices were issued.

Airline sector

4.15
To assess whether the CAA’s inspectors complied with their procedures (as noted in paragraph 4.5), we reviewed a sample of 8 certifications – 4 under Part 121, 3 under Part 125, and 1 that came under both parts. The sample covered almost a third of the total number of Airline operators.

The CAA’s compliance with its policies and procedures

4.16
Our review revealed few instances of non-compliance by CAA inspectors with the CAA’s policies and procedures.

4.17
In those cases where there was deviation by CAA inspectors from the policies and procedures, audits within a year of certification did not identify any significant problems with the operator. This contrasted with our findings relating to the General Aviation sector (see paragraph 4.13).

4.18
This situation may be due in part to the more complex entry requirements placed on the Airline sector than the General Aviation sector, which means that operators applying for Part 121 and 125 certification have more requirements to meet before being certificated. For example, on top of CAA certification and surveillance regimes, airlines are required to have their own quality assurance processes, the sophistication of which reflects the emphasis they place on safety.

4.19
Notwithstanding these findings, our file reviews highlighted 3 issues that we have raised with the CAA.

4.20
In the first instance, the certification process identified concerns with an operator’s quality systems and management acceptance of its Quality Management process. Instead of this matter being addressed at entry, it was transferred to the routine audit process for closure. Additionally, the recommended term for the certificate’s term from the compliance inspection report was exceeded – the certificate was granted for 2 years rather than the recommended 18 months.

4.21
A second instance was where the CAA’s files contained no evidence, other than Audit Work Request Control Sheets (see Part 7 of this report), that the required inspections for an operator’s certification had been conducted.

4.22
In the third instance, at the time an operator applied for a certificate, they suggested a time within which they wanted to be certificated, attaching to the application a schedule for meeting that deadline. The issue with this example is the appearance that the operator, rather than the CAA, determined the timeline for certification.

4.23
The Airline Group’s management expressed confidence to us that, while tight timeframes had been placed on the certification process, the standard of the process had not been compromised, and the dates in the plan would not have been agreed to if the standard had to be lowered to meet them. Nevertheless, the CAA’s internal auditors have commented that a number of operators have established “aggressive start-up times”, and that this has placed pressure on the Airline Group’s ability to meet its planned commitments for the year (on top of apparent staff shortages).

4.24
We consider that, while the CAA must be aware of the commercial realities facing airline operators, its paramount concern should be the compliance of those operators with the CARs, not the need to achieve certification by a date set by the operator.

Is the CAA’s certification process an effective “gatekeeper” for the Airline sector?

4.25
We consider that the certification process for the Airline sector is generally sound, in that the certifications we reviewed were not subsequently found to be deficient through audits. This may be due in part to the more complex entry requirements placed on the Airline sector.