March 29, 2010 - The National Transportation Safety Board adopted a study and concluded that small aircraft that had “glass cockpits” were not safer and had a higher fatal accident rate then similar aircraft with conventional instruments.

As a result the NTSB has proposed safety recommendations to the Federal Aviation Administration (FAA) and has requested the FAA respond within 90 days in addressing the actions taken or intends to take to implement the Board's recommendations.

The Board has recommended (in brief) that airman knowledge tests include questions regarding electronic flight and navigation displays. Training materials should cover electronic primary flight displays, initial and recurrent flight proficiency requirements should include flight displays requirements for pilots, as well develop and publish guidance for the use of equipment-specific.

Information supporting these recommendations is discussed in this letter and in the safety study. The NTSB would appreciate a response from you within 90 days addressing the actions you have taken or intend to take to implement our recommendations. 

In a span of only a few years, the cockpits of light aircraft have undergone a transition from conventional flight instruments to integrated, computerized displays commonly referred to as glass cockpits. This change has occurred rapidly. Data from the General Aviation Manufacturers Association (GAMA) indicate that by 2006, more than 90 percent of new piston-powered, light airplanes were equipped with full glass cockpit displays. Several manufacturers of glass cockpit displays now produce displays with supplemental type certification for retrofit installation in existing aircraft, suggesting that the number of aircraft equipped with full glass cockpits will continue to grow. 

The introduction of this advanced technology into light aircraft has brought with it a new set of potential safety concerns, including equipment design and operation; pilot performance and training; and new accident investigation techniques. 

This study was designed to test the hypothesis that the transition to glass cockpit avionics in light aircraft would improve the safety of their operation. The study also sought to evaluate the adequacy of resources and requirements supporting the transition to this new technology. To accomplish these goals, this study included three separate analyses, as described in the study report: 

·   A retrospective statistical analysis of accidents and activity data from two cohorts of recently manufactured airplanes produced with and without electronic PFDs, conducted to measure differences in activity, accident rates, and accident circumstances between glass cockpit and conventional aircraft. 

·    A qualitative review of FAA and industry training resources and requirements related to glass cockpit displays conducted to characterize the training and identify areas for potential safety improvement. 

·    A review of accident case studies conducted to identify emerging safety issues associated with the introduction of glass cockpit displays into this class of aircraft. 

Accident Involvement and Accident Rates 

The safety study compared a defined group of glass cockpit aircraft and a cohort of the same makes/models of aircraft with conventional instruments to reduce the potential for confounds associated with comparing aircraft of different age and capability. The study also used data from the FAA’s General Aviation and Air Taxi Activity and Avionics (GAATAA) Survey to make additional comparisons between aircraft using activity-based accident rates that reflect accident risk. 

Study analyses showed that glass cockpit-equipped aircraft experienced proportionately fewer total accidents than a comparable group of aircraft with conventional round-dial instruments. The fact that accident rates were higher for conventionally equipped aircraft than for glass cockpit aircraft might suggest a safety benefit resulting from the new technology—were it not for the fact that the study’s analyses also showed that the glass cockpit cohort had a significantly higher percentage of fatal accidents during the years 2002 through 2008 and that the fatal accident rate per 100,000 flight hours observed for this cohort in 2006 and 2007 was also higher. Data from the FAA’s GAATAA Survey confirmed that differences in the activity and usage of the two cohorts had likely influenced the type and severity of accidents experienced by each group. 

When considered as a whole, the study results describe two distinct aircraft operational profiles. Aircraft with conventional cockpit displays were more likely to be used for flight instruction. Accordingly, these aircraft were also found to have flown more hours per aircraft8 although they were used for shorter flights9 and flew less time in instrument conditions. As a result, aircraft in the conventional group were involved in more accidents during takeoffs and landings, which often resulted in less severe outcomes, most likely due to the relatively low speeds during those phases and the resulting low impact forces. 

Conversely, the operational profile of glass cockpit-equipped aircraft was found to involve fewer flight hours per year but longer trips. Consequently, the glass cockpit-equipped aircraft reportedly spent more time than conventional aircraft operating on instrument flight plans. The accident record is consistent with the way the aircraft were reportedly used. Glass cockpit aircraft experienced more accidents while on long trips and in instrument meteorological conditions (IMC) but also were reported as spending more time operating in instrument conditions. 

Previous NTSB research has identified a higher risk of aircraft on longer flights being involved in weather-related accidents and has noted that accidents occurring in IMC are more likely to be fatal due to the event profiles and impact forces typically associated with such accidents. The glass cockpit cohort experienced higher fatal accident rates and higher accident rates in IMC than the conventional aircraft—despite the fact that the pilots had higher levels of certification, were more likely to be instrument rated, had more total flight experience, and had more experience in the aircraft type. 

Based on the pattern of study results, the NTSB concluded that study analyses of aircraft accident and activity data showed a decrease in total accident rates but an increase in fatal accident rates for the selected group of glass cockpit aircraft when compared to similar conventionally equipped aircraft during the study period. Overall, study analyses did not show a significant improvement in safety for the glass cockpit study group. 

Safety Issues 

Training Resources and Requirements 

Despite efforts on the part of the FAA to develop resources and update training materials to address the needs of pilots transitioning to glass cockpit aircraft, the study identified several safety issues and areas for improvement. 

FAA airman knowledge tests, for example, do not currently assess pilots’ knowledge of glass cockpit displays. The NTSB concluded that pilots must be able to demonstrate a minimum knowledge of primary aircraft flight instruments and displays in order to be prepared to safely operate aircraft equipped with those systems, which is necessary for all aircraft but is not currently addressed by FAA knowledge tests for glass cockpit displays. The NTSB therefore recommends that the FAA revise airman knowledge tests to include questions regarding electronic flight and navigation displays, including normal operations, limitations, and the interpretation of malfunctions and aircraft attitudes. 

The NTSB’s review of the FAA’s training initiatives showed that the FAA worked with representatives from the general aviation industry and academia to develop its FAA Industry Training Standards (FITS) initiative in response to a recognized need for improved training for advanced aircraft systems. Initial planning documents show that although the FITS initiative intended to combine teaching techniques, such as scenario-based training, with requirements for equipment-specific training, the FAA has not implemented the equipment-specific training requirements suggested in the original FITS program documents. 

Providing Pilots with Information About Display Operation and Limitations 

The study considered several accident case studies that highlighted the complexity and unique functionality of glass cockpit displays in comparison to conventional instruments, as well as potential safety-critical issues associated with the design and operation of software-based systems. The case studies illustrate the importance of pilots’ receiving sufficient information about system operations and limitations so that they are prepared to identify and safely respond to system malfunctions and failures. 

The pilot involved in an accident on April 9, 2007, in Luna, New Mexico, found that glass cockpit displays may function differently than conventional displays under certain conditions. In that case, a blocked pitot tube intake that would have affected only the airspeed indicator of a conventional cockpit display resulted in a loss of airspeed, altitude, and rate-of-climb information in a glass cockpit display. The information provided to the pilot indicated only that the air data computer had failed, with no indication of why it had failed or whether the situation could be safely corrected in flight. 

The NTSB concluded that pilots are not always provided all of the information necessary to adequately understand the unique operational and functional details of the PFDs in their airplanes. Therefore, the NTSB recommends that the FAA require all manufacturers of certified electronic PFDs to include information in their approved aircraft flight manual and pilot’s operating handbook supplements regarding abnormal equipment operation or malfunction due to subsystem and input malfunctions, including but not limited to pitot and/or static system blockages, magnetic sensor malfunctions, and attitude-heading reference system alignment failures. 

Training Requirements 

As aircraft equipment becomes more complex, the demands placed on pilots to manage and monitor equipment operation will continue to increase. Findings of the FAA’s 2009 Part 23 - Small Airplane Certification Process Study, and comments included in pertinent draft FAA advisory circulars, suggest that the human-equipment interaction issues for Part 25 transport-category aircraft will become increasingly critical for Part 23 aircraft. 

In contrast to the generalized training traditionally required to operate the relatively simple systems in Part 23 aircraft, the complexity and variation of Part 25 aircraft systems have been addressed by requiring pilots to hold a type rating to act as pilot-in-command.14 However, now that light aircraft are incorporating integrated glass cockpit avionics that rival in complexity those in Part 25 aircraft, generalized systems training may not be sufficient for pilots of these aircraft. 

Different system architectures require different operating techniques, and responses to failure and knowledge of one type of glass cockpit display are not likely to transfer to other systems. The NTSB concluded that generalized guidance and training are no longer sufficient to prepare pilots to safely operate glass cockpit avionics; effective pilot instruction and evaluation must be tailored to specific equipment. 

Therefore, the NTSB recommends that the FAA incorporate training elements regarding electronic PFDs into its training materials and aeronautical knowledge requirements for all pilots. The NTSB also recommends that the FAA incorporate training elements regarding electronic PFDs into its initial and recurrent flight proficiency requirements for pilots of 14 CFR Part 23 certified aircraft equipped with those systems that address variations in equipment design and operation of such displays. 

Equipment Malfunction Training 

To be adequately prepared to respond to flight instrument system malfunctions and failures, pilots should be trained to identify and respond to all anticipated failure modes. However, in many cases it is neither appropriate nor practical to train for all anticipated types of glass cockpit avionics failures and malfunctions in the aircraft. The NTSB concluded that simulators or procedural trainers are the most practical alternative means of training pilots to identify and respond to glass cockpit avionics failures and malfunctions that cannot be easily or safely replicated in light aircraft. 

Pilots who do not have ready access to approved flight simulators or training devices could benefit from equipment-specific training using software applications or procedural trainers that replicate glass cockpit displays. Therefore, the NTSB recommends that the FAA develop and publish guidance for the use of equipment-specific electronic avionics display simulators and procedural trainers that do not meet the definition of flight simulation training devices prescribed in 14 CFR Part 60 to support equipment-specific pilot training requirements. 

Tracking Service Difficulties and Equipment Malfunctions 

NTSB investigations have revealed multiple instances of glass cockpit avionics malfunctions that were not required to be reported to the FAA and that did not result in a service difficulty report (SDR). Findings of the FAA Part 23 - Small Airplane Certification Process Study suggest a general difficulty with tracking Part 23 equipment performance due to SDR system underreporting for light aircraft. 

The NTSB concluded that identification and tracking of service difficulties, equipment malfunctions or failures, abnormal operations, and other safety issues will be increasingly important as light aircraft avionics systems and equipment continue to increase in complexity and variation of design, and current reporting to the FAA’s SDR system does not adequately capture this information for 14 CFR Part 23 certified aircraft used in general aviation operations.

The NTSB also concluded that the FAA’s current review of the 14 CFR Part 23 certification process provides an opportunity to improve upon deficiencies in the reporting of equipment malfunctions and defects identified by the FAA and aviation industry representatives in the July 2009 Part 23 - Small Airplane Certification Process Study.

However, the review of 14 CFR Part 23 and resulting regulatory actions will likely require considerable time. Therefore, to improve the voluntary submissions to the FAA SDR system in the interim, the NTSB recommends that the FAA inform aircraft and avionics maintenance technicians about the critical role of voluntary SDR system reports involving malfunctions or defects associated with electronic primary flight, navigation, and control systems in 14 CFR Part 23 certified aircraft used in general aviation operations.

Results

The results of this study suggest that, for the aircraft and time period studied, the introduction of glass cockpit PFDs has not yet resulted in the anticipated improvement in safety when compared to similar aircraft with conventional instruments. Advanced avionics and electronic displays can increase the safety potential of general aviation aircraft operations by providing pilots with more operational and safety-related information and functionality, but more effort is needed to ensure that pilots are prepared to realize that potential. Adoption of uniform training elements by the FAA to ensure pilots have adequate knowledge of aircraft equipment operation and malfunctions, as well as improved reporting of equipment malfunctions and service difficulties, is likely to improve the safety of general aviation operations beyond those involving aircraft with glass cockpit displays. However, such actions are particularly important in order to achieve the potential safety benefits associated with advanced cockpit technologies in light aircraft.

Therefore, the National Transportation Safety Board recommends that the Federal Aviation Administration:

·   Revise airman knowledge tests to include questions regarding electronic flight and navigation displays, including normal operations, limitations, and the interpretation of malfunctions and aircraft attitudes. (A-10-36)

·   Require all manufacturers of certified electronic primary flight displays to include information in their approved aircraft flight manual and pilot’s operating handbook supplements regarding abnormal equipment operation or malfunction due to subsystem and input malfunctions, including but not limited to pitot and/or static system blockages, magnetic sensor malfunctions, and attitude-heading reference system alignment failures. (A-10-37)

·    Incorporate training elements regarding electronic primary flight displays into your training materials and aeronautical knowledge requirements for all pilots. (A-10-38)

·    Incorporate training elements regarding electronic primary flight displays into your initial and recurrent flight proficiency requirements for pilots of 14 Code of Regulations Part 23 certified aircraft equipped with those systems that address variations in equipment design and operations of such displays. (A-10-39)

·    Develop and publish guidance for the use of equipment-specific electronic avionics display simulators and procedural trainers that do not meet the definition of flight simulation training devices prescribed in 14 Code of Federal Regulations Part 60 to support equipment-specific pilot training requirements. (A-10-40)

·   Inform aircraft and avionics maintenance technicians about the critical role of voluntary service difficulty reporting system reports involving malfunctions or defects associated with electronic primary flight, navigation, and control systems in 14 Code of Federal Regulations Part 23 certified aircraft used in general aviation operations. (A-10-41)

In response to the recommendations in this letter, please refer to Safety Recommendations A-10-36 through -41. If you would like to submit your response electronically rather than in hard copy, you may send it to the following e-mail address: correspondence@ntsb.gov. If your response includes attachments that exceed 5 megabytes, please e-mail us asking for instructions on how to use our secure mailbox. To avoid confusion, please use only one method of submission (that is, do not submit both an electronic copy and a hard copy of the same response letter).

Chairman HERSMAN, Vice Chairman HART, and Member SUMWALT concurred in these recommendations.

By: Deborah A.P. Hersman, Chairman