FAA Is Not Realizing The Full Benefits Of The Aviation Safety Action Program
By Dana Murphy
May 19, 2009, Office of Inspector General issued its report on the Federal Aviation Administration’s (FAA) Aviation Safety Action Program (ASAP). They conducted this audit following a complaint from an FAA inspector regarding the acceptance of a fatal accident into ASAP. ASAP is a joint FAA and industry program that allows aviation employees to self–report safety violations to air carriers and FAA without fear of reprisal. Incidents reported through ASAP by aviation employees are reviewed for acceptance by an Event Review Committee (ERC), which may also recommend corrective actions. Their audit objective was to assess FAA’s implementation of ASAP and identify any improvements that will help FAA to maximize the program’s safety benefits.
While ASAP is a potentially valuable safety tool, OIG found that FAA’s ineffective implementation and inadequate guidance have allowed inconsistent use and potential abuse of the program. Further, FAA has not devised a method to fully compile data reported through ASAP for analysis on a national level. Therefore, little is understood about nationwide trends in the types of violations reported under ASAP, and ASAP reports do not help FAA determine whether systemic, nationwide causes of those violations are identified and addressed.
As a result of these issues, ASAP, as currently implemented, is a missed opportunity for FAA to enhance the national margin of safety. To realize the full benefits of ASAP, FAA must improve the program in the following areas: (1) revising ASAP guidance to clarify which incidents should be excluded from the program, (2) emphasizing to employees that ASAP is not an amnesty program, (3) clarifying the ERC’s authority and role in ASAP and ensuring ERC members are unbiased, (4) requiring inspectors to examine repetitive reports of ASAP–related safety concerns and enhancements to ensure effective corrective actions, and (4) developing a central database of all air carriers’ ASAP reports that FAA can use for trend analysis at a national level. (See report below)
The Office of Inspector General conducted this audit following a complaint from a Federal Aviation Administration (FAA) inspector regarding the acceptance of a fatal accident into the Aviation Safety Action Program (ASAP). On January 16, 2006, a contract mechanic in El Paso, Texas, was killed while troubleshooting an engine oil leak as two pilots performed an engine run-up procedure. Within 24 hours, the pilots submitted a request for the accident to be accepted into the air carrier’s ASAP. ASAP is a joint FAA and industry program intended to generate safety information through voluntary disclosure that may not be otherwise obtainable to identify potential precursors to accidents. The program allows aviation employees to self-report safety violations to air carriers and FAA, including violations of Federal Aviation Regulations, without fear of reprisal through legal or disciplinary actions.
The inspector requested that we review the ASAP submission to determine whether FAA should have accepted the fatal accident into the program. To evaluate the complaint, we reviewed FAA’s guidance and procedures for implementing ASAP. Based on the results of that review, we expanded our audit to include ASAP at two other air carriers. Our objective was to assess FAA’s implementation of ASAP and identify any improvements necessary for FAA to obtain maximum safety benefits from the program. This report presents the issues we identified and provides recommendations for program improvements. This performance audit was conducted in accordance with generally accepted government auditing standards prescribed by the Comptroller General of the United States and included such tests as we considered necessary to provide reasonable assurance of detecting abuse or illegal acts. Exhibit A details our review objective, scope and methodology.
FAA’s Air Transportation Voluntary Safety Programs Branch manages voluntary aviation industry programs like ASAP. This office reviews program implementation and collects data and feedback from ASAP participants. The office uses these data to determine whether the program is achieving safety objectives.
The underlying principle of ASAP is to obtain voluntarily reported safety information to proactively prevent accidents and incidents. When properly implemented, this program could provide valuable safety data to FAA. We found, however, that FAA’s ineffective implementation and inadequate guidelines have allowed inconsistent use and potential abuse of the program. For example, the ERC that evaluated the ASAP report for the January 2006 fatal accident included an FAA inspector who had previously served as a pilot for the same air carrier as the two pilots involved in the accident. This inspector subsequently shared confidential hotline complaint information related to FAA’s acceptance of the accident into ASAP with the air carrier, which presented a conflict of interest.
Further, FAA has
limited the program’s effectiveness because it has not devised a method
to fully compile data reported through ASAP and analyze these data on a
national level to identify trends. FAA cites funding issues along with
air carriers’ concern for data protection as hindrances to developing
and implementing a program or method to identify safety trends on a
broader scale. While ASAP has proven highly beneficial to the airlines,
FAA currently obtains only limited aviation safety data through the
program for use in identifying systemic safety issues. As a result of
these issues, ASAP, as currently implemented, is a missed opportunity
for FAA to enhance the national margin of safety. To realize the full
benefits of ASAP, FAA must make program improvements in the following
Further, FAA guidance for ASAP does not clearly define the term “intentional disregard for safety.” FAA’s program surveys for ASAP in fiscal years (FY) 2005 through FY 2007 found that ERC members had difficulty interpreting this criteria for ASAP submissions. Without proper clarification, determination of intentional disregard becomes strictly subjective, which can impede FAA’s ability to take appropriate enforcement action. For example, the inspector who contacted our office about the 2006 accident believed that intentional disregard had contributed to the fatality, but the ERC members did not. At the time the accident was accepted into ASAP, FAA did not know the cause of the accident or whether it involved intentional acts. The inspector who made the complaint questioned whether events involving the loss of human life should be closed without an in¬depth investigation by FAA and with only minor administrative action taken against individuals that may have contributed to the accident. To enhance safety efforts, FAA should revise its guidance to refine this definition.
ASRS was established in
1975 under a Memorandum of Agreement between the FAA and NASA to
receive, process, and analyze voluntarily submitted incident reports
from aviation employees.
Further, FAA’s ASAP training does not address potential bias by inspectors assigned to an ERC. FAA’s ERC representative at the carrier involved in the 2006 fatal accident was a former pilot for the airline; he later shared confidential hotline complaint information related to FAA’s acceptance of the accident with the carrier.
According to FAA, US
Airways has reestablished its MOU with pilots, and American Airlines is
in the process of doing so.Check rides allow instructors to examine
flying proficiency and other factors that influence potential
performance, judgment, and stability. They examine how well a pilot
responds to the stress of in-flight emergencies in a flight simulator.
We are not advocating a
return to past practices where FAA relied primarily on penalties and
fines when airlines or aviation employees committed safety violations.
FAA believes safety partnership programs are valuable in forming
collaborative relationships with air carriers. Used properly, these
programs can indeed be important tools for FAA and the aviation
industry. However, partnership programs must be balanced with a strong
commitment to oversight so that they do not lapse into automatic amnesty
While this information
is valuable to individual air carrier safety programs, FAA has
overlooked an opportunity to enhance the national margin of safety by
collecting ASAP data that can be used for trending, identification of
risk factors and accident precursors, policy development, and
dissemination of collective safety data to FAA inspectors and other
carriers. An independent review team, convened by the Secretary of
Transportation in May 2008 to examine FAA’s safety culture and
management, also recommended that FAA compile and analyze ASAP data to
identify trends and patterns that represent risks.4 These actions would
also help to guarantee the integrity of voluntary reporting programs.
“Managing Risks in
Civil Aviation: A Review of the FAA’s Approach to Safety,” Report of the
Independent Review Team, September 2, 2008.
To determine whether accidents are valid contributions to ASAP, we attempted to examine air carrier ASAP data to identify the acceptance of other accidents into ASAP. Air carriers’ MOUs with FAA, which outline their ASAPs, stipulate that the air carriers keep all documents and records regarding the program. Due to strict confidentiality provisions, however, we were not able to access air carriers’ ASAP data needed to perform this analysis. FAA also could not provide us with the compiled data on a national level.
We therefore compared
ASAP to NASA’s voluntary reporting program, ASRS, in which analysts
collect, analyze, and respond to voluntarily submitted aviation safety
incident reports to lessen the likelihood of aviation accidents. FAA
provides most of the funding for ASRS, and NASA administers the program
and sets its policies in consultation with FAA and the aviation
community. Safety analysts identify safety hazards and issue alerting
messages to appropriate FAA offices or other aviation authorities.
De-identified information (i.e., all identifying carrier information is
removed) is also incorporated into the ASRS database, which can be
accessed by the public.
Another area in which ASAP guidance lacks clarity is with its use of the phrase “intentional disregard for safety.” FAA’s guidance states that violations involving an intentional disregard for safety should be excluded from ASAP. FAA guidance, however, does not provide any clarifying information on what this phrase means. FAA’s guidance5 provides only two examples of violations that would not be eligible for submission into the ASAP program because they involved an intentional disregard for safety. One example involves a first officer who falsely reported he had completed the exterior pre-flight inspection when he did not inspect the top wing surface for ice, resulting in an emergency landing. The other involves a mechanic who knowingly used an unauthorized lubricant on an engine valve to avoid delays in completing the job order. However, FAA’s guidance does not further explain this phrase under its “Key Terms,” which are defined “for the purposes of ASAP to ensure a standard interpretation of the guidance.”
In FAA’s annual reviews of the ASAP program in FY 2005 through FY 2007, several respondents indicated that they had experienced problems or interpretation issues with the meaning of “intentional disregard for safety” (see FAA’s pilot ERC survey results at exhibit C). We believe that this poses a significant problem because, without proper clarification, determination and interpretation of “intentional disregard” becomes strictly subjective and can complicate safety matters.
For example, the inspector who contacted our office after the El Paso mechanic’s death strongly believed that intentional disregard had contributed to the fatality, but the ERC members did not. The NTSB’s investigation report—issued on January 31, 2008—found that contributing factors in the accident were “the airline’s insufficient training of contract mechanics and the airport’s failure to disseminate a policy prohibiting ground engine runs above idle power in the terminal area.” The FAA investigation of the accident recommended a civil penalty of $55,000 against the airline involved.6 The two pilots received letters of warning, which stated that as the result of their “careless operation” of the aircraft, “passenger safety may have been jeopardized during the boarding operation….”
FAA Advisory Circular
120-66B, Aviation Safety Action Program (ASAP), November 15, 2002.
FAA Inspectors Do Not Have Timely Access to ASAP Reports: We found that FAA has not clarified when and how FAA inspectors assigned to ERC committees should obtain ASAP reports for review. The process for transmitting ASAP reports to FAA representatives was different at each of the three carriers we reviewed. When we initiated our review, we determined that some inspectors received the reports immediately upon employee submission, while others did not receive the reports until weeks after an event had occurred.
The inspector representing FAA on the committee that handled the January 2006 fatal accident did not receive ASAP reports until the ERC committee meeting on January 23—7 days after the accident. However, the air carrier and union ERC members obtained the ASAP report as soon as it was submitted.
It is important that inspectors receive the reports in a timely manner so they are able to review all pertinent FAA regulations and procedures prior to meeting with the other ERC committee members. The Director of FAA’s Air Transportation Voluntary Safety Programs Branch stated that a practice in which the Agency’s ASAP representative does not receive ASAP reports in a timely manner is inappropriate and certainly is not considered an industry “best practice.” The Director pointed out that FAA can revise the MOU language to address this issue.
The ERC that reviewed
an ASAP request from the other mechanics involved in the accident
rejected the request.
ASAP Is Viewed as an Amnesty Program: We found several instances in which FAA, air carrier, and union representatives incorrectly viewed ASAP as an “amnesty program” and believed that any corrective or enforcement actions for ASAP-reported incidents would be inappropriate. For example, pilots at one carrier objected to ERC members merely obtaining additional information about an incident disclosed in an ASAP report. FAA’s annual ASAP reviews have found similar concerns. Specifically, 25 percent of respondents in FY 2007 and 16 percent in FY 2006 indicated that ASAP was perceived as an amnesty program. In FY 2005, 50 percent of respondents indicated this perception (see exhibit C). We believe this is due to unclear FAA guidance on the authority of the ERC, which can adversely affect ASAP implementation.
In multiple, annual ASAP reviews FAA noted that ERC member and ASAP manager interviews indicated a need to further define what constitutes a corrective action, when an action is appropriate, and how an action should be recorded and tracked. According to FAA, in many cases, ERC members were reluctant to recommend corrective actions when the ASAP report was the only source of information about an incident due to concern that these actions might inhibit employee participation.
Confusion about the intent of ASAP has hindered the program at multiple air carriers. For example, Delta, American Airlines, US Airways, and Comair pilots ended their participation in ASAP because of disagreements regarding corrective actions. At Delta, pilot union representatives expressed concern about the airline’s policy of initiating additional corrective action (e.g., check rides) for pilots who have submitted ASAP reports. According to Delta managers, the union felt that once a pilot submitted a report to ASAP, Delta should have a “hands-off” policy and that it was inappropriate to interview the pilots. Delta pilots withdrew from ASAP in December 2006 and reinstated the ASAP program in January 2009 at the urging of FAA. Also, American Airlines and its pilots union were unable to agree on MOU language limiting the carrier’s authority to take disciplinary action for events reported under ASAP in instances where the carrier had independently obtained information on the reported event.
Under current FAA
guidance, employees submitting reports that are accepted under ASAP are
subject to the following FAA and air carrier actions:
To avoid the appearance
of bias in ERC reviews of ASAP reports, FAA needs to emphasize the need
for impartiality as part of its required ERC training and implement
procedures to require periodic refresher training in this area.
Additionally, FAA should clarify FAA field office management
responsibilities to ensure personal relationships between inspectors and
airline personnel do not influence decision-making. FAA Lacks a Process
To Effectively Collect and Analyze ASAP Data FAA needs to develop a
central ASAP database to collect information from ASAP reports in a
redacted form, but with sufficient information to identify trends.
Currently, FAA inspectors’ quarterly reports do not provide adequate
details about the nature of ASAP events or the effectiveness of
carriers’ corrective actions. Further, FAA does not have a process that
permits it to collect and analyze ASAP
FAA Inspector Reports Do Not Include Sufficient Information on ASAP Events or Carriers’ Corrective Actions FAA inspectors compile quarterly reports of ASAP activity for each participating air carrier and submit them to the Voluntary Safety Programs Branch at FAA Headquarters. We found, however, that these reports do not provide sufficient details about the nature of ASAP events to be effective safety data analysis and trending tools, and they vary dramatically in content among inspection offices.
Further, FAA guidance
for submitting quarterly reports of ASAP activity does not require
inspectors to provide summary information on all ASAP reports submitted
for the quarter. Each air carrier is required to identify notable
improvements made in the air carrier’s operation as a result of ASAP
reporting. However, these program improvements likely resulted from a
small fraction of the total ASAP reports submitted.
We also identified repetitive reporting of problems and safety enhancements from quarter to quarter. For example, we found that some areas of concern, such as including altitude deviations and checklist usage, continually showed up as unresolved in FAA inspectors’ quarterly reports. According to FAA’s guidance on ASAP, FAA requires carriers to follow through with corrective actions that are acceptable to all members of the ERC to resolve any safety deficiencies; otherwise, FAA could exercise the right to terminate the carrier’s ASAP. In our view, repetitive reports of the same problem would indicate that a carrier’s ASAP program is ineffective. Yet, we found that FAA inspectors do not flag these repetitive items or further examine the carriers’ corrective actions; rather, they simply include the issues in their report and send it on to the Voluntary Safety Programs Branch.
An independent review
team convened by the Secretary of Transportation in May 2008 to examine
FAA’s safety culture and management also identified this issue. The
team’s report stated that audits of safety violation disclosures and
carriers’ acceptances can validate adherence to program rules. Audits
would also ensure that FAA does not accept repeated disclosures from the
same regulated entity, which would indicate a failure to implement
sufficiently comprehensive fixes the first time. The report noted that
“any willingness on the part of the FAA (real or perceived) to accept
such repeat disclosures would undermine incentives for compliance.” Our
review found that at one air carrier incomplete or missed checklist
items were noted in the first quarter of 2006 as the most serious
events, which were to be emphasized in pilot retraining. Yet, this
problem persisted through the next two quarterly report submissions. The
fact that the same safety violations are recurring from quarter to
quarter indicates the need for heightened concern and stronger measures
to prevent recurring problems and ensure the success of the carrier’s
FAA Does Not Analyze
ASAP Data on a National Level To Identify Safety Trends In addition to
the lack of analysis of ASAP reports at the local level, we found that
FAA does not analyze ASAP reports at the national level. Although FAA’s
Voluntary Safety Programs Branch at Headquarters receives inspectors’
quarterly ASAP reports, FAA has been slow in developing a database that
would allow ASAP reports to be analyzed to identify safety trends.
Instead, the Voluntary Safety Programs Branch is only responsible for
developing policies and procedures to improve the program.
In 2008, the Secretary’s independent review team also identified this issue. Specifically, the team’s report stated that voluntarily disclosed safety data have not been routinely analyzed at a higher level within the FAA. The report recommended that FAA analyze Voluntary Disclosure Reporting Program (VDRP) and ASAP data (and data from many other sources) for two reasons: (1) to utilize these programs as potential contributors to the identification of trends and patterns that represent risks and (2) to guarantee the integrity of the voluntary programs themselves, eliminating any of the downside risks to compliance that might result from abuse.
To effectively use ASAP data, FAA needs to enhance the quarterly reporting process by requiring inspectors to submit summary information on all ASAP events, examine carriers’ repeatedly reported safety concerns, and ensure corrective actions were used to prevent a recurrence. Further, since the Voluntary Safety Programs Branch at Headquarters already receives inspectors’ quarterly reports, FAA should require this office to use these reports to develop a central ASAP database for national trending. These two efforts could provide FAA with valuable data for enhancing aviation safety nationwide.
FAA has contracted with Mitre Corporation to develop ASIAS, a system that will perform integrated queries across ASAP and other safety databases. Thirteen major air carriers are participating in ASIAS. FAA is using the system to assess the magnitude of problems that have already been identified. However, while FAA is continuing efforts to expand ASIAS functionality, the system does not have the ability to trend data to identify potential risk factors for FAA inspectors. In addition, access to ASIAS data is limited. Queries must be approved by an executive board, and results of ASIAS studies go to the Commercial Aviation Safety Team—a joint FAA and aviation industry group—and are not made public.
While FAA has established a good framework for obtaining valuable safety data through voluntary reporting by aviation employees, it needs to refine its current guidance to gain greater program benefit. FAA should consider developing a database tool similar to ASRS to include ASAP data. This would give FAA a centralized collection point of voluntarily reported safety data that could be used for national trending.
OIG Report Number
AV-2007-050, “Progress Has Been Made in Reducing Runway Incursions, but
Recent Incidents Underscore the Need for Further Proactive Efforts,” May
24, 2007. OIG reports are available on our website: www.oig.dot.gov.
ACTIONS REQUIRED: In accordance with DOT Order 8000.1C, we request that FAA reconsider its position regarding the part of recommendation 1 related to excluding accidents from ASAP reporting. Please provide your written response regarding recommendation 1 within 30 days of issuance of this report. FAA’s planned actions and target dates for recommendations 2, 3, 4, 5, 6, 7, and 8 are responsive, and we consider these recommendations addressed pending completion of the proposed actions. We appreciate the courtesies and cooperation of FAA representatives during this audit. If you have any questions concerning this report, please contact me at 202-366-0500 or Tina Nysted, Program Director, at (404) 562-3770.
From: Lou E. Dixon
Assistant Inspector General for Aviation and Special Program Audits
|©AvStop Online Magazine Contact Us Return To News|