FAA Certifies the Boeing 737 Max
The Boeing 737 Max has been grounded since March of 2019 after two aircraft crashes killing everyone on board.
The first crash killed 189 people, the second crash killed 157.
On Wednesday, November 18, 2020, the FAA Cleared Boeing’s 737 Max To Resume Passenger Service
After 20 months on the tarmac following two fatal crashes, Boeing’s troubled 737 Max airliner has been given the green light to resume passenger flights, the Federal Aviation Administration announced Wednesday.
The plane’s return to the skies will not be immediate, however. The FAA is requiring a series of design changes laid out in a 115-page directive. It also put forward training requirements for pilots and maintenance requirements for airlines.
“This airplane has undergone an unprecedented level of scrutiny by the FAA,” Dickson said. “We have not left anything to chance here.”
After the FAA announcement, the Air Line Pilots Association released a statement saying it “believes that the engineering fixes to the flight-critical aircraft systems are sound and will be an effective component that leads to the safe return to service of the 737 MAX.”
Culture of Concealment
Boeing was aware of issues but did not disclose them to pilots or the FAA.
Investigators found a “culture of concealment” as well as “grossly insufficient oversight by the FAA.”
A Few Lines of Code
Leeham News author Bjorn Fehrm has interesting details in his take FAA recertifies Boeing 737 MAX
Fehrm says “The 737 is a Safe Aircraft” and this “chain of events will not happen again on an updated 737 MAX“.
Much of the discussion in from a pilot’s perspective that is hard to follow but the key details are easy to understand even if you do not understand the terminology.
- The MCAS (Maneuver Characteristics Augmentation System) software was inaccurately classified as non “hazardous.”
- The inaccurate classification allowed a single sensor to control the MCAS.
- The MCAS was inaccurately coded.
- The original MCAS listened to the Speed Trim reset, “the Pilot trims,” instead of the correct “AoA is below the threshold again.” The result was MCAS trims, the Pilot trims, MCAS trims, the Pilot trims…. After 24 rounds in the Lion Air jet, the Pilots lost the race with MCAS.
- “MCAS went from a Pilot assist to a highly hazardous function by this single mistake in the MCAS software code. The whole drama came from the omission of a few code lines in the MAX Flight Control Computers software.“
Boeing Changes
I describe the above in sufficient detail so we can understand how little in MCAS needed change to take it from a hazardous function to one that would have caused no trouble if wrongly triggered.
In addition to this change, Boeing has made additional changes to increase safety further.
A single sensor no longer triggers MCAS. Both AoA sensors on the 737 MAX have to agree on the aircraft AoA, or Speed Trim including MCAS is deactivated (neither is needed to fly the plane. They are augmentation functions, i.e., good to have but not necessary).
On top of the dual-sensor activation of MCAS, its global authority, no matter what, is limited. The Pilot always has enough pitch control to fly the aircraft.
MCAS is Now Safe
To make MCAS safe, we only needed the correct reset criteria. But as the investigations dug deeper into how Boeing and FAA could miss how dangerous the original MCAS was, the requirements for changes grew. All eventualities, even remote ones, should be covered.
About Bjorn Fehrm
My Boeing contact who sent me the Leeham article notes Bjorn Fehrm is a former fighter test pilot and an aero engineer based in France.
Bjorn has said he would pilot the MAX as well as fly in it as a passenger.
Most Expensive Lines of Code in History
- Boeing is out $20 billion, not counting pending lawsuits.
- 346 people are dead.
- 450 aircraft are grounded worth about $45 billion.
- Mistrust of Boeing and the 737 Max will last for years.
Had a few lines of code been properly placed, there would not have been two crashes or 20 months of grounding even though other safety features were needed.
In retrospect, it is not really the lines of code that were the problem.
It was the “culture of concealment” coupled with “grossly insufficient oversight by the FAA.”
Addendum
The Seattle Times has an excellent article on what happened in both crashes that is very readable by a lay person.
Q&A: What led to Boeing’s 737 MAX crisis
For those who blame pilot error, note that “Boeing assumed that the pilots would realize what was wrong and react appropriately within four seconds” without even being aware there was an MACS fighting their decisions.
And in June 2018, before the first crash, another Boeing engineering memo acknowledged that a slow reaction by the pilots, if they took 10 seconds to react instead of four, would be “catastrophic.” These memos produced no change to the design.
The FAA did not see those memos.
Comments from my Boeing Contact
My experience in watching air crashes is, after a while, memory fades. Most people will don’t pay attention to the type of aircraft they fly. Happened with the DC1- and the Lockheed Electra, and earlier, with the British Comet. After 6 months to a year of safe operations, I think the MAX will be accepted. As Bjorn Ferhm said in his LNA piece, the 737 (airframe) is basically a safe aircraft with a 50 year history. But time will tell.
Mish
I disagree with the findings in two very specific ways:
(a) The aircraft is certified in the US but not in Europe or Asia — they will see that the FAA was coopted by Boeing in its approval process. they will want to review these things themselves and not believe the Americans…who can blame them
(b) the MAX is the first commercial Boeing aircraft that employs the Airbus aircraft management system — the aircraft is in charge and the pilot is an assistant. This is a major shift in Boeing’s operation philosophy and design that has been under-discussed
(3) the aircraft was built with an inferior wing that should have been built specifically for the aircraft. Hence the requirement for MCAS so that the aircraft can operate safely.
Now the truth is that there is nothing wrong with the concept of an unstable aircraft (that’s how military aircraft operate) and that’s to a certain extent how Airbus aircraft function. Still, it is an important change for Boeing that was “shoved under the carpet”
They had 4 seconds to get it right. From the addendum
“Boeing assumed that the pilots would realize what was wrong and react appropriately within four seconds.”
“And in June 2018, before the first crash, another Boeing engineering memo acknowledged that a slow reaction by the pilots, if they took 10 seconds to react instead of four, would be “catastrophic.” These memos produced no change to the design,”
“Although Boeing had installed a warning light to alert the pilot if the two AOA vanes disagreed, because of a software error this didn’t work. It was functional only if the airline had paid for an optional extra that added the AOA reading on the primary flight display.”
“Neither Lion Air nor Ethiopian Airlines had paid for that option. Boeing knew about this flaw in 2017, a year before the crashes, but didn’t consider it critical.”
While developing the fix for MCAS, the FAA discovered a separate problem, which is that a very unlikely glitch in the microprocessor inside the jet’s flight control computer could theoretically create a similar scenario to the two crashes even without MCAS activating.
If the two pilots had performed the correct procedures this would have never happened. It is called “Runaway Trim” and the procedure is to turn off the auto-trim function and trim by hand. They not only failed to do this but never pulled the power back and let the plane go uncontrollable. The problem still needed to be fixed but these pilots were not properly trained to fly this plane.
Nailed it. ALL the comments here are good, to the point, including Mish. Buuut, yours’ is the only pilot relevant ans. Therein is a fascinating study I’ve mentioned in other posts here. The study that revolutionized the international flight industry. Back in the day the world airlines were deeply concerned about the awful mileage/accident history world-wide. They commissioned a million dollar study: why were these accidents occurring? The airline industry gave the scientists 1yr to come back w/an ans. After months the researchers were despairing; no difference in monies spent by any country’s experts. Then, pow! lights went on. A researcher noted a compelling correlation/causation (?) in the data. They had their ans and immediately reported to the airline consortium: the ans? Religion!!! The industry reacted as expected, but the researchers convinced them of the ans and the training manuals were forever changed and re-issued. Result? the world airline industry thrived.
They had 4 seconds to get it right. From the addendum
“Boeing assumed that the pilots would realize what was wrong and react appropriately within four seconds.”
“Although Boeing had installed a warning light to alert the pilot if the two AOA vanes disagreed, because of a software error this didn’t work. It was functional only if the airline had paid for an optional extra that added the AOA reading on the primary flight display.”
“Neither Lion Air nor Ethiopian Airlines had paid for that option. Boeing knew about this flaw in 2017, a year before the crashes, but didn’t consider it critical.”
As a former USAF pilot, I am mystified by the lack of a simple “disconnect” button on the control column that would allow the pilot to disconnect the computerized system in the event that the system is overriding his manual input regarding trim. In fact, there is normally a circuit breaker that would disable the trim augmentation in the event of what is known as “runaway trim”. Pilots need to be able to manually override any computerized control systems and also need to be able to trim the flight controls independent of the augmentation system. Generally speaking, modern airliners are designed to be flown almost from takeoff to landing by computerized autopilot systems. If a pilot cannot fly the airplane from takeoff to landing manually, then he isn’t really a qualified pilot. He’s nothing more than a highly-paid computer operator.
I mentioned that many times in the past.
Thanks
Wrong computer language used for control, has to be functional, not procedural. I talked this over with software manager for a major fighter, she admitted I was right.
I worked on 2 fighters, problem #1 was WRONG computer language being used for the planes, has to be functional language, not procedural. problem #2 was no accountability – company afraid to confess failures, self-interest, govt won’t confess up, doesn’t want to look bad, self-interest feeding each other’s coverup. Where to start? Independent 3rd party auditing, but open to public scrutiny?
Don’t blame the contractors who wrote the code. They had to provide Boeing with development plans, system requirements, module requirements, code reviews, test plans, and test reviews. Boeing had to sign off all those reviews. DO-178 is the standard process to be followed for airborne software.
A lot more went wrong. The single sensor, changing the authority of the MCAS to 5× the original value without re-evaluating everything and without resubmitting the designs (which is a felony), regulators and industry in bed with each other, regulators outsourcing their own work to Boeing, an old design with motors too heavy and large requiring them to be hung to far to the front and changing the flights envelope.
It’s complicated. With better code there wouldn’t have been a crash, but a whole lot more things went wrong.
I used to fly a lot pre-Covid. Going forward, who knows. I swore I’d never set foot in a MAX, but I guess I’ll start riding them when I forget about the crashes.
American Airlines put out a statement that they would identify the MAX during booking, and that if you found yourself switched to one at the gate, they would accommodate you if you wanted a different aircraft.
To assign this to a couple of lines of missing code is amateur thinking. Systems architecture engineering was awful (information input channels, real-time logic, safety control, quality checking, etc.). This is what happens when you modify an old design from the standpoint of the end goal. The change ramifications are never properly explored – after all, the end goal (profits derived from an expedited, low cost solution) is what was required.
I flew on these planes a few times before they were grounded. Nice seats…good wifi…lots of comfortable amenities compared with the old 737’s…..only had that one little problem….hehehe.
Nothing has changed, don’t get, fly, or ride first models of anything until the bugs have been worked out by a few years of use. That being said Boeing seems to have some serious QC issues. Also Boeing’s space operations have been getting their doors blown off by Elon Musk/SpaceX. Again QC issues slowing them down big time.
Culture of concealment is common in corporate America. Flying isn’t free so caveat emptor.
You nailed the stark truth!
All well and good until the GRU is able to hack the max.
The BA CEO was shown the door taking a benefits package worth $62 Million with him. From everything I have read it sounds like someone or some folks should be doing some serious time for this fiasco. But there again you can’t believe everything you read I guess.
The culture of concealment is former`s CEO Dennis Muilenburg way of doing things to inflate company stock valuation. He is fully responsible for the destruction of boeing reputation.
I haven’t designed any flight control systems but I have designed many process automation systems. We used a PHA (Process Hazard Analysis) method which would have easily identified the issues with the MCAS. It would have been classified as a SIS (Safety Instrumented System) and with the hazard this high it would have used probably three AOA sensors, each of different type (to prevent a common mode failure) with 2oo3 (Two out of three voting) for any control action and at least two if not three processors (in case there was a failure with one processor). An alarm would have been indicated as soon as any of the three sensors did not agree within a designated tolerance. It is hard to believe that there isn’t some type of international standard for the development of aircraft flight control systems like there is for process control systems (ISA 84).
“It is hard to believe that there isn’t some type of international standard for the development of aircraft flight control systems like there is for process control systems (ISA 84).”
Not difficult to believe at all. In a world where “regulation bad” “bureaucrats bad” “committee rules bad” rule anything that regulates an industry will be crush by the political parties that think market forces will correct for human errors.
I am an engineer in heavy industry and I am very familiar with functional safety management. I just wanted to mention my appreciation for your post. I completely agree with what you said. And… i have thought at times of the risks that will start to show-up as we become more automated, and I expect the type of issues with the Max to show up more and more with coding and process control in other industries.
I would suggest you obtain USAF report on flight failure of SN4014 F22 and see what they had to say about software failure. Most interesting report! Would have been caught had functional language been used.
I would guess the software wasn’t developed and tested in the US. Most likely India. By people who have probably never flown in an airplane. You get what you pay for.
Developed at Boeing Seattle, not India.
By $9.00 per hour H1-B workers in Seattle, most likely Indians.
From: link to seattletimes.com
“
In his opening statement Wednesday at the House Aviation subcommittee hearing on the 737 MAX in Washington, D.C., the lead Republican congressman blamed errors by the Indonesian and Ethiopian pilots for the two deadly MAX crashes in those countries.
“Pilots trained in the United States would have successfully been able to handle” the emergencies on both jets, said Rep. Sam Graves of Missouri, ranking member of the House Transportation and Infrastructure Committee
”
How ignorant! How arrogant!
You are in good company Hottub!
You’re joking?
$9 an hour for programing by anyone in the US? For software in a regulated industry?!? Ha, ha, ha, ha, ha. Did you forget the zero on the right? 🙂
In 20+ years of IT project management in the US I have yet to witness or hear of any US company ceding all responsibility for application and development to an offshore software engineers. In addition, some of the best application developers, scrum masters and testing leads that I have worked with were located in India. I will admit that working with offshore developers does pose some additional challenges but none that can’t be overcome by strong onshore project management. At the end of the day complete responsibility for this debacle lies with Boeing and the FAA.
I never said Boeing in the US wasn’t involved. They almost certainly had managers overseeing the development. And I never said they weren’t to blame. I managed development groups in India for over a decade. I’m fully aware of what’s required. I’m guessing the vast majority of code and the vast majority of testing was done in India. The US managers likely didn’t have the skill to look at the code. They were just filling out progress reports and project plans. And there are some good technical personnel in India, but, on average, it’s been my experience they’re not as good as american educated workers.
737 Max problem:
“culture of concealment” coupled with “grossly insufficient oversight by the FAA”
Kidhorn’s problem:
“culture of arrogance” coupled with “grossly insufficient knowledge”
It’s more than the coding, they reconfigured the dual cut off switch making it not possible to use (manual) trigger trim adjustment with auto/mcas disabled.
…and downsized the manual trim wheel if I remember, making it difficult/impossible at high resistance (runaway trim) especially at low altitude, whole arrangement signed off by FAA, EASA approved without further testing. There is a story about the memory/processor used also, ties into cut-off switches but too complex for me to remember if it was valid.
Point is the programmer programs what the designer/management asks for, and I think this is what they did.
It really doesn’t matter where the programmers are from, when you have a programmer that doesn’t understand the process that needs to be controlled (in this case flight controls) then the result will be suboptimal at best. Boeing shouldn’t outsource to inexperienced people. Boeing’s desire to save a dollar cost them billions. And the programmer can’t make up for the design flaw of having only one AOA sensor. Here is an article on the outsourcing issue although it states that Boeing did not rely on these two Indian firms for the MCAS software.
Too sarcastic. Problem is the computer language being used. You must use functional language, not procedural. We condensed 187 lines of C++ code to 11 lines of Zspec. and that code is totally failure proof.
Airlines will now phase out the name “max” and refer to teh plane as 737-8
It is the same as the current “self driving” vehicles–where the driver is supposed to be in “watchful attendance” to the driving process.
Except, for most people, over time they drift into a deeper reverie, where the first reaction to an oncoming incident is over-reaction to something that has been ignored slightly too long.
The interface of human/machine is fraught with peril, even more so when the machine fights with the human for control.
I have long wondered why Boeing’s purchase of McDonnel-Douglas (sp?) was not an anti-trust issue. Only 2 biggies – and they merge? Cozy Washington relations? Airbus was not a big competitor back then. Anyway, what if Boeing had have had a competitor? Could that have changed their behavior? After all they seemed callus and ‘too big’ to be challenged.
I read several articles that explained that Boeing, in an attempt to compete with a new AirBus plane that was superior in design, decided to use larger engines than the Max was initially designed to use……which is what led to the need for the sophisticated fly-by-wire override in the first place.
The alternative would have been to go back to the drawing board and design a new plane from scratch….which they decided would cost too much and take too long.
“moving the engine nacelle (and a related change to the nose of the plane) changed the aerodynamics of the plane, such that the plane did not handle properly at a high angle of attack. That, in turn, led to the creation of the Maneuvering Characteristics Augmentation System (MCAS). It fixed the angle-of-attack problem in most situations, but it created new problems in other situations when it made it difficult for pilots to directly control the plane without being overridden by the MCAS.”
I expect the plane will be fine, given the scrutiny and the level of attention given to the problem and its software fix….but the underlying problem with the aircraft not handling well at certain angles of attack is not going to go away.
We can assume that any pilot who flies the airplane knows all about the problem now and has been vetted on the new system….and probably could avoid the problem in the first place by not taking the plane to the angle where it becomes vulnerable to misbehave.
I wouldn’t afraid to fly on one…..but that doesn’t make it a great airplane….and I think saying that a few lines of code “fixed the problem” is a little misleading.
Meant to say I wouldn’t be afraid to fly on one.
Not completely accurate. The overarching goal was to reduce pilot training costs by making the training of a larger plane the same as the existing smaller planes already in the fleet. Southwest, for example, only flies 737 planes and the addition of the Max would allow them to fly more passengers and with more efficient engines but not incur additional type-training costs because its “the same plane, but larger.”
Hard to build redundancy into a computerised system where workable management is dependent on every input and calculation being correct. Centralised system gives major failure when it goes into error. The older mechanical alternative , say dual wires with one redundant, dual controls with one only for emergency etc. doesn’t combine failure of any one facet of control with others. Usually it is found that disasters occur in those systems through unusual combination of simultaneous failure, whereas with the max failure was inbuilt to keep going to disaster repeatedly until the whole system was redesigned. This has happened mechanically as well on other aircraft previously, a design error for example , but then the first thing they do is ground all aircraft until cause is known. With the max and computer error they just decided to be able to blame the pilots 🙁 .
I have talked many times with a couple of my pilot friends that pilots have become dependent on computers and are losing valuable skills in the air.
Yep. You won’t see any Captain Sully’s ever again when a plane gets into real trouble.
The pilots fought the computer for control. Don’t blame the pilots, blame Boeing.