Uncorrected Design Flaws, Cyber-Vulnerabilities, and Unreliability Plague the F-35 Program
Uncorrected Design Flaws, Cyber-Vulnerabilities, and Unreliability Plague the F-35 Program
BY
DAN GRAZIER | FILED UNDER
ANALYSIS | MARCH 24, 2020
19 MINUTE READ
U.S. Air Force Brig. Gen. Brook Leonard, the commander of the 56th Fighter Wing, straps himself into the cockpit of an F-35A Lightning II at Luke Air Force Base, Arizona, Nov. 6, 2017. (Photo:
U.S. Air Force / Staff Sgt. Jensen Stidham)
Note: Portions of this analysis previously appeared in a recent POGO article.
As the F-35 program limps toward the end of its much-delayed operational testing period and subsequent full-rate production decision, unanswered questions about its combat effectiveness and suitability for service within the fleet remain. The Pentagon weapons testing office’s
2019 annual report, released earlier this year, paints a picture of an incompletely designed and vulnerable aircraft that may never be able to perform many of its intended functions.
The director of operational test and evaluation’s (DOT&E) report includes the following lowlights:
- The gun for the Air Force’s version not only can’t shoot straight, but breaks the aircraft when fired.
- There have been no appreciable improvements in the program’s overall reliability since 2016.
- The entire F-35 system remains vulnerable to cyber threats.
- The simulation facility necessary to fully test the aircraft and train pilots remains unfinished.
As the Project On Government Oversight (POGO) recently reported, a previously confidential
document produced by the F-35 program office plotted out the still-growing number of design flaws it was grappling with as of February 28, 2020. As the program approaches a full-rate production decision, the total number of reported unresolved design flaws has increased by 10, up from the total reported by the testing office in January—a time when the total number of flaws should be decreasing.
The F-35 Joint Program Office did not respond to comments for this report.
Unaddressed Design Flaws Adding Risk
The services set lofty goals for the F-35 program at its 2001 inception. The program established 536 performance specifications for the functional requirements of the aircraft and its components. As of September 17, 2019, the program had satisfied
only 493 of them.
The latest testing report on the F-35 shows there hasn’t been appreciable improvement in the program’s overall reliability since 2016.
The program office is expected to formally
revise the original contract with Lockheed Martin before the full-rate production decision to delete some of the requirements and complete the rest of the work during later development projects. The testing office’s report does not include a full list of the unmet goals, but mentioned that the aircraft is falling short of the airframe durability standards. That means the aircraft are
unlikely to last as long as planned, and readiness will be negatively impacted by the failure to build an effective logistics and maintenance network. The testing office reports that many of the unmet goals may never be achieved or will only be reached during later development projects.
Despite the Pentagon’s
self-congratulatory declaration in 2018 that the program had completed its troubled development process, the testing office reports that in fact the development phase contract “may take years to complete.” Meanwhile, pilots today are dealing with a problem-ridden aircraft that requires them to work around faulty components that, according to DOT&E, “may be observed from both operational testing
and fielded operations.” Translation: U.S. pilots will take underdeveloped and glitchy aircraft into combat for the foreseeable future.
In addition to the requirements that have not been—and possibly will not be—met, the program is still plagued by hundreds of unresolved design flaws. The Pentagon calls these design flaws “deficiencies,” and as of the release of the annual testing report,
the F-35 program had 873. Most were identified prior to the declared end of the program’s development phase—officially known as system development and demonstration—providing more evidence of just how premature the decision to end that phase was.
The Pentagon divides deficiencies into two categories based on severity and impact on safety and mission performance. The most serious are
Category I, or those that “may cause death, severe injury, or severe occupational illness; may cause loss or major damage to a weapon system; critically restricts the combat readiness capabilities of the using organization; or result in a production line stoppage.” The testing office stated in its annual report that the F-35 program still carries
13 unresolved Category I deficiencies.
The report does not say what the outstanding deficiencies are, but
a June 2019 report in
Defense News provided a list of the Category I deficiencies at the time. Engineers were contending with cabin pressure spikes that could cause the pilots ear and sinus pain, structural damage and stealth coating blistering caused by supersonic flight, and control issues in the Marines’ F-35B and the Navy’s F-35C that prevented pilots from fully controlling their aircraft during certain maneuvers.
The February 28 report on unresolved design flaws shows some improvement, with only nine remaining Category I deficiencies, four of which are listed as high priority. While it may be that the program in fact fixed four of the 13 most serious design flaws, POGO
previously found instances of officials in the program office simply altering the numbers.
Minutes from a 2018 F-35 program office Deficiency Review Board meeting showed that the office had been making paperwork changes
to reclassify some Category I deficiencies to a lower status instead of actually correcting them.
The endlessly patched software controlling all the F-35’s components and mission systems is unstable.
“The program shouldn’t enter OT&E [Operational Test & Evaluation] with deficiencies and certainly should not go into full rate production. You may be producing aircraft that are unsafe,” Tom Christie, who served as director of operational test & evaluation during the George W. Bush administration, told POGO. “Once you find out how bad things are and correct them, how many will you have to take out of the inventory to be fixed?”
The number of remaining design flaws is one thing, but the 2019 annual operational test report also highlights their persistent nature. The F-35 entered operational testing in December 2018 with a large “technical debt” of problems that had been identified but not corrected during developmental testing, as the Pentagon’s testing office
reported earlier this year. Of the 873 deficiencies identified by the testing office as of November 2019, approximately 576, or 66%, were carried over from the development phase. The program’s technical debt has only grown during operational testing as evaluators keep discovering new flaws, and the testing office’s report
cautioned that the unresolved flaws “should be addressed by the program to ensure the SDD [System Development and Demonstration] baseline configuration of software and hardware is stable, prior to introducing a large number of new capabilities to the software in the new hardware configuration associated with Block 4 [future development].”
What the testing office is saying in engineering parlance is that the endlessly patched software controlling all the F-35’s components and mission systems is unstable. The “
computer that happens to fly” is a densely integrated network of hardware, software, weapons, and mission data. Making a software change to any one component can,
and often does, have unintended negative effects on a seemingly unrelated component. The testing office wants to see the program correct all the existing flaws so the F-35 has a stable base on which to build as designers and engineers add new capabilities in the coming years. Unless this occurs, every time a new function is added, they will likely end up piling new flaws on top of old flaws, which will end up endangering schedules and increasing costs.
It remains to be seen how sustainable such a scenario is considering the program is already nearly a decade behind schedule with a budget that has more than doubled.
Cybersecurity Flaws Endangering the Entire Program
Software instability may be the least of the F-35 program’s cyber concerns. The F-35 relies on a tightly integrated network of computer systems both in the air and on the ground to operate. The problems with that system, called the Autonomic Logistics Information System (ALIS), have become so extensive that Pentagon officials announced in January they would
replace it with a new system by 2022.
For the time being, the program still requires the troubled system to run the F-35’s ground-based support. ALIS, which is operated by Lockheed Martin, integrates maintenance diagnosis, supply chain management, combat mission planning, and threat analysis. It is designed to capture maintenance and mission data generated on the aircraft, and then share it across the entire enterprise.
The wording of the testing office’s
assessment of the system is quite telling:
ALIS remains inefficient and cumbersome to use, still requires the use of numerous workarounds, retains problems with data accuracy and integrity, and requires excessive time from support personnel. As a result, it does not efficiently enable sortie generation and aircraft availability as intended. Users continue to lack confidence in ALIS functionality and stability.
The decision to scrap ALIS comes after the Pentagon spent years
trying to convince the American people that the $16.7 billion network would work. Leaders said they would switch to a new cloud-based system called Operational Data Integrated Network, or ODIN. This means that after Lockheed Martin failed to produce a workable system on its first try with ALIS, the Pentagon
will be paying the company again, to help build ODIN.
Only time will tell if engineers will be able to create a network that will be able to do what it is supposed to, but we already know that at present, program officials are not taking adequate steps to ensure the security of the F-35 and its support network while the program is still using ALIS. ODIN is expected to be up and running in 2022. Until then, the services are stuck trying to make do with the flawed ALIS program.
To a malign actor or a potential adversary, any digital network linking an entire fleet together looks like a gigantic target for a cyberattack. Michael Gilmore, the former head of the testing office
said in 2016 that the cyber vulnerabilities of the program were “significant.” Malware infiltrating the F-35 information systems is a concern, especially in the wake
of known cyberattacks. This concern is echoed in a March 2020 report by the
U.S. Cyberspace Solarium Commission, which recommended expanding the current efforts to hunt for malware in the Defense Department’s networks, saying that the current defensive posture has been “ineffective in preventing adversary cyber campaigns.”
The piecemeal manner in which the tests are being conducted is unlikely to reveal all the potential cyber vulnerabilities.
The F-35’s cyber-testing is not yet complete. The current plans leave much to be desired, and show that the testing to date has not been sufficiently rigorous. The testing office reports that the program has
yet to adequately address many of the vulnerabilities that have been found so far during testing. Several of these vulnerabilities have been known for years.
The piecemeal manner in which the tests are being conducted is unlikely to reveal all the potential cyber-vulnerabilities. Rather than testing on actual aircraft or support networks, all the testing to date has been done in isolated laboratories like the
Mission Systems Integration Lab in Fort Worth, Texas, where aircraft components are set up to mimic an F-35. DOT&E wants to move testing out of the laboratory and
onto the flight line because that is the only way to ensure the entire network is evaluated. The concern is that the laboratory does not fully replicate the program’s
attack surface or all the ways an attacker could send data to or pull data from the network.
In the case of the F-35 program, the possible penetration points are many and varied. Just on the aircraft itself, a hacker could potentially find access through the F-35’s communications components such as the navigation system, Link 16 datalink, and the Identification Friend or Foe system. On the ground, a hacker could access the aircraft’s systems through ALIS—and in the future through the cloud-based ODIN network—and through
the reprogramming laboratory where programmers create the F-35’s mission data loads. If malware capable of disrupting onboard systems, stealing data, or injecting false data were introduced to any of these systems, it could eventually spread throughout the entire F-35 fleet as data files are shared.
F-35 program officials do not want to test real aircraft or mission support equipment because they seem to believe doing so could disrupt F-35 operations. A series of cyber tests
were canceled in 2015 for fear that the tests could damage the software and essentially pollute the entire network with malware. By canceling tests or containing them within isolated laboratories, program officials tacitly acknowledge the vulnerability of the F-35 and its ground-based data networks, and that hackers or enemy cyber-warriors could inflict real damage.
A Fighter Plane That Can’t Shoot Straight
The F-35’s shortcoming that has so-far
garnered the most headlines in 2020 involves the gun installed in the Air Force’s model. The Air Force’s F-35A variant comes equipped with an internally mounted gun. Because the F-35 is supposed to eventually replace the A-10 in the all-important close air support role, an accurate and effective gun is of particular importance. History has proven that in most situations where friendly ground troops are in close contact with enemy fighters, or targets are close to civilians, accurate strafing is a better choice than missiles or bombs.
The testing office reports that little or no progress has been made to the F-35A’s inaccurate gun. As has been the case for years now, pilots cannot reliably hit targets with the F-35A’s gun. Testing
in 2017 showed that the gun hit long and to the right of the aiming cues projected into the pilot’s helmets. Engineers explored
multiple options to correctly align the gun’s boresight to the pilot’s aiming cues. They changed the way the gun is installed in the fuselage, updated boresight procedures, and modified the mission system software. But so far it’s not clear if the problem has been corrected because none of the solutions have yet to be successfully tested, according to the testing office.
Testing completed in 2019 revealed a new issue: Firing the gun causes structural damage the aircraft. The testing office reports that squadrons flying the newer-model F-35As found cracks in the underlying chine longeron skin near the gun’s muzzle following training flights during which the pilots fired the gun. A
longeron is a part of the aircraft’s frame used to reinforce the skin and the underlying bulkheads. The cracking problem is severe enough that the Air Force has taken the puzzling step of telling pilots they can only fire the gun for some F-35s in
combat only. Left unstated is how the Air Force expects them to shoot down an enemy plane if pilots are not allowed to fire the gun in training.
For all of these reasons, the testing office
reports, “the F-35A internal gun system remains unacceptable.”
To say something meets contract specifications is not the same as saying it is effective in combat.
The Marine Corps’ F-35B variant and the Navy’s F-35C use external belly-mounted gun pods. The testing office report provides far fewer details about them, saying little more than that the pods have not suffered the same inaccuracies as the F-35A’s gun. The
report says the accuracy results during testing have been consistent and “meet the contract specifications.”
The last statement is an odd one to appear in an operational test report, because it speaks to something that should have been completed during developmental testing. Retired Air Force Colonel Jim Burton described development testing in his book
The Pentagon Wars as “highly controlled, engineering-oriented tests designed to determine whether a new weapon system meets technical requirements and formal contract specifications.” According to Gilmore, the former head of the testing office, the contract specifications cannot capture everything a weapon system needs to do. “A combat aircraft that cannot fly or carry a weapons payload is clearly unacceptable,” he wrote in a
2015 article for
National Defense. “But an aircraft satisfying only those requirements will not be assured of penetrating enemy air defenses and destroying targets.”
Operational testing is supposed to determine whether the aircraft can perform in combat and is appropriate for use in the hands of the troops. To say something meets contract specifications is not the same as saying it is effective in combat. Any gun may be able to hit a target in laboratory conditions, but it is the ability to hit a target under combat conditions and produce the necessary effects that makes a weapon truly effective.
F-35 Testing Yet To Be Completed
Despite having had nearly two decades to prepare, the program failed to make adequate preparations to test the F-35. Program leaders planned from the outset to make extensive use of modeling and simulation to test the most complicated missions the F-35 is supposed to perform, but failed to sufficiently plan for or fund the necessary simulators. The open-air test ranges in the western U.S. lack the necessary equipment to recreate the same formidable anti-air defenses that a developed nation would field
against the F-35. Additionally, the ranges are not big enough to handle the planned tests of formations of up to eight aircraft to test the F-35’s ability to work in an integrated team by sharing radar and targeting information across the onboard mission systems.
To make up for the inadequate ranges, the program needs a specialized simulation facility that replicates the F-35 and the modern battlefield. The original plan dating back to 2001 tasked Lockheed Martin with creating a facility called the VSIM, or
Verification Simulator, to test the combat capabilities and scenarios that could not be replicated in the open-air ranges. But in the 14 years between 2001 and 2015, Lockheed Martin failed to produce a usable simulator.
In August 2015, the F-35 Program Office
decided to hand the project over to the Navy, which promised it could deliver a high-fidelity simulator renamed the Joint Simulation Environment in time for the beginning of operational testing.
Not surprisingly, the replacement simulator is not ready and is more than
two years behind schedule. The testing office reports that the simulator should be ready for the Initial Operational Test & Evaluation trials by the summer of 2020. The physical facility at Naval Air Station Patuxent River, in Maryland, is complete, but the simulation software programming is not.
In addition, designers have yet to complete the verification, validation, and accreditation process to ensure the simulation software accurately reproduces the performance of real aircraft in flight. Without verification, the simulator would just be a representation of what the contractors say the F-35 can do. In order to verify the simulation software design, programmers need to incorporate data gathered during flights over the test ranges, on how the onboard mission systems responded when the aircraft encountered threats or radar pings. This is a critical step to ensure the Joint Simulation Environment accurately simulates real-world conditions. The testing office reports this process is ongoing.
Once the simulation facility is completed, the operational testing plan calls for
64 missions to be “flown” in it. The facility is also important for the development and testing of the Block 4 upgrades in the years to come, so any further delays now could have a ripple effect with the testing of F-35 modernization efforts for years to come.
The 2015 decision to take the project away from Lockheed Martin at least made sense, unlike the original decision to have the company build the simulation facility in the first place. Giving Lockheed Martin this task was like paying a high school student to write their own exam, in that the simulator will be used to prove whether or not the aircraft they were trying to sell to the government could perform as advertised.
But the Navy team building the new simulator still relies heavily on Lockheed Martin to complete the project, which means the company was already paid to build the original, undelivered simulator and is now being paid again to help build the replacement. Even this arrangement has not been without hitches along the way. A prime example is the controversy over the “
F-35-in-a-box.” The Navy needed this simulation model from Lockheed Martin for basic F-35 operations. The company used a contracting dispute to delay delivery of the software, which contributed to the overall delay of operational testing.
The testing office deserves great credit for holding firm and insisting on the completion of the simulation facility so the full operational testing program can be completed. After nearly 20 years of development and testing,
political pressure continues to mount to move the F-35 program into full-rate production.
The delays associated with the Joint Simulation Environment could easily have been used as justification to cut the operational testing process short. The slow-rolling of the original simulator project and then holding up deliveries necessary for the completion of the replacement simulator at least gives the appearance that Lockheed Martin officials were attempting to engineer just such a scenario. Had Initial Operational Test & Evaluation been cut short, it is unlikely that we would ever know if the program was truly combat effective for the entire range of missions it is expected to perform.
F-35 Rarely Available To Fly
No matter what capabilities the F-35 could theoretically bring to aerial combat, they can only be put to use when the aircraft actually flies. Gaining a complete picture of the reliability of the F-35 fleet is a complicated task due to the various statistics used to measure the health of the fleet. But by any measure, the F-35 continues to underperform.
Traditionally, the Air Force’s go-to metric for judging the health of a fleet is the availability rate. This is calculated by dividing the number of hours aircraft in the fleet are able to perform at least one mission—known as the mission capable rate—by the total number of aircraft hours
of the fleet. By this metric, the health of the F-35 fleet remains below the program’s modestly low
65% availability rate target. The report does not mention the specific availability rate, saying only, “The average fleet-wide monthly availability rate for only the U.S. aircraft, for the 12 months ending September 2019, is below the target value
of 65 percent.” Without providing specific numbers, the testing office did report that the aircraft in the active operational fleet, about a third of the total, also fell short of the 65% goal for the 12 months before September 2019, but did achieve the goal for the
last three months of fiscal year 2019.
Pentagon leaders outside the Air Force apparently prefer to cite the mission capable (MC) rate as a measure of the fleet’s reliability. It is calculated by unit and represents the percentage of aircraft capable of performing at least one of the assigned missions. An aircraft that can simply take off is listed as mission capable. They prefer this to the more rigorous full mission capable (FMC) rate, which is the percentage of aircraft ready to perform all the assigned missions. For example, an F-35 might be able to conduct an air-to-ground strike, but a fault in the radar system might prevent it from tracking threats in the air. For a multi-role aircraft like the F-35, the full mission capable rate is a much more relevant figure. According to the
testing office:
No portion of the fleet, including the combat-coded fleet, was able to achieve and sustain the 80 percent MC rate goal set by former Secretary of Defense Mattis. However, individual units were able to achieve the 80 percent target for short periods during deployed operations. Similar to the trend in availability, the MC and FMC rates of the whole U.S. fleet improved slightly in 2019. FMC rates lagged the overall MC rates by a large margin, indicating low readiness for the mission sets requiring fully capable aircraft. All three variants achieved roughly similar MC rates, but significantly different FMC rates. The F-35A displayed the best FMC performance, while the F-35C fleet suffered from a particularly poor FMC rate; the F-35B’s FMC rate was roughly midway between the other two variants.
Unlike in last year’s report, the testing office did at least include a discussion of the F-35’s full mission capable rate, although it did not release the figure itself. The full mission capable rate is the single best metric for aircraft performance. At a March 9, 2020,
speech to the Mitchell Institute for Aerospace Studies, Air Force Brigadier General David Abba, the director of the F-35 Integration Office, said, “Frankly, in the fifth-generation fight, in the type of near-peer fight that we are anticipating and the reason we bought this airplane for, fully mission capable is where we’ve got to be.”
The testing office cites the lack of spare parts as the single biggest reason for the low readiness rates, with maintenance breakdowns and aircraft in the depots waiting for modifications
as additional drivers. Sources told POGO the full mission capable rates remain close to the most recent publicly available figures. Those figures date back to an April 2019 Government Accountability Office report on F-35 sustainment, which
included a chart that showed the entire U.S. fleet of F-35s, across all the services, averaging a 26.8% full mission capable rate.
The available data portrays an aircraft program that has demonstrated little improvement in its reliability. If the aircraft were pressed into service in a large-scale conflict today, the larger part of the F-35 fleet would watch from the sidelines.
Testing Office Under Threat Again
The only reason we know about any of these issues is that Congress created an independent operational testing office
in 1983. Before that, Congress and taxpayers had no choice but to rely on information generated and released by the service bureaucracies about the weapons they were purchasing with taxpayer dollars. The services are run by people with personal, and often financial, stakes in making sure the weapons they buy make it to full-rate production regardless of whether they performed as advertised. Members of Congress established the testing office, independent from the service bureaucracies and over the vehement opposition of defense contractors, to ensure the information they received had not been filtered or watered down to reflect parochial concerns.
Lacking the ability to control the operational testing office, defense contractors and their apologists have repeatedly tried to kill it. Defense Secretary Mark Esper, who’s currently on a sabbatical from his position
as a lobbyist for Raytheon to serve in government, now appears to be following in this tradition. Under the guise of freeing up funds to pay for higher-priority projects, Esper
released his plan to tackle waste in the “fourth estate,” or Pentagon support agencies outside the service branches. Buried in a chart in the
FY 2021 Defense Wide Review, we find that the operational testing office is once again on the chopping block.
There are plenty of wasteful programs that could be cut from the Pentagon bureaucracy. The operational testing office is not one of them.
There are plenty of wasteful programs that could be cut from the Pentagon bureaucracy, many of which would not be missed. The operational testing office is not one of them. Without it, the services would be able to pass off any self-serving information they need to in order to keep their pet projects on track, with little effective oversight.
Congress needs to step in to protect its own and the people’s interests to ensure this proposal fails. Additionally, Congress must take action to dash the defense industry’s greatest hope, the elimination of the testing office’s
annual report. The requirement for the report was due to sunset in 2021, but
an amendment to the 2020 defense authorization bill authored by Representative Katie Porter (D-CA) extended the requirement through 2025. Congress needs to make the annual report a permanent reporting requirement, and not one that is subject to review.
Conclusion
As the F-35 continues to stagger toward the operational testing finish line, we should gain a clearer picture of the program’s true capabilities and the flaws that must be corrected. If and when the Joint Simulation Environment is finished and the remaining testing events are completed, the testing office will evaluate all the testing data and include its assessment in an official Initial Operational Test & Evaluation report.
That will be the final legal hurdle before the much-anticipated full-rate production decision, currently
expected as early as October 2020. The original schedule had the program reaching this point six years ago,
in 2014.
The Pentagon has essentially treated the Initial Operational Test & Evaluation process as a mere formality, despite
federal law mandating operational testing as a precursor to full-rate production. More than 440 F-35s
have been delivered to the three services already, and additional
aircraft have been delivered to U.S. allies. Pentagon officials announced an agreement with the prime contractors for three years’ worth of F-35 production
in late 2019, placing another 478 aircraft in the pipeline for the U.S. and the international partners. Full-rate production for the F-35 is 160 aircraft per year, which is the number of aircraft to be produced in lot 13. By making such an agreement, the Pentagon made a de facto full-rate production decision even though the program has yet to meet the legal criteria to justify such a move.
As yet another annual operational testing report clearly shows, all we will get for our money in these deals are deeply flawed aircraft that will require extensive and expensive reworking and upgrades for years to come. In effect, Lockheed Martin will be paid at least twice to make these aircraft the right way. Any purported savings will quickly be subsumed as the bill comes due for rushing an incompletely designed aircraft into production.