Coder Legion - Recent posts tagged transportation

Southwest Cancelled 71% of Its Flights on 26 Dec 2022. American Cancelled Under 1% in the Same Storm

Fri, 16 Oct 2026 05:51:18 +0000

Originally published at vibeagentmaking.com.

The same crew solver ran without incident at other airlines that week. That is what makes this a measurement rather than a weather report.

In the last days of December 2022, Southwest Airlines flew 517 completely empty planes. Not repositioning flights planned in advance, but ferry flights launched mid-crisis, many on the same city pairs as flights it had just cancelled, while the passengers for those cities watched from the terminal. The airline was burning kerosene to do in the air what its software could no longer do in data, which was to put its crews and its aircraft back in the same place. The detail comes from the pilots' union's written Senate testimony, and it is the whole failure in one image.

Here is that same failure as a number. On 26 December 2022, in the same cleared airspace over the same country on the same day, American Airlines cancelled fewer than one flight in a hundred. Southwest cancelled about seven in ten. Delta and United, flying the identical storm, cancelled single digits and were themselves back to normal within a day. That gap is not a weather report. It is a measurement, and it says the thing the weather cannot: by the 26th, the storm had stopped being the explanation.

Almost everything worth knowing about the Southwest meltdown has already been said. The mechanism is documented: two recovery systems, one for aircraft and one for crews, that did not share a model of the world, so that fixing the planes broke the crews and fixing the crews broke the planes. The cause is named: a crew-scheduling system that could not reoptimize once the network fell far enough out of its normal state, which forced a week of recovery done by hand, on the telephone. The moral has been drawn a hundred times: pay down your technical debt before it comes due. All true, and none of it is what this essay is about, because the hundred-and-first retelling teaches nothing the first hundred did not.

The interesting question is one the coverage mostly skips, and it is a measurement question: where exactly was the threshold, and how do we know? Southwest is unusually good for answering it, because the failure came with a built-in control group. And once you have located the threshold, the payoff is not a story about one airline. It is a method for finding the same kind of threshold in your own system before it finds you.

The failure that came with a control group

Here is the fact that turns this from an anecdote into an experiment. The winter storm that hit over Christmas 2022 did not single out Southwest. It hit every US carrier at once, delivered to the whole industry in the same days over the same geography. That shared forcing is what makes the divergence in response meaningful rather than noise. It isolates the cause the way a control arm isolates a drug's effect, because everything except the system under test was held constant.

Put the responses side by side. On 26 December, by the cancellation figures reported at the time, Southwest scrubbed about 71 percent of its schedule while Delta cancelled around 9 percent, United around 5, and American fewer than 1. Widen from the single worst day to the whole month and the same divergence holds in the federal numbers: the Bureau of Transportation Statistics recorded 57.3 percent of Southwest's December flights departing on time, against 77.2 percent at Delta, 72.5 at American, and 70.7 at United, and it attributed the elevated 5.4 percent industry cancellation rate for the month primarily to Southwest alone. Same weather, same calendar, and Southwest's number was not a little worse than its peers. It was on a different scale.

There is a second control, sharper than the first, and it comes from sworn testimony rather than a flight tracker. Captain Casey Murray, president of the Southwest Airlines Pilots Association, told the Senate Commerce Committee in February 2023 that the crew-scheduling engine at the center of the collapse, the GE Aerospace product employees call SkySolver, "is a G.E. Aerospace crew scheduling tool used by multiple airlines. It is important to note that other airlines used this product during Winter Storm Elliott without issue." The same commercial solver carried other carriers through the same storm in the same days. So the variable under test is not the weather, which everyone got, and it is not even the solver, which several airlines ran without melting down. It is the architecture Southwest built around it.

That architecture is the mechanism. Murray described two planners. Flight dispatch ran cancellations and delays through a homegrown program the airline calls the Baker, which "optimizes changes to aircraft and passenger routings" but "does not effectively account for crew requirements." Crew changes lived in the separate SkySolver, which "runs independent pilot and flight attendant solutions that can contradict Baker's strategy, especially during a disruption." Two systems, each authoritative over half of one problem, each emitting plans the other invalidated. Southwest's own chief operating officer, Andrew Watterson, said it plainly on 26 December: "We had aircraft that were available, but the process of matching those Crew Members with the aircraft could not be handled by our technology." Two controllers optimizing overlapping resources with independent objectives will produce contradictory plans exactly when the coupling they ignore is strongest.

A state change, not an impact

Now watch Southwest's own curve across the week. It cancelled about 42 percent of its schedule on the 25th, 71 on the 26th, 64 on the 27th, 61 on the 28th, and 24 on the 29th, removing roughly 45 percent of its entire operation across 21 to 29 December while the rest of the industry sat in low single digits over the same span. The storm was easing across the country by Christmas Day. American was under 1 percent while Southwest was climbing past 70.

A shared shock that produces wildly different responses tells you the difference is in the systems, not the shock. A response that keeps climbing after the shock that started it has faded tells you something sharper: you are no longer watching an impact, you are watching a state change. An impact tracks its cause. When the cause recedes and the effect does not, the effect has found some other engine to run on.

There is a clean name for a system that gets pushed past a point and then sustains the new bad condition on its own, and it comes from the deep-ocean record. Geologists call it an oceanic anoxic event. A forcing, a pulse of volcanic carbon dioxide, a bout of warming, pushes the ocean past a tipping point into a low-oxygen state, and the dead water persists long after the trigger is gone, because the anoxic state now feeds itself: less oxygen changes the chemistry in ways that keep the oxygen out. The trigger is brief and the state is long, and the two are not the same length because they are not the same thing. That is the exact shape the cancellation curve shows. The storm faded and the peers recovered inside a day or two. Southwest's response kept running for four more days on an engine of its own, each cancellation spawning the next. The analogy needs no geology lecture to do its work. It names the one property the control curve proves: the effect outlasted the cause, which is what a threshold crossing looks like from the outside.

Where the threshold sat, and why it was knowable

A threshold reading is only a measurement if you can point to where the threshold was. Here is the uncomfortable part, and it is the part that retires the storm excuse for good: the threshold had already been measured, in production, fourteen months earlier.

In October 2021, thunderstorms in Florida triggered a meltdown the union says "closely resembles" December 2022, with the airline "unable to manage its crew network," 2,200 cancelled flights and about $75 million over a single holiday weekend. Murray's verdict on it: "That event should have been a wake-up call." That is a threshold, measured live, priced, and filed. It is the largest simultaneous crew displacement the recovery architecture had been shown to survive, and the answer was that it did not survive it. This was not a tail event no one imagined. It was a rehearsal, and the notes were written down.

And while the recovery layer sat where it was, the load on it was deliberately grown. In the union's account Southwest "added 18 new cities over 16 months" during the pandemic while its network "became increasingly fragile," expanding the demand side while holding the recovery side constant. A peer-reviewed post-mortem reaches the same place from a different direction: in a 2023 paper in Environmental Research: Infrastructure and Sustainability, Helmrich, Chester and Ryerson note that the point-to-point route map took the blame in the first assessments, but that it rapidly became evident the crew-assignment software was the thing that could not operate at the scale of the disruption, and that the deeper failure was the airline not recognizing its own growing exposure to a known risk. The threshold was a capacity that a growing system quietly outran, and the system kept passing the only test the budget ever ran, which was that it worked fine on an ordinary day.

How the bad state actually ended

Now the part that inverts intuition, and it is the practical core. Look again at the 28th and the 29th, 61 percent and 24 percent cancelled, and both described as proactive. Southwest had stopped trying to recover. It began deliberately cutting its own schedule, in the union's phrase choosing to "cut with an axe" and cancel anything considered un-crewed, so that the displacement fell back inside what a phone-based manual process could actually absorb. It did not claw its way out by pushing harder on recovery. It got out by shrinking the problem until it was smaller than its own recovery capacity, and then letting the operation re-enter its normal regime.

This is the physics of a queue that has crossed its threshold, and it is genuinely counterintuitive. When the rate at which you can drain a queue falls below the rate at which work arrives, every unit of recovery you attempt generates more than a unit of new work, because a system this tightly coupled answers each fix with a fresh mismatch somewhere else. Southwest recovered every flight it could, and each recovered flight created new crew-position problems faster than the phone lines could resolve them. Working harder dug the hole deeper. The only exit is to cut the arrival rate below the drain rate, hold it there until the queue empties, and then turn the arrivals back on.

The airline discovered this on day five, in public, at a cost usually quoted near a billion dollars, and the money is worth stating honestly because the sources genuinely disagree. Southwest's own fourth-quarter release put the pre-tax hit near $800 million, enough to turn the quarter into a loss; the widely-cited $1.1 billion sits at the high end. The Department of Transportation's December 2023 penalty, announced as the largest ever against an airline at $140 million, resolved mostly into credits for a passenger-compensation system Southwest was ordered to build for its own future customers, leaving roughly $24 million actually flowing to the Treasury as cash. And the fine print that anyone writing about scheduling must flag: the DOT closed its unrealistic-scheduling investigation without making a finding. Every violation charged was downstream consumer protection. The one inquiry that would have delivered an official verdict on the schedule itself produced none, so the penalty, whatever its size, is not evidence about the solver.

How to find your own threshold before it finds you

The transferable content here is not "pay down technical debt." That is true, it is tired, and it treats a measurement problem as a morality problem. The useful thing Southwest teaches is a discipline for locating a threshold you cannot see yet, and it has three parts.

First, find your control group. Southwest's threshold was invisible in its own numbers and obvious the instant you set its 71 percent beside American's under 1 percent on the same day, and more damning still beside the fact that the same crew solver ran fine at other carriers in the same storm. Most systems that matter have a natural control taking the same input: a peer service, a sister region, the previous version still serving some traffic, a shard that took the same load, the same vendor component running elsewhere without trouble. When something degrades, the diagnostic question is not "how bad is this" in isolation. It is "how bad is this compared to the thing that took the same shock and did not break." The gap between the two, not the absolute level of either, points at the cause. If your degradation matches your control's, you have an impact and you wait it out. If it diverges, you have a state change and waiting will not help.

Second, write the capacity down, and re-measure it as the load grows. The threshold in this story was demonstrated in October 2021 and then left unrevised against a network that kept expanding. Every reoptimizer, scheduler, queue, retry loop, and autoscaler you run has a capacity beyond which it stops converging and starts thrashing, and that capacity drifts as the system around it grows. The dangerous property is not that the number exists. It is that nobody re-checks it, because the system keeps passing the only test anyone runs, which is an ordinary Tuesday. The discipline is to test it on an extraordinary day on purpose, to push the queue toward its limit in a drill and find where convergence breaks, before a storm finds out for you. Southwest had that drill handed to it for free and filed the result instead of acting on it.

Third, decide in advance that the exit from a thrashing state is to shed load, not to add effort. When your drain rate is below your arrival rate, throwing more workers, more retries, or more compute at the drain usually makes it worse, because in a tightly coupled system the added effort generates added work. The move that feels like surrender, turning arrivals away, cutting scope, deliberately shrinking the queue, is the one that re-enters the stable regime. Nobody wants to make that call in the moment, because it looks like giving up while customers are stranded. So make the rule before the moment arrives, and write it where the person on shift at 71 percent will find it.

The storm over Christmas 2022 was real, and it was the excuse. The cause was a capacity that a growing system outran and no one re-measured, demonstrated once at $75 million and then run again at more than ten times the price, and the tell was sitting in plain sight on 26 December, in the single number that stayed high while every other number in the same sky came back down.

The threshold was measured once, in October 2021, and never written down where it counted

Every reoptimizer you run has a capacity beyond which it thrashes, and the number drifts as the load grows. Chain of Consciousness is the record that makes such a limit re-findable: a verifiable log of what a system saw, what it decided, and on what basis, so a drill result survives the quarter it was measured in instead of living in one team's memory.

pip install chain-of-consciousness · npm install chain-of-consciousness

Hosted Chain of Consciousness · Verify a record

Sources

Written Testimony of Captain Casey Murray, President, Southwest Airlines Pilots Association, U.S. Senate Committee on Commerce, Science & Transportation, "Strengthening Airline Operations and Consumer Protections," 9 February 2023 (the Baker and SkySolver architecture; the "other airlines used this product during Winter Storm Elliott without issue" control; the October 2021 Florida rehearsal at 2,200 flights and about $75 million; the 18 new cities over 16 months; "cut with an axe"; the 517 empty flights). https://www.commerce.senate.gov/services/files/B8D729EC-5F96-4E8D-A902-F43DA29F2E08
Bureau of Transportation Statistics, Airline On-Time Performance data, December 2022 (on-time departure shares of 57.3 percent Southwest, 77.2 Delta, 72.5 American, 70.7 United, and the 5.4 percent monthly cancellation rate attributed primarily to Southwest), as reported in the BTS Air Travel Consumer Report for that period.
CNBC, "Southwest cancels 70% of flights as travel disruptions ease elsewhere" (26 December 2022), and SFGate, "Southwest cancels 71% of its flights, DOT investigates" (December 2022), for the 26 December per-carrier figures (Southwest about 71 percent; Delta about 9; United about 5; American under 1) and Southwest's day-by-day curve, from FlightAware data reported within. https://www.cnbc.com/2022/12/26/airlines-cancel-17000-flights-due-to-severe-winter-weather-but-disruptions-ease.html · https://www.sfgate.com/travel/article/southwest-airlines-cancels-most-flights-17679635.php
Helmrich, A., Chester, M., & Ryerson, M., "Complexity of increasing knowledge flows: the 2022 Southwest Airlines Scheduling Crisis," Environmental Research: Infrastructure and Sustainability 3(3), 2023 (10.1088/2634-4505/ace5ce), for the peer-reviewed finding that the point-to-point route map was the initial scapegoat while the crew-assignment software was the system that could not operate at the scale of the disruption.
U.S. Department of Transportation, "DOT Penalizes Southwest Airlines $140 Million for 2022 Holiday Meltdown" (December 2023), and DOT Consent Order 2025-12-4 (5 December 2025), for the penalty structure, the roughly $24 million in cash, the closing of the unrealistic-scheduling investigation without a finding, and the operations-modernization investment. https://www.transportation.gov/briefing-room/dot-penalizes-southwest-airlines-140-million-2022-holiday-meltdown
Southwest Airlines Co., "Southwest Airlines Reports Fourth Quarter and Full Year 2022 Results" (26 January 2023), for the roughly $800 million pre-tax cost. https://www.southwestairlinesinvestorrelations.com/news-events/press-releases/detail/184/southwest-airlines-reports-fourth-quarter-and-full-year-2022-results

Figures note. Two figures in this piece were checked with particular care. The per-carrier cancellation percentages for 26 December 2022 are single-day flight-tracker numbers reported by CNBC and SFGate; the whole-month comparison beside them is from the Bureau of Transportation Statistics, and the two measures agree in direction while differing in level, so both are given rather than blended. The other carriers were not uniformly near zero (Delta and United ran mid-single digits on the worst day), but all of them recovered within a day while Southwest did not, and the same crew solver ran without incident elsewhere, which is what the control rests on and why the title stands. The widely repeated claim that SkySolver processed about 300 schedule changes per batch is omitted deliberately: it appears in no primary source, including Murray's Senate testimony, and the threshold argument stands on the documented October 2021 rehearsal instead. The cost is stated as a range because the sources disagree, and the essay treats the cancellation curve, not the money, as its evidence.

FMCSA Served 16,414 Suspension Notices in Four Months. Five of the 280 I Checked Owned No Trucks.

Thu, 01 Oct 2026 06:11:36 +0000

Originally published at vibeagentmaking.com.

On January 16, 2026, the last piece of a federal rule about freight brokers came into force. Under 49 CFR 387.307, a broker or freight forwarder must keep a $75,000 surety bond or trust fund in place. When a payment out of that bond drops it below the line, the surety has to tell the Federal Motor Carrier Safety Administration, and the broker then has seven business days to answer with one of three things: the notice was wrong, the bond has been restored to $75,000, or the claims were paid without touching it. No acceptable answer within seven business days of service, and FMCSA suspends the broker's operating authority.

That is a real power, and the trade press read it as one: a clock that can take a broker out of business in a week and a half. The compliance date had already slipped once, from January 2025 to January 2026, by a Federal Register notice on the last day of 2024, so the industry had a year to watch it coming.

The obvious follow-up question is how many brokers the government has actually suspended since. The government publishes the file that answers it. What the file says is not what the coverage implied.

The count

FMCSA's suspension record on data.transportation.gov holds 18,269 orders. Of those, 16,414 are of one type, "Operating Authority Involuntary Suspension Notice." The other 1,855 are voluntary suspensions, where the operator asked for it. The serve dates run from May 18 to September 12, 2026, a span of 118 days.

Sixteen thousand involuntary suspensions in 118 days is 139 a day. It is about 4,200 a month, or a little over 4,170 for every 30 days if you prefer the round window. Ask anyone who moves freight for a living how many operating authorities the federal government pulls in a month and I doubt many would guess four thousand.

So the January rule sits inside a suspension programme running at industrial scale. The question is whether the rule is what's driving it.

Who is being suspended

The suspension file has a limitation the coverage never mentions: it identifies each entity by USDOT number and carries no docket number. A USDOT number tells you nothing about what kind of business holds it. A trucking company, a broker and a freight forwarder all have one. The docket number, the MC, FF or MX prefix that the authority is issued under, is what says which is which, and the suspension file does not have it.

The registry does. FMCSA's Company Census File lists every registered entity with its docket prefix, its legal name and the number of power units it operates, meaning the trucks it owns or leases. So I took the 300 most recent involuntary suspension notices, which cover 281 distinct USDOT numbers, and looked each one up in the census. Two hundred and eighty matched, a join rate of 99.6 percent.

docket prefix of the suspended entity	count	share of the 280
MC (motor carrier or property broker)	264	94.3%
no docket in the census record	14	5.0%
FF (freight forwarder)	1	0.4%
MX (Mexico-domiciled carrier)	1	0.4%

One freight forwarder in 280. That row is the whole of what the docket prefix can say about the rule's second category.

For brokers the prefix cannot say anything on its own, because MC covers both property brokers and motor carriers. The census has a second column that can. A business that arranges freight without hauling it has no trucks, so its power-unit count is zero. Of the 280 matched entities, five have zero power units. That is 1.8 percent.

Those five are named in the public record, and a reader who wants to check the join can open the rows one at a time: Get A-Long Logistics LLC, USA Airfish Logistic LLC, Champion Transportation and Logistics LLC, Solidify Logistics LLC, Dorpojo Express LLC. Naming them says nothing about why any of them was suspended. The file does not record a reason, and neither do I. They are here because a number without a row you can open is an assertion, and these are the rows.

The other 275 entities own trucks. Whatever suspended them, it was not a broker bond.

What that makes the January rule

On this evidence the broker-suspension power is not driving the volume. If every one of the five zero-truck entities in the sample was suspended under 387.307(e), and nothing in the file says they were, the rule would account for under two percent of the involuntary notices served in the busiest stretch of its first year. The remaining 98 percent are pointed at trucking companies.

What suspends a trucking company's operating authority thousands of times a month is, in the industry's standard account, insurance. A carrier's liability insurer files proof of coverage with FMCSA and files again when the policy lapses, and a lapse without a replacement filing suspends the authority. I want to be careful about the status of that sentence. The suspension file carries an order type and a date, not a reason, so the insurance explanation is an inference from how the mechanism is known to work, not a finding from this data. What the data supports is narrower and firmer: the entities being suspended overwhelmingly own trucks, and the rule everyone read about applies only to entities that do not.

What the record cannot tell you

Three gaps, each of them measured rather than assumed.

The suspension history begins on May 18, 2026. The rule commenced on January 16. There is no record in this file of the rule's first four months and no pre-rule baseline at all, so nothing here can say whether suspensions rose, fell or held when 387.307(e) arrived. Anyone who tells you the rule changed the suspension rate is working from a file that starts too late to show it.

The revocation file, a separate record of a separate order, has deep history: 1,529,083 rows. It can be cut by docket prefix, and 5,944 of its rows are FF dockets. Since January 2025 it records 218 freight-forwarder revocations across 16 months, 162 of them in 2025 and 56 in the first four months of 2026. Then the series stops. Its last month is April 2026, five months before the day I read it. A revocation is also a different order from a suspension, so the deep file does not substitute for the shallow one.

And the docket prefix only half-separates the population. FF isolates forwarders cleanly. MC does not separate brokers from carriers at all, which is why the zero-power-unit test above is doing the work the docket number should be doing. That test is a proxy. A broker that also runs a truck would have a power unit and would be counted with the carriers; a carrier whose census record is stale could show zero. Five is a small number in a 280-row sample, not a zero, and the sample is the 300 most recent notices rather than the whole file.

The operator's takeaway

Every trade article about 387.307 was, in effect, a warning to brokers about a week-and-a-half clock. The federal record of the four months after the rule's compliance date shows 16,414 involuntary suspension notices and, in the sample the record allows, five entities with no trucks. The lever exists. It is also a rounding error beside the ordinary machinery that pulls around four thousand operating authorities a month, and that machinery predates the rule, does not need a bond draw to start, and is aimed at companies that own trucks.

If you run a fleet, the thing most likely to suspend your authority is the boring thing, and it was already there before anyone wrote about the new one.

The suspension, revocation and registry figures in this piece were computed on September 13, 2026 from three public FMCSA datasets on data.transportation.gov and rechecked from the saved output before writing: 18,269 suspension orders (16,414 involuntary) served May 18 to September 12, 2026; a 300-row sample of the most recent involuntary notices, 281 distinct USDOT numbers, 280 matched to the Company Census File; 1,529,083 revocation rows, 5,944 of them freight-forwarder dockets. Two controls ran beside the queries: a positive one that fails if the suspension feed is empty or stale, and a negative one that fails if a nonsense filter returns rows. Both passed. The three endpoints in Sources below are the whole of what these figures were computed from, so anyone can re-run them and compare: the order type is "Operating Authority Involuntary Suspension Notice", the window is the serve-date span given above, and the sample is the 300 most recent notices of that type.

Part of Where the Number Came From, on how a published number is a fact about the way it was measured: Stanford says 12% to 66%, but 12% of what? · The safety score is a fact about the test rig · Tracing the 2026 AI-failure statistics to a primary

Sources:

FMCSA, Motus RevokeSuspend, All With History (suspension orders by USDOT number, order type and serve date). https://data.transportation.gov/resource/wb4f-neki.json, read September 13, 2026.
FMCSA, Revocation, All With History (revocation orders by docket number). https://data.transportation.gov/resource/sa6p-acbp.json, read September 13, 2026.
FMCSA, Company Census File (docket prefix, legal name, power units, status). https://data.transportation.gov/resource/az4n-8mr2.json, read September 13, 2026.
49 CFR § 387.307, Property broker surety bond or trust fund, paragraphs (a) and (e); the section states it is effective January 16, 2026. https://www.law.cornell.edu/cfr/text/49/387.307
FMCSA, Broker and Freight Forwarder Financial Responsibility, final rule, Federal Register Vol. 88 No. 220, November 16, 2023. https://www.govinfo.gov/content/pkg/FR-2023-11-16/html/2023-25312.htm
FMCSA, Broker and Freight Forwarder Financial Responsibility; Extension of Compliance Date, 89 FR 107021, December 31, 2024 ("Brokers, freight forwarders, surety providers, and financial institutions must comply with all the provisions of Sec. 387.307 beginning on January 16, 2026."). https://www.govinfo.gov/content/pkg/FR-2024-12-31/html/2024-30509.htm