Coder Legion - Recent posts tagged pulp

S&OP Engineering IV: Scaling to Enterprise (Multi-SKU & Bottlenecks)

Mon, 06 Apr 2026 05:55:15 +0000

Your MVP works. One product, one model, one perfect plan. Congratulations: you just solved the easiest problem in Supply Chain.

Now add 3 products sharing the same factory. Product A needs 14,000 units in July. Product C needs 13,000 that same month. Your factory produces 15,000. Who gets shorted?

If your answer is "we'll figure it out in Thursday's meeting," your company has an engineering problem, not a management one.

Executive Summary: In chapters 1, 2 and 3 we built a working MVP for a single product. In this Chapter 4, we break it. We inject 3 years of history with 3 SKUs of radically different demand profiles (Core, Intermittent, and Seasonal), parallelize Prophet model training with MLOps, and build a unified Linear Programming model where products compete mathematically for limited factory capacity. Welcome to Theory of Constraints executed in code.

The Fatal Error: Optimizing in Silos

The biggest sin in Supply Chain Planning isn't using dirty data (we fixed that in Chapter 1). It's optimizing each product as if the factory were infinite.

Imagine three product managers, each with their optimized Excel:

SKU-001 Manager (Core): "I need to produce 6,000/month. Already calculated."
SKU-002 Manager (Spare Part): "I only need 15/month. No big deal."
SKU-003 Manager (Seasonal): "In July I need 14,000. Non-negotiable."

Sum: 6,000 + 15 + 14,000 = 20,015 units. Your factory produces 15,000. Each manager is right individually. Collectively, it's a collapse.

This is what Eli Goldratt described in The Goal as Theory of Constraints: the system is not optimized by summing local optima. It's optimized by identifying the bottleneck and letting mathematics allocate the resources.

Stress-Test: The 3 Profiles That Break Your MVP

To validate the Enterprise architecture, we generated a 3-year dataset (2023-2025) with three SKUs designed to break any naïve model:

SKU	Profile	Average Demand	Trap
SKU-001	Core Product	~200/day	High variance + 3x Black Friday
SKU-002	Intermittent Spare	~0.6/day	70% zero-demand days
SKU-003	Seasonal Summer	~250/day	500+ peaks in July, zero in winter

We injected real ERP noise: outliers (x50), corrupted dates, nulls, negatives. This isn't a toy tutorial. It's what comes out of a production SAP on any given Tuesday.

MLOps: Parallelizing the Forecasting

With a single product, training Prophet takes 2 seconds. With 50 SKUs in a for loop, you wait 100 seconds. With 500, you go home.

The solution is the same one production pipelines use: parallelization. Each SKU is statistically independent, so each model can train on its own thread:

from concurrent.futures import ThreadPoolExecutor

def process_sku(sku, df_sku, months, country_code):
    """Worker: trains Prophet for one SKU."""
    predictor = ProphetPredictor(df_sku, sku_name=sku)
    predictor.preprocess_daily_aggregation()
    predictor.train_model(country_code=country_code)
    return predictor.generate_forecast(months=months)

with ThreadPoolExecutor(max_workers=3) as executor:
    futures = {
        executor.submit(process_sku, sku, df, 12, "ES"): sku
        for sku, df in sku_dataframes.items()
    }

3 Prophet models, 3 seconds. Not 6. Three. If you add 50 SKUs like any mid-sized company has, the difference between sequential and parallel is the difference between an operational pipeline and a script nobody wants to run.

Negative Clipping: The Business Rule Prophet Ignores

Prophet can generate negative predictions. Mathematically it makes sense (the regression doesn't know demand can't be -12 units). Operationally it's a disaster.

Our forecasting_engine_v2.py applies a Negative Clipping post-prediction:

# Business Rule: Demand cannot be negative
for col in ["yhat", "yhat_lower", "yhat_upper"]:
    self.forecast[col] = self.forecast[col].clip(lower=0)

One line. One fewer production bug. This is what separates MLOps from "Machine Learning in a Jupyter Notebook."

Fault Tolerance: One SKU Doesn't Kill the Pipeline

What happens if SKU-002 has only 10 records and Prophet fails? In an amateur script, the entire pipeline dies. In an Enterprise system:

sku_name, forecast_df, error = future.result()
if error:
    print(f"[!!] {sku_name}: FAILED - {error}")
    result["skus_failed"] += 1
else:
    forecasts.append(forecast_df)
    result["skus_success"] += 1

The 2 healthy SKUs finish their work. The failed one gets logged. This is the difference between a script and a system.

The Mathematics of Bottlenecks: Unified PuLP Model

Here's the intellectual core of the entire chapter. In Chapter 3 we optimized one product. Now we create a single LP model where all SKUs coexist:

# ── MULTI-DIMENSIONAL decision variables ──
production[sku][t]  # How much to produce of SKU s in period t
inventory[sku][t]   # Inventory of SKU s at end of period t

# ── Objective Function: TOTAL system cost ──
problem += lpSum(
    prod_cost[sku] * production[sku][t]
    + hold_cost[sku] * inventory[sku][t]
    for sku in skus for t in range(T)
), "Total_System_Cost"

# ── THE KEY CONSTRAINT: Shared Capacity ──
for t in range(T):
    problem += (
        lpSum(production[sku][t] for sku in skus) <= SHARED_MAX_CAPACITY,
        f"SharedCapacity_t{t}"
    )

Read that last constraint. It's a single line of code, but it contains Goldratt's entire Theory of Constraints encoded:

lpSum(production[sku][t] for sku in skus) <= 15,000

You're telling the machine: "The factory produces at most 15,000 units per month. I don't care if it's 15,000 of SKU-001 or 7,500 of each. You decide the allocation that minimizes total system cost."

And the solver decides. No meetings. No politics. No "we always prioritize the big client." Mathematics.

The Result: Intelligent Capacity Allocation

Our solver with real Supabase data:

  SKU-001: demand=70,879 | prod=70,579 | avg_inv=5,754 | cost=855,394 EUR
  SKU-002: demand=143   | prod=128   | avg_inv=6     | cost=6,775 EUR
  SKU-003: demand=84,298 | prod=84,098 | avg_inv=115   | cost=1,265,970 EUR

  Total production cost:       1,973,660 EUR
  Total holding cost:            154,479 EUR
  ** TOTAL SYSTEM COST:        2,128,139 EUR **

Observe the decisions the algorithm made:

SKU-001 (prod_cost=€10, hold_cost=€2): Cheap to store. The solver produces ahead during cheap months and stockpiles inventory (avg 5,754 units). Intelligent pre-build.
SKU-002 (prod_cost=€50, hold_cost=€5): Expensive part. Strict just-in-time production (avg_inv=6 units). Doesn't stockpile a single extra piece.
SKU-003 (seasonal): Minimum inventory at safety stock. Produces exactly what it needs each month, following the seasonal wave.

No human with Excel can calculate this with 3 products and 13 months. And they definitely can't recalculate it every week when the forecast changes.

The Ah-Ha Moment: The Bottleneck Visualized

Look at the summer months. Combined demand from the 3 SKUs exceeds factory capacity (red line). What does the algorithm do? Instead of breaking stock, it decides to advance production of the cheapest-to-store SKU (SKU-001) to spring, freeing factory space for the critical seasonal product (SKU-003) in summer. This is Theory of Constraints executed by a machine.

If you look at the chart, it seems like SKU-002 (spare parts) doesn't exist. But it's there. It represents 0.1% of the factory's volume, but in manual management, it usually consumes 20% of the planner's mental time due to its high volatility. By delegating this to a Linear Programming model, the system manages large volumes and the long tail simultaneously without stress.

Open Kitchen: Break the System

I distrust theories that can't be put into practice. I've prepared a new Enterprise Google Colab where you can run the most revealing experiment of the entire series:

Lower the SHARED_MAX_CAPACITY variable to 12,000. Watch the solver start juggling: it sacrifices SKU-001 inventory to prioritize SKU-003 in July. Lower it to 10,000 and you'll see the model declare INFEASIBLE — the factory literally cannot meet demand even with the best possible planning. That's your inflection point for investing in capacity expansion.

Open the Multi-SKU Enterprise Google Colab

Change the maximum factory capacity, modify holding costs, add a fourth SKU. Do engineering, not faith.

The Full Chain: From MVP to Enterprise

With this fourth chapter, we've scaled a single-product prototype to a Multi-SKU Enterprise architecture:

Next Step: The Autonomous Brain

Our Supabase database now has the perfect master plan. Three products, thirteen months, every unit assigned to the optimal month considering costs, seasonality, and physical capacity.

But databases don't send emails to suppliers. They don't negotiate with plant directors. They don't generate purchase orders at 6am on Monday.

In Chapter 5 (Grand Finale), we'll connect this plan to AI Agents (CrewAI) to execute operations autonomously. The algorithm decides how much to buy. The agent executes the purchase. The human... supervises.

The difference between a company that plans and one that optimizes is a mathematical model between its data and its decisions. And between its decisions and its actions, an autonomous agent.

S&OP Engineering III: The End of Excel (Linear Programming for Supply Planning)

Wed, 01 Apr 2026 06:01:26 +0000

"We always want 4 weeks of coverage." This phrase, repeated like a mantra in every S&OP meeting on the planet, is financially toxic.

Why? Because it's a fixed rule applied to a dynamic system. If your demand in January is 200 units and in July is 20, you're forcing yourself to maintain 800 and 80 units respectively "just in case." January falls short. July immobilizes capital for no reason.

The alternative isn't more sophisticated intuition. It's mathematics.

Executive Summary: In this chapter, we connect the probabilistic forecast from Chapter 2 with a Linear Programming engine (PuLP) that calculates the exact production plan minimizing total cost (production + storage) while respecting Safety Stock constraints. We move from passive prediction to active prescription.

From Forecast to Decision: Architecture

In Chapter 1 we cleaned the signal. In Chapter 2 we predicted demand. Now we take the step Excel can't: optimize.

Our pipeline on GitHub connects to Supabase, reads the demand_forecasts table (the future we predicted), and generates a new supply_plans table. The system has evolved from descriptive (what happened?) to predictive (what will happen?) and now to prescriptive (what should I do?).

This is Operations Research. The same discipline that optimizes airline routes, military logistics, and global supply chains. And we run it with 50 lines of Python.

The Mathematics of Business

We're not going to hide the equations. They're the heart of the decision. Here's the core fragment from our SupplyOptimizer class:

# Objective Function: Minimize total cost
problem += pulp.lpSum(
    production_cost * production[t] + holding_cost * inventory[t]
    for t in range(T)
), "Total_Cost"

# Constraint: Inventory Balance (Conservation of Mass)
for t in range(T):
    prev_inv = initial_inventory if t == 0 else inventory[t - 1]
    problem += (
        inventory[t] == prev_inv + production[t] - demand[t],
        f"Balance_t{t}"
    )

# Constraint: Safety Stock (Risk Policy)
for t in range(T):
    problem += (
        inventory[t] >= safety_stock,
        f"SafetyStock_t{t}"
    )

Three engineering decisions worth explaining:

The Objective Function doesn't seek "lots of stock" or "high production." It seeks the minimum financial cost: the equilibrium between manufacturing (expensive) and storing (also expensive). The solver automatically finds the exact point where the combined cost is minimal.

The Mass Balance is a physical constraint: you can't sell what you don't have. Today's inventory equals yesterday's, plus what you produce today, minus what you sell. No magic. The equations forbid cheating.

The Safety Stock is the risk policy: never let inventory drop below a safety minimum. In our case, 1.5 months of average demand. The system calculates this, not a spreadsheet with a number pulled from thin air.

The Master Plan: From Algorithm to Business Decision

This is what the Operations Director needs to see. The algorithm doesn't produce uniformly: if it detects a massive demand spike in one period, it decides to "pre-build" inventory in preceding months to flatten the production load. If Holding Cost is high, it keeps the warehouse empty and manufactures Just-in-Time. Excel doesn't do this on its own; mathematics does.

The result from our solver with test data:

Production cost: €1,680
Storage cost: €540
Total optimized cost: €2,220

This number is not an estimate. It's the provable global minimum given the constraints. If someone proposes a cheaper plan with the same parameters, they're violating a constraint.

Open Kitchen: Play with the Solver

I distrust theories that can't be put into practice. That's why I've prepared a Google Colab where you can run the optimizer on a snapshot of our real data.

The most revealing experiment: change holding_cost to an extremely high value (e.g., €50/unit). Watch how the algorithm automatically decides to manufacture Just-in-Time and keep the warehouse practically empty. Then lower the production cost and watch it prefer to produce in bulk and store. Mathematics adapts. Excel's fixed rules don't.

Open the Interactive Google Colab

Modify the costs, capacity constraints, Safety Stock. Do engineering, not faith.

The Complete Chain: From Data to Decisions

With this third chapter, we've built an end-to-end S&OP system that goes from a dirty ERP CSV to an optimal production plan:

Next Step: Scaling to Enterprise

We now have the perfect plan in our database. But only for one product. What happens when you add 3 SKUs sharing the same factory?

In Chapter 4 we break the MVP: we inject multi-product data with radically different profiles, parallelize the forecasting with MLOps, and build a unified Linear Programming model where products compete mathematically for shared production capacity.

The difference between an Operations Director who plans and one who optimizes is an objective function between their intuition and reality.