CEROP
Stage 01 · Week 2

Framing the problem

What decision are we trying to support, for whom, and how would we know the model is any good?

What we asked

Companies that buy from or extend credit to partners abroad usually check two things separately: is the partner financially healthy, and do their goods arrive on time? Those checks live with different people. A credit or procurement analyst reads balance sheets. A logistics planner watches shipments.

Commercial supplier-risk scores already exist, which tells you there's demand for this. They are also proprietary, paid for, refreshed every so often rather than continuously, and they rarely say why a score is what it is. We wanted to see how far public data and course-level methods could get, with the reasoning visible.

What we decided

  • Two layers, two scores. A financial layer predicts bankruptcy. An operations layer predicts late delivery. Each answers its own team's question, so they're trained, judged and shown separately.
  • Supervised, batch classification. Both datasets carry a historical yes/no label, so both are binary classifiers trained offline on fixed data.
  • Missing a risky partner costs more than an extra review. The original bankruptcy study reached the same conclusion, so recall carries more weight than precision on the financial side.
  • Context from outside the company. The operations layer adds the New York Fed's Global Supply Chain Pressure Index by month. The financial data has no dates, so it can't take the same index.

Why accuracy was the wrong yardstick

Only 3.2% of the firms in the bankruptcy data failed. A model that says "safe" to every firm would be 96.8% accurate on our test set and would never catch a single bankruptcy. So the financial layer is judged on recall (how many failures it catches), precision (how many of its alarms are real), F1 (the balance of the two) and PR-AUC.

The shipment data is the opposite: about 55% of orders arrived late, so the classes are close to even. Accuracy means more there, but F1 and recall still tell you more about the cost of a miss.

Looking back: the Week 2 proposal also set a recall target for the financial layer. The final model doesn't reach it at the default threshold. That gap is discussed honestly on the results page rather than quietly dropped.

What changed because of it

Every later choice traces back to this page: the separate pipelines, the metrics on every chart, the decision to treat the threshold as a business setting, and the console's two tabs for two audiences.