Freight invoice discrepancy detection for a European 3PL
Client: A European third-party logistics provider operating 40 distribution centres across seven countries. Named on request with their permission.
The problem
Invoice reconciliation across four carriers consumed two full-time analysts. Because full manual audit was impossible at their volume — roughly 14,000 freight invoices a month — the team sampled around 8% and accepted that the rest went through unchecked. Finance knew there were discrepancies in the unsampled 92%. Nobody could say how many or how much.
Two previous attempts had stalled. A rules engine built in 2023 caught only the discrepancy types someone had already thought of. A vendor proof-of-concept in early 2025 demonstrated well on a curated set of 50 invoices and was never deployed, because nobody could agree what accuracy level would justify letting it act.
What we built
A document-intelligence agent over their TMS and the four carrier portals. The agent extracts line items from invoices in six formats, reconciles them against the shipment record and the contracted rate table, classifies discrepancies into eight categories, and routes anything above a value threshold to a human with the evidence assembled.
Two of the four carrier portals had no API. For those we worked from scheduled file exports rather than browser automation — slower to set up, dramatically less brittle to operate, and we said so at scoping rather than discovering it in month three.
The extraction pipeline runs a frontier model through a managed endpoint in the client's own cloud account, with a smaller model handling format classification and routing. Schema validation is deterministic. Anything the model can't ground in the source document is returned as absent rather than inferred.
How we measured it
Before any build work, we constructed a golden set of 340 invoices drawn from twelve months of real volume, stratified across carriers, formats and discrepancy types, including 60 invoices the team had already flagged and 40 they had passed that turned out on review to contain errors.
Each invoice was labelled independently by two analysts, with a third adjudicating disagreements. Inter-annotator agreement on discrepancy classification was 0.87 after one rubric revision — the first pass came in at 0.71, which told us the category definitions were ambiguous rather than the analysts inattentive.
In week one we agreed the thresholds in writing: recall above 75% at precision above 90%, with a false-positive rate low enough that human review of flagged invoices stayed under four hours a week. Both sides signed that document before the build started.
Week six ran in shadow mode against live invoice flow for eleven working days, scored against the golden set plus 90 new invoices the period surfaced.
Results
83% recall at 96% precision on the golden set, and 81% recall on the previously unseen shadow-period invoices — the gap between those two numbers matters more than either alone, and we report both.
Human review time on flagged invoices settled at roughly 2.5 hours a week against the four-hour ceiling. Coverage moved from an 8% sample to 100% of invoice volume.
The client has not authorised us to publish recovered value. What we can say is that the classification breakdown surprised them: the largest discrepancy category by count was not the one either previous project had been designed to catch.
What we'd do differently
We under-scoped carrier portal access negotiation by about two weeks. That's the item we now size deliberately at the start of every logistics engagement, because it depends on somebody else's calendar.
Talk to a supply chain engineer.
45 minutes with the engineer who'd lead the build. Bring your invoice volume and your carrier list.