Spurious Correlations, 150% Cost Savings and How to Justify Your Next Procurement Project Pierre Mitchell - July 2, 2015 6:27 AM | Categories: Learning / Research, Procurement Strategy & Planning | Tags: L1, Process & Best Practice Last week, I presented on a webinar alongside Tim Cummins, the founder of International Association for Contract and Commercial Management, and Ulf Zetterberg, the CEO of Seal Software. I was sharing some analysis that I did on Apple Inc.’s purchase agreement templates and how contracts sometimes can create more relationship risk than they prevent. I also discussed relationship risk and relationship reward, using Ford as an example. On the risk side, I referenced a great case study that David Simchi Levi’s team at MIT performed for Ford. On the reward side, I mentioned the oft-quoted research by John Henke on ‘Working Relations Index” in the Automotive industry. I took my data from the following press release, which stated: "If each of the 6 automakers had scored only 10% higher on the 2014 Working Relations Index, each would have gained an additional $58 to $152 in profit per vehicle, or $98 million to $400 million in operating profit, which can be attributed to the increase in better relations supply risk and supply reward." It’s an eye catching number, but can you spot the problem? (Hint: it’s italicized.) This is the classic problem of correlation being used to infer causation. One cannot say that automakers would have gained this additional profit through better supplier relations. One can only say that there is a model that demonstrates a mathematical correlation between a supplier management index and profit. The world is highly multivariate, and who knows the real reason this correlation is not captured in the study. Perhaps in worse economic times, when car prices are discounted, the whole supply chain gets stressed, and big buyers pass on that pain to their suppliers and not vice versa. Regardless of the sophistication of any model – and I have no doubt that the aforementioned is quite robust – you simply can’t hang your hat on a correlation model. And trust me, I spent quite a few years really diving deep into multiple regressions on procurement benchmark data that correlated procurement practices and company structural factors to procurement KPIs. When you do this, you tend to weed out practices that really don't seem to have any effect at all on procurement performance, and you’re left with ones that inherently do make sense in terms of, say, reducing process costs via increased automation. You can show the statistical correlations, as I often still do, and the bar charts look great, but you have to take them with a grain of salt. The most obvious reason is the dreaded spurious correlation, where 2 factors look highly correlated but actually have nothing to do with each other. Check out this excellent website – hat tip to Linda Michels on this – for some terrific examples. Did you know there is a near perfect correlation (r=0.95) between the number of people who drown after falling out of a fishing boat and the marriage rate in Kentucky? OK, that’s silly, so here’s another automotive example. Did you know that the annual number of automotive suicides is nearly perfectly correlated (r=0.94) to the number of Japanese passenger cars sold in the US? So, can we infer that driving a Japanese car makes you want to commit suicide? No, of course not – although I can think of a few that might. The point is to merely be wary of such numbers. As my friend Roy Andersen said back when he was CPO at MetLife, “If I added up all the promised savings from vendors and Aberdeen Reports, I’d be saving over 100% and suppliers would be paying me money!” But for practitioners looking for anything they can to help justify a project, any data is better than no data. In this case, yes, find what you can and use the best sources of data that you can. Also, you should be individually benchmarking with customers who’ve gone before you. Doing so within your industry might offer little extra true predictive value but can be useful in selling it upwards so that you don’t fall behind your industry peers. If your company is large enough and you have done or can do a pilot to show early successes to predict future success, that’s really the best. Finally, have a good ROI model that really looks at the “R” and “I” very realistically. Make sure to use ranges, even if you don’t show them, so you can play around with assumptions. The more you engage senior leadership and show them that you’ve thought through the details, the more credibility you’ll have in selling your project and moving the needle on your transformation. Discuss this: Cancel reply Your email address will not be published. Required fields are marked *Comment Name * Email * Website Notify me of follow-up comments by email. Notify me of new posts by email.