This weekend, I'd like to welcome Eric Hiller back to Spend Matters. Eric, who is founder and Chief Product Officer at aPriori, will be contributing a series of guest weekend posts on the subject of driving Spend Management upstream. And this is the seventh installment. Please join in welcoming Eric to Spend Matters!
The Oracle of Delphi was a prophetess in ancient Greece. One would go to the Oracle in search of an answer. She would ask a few questions about the dilemma, and, then, she would give an answer… often cryptic, but an answer. There was no questioning as to why this was the answer or the reasoning behind the answer; it was the answer -- take it or leave it and interpret it yourself (that's just how oracles are). Empirical (Statistical) Modeling is a costing methodology that is a lot like the ancient Oracle. In this method, the modeler takes as much past data as he can get, uses statistical methods to sift through many possible cost drivers, determines which drivers are active in determining cost, determines the magnitude of each variable’s effect, and constructs a model with this information. Once you have such a model, you can determine whether the costs you are paying for current parts are "outliers" from the model based on its active variables and investigate why. Perhaps, the outliers are stale costs or bad data, or perhaps these are cost reduction opportunities. Theoretically, you can also feed in the cost drivers for a new part and predict what its cost should be.
Statistical models can be fast or slow to construct depending on the cleanliness of the data set and whether you understand which variables are active cost drivers and what mathematical form the equations should take. Empirical models can be accurate, and they are obviously consistent, being a mathematical model. Empirical Models work well in narrow domains where the environment is static. In these domains they can be simple to construct and calculate cost well to a first order approximation, but empirical fits can have some difficulty in providing directional precision.
Empirical models also have trouble with resolution and are not very useful for helping users learn. They are top down models, i.e. they are a "black box." It is not easily possible to directly understand the physical action/reaction relationship between cost drivers and results (time or cost). It is difficult to actual causality in the model. However, this black box nature is actually one of the benefits of the approach, i.e., you don’t need to completely understand what is really happening or the cause and effect relationship. With empirical models, you will get an answer regardless. There is no why, only what. There also is no guarantee with an empirical model that the population of parts on which you are basing the model is a got representative sample of the possibilities for cost savings. The population in general may have been poorly sourced, designed, manufactured, etc.
However, the biggest problem with empirical models is that they are not very robust (they do not move from target to target automatically). Like the ancient maps that said on the fringes of the seas "Here there be dragons", there often are with empirical models. If you go outside the range of the input variables that the model is based on, you have no guarantee that the model is valid any longer. For example, let's say you are an injection molding company and you make a statistical model of time and cost for small injection molded medical parts you make for one of your customers. The model seems accurate and stable. Good news! You just picked up a new customer who makes industrial equipment. However, their parts are much bigger and thicker. Will your old model apply? Maybe, but it is likely that the original empirical cost model will start to diverge from the True Economic Cost™ or "blow up" on the new customer’s parts. Worse, it is unclear when you have left the region where a statistical model is valid and safe. Therefore, you will have significant work ahead to "retrain" your model.
Empirical models are notoriously hard to maintain in a dynamic business environment. If you change machines, technology, customers, suppliers, material pricing, routings, etc., it is likely that the model is not extensible. It will require a lot of rework. Empirical models take a lot of care and feeding, even within the same company. In this way, empirical models share some of the challenges of component models, although it is very possible to make process based models empirically.
(So, how do I know this empirical / curve fitting cost modeling approach? I did my original research in1996 using empirical modeling, but moved away from the methodology to Mechanistic Models, for several of the reasons discussed here, and that will be discussed tomorrow.)