Contract Management, reliance on data, and near disaster on the M1 (part 2)

Yesterday I told the story of how I ran out of petrol on the M1 Motorway because of a faulty fuel gauge. As well as a cautionary tale for any driver, it suggested some wider truths. We rely on measurement to tell us what is happening in real life. The truth was not the fuel gauge – the truth was that I had no petrol in my tank. But we have become accustomed to seeing the measurement as the truth.

So in a business context, if the metrics suggest that a supplier is performing well, or that our customers like us, we believe it and take that as a proxy for the true situation.

“The supplier is performing well”, we say. But that’s not true. “The measures we are using suggest that, all things being equal, and if the numbers reflect reality, then the supplier is performing well”.

The MoJ thought Serco and G4S were performing  well on the Electronic Monitoring contract, and Serco reported that targets were being met on the London prisoner transport contract. (See our recent posts on that topic). The customers of various call centres who turned out to have been mis-reporting their stats thought those suppliers were performing well.  I thought I had petrol in my tank ...

The message therefore is that we should always be cautious and careful with measurement and measures, and remember that these are just numbers, not ‘the truth’. So what can we do to get at the truth in a contract management situation?

The most fundamental message is a basic one – we should always engage our brains. That’s what I failed to do in the case of the petrol. I just didn’t think that the fact the gauge hadn’t moved might mean something important! I didn’t think to check with my wife whether she had filled up the car when I knew it was some time since Id done so. Keep your wits about you. If a reported measure looks too good to be true, it probably is (my car wasn’t suddenly running on fresh air).

And don’t rely on measures, certainly not just one measure. If the supplier tells you that your customers love the service they provide, check that. Go and talk directly to some customers. Dig into how performance metrics are derived, what process the supplier uses to collect the data.  (That is eventually what MoJ did of course, which exposed the over-billing issue. But it should have happened sooner).

To summarise - don’t take things at face value. Be suspicious of measures. Understand what lies behind the data. Don’t be afraid to question and dig into anything that doesn’t look right.

And above all, don’t run out of petrol on the Motorway.

Voices (4)

  1. b+t:

    If the KPIs look good, but the end users hate the supplier, then the KPIs are either unsuitable or untruthful or both.

  2. flog:

    So some of the metrics should, were it’s critical may need to complement each other. I once had a faulty petrol gauge, it used to sit on 3/4’s full for ages and then clunk it was at empty. So, I got into the habit of zeroing the trip every time I filled up. Now I have two metrics, the gauge and the mileage.

    Of course, from the contract management viewpoint, this highlights something else, I (the customer) modified my behaviour (compensating for the faulty gauge) rather than getting the supplier to sort the ‘delivery’ problem.

  3. Sam Unkim:

    Another Point – Cost
    Fortunately, we can always rely on photocopier suppliers, to accurately invoice copy-count each month. So no reason at all, to spend an hour wandering round your companies machines, auditing their usage figures for your own “peace of mind”.

  4. Dan:

    Another point – risk.

    The hard shoulder is there for drivers that break down. Because the hard shoulder spends a lot of time empty, they are allowing drivers to use it at peak times. If this had happened with you, you would likely be another statistic by now.

    Lesson – just because a risk happens rarely, it doesn’t necessarily mean that you can do away with any mitigation measures. It might be expensive for (seemingly) no gain a lot of the time, but when it is needed it’ll be worth it

Discuss this:

Your email address will not be published. Required fields are marked *