Suppliers will “game” the system if you make procurement evaluation too complex

It feels a little early to return to the subject of tender evaluation after our mammoth series a couple of months ago.  But one additional observation, based on a few tender documents I’ve seen or heard about recently. Procurement executives need to understand - the more complex you make your evaluation methodology, particularly for price / value criteria, the more chance you have of suppliers “gaming” the system.

So if you are evaluating a simple price, or even a series of prices for different items within a single bid,  that’s usually fine although bear in mind the comment we made last time around the different ways of converting price to a score. But when you start getting into complexities such as:

  • discounts or rebates as well as basic prices,
  • weighted price components that go into a total price which is scored
  • “added value” offerings that may translate in some sense into a final price score

- then you invite suppliers to do their upmost to get the best possible score for any given “total value” they offer.

I’m not going to go into exactly how that might happen, or specific examples  – trade secrets and all that. And we would get into some pretty detailed Maths to look at how it all works. But believe me, it does and it will happen if you make your scoring system overly complex.

Gaming the tender response...

If you’re in the public sector you also run the risk of breaking the basic “most economically advantageous tender” (MEAT) basis for awarding contracts.

I know of one successful challenge – settled before it came to court – where the challenger showed that a higher total cost bid could score better than a lower because of the evaluation process. That was clearly illogical and counter to the basic MEAT principle. There are others I’ve seen that have a similar potential failing.

The irony is, it is often driven by procurement people trying to do the right thing in terms of getting the best deal – hence asking for the discounts, added value etc. But there’s a real danger, if you don’t get it exactly right, that the end result will be worse than a simple method might have achieved.

So remember, every additional complexity you introduce will make it more likely that a “gamed” response will win - or you’ll get challenged be a losing bidder.

Voices (5)

  1. Janine Baker:

    I agree with you that as procurement evaluation becomes more complex, the opportunities for bid specialists to win by writing according to the evaluation criteria become greater. However, I don’t agree that looking at headline price is always the way forward either, and a simplistic adoption of MEAT does not really make sense.

    On one procurement I know of, the expected benefits were several hundreds of millions, with the ability to realise these driven largely by how good the solution was. The customer saved around £5m by picking a vastly inferior solution and ignored the impact this would have on benefits. Whilst they no doubt presented this to their lords and masters as a great success saving £5m off the headline price, the cost in lost benefits over the lifetime of the system will be an order of magnitude larger than this.

    I actually think the main problem with public sector procurement is that it is (supposed to be) so scrupulously “fair” and “transparent” which allows bidders to manipulate their responses to meet the evaluation criteria. In the real world (ie the private sector), businesses are allowed to use common sense and adapt and adjust to pick the best solution for them, rather than being shoehorned into a pre-determined and published evaluation methodology.

    1. bitter and twisted:

      2 thoughts

      When recruiting, would you send the candidates the evaluation criteria and marking system?

      If you have to choose between 2+ good and very close bids, does it really matter who wins?

  2. Phoenix:

    ?Que?

  3. Alan Holland:

    Every auction or tender is a game that can be vulnerable to strategic manipulation if the economic mechanism (rules for who receives what and the payments made in exhchange) are not carefully designed. The field of mechanism design in economics describes the properties of the mechanism that should be satisfied in order to incenitivise honest behaviour by bidders. Mechanism design is a form of inverse game theory whereby the designer assumes all the bidders in the auction will act selfishly. The “Nash Equilibria” describe a set of possible outcomes so this is the solution concept that informs our design. Computational mechanism design goes a step further to study the set of mechanisms that are tractable and don’t lead to impossibly complex problems for decision makers.

    I don’t fully agree with the claim that “every additional complexity you introduce will make it more likely that a “gamed” response will win”. Strategic manipulability depends upon features such as monotonicity of the allocation scheme etc. Some truthful schemes can be very complex whilst some very simple scheme (e.g. an all-pay auction or even a first price auction) have equilibria in which bidders lie about their true value.

    However, I should add that the example side-rules you listed above certainly do encourage gaming.

Discuss this:

Your email address will not be published. Required fields are marked *