The 3 types of Procurement error – which one did for the West Coast Franchise?

Following the West Coast Franchise debacle, I’ve been thinking about “procurement error”.  This is particularly relevant in a public sector context but actually everything I’m about to say applies in private sector activities too (except the specific EU regulatory comments). The main difference is we don’t often hear about private sector disasters ...

I’m excluding from this debate fraud, corruption and deliberate bias. Maybe we’ll come back to those options at a later date, which cover an insider helping one bidding firm to win, or where a a Minister / Chief Executive simply says “I want that supplier to win” and it happens!

I’m also focusing here on “error” in the sense of an outcome that could be challenged legally (in the public sector) or conceptually (private sector) – not simply a procurement that does not get the optimal result. So a poor negotiation or not researching the market fully would not count under my definition as an “error” in this sense - we might call it a shortfall against best practice instead.

So, I would suggest there are three types of error.

Type 1 – A faulty process

This covers a multitude of options. It could be a process that is non-compliant to regulations, or has clear logical or numerical mistakes.  So a method of calculating weighted scores that is mathematical incorrect, or uses an evaluation process that breaks EU regulations. (“ All Greek firms will be eliminated from the tender...”).

 Type 2 – Process not followed

In this case, a process is defined that is appropriate, but then it is not followed. That leads to an illegal process (in public sector terms) - not giving all the bidders the same information, or evaluating suppliers in an unequal or unfair manner perhaps  - or simply an error in the decision, perhaps through incorrect arithmetic in calculating scores. This is probably the most frequent cause of problems, I’d suggest, certainly in the public sector.

 Type 3 – Moving the goalposts

In this case, something changes, or the current process is bent into a different shape. That introduces error or unfairness into the procurement. Now technically, we might still see this as a variant of Type 2, in that if change is made and implemented  properly, it  can be managed. But errors arising from this are so common, it seemed worthy of its own error classification.

But then we have Type 4, which I would argue are not “errors” at all but a question of judgement. Maybe I’ve weighted price very highly in my supplier evaluation – you think quality should have been given more focus. I ruled out any bidders for this contract whose turnover was less than £10 million; you didn’t think that was important.  These are questions of judgement, not fact. My decision to weight price at 80% may be seen as bad judgement if the consensus says it should have been 20%, but I would argue strongly it wasn’t an error.

So, in terms of West Coast franchising, what has happened? Which type of error are we talking about?

Type 1 – perhaps a method was defined in the process for calculating some key number in the evaluation process, but that method was simply wrong. I’m simplifying, but that could be a “no, that’s not the right formula for calculating NPV” type moment! Or an out-of-date set of projected inflation figures was used in the financial model?

Type 2 – the methodology was fine, but something went wrong in the execution. Bidders were not treated the same, or the sums were done wrongly compared to the stated method. I’ve actually seen it come down to a * instead of a /  in a cell on an Excel spreadsheet.  One early report from the BBC yesterday suggested it might be as simple as this  - “The Department for Transport was using an entirely new model for evaluating bids, a spokesperson told the BBC. The mistakes appear to have been down to errors made in the inputting of the data, rather than with the model itself, he says”.  That focus seemed to change later though in the reporting.

Type 3 – half way through the process, someone said, ”let’s assess risk in a slightly different manner...”  and informal changes were made to the process, or numbers tweaked... but in a manner that on closer inspection turned out to be flawed or unfair.

Or was it a Type 4 error – i.e. not an error at all? Reports now talk about assumptions around inflation and passenger numbers being flawed. But assumptions are not facts. A new Minister might decide he doesn’t like these assumptions – but if they were made in good faith initially, that was not a process error, although it may have been poor judgement.  (And one would have thought that all key assumptions would have been discussed, checked and signed off, incidentally).

If the West Coast problem turns out to be a category  4 “non-error”, then the suspension of three staff is an over-reaction and we will probably find that no-one ultimately and directly carries the can..

One further point here, regarding the suspension. I’m all in favour of civil servants, consultants or whoever taking accountability for errors. But if we’re going to start suspending  everybody involved with faulty public procurements, we’ll quickly get into real problems. I’ve heard of two other  procurement exercises recently that were pulled at tender stages because of errors. Neither is as newsworthy as this one, or as expensive to the taxpayer, but how about the people who made “errors” there?  There could be some very depleted procurement functions around if we follow this principle everywhere...

Share on Procurious

Voices (8)

  1. Tom Graham:

    Where was the Gateway process??

  2. John Diffenthal:

    There is plenty of evidence that the number of challenges at all stages of public procurement are increasing sharply – if you need absolute figures then Achilles is a pretty good source.

  3. John D:

    …and it’s all because Branson decided to mount a challenge…an all too rare event. How many challenges were there across the public sector last year? A very small percentage.
    For me the lesson is that “the system” will only work as intended when there is sufficient perceived risk of challenge. Right now non-compliance/procurement error goes largely by the board.

  4. Janine Baker:

    One of the main problems with public sector procurement is that common sense gets taken out of the equation. Departments need to agree up front exactly how the tenders will be evaluated, and they need to tell the bidders as well. This leads to an army of bid professionals scrutinising this to see how they can optimise their scores rather than providing the best bid or the best value for money. This all stems from noble aims of transparency and fairness, but can end up achieving the wrong outcomes.

    The problem happens that when you get to the evaluation process and suddenly find how one of the bidders has manipulated their tender to exploit a hole in the scoring criteria, the procuring department is in a bind.

    If they move the goalposts to close the loophole, they can be challenged by the winner as they needed to make that clear up front. If they don’t move the goalposts and the loser finds out, they will want to challenge the outcome as it “seems” unfair. So the procurement function is damned if they do and damned if they don’t.

    The only solution they then have is to abandon the procurement and re-run it with clarified criteria to close the loophole. But – apart from the political fallout and costs that we are now seeing – this doesn’t help either. Until the bids go in, the approaches and prices are secret; after the bids go in, the cat is out of the bag and you can’t really run a fair re-run as the losing bidders will know what they now need to do. So all in all, you’re stuffed.

  5. John Jones:

    Frankly, one can debate the categories for types of errors till the cows come home in my view – it’s not the issue – we will find out soon enough (end of Oct) what the “official” report says. Might even de-bunk a few conspiracy theorists 🙂

    Irrespective of the type(s) of error that Peter describes, there is a fairly well tried and trusted process that I have seen used on many/most of the largest procurements UK government procurements (CD/negotiated) in the past 10 years and that is the use of a) moderation and b) external business assurance or scrutineering (by the “grey beards”) both during in-flight bids and prior to final award of contract.

    Moderation post ISOS/ITN has the role of quality assuring the evaluation and taking bias out of individual evaluators (can pick up numerical errors as well) – the purpose of in-flight scrutineering is to show adherance to the process (in all forms, think verification in software – have I followed the process correctly) and, prior to award, have I used the right process in the right way (think, validation in software – have I made the right product – a plane with one wing? – oh dear!)

    At the end of the day, procurements are run by people – inevitably, mistakes will be made – let the head’s role.

  6. Dave Orr:

    The Minister Justine Greening knew of the procurement flaws BEFORE West Coast rail franchise contract award:

    Was “rising star – tipped for the top” Justine Greening’s move out of DfT nothing to do with Heathrow then?

    First she gave Green light for HS2, then approved this against critical warning report……WAS THAT IN THE RISK LOG BEFORE SIGNING?

    Peter: FOI for latest Risk Log before franchise award??

    I still think that the risk & inflation model used was politically designed to get “the right answer” i.e. justify the HS2 hi speed rail project London~Birmingham mega bucks project (potential winner for the White Elephant of the Decade award 2019).

    When will we hear which BigCo consultancy company was hired in by the DfT to produce this franchising bid assessment model?

  7. Verity:

    An interesting thought process Peter, though I expect we’ll never hear the real reason!

Discuss this:

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.