“Watching the (CPO) Detectives” — Measuring digital procurement success

Organizations aiming to transform their procurement processes (for more lucrative supplier relationships, improved decision making, faster workflows, risk mitigation and overall efficiency – the list goes on) are investing heavily in digital automation and AI-powered solutions. However, a significant challenge they face is how to measure success, sustain it and quantify the return on investment (ROI).

Success for procurement has historically been measured in terms of cost savings. But procurement’s role has evolved: the procurement leader must be the detective, identifying gaps and shortcomings, analyzing data, finding tech solutions and solving problems. Today, ROI must be measured in a more structured and inclusive manner. This involves improved decision quality, faster cycle times and higher supplier engagement.

Digital transformation — then and now

Before we examine measuring and ROI, let’s consider why many tech implementations fail. We talked to Melissa Drew, who has been a buyer, supplier, procurement leader and consultant, alongside international keynote, thought leader and author, to mine her extensive experience in digital strategy and transformation.

“Looking at digital transformation over the past 30 years,” she says, “we typically transform because we are reacting to something triggered by external forces, like consumer demand or a new leader entering the business. Couple that with crises like the Covid pandemic or the 2008 financial slump, and we see how reactionary transformations have been.”  

Disruptions from tech advancements have also forced a reaction. “In 2005, we started to move more towards automation, away from the ERP; from 2008 to 2010, we shifted again to cloud technology. In 2015, IBM introduced a new chip that enabled faster processing times. We now had the tech capabilities to assess all the data we had been collecting for the past 20 years, much more quickly.”

Suffice it to say, every time there’s been a disruption, there’s been a reaction — the need to transform. “Given the scale of the trigger, our traditional approach to transformation has been to ‘redo’ everything. However, as we move forward, we can expect to see more disruptions, both positive and negative, which will become more frequent but on a smaller scale. So while traditionally the obvious reaction has been to transform completely, procurement won’t have the capacity to carry out months of transformation every time there’s a disruption.” That means we need to …

… Get it right first time

Melissa advocates that ideally, transformations “when done correctly, should enable you to adapt to the smaller and more frequent disruptions that continue to come your way, rather than completely transform each time. When a CPO states that they are now on their fourth transformation – that’s a red flag; it means the second and third ones failed because they didn’t carry out the first one in the right way. They didn’t build the foundation needed to allow the flexibility to adjust.”

In fact, she believes the whole concept needs to change, because a successful transformation will be the one that can flex and adapt to create sustainable capabilities.

And that’s an important point (in fact, a whole section of our Procurement Transformation Survival Guide is dedicated to Building the Foundations). 

“Every project starts with what you put in at the beginning. And if that’s broken, then you’ve broken the transformation before you’ve really started,” she says. “So you might want to think upfront about things like behavioral indicators of adoption, whether you are capturing complete and quality data that is consistent and standardized, whether it’s being routed through the right business rules or requirements, whether it’s transparent, and so on. 

“As for metrics, you need to connect the dots between that intake and any downstream inefficiencies. If you are focused on just the metrics for requisitions, for example, then you aren’t looking at the process as a whole – just placing a band-aid on something that happened downstream without recognizing that the issue was at intake.” So project success hinges on getting it right the first time.

The importance of company culture  

Melissa advises that transformations fail because of the decisions that are made before, during and after the initiative, and those decisions reflect culture. “Culture is at the heart of transformation success,” she says, “and it’s not something you can change overnight. The success of the transformation is quite heavily dependent on how ‘change management’ is carried out,” which she refers to as ‘readiness and adoption.’ 

“That,” she says, “focuses on making sure individuals are aware of the change and are willing to make the change. Suppose the culture is not right within the organization from the start. In that case, when the adoption or change management team moves on after deployment, people will often revert to their original way of working, like sending emails instead of using the system. The culture of a company affects how it approaches and embraces change and the importance it places on valuable assets (good data is one example). And it’s the leader who has to enforce this culture. 

“So sustainable success is ‘change + culture.’ It doesn’t matter how good the change management if you don’t reinforce behavior.” So how do we measure that? 

Not all metrics are equal 

“One of the lessons I’ve learned throughout my career is that not all metrics are equal,” she says. “The challenge comes when the KPIs and change management metrics on an organization’s dashboard haven’t been considered and generated by the organization itself; they’ve been replicated from another entity’s project or a consultant’s website. 

“The metrics you adopt must be relevant to you and what you are trying to achieve. It’s important to consider: 

• What was the original purpose of the project? 
• Does the metric I’m applying reflect that?

“Let’s take some examples:

• If one of your goals was to get more spend through catalogs, which you’ve spent months building, what exactly is the metric? Are you simply measuring how many people are creating requisitions on-catalog versus off-catalog? Or are you measuring true spend under management? So, in creating your metrics, you really have to go back to what the original driver for the change was. 
• Let’s say a company puts a policy in place that allows any requisition under a certain value to be automatically approved. It works on the assumption that the risk threshold is so low that it can go from requisition to PO without intervention. To track that, you run a report showing how many requisitions were created under that value that were automated. However, you must align this with the original purpose of the automation – to save a significant amount of time on the requisition-to-order cycle time for low-risk items like office supplies.
• Let’s assume you have a pilot program with 100 users. You can track the unique number of users who log in, and also determine whether it’s the same user logging in or a new one. So, if after three weeks of rollout only 30 users have logged in, that’s an actionable metric that you can hand to the adoption team to address the other 70 users. 

“Again, it’s not just about metrics being measurable but bridging the gap between measuring the outcome against why you instigated the change.”

Criteria for success measurement

A procurement leader who has invested a lot of the company’s money in a transformation needs to make sure they outline their criteria for success in advance. 

“An indicator for a procurement leader would be: did I align the outcomes of my goals with my expectations? Some of that will come from daily communication with super users and change champions to really understand what is being achieved. On the other hand, you need to spend time post go-live to understand what is not working and what can be strengthened.

“So indicators for the leader must be about the positives and the negatives. For example, looking at your workflow, you can track how many times AP had to intervene and how often the procurement approval process stopped because it was routed unexpectedly to someone for review. Those are the interventions that we didn’t expect to happen because the purpose of the transformation was to automate fully.

“In my experience, one of the best ways to observe the impact of change is to look at what doesn’t break during a disruption. If you have flexibility and adaptability built in from the start, then you’ll see your team pivoting faster when a crisis arises. 

“Look at how people adapted during a disruption before the transformation. A good example is Covid-19, when everyone had to suddenly shift towards emergency mode for things like health care supplies. Did the system allow for that adjustment? Or did everybody suddenly stop for three weeks because you didn’t have the flexibility to allow things to flow through it? 

“If adoption is done properly, and your transformation is set up to be flexible, there are fewer pivots because you can adjust and adapt. Then the next time there’s a disruption the leader has to look at how quickly they can pivot, not whether they can. And that is what we mean by ‘sustainable change.’ Being able to see what ‘broken’ looks like and being prepared for it next time. You can’t predict every single definition of disruption; you can only learn from what you’ve experienced before: weather, consumer demand, health, pandemic, supply chain. But there are many others we haven’t even thought of yet.”

All things considered, the terms of success have come to hinge on something more than cost savings. “Success is very much dependent on the procurement leader becoming a front-seat leader, asking the difficult questions like: did what we installed do what it was supposed to do? And they need to enforce culture and behavior, and continue to communicate it, because the more things get reinforced, the more people will remember and adopt. 

“So, each time there’s a disruption, you shouldn’t need a transformation; you just need to be able to make minor adjustments to allow you to adapt.”

Melissa’s book goes deeper into all these themes, find that here:
The Evolution of Procurement and Supply Management Transformations