Back to Hub

We have some hard questions about (and a soft spot for) Coupa’s new Business Spend Index (BSI)

12/11/2019 By

Coupa recently released its Q4 Business Spend Index (BSI). Indexes such as ISM’s PMI and NMI can be helpful tools for procurement — not to mention economists, investors and policymakers — to gauge overall economic trends as well as underlying supply, demand and inventory conditions.

Coupa is a relative newcomer to publishing an index, but it offers a potentially promising approach based on underlying transactional data flowing through the Coupa system — as opposed to the polling data that is used for the ISM indexes, for example — to complement other economic indicators.

Still, we recommend that firms not read too much into the BSI for making procurement, economic or investment decisions without some additional details. As long-time publishers of our own MMI indexes on MetalMiner, we are well aware of the benefits and limitations on index data as leading and lagging indicators of market conditions, in addition to their utility as contracting mechanisms in specific industries and categories (as an example, hedge fund managers wanted to backtest our data to the nth degree of commodity price predictability). Therefore, we’re especially sensitive to what an index can offer.

Coupa’s BSI is a potentially useful data point to consider in business spend planning because of the general scale of its data set (even though there aren’t hard numbers provided on the total participating firms, users and the spend itself) — and also the fact that it’s based on real-life spending behavior.

The core BSI metric is comprised of one after-the-fact metric on spend itself (a “spend per person metric that seems based on a system user vs. an employee) and then two “before-the-spend occurs” metrics: approval time and approval/rejection rate before the POs are even issued. When business conditions get tougher, those approvals take a little bit longer and sometimes get reduced or rejected outright.

Still, it’s unclear how to best use the BSI and how reliable of a leading indicator it actually is. The industry insights from the latest report tend to illustrate some BSI element, and then make some broader commentary on the industry that is not necessarily directly drawn from the data. There are important supporting details missing from the index, and without them, it’s unclear how useful the BSI actually is in the trenches, and what for, specifically.

There are a few issues we’d like to see addressed if Coupa truly wants to make this a useful planning tool and not just a marketing vehicle for its community intelligence strategy:

  • How predictive is the index actually? What economic growth indicator is correlated against the BSI? What is the r-squared of BSI versus this indicator at how many months out? Having some insights on this would be useful beyond just saying “trust us, it’s predictive.”
  • To what extent does the BSI actually have enough data to properly drive its model? What’s the sample size by mega industry? How about sub-sectors? There are huge differences within the industries shown in the report (e.g., Manufacturing or Health & Life Sciences)!
  • How important are geographic considerations in the index inputs and values (e.g., for cross-border transactions or those assigned split between different GL codes in different regions, how is the data evaluated and considered?)
  • How did Coupa’s changing of the BSI model elements and calculations affect this period’s data? Have previous periods of data been converted to this new methodology? A more detailed version of the report with this type of footnoting would be helpful.
  • What spend data and associated spend management data is really in the BSI — and what does it really predict (or not)? The BSI does give some view into large corporations’ behaviors, primarily around operating expenses, secondarily around capital expenditures, and minimally on supply chain spending (i.e., COGS). Coupa mostly measures “indirect” spend, so one shouldn’t read too much into the broader conclusions based specifically on this data unless more details are provided. For example:
    • For supply chain-focused industries, what percent of the spend is actually direct?
    • For that direct spend, is there accompanying insightful/relevant metrics on approval times and percentages, and does it make sense? This spend (and the PO/invoice activity) is dependant on supply/production planning, not discretionary approval decisions made by finance.
    • Can Coupa split out SG&A spend vs. CapEx spend vs. COGS? It’s possible if clients map those G/L spend types back up to POs, but likely not.
  • How do these metrics filter out other key variables that are missing or misleading? For example, if the number of persons in the systems are growing as more users are getting online and implemented for Reqs/POs, won’t that drive the spend per person down? Are adjustments being made? Also, to what extent does “seasonality” drive the numbers? For example, Q4 can be a high-spend quarter because of the “hockey stick effect” and use-it-or-lose-it budgeting processes. Our research indicates that almost 15% of indirect spend is wasted because of these policies. We’d love to see some quantitative understanding of these effects.
  • How should a procurement and finance professional actually use the index? For this new Q4 data published two-thirds of the way through Q4, what time period is it for compared to Q3? And how relevant is it to me? Can I get some confidence score that goes along with the number based on my company attributes?
  • Why is Coupa using forensic “spent management” data alone rather than also including a survey-based sentiment assessment? Answering this question would be valuable to those who might use the index. For example, when I was doing benchmarking at The Hackett Group, we had a service called “Hackett Performance Exchange” (see old write-up here) that combined transactional/spend data mining alongside validated survey-based benchmarking for FTE/outsourcing investment, technology investment, best practices adoption metrics, and other data. Coupa might want to consider administering its own annual sentiment-based survey to augment its forensic insights (e.g., adding questions about current/expected budget levels). It could also partner with ISM (with its survey-based PMI) or CIPS to combine both data sets and essentially crowdsource some prediction market data on planned spend (rather than just waiting for the “spend exhaust”) and broader business planning sentiment.

We applaud Coupa for investing in the BSI and especially for making it a public service that does not require someone to be a Coupa customer to use it (we also like Coupa’s continuous improvement philosophy and its current efforts to drill down into high-level category insights and other areas where they’ll take it). But for the BSI to become invaluable for us and others, we would like to know on a more granular basis what it is truly comprised of, what it means and usage recommendations. We can also foresee how Coupa could use this to engage non-Coupa customers with the core BSI data as well as some adjunct survey-based data (which could use spend/process mining tools from its Value Engineering teams or via Coupa’s deep bench of consulting partners).

We reached out to the Coupa team for their insights on the topic but they declined to comment prior to publication. However we also welcome any reader feedback in the comments section below.