How to Interpret SolutionMap Procurement Provider Technology Rankings

When you first lay eyes on a SolutionMap Ranking you’ll notice the ‘Reading the Ranking Chart’ box on the right hand side stating a key — but there’s more to interpreting SolutionMap. Let’s explore the different elements of which you should be aware.

Please note: In the Q2 SolutionMap ranking graphics, providers with an asterisk (*) next to their company name not yet evaluated against Q2 2019 RFI requirements, but rather normalized based on scoring from Q1 2019 publication.

1. What You're Looking at in Each Persona Graphic

Each graphic has two axes in play. The (y) axis represents the analysts' score of the vendor's solution. The (x) axis represents the customer survey score of the vendor.

To further help frame what's actually represented in each graphic, here's a helpful note on their creation:

All SolutionMap persona graphics are centered at the average of participating vendor scores across the analyst score (y) axis and customer score (x) axis for that specific graphic and scaled accordingly. Vendor and customer scores, and hence relative placement of a vendor's bubble, will differ by graphic as each score or response is weighted differently across the different personas.

2. Customer References

You will notice the varying colors of the bubbles on the chart. The colors indicate how many customer references were taking into account for a respective provider. Dark purple indicates a ‘maximum’ and grey a ‘below average’ quantity of references. I’m sure you’re wondering how many references the different bubble colors represent, but there is no single answer to that question. You see, both the providers and Spend Matters continuously solicit references year-round. This changes how many references are considered the average.

For instance, there were nearly 1,000 individual, validated references taken into account for the Q2 2019 SolutionMap rankings. These customer references are valid for 2 years upon submission and manually validated by the Spend Matters team by checking the identity of the submitter and ensuring the responses do not seem suspicious (such as extreme scoring throughout the survey out of line with the average for that provider or multiple submissions from a single IP address).

The greater number of references for a SolutionMap category, the higher the average number of references. For any new release, Spend Matters requires a minimum of three references for a single provider per SolutionMap category in which they participate. (For example, if I participate in Sourcing and Analytics, I need to obtain a minimum of six references. It is possible there is overlap if a single customer uses both modules and selects both of these as applicable in the review survey — I would still need three references for each category but it may be that I only need three customers to get them if they all use both modules and include them in their review).

3. Number of Customers per solution category

Then there is the different size of the bubbles on the ranking charts. These indicate how many customers a provider has in that particular SolutionMap area (so a provider’s bubble can be larger or smaller from one SolutionMap category to the next — it could be that they are well-established in E-Procurement but their Invoice-to-Pay capabilities are relatively new and adoption is not as high).

4. The Four Quadrants

In terms of interpreting the four quadrants of the ranking charts: Providers can be recommended by either achieving Customer Value (horizontal axis) or Solution scoring (vertical axis) that is above the average based on the weighted criteria for each specific persona, which are as follows:

  • Solution Leaders are recommended providers with above average analyst scores (upper left quadrant).
  • Customer Leaders are recommended providers with above average customer scores (lower right quadrant).
  • Value Leaders are providers with both above average analyst and customer scores (upper right quadrant).

Emergent Contenders are providers who did not achieve above-average scores (bottom left). To encourage participation among the widest range of service providers — especially those specializing in a specific market or sub-process niche — and to simplify how procurement practitioners use the rankings, SolutionMap graphics only feature recommended providers for each persona. Specialists and emerging providers may not (yet) score well enough overall to appear on a SolutionMap graphic but might be worth your consideration. Feel free to inquire to learn more.

5. More on a Provider's Relative Placement on the ranking graphic

Again, all persona graphics are centered at the average of participating vendor scores across the analyst score (y) axis and customer score (x) axis for that specific SolutionMap category and scaled accordingly. Vendor and customer scores, and hence relative placement of a vendor's bubble, will differ by graphic as each score or response is weighted differently across the different personas.

You’ll also notice that providers are situated in a different spot on the ranking graphic each quarter. That’s because SolutionMap is not static: As solution providers continually invest in R&D and improve their capabilities, some features and requirements which originally yielded higher solution/analyst scoring will see less differentiation with each SolutionMap release. Three possible reasons:

  • Providers did not make additional investments in a functional area and lost ground to competitors
  • Providers did not demonstrate newly added capability to the Spend Matters team during the quarterly review process
  • Newly added providers set a higher bar for differentiation and advanced capability

That’s why Spend Matters recommends that existing participants in the SolutionMap process provide RFI and solution demonstration updates every other SolutionMap release cycle (i.e., twice each year), or more frequently in cases of major solution releases or product launches.

The same goes for the customer references scores — they aren’t static either: Spend Matters both encourages providers to obtain customer references year-round and solicits procurement practitioners directly. The more customer references, the more accurately the user experience with the provider is represented. Also, customer references need to be renewed annually or they will be removed and new providers will alter the average benchmark and thus the relative ranking of other providers on the chart as well.

If you haven’t yet explored SolutionMap procurement technology service provider rankings, do peruse the site at your leisure and feel free to contact Spend Matters if you have further questions.

Last updated: June 27, 2019

Share on Procurious

Discuss this:

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.