Q4 2019 SolutionMap Release Notes: Source-to-Pay Suites

This Spend Matters SolutionMap Insider release note provides insight into the Q4 2019 SolutionMap release for Source-to-Pay, reviewing the process that we followed for this quarter’s release.

The providers in the Q4 2019 Source-to-Pay SolutionMap include Coupa, Determine, GEP, Ivalua, Jaggaer One, Synertrade and Zycus. All of these providers were required to participate in SolutionMap for each underlying module, as well. These individual areas include Sourcing, Spend and Procurement Analytics, Supplier Relationship Management & Risk, Contract Lifecycle Management, E-Procurement and Invoice-to-Pay. All of these providers are also included in the individual Strategic Procurement Technologies (SPT) and Procure-to-Pay (P2P) suite views.

*Get a first look at the Q4 SolutionMap comparative quadrant graphics here.

The Q4 2019 SolutionMap marks the first time Jaggaer One (formerly Jaggaer Indirect) is participating in the S2P Suite view, having participated in each underlying module.

SAP Ariba did not update their scoring and, as a result, saw their scores adjusted based on a common shift factor (that preserves positional integrity relative to peers) and any platform updates, or lack thereof, that the analysts have received since the last time the RFIs were filled out. We expect them to participate in upcoming releases.

The Source-to-Pay customer data set is composed of a subset of the 886 individual organization references included in the overall SolutionMap scoring as of Q4 2019*.

This SolutionMap Insider research note provides insight into the nature of the expanded RFI, methodology, customer references and additional changes in the Q4 2019 SolutionMap release.

*Customers using more than one module that a vendor provides can opt to fill out a single survey within each individual suite area (e.g., a customer using two modules for P2P and four modules for SPT would need to fill out only two surveys).

Additional Source-to-Pay Suite Scoring Considerations

Only providers participating in the source-to-pay suite view had their data used to drive the inputs for scoring. The S2P scores, charts, averages and axes are all created by using only S2P participant vendor scoring (i.e., Coupa, Determine, GEP, Ivalua, Jaggaer One, Synertrade and Zycus).

Only Recommended Vendors in Each Persona are Shown

To encourage participation among the widest range of vendors — especially those specializing in a specific market niche — and to simplify how procurement practitioners interact with and use each persona-based graphic, the SolutionMap release features only the recommended vendors for each persona.

Vendors can be recommended by either achieving Customer Value or Solution scoring that is above the average based on the weighted criteria for specific personas, which are as follows:

• Solution Leaders are recommended providers with above average analyst scores (upper left quadrant)
• Customer Leaders are recommended providers with above average customer scores (lower right quadrant)
• Value Leaders are providers with both above average analyst and customer scores (upper right quadrant)

Our standard graphical displays show the geometric averages of the providers, but we may occasionally expand the view when the map has certain providers that skew the results of the overall population being displayed.

Customer References

When using the Q4 2019 SolutionMap release compared with the Q3 2019 release, it is important to consider how underlying customer references affected the scoring. The SolutionMap customer survey reference database currently contains 938 active responses from different customers. The Q4 2019 SolutionMap incorporated 34 new individual customer survey responses for the Source-to-Pay suite view.

Beginning in Q2 2018, Spend Matters began directly soliciting references from the procurement practitioner community by inviting them to complete the public version of the provider reference survey, in addition to the traditional provider-solicited-reference method. Entries are manually reviewed and validated by the Spend Matters team by checking the identity of the submitter and ensuring the responses do not seem suspicious (i.e., extreme scoring throughout the survey out of line with the average for that provider). Spend Matters does not influence which provider the submitter chooses to review in any way.

The inclusion of additional customer feedback can positively or negatively affect each provider’s Customer Value score regardless of whether a vendor submitted updated references, as each individual survey affects the benchmark. This effect can also be magnified when references beyond the two-year lifespan drop out of the database. However, providers have the opportunity to have their references “refresh” their surveys to reflect the latest customer satisfaction status.

It is also important to note that Spend Matters has no control over this variable and that the only way for a solution provider to improve (or decline) in Customer Value scoring is to incorporate new customer references until previously submitted references “age out” (which again, can be mitigated if customers refresh their scoring and come back in).

Persona Names and Descriptions

The Q4 2019 persona names and definitions are the same as in the previous quarter, and the weightings for the new common categories in each persona had been defined in a manner consistent with the weightings for the questions they replaced in an attempt to keep each persona as consistent as possible with the definition in the Q1 2019 SolutionMap, including the Nimble persona. (The Nimble persona had been slightly re-defined on the back end in Q2 2019 to produce a result that is more consistent with the definition of “nimble” that we have been using since day one — fast to implement, focused on specific needs, affordable, and easy to get going.)

The personas remain:

• Nimble
• Deep
• Configurator
• Turn-Key
• CIO-Friendly

We base the SolutionMap ratings approach on an expectation of evolving requirements. However, we did not change any of the weightings for Q4 2019.

For detailed definitions of and functional emphasis placed on each persona within individual SolutionMap area, see our Persona Definitions pages.

Scoring Criteria: Solution/Technology Inputs

We’re not believers in “grade inflation” when it comes to scoring technology vendors based on features and requirements. In fact, the average analyst score for each individual field (e.g., supplier onboarding via a network) across all SolutionMaps for Q4 2019 hovers around 2 (on a 0-5 scale as defined below). For a particular field, the criteria we follow for scoring is that the first three points are entirely based on whether or not a vendor meets specific functional requirements, or equivalents thereof. Our scoring guide is as follows (detailed question-specific scoring criteria are also of course provided):

0 = Not supported

1 = Partial support for select requirements

2 = Core support for Standard requirements

3 = Advanced support: moderate to high levels of supported complexity/value

Beyond a 3, a vendor needs to be able to point to the feature or capability as differentiating its solution, either on the requirement/feature level or overall:

• To achieve a score of 4 for a given feature, a provider must demonstrate significantly differentiated capabilities compared with peers for a particular requirement or a functionality that is significantly beyond a 3 on our scoring scale. Our scoring scale is designed with the intent that not more than 10% of all scores will be a 4.
• A score of 5 requires that a provider score at least a 4 in functionality and either offer functionality that is so distinct it stands out, or a capability for which the provider ultimately wins business due to significant strategic differentiation. A score of 5 is at the discretion of the lead analyst for the SolutionMap area. Our scoring scale is designed with the intent that only 1% of scores should be a 5.

It is important to note that our scoring is based on the knowledge and visibility that Spend Matters has from having studied each technology segment since its inception, going back 20 years (or more) in certain cases. This is what enables our analysts to objectively label features and requirements that are partial, standard, advanced, significantly differentiated or truly extraordinary, and to define a scale which specifies concretely what we look for, by default, for a score in the 1 to 3 range.

Scoring Criteria: A Rising Bar

Because solution providers are continually investing in R&D and improving their capabilities, some features and requirements that factor into higher Solution/Analyst scoring (4 and 5) will see less differentiation with each SolutionMap release. This statement holds true for the latest individual Sourcing, Spend and Procurement Analytics, Supplier Relationship Management & Risk, Contract Lifecycle Management, Strategic Procurement Technologies (Suite), E-Procurement, I2P and P2P (Suite) SolutionMaps.

For example, a feature or requirement for a particular provider that may have been a competitive advantage in a 2018 release may not be in 2019, as what we previously considered as differentiated becomes more common based on existing and newly participating vendors in the SolutionMap benchmark. That’s why many of the questions across the Strategic Procurement Technology SolutionMap areas received updated scoring scales in Q2 2019, to allow for some of this differentiation to be noted. And for those questions that did not realize a revised scoring scale, this is also why some vendors who may have implemented additional functionality did not necessarily attain a higher score.

In this round we continued to assign half scores to those vendors that made a considerable effort but did not advance to the next level on the scoring scale. Finally, in future SolutionMap releases, in 2020, we expect to continue to lower requirement weightings in commodity-level functionality that basically all providers support, and also add in more/deeper requirements for emerging capabilities such as direct sourcing, advanced analytics, GRC, quality management, and/or other related (and emerging) areas. We plan to gather feedback that we’ve received from providers and practitioners and also perform formal requirements-gathering to make sure that future versions remain in line with the market reality and market trends.