Methodology

The SolutionMap methodology has been updated by the Spend Matters analyst team for the Q2 2019 release. This marks the first major “revamp” since the SolutionMap’s Q2 2017 launch

Having a platform-level understanding allows potential technology users to discern the level of integration between product modules and suites, overall interoperability with ERP and broader technologies, as well as the quality and effectiveness of advanced technologies such as artificial intelligence (AI).

All ranked providers were required to re-do the SolutionMap review and scoring process in its entirety. Although the core methodology as outlined below hasn’t changed, the RFI requirements were expanded to well over 1,000 and divided into platform and functional components across source-to-pay and services procurement modules and suites.

Adding such a level of granularity allows for deeper functionality and platform capability assessments and ensures objective scoring scales (see example below).

To illustrate the objective scoring format, here is a sample common configurability, globalization platform requirement for Source-to-Pay (S2P):

Multi-Lingual
How extensive are the multi-lingual support capabilities?
0 - not currently supported / not applicable
1. flat file menu mappings for a small set of languages;
2. dynamic mappings of menu options and help text based on standard translations and regional linguistic variances and support for over a dozen languages;
3. override features that allow a buyer to override mappings on documents / menu options being shared with a supplier and over twenty languages supported;
4. Includes capability beyond which is previously addressed and beyond peers

As before, validated user/peer reviews (1,000+) are also factored into the transparent vendor ratings. Analyst scoring makes up the left side of the quadrant, customer summary scoring determines the right side of the quadrant. SolutionMap rankings are updated and published quarterly. Procurement professionals consult the free rankings, subscription research and data to drive procurement technology selection decisions.

Here’s how the rankings are created:

  1. Every SolutionMap has a dedicated lead analyst supported by the greater analyst team plus operational support staff
  2. The lead analyst develops the RFI in collaboration with the internal analyst team, customers and consultants to ensure requirements are comprehensive and relevant to practitioners following a set methodology
    1. Capabilities definitions, personas and rating weights defaults are determined ‘up front’ before issuing the RFI
  3. During the RFI process, vendors need to:
    1. Provide detailed company and customer demographics
    2. Self-score solution requirements (with detailed specifications )
    3. Describe all solution related services available to customers
    4. Provide customer references
    5. Supply documentation regarding:
      1. Revenue models
      2. Buyer and supplier fees
      3. New vs. replacement customers
      4. Customer TCO, ROI and success metrics
      5. Case studies
  4. Customer (user) references complete a detailed survey focused on key provider differentiators, delivery against expectations/requirements, strengths, promoter score, collaborativeness, improvement areas, problem solving skills, features/capabilities improvement needs, shortcomings and more (making up 50% of the score)
    1. Individual customer inputs are never shared with the vendor community
    2. Customer identities are known to the core Spend Matters project team only
    3. Customer inputs are aggregated to form the basis of the customer value ratings
    4. Spend Matters also directly solicits references from the procurement practitioner community
      1. Entries are manually reviewed and validated by the Spend Matters team by checking the identity of the submitter and ensuring the responses do not seem suspicious (extreme scoring throughout the survey out of line with the average for that provider)
      2. Upon approval, the survey participants receives a participation reward of choice
  5. Analysts participate in a 90-minute technology demo (per module) for each RFI participant to validate the self-scoring and adjust the scores up and down based on the technology demonstrations
  6. Analysts rate providers against strict functional / solution definition requirements after reviewing vendors’ self-scoring RFI inputs and demonstrations
    1. Demonstration notes and documentation may also be reviewed and extra spot checks may be performed
    2. Solution draft scoring is finalized for each vendor
  7. Vendors are invited to fact check their scores on a granular level against analyst scoring and request clarification
    1. Vendors have the option to appeal individual functional scores at a granular level via a defined process involving additional written documentation, calls and demonstrations
    2. SolutionMap placements are never shared prior to final publications with providers
  8. Final SolutionMap rankings are created and produced in due course based on finalized data and previously defined criteria, weightings and inputs
  9. Participating vendors are required to review and update their full RFI scoring inputs every 12 months (‘general’ platform-level inputs as well as category-specific self-scores)
    1. Vendors may submit intermittent category updates (max. twice every 12 months for a each single RFI category)
    2. Any scoring updates require validation demos.

The following assessment elements are taken into account:

  • All elements of a technology*
  • Integrations
  • Supplier experience
  • Front end users and administrators’ user experience

* The technology elements are very detailed and vary by the technology being ranked