In the past couple of weeks, I've been asking vendors about their current thoughts on the state of the industry analysts. Most are more up-beat than they were a year ago about the direction and frequency of coverage, but many are still clamoring for more domain expertise and depth than they're seeing in the research and recommendations that the firms spout. Below are a few random observations that I've heard of late, as well as some personal commentary on them:
"We're excited that Forrester will be issuing a new series of Wave reports covering more than just procurement and sourcing. Their spring research agenda looks serious, as they embark on their vendor rankings." My personal view here remains that while Andrew Bartels is a talented analyst, he is simply covering too many areas to make substantive recommendations in the Spend Management world. While, I too, await his Wave reports, I'd argue that everyone should at least mildly discount their credibility because Forrester simply does not devote enough effort throughout the course of the year to covering the sector and staying current with real-world customer needs. In my view, you can't dive deep on this sector once or twice a year and "get it" at the level you need to if you're going to issue vendor recommendations and rankings. The analogy here for fully taking stock in what they say would be sending the National Guard instead of an Active Duty Regiment of Marines to fight the most troublesome battles at the outbreak of a conflict. Wouldn't be prudent.
"It would appear that Debbie Wilson's grand research agenda when she came to Gartner has been reduced. She is succumbing to Gartner's "Borg-ish" tendency to assimilate those who join and limit their research ambitions." Personally, I think this is too bad considering that Debbie is covering the sourcing and procurement technology world 100% of her time, and she could make a huge difference in the overall analyst coverage of the sector (and plus, she knows how to write). Still, despite the scaling back of her research ideas, it's rumored that she'll be tackling magic quadrants for at least some Spend Management technology areas in 2007. Let's hope there's more transparency this time than last in the methods and analysis.
"Some of the research Aberdeen is cranking out is excellent, but the quality and level of analysis is somewhat inconsistent". Almost universally, I've heard praise from vendors about the direction that Vance Checketts is taking Aberdeen's research practice. It is important to point out, though, that some vendors have criticized the demographics, approach and sample sizes of Aberdeen's research. For example, one vendor I spoke with pointed out that the eProcurement enablement slam might have been in part due to the number of references and names that a best of breed vendor had provided to them in their research process (which in turn might have skewed the results). NB: I cannot confirm how this research was conducted -- or whether this is just sour grapes or an attempt to spin the results.
As a final side note, having been on the outside, inside and just about every area of Spend Management technology for the past decade, I can say that it scares me that some of the analysts -- not all, but some -- get involved in detailed technology recommendations, rankings and short-lists when they really don't understand all of the solution details at the level they should. And it is cliché -- though important -- to note that the level of vendor interaction time is biased by those who have analyst research contracts. So the net of analyst vendor recommendations and rankings is that while they may be directionally accurate, companies should never use them as the sole basis to select a particular technology or service provider.