While taking a break from an all day podcast recording session earlier today -- stay tuned for more details on it -- I got a call from a friend and colleague at an analyst firm asking me if I had read the latest story on Aberdeen in the Wall Street Journal. It turned out that I had not, in fact. But I quickly opened up my hard-copy version of the paper -- one of the small luxuries of being on the road (my neat-freak wife will not let me get newspapers at home because she thinks they attract roaches) -- and thumbed my way to page B1. And sure enough, the headline was not positive: "Vendors Still Paying for IT Research That Flatters Them."
What followed was essentially a critical examination of the Aberdeen business model. To summarize, vendors pay what the story suggests is around $30,000 on average, to be featured as a sponsor of a single survey (there were 212 reports published last year each "typically with four or five sponsors"). According to the columnist, Lee Gomes, "Most of the half-dozen Aberdeen sponsors I talked with described the attraction of sponsoring a report as a chance to rise above the noise of the marketplace by being associated with something customers consider 'research'."
But how unbiased this research is, the article suggests, is open to interpretation. "The reports seem to invariably discover that 'best in class' companies use, or are thinking of using, or somehow embody, whatever technology the report happens to be discussing," the column notes. Now to be fair to Aberdeen, I do believe the firm has published at least a few critical reports in the past year in the Spend Management space (Vance Checkett's expose on supplier enablement -- and how most companies come up short -- is a perfect example).
But in general, there certainly does seem to be a formula to the Aberdeen model -- and one where the reports could almost write themselves without any critical thinking or subject matter expertise wrapped around them. Perhaps that's why I've heard that Aberdeen is talking to numerous people I've spoken with about "outsourcing" the research and analysis for report writing to them. Then, the firm will essentially slap its logo onto a report and sell sponsorships. So much for quality control -- if it ever existed -- I suppose ...
But my real issue with Aberdeen has nothing to do with the formulaic research findings or the lack of expert analysis in what they present. Rather, my beef comes down to two main issues. First, I don't believe the firm should pretend to be an analyst firm when it is in fact something entirely different -- a lead generation source for vendors (and sometimes, a good one, from what I hear). Second, I seriously question the quality of the data that goes into the benchmarking research. Unlike firms like the Hackett Group which specifically pre-qualify all potential responders to survey research, Aberdeen lacks a data quality control process that would stand up to outside scrutiny (e.g., verifying the employment, companies, titles and responsibilities of those responding to the web-based requests).
At the end of the article, Gomes asks Aberdeen's President Steven Gold why the old Aberdeen business model came up short. His response is that Aberdeen "had become complacent about actual research. We were following a non-scalable and non-value-added model". Hmmm ... I would agree that such research minds as Tim Minahan, who single handily built and preserved Aberdeen's reputation in the dark years following the B2B bust, are not scalable.
After all, talent is hard to find. It's much easier -- as Aberdeen is in the process of doing -- to simply outsource the analysis of questionable survey data to some unnamed third party in its entirety. But to argue that the old Aberdeen model was non-value added is a joke. For years when I was at FreeMarkets, I hired Aberdeen analysts to speak at events, conduct expert research and give us unbiased, hard-hitting advice. These types of activities, ironically, are precisely what the new Aberdeen is not focusing on, despite having a number of smart, capable minds in our sector that could provide such advice and counsel to their vendor clients if they made it a focus. I suppose "non-value added" is in the eyes of the beholder.
In any event, I'd be curious to get your thoughts. Was the Journal article too harsh?
- Jason Busch