Gartner, why the confusion?

Earlier this year Gartner released their Magic Quadrant and Critical Capabilities for Analytics.

It’s always an honor to be included in these prestigious documents. However, while Gartner’s analysis is highly regarded, the two documents continue to cause a lot of confusion in the industry and amongst buyers.

The two documents are very different

As they’re positioned today, the Magic Quadrant and the Critical Capabilities are two very different documents.

From my perspective, the most important difference is the Magic Quadrant rates the organization, not the technology. Gartner looks at a lot of variables to determine how to rate an organization like whether it is global -  an organization must pass this hurdle to be featured. Other things, like social media following, are also considered (although I’m still not sure how important Twitter followers are for the buyer of a technology solution).

While an understanding of the vendor is helpful, it’s important for customers to understand the technology - that’s why organizations need to look beyond the Magic Quadrant. For many people, the Critical Capabilities is actually a much better framework to use to bring a shortlist together. Vendors have already passed Gartner’s organizational viability threshold set for the Magic Quadrant. So this document looks purely at the technology and helps people answer the question “Is this the right solution for me?”

The document breaks down the technical capabilities of a modern BI platform into 15 key critical capabilities - like automating analytics, data transformation, and dashboards - and then maps and weights those against five specific use cases. The five use cases are:

• Agile, centralized BI provisioning
• Decentralized analytics
• Governed data discovery
• OEM or embedded BI
• Extranet deployment

These are designed to help organizations determine if a solution will help them achieve their objectives. The individual capabilities then sit below and are weighted for each individual use case. These critical capabilities are also closely aligned with the Forrester report that has a criteria-based view of who the best vendors in the space are.

For us at Yellowfin, while we sit in the “Niche” quadrant, we were rated in the top five for all 15 of the critical capabilities. I struggle to explain this contradiction to customers, and I don’t think Gartner has a better answer.  

The reality is that most organizations work across multiple use cases, not a single one. While a customer may initially think they have one specific use case when they are searching for a BI platform, it’s common for them to discover that they actually have more use cases downstream. This can constrain the organization if they have a platform that is specific to just one capability.

Gartner needs to merge these two documents

Ostensibly, the Magic Quadrant and Critical Capabilities documents are meant to do the same thing - provide Gartner’s customers with a mechanism to determine which products are the most appropriate for them. The problem is that they seem to be saying very different things about the same set of vendors. This is causing confusion amongst buyers and in the industry.

Some industry commentators have called out this disconnect - for example, Jen Underwood has tried to reconcile the two documents in her analysis here. My personal view is it will be difficult for Gartner to continue to maintain these two very different documents moving forward. Ideally, the two pieces of analysis should be merged together into one concise document that anyone can pick up and use to inform their BI buying choices.