Big Data Analytics: The need for pragmatism, tangible benefits and real-world case studies

You can feel it in your fingers. You can feel it in your toes. Big Data is all around you, and so the feeling grows…

*insert deep sigh and swoon here*

But, unlike love, Big Data can be tangibly harnessed, through reporting and analytics, to solve real challenges.

Whilst many media outlets, industry authorities and technology evangelists have created a Big Data love-in, its real benefits can only be realized when the fluffy clouds of unchecked adoration are brushed aside, and the self-perpetuating cycle of purposeless idolization halted.

Because when pursuing Big Data programs, divergent from love, getting swept-up in hype isn’t the least bit helpful. In fact, it distracts you from its potential power.

Unabashed Big Data fanfare leads to confusion, stagnation

There’s a mounting body of research that indicates the potential damage being caused by Big Data hype.

Gartner’s 2013 Hype Cycle for Business Intelligence and Analytics suggests that in 2014, the concept (and technological term) ‘Big Data’ will be at the very peak of ‘inflated expectations’. By comparison, dashboards – a tried, true and tested Business intelligence (BI) mainstay – sit comfortably atop the highest arc on the ‘plateau of productivity’.

While it’s accurate that Big Data Analytics has the capacity to reshape entire industries, the rampant silver-bullet-like discussions that abound the industry do nothing to assist organizations to actually leverage their bourgeoning datasets for competitive advantage.

Recent Gartner research suggests that extensive confusion among potential technology purchases and implementers – regarding the definitions and differences between the terms ‘Big Data’, ‘BI’ and ‘analytics’ – is “blunting BI spend”. Principal Research Analyst at Gartner, Dan Sommer, said in an associated media statement that “confusion related to emerging technology terms causing a hold on purse strings”.

But, Big Data hype isn’t just stalling purchase decisions, it’s also inhibiting deployment and ROI. Gartner’s research report – Big Data Adoption in 2013 Shows Substance Behind the Hype – found that while 64 percent of enterprises have Big Data investment plans, 57 percent are still grappling with “how to get value from Big Data”. Similarly, a 2013 Forrester Research report, based on a survey of 259 business development executives, reported that one-third of respondents found the term Big Data “very confusing”.

Much of this inertia, according to industry experts, is a result of massively overblown expectations (as a result of sustained industry hype) that leave organizations attempting to reach for the moon, whilst ignoring current operations and strategies to which Big Data Analytics can be successfully applied.

Big Data growth

Speaking of the moon, we know that Big Data (the volume, variety and velocity of data) is growing, as existing data sources and types mushroom, and new ones proliferate. So there’s an obvious need for technology capable of capturing, collating, exploring, analyzing and communicating that bourgeoning information. Technology; which can enable people and organizations to derive actionable insights that lead to faster, smarter decision-making.

To illustrate the scale of this growth, International Data Corporation (IDC) estimated that there was enough global data in 2009 to fill a stack of DVDs stretching to the moon and back.

They predict the volume of information, as of 2009, to grow 44X over by 2020. This analogy is good for encapsulating the Big Data challenge. But it’s clear that many organizations – regardless of being overawed or overexcited – view Big Data as an opportunity, rather than a challenge. Actually, according to TDWI research, 89% of organizations see Big Data as a “significant business opportunity”. And that mentality is reflected in current investment figures and projections.

Big Data investment

Ninety-one percent of respondents to New Vantage Partners’ 2013 Big Data Executive Survey of Fortune 1000 companies said they already had a Big Data initiative planned or in progress. Of these, 60 percent of executives reported that at least one Big Data initiative had already been implemented. Fifty percent of respondents also indicated that they planned to spend more than $10 million on Big Data investments by 2016.

While varying from the findings of New Vantage Partners, results from QuinStreet Enterprise’s survey also underscore the growing emphasis on Big Data technology. The survey interviewed 540 decision-makers involved in Big Data purchase decisions. The report, 2014 Big Data Outlook: Big Data Is Transformative – Where Is Your Company?, found that 77 percent of respondents considered Big Data Analytics a priority for their organization in 2014. The survey also demonstrated that Big Data projects aren’t the exclusive domain of the Fortune 1000, with 71 percent of mid-sized and large organizations having plans for, or currently implementing, Big Data initiatives throughout 2014. Backing these findings is a 2014 Competitive Edge research report, based on a global survey of mid-market executives. In this survey, 41 percent of respondents said that they had already implemented one of more Big Data projects, with a further 55 percent stating that they planned to kick-off a Big Data deployment in “the foreseeable future”.

ABI Research found total Big Data spend exceeded $31bn in 2013, forecasting it to reach $114bn in 2018 in their Unlocking the Value of Big Data in Enterprises study. To put that in perspective, IDC predicts the Big Data technology and services market will enjoy a 27 percent compound annual growth rate (CAGR) to $32.4 billion through 2017. That’s roughly six times the growth rate of the overall ICT market.

Wikibon’s Big Data Vendor Revenue and Market Forecast 2012 – 2017 report claims that the 2012 Big Data market reached at total size of $11.4 billion, climbing to $18.1 billion in 2013 – a staggering annual growth rate of 61 percent. If sustained, the global Big Data market would exceed $47 billion by 2017, amounting to a CAGR of 31 percent across 2012 – 2017.

Similarly, the research firm MarketsandMarkets predicts that the global Big Data market will climb from a more conservative estimate of $14.87 billion in 2013, to $46.34 billion by 2018. The report said that one of the major forces driving investment was the demand for advanced and predictive analytics.

Investment in Hadoop leading the charge, but…

The Wikibon report attributes much of the explosive growth in Big Data investment to two key factors: A better understanding of Big Data use cases leading to definitive ROI, in combination with the maturation of a range of Big Data specific products and services – in particular capabilities relating to Hadoop, and associated Big Data technologies such as massively parallel processing analytic databases.

In fact, IDC estimates the revenue of Big Data companies offering Hadoop-oriented solutions to grow tenfold by 2016. But, while IDC’s 2014 study Move Over Hadoop: Emerging Database Technologies for Tackling Big Data Problems does predict that “the dominant Big Data technology will continue to be Hadoop and its tributaries”, it does acknowledge that Hadoop is designed for data storage. Therefore, organization will have to also consider other database and analytic technology to leverage Hadoop-based data for reporting and analysis purposes.

"Big Data technologies offer great benefits to businesses by enabling them to tap the value inherent in various kinds of data that could never be leveraged before in an affordable way," said Carl Olofson, Research Vice President for Database Management and Data Integration Software research at IDC. "Anyone investigating Big Data technology for the enterprise must first identify the type of data to be managed and the problems to be addressed."

These findings were inline with those reported in the 2014 Big Data Outlook: Big Data Is Transformative – Where Is Your Company?, which found 44 percent of respondents would use analytic databases to underpin their Big Data Analytics initiative(s), with 55 percent also stating that BI tools would be a top technological priority.

Benefits of Big Data Analytics

Despite inflated expectations, and some short-term resultant stagnation, the business benefits of Big Data Analytics are beginning to be documented.

Tom Davenport and Jill Dyche’s investigative study Big Data in Big Companies, as well as research from Harris Interactive, has found the top benefits of Big Data Analytics to be:

• Increasing sales of existing products or services
• Boosting revenue by using new information to create new products or services
• Attracting and retaining more customers
• Increasing business agility and responsiveness
• Achieving more efficient business operations
• Better internal decision-making

These findings were backed by similar results in Gartner’s research report, Big Data Adoption in 2013 Shows Substance Behind Hype, which disclosed the most common Big Data related business goals were to “improve process efficiency” and “customer experience”. In the 2013 survey, 55 percent of organizations said that they are currently addressing enhanced customer experience using Big Data, while 49 percent are using Big Data to address process efficiency.

In QuinStreet Enterprise’s report, 2014 Big Data Outlook: Big Data Is Transformative – Where Is Your Company?, 72 percent of respondents cited that enhancing the speed and accuracy of business decisions was a top benefit of Big Data Analytics. Other organizational benefits at the top of the list included providing real-time access to data (65%), more timely integration of data (63%), quicker determination when changes needed to be made (63%), and automated decision-making (61%).

According to recent research conducted by Avanade, 84 percent of respondents actively leveraging Big Data say they can now make better decisions. Seventy-three percent said that they had used their Big Data programs to increase revenue via established revenue streams (57%) or create completely new ones (43%).

Eighty-nine percent of respondents to the abovementioned 2014 Competitive Edge research report, who stated they already had a Big Data project in place, said they had experienced “significant improvements in company decision-making”. Respondents listed the top three goals of their Big Data projects as the ability to provide “better quality products and services”, “take advantage of new business opportunities”, and “improve the quality and speed of decision-making”. The report also compared organizations with and without Big Data implementations, and discovered that:

• Fifty percent of organizations with a Big Data implementation were satisfied with the quality and speed of their decision-making, compared to 23 percent of organizations without a Big Data deployment
• Forty-nine percent of organizations with a Big Data implementation were satisfied with their ability to improve product quality, compared to 32 percent of organizations without a Big Data deployment
• Forty-seven percent of organizations with a Big Data implementation were satisfied with their ability to identify and take advantage of new business opportunities, compared to just 24 percent of organizations without a Big Data deployment

Perhaps most tellingly, respondents to the Competitive Edge research report identified data visualization as one of the most important technologies for extracting value from their Big Data project.

Big Data: Unlocking value is about analysis and sharing, not collection

Data visualization and analysis capabilities give organizations the means to explore, analyze and share insights attained via Big Data. Simply sitting on mountainous volumes of information does not lead to Big Data ROI.

But, a recent MIT Sloan Management Review survey found that 60 percent of organizations collect more data than they can effectively use. In fact, research from McKenzie & Company suggests that 88 percent of the data captured has never even been accessed.

So, how do you exploit that data growth, to develop a deeper understating of your business and your customers? According to TDWI analyst and industry commentator, Philip Russom – as written in his 9 Ways to Get Business Value from Big Data article – “the primary path to business value is through analytics”. Russom emphasizes that Big Data ROI cannot be attained from simply capturing information. Rather, that data must be able to be explored, analyzed and the results of that analysis shared with decision-makers to enable them to take appropriate action.

Likewise, New Vantage Partner’s 2013 Big Data Executive Survey revealed that respondents considered the biggest factor in ensuring the success of Big Data projects was the ability to make better, faster fact-based decisions. Survey participants said their organization would realize this ability by accelerating the speed at which they could gain insight, and answer critical business questions, as a result of delivering timely, understandable and actionable reporting and analytics. “The ability to Accelerate Time-to-Answer (TTA) is a metric for business success” stated the report.

However, unlocking the value of Big Data requires more than simply applying basic reporting and analytics. You also need to make that information as easily shareable and accessible to business decision-makers as possible – because, after all, there’s no ROI for BI or Big Data unless a decision is made. Seventy-four percent of respondents to the aforementioned QuinStreet Enterprise report, titled 2014 Big Data Outlook: Big Data Is Transformative – Where Is Your Company?, said that they hoped their Big Data programs would “make the information more transparent and more usable quickly”.

Highlighting this point is IDC’s Collaboration in the Age of Big Data study, which presents findings from its 2013 Big Data and Analytics Benchmark Survey. The report found a distinct link between the levels of collaboration between IT, analytics groups and line-of-business employees, and the ultimate success of Big Data and analytics projects.

"Many line of business and analytics groups pride themselves on their ability to bypass IT to fulfill their own needs for data access and analysis,” said Dan Vesset, Program Vice President of Big Data and Analytics at IDC. “But there is a fine line between data and analytics democracy and anarchy.”

Vesset’s point is that, with the right Collaborative BI capabilities, decision-makers are able to overlay and complement data exploration and analysis with appropriately experienced human knowledge, enabling fast and accurate decision-making without plunging their BI environment into chaos.

2014 research conducted by analyst firm Ventana Research found the top benefits sought by organizations deploying Big Data Analytics to be “faster response to opportunities and threats” (57%), “improving efficiency” (57%), “improving the customer experience” (48%) and “gaining competitive advantage” (43%). However, what was interesting about this research – as reported in the blog Big Data Analytics Research Reveals Benefits of Investment on information-management.com – was that post production, the number one benefit sought or attained was “better communication and knowledge sharing” (51%).

So, we now know that making the insights attained via Big Data Analytics highly accessible, whenever and wherever required, is critical. But it’s not just the ability to analyze data, and share any resultant insights which is important. Anywhere, anytime access is imperative too.

According to a recent report, based on a survey of 154 senior US executives from international companies across a range of industries, the ability to deliver the results of Big Data Analytics to mobile devices, to boost timely decision-making, is crucial. The survey, conducted by Harris Interactive from April 10 – 23 2013, asked participants to describe the importance of instant access to Big Data from their mobile devices (Mobile BI). Ninety percent of respondents said it was “somewhat important”, with 57 percent stating that it was “absolutely essential” or “very important”.

So what have we learnt?

We know that Big Data has serious potential across most major industries, but that unrestrained hype is inhibiting people’s understanding of what’s possible. Despite this confusion, Big Data investment is on the rise as organizations realize the potential power of capturing expanding existing, and multiplying new, data sources and types. And, while Hadoop has opened up new possibilities for Big Data storage, new reporting and analytics technologies are needed to unlock Big Data’s potential value.

But, despite the fact that the benefits of Big Data Analytics are becoming more widely researched and publicized, there remains a lack of clear, specific and tangible use cases. As a result, many organizations are still struggling with how to best leverage and apply Big Data Analytics to their own operations and strategies.

If this is you, why not join us to explore a Big Data Analytics with Hadoop use case that has transformed the business operations of a telecommunications provider?

Register for this exclusive Webinar event – Big Data Analytics with Hadoop – to find out how, as we go from data in Hadoop to real-time reporting, to predictive analytics, in just 40-minutes.

Discover how to combine Hadoop with the most advanced Big Data technologies, and world’s easiest BI solution, to quickly generate real business value from Big Data Analytics.   REGISTER NOW >