The Impact of AI on the Data Analyst

In this special guest feature, Glen Rabie, CEO of Yellowfin, believes that while many analysts may fear they will be replaced by automation and AI, the role of the data analyst will increase in significance to the business and breadth of skills required. Yellowfin is an Analytics and Business Intelligence software company focused on helping businesses understand their data. Rabie is passionate about data and improving business performance through analytics. Prior to starting Yellowfin, he worked in various roles at National Australia Bank including senior e-business consultant and global manager of employee self-service. Rabie holds a Masters in Commerce from the University of Melbourne.

The introduction of AI, automation and data storytelling to the world of analytics has not only had an immediate impact on the end users of analytics but also the people that work in the field. While many analysts may fear they will be replaced by automation and AI, I believe that the role of the data analyst will increase in significance to the business and breadth of skills required.

Data analysts have traditionally spent a significant amount of their time doing mundane and repetitive tasks like preparing data for analysis, creating reports and dashboards then using these to manually search for meaningful changes in their data.  With traditional analytical and business intelligence tools, analysts simply cannot explore every combination or permutation of their data.  And if they do find something of interest, how do they determine if it’s statistically relevant and of meaningful benefit to the business?  The introduction of automated data discovery addresses these issues.  It reduces the time to find insights, subsequently leaving far more time for analysts to add value by interpreting their findings. This will require analysts to become business savvy, (understanding the business, not just the data) and storytellers with improved literacy skills to better communicate their findings.

The role of the data analyst today encompasses a broad range of data management and analysis activities.  These include procuring, preparing, cleansing and modelling data, then creating reports and dashboards through to bespoke analysis for the business to support decision making.  Of all these activities, the true value to the business are those activities that are related to the identification of critical changes or trends that impact the business and the interpretation of that information to determine the possible impact to the business.

The dilemma that business analysts face is that, although interpretation is the most valuable activity that they undertake, it’s the one where they spend the least amount of time.  Most data analysts spend only 20 percent of their time on actual data analysis and 80 percent of their time doing tasks of little business benefit like finding, cleaning, and modelling data, which is highly inefficient and adds little value to the business.

It’s not just data preparation that is inefficient.  The traditional tools for data analysis and visualization require a completely manual approach for data discovery. Users must choose from a large array of fields and filters and then slice and dice data in the search for patterns, changes in trends and anomalies.  This manual process is incredibly time consuming, and highly prone to human error and bias, especially in today’s data rich world.  The result? The identification of critical changes in business data is accidental rather than something that will happen with certainty.  This creates risk for business leaders who want certainty in the data they use for decision making.

AI and automation promise to radically change this paradigm.  Applied to analytics and business intelligence, many of the tedious and time consuming processes will be done by machines.  Smart data preparation that uses machine learning to streamline the data profiling, matching and cleaning processes will significantly reduce the time that analysts spend preparing data for analysis. This in combination with AI driven data discovery, which applies a range of sophisticated algorithms to data, will reduce time consuming data exploration and the discovery of relevant business insights.  

These advances however, do not mean that AI will replace the data analyst. AI is great for automation but it has fundamental limitations. Machines cannot understand context.  Only humans have the capacity to contextualize data in complex terms such as the organizational environment, external market factors, customer dynamics and much more.  For instance, the ability to find meaning in a downwards trend in product sales based on the anecdotal ramp up of marketing by a competitor is far more than AI can process but it is relatively simple for a human to do so.

The result of this shift will see data analysts spending far more time doing what machines can’t – providing context and interpreting data.  Data analysts will be elevated to that of significant business partners, where they will use their data literacy skills to assist the business to interpret the data, contextualize the insights discovered and to tell compelling stories with that data.  The result of this will be that the data analysts of today need to become far more business savvy and build their skills to develop narratives.

This does not mean that repetitive data analyst jobs will not disappear.  For data analysts whose primary focus is the preparation of data and building of dashboards, their time will come sooner rather than later.  However, organizations will rely more heavily on those with the skills to provide insight into what the data means.  Data analysts will rely on AI driven tools that make the mundane aspects of their jobs easier, so that they can spend more time on highly valuable activities such as data interpretation and storytelling.  As a result, they will able to provide meaningful analysis to the business to make better data-driven decisions.

Sign up for the free insideBIGDATA newsletter.