You are here

The war on analytics talent & BICC project backlog: can data estate automation be the solution?

The war on talent is affecting most industries, but none more than the IT field. It is increasingly difficult to find quality data professionals and to hold onto these talented individuals. What is worse, is that most of these professionals are spending their time and valuable expertise on hand-coding and meticulously managing data systems.

In order to win this war on talent, companies need to not only provide their employees with the latest digital tools, but also allow them to use their talent for purposeful growth, innovation, and game-changing breakthroughs, rather than tedious coding and data management.

Data is no longer a byproduct or just an accidental creation of business processes or the storage of information that is required by regulatory authorities. The term data estate reframes the narrative. Think of data as a raw resource. Data is the new oil and data science the refinery. Enterprises – large and small – need to handle their data with the respect it deserves. Business leaders understand that data is the potential for massive business transformation and profits. They manage their data effectively, entrust it to good stewards and grow it.

Analyst research reveals that the amount of data in the world is actually doubling every two years. For many organizations this goes even faster. As data increases it becomes more complex and increasingly more difficult to access, govern, and maintain compliance.

So, companies are looking into solving their current pains, but what about in a few years? How will they keep up with their data? They want to make sure that the solution is not only able to help their current organization but also prepares for the future.

Self-service

On the one side, companies have their ever-growing data sources. On the other side, there are the business analytics instruments at their disposal: dashboards, reports, predictive tools, etc.

 

The war on analytics talent & BICC project backlog: can data estate automation be the solution?
click to enlargeThe war on analytics talent & BICC project backlog: can data estate automation be the solution?

Figure 1 – Self-service approach to BI

Some companies begin connecting these tools directly to the data sources, either using connectors within the tool, or writing scripts to extract the data. In taking this approach each analytical tool has its own data pipeline and set of transformations. While you get quick access to your data, it becomes increasingly difficult to manage. Data silos begin to develop with limited control of data quality and security.

Then, when a business user needs more data or new data, it is common to encounter an extended delay to address connectivity to source systems and data infrastructure issues.

Patchwork of tools

Many organizations begin to see the shortcomings of the previous approach and attempt to simplify matters. They may implement a staging layer to help simplify connections and improve security.

The war on analytics talent & BICC project backlog: can data estate automation be the solution?
click to enlargeThe war on analytics talent & BICC project backlog: can data estate automation be the solution?

Figure 2 – Patchwork of tools

Then when they encounter specific difficulties or pains, they try and implement a tool to treat each symptom. A tool for extracting data, a tool for transforming, a tool for modeling, a tool for scheduling and many others. While this approach may make individual problems easier to handle, it ends up only making the overall implementation more difficult to manage. Each tool requires an expert to maintain it and none of these tools talk to one another, so you need to orchestrate the entire operation. In addition, maintaining compliant documentation for the solution often requires the same number of hours as it does to implement. This creates a serious draw on resources and results in a backlog of IT requests and delayed access to data.

Data estate automation

Companies do not want to simply treat the symptoms. They want to address the underlying condition. To run an effective business, users can’t wait days or weeks for new data. And not all user’s needs are the same, power users need raw data, business users need governed data, and casual users need pre-build data models. Companies looking for a platform that is specifically designed to build and maintain this type of data discovery architecture should consider to deploy their data estate to Microsoft Azure and enable instant access to data with timeXtender’s Discovery Hub. Because this data architecture persists in the users’ environment, it provides the users with instant access to data enabling immediate analytics. The software is able to bridge the gap between business and IT by transforming IT from a gatekeeper to a shopkeeper.