ETL Hinders Real-Time Analytics
August 7, 2017
A study has revealed that organizations using extract, transfer and load (ETL) and Changed Data Capture (CDC) technologies are struggling to keep up with today’s demand for real-time data analysis, negatively affecting business opportunities and efficiency.
The study highlighted the growing need for reliable real-time data analytics in today’s enterprise organizations, with more than 75 percent of respondents believing that untimely data has inhibited business opportunities and 27 percent indicating it has negatively affected productivity/agility. Untimely data is also slowing the pace of business, with 54 percent of respondents claiming it limits operational efficiency.
Looking at where data is languishing, the study revealed that nearly two-thirds of data moved via ETL was at least five days old by the time it reached an analytics database. When it comes to CDC, a real-time data replication technology, the survey revealed that on average it takes 10 minutes or more to move 65 percent of CDC data into an analytics database. This proves to be problematic for most organizations, as making decisions based on live data requires the ability to blend analytical queries into transactions in real-time.
The study also discovered that organizations consider all new data types important, making it more alarming that they are not finding their way to analytical databases where they can more immediately impact business. In fact, relational, IoT data, streaming data from external sources, sensor data, graphs, key value, video/audio/image, object, JSON documents and geospatial data, were all listed as “very important” by the 502 organizations surveyed across Australia, Brazil, China, Germany, Japan, UK and the United States.
“As organizations look to compete and accelerate innovation, this study highlights the importance of concurrent transaction processing and real-time data analytics for improving customer experience, business productivity, operations and more,” said Paul Grabscheid, Vice President, InterSystems. “InterSystems works directly with organizations across the globe to reduce data management complexity, enable real-time data analysis and insights at the time critical decisions are made, speeding innovation and driving improved outcomes across the entire enterprise.”