Disconnect, confusion and challenges. Are organisations failing their data management strategies?

“When a chef has an abundance of ingredients in front of him or her, the issue isn’t the quantity of produce that they have, it’s sorting and organising of that produce in such a way that they can produce a tantalising meal.”

The constant creation of data can be as much of a benefit to any business as a headache to manage and handle.

Poor IT systems – which sometimes are down to poor It infrastructure as well – are often behind most company’s lack of ability to handle the data they own appropriately.

Yet, the global enterprise data management space is booming and was in 2016 worth $68.6bn, according to Stratistics MRC.

Growth in this segment is expected to continue to occur at pace, with the market projected to reach $142.67bn by 2023 growing at a CAGR of 11.0%.

João Marques Lima speaks to Rob Mellor, VP & GM EMEA, at data warehouse automation and big data software company WhereScape, about the challenges organisations face when it comes to data and how automation can help.


Can you outline the data challenge facing organisations today?


Rob Mellor, VP & GM EMEA, WhereScape

RM: Organisations don’t have a data challenge, they have a data management challenge.  Think about it this way, when a chef has an abundance of ingredients in front of him or her, the issue isn’t the quantity of produce that they have, it’s sorting and organising of that produce in such a way that they can produce a tantalising meal.

It’s the same for businesses; today, the explosion in data means that organisations don’t want for a supply of data.  If anything, they have too much data (heard the cliché ‘drowning in data?’)  Instead, businesses have to manage their data in such a way that they can create the kind of insights and information flows that allows them to make the right decisions at the right time.

Those organisations that solve this data management challenge the best are those that are able to be agile.  To innovate.  To differentiate.  And, ultimately, to grow their business faster than their competitors; getting data management right today is therefore as fundamental to business success as having a unique product or employing the best people.


Do we still have a disconnect between IT and business?

RM: Yes.  Friction has occurred in the past between business users moving at a significantly faster pace than IT could deliver on.  IT departments are facing increasing pressure to deliver insights that businesses can use to make smarter decisions and the ‘time to value’ is quickly becoming one of the most crucial metric that they are judged on.

Nowhere has this become more apparent than when investing in data analytics capabilities.  Using automation software can alleviate this friction by eliminating manual tasks and instead focusing on the high value work that turns IT requestors into IT advocates.


Do companies fully understand the impact of the data explosion and are they clear on how to derive value from it?

RM: No. There is a lot of confusion out there!  But to my earlier point, it’s a data management issue.  When I talk to customers about the explosion in forms of data, the common phrase I hear is ‘we’re drinking from the firehose!’

It’s no surprise given we’ve moved from the world’s annual collection of data being 2700 exabytes in 2012 to a projected 40,000 exabytes in 2020.  Businesses absolutely understand the fact that the deluge of data is already overwhelming and they know it is only set to increase.

They also understand that they need to try and manage that data better because there is significant value to be gained from it. But that’s where the real challenge lies: the ‘how’ to derive value is the thing that has senior executives in organisations scratching their heads.

And the answer?  Get expert advice from those organisations who live and breathe extracting value from data, because that is the best and least painful path to value.

How does automation of analytical data infrastructure work?

RM: Automation software helps businesses to automate the gathering of data intelligently, allowing them to dramatically speed up the time it takes to drive value from it.

Automating the process of data gathering can drive real business value and provide a flexible, templated approach to automation, personalised for every business requirement.

In fact, the more you interrogate the software, the more personalised and efficient the templates become and the more accurate and valuable the output.


Is there a particular type of organisation for whom automation does or doesn’t work?

RM: No, that’s the beauty of it!  Every company that is built on data – and let’s face it, show me a company that doesn’t rely on data for all that it does – would benefit from automation of data infrastructure, no matter how big or small, or in what sector they operate.


How does automation play into organisations developing a wider AI/Machine Learning strategy?

RM: Both AI & machine learning require data as input.  You have to think of them as another type of consumer, rather like you would a business user of a report.

You need data foundations to drive these advanced methods and automation can help speed up the delivery of data to these AI/ML components.

In addition, as the AI/machine learning consumers’ needs evolve, automation can help companies make adjustments more rapidly.


What about real time data?  How can you automate that?

RM: Yes, even real time data analytics can be automated. Analysing streaming data—think Internet of Things (IoT), sensor data, log files, retail purchases, social media content, or any other continuously generated data—can give your business unprecedented visibility into trends, trouble spots and opportunities as they emerge.

Real-time data is also the fuel that drives better outcomes from applications such as fraud detection, cybersecurity, machine learning, supply chain optimization, and more.  So it’s important you get this information quickly.


What is the challenge here, especially for IT?

RM: The challenge for IT teams is how to incorporate these new data sources and streaming technologies into existing analytic environments, making them quickly and easily accessible to the business.

When automation supports streaming data, it minimises the learning curve for IT teams and reduces the complexity of managing a hybrid of streaming and traditional batch-based data. With automation, you can deliver real-time data at the speed of the business.