Demystifying the Big Data hype


Big Data is a topic that has a lot of hype riding behind it, but it may be little more than a vendor-driven initiative, as most banks are already working on what is being defined as Big Data, according to analyst firm, Gartner.

Ian Bertram, Gartner’s Managing Vice President, Analytics, BI and Performance Management Research Team, said the Big Data story is one largely manufactured by vendors, and indeed many Australian and New Zealand financial institutions are already doing what is being called Big Data.

“I would argue that many financial institutions have been dealing with Big Data problems for a number of years, and now they’ve just got a name for it,” Bertram said. “There’s a lot of hype around Big Data, and yes in the Hadoops of the world there are some technologies around that will help people manage that data.”

Gartner does field a lot of questions around Big Data, Bertram added. However, often those questions come from customers who have driven by the vendor message that organisations need to be prepared to handle Big Data. “We’ll ask them some fundamental questions around social media and the like and we’ll find they’re doing Big Data already, perhaps in a rudimentary form but it’s something to build on,” he said.

Indeed, according to Suncorp Executive Manager, Business Intelligence, Jason Catania, Australian organisations may not even want to invest in the technologies that vendors are pushing out as Big Data solutions. Often they are not in an ideal position to shift investment away from data warehousing (“Small Data”), and the scale of the local market also means it’s an unnecessary consideration at this stage.

“This is not one of those things where you can say “Big Data is better than warehousing, or warehousing is better than Big Data,” Catania said. “It’s a choice. For 20 years we’ve been going around a warehousing approach and for those companies that made a lot of investment into streamlining that approach there’s not a lot to be saved changing course to a Big Data approach right now.”

However, this is not the case globally, Catania added. In the US and Asia, financial institutions are dealing with far greater volumes of data, and this is forcing them to turn to Big Data solutions from a cost perspective. As the quantity of unstructured data increases, as do the size of the teams required to manage the data using a data warehousing solution. However, according to Catania, Australian and New Zealand banks have not hit that tipping point just yet.

“There’s that point where the cost of technology will be less than the cost of people. But I think waiting is good in Australia,” he said. “The technology is a little bit immature and we are privileged that we don’t have to be the first ones, as in many cases the early adopters are the sweepers for technology. One of the other problems we are facing at the moment is that there isn’t really one meaning that everyone aligns to. A lot of Big Data messaging is vendor driven, which shows its immaturity at the moment.”

Gartner defines Big Data in terms of three ‘V’s: Volume, Velocity and Variety, which is different from a common definition of the issue that exclusively focuses on volume. Until the industry can arrive at a unified definition around Big Data it will be a hard sell, but the core issues around unstructured data is an area that Bertram sees a lot of innovation moving forward.

For now, the best approach in the Australian and New Zealand market might be for financial institutions to use data warehousing around structured data to inform the development of an approach to unstructured data.

“When we go into an organisation and look into it around structured data we see teams in place with best practices and well defined processes,” Bertram said. “Then we look at the content side of things and all the other information inside the organisation that’s not structured. What type of organisational model do they have? Quite often it’s all over the place, so why not bring the structured and content parts of the organisation together to learn off each other?”


Click to view FST Media’s Business Intelligence Melbourne or Sydney agendas.