NSW builds data lake to navigate information streams

12

A ‘crystal clear’ data lake is the next big thing for NSW, says the minister for innovation, Victor Dominello, while foreshadowing a launch in the New Year.

New South Wales is fast-tracking plans to build a data lake under moves to navigate information streams across government and industry.

Minister for innovation, Victor Dominello, told FST Government the plumbing is already in place. This data lake, to start filling up in the New Year, will be administered by the Data Analytics Centre (DAC) under the CEO, Dr Ian Oppermann.

Minister Dominello, expected to present next year at the Annual FST Government NSW Conference, said the data-sharing foundations have already been built.

“The job now is to start populating the lake. The DAC was in many ways our stake in the ground. We’re changing gears to embrace the digital and data-centric age.”

Increasingly more data streams will flow into the lake. This enables the centre to deliver on its programs. “I can safely say now that DAC is over-subscribed, rather than looking for work,” noted Dominello.

The blueprint is around the quality of information. Data lakes store both the structured and unstructured data, while improving the reporting, visualisation, analytics and machine-learning capabilities.

Crystal-clear waters

Initial work involved building a prototype. “We need quality data to gain value from this lake,” noted Dominello. “While all data streams flow into the lake, this is about access to pure and crystal-clear water.”

Dominello said he wants to change the narrative “beyond the pipes to what we can do with the water. This is about the insights we can get, and how these insights streamline services.”

Consultations have involved the NSW GCIO, Damon Rees, together with feedback from industry and government stakeholders.

Smart and nimble

Use of big data needs to be smart, nimble, and adapt to consumers. “We now have FuelCheck where we get real-time information from every petrol station in NSW. “A great regulator of the market is competition. This is where you want to reduce prices, improve services, and open up the competition.”

The FuelCheck website went live in August 2016. This service was developed to relay accurate and reliable information in real-time about fuel prices.

“This is about boosting competition and transparency in the fuel sector,” said Dominello. This data is available on any device connected to the internet, including smartphones, tablets, or desktops.

So far, 2,149 petrol stations have connected to the FuelCheck network. “Consumers can track what the prices are, and where they get the best deal. This app demonstrates the tracking power of the marketplace.”

On the money

Similar apps are planned for certifiers across building and construction, as well as green slips. “For construction, we now use something as blunt as the crane-meter and count how many cranes you have up in the air.

“That’s how slow it is. I don’t want to do crane-meters. I want to get to the point where we start to be predictive with our modelling, and that’s when we start to become seriously smart.”

Building consumer apps around quality data does not involve big ICT spending programs, noted Dominello. “With FuelCheck, and other apps, we’re not talking billions of dollars. If we had to ask a big multinational to do this, it would’ve cost us a fortune. There’re a whole lot of designers at university that can do this stuff, once the legislation is in place. This is just about using data in clever and more innovative ways.”