UniSuper systems back online after ‘unprecedented’ outage

Google Cloud, UniSuper

UniSuper has confirmed that it has restored the majority of its member-facing services after a more than week-long unscheduled outage prevented members from accessing or modifying their accounts.

The super fund has been progressively bringing its system back online since 9 May, with its cloud provider, Google Cloud, taking the blame for the prolonged outage, admitting it had inadvertently deleted UniSuper’s private cloud subscription.

The outage, reportedly affecting members from the evening of 1 May, prevented members from accessing their online accounts, altering their investment allocations, viewing their balances and investment performance, using on-site super calculators, and making withdrawal requests.

As of 13 May, UniSuper confirmed that, bar its retirement calculators, these systems are now all back online and accessible to members.

In an email to members last week, the superannuation provider confirmed that the outage was related to an error made by its cloud provider Google Cloud, which since mid-2023 has hosted key UniSuper services following its migration from on-premise, VMware-supported infrastructure to private and public clouds.

The super fund was reported to have lost its disaster recovery backup, preventing timely recovery, after Google inadvertently deleted UniSuper’s infrastructure subscription.

In a joint statement issued by UniSuper and Google Cloud on 7 May, the pair disclosed that disruption “was caused by a combination of rare issues at Google Cloud that resulted in an inadvertent misconfiguration during the provisioning of UniSuper’s Private Cloud, which triggered a previously unknown software bug that impacted UniSuper’s systems”.

The pair added that the incident was “an isolated, ‘one-of-a-kind occurrence’ that has never before occurred with any of Google Cloud’s clients globally”; Google further stated that it had “taken measures to ensure this does not happen again”.

“This was an unprecedented occurrence, and measures have been taken to ensure this issue does not happen again.”

The issue, Google hastened to add, was “not the result of a malicious behaviour or cyber-attack, and that no UniSuper data has been exposed to unauthorised parties”.

As part of its disaster recovery (DR) measures, the pair confirmed that UniSuper had duplication across two geographies as a fallback against outages and data loss.

However, the deletion of UniSuper’s Private Cloud subscription effectively resulted in the deletion of its archives across both geographies.

While UniSuper has openly pursued a multi-cloud strategy, it did not appear in this case to have a full and ready-to-go external backup for DR. This has resulted in a slower-than-desired recovery process, and the likelihood of data being lost.

UniSuper noted in its statement: “UniSuper had backups in place with an additional service provider. These backups have minimised data loss, and significantly improved the ability of UniSuper and Google Cloud to complete the restoration.”

It added: “Restoring UniSuper’s Private Cloud instance has called for an incredible amount of focus, effort, and partnership between our teams to enable an extensive recovery of all the core systems.

“The dedication and collaboration between UniSuper and Google Cloud has led to an extensive recovery of our Private Cloud which includes hundreds of virtual machines, databases and applications.”

Owned by Australia’s 37 universities, UniSuper is one of Australia’s largest industry super funds, counting more than 615,000 members (primarily from the tertiary education sector) and $124 billion in funds under management as of June 2023.