An opinion piece published in the Canberra Times by a federal Member of Parliament (MP) has outlined examples of responsible artificial intelligence (AI) in government in an effort to quash concerns about data misuse or unsafe technologies.
The Hon Dr Andrew Leigh MP, Assistant Minister for Competition, Charities and Treasury and Assistant Minister for Employment, said a case study of the Australian Bureau of Statistics (ABS) using generative artificial intelligence to help update the Australian and New Zealand Standard Classification of Occupations, a data set that tracks labour market conditions, shows how the technology can be used safely, reliably and transparently.
Last updated in 2006, the list covers 1,076 occupations from acupuncturists, blacksmiths and cartographers to wool classers, youth workers and zookeepers.
“The Australian Bureau of Statistics looked to artificial intelligence for help to create a preliminary list of tasks undertaken in each occupation. Australian Bureau of Statistics data scientists gave ChatGPT a comprehensive prompt of over 480 words based on the existing publicly available occupational classification. This helped the artificial intelligence to generate its output in the right format and style,” Leigh wrote in the piece.
“Australian Bureau of Statistics analysts spent time testing and refining ‘the prompt’ or the question they ask ChatGPT – so that the machine delivered what the humans needed. After each test, the analysts used a mathematical formula to calculate the quality of the responses within tolerance levels around precision and recall.
“The Australian Bureau of Statistics data scientists iterated the prompt until it consistently produced a high‑quality output. The final prompt ended up optimising the output from ChatGPT to about 69 per cent for both precision and recall.
“The results were of such high quality that in one test, experts themselves were asked to distinguish between the description of occupational tasks written by experts and those written by generative artificial intelligence. Two‑thirds got it wrong.
“The results from ChatGPT were not perfect. But they did provide enough of a starting point for Australian Bureau of Statistics analysts to review and build on. As a result, the project team saved approximately 1,600 hours. It amounted to an approximate seven‑fold return on investment.”
Leigh said a key part of the trial was that the ABS data scientists confirmed AI was used to support, not replace, human analysts. It also ensured all outputs were measured against four criteria: quality, ethics, legality and security.
“This delivers on the Australian Government’s commitment to fostering an innovative culture in the Australian Public Service, while managing the risks of emerging technologies.
“The Australian Bureau of Statistics is also at the forefront of harnessing digital technologies to build high quality, big data sets. The Australian Bureau of Statistics is continuing its work to modernise collection methods to gather high quality and granular household spending data.
“The current revolutions in artificial intelligence and big data in national statistics are not simply good just because they use new, cutting‑edge technologies. They matter because they offer the Australian Government a way of improving our administrative practices, and therefore the way we deliver for all citizens. Artificial intelligence and big data can help to structure and collect data productively, safely and responsibly.
“Big data and generative artificial intelligence offer opportunities across government. The Australian Government remains committed to exploring these technologies within a clear ethical framework. Better data allows us to better serve the nation.”
