Deloitte’s AI-Generated Errors Lead to Refund
Deloitte Australia has agreed to make a partial refund to the government after providing a report riddled with errors. Artificial intelligence generated some of the content. The Department of Employment and Workplace Relations (DEWR) commissioned the report to identify problems in the welfare system. But the paper contained fabricated academic references. As well as misquoting a federal court ruling, triggering public outcry and revisions to the paper.
The Role of AI in the Report
The original report was a lengthy 237-page document. It was designed to review the Targeted Compliance Framework. It is a system that penalises welfare recipients. A University of Sydney researcher specialising in health and welfare law, Chris Rudge, identified the errors in the report. These included invented academic references and a fabricated quote falsely attributed to a federal court decision. Deloitte subsequently reviewed the report, confirmed the inaccuracies. They said some of the errors were caused by the use of generative AI tools. Specifically, Azure OpenAI GPT-4o. The firm admitted that the AI-generated material was a factor in the mistakes, a phenomenon described by the term “hallucination” in AI systems.
Government’s Response and Refund Agreement
After identifying these errors, DEWR released an updated version of the report that corrected the errors. It also acknowledges using AI in the report’s preparation. The department clarified that the revisions did not affect the report’s fundamental findings or recommendations. Due to the errors and the use of AI in producing sections of the report, Deloitte also consented to returning the final payment of its contract fee. This was AUD 440,000 (USD 290,000). They will make the exact refund amount public after completing the transaction.
AI Adoption of Deloitte
Undeterred by this setback, Deloitte continues to invest in AI. The company has also partnered with Anthropic to bring the Claude chatbot to nearly 500,000 Deloitte employees worldwide. The move further highlights Deloitte’s drive to embed AI tooling to streamline and boost productivity within its offices worldwide.
Future Outlook for AI Strategy of Deloitte
Deloitte’s partial refund to the Australian government illustrates the perils and challenges in using artificial intelligence to produce official reports. AI can increase productivity. But this incident highlights the need for strict human oversight to confirm the accuracy and reliability of information. Especially when it is used to make decisions on public policy and governance. In the future, it will be important for companies to find a way to respect technological progress. While keeping traditional options available in order to preserve the quality of their results.
About The Author
