HIGHLIGHTS

AI in Court Without a Lawyer: The Hidden Costs of Self-Representation

Introduction

• Generative Artificial Intelligence (AI) tools are increasingly being used by self-represented litigants as an alternative to legal counsel.
• While AI can make legal information more accessible, courts across several jurisdictions have warned that unverified reliance on AI can seriously damage a case.
• The growing use of AI in litigation has raised concerns about fabricated citations, inaccurate legal advice, and violations of professional and procedural obligations.

Why Are Litigants Turning to AI?

1. Rising Legal Costs

• Many individuals cannot afford professional legal representation.
• Limited access to legal aid and public funding has encouraged litigants to seek assistance from AI tools.

2. Easy Accessibility

• AI platforms provide instant responses and draft legal documents at little or no cost.
• Their ability to produce confident and persuasive language often creates the impression of legal expertise.

3. Growing Adoption

• Legal professionals, litigants, and expert witnesses are increasingly experimenting with AI tools.
• Research has identified more than 84 reported cases involving AI use in Australian courts since the launch of ChatGPT in late 2022.

The Hallucination Problem: When AI Invents the Law

4. What Are AI Hallucinations?

• AI hallucinations occur when a system generates false information that appears credible.
• In legal contexts, this may include:
• Non-existent cases,
• Fabricated citations,
• False quotations,
• Incorrect legal principles.

5. Judicial Examples

• In Luck v Secretary, Services Australia [2025] FCAFC 26, a self-represented litigant relied on a completely fictitious case.
• In DPP v GR [2025] VSC 490, AI-generated submissions contained authorities that could not be located.
• A California attorney was fined USD 10,000 after filing a brief containing numerous fabricated cases generated by ChatGPT.
• Several courts in Australia and the United States have sanctioned lawyers and litigants for relying on non-existent AI-generated authorities.

Consequences of Using Unverified AI Content

6. Breach of Duty to the Court

• Parties appearing before a court must ensure that all legal authorities and factual statements are accurate.
• This obligation applies equally to lawyers and self-represented litigants.

7. Possible Sanctions

Courts may impose:

• Adverse costs orders,
• Financial penalties,
• Striking out of pleadings,
• Professional disciplinary proceedings,
• Adverse findings affecting the outcome of the case.

8. Loss of Credibility

• AI-generated errors can undermine a litigant's credibility.
• Judges have expressed concern about litigants using AI-generated responses during hearings, affecting perceptions of honesty and reliability.

How Courts Are Responding

9. Development of AI Guidelines

• Courts have begun issuing formal guidance on the responsible use of AI.
• These guidelines emphasise:
• Independent verification,
• Disclosure obligations,
• Confidentiality concerns,
• Risks of hallucinated content.

10. Regulatory Developments

• The Supreme Court of Victoria and Queensland courts have issued AI-use guidelines.
• The Victorian Law Reform Commission has conducted a dedicated inquiry into AI in courts and tribunals.
• Several jurisdictions are considering mandatory disclosure requirements when AI is used in legal filings.

What AI Can and Cannot Do

11. Useful Functions of AI

AI may assist with:

• Explaining legal concepts in plain language,
• Organising facts chronologically,
• Drafting correspondence,
• Identifying general legal issues.

12. Limitations of AI

AI cannot reliably:

• Verify that a legal authority exists,
• Determine the current state of the law,
• Provide jurisdiction-specific legal advice,
• Predict the outcome of litigation,
• Assume responsibility for mistakes made in court filings.

Conclusion

• AI can be a useful research and drafting tool, but it is not a substitute for legal expertise.
• Courts increasingly view unverified AI-generated content as a serious litigation risk.
• The responsibility for ensuring accuracy remains entirely with the litigant or lawyer who submits the document.
• Until AI systems can guarantee legal accuracy and accountability, reliance on AI without independent verification may expose litigants to significant legal and financial consequences.

Reference

• Michael Legg, The Conversation (based on research conducted by the University of New South Wales (UNSW) Centre for the Future of the Legal Profession).
• Federal Court of Australia, observations of Justice Needham concerning the use of generative AI in litigation (June 2025).
• Barry Nilsson Lawyers, commentary on AI-generated hallucinations and litigation risks.
• OBITER, AI Hallucination Cases Tracker.
• Lawyers Weekly Australia reports on judicial responses to AI-generated court filings.
• Thomson Reuters Institute, research on fabricated citations and AI-related litigation risks.
• Futurism, reporting on AI-generated legal errors and sanctions.
• Texas State Office of Administrative Hearings (SOAH), Advisory on the Use of Artificial Intelligence in Legal Proceedings (October 2025).