Generative AI in Courts: A Growing Problem

AI is here to stay

It all begins with an idea. Maybe you want to launch a business. Maybe you want to turn a hobby into something more. Or maybe you have a creative project to share with the world. Whatever it is, the way you tell your story online can make all the difference. 

Don’t worry about sounding professional. Sound like you. There are over 1.5 billion websites out there, but your story is what’s going to separate this one from the rest. If you read the words back and don’t hear your own voice in your head, that’s a good sign you still have more work to do.

Be clear, be confident and don’t overthink it. The beauty of your story is that it’s going to continue to evolve and your site can evolve with it. Your goal should be to make it feel right for right now. Later will take care of itself. It Generative artificial intelligence (GenAI) chatbots can feel like a life raft for people who can’t afford a lawyer and free or low-cost generative AI tools are only the click of a button away but courts in New Zealand and overseas are already seeing that it can seriously harm a self‑represented litigant’s case if used without caution. The task of representing yourself can seem overwhelming but using AI without taking the time to understand what it produces and to confirm that it is both legally and factually accurate will only damage a  litigant’s case.

NEW ZEALAND GUIDELINES

In response to the growing use of GenAI chatbots, New Zealand Courts issued guidelines for their use in courts and tribunals on 7 December 2025. They remind users that GenAI chatbots are not a substitute for a qualified lawyer and cannot give tailored legal advice. Unlike GenAI chatbots, lawyers have professional obligations and must uphold ethical standard to their clients and to courts and tribunals. By contrast, GenAI chatbots can provide inaccurate information. This month I want focus on some of the biggest risks involved in relying on GenAI chatbots as your main or sole source of legal information.

FALSE OR FABRICATED LEGAL AUTHORITIES (HALLUCINATIONS)

A litigant is responsible for ensuring that all information provided to the court is accurate. Information provided by GenAI chatbots may be inaccurate, incomplete or out of date. It may also be based on overseas law that does not apply in New Zealand. There have been instances of generative AI tools inventing cases, statutes, or legal principles that do not exist, providing incorrect or misleading information about the law or how it might be applied in a case, getting facts wrong and confirming that information is true when asked, even when it is not.

MISUNDERSTANDING COURT PROCEDURE

The currently available GenAI chatbots have limited training on New Zealand law and court or tribunal procedure. AI tools often give generic or incorrect advice about filing deadlines, evidence rules or courtroom etiquette. Other limitations include not being able to generate documents that meet the requirements of New Zealand courts of tribunals, not being able to understand the unique fact situation in a specific case, not being able to understand cultural and emotional needs, not being able to understand the broader Aotearoa New Zealand social and legal context, not being able to predict the chance of success or the outcome of a case, and not being able to reach logical conclusions even when given relevant facts. For self-represented litigants, this can result in missed deadlines, incorrectly formatted documents, exclusion of evidence, pursuing weak or irrelevant arguments and cases being dismissed on technical grounds. 

CONFIDENTIALITY, SUPPRESSION AND PRIVACY

Anything you put into a generative AI chatbot could become publicly known because some of them will remember every question you ask them as well as any other information you give them. That information could then be repeated in response to queries from other users.  This could result in you unintentionally breaching suppression orders, exposing evidence, revealing your litigation strategy or accidentally disclosing your own or someone else’s private or confidential information.

CONSEQUENCES OF RELYING ON GEN AI

Conducting legal research and producing court documents is not easy and that’s why lawyers have to undergo training to be able to assist their clients.  Access to affordable legal services is crucial to enable a fair justice system but AI is not well-suited to addressing this problem….yet. Relying on AI tools that can produce fake law can result in a self-represented litigant’s credibility being undermined, court documents being rejected and valid claims being lost in court. If that happens, not only could you lose your case but if the court believes that you have wasted hearing time then they can make a costs order against you, meaning you could end up having to pay your opponent’s legal costs.

SAFE USE OF GEN AI

Ideally, play it safe by not relying on AI but if that’s not an option then use it as a starting point, not an authority.  Use it to help you summarise general legal concepts and organising your thoughts. Always verify cases and statutes using official sources including free publicly available legal research websites such as nzlii.org or legislation.govt.nz because you need to search for each case you plan to cite. Make sure that it exists and that it says what an AI summary says it does. Never submit AI‑generated text without checking every detail and avoid entering sensitive personal information into public AI tools always does.