Weekly take: Fake judgment row shouldn’t spark AI concern

Managing IP is part of Legal Benchmarking Limited, 1-2 Paris Gardens, London, SE1 8ND

Copyright © Legal Benchmarking Limited and its affiliated companies 2025

Accessibility | Terms of Use | Privacy Policy | Modern Slavery Statement

Weekly take: Fake judgment row shouldn’t spark AI concern

Chatgpt.jpg

A New York lawyer could face sanctions after citing fake judgments generated by ChatGPT, but that doesn’t mean practitioners should shy away from AI

Many posts have been circulating on social media about the dangers of using artificial intelligence tools after it emerged that a US-based practitioner relied on non-existent judgments generated by ChatGPT at a court hearing.

The opposing counsel pointed out during arguments that they couldn’t find the cited judgments.

Upon examination, the US District Court for the Southern District of New York found that they were in fact fake citations.

The court then asked the responsible attorney to show cause why he shouldn’t be sanctioned for relying on fake cases to support his arguments.

The lawyer, Steven Schwartz, an attorney at Levidow, Levidow & Oberman in New York, said he was unaware that content provided by ChatGPT could be false and that he was at fault for not confirming the information.

The current case doesn't concern intellectual property, but there is a lesson or two to be learned for the entire legal industry, including the IP sector.

I understand why the incident may have led to increased scepticism towards ChatGPT, but that doesn’t mean that the tool is the only one to blame or that lawyers should stop using AI systems completely.

To me, the case highlights what you shouldn’t do as a lawyer more than it points to the fallibility of a tool launched just six months ago and which has yet to reach its full potential.

According to reports, Schwartz asked ChatGPT to verify whether the cases it found were real and the tool assured him that they were.

If a first-year associate had conducted the same research, the lawyer in charge would probably have verified the citations themselves.

Why shouldn’t the same level of due diligence apply to AI-generated research?

No shortcuts

AI, as it stands, is perhaps not that different from a first-year associate. It’s taking baby steps into a whole new world and is bound to make mistakes.

It is, therefore, incumbent on the users of such tools to ensure that they diligently verify all work it produces and not blindly rely on it.

If someone wants to take the shortcut to success by relying on generative AI without any due diligence on their part, they must take the fall if things backfire.

In this case, ChatGPT provided research that could have been easily verified.

But there are bound to be circumstances that will be more complex.

For example, IP service providers have now started using AI tools to enhance their offering – such as generating names that are not trademarked or assessing chances of success in IP oppositions or infringement cases.

The AI tools analyse thousands of datasets to provide such information.

It could be virtually impossible for a human to physically scan through all those datasets to confirm if the results were accurate.

If a tool can generate fake results, is there any way to make sure it has scanned through all the relevant data provided to it to generate answers?

To my knowledge, the answer is no.

This, again, demonstrates why lawyers must place limited reliance on AI tools.

Hard graft

At the same time, I don't support boycotting AI systems altogether.

Blaming the tool for human errors isn’t the right approach to dealing with this new technology.

AI is growing at an unprecedented rate, and it could help IP lawyers with several tasks, including preparing first drafts of licensing agreements, trademark and patent specifications, and even coming up with available names for new products.

But expecting it to do all your work is unreasonable.

Lawyers must see AI as a tool that can scale up but not replace human effort.

For now, this means using AI only to assist with initial research, preparing working drafts, or ruling out possibilities. But one must put in the necessary hours to review the generated work products for accuracy and suitability.

If more users did that, the conversation within the legal profession would probably shift from the dangers posed by AI to the opportunities it presents.

more from across site and SHARED ros bottom lb

More from across our site

A $110 million US verdict against Apple and an appellate order staying a $39 million trademark infringement finding against Amazon were also among the top talking points
Attorneys are watching how AI affects trademark registrations and whether a SCOTUS ruling from last year will have broader free speech implications
Patent lawyers explain why they will be keeping an eye on the implications of a pharma case and on changes at the USPTO in the second half of 2025
The insensitive reaction to a UK politician crying on TV proves we have a long way to go before we can say we are tackling workplace wellbeing
Adrian Percer says he was impressed by the firm’s work on billion-dollar cases as well as its culture
In our latest interview with women IP leaders, Catherine Bonner at Murgitroyd discusses technology, training, and teaching
Developments included an update in the VAR dispute between Ballinno and UEFA, the latest CMS updates, and a swathe of market moves
The LMG Life Sciences Americas Awards is thrilled to present the 2025 shortlist
A new order has brought the total security awarded to a Canadian tech company to $45 million, the highest-ever by an Indian court in an IP case
Andrew Blattman reflects on how IP practices have changed and shares his hopes for increased AI use and better performance on the stock market
Gift this article