Cheap and cheerful: why ChatGPT is no trademark filer
Managing IP is part of Legal Benchmarking Limited, 4 Bouverie Street, London, EC4Y 8AX
Copyright © Legal Benchmarking Limited and its affiliated companies 2024

Accessibility | Terms of Use | Privacy Policy | Modern Slavery Statement

Cheap and cheerful: why ChatGPT is no trademark filer

ChatGPTAdobe2000x.jpeg
Timon - stock.adobe.com

Despite its enormous potential in internet search, ChatGPT is unable to get trademark drafting right, says Robert Reading of Clarivate

ChatGPT – a chatbot using artificial intelligence (AI) trained on a vast amount of online text – has received enormous interest since publicly launching in December 2022. Online search giants Google and Microsoft (Bing) have announced plans to use AI chatbot technology to radically change the way we find information. Instead of searching using specific words and terms, ChatGPT simulates conversation, with questions receiving answers that have the look and feel of a human response.

Much of the publicity around ChatGPT has focused on the apparent quality of its responses and its ability to produce detailed answers, even in relation to questions in specific technical areas where we currently rely on human expertise and experience.

A huge amount of trademark-related information is now available online. In just a few clicks of a mouse button it is possible to find millions of trademark records, websites of intellectual property firms, explanations of the registration process, the Nice Classification, case law, and a treasure trove of expert analysis and opinion. Which raises a question: can ChatGPT help (or replace!) trademark attorneys?

We started by asking ChatGPT to draft a trademark application covering “chatbots” and to include the relevant Nice classes and specification. After a few seconds of AI thought it produced something that looked impressive:

ChatGPT1.png

Later that day we tried ChatGPT again and started by asking the same question a second time. Expecting to get the same result, we were surprised to be given a different answer:

ChatGPT2.png

Drafting a trademark specification often doesn’t have a single, perfect answer. If five trademark attorneys were given the task independently, we would expect some variation (with considerable overlap). However, a machine is expected to be consistent and produce the same result each time.

While the results look to be good, the lack of consistency may be a cause for concern, especially when the tool is used by someone who doesn’t have a solid feel for what the ‘right answer’ should look like. Over 80% of trademark applications filed at the USPTO last year belonged to individuals or businesses with a small trademark portfolio (fewer than five records); self-filers relying on an online tool for assistance might not realise they are being given suboptimal ‘advice’.

To get a better understanding of how well ChatGPT ‘understands’ the Nice Classification system, we asked it to classify a term from the USPTO ID Manual – “Planters for flowers and plants” – which is assigned to class 21.

ChatGPT again provided a very quick reply that looked authoritative – but it was completely wrong:

ChatGPT4.png

Nice class 7 covers mainly machines and machine tools, motors and engines – it does include agricultural implements, but because they are machines, not because they are agricultural.

Repeating the question, ChatGPT generated a new, completely different answer. Which again was wrong:

ChatGPT5.png

To understand why ChatGPT is making these mistakes despite having a vast amount of online text available during training, it’s important to consider how answers are generated. Although it appears to be mimicking a human and summarising relevant information on a given subject, it is actually using the text it is given (the question) to predict and generate subsequent words (the answer it provides).

This explains why it is not consistent – each time it is asked a question, it uses statistics to generate the next word, and the word after that, etc. There is rarely a ‘perfect’ word, just a series of choices that have probabilities and weighting and which are selected in a fashion that superficially appears to be random. If a question is asked twice and a new word generated early in the second answer, then the entire second answer can look very different to the first.

To illustrate the way ChatGPT works with words rather than understanding concepts, here’s a relatively simple mathematics question and ensuing conversation:

ChatGPT6.png

The correct answer is in fact 7,006,652.

ChatGPT is confident, polite and provides an answer that seems reasonable. But it’s wrong. It hasn’t performed any calculations (the way a calculator would tackle the task) – it has relied on the words in the question to try to figure out which words should be generated to follow them based on text that it has seen previously. The answer it gives is impressively close to the correct answer – it clearly isn’t a completely random or unconnected guess – but when precision matters, it has missed the mark.

The same happens with trademarks. ChatGPT is a very impressive piece of technology. For certain tasks and questions, it may well be the future of online search. It may even replace some tasks currently performed by humans. But in its current form it doesn’t appear to have the specialised training required to deal with trademark classification and specification tasks.

An expert at CompuMark can easily spot that it is making mistakes, but users of the service are not experts – they are likely to be individuals and small businesses with little trademark experience who go online for help. Anyone using ChatGPT as a cheap substitute for professional trademark advice needs to be aware that they are likely to get what they pay for from a free service.

Robert Reading is director of corporate strategy at Clarivate, and is based in the UK.

more from across site and ros bottom lb

More from across our site

Partners and other senior leaders must step up if they want diverse talent at their firms to thrive
European and US counsel reveal why they are (or aren't) concerned about patent quality and explain how external counsel can help
Firms such as Bird & Bird and Taylor Wessing have reported rising profits and highlighted the role of high-profile IP disputes and hires
We provide a rundown of Managing IP’s news and analysis from the week, and review what’s been happening elsewhere in IP
Lawyers in the corporate and IP practices discuss where the firm can steal a march on competitors, its growth plans in London, and why deal lawyers are ‘concertmasters’
Kathleen Gaynor, DEI specialist at Phillips Ormonde Fitzpatrick, says deliberate actions can help law firms reach diversity goals
Scott McKeown, who moved to Wolf Greenfield one year ago, says the change has helped him tap into life sciences work and advise more patent owners
The winners of our Asia-Pacific Awards 2024 will be revealed during a ceremony in Malaysia on September 26
Zach Piccolomini of Wolf Greenfield explains how to maximise your IP portfolio’s value while keeping an eye on competitors
Witnesses at a Congressional hearing debated whether reforming the ITC is necessary and considered what any changes should look like
Gift this article