Future AI chatbots may secretly dob on their users

Google DeepMind is testing an algorithm which may help AI detection programs. (Rounak Amini/AAP PHOTOS)

You might not be able to pass generative AI writing off as your own for much longer: data scientists have developed a way for the technology to dob on its users. 

Researchers from Google DeepMind released the findings on Thursday, outlining a technical solution that could watermark text produced by large-language models such as Google Gemini and ChatGPT. 

Artificial intelligence experts say the breakthrough could prove useful in fields including education, but warned it would not be a failsafe solution for identifying AI content or plagiarism. 

The research project outlined its solution to watermarking text, called SynthID-Text, in Springer’s Nature journal

Google DeepMind researchers investigated whether an algorithm could be used to “bias the word choice” of text produced by AI tools, producing a signature that could be read by AI detection programs.

A person typing on a keyboard (file image)
The technology has been tested on more than 20 million Gemini chatbot interactions.

The study investigated two approaches, including a “distortionary” method that affected the quality of the writing that AI tools produced, and “non-distortionary” approach that did not noticeably impact the text it produced.

Researchers tested the technology using more than 20 million Gemini chatbot interactions and found both made the AI-generated text easier to identify as computer-made.

“Our work provides proof of the real-world viability of generative text watermarks,” the paper said. 

“(It) sets a practical milestone for accountable, transparent and responsible (large-language model) deployment.”

However, the study noted these text-based watermarks could be “weakened” if users edited or paraphrased portions of the text. 

Despite the workaround, University of the Sunshine Coast computer science lecturer Erica Mealy said the watermarking technology could help to tackle a difficult modern challenge. 

“It is one of the biggest problems (for generative AI), so steps in the right direction are good,” Dr Mealy said.

“Theoretically, you can watermark images and you can watermark videos but text, because it’s such a simple mechanism, was going to be very interesting.”

Students work on computers (file image)
Experts say the technology may prove useful in the education sector, but isn't a failsafe solution.

Text-based watermarks could be particularly useful in school settings, Dr Mealy told AAP, where detection programs were regularly delivering false positives and false negatives about the use of generative AI. 

She said AI watermarking technology could also prove useful in other academic settings, to detect whether chatbots had been used to contribute to studies. 

“It could be quite relevant in research papers because that’s another area that people are looking at and working out whether people are using AI to generate their work,” Dr Mealy said. 

“It’s a dodgy topic and a bit of a dangerous area to go into because people get very fiery about their opinions.”

Despite the research, Google has yet to reveal whether it will add SynthID-Text in its AI tools. 

License this article

What is AAPNews?

For the first time, Australian Associated Press is delivering news straight to the consumer.

No ads. No spin. News straight-up.

Not only do you get to enjoy high-quality news delivered straight to your desktop or device, you do so in the knowledge you are supporting media diversity in Australia.

AAP Is Australia’s only independent newswire service, free from political and commercial influence, producing fact-based public interest journalism across a range of topics including politics, courts, sport, finance and entertainment.

What is AAPNews?
The Morning Wire

Wake up to AAPNews’ morning news bulletin delivered straight to your inbox or mobile device, bringing you up to speed with all that has happened overnight at home and abroad, as well as setting you up what the day has in store.

AAPNews Morning Wire
AAPNews Breaking News
Breaking News

Be the first to know when major breaking news happens.


Notifications will be sent to your device whenever a big story breaks, ensuring you are never in the dark when the talking points happen.

Focused Content

Enjoy the best of AAP’s specialised Topics in Focus. AAP has reporters dedicated to bringing you hard news and feature content across a range of specialised topics including Environment, Agriculture, Future Economies, Arts and Refugee Issues.

AAPNews Focussed Content
Subscription Plans

Choose the plan that best fits your needs. AAPNews offers two basic subscriptions, all billed monthly.

Once you sign up, you will have seven days to test out the service before being billed.

AAPNews Full Access Plan
Full Access
AU$10
  • Enjoy all that AAPNews has to offer
  • Access to breaking news notifications and bulletins
  • Includes access to all AAPNews’ specialised topics
Join Now
AAPNews Student Access Plan
Student Access
AU$5
  • Gain access via a verified student email account
  • Enjoy all the benefits of the ‘Full Access’ plan at a reduced rate
  • Subscription renews each month
Join Now
AAPNews Annual Access Plan
Annual Access
AU$99
  • All the benefits of the 'Full Access' subscription at a discounted rate
  • Subscription automatically renews after 12 months
Join Now

AAPNews also offers enterprise deals for businesses so you can provide an AAPNews account for your team, organisation or customers. Click here to contact AAP to sign-up your business today.

SEVEN DAYS FREE
Download the app
Download AAPNews on the App StoreDownload AAPNews on the Google Play Store