OpenAI announced it will begin automatically watermarking text generated with ChatGPT in the European Union, starting in the coming weeks, to comply with the EU AI Act.

The company said it will also offer the watermarking feature in other regions, but it will be off by default outside of the EU. The move is driven by a need to comply with the EU AI Act, which took effect in August.

It requires marking content produced by AI models in a way that another tool can detect. Unfortunately, there is still no completely effective and reliable way to do that.

A few standards already exist, like SynthID and the C2PA project, but they are relatively easy to circumvent for anyone with basic know-how. The same is likely true for OpenAI’s watermark.

Its method is proprietary; the company calls it textGrain, and has published a technical paper explaining how it works.

But in general, it works like other LLM watermarking tools we’ve seen in the past: It puts patterns in the word choices that are not clear to a human reader, and that don’t meaningfully change the general quality of the output, but that someone with a key can use a specialized detector to find.

OpenAI says it will be giving access to the detector to a limited number of researchers and organizations, and providing a request-for-approval process for others to be added over time.

While the best case OpenAI’s tests of textGrain show a respectable but not entirely reliable 92 percent successful detection rate, the company’s tests also show that changing just 10 percent of the text in an output reduces the successful detection rate by almost 30 percent, and changing 20 percent of the text can lower the success rate by almost 75 percent. (Also worth noting that in general, the success rate is lower for shorter or translated text than for longer text.) All this is roughly in line with what we’ve seen with other, similar watermarking solutions in the past.

In August, OpenAI competitor Anthropic also introduced watermarking for text generated by its models—but Anthropic has enabled it globally, while OpenAI is currently only making it the default where regulators require it—at least in the ChatGPT and Codex apps.

It will remain off but optionally available in the API, it seems. The new watermarking will roll out to users in the EU “in the coming weeks,” the company says.

Source: arstechnica