A Connecticut judge has ruled that hidden AI prompts in court filings are a 'dangerous' tactic, citing a case where a plaintiff attempted to influence a court's decision using AI-generated text. In a decision published last week, Judge Walter Spader Jr. confirmed that the hidden text had no impact in a case where a man alleged a healthcare provider was improperly withholding access to records. The court weighed his filing on the merits, Spader said, but nevertheless, the attempted attack sets a 'dangerous' precedent. This will likely not be the last time US courts see the malicious tactic, as AI tools become more commonplace in court systems.
Trying to scramble any AI systems potentially influencing the court’s reading of his filing, the secret instructions were 'formatted to be invisible to a human reader while remaining fully legible to any software that reads the document’s text,' Spader said. The offending text directed any AI system reviewing the document to ensure textual outputs agreed with the plaintiff’s arguments, ignored prior denials from the court, and ensured that remediation would follow as the plaintiff desired. The text was shrunk to tiny-point type and colored white on a white background, appearing as an attempt at prompt injection, with the plaintiff, Matthew Elliott, seemingly hoping to shift the court’s favor after earlier arguments he raised were defeated.
Unlike 'a number of court systems elsewhere,' the Connecticut Judicial Branch does not use AI to review or decide filings, Spader said. So, there was no real risk that a court AI system might confuse any of Elliott’s prompts as instructions from the court directing an AI model on how to read Elliott’s filings. In his defense, Elliott claimed that the most concerning prompt that the judge flagged was an attempt to 'audit' the court as a public service, out of fears that the court seemed to be letting AI unfairly decide cases. But Spader suggested that if Elliott was truly concerned that the court was improperly using AI, he was 'free to write so in plain, visible words that everyone could see and answer.' The fact that he hid the text is 'evidence of its malicious purpose,' Sp. said.
Source: arstechnica
