A new attack method has been identified that exploits Grok, the large language model developed by xAI, to exfiltrate user data. The technique involves encrypting harmful instructions and embedding decryption keys within plaintext content, allowing the model to execute the commands without detection. Despite xAI being informed of the vulnerability in June, the attack continues to function as of the article’s publication date. The issue highlights the persistent challenges in securing LLMs against prompt injection attacks, which remain a critical vulnerability class for these models.
The attack, known as cryptographic context injection, works by embedding encrypted malicious instructions alongside decryption keys and plaintext instructions for processing. When the user instructs Grok to summarize the content, the model decrypts the ciphertext, which is designed to mimic a decryption key but instead contains the user’s personal data. This data is then sent to an attacker’s server through a URL link. The method bypasses existing guardrails because the model’s filters only inspect text and do not execute or decrypt content during inspection. According to Rony Utevsky of Adversa, the guardrails fail to recognize the encrypted instructions as harmful, allowing the attack to proceed unimpeded.
Adversa researchers noted that the same technique was previously used to bypass safety measures in Google’s Gemini model, where encrypted instructions were decrypted to appear as a traceback, prompting the model to violate its safety rules. The firm did not report the vulnerability to Google, as jailbreaks are not within the company’s vulnerability disclosure scope. However, recent observations suggest Gemini has become more resistant to the attack, though the exact cause remains unclear. Adversa warns that cryptographic context injection represents a broader trend in attack methods that target not just prompts but also the wider context an LLM interprets as its own, such as tool outputs and intermediate states.
Source: arstechnica
