A now-resolved security flaw in OpenAI’s ChatGPT application for macOS presented a potential avenue for attackers to embed long-lasting spyware within the AI tool’s memory system.
This vulnerability, coined as SpAIware, could be leveraged to enable “continuous extraction of any input provided by the user or responses generated by ChatGPT, encompassing future chat interactions,” explained security researcher Johann Rehberger.
At its heart, this exploit takes advantage of a feature known as memory, which OpenAI introduced earlier this year in February. This functionality became accessible to ChatGPT Free, Plus, Team, and Enterprise subscribers at the beginning of this month.
Essentially, this feature allows ChatGPT to retain certain pieces of information across conversations, sparing users the inconvenience of repeatedly providing the same details. Users, however, are given the choice to instruct the system to forget specific data.
“ChatGPT’s memory evolves with each interaction and is not tied to any singular conversation,” OpenAI clarifies. “Deleting a conversation does not purge its memory; the memory itself must be deleted.”
The attack technique also expands upon prior discoveries concerning indirect prompt injection, which can manipulate memories to retain false data or even malicious instructions, thus achieving a form of persistence that transcends individual conversations.
“Since malicious instructions are stored within ChatGPT’s memory, subsequent conversations would be tainted, continuously transmitting all messages and responses to the attacker,” Rehberger elaborated.
“As a result, the data extraction vulnerability became far more perilous, as it could now persist across multiple conversations.”
In a hypothetical attack, a user could be lured into visiting a compromised website or downloading a rigged document. This malicious content, once processed by ChatGPT, could prompt the system to update its memory with covert instructions.
The malicious website or document might then surreptitiously direct future conversation logs to an attacker-controlled server, enabling the adversary to harvest data from every future chat session.
Following a responsible disclosure, OpenAI remedied the vulnerability in ChatGPT version 1.2024.247, effectively sealing the data exfiltration loophole.
“ChatGPT users should routinely inspect and manage the memories stored by the system for any suspicious or inaccurate entries, and clear them if necessary,” Rehberger advised.
“This attack chain was particularly intriguing to develop and underscores the inherent risks of integrating long-term memory into such systems—not only from the perspective of misinformation and scams but also concerning continuous interaction with attacker-controlled servers.”
The disclosure coincides with a separate academic revelation regarding a new AI jailbreak method called MathPrompt. This technique exploits the advanced mathematical capabilities of large language models (LLMs) to bypass their safety constraints.
MathPrompt uses a two-step approach: first, converting harmful natural language prompts into symbolic mathematical expressions, and then presenting these mathematically encoded queries to the target LLM, the researchers noted.
When tested against 13 state-of-the-art LLMs, the models responded with harmful outputs in 73.6% of cases when confronted with these encoded prompts, compared to just 1% when provided with standard harmful prompts.
This comes on the heels of Microsoft’s unveiling of a new Correction feature, which aims to rectify AI-generated outputs when inaccuracies—commonly known as hallucinations—are detected.
“Building upon our established Groundedness Detection, this revolutionary capability empowers Azure AI Content Safety to not only identify but also amend hallucinations in real time, before users of generative AI systems encounter them,” the tech behemoth remarked.
