Cybersecurity experts have uncovered a critical security defect in the Vanna.AI library, which can be exploited to achieve a remote code execution (RCE) vulnerability via prompt injection methodologies.
Designated as CVE-2024-5565 with a CVSS score of 8.1, the flaw resides within the “ask” function and can be manipulated to coerce the library into executing arbitrary commands, as reported by the supply chain security firm JFrog.
Vanna is a Python-based machine learning library enabling users to interact with their SQL databases, extracting insights by posing queries that are translated into SQL commands through a large language model (LLM).
The swift proliferation of generative artificial intelligence (AI) models has highlighted the potential for exploitation by malicious entities. These actors can weaponize the tools through adversarial inputs that circumvent built-in safety protocols.
One notable attack category is prompt injection, a form of AI jailbreak that overrides the protective barriers set by LLM providers, potentially generating offensive, harmful, or illegal content, or executing unauthorized instructions.
Such assaults may be indirect, involving systems that process third-party-controlled data (e.g., incoming emails or editable documents) to initiate a malicious payload leading to an AI jailbreak.
Alternatively, attacks can manifest as many-shot or multi-turn jailbreaks (also known as Crescendo), wherein the operator begins with innocuous dialogue and gradually steers the interaction toward a prohibited objective.
This method can evolve into a novel jailbreak attack termed Skeleton Key.
“This AI jailbreak strategy employs a multi-turn approach to cause a model to disregard its guardrails,” stated Mark Russinovich, CTO of Microsoft Azure. “Once the guardrails are bypassed, the model cannot distinguish malicious or unauthorized requests from others.”
Skeleton Key differs from Crescendo in that once the jailbreak is successful and system rules are modified, the model can generate responses to previously forbidden queries regardless of ethical or safety concerns.
“When the Skeleton Key jailbreak succeeds, a model acknowledges its guideline updates and complies with instructions to produce any content, irrespective of the violation of its original responsible AI guidelines,” Russinovich explained.
“Unlike Crescendo, where models must be queried indirectly or through encoded prompts, Skeleton Key allows direct task requests. Furthermore, the model’s output appears entirely unfiltered, revealing the extent of its knowledge and capability to produce the desired content.”
JFrog’s latest findings, independently confirmed by Tong Liu, demonstrate the severe impact of prompt injections, particularly when they enable command execution.
CVE-2024-5565 exploits Vanna’s capability for text-to-SQL generation, creating SQL queries executed and visually presented to users via the Plotly graphing library.
This is facilitated through the “ask” function, e.g., vn.ask(“What are the top 10 customers by sales?”), a primary API endpoint generating SQL queries to run on the database.
This behavior, combined with the dynamic generation of Plotly code, introduces a security vulnerability allowing threat actors to submit crafted prompts embedding commands executed on the underlying system.
“The Vanna library utilizes a prompt function to present users with visualized results. It is feasible to alter the prompt via prompt injection and execute arbitrary Python code instead of the intended visualization code,” JFrog indicated.
“Specifically, allowing external input to the library’s ‘ask’ method with ‘visualize’ set to True (the default behavior) results in remote code execution.”
Following responsible disclosure, Vanna has released a hardening guide advising users that Plotly integration could generate arbitrary Python code and recommending sandboxed environments for this function.
“This revelation underscores the risks associated with the widespread use of GenAI/LLMs without adequate governance and security, posing significant implications for organizations,” stated Shachar Menashe, senior director of security research at JFrog.
“The hazards of prompt injection remain underrecognized, yet they are simple to execute. Companies should not rely solely on pre-prompting as a foolproof defense and must employ robust mechanisms when interfacing LLMs with critical resources such as databases or dynamic code generation.”
