Exploring the threat landscape for GenAI cloud workloads

GenAI is transforming our lives in various ways, and organisations are rapidly exploiting its potential to enhance productivity, innovation, and business advantages. However, only some organisations can easily access GenAI despite the perceived benefits.  

Consequently, cloud service providers (CSPs) now provide GenAI-as-a-Service offerings, such as Amazon Bedrock, Azure AI Services, and Google Vertex AI. These GenAI-as-a-Service offerings drastically lower the entry barrier for organisations and fast-track development, deployment, and maintenance of GenAI workloads without bothering much about the infrastructure and knowledge requirements.  

However, these GenAI workloads introduce several security and safety challenges that require attention, given the increasing need for Responsible AI systems, e.g., the EU AI Act. Organisations must understand these risks and possible countermeasures to retain customer trust and business excellence and thwart potential cloud attacks. 

Top threats against GenAI cloud workloads  

LLMJacking

LLMJacking is an attack where criminals gain illegal access to Large Language Models (LLMs) by first stealing cloud credentials, e.g., AWS API Keys, and using the credentials to access cloud GenAI workloads e,g, LLMs. The primary motivation is to pass over the vast bills accrued to the victim account organisation (up to $46,000 of LLM consumption/day). 
KrebsonSecurity recently published a story on LLMJacking attacks based on research investigation conducted by Permiso Security. The report reveals the lucrativeness of LLMJacking for cybercriminals; a prominent cyber criminal organisation behind this attack generated $1 million in annualised revenue via a business model involving the reselling of sex chatbots. LLMJacking was first reported earlier this year by Sysdig Threat Research Team. 

Data poisoning

Data poisoning attacks are malicious actions where attackers inject malicious or nonsensical data into the training datasets to corrupt the output produced by LLMs. Data is the core of GenAI; training and fine-tuning data must be easily accessible to Cloud GenAI workloads to allow for customisation and contextualisation via Retrieval Augmented Generation (RAG) techniques. 
The LLMs provided by CSPs are usually pre-trained and general-purpose. While this drastically reduces the effort and resources needed for training, the responses from the models need to be customised to meet specific use cases required by organisations. However, fine-tuning data is commonly kept in S3 buckets, thus exposing the data to data-poisoning attacks - attackers are familiar with the vulnerabilities related to S3.

Data exfiltration

Similar to the data poisoning attacks, data used by cloud GenAI workloads are attractive targets for exfiltration attacks. There are several reasons attackers would be interested in training data more than other types of data, the primary reason being the need to acquire proprietary information or intellectual property. Access to intellectual property might give a competing organisation a considerable advantage. 

Prompt injection

Prompt injection attacks are one of the most commonly discussed threat vectors against LLMs. Attackers implement this attack by crafting malicious prompts that trick LLMs into acting in an unintend way, such as divulging sensitive information. 
Prompt injection occurs at the application layer. However, similar to other application security attacks, there are implications and challenges when applications are deployed on the cloud, e.g., to take advantage of rapid scaling and elasticity. With the need to scale GenAI applications, organisations would prefer central management of GenAI applications for the sake of resilience and scalability. This would eventually mean exposing prompt injection attacks to the infrastructure rather than the application layer.

Model attacks

Cloud GenAI workloads might include both provider-managed and customer-managed Machine Learning (ML) models. Provider-managed models are less vulnerable to attacks since the providers implement extra security measures, and these are the least exposed. 
However, customer-managed models are more susceptible to attacks due to higher levels of exposure. Customer-managed models are often stored in container image repositories (e.g., Azure Container Registry), virtual machines, or Kubernetes clusters. Several ML image attack vectors are documented on the MITRE ATLAS, including Backdoor ML Model, Full ML Access, Evade ML Model, and Erode ML Model Integrity. Several of these model attacks were recently demonstrated against SAP AI Core by Wiz research. 

Cloud misconfiguration

Most cloud attacks are due to misconfigured cloud resources and customers' faults. Naturally, GenAI cloud workloads are not isolated from this menace. Some important aspects are Identity and Access Management (IAM) controls, encryption, and secret management. Each type of misconfiguration could have substantial security impacts on GenAI workloads, resulting in successful attacks. 

Addressing threats against GenAI workloads

Several threats against GenAI cloud workloads are well known, and mitigation strategies are established. However, many of these threats are AI-specific and require novel mitigation strategies. Let’s examine two of these essential strategies for securing GenAI cloud workloads.   

AI Security Posture Management (AISPM)

Several security posture management approaches, including CSPM and DSPM, are used to manage diverse aspects of the cloud. These approaches only partially cover the AI-specific gaps; hence, there is a need for a specific approach: AI Security Posture Management (AISPM).