Security and Privacy Considerations for AI Systems

Security considerationsCopy Anchor Link

In the context of AI and generative AI, there are a number of security tasks, such as threat detection, vulnerability management, infrastructure protection, prompt injection, and data encryption. Following is a description of each of these tasks.

Threat detectionCopy Anchor Link

To detect threats to your AI systems, do the following:

• Identify and monitor for potential security threats, such as malicious actors attempting to exploit vulnerabilities in AI systems or using generative AI for malicious purposes. The following are some examples:
• Generating fake contentManipulating dataAutomating attacks
• You can assist threat detection by developing and deploying AI-powered threat detection systems. You can analyze network traffic, user behavior, and other data sources to detect and respond to potential threats.

For more information, see Threat Detection and Protect Against Adversarial and Malicious Activities.

Vulnerability managementCopy Anchor Link

To help manage vulnerability, do the following:

• Identify and address vulnerabilities in AI and generative AI systems, including software bugs, model weaknesses, and potential attack vectors (for example, malware, viruses, and email attachments).
• Regularly conduct security assessments, penetration testing (attempt to find and exploit vulnerabilities), and code reviews to uncover and address vulnerabilities.
• Implement robust patch management and update processes to ensure that AI systems are kept up to date and secure.

For more information, see Vulnerability Management.

Infrastructure protectionCopy Anchor Link

To ensure that your infrastructure is protected, do the following:

• Secure the underlying infrastructure that supports AI and generative AI systems, such as the following:
• Cloud computing platformsEdge devicesData stores
• Implement strong access controls, network segmentation, encryption, and other security measures to protect the infrastructure from unauthorized access and attacks.
• Ensure that the AI infrastructure is resilient and can withstand failures, attacks, or other disruptions.

For more information, see Infrastructure Protection.

Prompt injectionCopy Anchor Link

You need to mitigate the risk of prompt injection attacks. In these attacks, adversaries attempt to manipulate the input prompts of generative AI models to generate malicious or undesirable content. To reduce the risk, do the following:

• Employ techniques, such as prompt filtering, sanitization, and validation, to ensure that the input prompts are safe and do not contain malicious content.
• Develop robust models and training procedures that are resistant to prompt injection attacks.

For more information, see Protect Against Data Poisoning Threats.

Data encryptionCopy Anchor Link

To protect the confidentiality and integrity of the data used to train and deploy AI and generative AI models, do the following:

• Implement strong encryption mechanisms to secure both data at rest and data in transit. Data at rest refers to data that is stored on servers, in databases, or on local devices. Data in transit refers to data that is transmitted during communication between different components of the AI system.
• Ensure that the encryption keys are properly managed and protected from unauthorized access.

For more information, see Data Protection and Protect Sensitive Data Privacy.

The OWASP Top 10 for LLMsCopy Anchor Link

The Open Web Application Security Project (OWASP) Top 10 is the industry standard list of the top 10 vulnerabilities that can impact a generative AI LLM system. These vulnerabilities are as follows:

1. Prompt injection: Malicious user inputs that can manipulate the behavior of a language model
2. Insecure output handling: Failure to properly sanitize or validate model outputs, leading to security vulnerabilities
3. Training data poisoning: Introducing malicious data into a model's training set, causing it to learn harmful behaviors
4. Model denial of service: Techniques that exploit vulnerabilities in a model's architecture to disrupt its availability
5. Supply chain vulnerabilities: Weaknesses in the software, hardware, or services used to build or deploy a model
6. Sensitive information disclosure: Leakage of sensitive data through model outputs or other unintended channels
7. Insecure plugin design: Flaws in the design or implementation of optional model components that can be exploited
8. Excessive agency: Granting a model too much autonomy or capability, leading to unintended and potentially harmful actions
9. Overreliance: Over-dependence on a model's capabilities, leading to over-trust and failure to properly audit its outputs
10. Model theft: Unauthorized access or copying of a model's parameters or architecture, allowing for its reuse or misuse

OWASP Top 10 for LLMs

For examples of these vulnerabilities, common attack scenarios, and prevention methods, choose the following link.

OWASP Document

Additional resourcesCopy Anchor Link

There are a number of resources that are helpful for addressing the overall security and privacy requirements of your AI systems. The following are a few that you might explore.

AWS Cloud Adoption Framework: Security Perspective

The security perspective helps you achieve the confidentiality, integrity, and availability of your data and cloud workloads. To learn more, choose the following link.

AWS Whitepaper

Mitre ATLAS

Adversarial Threat Landscape for Artificial-Intelligence Systems (ATLAS) is a knowledge base of adversary tactics and techniques. To learn more, choose the following link.

ATLAS Webpage

Addressing Open Worldwide Application Security Project (OWASP) Top 10 Risks

The OWASP Top 10 is a standard awareness document for developers and web application security. To learn how to address these risks within AWS, choose the following link.

Developer Article

Architect Defense-in-Depth Security for Generative AI Applications Using the OWASP Top 10 for LLMs

This blog post provides a common mental model and framework to apply security best practices. To learn more, choose the following link.

AWS Blog

Next, you will learn about AWS services and features for securing AI systems.