Security, compliance, and governance
Approaches for Implementing Governance Strategies
Five governance approaches
- Policies: clear policies on the organization's approach to generative AI, covering data management, training, output validation, safety, human oversight, IP, bias, and privacy. Review them regularly.
- Review cadence: regular technical, legal, and responsible AI reviews, for example monthly, quarterly, or twice a year depending on risk, with diverse stakeholders.
- Review strategies: technical reviews (performance, data quality, robustness) and non-technical reviews (policy, ethics, regulation), testing before deployment, and clear rules for when to intervene.
- Transparency standards: publish information about models, training data, and key decisions. Document capabilities, limitations, and intended use. Give stakeholders a feedback channel.
- Team training: bias mitigation and responsible AI training, cross-functional collaboration, ongoing certification.
Monitoring an AI system
These are operational practices, separate from the governance approaches above:
- Performance metrics: accuracy, precision, recall, F1, latency
- Infrastructure: CPU, GPU, memory, network, storage, logs
- Bias and fairness, especially in healthcare, finance, and HR
- Compliance and responsible AI
Generative AI Security Scoping Matrix
Classifies a generative AI application into one of five scopes, from least to most ownership:
| Scope | You... | Example |
|---|---|---|
| 1. Consumer app | Use a public generative AI service | A public chatbot |
| 2. Enterprise app | Use a third-party business app with generative AI built in, under a business agreement | A scheduling app that drafts agendas |
| 3. Pre-trained models | Build your app on an existing FM | An app calling a model in Amazon Bedrock |
| 4. Fine-tuned models | Fine-tune an existing FM with your data | A model tuned on your support tickets |
| 5. Self-trained models | Train a model from scratch on your data | Your own FM |
For each scope, the matrix covers four security disciplines: governance and compliance, legal and privacy, risk management, and controls and resilience.
- Controls are security measures that reduce risk, such as controlling who can use which FMs and who can reach inference endpoints.
- Resilience means staying available and meeting SLAs, for example by checking each service is available in your Region.