Responsible AI practices
Amazon Services and Tools for Responsible AI
Amazon SageMaker AI and Amazon Bedrock have built-in tools for each area of responsible AI.
| Need | Tool | What it does |
|---|---|---|
| Evaluate foundation models | Model evaluation on Amazon Bedrock | Compare and pick FMs. Automatic evaluation uses predefined metrics (accuracy, robustness, toxicity). Human evaluation covers subjective metrics (friendliness, style, brand voice) with your own team or an AWS-managed team |
| SageMaker Clarify | Also evaluates FMs | |
| Safeguard generative AI | Guardrails for Amazon Bedrock | Blocks denied topics, filters harmful content (hate, insults, sexual, violence) with thresholds, and redacts or blocks PII. Works with any FM, including fine-tuned ones, and with Agents |
| Detect bias | SageMaker Clarify | Analyzes chosen features such as age or gender and reports bias metrics |
| Balance data | SageMaker Data Wrangler | Random undersampling, random oversampling, and SMOTE |
| Explain predictions | SageMaker Clarify (with SageMaker Experiments) | Scores showing which features contributed most to a prediction, plus feature importance charts for tabular data |
| Monitor in production | SageMaker Model Monitor | Watches model quality on endpoints or batch jobs and alerts on deviations |
| Human review | Amazon Augmented AI (A2I) | Routes predictions to people for review |
| Governance | SageMaker Role Manager | Defines minimum permissions quickly |
| SageMaker Model Cards | Documents intended use, risk rating, and training details | |
| SageMaker Model Dashboard | One place to track model behavior in production | |
| Transparency of AWS services | AWS AI Service Cards | For each AWS AI service: basic concepts, intended use cases and limitations, responsible AI design, and deployment and performance best practices |
Easy to mix up
- Clarify covers bias and explainability. Model Monitor covers drift and quality in production. A2I covers human review.
- AI Service Cards document AWS's services. Model Cards document your own models.
- Guardrails filter inputs and outputs at runtime. They don't retrain the model.