Responsible AI practices
Responsible Considerations to Select a Model
Model selection affects everything from user experience to profitability. Evaluate candidates with Model evaluation on Amazon Bedrock or SageMaker Clarify.
Define the use case narrowly
A narrowly defined use case lets you tune the model for it. Face recognition is a technology, not a use case.
- Gallery retrieval (finding missing persons) should favor recall: better to return many possible matches.
- Celebrity recognition or virtual proctoring should favor precision: too many results are not useful.
Generative AI example for an online store:
| Catalog a product | Persuade to buy | |
|---|---|---|
| Audience | Broad | Narrow |
| Risks | Veracity | Veracity, unwanted bias, toxicity |
| Tuning | Neutral, clear, complete | Focused on what matters most to that group |
Performance factors
- Level of customization: from prompting to full retraining
- Model size: parameter count
- Inference options: self-managed or API
- Licensing: some licenses restrict commercial use
- Context window: how much fits in one prompt
- Latency: time to generate output
Performance is a function of the model and the test dataset, not the model alone. Datasets evolve, so track both.
Other considerations
- Sustainability: socially, environmentally, and economically sustainable over the long term.
- Responsible agency: value alignment, responsible reasoning, an appropriate level of autonomy with human oversight, and transparency and accountability.
- Environmental: energy consumption, resource use (GPUs and data centers), and environmental impact assessments.
- Economic: efficiency gains weighed against job displacement, inequality, and concentration of power in a few companies.