Responsible AI practices
Transparent and Explainable Models
- Transparency answers how a model makes decisions. It supports accountability, trust, and auditing.
- Explainability answers why the model made a particular decision. It shows the model's limitations and helps with debugging.
Models without either are black box models, such as deep neural networks with many layers.
Why transparent models are worth it
- More trust, especially in healthcare, finance, and transportation
- Easier to debug and improve
- Better understanding of the data and the decision process
They don't always outperform black box models.
Ways to add transparency and explainability
- Explainability frameworks: SHAP (SHapley Additive exPlanations), LIME (Local Interpretable Model-agnostic Explanations), and counterfactual explanations.
- Transparent documentation of architecture, data sources, training, and assumptions
- Monitoring and auditing for bias and unusual behavior
- Human oversight of high-stakes decisions
- Counterfactual explanations: show how the output would change if an input changed
- User interface explanations of outputs, rationale, and limits
The course text expands SHAP and LIME incorrectly. The names above are the correct ones.
Risks
More complexity and cost, vulnerabilities that attackers can exploit, unrealistic expectations of full transparency, and exposing information that hurts privacy, security, or competitive edge.
AWS tools
| Goal | Tool |
|---|---|
| Transparency about AWS AI services | AWS AI Service Cards |
| Transparency about your own models (intended use, risk rating, training details and metrics, evaluation results) | SageMaker Model Cards |
| Explain feature contributions per prediction | SageMaker Clarify |
| Explanations for AutoML models | SageMaker Autopilot (uses Clarify) |