Responsible AI practices
Model Trade-Offs
Interpretability versus explainability
| Interpretability | Explainability | |
|---|---|---|
| Meaning | A human can explain the output directly from the model's weights and features | The behavior of any model, even a black box, is explained in human terms |
| How | Inspect the inner mechanics | Model-agnostic methods: partial dependence plots, SHAP dependence plots, surrogate models |
| Example | An economist reads the coefficients of a regression model for inflation | A news site discovers its neural network labels business articles about sports teams as "sports" |
High interpretability usually costs performance. If you need exact transparency, your choice of algorithms is limited. If you need high performance with a general understanding, use explainability.
Safety versus transparency
Model safety is avoiding harm: bias, privacy exposure, and security vulnerabilities. Safety protects information and transparency exposes it, so they pull against each other:
- Accuracy: complex neural networks are more accurate but less interpretable than linear models.
- Privacy: techniques like differential privacy improve safety but make models harder to inspect.
- Safety: filtering outputs hides the original reasoning.
- Security: models trained on air-gapped networks are harder to audit externally.
Controllability
A controllable model is one whose predictions and behavior you can influence by changing the training data. It's more transparent and makes it easier to correct bias.
- Linear models are more controllable than complex neural networks.
- Test it by adding or removing examples and checking that the output changes as expected.
- Improve it with data augmentation and training constraints.