Title

AWS re:Invent 2022 - Explainable attention-based NLP using perturbation methods (BOA401)

Summary

Cyrus, a developer advocate specialist, and Sosan, a research scientist, present on explainability in NLP projects.
Explainability is crucial for trust, transparency, fairness, and accountability in AI models.
Two types of explainability are discussed: global (general decision principles) and local (specific decisions).
Perturbation methods involve changing inputs and observing outputs to determine feature importance.
LIME and SHAP are two perturbation-based explainability models introduced.
Sosan presents a case study from Amazon Transportation using perturbation to understand BERT model decisions.
Amazon SageMaker Clarify is highlighted for providing out-of-the-box explainability, detecting biases, and monitoring models over time.

Explainability in AI is becoming increasingly important due to regulatory requirements and ethical considerations.
Local explainability is essential for end-users to understand specific decisions made by AI, such as loan rejections.
Perturbation methods are a powerful tool for explainability, especially in black-box models like deep learning.
LIME and SHAP are two prominent methods for feature attribution, with SHAP based on game theory providing a unique solution for reward distribution among features.
The case study demonstrates the practical application of perturbation methods in a real-world business scenario, improving model reliability and providing insights to stakeholders.
Amazon SageMaker Clarify offers a comprehensive solution for model explainability and bias detection, which is crucial for maintaining fair and accountable AI systems.
The talk emphasizes the importance of both model interpretability and robustness, suggesting different perturbation strategies for each.
The integration of explainability tools like SageMaker Clarify into AWS infrastructure underscores the commitment to responsible AI development and deployment.