Title

AWS re:Invent 2023 - Threat modeling your generative AI workload to evaluate security risk (SEC214)

Summary

The session focused on evaluating security risks associated with generative AI workloads using threat modeling.
The speakers, Kareem, Danny, and Ana, from AWS, discussed the importance of threat modeling in building secure applications.
They used a healthcare company's generative AI chatbot as a case study to demonstrate the threat modeling process.
The process involved identifying what the team was working on, understanding the architecture, and creating a data flow diagram.
They discussed the importance of understanding the business context, using threat frameworks, and having existing threat information.
The speakers walked through the creation of a threat statement and the categorization of threats using the STRIDE framework.
They demonstrated how a malicious actor could exploit the system using crafted prompts and how to mitigate such threats.
The session concluded with recommendations for penetration testing, automated testing, and stakeholder reviews to validate the threat model.
Resources such as the Threat Modeling Workshop, Threat Composer, and Generative AI Security Scoping Matrix were shared.

Generative AI poses unique security challenges that require careful consideration and threat modeling to ensure data confidentiality and system integrity.
The session highlighted the iterative nature of threat modeling, emphasizing that an incomplete threat model is better than none, and it should evolve over time.
The use of frameworks like STRIDE and resources like OWASP Top 10 for LLMs and MITRE Atlas can provide structure and guidance in identifying and categorizing threats.
The case study demonstrated the importance of sanitizing user inputs, pre-defining query structures, and validating responses to prevent unauthorized data access.
The session underscored the need for defense-in-depth strategies and the importance of involving security experts early in the design phase to build secure applications.
The speakers recommended regular penetration testing and automated testing, especially for new releases and when new threats are identified, to ensure the effectiveness of security measures.
The resources shared at the end of the session, such as the Threat Modeling Workshop and Threat Composer, can be valuable tools for organizations looking to implement threat modeling for their generative AI workloads.