Title: AWS re:Inforce 2024 - Safeguarding sensitive data used in generative AI with RAG (DAP223)

Insights:

• Generative AI Customization Challenges: Customers initially tried using pre-built foundation models for their chatbot services but found them inadequate for specific needs, leading to the exploration of custom models and prompt engineering.
• Retrieval Augmented Generation (RAG): RAG emerged as a cost-effective and efficient solution, allowing the use of context with prompts to get accurate answers without retraining foundation models.
• Traditional RAG Architecture: Involves uploading internal data to Amazon S3, converting it to vector format via Amazon Bedrock, and storing it in Amazon OpenSearch Service. Queries to Amazon Bedrock include this context.
• Simplified RAG Architecture: New features in Amazon Bedrock, such as Agent for Amazon Bedrock and Knowledge Basis, simplify the RAG architecture.
• Security Focus: Emphasis on securing the RAG architecture, particularly in data protection and network isolation.
• Network Security: Recommendations include using AWS Direct Connect or AWS Site-to-Site VPN instead of the internet to upload data to Amazon S3, and employing AWS WAF to protect against DDoS attacks.
• Network Isolation: Use of VPCs and VPC endpoints to ensure data flows only within AWS, crucial for compliance with stringent regulations in the financial sector.
• Data Protection with Amazon Macie: Automatically detects sensitive data uploaded to Amazon S3, identifying policy issues and sensitive information like PII and financial data.
• Guardrails for Amazon Bedrock: Filters harmful content and ensures responsible AI by blocking or masking sensitive information in responses.
• Core Security Services: AWS IAM for managing permissions and KMS for data encryption provide a robust security foundation.

Quotes:

• "My customers are all interested in generative AI as you are; they wanted to apply generative AI to their chatbot service."
• "The easiest and cheapest way to do this is with prompt engineering. But this method is not effective."
• "The RAC is a way to pass context with the prompt to get the right answer without retraining the foundation model."
• "AWS recommends that you create a dedicated network over the AWS Direct Connect or AWS Site-to-Site VPN instead of the Internet."
• "AWS WAF blocks requests from specific sources that exceed certain thresholds and prevents excessive costs and service outage."
• "Using a VPC endpoint, you can ensure that data flows only within AWS, not over the Internet."
• "Amazon Macie service can find two main types of data: data that may have policy issues and sensitive data."
• "Guardrails for Amazon Bedrock features filter out harmful content from Amazon Bedrock's answer."
• "If you only apply what you see on the screen now, you will be able to enjoy great results with little effort."