Title

AWS re:Invent 2023 - Solving Large-Scale Data Access Challenges with Amazon S3 (STG337)

Summary

Rob Wilson and Becky Weiss presented on solving large-scale data access challenges with Amazon S3.
They introduced a new feature called S3 Access Grants.
The session covered permissions overview, prefix-based access, access points, structured data considerations, IAM session broker pattern, and the new S3 Access Grants feature.
Security best practices such as block public access and disabling ACLs were emphasized.
S3 Access Grants is a managed service that allows granular access control to S3 data, integrating with IAM and supporting end-user identities from directories.
The session included a demonstration of how S3 Access Grants work and how they can be integrated into applications.
The feature is generally available and can be used to manage access to data lakes, with auditing capabilities that track user access down to the individual level.
S3 Access Grants can coexist with other AWS services like Lake Formation and EMR, and they are designed to work on top of IAM, not as a bypass.

S3 Access Grants provide a solution to the complexity of managing large-scale data access policies by allowing discrete, manageable grants.
The feature supports direct granting to IAM principals as well as end-user identities from directories, making it easier to map users to data sets.
S3 Access Grants enhance audit capabilities by including the original user identity in CloudTrail events, simplifying the process of determining who accessed what data.
The service is designed to work with IAM, ensuring that existing security measures and permissions are not bypassed.
S3 Access Grants can be particularly useful for applications that interface with S3 data using the S3 object API rather than SQL queries, which would be more suited for Lake Formation.
The integration with IAM Identity Center allows for trusted identity propagation, enabling authenticated end users to access S3 data through applications without needing to understand IAM roles.
The session highlighted the importance of considering the scale and dynamism of access patterns when choosing between S3 Access Grants, IAM policies, access points, and other access control mechanisms.
The new feature is expected to streamline the management of access to S3 data for a variety of use cases, including analytics, machine learning, and AI workloads.