Title: AWS re:Inforce 2024 - Data lake deep dive: Using accelerated indexing for threat detection (TDR228-S)

Insights:

• Integration Announcement: Splunk announced new integrations with AWS Security Lake, enhancing their federated analytics capabilities. These integrations were highlighted at both AWS re:Inforce and Splunk Conf.
• Federated Analytics: The new integrations fall under the umbrella of federated analytics, which includes federated search for Security Lake and Data Lake Index. These tools allow users to search and analyze data without storing it in Splunk.
• Federated Search for Security Lake: This feature allows users to search data stored in Security Lake directly from the Splunk interface, similar to the previously announced federated search for S3.
• Data Lake Index: This feature enables users to pull data from Security Lake into a temporary index in Splunk, allowing for searches and detections without committing the data to long-term storage.
• Use Cases: Common use cases include threat hunting and threat detection. Users can run queries and detections on data stored in Security Lake without the need to store high-volume, low-fidelity data in Splunk.
• Customer Feedback: Customers have expressed a need for accessing data like VPC flow logs and CloudTrail data for investigations without the cost and storage implications of pulling all data into Splunk.
• Retention and Indexing: Users can set different retention periods for various data types, such as CloudTrail or VPC flow logs, allowing for more efficient data management and cost control.
• Third-Party Data Sources: Security Lake supports third-party data sources, enabling integration with vendors like CrowdStrike Falcon and Palo Alto, providing broader data access within Splunk.
• Setup Process: The setup involves creating subscribers and providers in AWS Security Lake and configuring them in Splunk. This process includes enabling token authentication, setting up federated providers, and configuring data access and search permissions.
• Demo Overview: A recorded demo showcased the process of setting up subscribers and providers, highlighting the ease of configuration and the granular control over data access and indexing.

Quotes:

• "We announced two new integrations under the umbrella of federated analytics: federated search for Security Lake and Data Lake Index."
• "Customers were asking for access to high-volume, low-fidelity data like VPC flow logs without the need to store it in Splunk."
• "Federated search for Security Lake allows you to search data in Security Lake directly from the Splunk UI without committing it to disk."
• "Data Lake Index enables pulling data into a temporary index in Splunk, allowing for searches and detections without long-term storage."
• "We wanted to give customers the ability to access data in Security Lake while letting the data stay in Security Lake."
• "Typical use cases include threat hunting and threat detection, where users can run queries and detections on data stored in Security Lake."
• "Security Lake supports third-party data sources, enabling integration with vendors like CrowdStrike Falcon and Palo Alto."
• "The setup process involves creating subscribers and providers in AWS Security Lake and configuring them in Splunk, with a focus on ease of configuration and granular control over data access."
• "The key advantage is access to data in Splunk that you didn't have before because you were not sending that data into Splunk."