Title

AWS re:Invent 2023 - Best practices for container observability (COP319)

Summary

• Speakers: Avi Khanna (Product Manager at AWS) and Omer Krikci (Product Manager in AWS CloudWatch).
• Enhancements in Container Observability:
  • Enhanced Container Insights launched in early November 2023 with additional metrics.
  • Agentless onboarding for Managed Prometheus added for EKS clusters.
• AWS Observability Categories:
  • Native services in the CloudWatch ecosystem.
  • Open source managed services (Managed Grafana, Managed Service for Prometheus, Amazon Open Search Service).
• Collection Mechanisms:
  • CloudWatch Agent and AWS distribution for OpenTelemetry.
• Customer Preferences:
  • Security-first guarantees, production scale, seamless AWS integrations, and contributions to open source.
• Onboarding Barriers:
  • Simplified onboarding for ECS and EKS with one-click experiences.
• Metrics to Collect:
  • CloudWatch Container Insights provides essential metrics.
  • Open-source approach aligns with community best practices.
• Opinionated Observability:
  • AWS best practice alarms based on customer data.
  • Integration of Container Insights with Application Performance Monitoring (APM) for end-to-end visibility.
• Cost vs. Benefit:
  • Enhanced Container Insights for EKS with a new pricing model that offers up to 97% savings.
• Demonstration:
  • Showcased the integration of Container Insights with APM and best practice alarms.
  • Demonstrated the ease of accessing logs and metrics and identifying issues and their impacts.

Insights

• AWS's Commitment to Observability: AWS continues to invest in simplifying the observability of containerized environments, recognizing the importance of quick and effective incident resolution.
• Agentless Collection: The introduction of agentless collection for Prometheus metrics signifies a move towards simplifying infrastructure and reducing the overhead of running and managing agents.
• Integration of Observability and APM: The integration of Container Insights with APM tools like Application Signals demonstrates AWS's focus on providing end-to-end visibility, from infrastructure to application performance.
• Cost-Effective Observability: AWS's new pricing model for Enhanced Container Insights reflects a response to customer feedback about the cost of observability. The significant cost savings could lead to broader adoption of AWS's native observability tools.
• Community and Open Source Alignment: AWS's approach to open-source observability services, such as Managed Grafana and Managed Service for Prometheus, shows a commitment to maintaining alignment with community standards and best practices.
• Future Developments: The mention of cross-account observability indicates upcoming features that will allow organizations to have a unified view of their container environments across multiple AWS accounts.
• Workshops and Learning Resources: AWS encourages hands-on learning through workshops, which can help users better understand and utilize the full range of AWS observability tools.