Use case
Automating Athena Query Reports to Slack via Lambda Function
- Lakshya Chhajed
About the Customer
BrightPay revolutionizes online payment card management with its card-on-file solution, enabling users to effortlessly save and update their payment cards across various online accounts and subscriptions. By centralizing card updates, users can conveniently manage their payment information without leaving their preferred platforms. This seamless integration benefits merchants and banks by providing them enhanced control over the card-on-file process, driving increased card usage, spending, and transaction volume from the outset.
Customer Challenge
BrightPay faced challenges in tracking and reporting latency metrics (p90, p95, p99) for critical API endpoints. With a large volume of CloudFront access logs stored in an S3 bucket, manually analyzing and generating reports was inefficient and prone to errors. This manual process made it difficult to monitor API performance trends effectively, posing a risk of missing early indicators of latency issues, which could impact user experience and overall application reliability.
The customer needed an automated process to collect, aggregate, and report latency metrics without requiring human intervention. Regular, reliable reporting was essential for maintaining optimal performance and quickly identifying potential bottlenecks.
Solution
IAMOPS implemented a comprehensive solution leveraging AWS services, including Lambda, Athena, S3, CloudFront and EventBridge to automate latency metric reporting.
1) Workflow Design and Automation
- An automated workflow was created to fetch CloudFront logs stored in S3, analyze them using Athena, and generate latency metrics (p90, p95, p99). The data was aggregated into a CSV report and delivered directly to a predefined Slack channel for easy access by BrightPay’s team.
2) Lambda Function Development:
- A Lambda function was configured to execute scheduled Athena queries, fetch and process the required latency metrics, and save the output in CSV format within an S3 bucket. The Lambda function then retrieved the report and sent it as a message to a Slack channel, ensuring prompt and consistent report delivery.
3) Scheduled Execution with EventBridge
- EventBridge was used to automate report generation on a weekly and monthly basis, eliminating manual triggers. Reports were set to be sent every Thursday at 9 AM UTC for the past week’s data, and on the first day of each month at 9 AM UTC for the last month’s data.
4) Post-Deployment Monitoring
- IAMOPS conducted tests to validate the solution, ensuring that the Athena queries were accurate, the Lambda functions were triggered as scheduled, and the reports were reliably delivered via Slack. In case of any failure in lambda function, CloudWatch Alarm will notify us in Slack.
The diagram below illustrates the flow of the automated latency metrics report generation:
Results & Benefits
- Reduced manual effort in latency reporting, saving significant time for BrightPay and IAMOPS team.
- Provided consistent weekly and monthly performance insights, enabling proactive API performance management.
- Enhanced decision-making by aggregating CloudFront logs into actionable metrics.
- Streamlined the log analysis process through automation of metric reporting.
About IAMOPS
IAMOPS is a full suite DevOps company that supports technology companies to achieve intense production readiness.
Our mission is to ensure that our clients’ infrastructure and CI/CD pipelines are scalable, mitigate failure points, optimize performance, ensure uptime, and minimize costs.
Our DevOps suite includes DevOps Core, NOC 24/7, FinOps, QA Automation, and DevSecOps to accelerate overall exponential growth.
As an AWS Advanced Tier Partner and Reseller, we focus on two key pillars: Professionalism by adhering to best practices and utilizing advanced technologies, Customer Experience with responsiveness, availability, clear project management, and transparency to provide an exceptional experience for our clients.
