Use case
Accelerating Image Processing with AWS Lambda for Real-Time Results
- Vishalkumar Chaurasia
About the Customer
Freshfruit.ai is an innovative company that specializes in quality assessment of fruits and vegetables using cutting-edge machine learning and AI tools. Their product provides detailed insights into the quality of fresh produce, helping revolutionize the global supply chain by improving quality assurance processes. Freshfruit.ai’s application is available on both Android and iOS platforms, targeting efficient quality analysis to enhance the overall produce market.
Customer Challenge
Freshfruit.ai processes thousands of images to assess the quality of produce, and this requires a scalable and time-efficient infrastructure. Initially, the company was relying on ECS (Elastic Container Service) to handle image processing tasks. However, ECS has a startup time of 1-2 minutes, which slowed down the entire processing pipeline, especially for large datasets. This delay impacted their ability to provide real-time quality assessments, critical to their customers in the supply chain industry. Moreover, Freshfruit.ai needed a solution that could scale automatically to accommodate fluctuations in workload without incurring high operational costs.
If the challenge wasn’t addressed, Freshfruit.ai risked delays in image processing, reduced operational efficiency, and unsatisfied customers waiting for quality reports on perishable goods. A faster, more scalable solution was imperative for their operations.
Solution
IAMOPS developed a solution leveraging AWS Lambda’s serverless architecture to reduce the startup time and enable parallel processing. This new approach shifted the image processing from ECS to Lambda functions, which can handle multiple messages from the Simple Queue Service (SQS) concurrently, processing images in parallel.
The primary components of the solution included:
- AWS Lambda with Container Images: AWS Lambda was configured to handle container images, ensuring compatibility with the existing ECS-based image processing tasks. Lambda’s event-driven architecture allowed it to automatically scale when triggered by new messages in the SQS.
- SQS as a Trigger: The SQS service was used as a trigger to invoke AWS Lambda functions every time a new message was available. This allowed automatic and instantaneous processing of images as soon as they were queued, eliminating the need for ECS to continuously monitor the queue.
- Terraform for Infrastructure as Code (IaC): Terraform scripts were created to manage the infrastructure, ensuring that the environment remained consistent and scalable. This also provided greater control over resource allocation and minimized configuration drift.
- CI/CD Deployment with GitHub Actions: To automate the deployment of the Lambda functions and the required infrastructure, GitHub Actions were used to streamline the CI/CD pipeline.
- Lumigo for Monitoring: Lumigo was integrated to trace, monitor, and log the execution of Lambda functions, ensuring transparency and efficient debugging during production.
Hawkeye ECS Service Diagram
Hawkeye Service as Lambda Diagram
Results & Benefits
- Reduced Image Processing Time: The shift to AWS Lambda reduced the startup time from minutes to milliseconds, and the concurrent execution allowed processing thousands of images in parallel. The overall time taken to process 10,000 images was drastically reduced.
- Scalability and Cost Optimization: AWS Lambda’s pay-as-you-go model ensured that Freshfruit.ai only paid for the actual processing time, resulting in reduced costs compared to ECS, which required constant resource allocation. Furthermore, Lambda scaled automatically to accommodate varying workloads without requiring manual intervention.
- Enhanced Monitoring and Debugging: With Lumigo tracing, Freshfruit.ai gained deep insights into the performance of the Lambda functions, enabling them to troubleshoot issues quickly and effectively.
About IAMOPS
IAMOPS is a full DevOps suite company that supports technology companies to achieve intense production readiness.
Our mission is to ensure that our clients’ infrastructure and CI/CD pipelines are scalable, mitigate failure points, optimize performance, ensure uptime, and minimize costs.
Our DevOps suite includes DevOps Core, NOC 24/7, FinOps, QA Automation, and DevSecOps to accelerate overall exponential growth.
As an AWS Advanced Tier Partner and Reseller, we focus on two key pillars: Professionalism by adhering to best practices and utilizing advanced technologies, and Customer Experience with responsiveness, availability, clear project management, and transparency to provide an exceptional experience for our clients.
