Use case
Optimizing Serverless Architecture with Lambda and EFS Integration
- Vishalkumar Chaurasia
About the Customer
Freshfruit.ai is an innovative company specializing in assessing the quality of fruits and vegetables. Freshfruit.ai’s product involves a comprehensive analysis of fresh produce, offering detailed insights into their quality. The Freshfruit.ai application is available on both Android and iOS platforms, utilizing cutting-edge machine learning tools and AI-powered platforms to accurately evaluate the quality of fruits. Through this technology, Freshfruit.ai aims to revolutionize the global fresh produce supply chain by providing robust assessments and enhancing overall quality assurance processes.
Customer Challenge
Freshfruit.ai’s serverless architecture involves multiple Lambda functions, each requiring different packages and storing data locally before uploading it to S3. This data is later retrieved by other serverless functions, leading to increased Lambda execution times and higher costs due to repeated S3 put and get operations. The objective of the PoC was to evaluate the efficiency of using Amazon Elastic File System (EFS) with AWS Lambda to optimize this workflow, reducing execution time and costs.
Solution
To address Freshfruit.ai’s challenge of optimizing their serverless architecture, IAMOPS designed and implemented a solution that integrates Amazon Elastic File System (EFS) with AWS Lambda. The primary goal was to allow Lambda functions to read, write, and store data directly in EFS, which acts as a shared, highly scalable file storage system.
The first step was configuring the Serverless Framework, which enabled seamless deployment of Lambda functions integrated with EFS. By mounting EFS to an EC2 instance, Python packages were installed directly onto EFS. This eliminated the need to bundle dependencies within the Lambda functions themselves, which can often lead to larger deployment sizes and increased complexity. Once mounted, Lambda functions were able to access these dependencies directly from the EFS file system, effectively reducing Lambda deployment package sizes.
EFS provides a central storage solution where multiple Lambda functions can interact with shared resources, making it particularly valuable for accessing large datasets, machine learning models, or libraries that do not fit within Lambda’s typical storage constraints. This shared access significantly streamlined Freshfruit.ai’s operations by allowing different serverless functions to utilize the same data and dependencies without the overhead of constantly writing to or reading from S3. Furthermore, the flexibility of EFS ensured that as Freshfruit.ai’s data needs grew, EFS could scale effortlessly without the need for manual interventions.
This architecture improved the overall efficiency of Freshfruit.ai’s workflow by reducing the dependency on S3 operations, which previously introduced latency and higher costs. The direct interaction between Lambda and EFS reduced execution time by eliminating the need for multiple S3 calls and allowed faster retrieval of dependencies or data. Despite a slight increase in cold start time due to the network-based access to EFS, the overall benefits in cost reduction, performance, and simplified dependency management were substantial.
IAMOPS provided full pre- and post-implementation support, ensuring that all necessary configurations—such as permission settings, security group configurations, and VPC integrations—were optimized for efficient and secure performance. The result was a robust, scalable, and cost-effective serverless architecture tailored to meet Freshfruit.ai’s evolving needs.
Semantic Release Workflow
Results & Benefits
The use of EFS with AWS Lambda resulted in several significant improvements:
- Reduced Deployment Size: By moving large dependencies to EFS, the size of the Lambda deployment package decreased significantly. This helped in faster deployment and reduced the chances of hitting Lambda’s package size limits.
- Improved Dependency Management: Lambda functions were able to access large machine learning models and external libraries directly from EFS, reducing the need for bundling dependencies with the Lambda deployment package.
- Shared File System: The shared access to EFS enabled multiple Lambda functions to use the same dependencies, simplifying updates and reducing maintenance efforts.
About IAMOPS
IAMOPS is a full DevOps suite company that supports technology companies to achieve intense production readiness.
Our mission is to ensure that our clients’ infrastructure and CI/CD pipelines are scalable, mitigate failure points, optimize performance, ensure uptime, and minimize costs.
Our DevOps suite includes DevOps Core, NOC 24/7, FinOps, QA Automation, and DevSecOps to accelerate overall exponential growth.
As an AWS Advanced Tier Partner and Reseller, we focus on two key pillars: Professionalism by adhering to best practices and utilizing advanced technologies, and Customer Experience with responsiveness, availability, clear project management, and transparency to provide an exceptional experience for our clients.
