Use case

Continuous Integration and Deployment using ArgoCD and GitHub Action

About the Customer

BizROI.AI is a platform designed to optimize data costs and improve the return on investment (ROI) for organizations utilizing modern data infrastructures. 

The platform provides end-to-end visibility into data pipelines, helping organizations manage their data costs more efficiently. Some of the key features includes Cost Management and Reduction, Data Lineage and Root Cause Analysis, AI-Powered Optimization, Budget Management and Seamless Integration.

Customer Challenge

The customer needed a robust Continuous Integration and Deployment (CI/CD) pipeline for their NestJs application, which consists of multiple microservices.

They wanted to ensure that any changes pushed to the main branch trigger the CI/CD pipeline, resulting in the creation and deployment of Docker images to the Amazon Elastic Container Registry (ECR).

Additionally, they required a mechanism to handle semantic releases, which, upon publication, would trigger the deployment process to the production environment.

The main challenges included:

  • Application Build: The build process was time-consuming and error-prone. It required 20-30 minutes to build and deploy all 15 microservices, leading to significant delays. This was particularly time-consuming for multiple developers working in the Dev/Stage environment, as they had to wait for their changes to be reflected.
  • Integration Issues: Maintaining consistent integration of code changes was challenging. Initially, the automation process would build and deploy all 15 microservices to the staging environment, even when only a few microservices had code changes. This approach was both time and resource-intensive.

Solution

  • Change Detection: Utilize NX’s capability to detect affected microservices, so only those services that have had changes will go through the build process. This significantly reduces the build time and the computational resources required for building the entire suite of microservices.
  • Microservice Isolation: Each microservice will have its build process isolated, meaning that builds can occur independently of one another. This isolation ensures that changes in one service don’t unnecessarily trigger builds in unrelated services.
  • Parallel Builds: By isolating the builds, we can execute the build processes in parallel across different microservices. This parallelism leverages available computational resources efficiently, reducing the overall time required for the build process.
  • Reliable Deployment: We leveraged AWS Managed EKS (Elastic Kubernetes Service) to deploy services more reliably and faster. EKS allows us to scale deployments efficiently while ensuring high availability and resilience, further streamlining the overall deployment process.
  • Automated Synchronization: ArgoCD automates the synchronization of the desired application state, as defined in Git, with the actual running state in the EKS cluster. This reduces manual intervention and speeds up the deployment process.
  • Progressive Delivery: With ArgoCD, we can implement progressive delivery strategies like canary deployments and blue-green deployments, allowing for safer, controlled rollouts.

CI/CD Workflows after the changes:

For Staging:

Whenever changes are pushed to the main branch, GitHub Actions workflows are triggered.

These workflows build Docker images for all microservices and push them to the ECR repository. Post image creation, the workflows update the image tags in the Helm charts for the microservices.

ArgoCD then synchronizes these changes and updates the deployments in the EKS cluster.

Semantic Release for Production:

A manual workflow is configured to handle semantic releases. Upon publishing a release, a GitHub Actions workflow is triggered automatically.

This workflow follows a similar process of building and pushing Docker images to ECR, updating Helm charts, and synchronizing deployments via ArgoCD for the production environment EKS Cluster.

CI/CD with ArgoCD and github
CI/CD workflow

Results and Benefits

We achieved a 70% reduction in overall build and release time, reducing the time from 20-30 minutes to 5-7 minutes, enabling faster delivery of new features and bug fixes.

Developers can now focus more on writing code and testing their changes quickly as the reduced build time allows for faster feedback in the Dev/Stage environments.

Also, running all builds in parallel has significantly reduced the time needed for builds, leading to enhanced productivity for all developers.

Reduce the time required for rollback procedures by 70%, with rollback options from ArgoCD. Developers can quickly revert deployments in case of issues, minimizing downtime and mitigating risks.

ArgoCD provides a user-friendly UI that allows developers to visualize deployments and quickly identify potential issues, leading to faster troubleshooting and resolution.

Our key learnings from the project were:

  • CI/CD Best Practices: Gained insights into best practices for setting up CI/CD pipelines using GitHub Actions, NX Releases, Semantic Versioning and ArgoCD, particularly in a Monorepo context.
  • Tools Integration: Learned effective strategies for integrating various tools (GitHub, Docker, NX, ArgoCD, Helm, and EKS) to create a seamless and efficient CI/CD process.

About IAMOPS

IAMOPS is a full DevOps suite company that supports technology companies to achieve intense production readiness.

Our mission is to ensure that our clients’ infrastructure and CI/CD pipelines are scalable, mitigate failure points, optimize performance, and minimize costs.

Our DevOps suite includes DevOps Core, NOC 24/7, FinOps, QA Automation, and DevSecOps to accelerate overall exponential growth.

As an AWS Advanced Tier Partner and Reseller, we focus on two key pillars: Professionalism by adhering to best practices and utilizing advanced technologies, and Customer Experience with responsiveness, availability, clear project management, and transparency to provide an exceptional experience for our clients.

Want to transform your CI/CD workflows with DevOps expertise?

Book a Free Call