Use case
Automating Secret Management with AWS Secrets Manager and External Secrets Operator
- Lavish Gupta
About the Customer
CyvexTech is a company specializing in securing the software supply chain by providing solutions for code assurance, tamper-proof documentation, and anomaly detection. Their platform offers enhanced visibility and protection throughout the entire software lifecycle, ensuring the integrity and security of code artifacts from production to delivery.
Customer Challenge
CyvexTech encountered significant challenges in managing secrets between their Production and Staging environments. The two environments relied on different secret management tools, with Production using the Secret CSI Driver synced from AWS Secrets Manager and Staging using GitHub Secrets. This disparity led to deployment failures, inconsistencies, and an increased risk of human error. The discrepancies between environments made troubleshooting more difficult and resulted in delays in deployment cycles.
Solution
IAMOPS proposed a unified, automated approach to secret management to address these challenges. The solution focused on migrating both environments to AWS Secrets Manager and replacing the Secret CSI Driver with External Secrets operator. This eliminated the use of GitHub Secrets for Staging and ensured consistent secret handling across both environments.
Steps taken:
1. Unified Secret Management:
- Migrated both the Staging and Production environments to AWS Secrets Manager, ensuring consistency and eliminating environment discrepancies.
2. Automated Secret Key Verification:
Developed a Python script to automatically verify secret keys between Staging, Production, and the Production Secrets Manager, reducing the risk of human error. The script runs as a GitHub Action and logs any discrepancies, simplifying secret verification.
3. Simplified Secret Integration:
Replaced the Secret CSI Driver with External Secrets operator Helm charts, allowing for direct secret mapping from AWS Secrets Manager to Kubernetes secrets without manually specifying each key-value pair. Deployment files were updated to streamline the referencing of environment variables.
The diagram for the solution is shown below:
Semantic Release Workflow
Results & Benefits
- Consistency: Ensured identical secret management setups between Staging and Production environments, reducing deployment failures and discrepancies.
- Efficiency: The automated verification process decreased the time spent on troubleshooting, saving the team effort and reducing delays.
- Reliability: Human errors in secret management were minimized due to automation, improving the overall reliability of deployments.
- Simplified Processes: Integrating secrets became more efficient, enabling smoother and faster deployments.
About IAMOPS
IAMOPS is a full DevOps suite company that supports technology companies to achieve intense production readiness.
Our mission is to ensure that our clients’ infrastructure and CI/CD pipelines are scalable, mitigate failure points, optimize performance, ensure uptime, and minimize costs.
Our DevOps suite includes DevOps Core, NOC 24/7, FinOps, QA Automation, and DevSecOps to accelerate overall exponential growth.
As an AWS Advanced Tier Partner and Reseller, we focus on two key pillars: Professionalism by adhering to best practices and utilizing advanced technologies, and Customer Experience with responsiveness, availability, clear project management, and transparency to provide an exceptional experience for our clients.
