21 Aug 2024 Detecting Unused Secrets in AWS Secrets Manager
Have you ever noticed secrets lingering indefinitely in AWS Secrets Manager, accumulating unnecessary costs? You’re not alone. This common issue can quietly inflate your AWS bill over time. Fortunately, there are solutions to prevent this which we will explore in this blog.
Firstly, for those unfamiliar with AWS Secrets Manager, let’s start with a high level overview of what it is – AWS Secrets Manager is a service designed to securely manage and rotate secrets such as application credentials, OAuth tokens, API keys and other sensitive data. It provides a central repository for managing access to these secrets with built-in security features to ensure they remain protected.
The Problem: Stale Secrets
AWS Secrets Manager is a fantastic tool for managing sensitive information, however, without proper management, these secrets can accumulate and remain in your account long after they are needed. Each secret, even if unused, incurs a monthly cost. Over time, this can add up significantly, impacting your budget without delivering any value.
The Solution: AWS Config
AWS Config provides a powerful way to monitor and manage your AWS resources by providing a detailed view of the configuration of AWS resources in your account. It provides details of how they are related to each other and how they have changed over time.
One of its key features is the AWS Config rule that you can define to evaluate the configuration of your AWS resources overtime. These rules will continuously monitor your resources to ensure they comply with rules defined based on your organisation policies and best practices.
By using the secretsmanager-secret-unused AWS Managed Config rule, you can automatically identify and handle stale secrets in AWS Secrets Manager. Here’s a step-by-step guide on how to set this up:
- Go to your AWS account > AWS Config.
- Go to Rules and click on Add Rule.
- Enter secretsmanager-secret-unused in ‘Find Rules’ search box.
- Select the rule and click Next.
- Modify the name and description as needed. And provide unusedForDays parameter’s value. For the purpose of the demo, I have left this unchanged.
- Click Next. Review the setup and save it to create the AWS Config rule.
Now, this will run periodically and report non compliant secrets in the Resources in scope section. Regularly review the rule’s performance and adjust the criteria as necessary to balance security and cost management.
You can further modify the AWS Config rule to add a remediation action for deleting unused secrets. However, exercise caution with this automation, as automatically deleting secrets may have unintended consequences.
Customisation for specific needs
What if we need customisation? There could be custom requirements such as suppressing some secrets from being reported, which are known to be un-accessed for a longer period. For these types of requirements, we need a custom solution because the AWS Config rule secretsmanager-secret-unused is limited to reporting secrets only.
To accommodate custom needs, you can implement a solution using Amazon EventBridge, Parameter Store, Lambda and S3 bucket.
Explore AWS services for customisation
Let’s dive through what each of these AWS services are before moving to the solution.
EventBridge – EventBridge is a serverless event bus service that makes it easier to connect different applications using events. This allows us to build event driven architectures by routing data from sources to destinations such as AWS services or customer applications. One of the key features of EventBridge is EventBridge rules that match a rule or event to a specific pattern and route them to the appropriate target, making it possible to filter and process events based on content.
Parameter Store – AWS Systems Manager Parameter Store is a service that provides secure storage for configuration data management and secrets management. It is used for storing data such as passwords, API keys, database connection strings and other configuration information as parameters.
S3 – Amazon S3 (Simple Storage Service) is a scalable object storage service that allows you to store and retrieve objects such as files, images, videos and other data types. This service is a highly durable, scalable and secure solution for storing and managing data in the cloud.
The Customised Solution
Alright! That was a brief introduction of the services involved. Now, let’s dive into the solution.
An EventBridge rule is created to trigger on a defined schedule, such as every Monday at 1 PM. The primary task of this EventBridge rule is to invoke a Lambda function at the scheduled time to scan Secrets Manager for unused secrets.
A parameter is created in the Parameter store that contains a list of secret names that need to be suppressed from being reported.
When the lambda function is triggered by EventBridge, the handler is designed to retrieve the name of unused secrets and last accessed date from Secrets Manager. This is achieved by lambda using the AWS Software Development Kit (SDK). This lambda can also be customised to obtain more details if needed. The handler checks the Parameter Store for the list of secret names to be suppressed and excludes them from the reports. Finally, the lambda uploads the results to a S3 bucket.
The solution can be further customised to integrate with services for sending notifications, such as email or Slack messages, to developers or other interested parties.
The code for this solution, implemented using AWS TypeScript CDK and SDK, is available here.
Conclusion
Managing secrets in AWS Secrets Manager doesn’t have to be a costly, disorganised affair. With AWS Config rules, you can automate the identification and handling of stale secrets. For more complex requirements, custom solutions using Amazon EventBridge and AWS Lambda offer flexibility and precision. Start implementing these practices today and take control of your AWS resources, balancing cost management and security.
No Comments