How to retrieve credentials stored in AWS Secrets Manager from AWS Lambda running in VPC

AWS Secrets Manager is a secrets management service that enables you to store credentials and retrieve it dynamically when you need them. It helps protect access to your applications and services. With AWS Secret Manager you can –

• Programmatically retrieve encrypted secret values at runtime
• Store different types of secrets
• Encrypt secret data
• Automate secret’s rotation

When you create a AWS Secrets Manager, AWS provides you with a sample code which you can use to retrieve the secrets in your application. The sample code is available in Java/JavaV2/JavaScript/C#/Python3/Ruby/Go.

In this blog post, I will point out two things which you need to do in order to retrieve credentials stored in AWS Secrets Manager from AWS Lambda function running inside a VPC.

Let’s build a scenario –

Suppose you have the application data stored in Aurora PostgreSQL database. The database is running in Private Subnets and is not publicly accessible. You want to access this data from API. To do so, you create an API using Amazon API Gateway. The GET method of API, calls the Amazon Lambda function which retrieves the data from Amazon PostgreSQL database. You have stored the database credentials in AWS Secrets Manager. Within AWS Lambda function, you call AWS Secrets Manager to retrieve the credentials and create connection to the database successfully.

Note, all resources are running in the same AWS account.

For the Lambda Function to access the data from database running in private subnet you will need to create it in the same VPC. The AWS Lambda function can be in public or private subnet. For this use-case I have created the Lambda function in public subnet with the same security group attached to the database.

When you create the AWS Lambda function within a VPC, it creates an elastic network interface (ENI) for each subnet in the VPC configuration. In my case I have attached Lambda to 1 subnet and hence it created 1 ENI.

EC2 > Network & Security > Network Interfaces

With this basic setup, now let’s look at what needs to be done to successfully retrieve credentials from AWS Secrets Manager through AWS Lambda function.

1. Create Amazon VPC endpoint for AWS Secrets Manager. Make sure to keep Private DNS names enabled.

VPC > Endpoints > Create Endpoint

2. In the security group attached with AWS Lambda, edit the inbound rules and add TCP protocol for 443 range with source as itself.

By setting up the endpoints and having proper inbound rule in security group, you can retrieve credentials from AWS Secrets Manager through AWS Lambda function running in VPC.

Below is the code snippet of Lambda function for reference –

import json
import boto3
import base64
from botocore.exceptions import ClientError
import pg8000.dbapi


# Use this code snippet in your app.
# If you need more information about configurations or implementing the sample code, visit the AWS docs:   
# https://aws.amazon.com/developers/getting-started/python/
def get_secret():

    secret_name = "pg/fleetdb/admin"
    region_name = "us-west-2"

    # Create a Secrets Manager client
    session = boto3.session.Session()
    client = session.client(
        service_name='secretsmanager',
        region_name=region_name
    )

    # In this sample we only handle the specific exceptions for the 'GetSecretValue' API.
    # See https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_GetSecretValue.html
    # We rethrow the exception by default.

    try:
        get_secret_value_response = client.get_secret_value(
            SecretId=secret_name
        )
    except ClientError as e:
        if e.response['Error']['Code'] == 'DecryptionFailureException':
            # Secrets Manager can't decrypt the protected secret text using the provided KMS key.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'InternalServiceErrorException':
            # An error occurred on the server side.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'InvalidParameterException':
            # You provided an invalid value for a parameter.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'InvalidRequestException':
            # You provided a parameter value that is not valid for the current state of the resource.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'ResourceNotFoundException':
            # We can't find the resource that you asked for.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
    else:
        # Decrypts secret using the associated KMS CMK.
        # Depending on whether the secret is a string or binary, one of these fields will be populated.
        if 'SecretString' in get_secret_value_response:
            secret = get_secret_value_response['SecretString']
        else:
            secret = base64.b64decode(get_secret_value_response['SecretBinary'])
        return secret

def lambda_handler(event, context):

    # TODO implement
    credentials = json.loads(get_secret())
    #print(credentials)
    hostname = credentials['host']
    dbname = credentials['dbname']
    port = credentials['port']
    username = credentials['username']
    password = credentials['password']

    conn = pg8000.dbapi.connect(
        host=hostname,
        database=dbname,
        port=port,
        user=username,
        password=password
        )

    cursor = conn.cursor()
    cursor.execute("SELECT 1")
    result = cursor.fetchone()
    
   
    return {
        'statusCode': 200,
        'body': result
    }

Hope you find this helpful!

Advertisement