How to use AWS DocumentDB in your Altostra projects

When it comes to databases, most developers struggle between a relational, SQL-like databases (strict schema, but the information is easily accessible), and NoSql ones, like DynamoDB - where the data is accessible in specific patterns, but the schema is dynamic (you can always add fields). One popular compromise between these two options is MongoDB - or in the AWS version, DocumentDB. You get a dynamic schema, but you can also use joins and schema validations.

In this post, we'll see how you can use Altostra to easily build your own basic Lambda-based, DocumentDB CRUD service in no time.

You can now quickly add a DocumentDB Cluster resource to your Altostra project and start using it as your database, which is fully managed and MongoDB-compatible. Once you add it to your project, Altostra makes it easy to modify it to your needs using the visual editor.

Creating a simple CRUD service with DocumentDB:

Prerequisite - an Altostra project (Don't have one yet? Learn how to create an Altostra project here)

We will create a simple CRUD service that allows read and write operations to a database.

When we use Amazon DocumentDB, we begin by creating a cluster. A cluster consists of 1-16 instances, which provide the processing power for the database, and allow performing read/write operations on the cluster storage volume. To learn more about DocumentDB Cluster, see AWS DocumentDB docs.

Our service will consist of the following resources:

• DocumentDB cluster (with one instance) - to use as our database.
• Two Lambda functions - one to write documents to our DB, and the other to read documents from the DB.
• REST API Gateway - to get the requests and direct them to our relevant Lambda functions.  
  

Creating our project:

• Add a DocumentDB cluster resource with one instance (with default configurations).
  • The cluster's password is generated automatically with Secret Manager.
  • The secret ID will be stored in an environment variable that you can access in the code, without exposing the stored value of the password.
  • If you want to see the the secret value, it's accessible in the AWS Secrets Manager web console*.  
    
• Add a Lambda Function that connects to MongoDB and inserts a document to the DB.
• Add a Lambda Function that connects to MongoDB and gets an existing document from the DB.
• Connect both Lambda functions to DocumentDB.
  • Insert document should have a read-write connection.
  • Get document should have a read-only connection.
• After adding the connections, we will see the generated environment variables, that are now accessible for the Lambdas, in the Lambdas settings. We will use these variables later in our code to connect to MongoDB (the names of the variables depend on the resource name):
  • DOC_DB_DOCUMENTDB01 - the endpoint of the cluster (host and port).
  • DB_SECRET_DOCUMENTDB01 - the secret IDs of user name and password of the cluster.  
    
• Lambda's accessible environment variables:
  

• Add an API Gateway resource and create endpoints for both Lambda Functions.  
  
  When we create a DocumentDB Cluster, it's created by default with TLS enabled, so we need a certificate for connecting to the DB. We will download the certificate file in our Lambdas code, and send it to the connection method.

Let’s update the following code in the insert-doc Lambda:

• First, install mongodb and axios packages

$ npm i mongodb axios

const MongoClient = require('mongodb').MongoClient;
const aws = require('aws-sdk');
const path = require('path');
const axios = require('axios');
const fs = require('fs');
const { tmpdir } = require('os');

exports.handler = async (event, context) => {
  try {
    //get environment variables values
    const [host, port] = process.env.DOC_DB_DOCUMENTDB01.split(':')
    const secretManager = new aws.SecretsManager()
    const secret = await secretManager.getSecretValue({
      SecretId: process.env.DB_SECRET_DOCUMENTDB01
    }).promise()

    //create MongoDB connection uri
    let { username, password } = JSON.parse(secret.SecretString);
    username = encodeURIComponent(username);
    password = encodeURIComponent(password);
    const uri = `mongodb://${username}:${password}@${host}:${port}/sample-database?tls=true&replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false`;

    //download the certificate and create a pem file
    const { data: certificate } = await axios.get('https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem');
    const dirPath = path.join(tmpdir(), 'rds-combined-ca-bundle.pem');
    try {
      fs.writeFileSync(dirPath, certificate, {
        encoding: 'utf-8'
      });
    }
    catch (err) {
      console.error(err);
    }

    //connect to MongoDB with certificate
    const client = await MongoClient.connect(
      uri,
      {
        useNewUrlParser: true,
        tlsCAFile: dirPath,
      }
    );

    //Specify the database and collection to be used
    const db = client.db('sample-database');
    const col = db.collection('sample-collection');

    //insert a new document to MongoDB
    const doc = await col.insertOne({ 'hello': 'Amazon DocumentDB;' });

    client.close();

    return {
      statusCode: 200,
      body: JSON.stringify(doc)
    };
  }
  catch (err) {
    return {
      statusCode: 500,
      body: err.message
    };
  }
}

Use the same code for the get-doc Lambda, but with a call to fineOne instead of insertOne:

//get an existing document from MongoDB
const doc = await col.findOne({ 'hello': 'Amazon DocumentDB;' });

Deploy

Now that we added the code, let’s deploy the first version of our project, using the Altostra CLI:

$ alto push v1
$ alto deploy docdb:v1 --env dev

Testing our project:

Let’s make an insert-doc request:

• You can see the API Gateway endpoint in the endpoints tab:  
  
• insert-doc request:  
  

// $ curl --location --request POST '<your-api-gateway-endpoint>/insert-doc'
$ curl --location --request POST 'https://eigq556733.execute-api.us-east-1.amazonaws.com/Prod/insert-doc'

We are getting an error. If we look in the lambda’s logs, we can see that there was a timeout:  

When we connected the Lambda functions to DocumentDB, we saw a message:  

As a rule, databases should be secured, and only accessible from within our internal network, which consists of the private subnets of our VPC. In this case, to allow our Lambda functions to access our DB, they need to run in the same subnets. When we deploy a DocumentDB without specifying a VPC, it will be deployed to the default VPC in the region you’re working on. In the next part, we’ll see how to define the VPC.

Configuring the VPC:

Let’s update the Lambda functions’ VPC – by setting the subnets to the same subnets of DocumentDB.

• Our cluster is in the default VPC. We can see the default subnets in the AWS DocumentDB web console:  
  
• In addition, MongoDB default port is 27017, so (only) this port should be allowed in the Security Group of the DocumentDB Cluster. We will define the port in the inbound rules of the Security Group of the Cluster. The Security Group of the Cluster should only allow traffic from inside the VPC, meaning, only from the private subnets of the VPC.
• First, we’ll create a security group for the Lambda functions (with an inbound rule of all traffic from the current security group, for simplicity).  
  
• Second, we’ll create a security group with an inbound rule for the MongoDB port, that only allows traffic from our Lambda functions’ Security Group.  
  

Now we have a list of subnets from the DocumentDB cluster's default VPC, and two security groups – one for the Lambda functions and the other for our cluster. Let’s configure the Subnets and the Security Group in our project:  

The cluster's subnets update requires cluster deletion (and recreation), so you should also disable deletion protection (to avoid errors when updating the stack).  

Deploy and Test

let’s deploy our project again and test that it works:

$ alto deploy docdb --push

Now that our VPC is configured, let’s make an insert-doc request again:  

// $ curl --location --request POST '<your-api-gateway-endpoint>/insert-doc'
$ curl --location --request POST 'https://eigq556733.execute-api.us-east-1.amazonaws.com/Prod/insert-doc'

And a get-doc request:  

// $ curl --location --request GET  '<your-api-gateway-endpoint>/get-doc'
$ curl --location --request GET 'https://eigq556733.execute-api.us-east-1.amazonaws.com/Prod/get-doc'

And now we can see that we are able to access the DB.

Summary

In this post, we saw how to get an AWS DocumentDB cluster up and running with Altostra, and how to access it with Lambda functions. We also learned that the VPC must be configured in a secure way, to only allow traffic from within its VPC, and that the Lambdas that access the cluster should be in that VPC.

*Accessing the cluster's password value:

The best secrets stay secret, but if you want to access the cluster's generated password value, you can retrieve it in the Secret Manager:  

Next steps

Want to give it a try? We provide a free-forever plan for developers. Create your free account today at https://app.altostra.com.

Want to stay up to date with the latest Altostra news? Join our community on Slack.

We want to hear from you! Share your Altostra experience with us on Twitter @AltostraHQ and LinkedIn and any suggestions that can make your work with Altostra even better.

Happy coding!