How to use Aurora Cluster in your Altostra projects

When you are deciding on what database to use, you have many solutions to choose from. If your choice falls onto Relational Database, Amazon gives you the following options:

• RDS Instance - a single Db Instance that gives you maximum control over a database creation. Supports MySQL, PostgreSQL, MariaDB, Oracle, or Microsoft SQL Server.
• Aurora Cluster - may include more than one read replica of db instance which provides more scalability and reliability. Supports MySQL and PostgreSQL.
• Aurora Serverless - automatically allocates cloud database resources based on load demand, automated capacity scaling. Supports MySQL and PostgreSQL.

In this post, We will discuss the advantages of the Aurora cluster and show how you can use Altostra to easily create with Aurora Cluster in no time.

Amazon Aurora is a relational database built for the cloud. According to Amazon, Aurora Database is up to five times faster than standard MySQL databases and three times faster than standard PostgreSQL databases.

Why cluster

An Amazon Aurora DB cluster consists of one or more DB instances and a cluster volume that manages the data for those DB instances.
An Aurora Cluster volume is a virtual database storage volume that spans multiple Availability Zones, with each Availability Zone having a copy of the DB cluster data.
Two types of DB instances make up an Aurora DB cluster:

• Primary DB instance – Supports read and write operations, and performs all of the data modifications to the cluster volume. Each Aurora DB cluster has one primary DB instance.

• Aurora Replica – Read-only replica. Connects to the same storage volume as the primary DB instance. Each Aurora DB cluster can have up to 15 Aurora Replicas in addition to the primary DB instance.

To conclude, having multiple instances in one cluster helps to maintain high availability by locating Aurora Replicas in separate Availability Zones. Aurora automatically fails over to an Aurora Replica in case the primary DB instance becomes unavailable.

All being said, it's time to see how it works in practice.

Creating database service with Aurora Cluster:

Prerequisite - an Altostra project (Don't have one yet? Learn how to create an Altostra project here)

We will create Aurora Cluster, connect and test it with Lambda.

Adding RDS Cluster

Choose an Aurora Cluster resource.

Aurora cluster settings

When you are adding Aurora Cluster into altostra project, it sets default values for your database.

• By choosing instance count you define the amount of "read replica" instances being created. The default value is 1, it will create the primary read-write db instance. For this demo, we will leave the default.
• The cluster's password is generated automatically with Secret Manager.
• The secret ID will be stored in an environment variable that you can access in the code, without exposing the stored value of the password.
• If you want to see the secret value, it's accessible in the AWS Secrets Manager web console.

Creating Lambda

• Add a Lambda Function, name it "health check".
• Connect Lambda function to Aurora Cluster.
  
  
• After adding the connection, we will see the generated environment variables, that are now accessible for the Lambda, in the Lambda settings. We will use these variables later in our code to connect to Aurora Cluster (the names of the variables depend on the resource name):
  • DB_CLUSTER01 - the endpoint of the cluster (host and port).
  • DB_SECRET_CLUSTER01 - the secret IDs of the user name and password of the cluster.
    

Lambda's code

Add npm to your project by executing

npm init

install mysql package

npm i mysql

const { inspect } = require('util')
const mysql = require('mysql')
const aws = require('aws-sdk')

module.exports.handler = async (event, context) => {
  try {
    const [host, port] = process.env.DB_CLUSTER01.split(':')

    const secMan = new aws.SecretsManager()
    const secret = await secMan.getSecretValue({
      SecretId: process.env.DB_SECRET_CLUSTER01
    }).promise()

    const { username, password } = JSON.parse(sec.SecretString)

    const dataApi = mysql.createConnection({
      connectTimeout: 30000,
      host,
      port: Number(port),
      user: username,
      password,
    })

    await new Promise((s, j) => dataApi.connect((err, ...x) => err ? j(err) : s(x))))

    return {
      statusCode: 200,
      body: `successfully connected to DB`
    }
  } catch (err) {
    return {
      statusCode: 500,
      body: err.message
    }
  }
}

Configuring the VPC

By default after creation, our database is not accessible to other resources and is located in a Virtual Private Cloud (VPC). In order for Lambda to get access to the Database they need to be in the same VPC.
Now, where can I find "those VPCs" and what exactly do I need from there?

Subnets

If you are using a specific VPC, then just provide subnet ids from the AWS web console

In our demo project, we will use a default VPC. When you create an Aurora cluster with default settings it automatically joins the default VPC.
You can get subnet ids from the default VPC.

Security group

Aurora cluster will create your Database within a default port unless you set a custom port on resource creation. Default DB port - Postgres 5432, MySql 3306.

If you are using a default security group, it will have a wildcard for all ports, so no additional actions required. You can get the group from the AWS web console If you are building your own security group make sure you have 2 security groups. One for Lambda, one for RDS. After creating them, add inbound/outbound rules according to the following scheme:

Lambda SG:

• Direction: Outbound
  Protocol: TCP
  Port: 3306
  Source: ALL

RDS SG:

• Direction: Outbound
  Protocol: TCP
  Port: 3306
  Source: ALL
  
• Direction: Inbound
  Protocol: TCP
  Port: 3306
  Source: Lambda SG

Setting VPC configuration

The most flexible way is to put your VPC configuration into Altostra environments parameters

and then set it in the VPC configuration section on both Lambda and Cluster

If you are using default VPC, it is enough to set values only on Lambda

Deploy

Now that we added the code, let’s deploy the first version of our project, using the Altostra CLI:

$ alto push v1
$ alto deploy cluster:v1 --env dev

Testing our project:

All what left is to test the resulted flow. Open deployed stack with Altostra, and press "open in AWS". From there, locate the Lambda resource and execute the code.

Summary

In this post, we have learned how to create and deploy Aurora cluster.
Now, in 5 minutes, you can build your own production-ready Database Service.

What next

Want to make your database more stable under high load and have control over connection usage by utilizing a connection pool?
Consider adding an RDS Proxy.