Deploying MongoDB Replicaset Architecture in AWS Private VPC
This reference architecture provides a set of YAML templates for deploying the following AWS services :
- Amazon IAM
- Amazon Security Group
- Amazon EC2
- Amazon Route53
Prerequisites Notes
The Cloudformation Security Group IP address is open by default (testing purpose). You should update the Security Group Access with your own IP Address to ensure your instances security.
Before you can deploy this process, you need the following:
- Your AWS account must have one VPC available to be created in the selected region
- Amazon EC2 key pair
- Installed Domain in Route 53.
- cloudformation-vpc (Assuming you already have installed VPC https://github.com/thinegan/cloudformation-vpc )
We have test launch this CloudFormation stack in the following Region in our account:
- US East (N. Virginia)
The repository consists of a set of nested templates that deploy the following:
- A tiered VPC with public and private subnets, spanning an AWS region.
- A highly available ECS cluster deployed across two Availability Zones in an Auto Scaling group.
- Two NAT gateways to handle outbound traffic.
- Two interconnecting microservices deployed as ECS services (website-service and product-service).
- An Application Load Balancer (ALB) to the public subnets to handle inbound traffic.
- ALB path-based routes for each ECS service to route the inbound traffic to the correct service.
- Centralized container logging with Amazon CloudWatch Logs.
MongoDB Cloud Manager Setup
Create a New Project, click "New Project"
Select "Cloud Manager" and Click "Next"
Enter Name of your project and click "Next"
Goto Project "Deployment". Under "Crytera > Timeclonedbrep", select "Agents" and "Downloads & Settings". Since, I'm using Debian Os, select Automation "Ubuntu (15.x, 16.x) - DEB"
Use mmsGroupId and mmsApiKey to setup mms agent in your cloudformation script.
A completed deployed mms automation agent running after completed cloudformation run.
Goto Deployment > Security > Edit Setting. Select "Authentication Mechanisms [X] Username/Password
Continue "Next" without enabling SSL. We will enable it on the process.
Save and Initiate first Credential will be blank password. Remember, you need re-run this credential process again to generate new password.
Re-run the entire credential process again, only this time Agent mms-automation user will generate a password. Don't Save and Deploy yet.
Login to your Mongo Replica Master and create admin user first, based on the credential you got from Cloud Manager.
Now, Save, Review and Deploy your changes,
Next, Goto Deployment > Server.
Install Monitoring Agent in Master Replica
Install Monitoring and Backup Agent in Secondary Replica
Goto Deployment > Processes Click "Manage Existing"
Add Master hostname and mongo port. Turn on "Enable Authentication".
Choose, Auth Mechanism "Username/Password". Enter Username and Password. Select "Continue".
Continue but make sure you see all the processes in your deployment.
Check, "I understand that this require..." and click "Continue".
Check, "Yes, import users and roles from this deployment item".
Click "Continue".
Proceed after "Automation Agent Successfully Verified".
Proceed after "Initialing Automation for your Deployment".
Replicaset Processes Display Completed!
Goto Deployment > Security > MongoDB User.
Turn on "Enforce Consistent Set".
Confirm "Enforce Consistent Set".
Now, Lets start the step to enable TLS/SSL setting.
Please ensure you already have certs/pem install in your servers.
Goto Deployment > Security > Authentication & TLS/SSL.
Edit Setting and proceed to "Authentication Mechanisms" and Click "Next".
Enable TLS/SSL option.
Enter TLS/SSL CA File Path.
Switch "Client Certificate Mode" to "Require".
Enter PEM file for Automation, Backup and Monitoring Agent.
Next Click "Save".
Changes will shows as Enabled in TLS/SSL.
Next, to Ensure the TLS/SSL support enabled in the Mongo replicaset,
Goto Deployment > Processes. Select Replicaset Name and choose "Modify" setting.
Update the Following:
DB Directory Path Prefix = /data
bindIp = 0.0.0.0
sslMode = requireSSL
sslPEMKeyFile = /etc/ssl/certs/mongodb.pem
Then, click "Apply".
Now continue the previous step for the rest of the servers.
Mostly the update is just the following :
sslMode = requireSSL
sslPEMKeyFile = /etc/ssl/certs/mongodb.pem
You will see the icon changes in your replicaset during this process.
Save, Review, Confirm and Deploy.
Once Deploy is completed, you can double check the SSL/TLS changes by select a host and click the connect option to see example of connection command.
Click "Metric" to monitor all MongoDB Traffic/Usage.
Refer to "Data Explorer" for overall Data list.
Adding a New User. Click "Add New User".
Add the following.
Identitier: test (dbname)
username: user1
Roles: dbOwner
Password: xxxxxx
Click "Add User".
Once changes take effects. You can double check your changes in your cli.
Troubleshoot Slow Query by Checking "Real Time" and check slowest operation.
Also you can set log rotate from by your preference.
Finaly, you can remove the replicaset if you don't like and rebuild all over again.
Infrastructure-as-Code
A template can be used repeatedly to create identical copies of the same stack (or to use as a foundation to start a new stack). Templates are simple YAML- or JSON-formatted text files that can be placed under your normal source control mechanisms, stored in private or public locations such as Amazon S3, and exchanged via email. With CloudFormation, you can see exactly which AWS resources make up a stack. You retain full control and have the ability to modify any of the AWS resources created as part of a stack.
Self-documenting
Fed up with outdated documentation on your infrastructure or environments? Still keep manual documentation of IP ranges, security group rules, etc.?
With CloudFormation, your template becomes your documentation. Want to see exactly what you have deployed? Just look at your template. If you keep it in source control, then you can also look back at exactly which changes were made and by whom.
Intelligent updating & rollback
CloudFormation not only handles the initial deployment of your infrastructure and environments, but it can also manage the whole lifecycle, including future updates. During updates, you have fine-grained control and visibility over how changes are applied, using functionality such as change sets, rolling update policies and stack policies.
Add a new item to this list
If you found yourself wishing this set of frequently asked questions had an answer for a particular problem, please submit a pull request. The chances are that others will also benefit from having the answer listed here.
Contributing
Please create a new GitHub issue for any feature requests, bugs, or documentation improvements.
Where possible, please also submit a pull request for the change.
Author
Thinegan Ratnam
Copyright and License
Copyright 2018 Thinegan Ratnam
Code released under the MIT License.