Continuous Deployment tool
Mahesh Veerabathiran
AWS Certified Developer Associate
AWS Certified Sysops Administrator Associate
Agenda

Stages of Transformation

High level design overview

Significant features

Big picture - CICD Pipeline

Prerequisites

Demo

Tool information
Stages of Transformation

Developing API and deploy services majorly using
CFT and Chef since couple of years ago.
Stages of Transformation

Our initial pipeline uses following tools

Git – Used for Version controlling.

Jenkins – Runs code quality check and prepare artifacts
for deployment

Uses Sonar for code quality validation.

Stores artifact in to Nexus.

Provision cloud infrastructure using Cloudformation.

Infrastructure configuration is handled by Chef.

Using chef, one can build reusable libraries for infrastructure
configuration called Cookbooks.

Each cookbook can have more than one recipe.

Using Chef Role, one can define list of cookbook/recipe to
be executed in order as per their requirement.

Chef Environment – Let you define any environment specific
configuration for your requirement

Version of application artifact information is defined in chef
environment artifacts
Stages of Transformation

Maintenance of multiple CFT’s and Chef artifacts though
does not have major differences
1st API Stack
2nd API Stack
3rd API Stack
Cloudformation
Templates
Cloudformation
Templates
Cloudformation
Templates
Chef Artifacts
Chef Artifacts
Chef Artifacts
(Role, Env.)
(Role, Env.)
(Role, Env.)
Stages of Transformation

Built Generic CFT’s to maintain one set of CFT templates for all
our API’s
Stages of Transformation

But it did make our deployment complex, especially for
Deployment team as deployment is still manual.
ASG
ELB
Stages of Transformation

Expanded our API footprint further in West region for high
availability.

Enabled Blue/Green deployment for zero application
downtime.

Ended up spending more hours in deployment.
2x
ASG
ELB
for Feature deployment
4x
ASG
ELB
for AMI rehydration
Stages of Transformation

It complicated further our deployment process. Also, It
became difficult to get the release team availability.
Stages of Transformation

Realized the need to automate our releases and looked for
a solution to leverage automation solution using Chef and
CFT’s
Stages of Transformation

All the available automation tools are based on Terraform, Docker
etc. As our deadlines are narrow, could not spend time in new
tools.
Expectation
EC2
vs
Reality
Terraform
Stages of Transformation

Hence started coding simple script for automation and it
became Continuous deployment tool in a week time.
High level flow diagram
Config. file
Significant Features

All Cloudformation parameters are config. file driven
Significant Features
Artifact info. hardcoded
in Chef Env.
Significant Features
Now

API artifacts (*.war, *.jar) version are not hardcoded in CHEF.

Calls Nexus API to get latest artifact version & store artifacts info. in S3.

During Instance bootstrapping, pull down artifact info. from S3 and
deploy the artifacts using custom cookbooks.

Therefore, it eliminates the need for maintaining chef artifacts one for
blue and other for green.
Significant Features

Once the stack (blue/green) is created, the DNS recordset
to create/update R53 is prepared and stored in S3.

Configure cloud watch to monitor ELB instance health and
triggers SNS when at least one of the instance becomes
healthy

One could write lambda which subscribes SNS event. When
triggered, pull R53 update info. from S3 and update R53
DNS recordset to route traffic to new stack.
Significant Features

One could also trigger R53 update manually when application
traffic is low using automation job.

Sample R53 update JSON file.
Significant features
Big Picture - CICD Pipeline
Prerequisites

S3 bucket should be setup for staging temporary
information

AWS System account & Access keys.

System account should have read/write
permissions to your S3 bucket.

System account should also have following IAM
policies enabled


iam:PassRole

route53:ListResourceRecordSets

route53:ChangeResourceRecordSets
Need IAM role to access your S3 bucket from EC2
Demo
Questions
Tool information


GitHub repo

Deck:
https://github.com/dironman/continuousdeployme
nt-meetup.git

Sample config:
https://raw.githubusercontent.com/dironman/conti
nuousdeploymentmeetup/master/mortagage_loan_config.json
Contact me

https://www.linkedin.com/in/aboutmahesh