Running Jenkins on AWS EC2 using CloudFormation (updated July 2017)

Running Jenkins on AWS EC2 using CloudFormation (updated July 2017)

How to get it to work.

Scroll Down for more info

Introduction:

Updated July 5th:
  • New content.
  • Updated CloudFormation template now emails you the temporary password, more robust deployment, allows you to specify IP to whitelist out of the box, and launches instances with Java 8 (necessary for Jenkins 2.60.1 installation).
  • Two new tutorial videos below (AWS account setup tutorial and launch tutorial) for exact instructions on how to get your own Jenkins instance setup in the cloud.

 

Download the Jenkins CloudFormation template here. Right Arrow

 


 

Setup work:
  1. Route53: you need to set this up so that Amazon hosts the DNS for a domain name. The CF template will ask for your domain name and the subdomain for the Jenkins host instance.
  2. S3: you need to create a S3 bucket to hold the Jenkins nightly (and before destruction) backup.
  3. VPC: you need to create a VPC (in the EC2 console).
  4. Security group: you need to create a new security group (in the EC2 console).



 

Launching instructions:

You are now ready to start your own Jenkins host from the CF template.

  1. Download the Jenkins template
  2. Go to the CloudFormation console and click “Create Stack.”
  3. Upload the template (as of October 17, 2016, it is the middle option: “Upload a template to Amazon S3”), proceeding to the next screen.
  4. Specify the details (located in the parameters section of the template file), and proceed to the next screen.
  5. Tag the instance if you wish, and proceed to the next screen.
  6. Verify the settings, check the acknowledgement, and click Create.
  7. It will show a detailed log of how the process is going.
Read why we transitioned from Bamboo Cloud to Jenkins here.

Background and Details

It is very important to build a system that is maintainable. Installing a piece of software only to have it go out of date (and thus insecure and vulnerable) is not good. The last thing I want is to have to maintain and keep up-to-date a platform that could turn into being a dinosaur.

While we could use any server to run Jenkins (the install process is super easy), the problem is how to keep Jenkins and its host up-to-date with feature and security updates. I have learned the hard way of setting up a server, letting it run for 7 years, and slowly watching it turn into a monster, consuming all hard-drive space, forcing me to log in periodically to free up disk space. Updates didn’t happen, and the OS that it was running (at the time of decease) was released over 10 years prior.

I have some experience with Linux and Amazon, so I turned to AWS to provide the infrastructure for this system.

CloudFormation is the solution of choice for the following reasons:

  • Elastic Beanstalk
    An incredibly attractive for deploying a web application, but I have not seen that it has the built-in capacity to install Jenkins.

  • Puppet / Chef / Ansible
    Super cool technologies, but would take too much time to set up. As a side note, we are investing in using Puppet for deploying Magento.

  • CloudFormation
    A very customizable solution that is easy to learn.

About the template:

You can find it here (updated July 4th, 2017). You should be able to load the template into the CloudFormation (CF) wizard, specify the options, and log into Jenkins.

I will provide some information about how it works. I took quite a few templates and Jenkins install instructions and merged them together for my liking. Feel free to do the same with this.

One thing to understand is that this template makes it possible to start two types of EC2 instances: the host (for running Jenkins) and child build nodes (for running build jobs). In the resources discussed below, most of the Jenkins host ones are prefixed with “Jenkins” and the child build nodes are prefixed with EC2Builder.

A CF template is a glorified JSON file. There are seven root nodes:

  • AWSTemplateFormatVersion: 2010-09-09
  • Description: a brief note about what this template does.
  • Parameters: user-specified details about the instance.
    • We are using them to specify the instance type, SSH key, DNS Zone, and other things. You can set up whatever parameter you want. To reference it later in the install script, you can use the {“ref”: “parameter-name”} function (see my slideshow here)
  • Mappings: AMI (Amazon Machine Identifier) mapping to the region.
  • Resources
    • CloudFormationLogs: log collector to push logs back to CloudFormation
    • JenkinsIamUser: the user that is associated with the Jenkins host instance
    • EC2BuilderIamUser: the user that is associated with the build instances that are spawned by the Jenkins host instance.
    • RolePolicies: the policies associated with these users
    • Ec2BuilderProfile: instance profile for build nodes
    • JenkinsHostKeys: the keys that will be used to access either the host or the build nodes
    • JenkinsServerGroup: an auto-scaling group for Jenkins. This is limited to 1 node only. We are using an auto-scaling group as this enables us to specify a launch configuration so that our Jenkins will re-appear if deleted (see my improvements section).
    • JenkinsLaunchConfig: where the magic happens. This huge node details how the host machine is started and configured.
    • LoadBalancerSecurityGroup: the firewall and permissions for the load balancer
    • EC2BuilderSecurityGroup: the firewall and permissions for the build instances
    • ElasticLoadBalancer: the load balancer for the auto-scaling group. In such an auto-scaling group, traffic is routed to any node in the group. The load balancer is what does this. This isn’t totally necessary considering the auto-scaler is limited to one node at a time. One other thing to note here is the “LoadBalancerPort” (what is visible to the Internet) is 80, whereas the “InstancePort” is 8080 (what is exposed from the Jenkins host instances—Jenkins runs on port 8080).
    • JenkinsDnsRecord: where you can access the Jenkins host.
    • WaitHandle
    • Outputs: this will output the address to access Jenkins

Improvements:

I haven’t found a way to have the Jenkins host rebuild itself every night. The issue is destroying the current host. With CloudFormation, it seems that specifying that the Shutdown Behavior is to terminate doesn’t work. At this point, it is backing up every night, but it is not destroying itself (to be rebuilt by the auto-scaling group).

To re-create the instance, I SSH into Jenkins host and run:

  • sudo ./etc/cron.daily/jenkins


In conclusion
  • Setting up Jenkins on EC2 with CloudFormation is a very easy process.
  • Jenkins has saved us many hours of time, thanks to automating deployment of code. It just works.
  • Using a repeatable system like this takes time, but in the end, it is very worth it. Consider it an investment.
  • If you have questions, feel free to contact me.
  • If you want to read why we transitioned to Jenkins, you can get that here.
SwiftOtter, Inc.
It relates to Continuous Integration, Developer Environment, Bamboo and Jenkins.
Joseph Maxwell - president / senior developer at Swift Otter

President / senior developer at SwiftOtter - @josephmaxs