Automated Deployment with Cargo - the Drive-By Example

"Automate Deployments" - this is one of the tenets of Continuous Integration as espoused by Martin Fowler in his seminal definition of what Continuous Integration is. The automation of deployments, in my experience, has tended to be decoupled from the build process - and for good reason. The mere act of building a piece of software doesn't necessarily warrant deploying it to some environment (Continuous Deployment apologists will disagree with me on this one). However, it may be useful to automatically deploy software to a specific environment on each build... let's say to a "developer sandbox" type of environment.

If your target deployment environment is a Java EE container, then you should be using Cargo for this. Cargo can be called directly through Java or can be invoked through build script plugins - there are plugins for ant and maven (1 and 2).

The following example shows how I've setup a maven project that can deploy a .war artifact immediately after building. This isn't ground breaking stuff, but useful to implement in case you aren't doing it already.

Without further ado, here are relevant parts of the pom.xml file:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
   <modelVersion>4.0.0</modelVersion>
   <groupId>com.jasonwhaley.sample</groupId>
   <artifactId>sample</artifactId>
   <packaging>war</packaging>
   <version>1.0-SNAPSHOT</version>
   <name>sample</name>
    <!-- snip -->
    <build>
        <plugins>
           <plugin>
               <groupId>org.codehaus.cargo</groupId>
               <artifactId>cargo-maven2-plugin</artifactId>
               <version>1.0</version>
               <configuration>
                   <container>
                       <containerId>tomcat6x</containerId>
                       <type>remote</type>
                   </container>
                   <configuration>
                       <type>runtime</type>
                       <properties>
                           <cargo.tomcat.manager.url>http://ec2-204-236-241-59.compute-1.amazonaws.com:8080/manager</cargo.tomcat.manager.url>
                           <cargo.remote.username>manager</cargo.remote.username>
                           <cargo.remote.password>manager</cargo.remote.password>
                       </properties>
                   </configuration>
               </configuration>
           </plugin>

            <!-- snip -->
       </plugins>
   </build>
</project>

Basically, by including the cargo plugin in the section you can issue several goals in your `mvn` invocation to deploy or undeploy your .war to the specific container listed (you can have multiple containers listed here, i've used only one for simplicities sake). For instance:

mvn clean install cargo:redeploy

which results in:

[INFO] [cargo:redeploy {execution: default-cli}]
[INFO] [mcat6xRemoteDeployer] Redeploying [/Users/whaley/Repositories/Personal/sample_ci/webapp/target/sample-1.0-SNAPSHOT.war]
[INFO] [mcat6xRemoteDeployer] Undeploying [/Users/whaley/Repositories/Personal/sample_ci/webapp/target/sample-1.0-SNAPSHOT.war]
[INFO] [mcat6xRemoteDeployer] Deploying [/Users/whaley/Repositories/Personal/sample_ci/webapp/target/sample-1.0-SNAPSHOT.war]

When it comes to continuous integration, what I prefer to do is setup one build project that builds and tests the .war through `mvn clean install`. Upon completion, I then start another project that does `mvn clean install cargo:redeploy`.

The reason for this separation is that your notification lists for the initial build project should be different than the notification list for deployment project. Developers should be the primary recipient of any notifications from the initial build. After that, any operations member responsible for the target container or development lead probably only need to know about notifications regarding the automated deployment project.

The advantage of orchestrating this through your CI server is that the process is visible to all. If someone notices that the latest commit isn't reflected on the target container (or if the target container doesn't have the build at all), then a quick peek at the CI server can show what step of the process failed simply by looking at the build status and more detail in the build logs.

Terminate All Of Your EC2 Instances In Three Fell Swoops

I am a forgetful person.

I am also a person who uses Amazon EC2 for a variety of purposes. I've helped migrate a startup from a colo with 4 machines to EC2 with tens of virtual instances in production. I also use EC2 is a bit of a playground from time to time in a separate account. I call these "lab" environments - which is something I actually consider a pattern (with a little "p") in terms of Continuous Integration... more on that in another post down the road.

With lab environments the goal is to quickly bring up a working set of servers to mimic a real environment somewhere else and allow a single developer or group of developers to toy around with the environment as needed and just throw it away once they are done. The course of the work, for me at least, hardly ever lasts more than an afternoon. EC2 makese this awfully easy to do by simply charging you a nominal hourly fee for spinning up some instances and letting you throw them away once you are done.

But since I'm quite forgetful, I often forget to destroy my instances, and don't realize it until I either get my monthly bill or do something on ec2 again with the account I have set aside for experimenting.

So one solution would be to put in a nightly cron a script that will kill off all of my instances. Python+boto makes writing this script easy:

import boto
ec2 = boto.connect_ec2(aws_access_key_id="foo", aws_secret_access_key="bar")
ec2.terminate_instances([reservation.instances[0].id for reservation in ec2.get_all_instances()])