James Shore: The Agile Startup: Build and Deploy

The Agile Startup: Build and Deploy

April 6, 2009

Q: What's the difference between an unemployed programmer and an entrepreneur?

A: You don't know either?

Kent Beck

The economy's down, and that sucks, but it's good news for innovation. People everywhere are creating startups. A lot of you will fail--some due to endless procrastination, some due to poor marketing, and some due to old fashioned bad luck. But a few of you will succeed. That's what makes it worthwhile.

I've worked with startups from all ends of the spectrum. Some had successfully cashed out; some with several rounds of funding, and some that were struggling to survive. Just recently, I've been working with Arlo Belshee and Kim Wallmark on our own startup. More about that in a later post.

From these experiences, I've learned a thing or two about software development in an entrepreneurial environment. This essay is the first of a series to help your startup succeed--or at least, not fail for software development reasons. Turning off World of Warcraft and getting down to work? That's your job.

Why Process Matters

My field is software process--specifically, Agile methods. One founder I talked to pooh-pooh'd the idea of process. "We don't have time for process," he said. "We need to focus on satisfying our customers."

That's a great attitude, really. And this founder was fortunate to have paying customers and actual profit. But the part about process? Totally wrong.

"Process" is nothing more than the way you do your work. Process improvement: improving the way you do your work. Agile processes are ways of working better. It's that simple. Don't be fooled by the suckers who buy into CMM Maturity Models (or, heaven forfend, "Agile" maturity models)--process improvement does not mean lots of bureaucracy, overhead, or expensive Certified anythings. It means doing your job better. That's what this series is about: doing your job better.

Enough talk. In this essay, we'll look at one way to make your work habits a little bit better.

Automate Your Release

Part of the secret to high performance software development is staying in the zone. Some people call it flow. It's that state of mind when you have everything clear in your head, know exactly what you're doing, and you've tuned out the rest of the world.

Flow is difficult to achieve and easy to lose. Any significant interruption pulls you out of the zone and all of those delicate mental structures come crashing down. Your productivity comes to a screeching halt as you deal with the interruption. When you're ready to get back to work, it takes a long time--fifteen minutes, according to Tom DeMarco and Timothy Lister--to get back in the zone.

(Pair programming is one way to reduce the cost of interruptions, but that's an essay for another day.)

One of the ways to kill flow is to have a clunky build and deployment process. Builds sort of grow organically, like fungus. An IDE build here, an FTP there, an ssh session there. Before you know it, building and deploying your app is a major hassle that takes half an hour on a good day, and you're lucky if only one thing breaks.

Prevent this disruption by creating an automated build and deployment script. It doesn't have to be perfect, but make a point of making tiny improvements as you identify pain points. One of the keys to process improvement is to realize that imperfect improvement is better than no improvement. Six months of imperfect improvements in a row leads to some pretty awesome results.

A Real-World Example

As I mentioned, I'm currently working on a startup with Arlo Belshee and Kim Wallmark. Here's the build and deploy script we wrote for our startup's first product. The version I'm about to show you is from about 12 days into development. The script isn't perfect--far from it--but it does allow us to release our software painlessly. We've improved it over time and we'll continue to do so.

First, here's our rakefile. (Rake is a Ruby-based build system that I like.) The rakefile is the master build for the system. To build and test, we type rake; to build, test, and deploy, we type rake ship. I've added a few comments to help you understand what's going on. The highlighted targets are the ones we run from the command-line.

require 'qunit_runner.rb' # A runner we wrote for unit testing Javascript

task :default => :test # Run the 'test' target if no target provided

task :python do # Builds the python (server-side) code
   sh "cd app && python setup.py develop"
end

task :database_schema => :python do # Rebuilds the database schema
   touch "app/devdata.sqlite"
   rm "app/devdata.sqlite"
   sh "cd app && tg-admin sql create"
end

task :server_test do # Tests the server-side Python code
   sh "testserver"
end

task :client_test do # Tests the client-side Javascript code using our custom qunit_runner.rb
   run_qunit
end

desc "Run tests" # Used from command-line
task :test => [:server_test, :client_test]

desc "Start local server" # Used from command-line
task :serve => [:python, :database_schema] do
   sh "cd app && python start-product.py"
end

desc "Deploy to production server" # Used from command-line
task :ship => :test do
   sh "ship.bat"
end

There's a few things of note in this build. First, there's not much to it. A lot of the heavy lifting is done by batch files. That's an example of its imperfection. The automation was put together piecemeal over time, and there hasn't yet been a good enough reason to merge the files into the master build. As the build gets bigger, we'll have to improve the way we handle modularity, because the scripts will get unwieldy if we don't.

On the positive side, one of the best things we've done is to take the time and pain to get a localhost server working and automated. We type rake serve and we can run everything disconnected from the network. This allows us to build and test changes without breaking others' work. I highly recommend that you do this if you aren't already.

(Do localhost servers seem like a no-brainer to you? Good. You'd be surprised how many people don't do it.)

The Python build is a standard TurboGears build file, and I'm going to cover the automated tests in a later essay, so let's skip ahead to the automated deployment. When we type rake ship, the rakefile builds and tests, then calls ship.bat, a Windows batch file.

# Change backwards slashes into forward slashes so rsync works
set PRODUCT=%CD:~2,255%\app\
set PRODUCT=%PRODUCT:\=/%

set BIN_DIR="%CWRSYNC_LOCATION%\bin\"
if DEFINED CWRSYNC_LOCATION (
   pushd %BIN_DIR%
   # Copy the deployment tree to the server
   rsync.exe --recursive --perms --times --delete --force --delete-excluded --progress --exclude=.svn --exclude=.DS_Store --exclude=*.pyc --exclude=test --include=.* %PRODUCT% user@example.com:/home/example/app
   # Bounce the server so it sees the new code
   ssh user@example.com ~/bounce-webserver.sh
   popd
) else (
   # Tell us what environment variables we need to set
   echo Set the variable CWRSYNC_LOCATION TO THE directory cwRsync was installed in.
(Example: C:\Program Files\cwrsync)
)

In the very beginning, we deployed by running Putty and manually telling it to copy files. That was good enough for about a day, and then we beat our heads against Windows for a while and figured out how to use rsync to copy our code to the server. Later, we added ssh public key authentication so we didn't have to type in our passwords, and then a little script to bounce the server for us.

There's still a lot to improve. A lot of the server configuration is still done manually on the server. That will hurt if something catastrophic happens to the server. We could also do a better job of building out our directory structure.

But overall, deploying is pretty painless. We've directed our attentions elsewhere for now. Our deployment isn't perfect, but it's good enough that it no longer interrupts flow.

The Relentless March of Imperfect Improvements

In a startup, you constantly have to balance delivering software with improving your capacity to deliver. The pressure of the situation tends to emphasize "deliver" over "improve." That's okay, as long as you don't go backwards. Many of the startups that purchase my consulting services have accumulated years of shortcuts, and it's left them with an inability to innovate. Changes to their code are so costly and bug-prone, they decide to throw it all away and start over. By this point, there's severe frustration with the dev team and discussions about outsourcing development entirely. Yikes. Don't do that.

Greatness emerges from a relentless march of imperfect improvements. A tiny improvement today enables a better improvement tomorrow, which enables even more improvement the next day and the day after that. Before you know it, you have more fun working in your "legacy" codebase than on brand-new code. It's possible! I've seen it.

As for us, it took about a half a day to figure out how to get rsync to play nice with Windows. Creating the rest of the build script wasn't free, either. Given that we have less than a month of development under our belts, I don't think we've recouped the cost yet. (It's hard to see all of the pain we've prevented, though.) Despite that cost, I think it was worth it. The real benefit is that we've offloaded all the thought and work around deployment. We don't have to interrupt flow to carefully consider all the issues. We just type rake ship and we're done.

If building and deploying your code is a distraction (or worse, if you've yet to do it), invest a bit of time in a good automated build. It doesn't have to be perfect. Just make it good enough so that you can stop worrying about shipping and get back to focusing on code.

The Art of Agile^SM