James Shore

Aug 24th at Agile 2009: What Makes Agile Ours? (Panel)

Tue, 02 Jun 2009 00:04:00 PST

02 Jun 2009

James Shore/Calendar

Aaron Sanders is moderating the panel discussion "What Makes This Agile Ours? A Talk with Previous Gordon Pask Award Winners" at 11:00am on the Leadership & Teams stage of Agile 2009. I'll be there, as will Bob Payne, Arlo Belshee, Steve Freeman, Kenji Hiranabe, Laurent Bossavit, J. B. Rainsberger, and Jeff Patton. The panel has a neat iterative structure and should be a lot of fun.

Read More...

Comments

Aug 27th at Agile 2009: Replacing End-to-End Testing (Workshop)

Tue, 02 Jun 2009 00:03:00 PST

02 Jun 2009

James Shore/Calendar

Arlo Belshee and I are delivering "Slow and Brittle: Replacing End-to-End Testing" at 4:00pm Thursday on the Testing stage of Agile 2009. This should be a fun session: we're asking gurus to join us in a workshop format to figure out alternatives to end-to-end tests.

Read More...

Comments

Aug 27th at Agile 2009: Agile Metrics (Presentation)

Tue, 02 Jun 2009 00:02:00 PST

02 Jun 2009

James Shore/Calendar

I've teamed up with my buddy Rob Myers to deliver "Metrics in an Agile World" at 11:00am Thursday on the Leadership & Teams stage of Agile 2009. We're going to be looking at how metrics work with Agile, which ones are appropriate, and pitfalls to avoid.

Read More...

Comments

Aug 27th at Agile 2009: The Agile CTO (Workshop)

Tue, 02 Jun 2009 00:01:00 PST

02 Jun 2009

James Shore/Calendar

Diana Larsen and I are building on our CTO research to present "The Agile CTO" at Agile 2009. In this session, we'll be working with executives with Agile experience to discover and document patterns for Agile executives to use.

Read More...

Comments

Art of Agile Planning and Delivery Next Week

Mon, 01 Jun 2009 00:00:00 PST

01 Jun 2009

James Shore/Blog

Quick reminder: My training courses with Diana Larsen, "The Art of Agile Planning" and "The Art of Agile Delivery," are coming up next week in Portland, Oregon.

I've talked about these courses before, so I won't do so again, although I do want to share a nice testimonial that came in a few weeks ago:

We sent developers to [The Art of Agile Delivery] last year and it was outstanding. We also sent several people including myself to [The Art of] Agile Planning, and we loved it. Both classes had a huge impact to how we work, even for those we weren?t able to send.

Don Dornblaser, Director, WebMD

Hearing that your course made a lasting impact, particularly to people who didn't even attend, is just about the highest praise I can get from training. Thanks, Don.

The courses are next week, so if you're planning to attend, now is your last chance to register. The courses are $1,395 for "The Art of Agile Planning" on June 8th and 9th, and $1,795 for "The Art of Agile Delivery" on June 10th, 11th, and 12th. There's a nice discounted rate of $2,495 if you attend both.

Comments

How TDD and Pairing Increase Production

Tue, 26 May 2009 00:00:00 PST

26 May 2009

James Shore/Blog

Mike Hill has a lovely little essay titled "How TDD and Pairing Increase Production."

If you want more production, look first to raising your internal quality.

...

Do I really still need to rehearse this? All day long, every time you make a move, you will be depending on the code that?s already there. Every line of it you have to study will slow you down. Every extra open dependency will slow you down. Every bad variable name will slow you down. Every flawed design decision, be it big or small, will slow you down.

If you want to work as fast as you can, you want to work with clean code. Period.

There's a lot of truth in that essay. Go read it.

Comments

Agile Planning & Delivery: Last Chance for Early-Bird Registration

Wed, 29 Apr 2009 00:00:00 PST

29 Apr 2009

James Shore/Blog

Quick training course reminder: Tomorrow is the early-bird registration deadline for The Art of Agile Planning (held June 8th & 9th in Portland, Oregon) and The Art of Agile Delivery (June 10-12). With the discount, the courses are $995 and $1,395 respectively, or $1,995 for both. Starting in May, the price will be $1,395 and $1,795, or $2,995 for both.

To get the best price, sign up now.

(Read about our approach to training in my essay, "Come Drink from the Firehose.")

Comments

Come Drink from the Firehose

Thu, 23 Apr 2009 00:00:00 PST

23 Apr 2009

James Shore/Blog

I've previously mentioned that Diana Larsen and I are teaching two courses in June: The Art of Agile Planning and The Art of Agile Delivery. But I don't think I've mentioned what makes these courses unique. Diana and I share two core beliefs about training: we want to our students to have experiences, not lectures; and whenever possible, we want those to be real-world experiences rather than metaphors or simulations.

Experiences over Lectures

In The Art of Agile Planning, we're helping you understand how to plan Agile projects, both from an iteration/Sprint perspective, and from the larger release/organizational perspective. There are a lot of skills to learn, and one thing we know is that you can't build skills passively. You have to experience the thing being taught--do it--before it really sinks in.

Part of the reason is that "textbook" answers tend to strip out the messy complexity that occurs when you put work into practice. In real-world planning, there's tension and pressure. Different people want different things. There's a fear about what will happen if you don't get everything done by a certain time. Existing interpersonal friction is magnified.

You'll never learn this stuff by hearing a lecture. Most instructors don't even mention it. (Sadly, some teach from books, not experience, and don't know that they don't know it.)

So we do it. We have people practice planning, many times, and we intentionally put real tension and pressure into the mix by making sure the planning outcomes actually matter. Suddenly the stuff that seemed so easy when it was up on the board isn't as easy any more. Mistakes are made. And real learning happens.

Reality over Simulations

Hidden complications explain why we prefer real-world experiences over metaphorical simulations, too. Sure, you can experience aspects of Agile delivery by sketching a mousetrap on a flip chart. You'll learn about cross-functional teamwork, communication challenges, all kinds of good stuff.

But it's nothing compared to the experience of actually delivering working software in a time-boxed iteration. Version control. Flaky computers. Cross-platform compatibility. Strict deadlines. These are all part of reality, and they make the cross-functional teamwork, communication challenges, and so forth even more challenging.

In The Art of Agile Delivery, we do something that I'm exceedingly proud of: we have students define, develop, and deliver a complete software product in several 90-minute iterations. We make it as close to the real world as we possibly can, with things like on-site customers, version control, automated builds, and a third-party deployment environment.

As you can probably guess, it's hard. Everything I see on real software teams is repeated in miniature in that classroom. The last time we taught this course, some teams struggled with getting their computers working. Some tossed out the rigorous practices they had learned and started hacking. And some refused to acknowledge the deadline and scale down their ambitious plans. Nobody delivered software at the end of the first iteration. Nobody.

Sound like chaos? Sound painful? Yes. It was. That's what software development is like. And yet we have to deliver anyway.

And at the end of the second iteration, some teams did. By the fourth iteration, everyone did. Learning happened.

The Firehose

These core beliefs--experiences over lectures and reality over simulations--does mean that we expect a lot from our students. You'll be drinking from the firehose. You have to be willing to make mistakes, and learn from them. It's not for everybody.

We're okay with that. We figure the challenge will attract the kinds of students we want to teach. If that's you, come join us.

Early Bird Discount Expires Soon

We have a nice early-bird discount that expires in seven days, on April 30th, so you should register soon in order to get the best price. The Art of Agile Planning is on June 8th & 9th; with the discount, it's $995. The Art of Agile Delivery is on June 10th through 12th; it's $1,395. They go well together; the combined early bird rate is $1,995, a great deal.

Comments

Monthly in Portland, Oregon: Come Hang With the Agilists (Lunch)

Fri, 17 Apr 2009 00:01:00 PST

17 Apr 2009

James Shore/Calendar

The local Portland Agile community gets together every first Friday to eat, drink, and make merry. Oh, and sometimes we talk about agile stuff, too. Come join us! I don't always attend, but I come to as many sessions as my schedule permits. You can find us at the Broadway McMenamins (on the corner of NE Broadway and 15th) on the first Friday of every month at 1:00pm.

Comments

The Agile Startup: Build and Deploy

Tue, 07 Apr 2009 00:00:00 PST

07 Apr 2009

James Shore/Blog

Q: What's the difference between an unemployed programmer and an entrepreneur?

A: You don't know either?

Kent Beck

The economy's down, and that sucks, but it's good news for innovation. People everywhere are creating startups. A lot of you will fail--some due to endless procrastination, some due to poor marketing, and some due to old fashioned bad luck. But a few of you will succeed. That's what makes it worthwhile.

I've worked with startups from all ends of the spectrum. Some had successfully cashed out; some with several rounds of funding, and some that were struggling to survive. Just recently, I've been working with Arlo Belshee and Kim Wallmark on our own startup. More about that in a later post.

From these experiences, I've learned a thing or two about software development in an entrepreneurial environment. This essay is the first of a series to help your startup succeed--or at least, not fail for software development reasons. Turning off World of Warcraft and getting down to work? That's your job.

Why Process Matters

My field is software process--specifically, Agile methods. One founder I talked to pooh-pooh'd the idea of process. "We don't have time for process," he said. "We need to focus on satisfying our customers."

That's a great attitude, really. And this founder was fortunate to have paying customers and actual profit. But the part about process? Totally wrong.

"Process" is nothing more than the way you do your work. Process improvement: improving the way you do your work. Agile processes are ways of working better. It's that simple. Don't be fooled by the suckers who buy into CMM Maturity Models (or, heaven forfend, "Agile" maturity models)--process improvement does not mean lots of bureaucracy, overhead, or expensive Certified anythings. It means doing your job better. That's what this series is about: doing your job better.

Enough talk. In this essay, we'll look at one way to make your work habits a little bit better.

Automate Your Release

Part of the secret to high performance software development is staying in the zone. Some people call it flow. It's that state of mind when you have everything clear in your head, know exactly what you're doing, and you've tuned out the rest of the world.

Flow is difficult to achieve and easy to lose. Any significant interruption pulls you out of the zone and all of those delicate mental structures come crashing down. Your productivity comes to a screeching halt as you deal with the interruption. When you're ready to get back to work, it takes a long time--fifteen minutes, according to Tom DeMarco and Timothy Lister--to get back in the zone.

(Pair programming is one way to reduce the cost of interruptions, but that's an essay for another day.)

One of the ways to kill flow is to have a clunky build and deployment process. Builds sort of grow organically, like fungus. An IDE build here, an FTP there, an ssh session there. Before you know it, building and deploying your app is a major hassle that takes half an hour on a good day, and you're lucky if only one thing breaks.

Prevent this disruption by creating an automated build and deployment script. It doesn't have to be perfect, but make a point of making tiny improvements as you identify pain points. One of the keys to process improvement is to realize that imperfect improvement is better than no improvement. Six months of imperfect improvements in a row leads to some pretty awesome results.

A Real-World Example

As I mentioned, I'm currently working on a startup with Arlo Belshee and Kim Wallmark. Here's the build and deploy script we wrote for our startup's first product. The version I'm about to show you is from about 12 days into development. The script isn't perfect--far from it--but it does allow us to release our software painlessly. We've improved it over time and we'll continue to do so.

First, here's our rakefile. (Rake is a Ruby-based build system that I like.) The rakefile is the master build for the system. To build and test, we type rake; to build, test, and deploy, we type rake ship. I've added a few comments to help you understand what's going on. The highlighted targets are the ones we run from the command-line.

require 'qunit_runner.rb' # A runner we wrote for unit testing Javascript

task :default => :test # Run the 'test' target if no target provided

task :python do # Builds the python (server-side) code
   sh "cd app && python setup.py develop"
end

task :database_schema => :python do # Rebuilds the database schema
   touch "app/devdata.sqlite"
   rm "app/devdata.sqlite"
   sh "cd app && tg-admin sql create"
end

task :server_test do # Tests the server-side Python code
   sh "testserver"
end

task :client_test do # Tests the client-side Javascript code using our custom qunit_runner.rb
   run_qunit
end

desc "Run tests" # Used from command-line
task :test => [:server_test, :client_test]

desc "Start local server" # Used from command-line
task :serve => [:python, :database_schema] do
   sh "cd app && python start-product.py"
end

desc "Deploy to production server" # Used from command-line
task :ship => :test do
   sh "ship.bat"
end

There's a few things of note in this build. First, there's not much to it. A lot of the heavy lifting is done by batch files. That's an example of its imperfection. The automation was put together piecemeal over time, and there hasn't yet been a good enough reason to merge the files into the master build. As the build gets bigger, we'll have to improve the way we handle modularity, because the scripts will get unwieldy if we don't.

On the positive side, one of the best things we've done is to take the time and pain to get a localhost server working and automated. We type rake serve and we can run everything disconnected from the network. This allows us to build and test changes without breaking others' work. I highly recommend that you do this if you aren't already.

(Do localhost servers seem like a no-brainer to you? Good. You'd be surprised how many people don't do it.)

The Python build is a standard TurboGears build file, and I'm going to cover the automated tests in a later essay, so let's skip ahead to the automated deployment. When we type rake ship, the rakefile builds and tests, then calls ship.bat, a Windows batch file.

# Change backwards slashes into forward slashes so rsync works
set PRODUCT=%CD:~2,255%\app\
set PRODUCT=%PRODUCT:\=/%

set BIN_DIR="%CWRSYNC_LOCATION%\bin\"
if DEFINED CWRSYNC_LOCATION (
   pushd %BIN_DIR%
   # Copy the deployment tree to the server
   rsync.exe --recursive --perms --times --delete --force --delete-excluded --progress --exclude=.svn --exclude=.DS_Store --exclude=*.pyc --exclude=test --include=.* %PRODUCT% user@example.com:/home/example/app
   # Bounce the server so it sees the new code
   ssh user@example.com ~/bounce-webserver.sh
   popd
) else (
   # Tell us what environment variables we need to set
   echo Set the variable CWRSYNC_LOCATION TO THE directory cwRsync was installed in.
(Example: C:\Program Files\cwrsync)
)

In the very beginning, we deployed by running Putty and manually telling it to copy files. That was good enough for about a day, and then we beat our heads against Windows for a while and figured out how to use rsync to copy our code to the server. Later, we added ssh public key authentication so we didn't have to type in our passwords, and then a little script to bounce the server for us.

There's still a lot to improve. A lot of the server configuration is still done manually on the server. That will hurt if something catastrophic happens to the server. We could also do a better job of building out our directory structure.

But overall, deploying is pretty painless. We've directed our attentions elsewhere for now. Our deployment isn't perfect, but it's good enough that it no longer interrupts flow.

The Relentless March of Imperfect Improvements

In a startup, you constantly have to balance delivering software with improving your capacity to deliver. The pressure of the situation tends to emphasize "deliver" over "improve." That's okay, as long as you don't go backwards. Many of the startups that purchase my consulting services have accumulated years of shortcuts, and it's left them with an inability to innovate. Changes to their code are so costly and bug-prone, they decide to throw it all away and start over. By this point, there's severe frustration with the dev team and discussions about outsourcing development entirely. Yikes. Don't do that.

Greatness emerges from a relentless march of imperfect improvements. A tiny improvement today enables a better improvement tomorrow, which enables even more improvement the next day and the day after that. Before you know it, you have more fun working in your "legacy" codebase than on brand-new code. It's possible! I've seen it.

As for us, it took about a half a day to figure out how to get rsync to play nice with Windows. Creating the rest of the build script wasn't free, either. Given that we have less than a month of development under our belts, I don't think we've recouped the cost yet. (It's hard to see all of the pain we've prevented, though.) Despite that cost, I think it was worth it. The real benefit is that we've offloaded all the thought and work around deployment. We don't have to interrupt flow to carefully consider all the issues. We just type rake ship and we're done.

If building and deploying your code is a distraction (or worse, if you've yet to do it), invest a bit of time in a good automated build. It doesn't have to be perfect. Just make it good enough so that you can stop worrying about shipping and get back to focusing on code.

James Shore

Aug 24th at Agile 2009: What Makes Agile Ours? (Panel)

Aug 27th at Agile 2009: Replacing End-to-End Testing (Workshop)

Aug 27th at Agile 2009: Agile Metrics (Presentation)

Aug 27th at Agile 2009: The Agile CTO (Workshop)

Art of Agile Planning and Delivery Next Week

How TDD and Pairing Increase Production

Agile Planning & Delivery: Last Chance for Early-Bird Registration

Come Drink from the Firehose

Experiences over Lectures

Reality over Simulations

The Firehose

Early Bird Discount Expires Soon

Monthly in Portland, Oregon: Come Hang With the Agilists (Lunch)

The Agile Startup: Build and Deploy

Why Process Matters

Automate Your Release

A Real-World Example

The Relentless March of Imperfect Improvements

Further Reading