Quick reminder: My training courses with Diana Larsen, "The Art of Agile Planning" and "The Art of Agile Delivery," are coming up next week in Portland, Oregon.

I've talked about these courses before, so I won't do so again, although I do want to share a nice testimonial that came in a few weeks ago:

We sent developers to [The Art of Agile Delivery] last year and it was outstanding. We also sent several people including myself to [The Art of] Agile Planning, and we loved it. Both classes had a huge impact to how we work, even for those we weren?t able to send.

Don Dornblaser, Director, WebMD

Hearing that your course made a lasting impact, particularly to people who didn't even attend, is just about the highest praise I can get from training. Thanks, Don.

The courses are next week, so if you're planning to attend, now is your last chance to register. The courses are $1,395 for "The Art of Agile Planning" on June 8th and 9th, and $1,795 for "The Art of Agile Delivery" on June 10th, 11th, and 12th. There's a nice discounted rate of $2,495 if you attend both.

Register here.

Mike Hill has a lovely little essay titled "How TDD and Pairing Increase Production."

If you want more production, look first to raising your internal quality.

...

Do I really still need to rehearse this? All day long, every time you make a move, you will be depending on the code that?s already there. Every line of it you have to study will slow you down. Every extra open dependency will slow you down. Every bad variable name will slow you down. Every flawed design decision, be it big or small, will slow you down.

If you want to work as fast as you can, you want to work with clean code. Period.

There's a lot of truth in that essay. Go read it.

Quick training course reminder: Tomorrow is the early-bird registration deadline for The Art of Agile Planning (held June 8th & 9th in Portland, Oregon) and The Art of Agile Delivery (June 10-12). With the discount, the courses are $995 and $1,395 respectively, or $1,995 for both. Starting in May, the price will be $1,395 and $1,795, or $2,995 for both.

To get the best price, sign up now.

(Read about our approach to training in my essay, "Come Drink from the Firehose.")

I've previously mentioned that Diana Larsen and I are teaching two courses in June: The Art of Agile Planning and The Art of Agile Delivery. But I don't think I've mentioned what makes these courses unique. Diana and I share two core beliefs about training: we want to our students to have experiences, not lectures; and whenever possible, we want those to be real-world experiences rather than metaphors or simulations.

Experiences over Lectures

In The Art of Agile Planning, we're helping you understand how to plan Agile projects, both from an iteration/Sprint perspective, and from the larger release/organizational perspective. There are a lot of skills to learn, and one thing we know is that you can't build skills passively. You have to experience the thing being taught--do it--before it really sinks in.

Part of the reason is that "textbook" answers tend to strip out the messy complexity that occurs when you put work into practice. In real-world planning, there's tension and pressure. Different people want different things. There's a fear about what will happen if you don't get everything done by a certain time. Existing interpersonal friction is magnified.

You'll never learn this stuff by hearing a lecture. Most instructors don't even mention it. (Sadly, some teach from books, not experience, and don't know that they don't know it.)

So we do it. We have people practice planning, many times, and we intentionally put real tension and pressure into the mix by making sure the planning outcomes actually matter. Suddenly the stuff that seemed so easy when it was up on the board isn't as easy any more. Mistakes are made. And real learning happens.

Reality over Simulations

Hidden complications explain why we prefer real-world experiences over metaphorical simulations, too. Sure, you can experience aspects of Agile delivery by sketching a mousetrap on a flip chart. You'll learn about cross-functional teamwork, communication challenges, all kinds of good stuff.

But it's nothing compared to the experience of actually delivering working software in a time-boxed iteration. Version control. Flaky computers. Cross-platform compatibility. Strict deadlines. These are all part of reality, and they make the cross-functional teamwork, communication challenges, and so forth even more challenging.

In The Art of Agile Delivery, we do something that I'm exceedingly proud of: we have students define, develop, and deliver a complete software product in several 90-minute iterations. We make it as close to the real world as we possibly can, with things like on-site customers, version control, automated builds, and a third-party deployment environment.

As you can probably guess, it's hard. Everything I see on real software teams is repeated in miniature in that classroom. The last time we taught this course, some teams struggled with getting their computers working. Some tossed out the rigorous practices they had learned and started hacking. And some refused to acknowledge the deadline and scale down their ambitious plans. Nobody delivered software at the end of the first iteration. Nobody.

Sound like chaos? Sound painful? Yes. It was. That's what software development is like. And yet we have to deliver anyway.

And at the end of the second iteration, some teams did. By the fourth iteration, everyone did. Learning happened.

The Firehose

These core beliefs--experiences over lectures and reality over simulations--does mean that we expect a lot from our students. You'll be drinking from the firehose. You have to be willing to make mistakes, and learn from them. It's not for everybody.

We're okay with that. We figure the challenge will attract the kinds of students we want to teach. If that's you, come join us.

Early Bird Discount Expires Soon

We have a nice early-bird discount that expires in seven days, on April 30th, so you should register soon in order to get the best price. The Art of Agile Planning is on June 8th & 9th; with the discount, it's $995. The Art of Agile Delivery is on June 10th through 12th; it's $1,395. They go well together; the combined early bird rate is $1,995, a great deal.

Register here.

Q: What's the difference between an unemployed programmer and an entrepreneur?

A: You don't know either?

Kent Beck

The economy's down, and that sucks, but it's good news for innovation. People everywhere are creating startups. A lot of you will fail--some due to endless procrastination, some due to poor marketing, and some due to old fashioned bad luck. But a few of you will succeed. That's what makes it worthwhile.

I've worked with startups from all ends of the spectrum. Some had successfully cashed out; some with several rounds of funding, and some that were struggling to survive. Just recently, I've been working with Arlo Belshee and Kim Wallmark on our own startup. More about that in a later post.

From these experiences, I've learned a thing or two about software development in an entrepreneurial environment. This essay is the first of a series to help your startup succeed--or at least, not fail for software development reasons. Turning off World of Warcraft and getting down to work? That's your job.

Why Process Matters

My field is software process--specifically, Agile methods. One founder I talked to pooh-pooh'd the idea of process. "We don't have time for process," he said. "We need to focus on satisfying our customers."

That's a great attitude, really. And this founder was fortunate to have paying customers and actual profit. But the part about process? Totally wrong.

"Process" is nothing more than the way you do your work. Process improvement: improving the way you do your work. Agile processes are ways of working better. It's that simple. Don't be fooled by the suckers who buy into CMM Maturity Models (or, heaven forfend, "Agile" maturity models)--process improvement does not mean lots of bureaucracy, overhead, or expensive Certified anythings. It means doing your job better. That's what this series is about: doing your job better.

Enough talk. In this essay, we'll look at one way to make your work habits a little bit better.

Automate Your Release

Part of the secret to high performance software development is staying in the zone. Some people call it flow. It's that state of mind when you have everything clear in your head, know exactly what you're doing, and you've tuned out the rest of the world.

Flow is difficult to achieve and easy to lose. Any significant interruption pulls you out of the zone and all of those delicate mental structures come crashing down. Your productivity comes to a screeching halt as you deal with the interruption. When you're ready to get back to work, it takes a long time--fifteen minutes, according to Tom DeMarco and Timothy Lister--to get back in the zone.

(Pair programming is one way to reduce the cost of interruptions, but that's an essay for another day.)

One of the ways to kill flow is to have a clunky build and deployment process. Builds sort of grow organically, like fungus. An IDE build here, an FTP there, an ssh session there. Before you know it, building and deploying your app is a major hassle that takes half an hour on a good day, and you're lucky if only one thing breaks.

Prevent this disruption by creating an automated build and deployment script. It doesn't have to be perfect, but make a point of making tiny improvements as you identify pain points. One of the keys to process improvement is to realize that imperfect improvement is better than no improvement. Six months of imperfect improvements in a row leads to some pretty awesome results.

A Real-World Example

As I mentioned, I'm currently working on a startup with Arlo Belshee and Kim Wallmark. Here's the build and deploy script we wrote for our startup's first product. The version I'm about to show you is from about 12 days into development. The script isn't perfect--far from it--but it does allow us to release our software painlessly. We've improved it over time and we'll continue to do so.

First, here's our rakefile. (Rake is a Ruby-based build system that I like.) The rakefile is the master build for the system. To build and test, we type rake; to build, test, and deploy, we type rake ship. I've added a few comments to help you understand what's going on. The highlighted targets are the ones we run from the command-line.

There's a few things of note in this build. First, there's not much to it. A lot of the heavy lifting is done by batch files. That's an example of its imperfection. The automation was put together piecemeal over time, and there hasn't yet been a good enough reason to merge the files into the master build. As the build gets bigger, we'll have to improve the way we handle modularity, because the scripts will get unwieldy if we don't.

On the positive side, one of the best things we've done is to take the time and pain to get a localhost server working and automated. We type rake serve and we can run everything disconnected from the network. This allows us to build and test changes without breaking others' work. I highly recommend that you do this if you aren't already.

(Do localhost servers seem like a no-brainer to you? Good. You'd be surprised how many people don't do it.)

The Python build is a standard TurboGears build file, and I'm going to cover the automated tests in a later essay, so let's skip ahead to the automated deployment. When we type rake ship, the rakefile builds and tests, then calls ship.bat, a Windows batch file.

In the very beginning, we deployed by running Putty and manually telling it to copy files. That was good enough for about a day, and then we beat our heads against Windows for a while and figured out how to use rsync to copy our code to the server. Later, we added ssh public key authentication so we didn't have to type in our passwords, and then a little script to bounce the server for us.

There's still a lot to improve. A lot of the server configuration is still done manually on the server. That will hurt if something catastrophic happens to the server. We could also do a better job of building out our directory structure.

But overall, deploying is pretty painless. We've directed our attentions elsewhere for now. Our deployment isn't perfect, but it's good enough that it no longer interrupts flow.

The Relentless March of Imperfect Improvements

In a startup, you constantly have to balance delivering software with improving your capacity to deliver. The pressure of the situation tends to emphasize "deliver" over "improve." That's okay, as long as you don't go backwards. Many of the startups that purchase my consulting services have accumulated years of shortcuts, and it's left them with an inability to innovate. Changes to their code are so costly and bug-prone, they decide to throw it all away and start over. By this point, there's severe frustration with the dev team and discussions about outsourcing development entirely. Yikes. Don't do that.

Greatness emerges from a relentless march of imperfect improvements. A tiny improvement today enables a better improvement tomorrow, which enables even more improvement the next day and the day after that. Before you know it, you have more fun working in your "legacy" codebase than on brand-new code. It's possible! I've seen it.

As for us, it took about a half a day to figure out how to get rsync to play nice with Windows. Creating the rest of the build script wasn't free, either. Given that we have less than a month of development under our belts, I don't think we've recouped the cost yet. (It's hard to see all of the pain we've prevented, though.) Despite that cost, I think it was worth it. The real benefit is that we've offloaded all the thought and work around deployment. We don't have to interrupt flow to carefully consider all the issues. We just type rake ship and we're done.

If building and deploying your code is a distraction (or worse, if you've yet to do it), invest a bit of time in a good automated build. It doesn't have to be perfect. Just make it good enough so that you can stop worrying about shipping and get back to focusing on code.

Further Reading

Chromatic has a nice series of essays over at Modern Perl Books about how to release software:

1. Shooting Yourself in the Foot with Customer Branches
2. How to Ruin Your Ability to Release Software
3. The Rapid Release Tautology

Shane Warden and I write about automated builds in The Art of Agile Development. There's a summary at Ten Minute Build and additional commentary at Living in the Punch-Card Era.

Diana Larsen and I teach a hands-on Agile delivery course called The Art of Agile Delivery. In that course, you'll form a team to build a full software product from scratch, and you'll use build automation and other techniques to ship that product in 90-minute increments. It's pretty cool. Once you've successfully built and released software in a 90 minute cycle, shipping once a week is a doddle.

At the time of this writing, the next course is June 10-12, 2009. There's a nice early-bird discount if you register prior to April 30th.

It's a great time to purchase consulting and training. For many teams, business is slow, which means they have time to invest in improving their work habits. It's a cliché, but there really is value in doing more with less.

Business is slow for consultants, too, which means there are some great deals out there. I'm pricing my consulting services aggressively, and Diana Larsen and I have some great deals on our Art of Agile training courses. In May, we're offering The Art of Agile Teams: Designing Roles and Responsibilities at the government-subsidized price of $200. For our June courses, The Art of Agile Planning and The Art of Agile Delivery, we have some very nice early bird discounts if you register by April 30th, bringing the prices to $995 and $1,395 respectively.

A lot of businesses have cut way back on expenses, so they're not taking advantage of these opportunities. That's understandable. But the need hasn't gone away, so there's a backlog of pent-up demand forming. As the economy starts to improve, I expect to get more requests than I can fulfill. (It's the nature of independent consulting: too many customers, or too few. Sigh.) I'm already starting to see the signs.

If you're interested in consulting or training, now is a great time. There are some great deals available, and the best consultants' calendars are going to fill up quickly when the economy turns around.

There's been a lot said about certification lately. I think it's a natural outgrowth of Agile's growing popularity. Unfortunately, I think it's also an outgrowth of the continued watering-down of Agile, which--if we're not careful--will lead to Agile becoming irrelevant.

So although there has already been a fair amount of outcry about certification, I think it's important for the health of the industry that we all continue to argue against it.

Actually, I Do Provide Certification

Diana Larsen and I provide training courses based on The Art of Agile Development (in fact, we're teaching two of them in June), but we don't offer certification.

Actually, that's not entirely true. Diana and I do provide a "Certificate of Completion" for anybody who wants it. It says "This certificate of completion of 14 hours in workshop is hereby granted to So-and-So." Basically, if we remember seeing you at the breakfast buffet and you didn't snore too loudly the rest of the time, you get a certificate. Yay you. Some people use it for PMP credit, or to get reimbursed for the cost of training, or whatever.

Now, if I was feeling dishonest, I could give that certificate a fancy name. Rather than a fairly plain, unadvertised "certificate of completion," I could announce that attendees would be "Certified Rigorous Agile Practioners!" or "Certified Agile Artists" or some silly thing. The requirements wouldn't change (show up, don't snore), but the name would, and all of the sudden Diana and I would sell more courses. This is how the Certified ScrumMaster courses do it, and boy, does it work. Certified ScrumMaster courses cost several hundred dollars more than their counterparts and sell out quicker, too. All you need to do to get certified is show up.

(Speaking of selling out, our June courses are really good. Go forth and register.)

Why People Like Certification

Certification can't be all bad, can it? Those Certified ScrumMaster courses do cost more than their uncertified counterparts. Somebody must feel that premium is worth paying for.

Employers like certifications because it allows them to filter through the tons of resumes they get. It's an easy way to feel like they're hiring someone who has some basic minimum of competence.

Unfortunately, that sense of confidence is misplaced. I've seen no evidence that certification can do that. For example, the Project Management Institute's PMP (Project Management Professional) certification seems to have quite rigorous requirements--they require their PMPs to take ongoing education classes, have a certain amount of experience, and so on. And I'm sorry to say that, although I've known good PMPs, it's also true that the worst project managers I've met were PMPs who should never have been put in charge of a project. They were also the ones most proud of their certification, and most unaware of their deficiencies. I don't know what the PMP means, but it does not mean "basic minimum of competence."

Meanwhile, employees like certifications because it allows them to get noticed. It's an easy way to stand out amongst the piles of resumes.

These two things combine to lead to a vicious cycle: the more people there are with a certification, the more that certification gets visibility, and the more employers think it has meaning. Once employers think it has meaning, they start filtering out resumes that don't have the certification, leading more employees to obtain the certification so they can get hired.

Eventually, companies start making the certification a prerequisite for employment. One company I worked with required that all their project managers have a PMP certification. They also had the worst-run projects.

Escaping Mediocrity

These are great reasons to like certification... if you're interested in being mediocre. There's nothing in there about "superior training" or "proven ability." Far from it. No, certification isn't about ability. It's about the easy way out: for employers, an easy way to filter resumes; for employees, an easy way to get noticed. And one thing I know for sure is that people and companies who want the easy way out are not going to be great. That's fine for the mediocre, I suppose, but you can count me out. I'm interested in greatness.

Worse, the bigger Agile gets, the more opportunists enter the field. They can smell easy money, and certification looks like easy money. Perhaps that's why fly-by-night companies like [redacted--they'll get no publicity from me] and [scum-sucking bottom feeder] have recently offered agile certifications. These courses are taught by people with no experience and no ability. They'll disguise their incompetence with increasingly formal and glossy bullshit, and before you know it Agile will be gone in everything but name.

And that's why I don't provide Agile certification. It's a trap for the mediocre. I'm better than that, and I hope you are too.

From the beginning of the Agile movement, I've heard the question: "But does it scale?" And at first, I was interested in that question. "We're not sure yet," I would reply. "Here are the challenges of scaling Agile, and here's what we're doing about it." A few people listened intently, but others seemed to tune out.

Scaling Mediocrity

In sales, they teach that the first objection is often a false objection--misdirection rather than a real concern. "Does Agile scale?" can be one of these false objections.

I wasn't really being asked "Does Agile scale?" (By now, though, we know the answer: yes.) What I was really being asked was, "Does Agile work in large, dysfunctional organizations? Can I keep doing all of the ineffective things I'm required to do and still say I'm Agile? I can't have a co-located team--it's out of my control. I can't have active, involved product managers--they're too busy. I can't create cross-functional teams--it would disrupt our reporting hierarchy. How does Agile scale to my situation?"

It doesn't... at least, not well. Welcome to Mediocrity.

Pursuing Greatness

Agile involves (among other things) cross-functional teams, high-bandwidth communication, and adaptive planning. We know the best ways of achieving these things. The details vary, but when it comes to broad strategy, I don't think there's even any debate any more.

• We know that the most effective way for business experts, programmers, and testers to collaborate is to put them on a single team that shares responsibility and authority for success.

• We know that the most productive way for teams to work together is to sit together in a shared workspace and communicate primarily via ad-hoc face-to-face discussion.

• We know that the most successful way for teams to meet market needs is to adjust their plans as they go, forgoing detailed long-term plans in favor of rapid feedback and maximizing throughput and value.

We know this, and yet it rarely happens.

What's really amazing is that none of these things are mechanically difficult. Greatness--at least in software development--doesn't require great talent. Talent doesn't hurt, by any means, but I've seen ordinary people work together to create great teams. No, it's not talent that's needed. What's needed is will.

Sure, if you ask, everyone will tell you that they want to be great. But put the above list in front of them and they shrink back. Being great requires making waves. It requires taking risks. And it requires saying "no" to people who want to hear "yes." In the end, people value calm waters and safety more than they value being great.

And so it doesn't happen. Already, there are people lining up to write comments about how shared workspaces Just Aren't Practical in the "real world." In the "real world," they'll say, cross-functional teams can't work because of time, distance, and reporting constraints. These ideas would never be accepted.

My point exactly.

There are alternatives to these ideas that preserve the underlying ideals--for example, you can have high-bandwidth communication with a distributed team--but those alternatives are even more expensive and difficult than the basic scenarios I've outlined. And so many putatively Agile teams don't have cross-functional teams, high-bandwidth communication, or adaptive planning. They stumble through Mediocrity, blinding themselves with trivialities while ignoring the signposts that lead to Greatness.

Refining My Direction

I've been coaching teams in agile development for nearly ten years. (September is my ten-year anniversary.) Since the beginning, I've been fortunate to work with a lot of teams who wanted to be great, and I think I've been successful in helping many of them do so. In the last several years, though, Agile has become more popular. The market has grown, but the number of organizations willing to be great has shrunk. The emphasis has shifted from "be great" to "be Agile." And that's too bad. As much as I like it, there's really no point in Agile for the sake of Agile.

Worse, with so many organizations interested in "being Agile" rather than "being great," a whole industry is springing up around providing watered-down, non-threatening, non-boat-rocking (and non-functioning) Agile. If the point is to Be Agile, there's no need for Agile to actually work. Sell a certificate and walk away. Everyone's happy, right? Right?

Yuck.

Starting now, I'm reorienting my business to focus on people who want to be great. I hope there's enough of you still out there. Agile continues to be the best way I know to get to greatness, so that's what I'm using, but I'm no longer interested in helping people find the lowest-impact way to slap an Agile sticker on their door.

I want to work with people who want to be great. People who aren't satisfied just fitting in. People who are willing to take risks, rock the boat, and change their environment to maximize their productivity, throughput, and value. If that's you--particularly if you're in a product-focused, entrepreneurial environment--I want to hear from you. We can do great things together.

The Gordon Pask Award program is looking for volunteers. In case you're not familiar with it, the Gordon Pask Award for Contributions to Agile Practice is the premier (and only) award of the Agile Alliance. Every year, the award committee recognizes two people that they believe others in the field should emulate. It comes with a significant grant for travel to international conferences so that others may hear and speak with them.

Here's the call for volunteers:

Call for Volunteers

Gordon Pask Award Program

Laurent Bossavit and J. B. Rainsberger invite you to participate in the Gordon Pask Award program. We would like to improve the program in a number of ways, including collecting nominations throughout the year, researching candidates' contributions to the agile community and giving everyone a greater chance to get to know past winners. We need your help to do this well.

For the moment, we seek volunteers to help with work such as:

• advertise and promote our initiatives on weblogs, in discussion groups, in user groups, at conferences, and wherever you can garner attention
• design a physical Gordon Pask Award prize
• interview Brian Marick about the story behind instituting the Gordon Pask Award in 2005
• interview past award winners and write articles about them
• connect the program with public relations or press outlets to help promote the award and raise its profile in the larger software community
• suggest changes to the nomination process
• suggest something special for the fifth year of the award (2009)
• design a logo and color scheme for the Gordon Pask Award

If you would like to help, please send email to gordonpaskaward@agilealliance.org and tell us how you'd like to help. You might tell us you'd simply like to help in whatever capacity we need, or that you have a specific task or project in mind to which to contribute. We welcome all comers.

Thank you for your time, attention and interest.

Laurent Bossavit and J. B. Rainsberger,
Directors of the Gordon Pask Award Program

Rod Hilton at Absolutely No Machete Juggling has written a neat essay about pair programming from an in-the-trenches perspective. If you've been skeptical about pairing, take a look. It's not a sales job; it's the experience of someone who programs for a living. Worth reading.

Here's an excerpt:

The biggest challenge for me personally was essentially mourning for the death of "Programmer Man". Programmer Man is how I think of myself when I've got my headphones in, speed metal blaring in my ears, and I'm coding like a motherfucker. My fingers can't keep up with my brain. I'm In The Zone. For most of my career, this image is what I've considered to be the zenith. When I come home and was in Programmer Man Mode most of the day, I feel like I've had a good day.

Pair Programming undeniably killed Programmer Man. This was a tough adjustment, since I've considered that mode to be my favorite for so long. I now see, however, that Programmer Man was, without me knowing it, Technical Debt Man.

Read the whole essay here.

(Now I have the Spiderman theme running through my head. "Programmer Man, Programmer Man, writes the code, no one can; slams out debt, no one buys; watches the code grow in size. Look Out! Here comes the Programmer Man!")

I've finally caught up to the rest of the 21st century and joined Twitter. You can follow me as 'jamesshore'. I'm pleasantly surprised by how much I like it so far; I already feel more connected to the community. I used to get that sense of connection by following mailing lists, but for the last few years there's been way too much traffic for me to keep up with.

Now that I'm tweeting, I'd like to know more about my audience. I know that I can tweet any way I like, but I'd like to hear about your preferences. When you follow someone, do you like to hear about personal aspects of their life ("Kids were cute today. I love working from home!") or would you rather just see tweets that are more impersonal? How often do you like to see tweets?

I could also use some tips on how you use Twitter. I'm currently using Twitteriffic on MacOS and moTwit on PalmOS. Which tools do you prefer? Are there any hidden gems or tricks that I should know about? Who do you follow that I shouldn't miss?

You can leave your advice in the comments. Thanks!

Sebastian Hermida has put together something very cool: abetterteam.org. It's an online version of the "Assess Your Agility" quiz that Shane and I put in our book, and it's just beautifully done.

It's free. Try it out.

Note: abetterteam.org comes to us thanks to the hard work and initiative of Sebastian Hermida. My only involvement was to give my blessing and provide a few small suggestions. Please direct kudos and critiques to Sebastian.

There's been a bit of Scrum bashing going on lately. It started with my "Decline and Fall of Agile" essay last November and then reignited with Martin Fowler's recent "Flaccid Scrum" posting. InfoQ has a summary, if you want more links.

(Actually, bashing Scrum wasn't my intent. My real point can be found in the closing paragraphs of my essay: "It's human nature to only do the stuff that's familiar and fun, and that's what has happened with Agile." But I did call out Scrum and the Scrum Alliance for making the situation worse.)

A few people have risen to Scrum's defense. Tobias Mayer, commenting on my original essay, was particularly eloquent:

Listen. Scrum is not a methodology. It is not a process. It is a framework for surfacing organizational dysfunction. Scrum does not actually DO anything. People do things. The framework of Scrum, if taught effectively gives people the opportunity to wake up to themselves. If they don't they'll go out of business. Good. The others will know to make changes fast, and will likely call in good engineering consultants. Forget "Agile", these are just good software practices, they have been around long before the "Agile" label was slapped on them. Apart from a manifesto, Agile isn't actually anything.

Absolutely right.

So what's going on? Why are so many agile adoptions going wrong?

Kaizen

Kaizen means "continuous improvement." It's Japanese, and Lean, so it's a trendy word right now. But continuous improvement has been a core part of Agile since its inception. It's even the 12th principle in the manifesto ("At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly").

An Agile team should continuously look at what it's doing and strive to improve. Agile methods support this idea by exposing problems. Diana Larsen described it to me this way (I believe she was quoting Esther Derby): "It's like driving with a dirty windshield. Everything's fine until you turn into the setting sun. Then you can't see anything. You say, 'where did this dirt come from?' But it was there all along--the sun didn't create the dirt, it just made it visible.'"

So, yes, an Agile team should continuously inspect and adapt, and Agile gives you the tools to do so. And... there are some things that won't be visible to teams at first.

Will kaizen help these teams? Eventually. Does that mean we should hang them out to dry--wait for them to crash and burn, then hope they'll learn from their mistakes? Keep in mind that this is what we're seeing in the field: teams start out well, then struggle. It's good money for us consultants, but I'd rather people didn't get into that mess in the first place.

Kaikaku

Kaikaku is also Japanese, although it's not yet as trendy as kaizen. (Give it time.) It means "transformation." It's rapid change--introducing completely new approaches.

Now, in fairness, kaikaku is a form of kaizen. Kaizen doesn't necessarily mean gradual change, but that's the popular usage.

In popular usage, kaikaku is in contrast to kaizen. It's a rapid shift: hiring a coach to lead your team; reading a book and implementing its ideas as written; conducting a value-stream mapping workshop and making significant changes.

Where They Go Wrong

Now we come to where so many agile adoptions have gone wrong. (Not all of them, mind you. Not yours, certainly. But many.)

Agile adoptions go wrong in two ways.

1. First, teams apply kaizen within the constraints of their established habits and patterns. They make minor improvements, but miss out on transformative ideas.

2. Second, teams underestimate the human cost of change. When they do implement significant changes, they do so gradually and eventually burn out.

(Actually, there are plenty of other ways to go wrong, too, not least including organizational culture and management buy-in, but in this essay I'm looking at how teams put in new practices.)

Missing Out on Transformative Ideas

Jeffrey Liker says in The Toyota Way:

[The manager] says it's time to become team. So everyone piles into the conference room to work on improving productivity. What is likely to happen is that the team will focus on reducing the amount of time it takes to perform the value-added processes, the work they perform, or work on creature comforts like the lighting and putting in a water cooler. In the [example] process, the workers work individually, so it is natural that they focus only on their individual tasks.

Now let's consider the case of a TPS [Lean] expert coming in and analyzing the [example process]. The expert would immediately observe that there is no flow and that there is a great deal of waste. The first task of the TPS expert might be to improve the flow and eliminate most of the inventory that is getting in the way of tying together operations.

It's so hard for teams to change their work habits. They'll try, and they'll make changes, but without exposure to new ideas those changes will be incremental and evolutionary, not revolutionary. Waterfall teams will make smaller waterfalls. Document-centric teams will streamline their documents. Blame-oriented cultures will come up with more efficient ways of assigning culpability.

Some of the biggest opportunities in agile are transformative ideas. They're ideas like test-driven development, co-located and cross-functional teams, intensive customer participation, simultaneous phases, and a zero-defect mindset. These are also the most foreign ideas. Most putatively "Agile" teams aren't doing any of these things, and very few are doing all of them.

Some things can't be reached with gradual change.

Eventually Burning Out

In The Art of Agile Development, Shane and I wrote:

Discomfort and a feeling of chaos is normal for any team undergoing change, but that doesn't make it any less challenging. Expect the chaotic feeling to continue for at least two months. Give yourselves four to nine months to feel truly comfortable with your new process. If you're adopting XP incrementally, it will take longer.

...You can [adopt XP incrementally], but if you have a greenfield project, it's actually easier and faster to adopt all the practices at once. It's the chaos and uncertainty of change that makes adopting XP difficult, not the practices themselves. If you adopt XP incrementally, every new practice will disrupt the equilibrium you'll be fighting to achieve. You'll actually extend the period of chaos and uncertainty, making the transition all the more difficult.

You can introduce iterations (or Sprints), and then continuous integration, and then test-driven development, and then continuous incremental design, and then on-site customers, and then a shared workspace, and then pair programming, and then... or at least, you can try. Teams take years to get through this list, and it's exhausting. I don't know any that have made it. Really! None. (And before you tell me that they didn't need all those things, let me tell you that they didn't even try them. That's not kaizen; that's giving up.)

Or you can take about six months, give or take, and do it all at once. Gradual change is less scary, but it isn't easier or more successful.

Start with Kaikaku

Download

I've worked with a lot of Agile teams, and I've talked to many more Agile team members. The bottom line is this: the high performance Agile teams start with kaikaku. They start with something proven, say they're going to give it their best shot, and then... they really do.

They follow up with kaizen, continuously inspecting and adapting, asking why something they didn't like works, and why something they did like doesn't. They never stop learning or improving. But it starts with a transformation.

To achieve a high-performance Agile team, start with kaikaku. You'll need other things, too, like executive buy-in and bottom-up support. Those things are beyond the scope of this little essay. But once the conditions are in place to go Agile, use kaikaku. Make a big change. Hire a coach or do it by the book. From there, apply kaizen. Pay attention, be mindful, inspect and adapt.

Kaizen is important... and it's not the whole story. You need more than gradual change to get you where you need to be.

I've had the announcement up on my event calendar for a while now, but I haven't blogged it yet: in March, Diana Larsen and I are delivering our "Art of Agile" training courses in Europe.

Diana and I put these courses together last year, based on The Art of Agile Development. I'm biased, sure, but I think they came out great. We had a sold-out crowd of 40 people and we got a lot of positive feedback. There are two courses: The Art of Agile Planning and The Art of Agile Delivery.

We had a beautiful room last year.

The Art of Agile Planning

"The Art of Agile Planning" is a two-day course that covers all aspects of agile planning, from visions, to release planning and risk management, to iteration planning and estimating. Students use these skills over the course of multiple iterations as they plan an actual product--a miniature training course that we deliver.

If you've ever had trouble meeting the commitments you make to your stakeholders, you need to come to this class.

"The facilitators were excellent! I really enjoyed the 'jump in and swim' approach to applying what we learned as we went."

Bill Jackson III, Senior Software Engineer, Oracle Corp.

Diana demonstrates a concept.

The Art of Agile Delivery

"The Art of Agile Delivery" is a three-day course that does a deep dive into what happens during an agile iteration. It's an intensive course that replicates real agile development. We group people into cross-functional teams consisting of programmers, customers, and testers, and then we have them develop actual working software in several 90-minute iterations, complete with version control, continuous integration, and automated builds. (If you think that's impossible, you should come to our course!) Peppered throughout are lectures on topics as diverse as test-driven development, working with stakeholders, exploratory testing, and incremental design.

It's an intensive class, and one you shouldn't miss.

"I don't know how they pulled off the [class project], but going through four iterations brought the concepts home. Also I was a programmer wanting to learn about the project development side. Diana's four-quadrant diagrams (about stakeholders) were enlightening as was Jim's [incremental design] box diagrams and analogy of TDD to double-entry bookkeeping. Thank you!!"

Steve Tamura, Developer

Jim talks about customer testing.

How to Register

If you're in Europe and interested in one or both of these courses, I hope you'll join us! We're partnering with ProgramUtvikling in Norway to deliver the courses during the week of March 16th, and with MultiSoft in Hungary to deliver the courses during the week of March 30th. To register, follow the links below:

We work hard, and we have fun doing it.

It's that time of year again... I've updated my Essays Index with my best essays from the past year. I've also updated my list of most popular essays based on my traffic and readers' reviews.

I've also revamped the categories in the index page to make it easier for you to find what you're interested in. The new categories include "Adopting Agile," "Collaboration," "Coding," "Design," "Metrics," "Release Planning," and more. See the index for the full list.