There's been a bit of Scrum bashing going on lately. It started with my "Decline and Fall of Agile" essay last November and then reignited with Martin Fowler's recent "Flaccid Scrum" posting. InfoQ has a summary, if you want more links.

(Actually, bashing Scrum wasn't my intent. My real point can be found in the closing paragraphs of my essay: "It's human nature to only do the stuff that's familiar and fun, and that's what has happened with Agile." But I did call out Scrum and the Scrum Alliance for making the situation worse.)

A few people have risen to Scrum's defense. Tobias Mayer, commenting on my original essay, was particularly eloquent:

Listen. Scrum is not a methodology. It is not a process. It is a framework for surfacing organizational dysfunction. Scrum does not actually DO anything. People do things. The framework of Scrum, if taught effectively gives people the opportunity to wake up to themselves. If they don't they'll go out of business. Good. The others will know to make changes fast, and will likely call in good engineering consultants. Forget "Agile", these are just good software practices, they have been around long before the "Agile" label was slapped on them. Apart from a manifesto, Agile isn't actually anything.

Absolutely right.

So what's going on? Why are so many agile adoptions going wrong?

Kaizen

Kaizen means "continuous improvement." It's Japanese, and Lean, so it's a trendy word right now. But continuous improvement has been a core part of Agile since its inception. It's even the 12th principle in the manifesto ("At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly").

An Agile team should continuously look at what it's doing and strive to improve. Agile methods support this idea by exposing problems. Diana Larsen described it to me this way (I believe she was quoting Esther Derby): "It's like driving with a dirty windshield. Everything's fine until you turn into the setting sun. Then you can't see anything. You say, 'where did this dirt come from?' But it was there all along--the sun didn't create the dirt, it just made it visible."

So, yes, an Agile team should continuously inspect and adapt, and Agile gives you the tools to do so. And... there are some things that won't be visible to teams at first.

Will kaizen help these teams? Eventually. Does that mean we should hang them out to dry--wait for them to crash and burn, then hope they'll learn from their mistakes? Keep in mind that this is what we're seeing in the field: teams start out well, then struggle. It's good money for us consultants, but I'd rather people didn't get into that mess in the first place.

Kaikaku

Kaikaku is also Japanese, although it's not yet as trendy as kaizen. (Give it time.) It means "transformation." It's rapid change--introducing completely new approaches.

Now, in fairness, kaikaku is a form of kaizen. Kaizen doesn't necessarily mean gradual change, but that's the popular usage.

In popular usage, kaikaku is in contrast to kaizen. It's a rapid shift: hiring a coach to lead your team; reading a book and implementing its ideas as written; conducting a value-stream mapping workshop and making significant changes.

Where They Go Wrong

Now we come to where so many agile adoptions have gone wrong. (Not all of them, mind you. Not yours, certainly. But many.)

Agile adoptions go wrong in two ways.

1. First, teams apply kaizen within the constraints of their established habits and patterns. They make minor improvements, but miss out on transformative ideas.

2. Second, teams underestimate the human cost of change. When they do implement significant changes, they do so gradually and eventually burn out.

(Actually, there are plenty of other ways to go wrong, too, not least including organizational culture and management buy-in, but in this essay I'm looking at how teams put in new practices.)

Missing Out on Transformative Ideas

Jeffrey Liker says in The Toyota Way:

[The manager] says it's time to become team. So everyone piles into the conference room to work on improving productivity. What is likely to happen is that the team will focus on reducing the amount of time it takes to perform the value-added processes, the work they perform, or work on creature comforts like the lighting and putting in a water cooler. In the [example] process, the workers work individually, so it is natural that they focus only on their individual tasks.

Now let's consider the case of a TPS [Lean] expert coming in and analyzing the [example process]. The expert would immediately observe that there is no flow and that there is a great deal of waste. The first task of the TPS expert might be to improve the flow and eliminate most of the inventory that is getting in the way of tying together operations.

It's so hard for teams to change their work habits. They'll try, and they'll make changes, but without exposure to new ideas those changes will be incremental and evolutionary, not revolutionary. Waterfall teams will make smaller waterfalls. Document-centric teams will streamline their documents. Blame-oriented cultures will come up with more efficient ways of assigning culpability.

Some of the biggest opportunities in agile are transformative ideas. They're ideas like test-driven development, co-located and cross-functional teams, intensive customer participation, simultaneous phases, and a zero-defect mindset. These are also the most foreign ideas. Most putatively "Agile" teams aren't doing any of these things, and very few are doing all of them.

Some things can't be reached with gradual change.

Eventually Burning Out

In The Art of Agile Development, Shane and I wrote:

Discomfort and a feeling of chaos is normal for any team undergoing change, but that doesn't make it any less challenging. Expect the chaotic feeling to continue for at least two months. Give yourselves four to nine months to feel truly comfortable with your new process. If you're adopting XP incrementally, it will take longer.

...You can [adopt XP incrementally], but if you have a greenfield project, it's actually easier and faster to adopt all the practices at once. It's the chaos and uncertainty of change that makes adopting XP difficult, not the practices themselves. If you adopt XP incrementally, every new practice will disrupt the equilibrium you'll be fighting to achieve. You'll actually extend the period of chaos and uncertainty, making the transition all the more difficult.

You can introduce iterations (or Sprints), and then continuous integration, and then test-driven development, and then continuous incremental design, and then on-site customers, and then a shared workspace, and then pair programming, and then... or at least, you can try. Teams take years to get through this list, and it's exhausting. I don't know any that have made it. Really! None. (And before you tell me that they didn't need all those things, let me tell you that they didn't even try them. That's not kaizen; that's giving up.)

Or you can take about six months, give or take, and do it all at once. Gradual change is less scary, but it isn't easier or more successful.

Start with Kaikaku

I've worked with a lot of Agile teams, and I've talked to many more Agile team members. The bottom line is this: the high performance Agile teams start with kaikaku. They start with something proven, say they're going to give it their best shot, and then... they really do.

They follow up with kaizen, continuously inspecting and adapting, asking why something they didn't like works, and why something they did like doesn't. They never stop learning or improving. But it starts with a transformation.

To achieve a high-performance Agile team, start with kaikaku. You'll need other things, too, like executive buy-in and bottom-up support. Those things are beyond the scope of this little essay. But once the conditions are in place to go Agile, use kaikaku. Make a big change. Hire a coach or do it by the book. From there, apply kaizen. Pay attention, be mindful, inspect and adapt.

Kaizen is important... and it's not the whole story. You need more than gradual change to get you where you need to be.