James Shore

in 99 words

Rather than fixing bugs, agile methods strive to prevent them.

Test-driven development structures work into easily-verifiable steps. Pair programming provides instant peer review, enhances brainpower, and maintains self-discipline. Energized work reduces silly mistakes. Coding standards and a "done done" checklist catch common errors.

On-site customers clarify requirements and discover misunderstandings. Customer tests communicate complicated domain rules. Iteration demos allow stakeholders to correct the team's course.

Simple design, refactoring, slack, collective code ownership, and fixing bugs early eliminates bug breeding grounds. Exploratory testing discovers teams' blind spots, and root-cause analysis allows teams to eliminate them.

Inside The Book

• No Bugs
• How Is This Possible?
• How to Achieve Nearly Zero Bugs
• Ingredient #1: Write Fewer Bugs
• Ingredient #2: Eliminate Bug Breeding Grounds
• Ingredient #3: Fix Bugs Now
• Sidebar: Don't Play the Bug Blame Game
• Ingredient #4: Test Your Process
• Ingredient #5: Fix Your Process
• Sidebar: Making Mistakes Impossible
• Invert Your Expectations
• Questions
  • If novices can do this, what's stopping management from firing everybody and hiring novices?
  • How do we prevent security defects and other challenging bugs?
  • How long should we spend working on a bug before we convert it to a story?
  • If our guideline is to fix bugs as soon as we find them, won't we have the unconscious temptation to overlook bugs?
  • If we don't find any bugs, how do we know that our testers are doing exploratory testing correctly?
  • We have a large amount of legacy code. How can we adopt this policy without going mad?
• Results
• Contraindications
• Alternatives
• Further Reading

Behind the Scenes

This may be my favorite section of the book. It describes, in clear and pragmatic terms, exactly how all of the practices in the book fit together to achieve high quality and low defects.

It didn't start out so strong. Actually, it was terrible. The practice was originally titled "No Bug Database." We got more flack on this section than any other. At one point, the Joel on Software forums picked up on the material, and flamed us thoroughly. (In three parts: one, two, three.)

It took a little while to get over the singed egos, but once we did, we were able to use the feedback to create a much stronger result. We still got flamed in the forums, but they went from "you're obviously stupid," to "I'm not convinced." By Internet forum standards, that's high praise.

I think I've mentioned before that we had an open review process, with every section of the book posted on the web for public comment. (We ended up with over 1,000 messages on our review mailing list.) This section is a great example of how important that review process was to the quality of the final book. To illustrate, I'd like to share our initial draft so you can compare it with the final.

First, here's the version from our final manuscript. You'll notice minor differences between this draft and the published book as a result of the production editing process.

---

No Bugs

Audience

Whole Team

We confidently release without a dedicated testing phase.

Let's cook up a bug pie. First, start with a nice, challenging language; does C sound okay? We'll season it with a dash of assembly.

Next, add extra bugs by mixing in concurrent programming. Our old friends Safety and Liveness are happy to fight each other over who provides the most bugs. They supplied the Java multithreading library with bugs for years!

Now we need a really difficult problem domain. How about real-time embedded systems?

To top off this disaster cocktail, we need the right kind of programmers. Let's see... experts, senior developers... no... ah ha! Novices! Just what we need.

Take your ingredients—C, real-time embedded systems, multitasking... don't forget the novices... add a little assembly for seasoning, shake well, and bake for three years. (I do love me a bug pie.)

Let's see how it turned out.

The GMS team delivered this product after three years of development [60,638 lines of code], having encountered a total of 51 defects during that time. The open bug list never had more than two items at a time. Productivity was measured at almost 3 times the level for comparable embedded software teams. The first field test units were delivered after approximately six months into development. After that point, the software team supported the other engineering disciplines while continuing to do software enhancements.

—"Embedded Agile Project By the Numbers with Newbies" [Van Schooenderwoert]

These folks had everything stacked against them—except their coach and her approach to software development. If they can do it, so can you.

How Is This Possible?

If you're on a team with a bug-count in the hundreds, the idea of "no bugs" probably sounds ridiculous. I'll admit: "no bugs" is an ideal to strive for, not something that your team will necessarily achieve.

However, XP teams can achieve dramatically lower bug rates. [Van Schooenderwoert]'s team averaged one and a half bugs per month in a very difficult domain. In an independent analysis of a company practicing a variant of XP, QSM Associates reported an average reduction from 2,270 defects to 381 defects [Mah].

You might think that improvements like this are terribly expensive. They're not. In fact, agile teams tend to have above-average productivity.¹

¹See, for example, [Van Schooenderwoert], [Mah], and [Anderson].

Evidence for these results is as yet anecdotal. Rigorous scientific studies of complete software development methodologies are rare due to the large number of variables involved. While there are organizations that draw performance conclusions by aggregating hundreds of projects, none that I am aware of have enough data to draw conclusions on agile projects in general, let alone XP specifically. (QSM Associates is a well-regarded example of such an organization; as of June 2006, they only had data from a few agile projects.²)

²Personal communication with Michael Mah of QSM Associates.

In the absence of conclusive proof, how can you know if your team will achieve these results? There's only one way to know for sure: try it and see. It doesn't take super-powers. Teams of novices coached by an experienced developer have done this. All you need is commitment to follow the XP practices rigorously and support from your organization to do so.

How to Achieve Nearly Zero Bugs

Many approaches to improving software quality revolve around finding and removing more defects³ through traditional testing, inspection, and automated analysis.

³I use "defect" synonymously with "bug".

The agile approach is to generate fewer defects. This isn't a matter of finding defects earlier; it's a question of not generating them at all.

For example, [Van Schooenderwoert] delivered 21 bugs to customers. Working from Capers Jones' data, Van Schooenderwoert says that a "best in class" team building their software would have generated 460 defects, found 95% of them, and delivered 23 to their customer.⁴ In contrast, Van Schooenderwoert's team generated 51 defects, found 59% of them, and delivered 21 to their customer. At 0.22 defects per function point, this result far exceeds Capers Jones' "best in class" expectation of two defects per function point.

⁴An "average" team would have generated 1,035, found 80%, and delivered 207.

To achieve these results, XP uses a potent cocktail of techniques:

1. Write fewer bugs by using a wide variety of technical and organizational practices.

2. Eliminate bug breeding grounds by refactoring poorly-designed code.

3. Fix bugs quickly to reduce their impact, writing tests to prevent them from reoccurring, then fix the associated design flaws that are likely to breed more bugs.

4. Test your process by using exploratory testing to expose systemic problems and hidden assumptions.

5. Fix your process by uncovering categories of mistakes and making those mistakes impossible.

This may seem like a lot to do, but most of it comes naturally as part of the XP process. Most of these activities improve productivity by increasing code quality or removing obstacles. If you do them as part of XP, you don't have to do many of the other, more expensive activities that other teams perform: you don't need to perform an exhaustive up-front requirements gathering phase, you don't need to interrupt the team for code inspections, and of course you don't need a separate bug-fixing phase.

Ingredient #1: Write Fewer Bugs

Don't worry—I'm not going to wave my hands and say, "Too many bugs? No problem! Just write fewer bugs!" To stop writing bugs, you have to take a rigorous, thoughtful approach to software development.

Ally

Test-Driven Development, §X.Y

Start with test-driven development (TDD). TDD is a proven technique for reducing the number of defects you generate [Janzen & Saiedian]. It leads to a comprehensive suite of unit and integration tests and, perhaps more importantly, structures your work into small, easily-verifiable steps. Teams using TDD report that they rarely need to use a debugger.

Allies

Energized Work, §X.Y

Pair Programming, §X.Y

To enhance the benefits of test-driven development, work sensible hours and program all production code in pairs. Both of these techniques enhance your brainpower, helping you make fewer mistakes and allowing you to see mistakes more quickly. Pair programming also provides positive peer pressure, which helps you maintain the self-discipline you need to follow defect-reduction practices.

Allies

Whole Team, §X.Y

Sit Together, §X.Y

Customer Tests, §X.Y

Exploratory Testing, §X.Y

Iteration Demo, §X.Y

Test-driven development is a powerful technique for eliminating coding defects, but code isn't your only source of defects. Software teams also produce good code that does the wrong thing. To prevent these requirements-oriented defects, work closely with your stakeholders. Enlist on-site customers to sit with your team. Use customer tests to help communicate complicated domain rules. Have testers work with customers to find and fix gaps in their approach to requirements. Demonstrate your software to stakeholders every week and act on their feedback.

Allies

Coding Standards, §X.Y

"Done Done", §X.Y

Supplement these practices with good coding standards and a "done done" checklist. These will help you remember and avoid common mistakes.

Ingredient #2: Eliminate Bug Breeding Grounds

Writing fewer bugs is an important first step to reducing the number of defects your team generates. If you accomplish that much, you're well ahead of most teams. Don't stop now, though. You can generate even fewer defects.

Even with test-driven development, your software will accumulate technical debt over time. Most of it will be in your design, making your code defect-prone and difficult to change, and it will tend to congregate in specific parts of the system. According to [Boehm], about 20% of the modules in a program are typically responsible for about 80% of the errors.

These design flaws are unavoidable. Sometimes a design that looks good when you first create it won't hold up over time. Sometimes a shortcut that seemed like an acceptable compromise will come back and bite you. Sometimes your requirements will change and your design will need to change as well.

Technical debt breeds bugs.

Whatever its cause, technical debt leads to complicated, confusing code that's hard to get right. It breeds bugs. To generate fewer defects, pay down your debt.

Allies

Simple Design, §X.Y

Refactoring, §X.Y

Slack, §X.Y

Although you could dedicate a week or two to fixing these problems, the best way to pay off your debt is to make small improvements every week. Keep new code clean by creating simple designs and refactoring as you go. Use the slack in each iteration to pay down debt in old code.

Ingredient #3: Fix Bugs Now

Programmers have long known that the longer you wait to fix a bug, the more it costs to fix [McConnell, p.75]. In addition, unfixed bugs probably indicate further problems. Each bug is the result of a flaw in your system that's likely to breed more mistakes. Fix it now and you'll improve both quality and productivity.

To fix the bug, start by writing an automated test that demonstrates the bug. It could be a unit test, integration test, or customer test, depending on what kind of defect you've found. Once you have a failing test, fix the bug. Get a green bar.

Don't congratulate yourself yet, though. You've fixed the problem, but you haven't solved the underlying cause. Why did that bug occur? Discuss the code with your pairing partner. Is there a design flaw that made this bug possible? Can you change an API to make such bugs more obvious? Is there some way to refactor the code that would make this kind of bug less likely? Improve your design. If you've identified a systemic problem, discuss it with the rest of your team in your next stand-up meeting or iteration retrospective. Tell people what went wrong so they can avoid that mistake in the future.

Allies

Whole Team, §X.Y

Collective Code Ownership, §X.Y

Sit Together, §X.Y

Fixing bugs quickly requires the whole team to participate. Programmers, use collective code ownership so any pair can fix a buggy module. Customers and testers, personally bring new bugs to the attention of a programmer and help them reproduce it. These actions are easiest when the whole team sits together.

In practice, it's not possible to fix every bug right away. When you find a bug, you may be in the middle of working on something else. When this happens to me, I ask my navigator to write the problem on our to-do list. We come back to it in ten or twenty minutes, when we come to a good stopping point.

Ally

Slack, §X.Y

Some bugs are too big to fix while you're in the middle of another task. For these, I write the bug on a story card and announce it to the whole team. (Some teams use red story cards for this purpose.) We collectively decide if there's enough slack in the iteration to fix the bug and still meet our other commitments. If there is, we create tasks for the newly-created story and pairs volunteer for them as normal. Sometimes your only task will be "fix the bug". I use the story card as its own task card when this happens.

If there isn't enough slack to fix the bug, estimate the cost to fix it and ask your product manager to decide whether to fix it in this release. If it's important enough to fix, schedule it into the very next iteration.

Although you may wish to fix every bug right away, you need to consider its cost and value before doing so. Some bugs are expensive to fix but have limited impact; these are often not worth fixing. However, because bugs generally become more expensive to fix over time, you should typically only choose between "fix" (as soon as possible) or "don't fix" (until a later release).

Allies

Whole Team, §X.Y

"Done Done", §X.Y

Don't Play the Bug Blame Game

FEATURE

—License plate seen on the back of a Volkswagon Beetle

If you tell a programmer that there's a bug in his software, you're likely to get a prickly response. "That's not a bug, it's a feature!" or, "What did you do wrong?"

I suspect that this is because "bug" usually means "you screwed up"⁵. Deciding whether something is a bug or not often seems to be a fancy approach to finger-pointing. Some folks have taken this to a high art, making elaborate distinctions between bugs, defects, faults, errors, issues, anomalies, and of course, unintentional features. I prefer to avoid formal categorizations. What really matters is whether you will do or not do something, regardless of whether that thing is a fixing a bug or implementing a feature.

Mistakes do occur, of course, and we want to prevent those mistakes. Pointing fingers, though, is counter-productive. The whole team—on-site customers, testers, and programmers—is responsible for delivering valuable software. Regardless of who made the mistake, the whole team is responsible for preventing it from reaching stakeholders.

To that end, when I think about bugs, I focus on what the team delivers, not where the bug came from.

A bug or defect is any behavior of your software that will unpleasantly surprise important stakeholders.

There are a few caveats to the definition:

• If the team finds and fixes a problem as part of their normal work—in other words, before a story is "done done"—it's not a bug.

• If the on-site customers have intentionally decided that the software should behave this way, it's not a bug. (Some of your software's behavior will be unpleasant to stakeholders—you can't please all people all the time—but hopefully it won't be a surprise.)

This definition is broad and it won't help you settle contract disputes, so don't use it for formal bug categorization. Instead, use it to change the way you think about bugs. Stop worrying about who made what mistake and start focusing on what you can do, as a team, to increase value by delivering fewer bugs.

⁵Thanks to Ron Jeffries for this insight.

Ingredient #4: Test Your Process

These practices will dramatically cut down on the number of bugs in your system. However, they only prevent bugs that you expect. Before you can write a test to prevent a problem, you have to think that the problem can occur.

Ally

Exploratory Testing, §X.Y

Exploratory testing, in the hands of a good—some might say diabolical—tester, is an invaluable way to broaden your understanding of the types of problems that can occur. An exploratory tester uses her intuition and experience to tell her what kinds of problems programmers and customers have the most trouble considering. For example, she might unplug her network cable in the middle of an operation or perform a SQL injection attack on your database.

If your team had a typical adversarial relationship between programmers, customers, and testers, these sorts of unfair tests might elicit bitter griping from programmers. Remember, though—you're all in this together. The testers have exposed a hole in your thought process and, by doing so, saved you from uncomfortable apologies to stakeholders or dramatic failures in production. Congratulate your testers on their ingenuity... and as you write tests, remember the problems they have exposed.

Exploratory testing is a very effective way of finding unexpected bugs. It's so effective that the rest of the team might start to get a little lazy.

Think of exploratory testing as testing your process, not your software.

Don't rely on exploratory testing to find bugs in your software. (Really!) Your primary defense against bugs is test-driven development and all of the other good practices I've mentioned. Instead, use exploratory testing to test your process. When an exploratory test finds a bug, it's a sign that your work habits—your process—have a hole in them. Fix the bug, then fix your process.

Testers, only conduct exploratory testing on stories that the team agrees are "done done". Programmers and customers, if your testers find any problems, think of them as bugs. Take steps to prevent them from occurring, just as you would do for any other bug. Aim for a defect rate of under one or two bugs per month including ones found in exploratory testing.

A good exploratory tester will find more bugs than you expect. To make the bug rate go down, fix your process.

Ingredient #5: Fix Your Process

Some bugs occur because we're human. (D'oh!) More often, bugs indicate an unknown flaw in our approach. When the number of bugs you generate gets low enough, you can do something usually associated with NASA's Space Shuttle software: root-cause analysis and process improvement on every bug.

Even if your defect count isn't low enough to allow you to do root cause analysis of every bug, consider randomly picking a bug each iteration for further analysis.

When you fix a bug, start by writing an automated test and improving your design to make the bug less likely. This is the beginning of root-cause analysis, but you can go even further.

Ally

Root-Cause Analysis, §X.Y

As you write the test and fix the design, ask questions. Why was there no test preventing this bug? Why does the design need fixing? Use the "five whys" technique to consider the root cause. Then, as a team, discuss possible root causes and decide how best to change your work habits to make that kind of bug more difficult.

Making Mistakes Impossible

The best way to fix your process is to make mistakes impossible. For example, if you have a problem with UI field lengths being inconsistent with database field lengths, you could change your design to base one value off of the other.

The next best solution is to find mistakes automatically. For example, you could write a test to check all UI and database fields for consistency and run that test as part of every build.

The least effective approach (but better than nothing!) is to add a manual step to your process. For example, you could add "check modified database/UI fields for length consistency" to your "done done" checklist.

Next, ask yourselves if the root cause of this bug could also have led to other bugs that you haven't yet found. Testers, ask yourselves if this type of mistake reveals a blind spot in the team's thinking. Perform additional testing to explore these questions.

Invert Your Expectations

If you follow these practices, bugs should be a rarity. Your next step is to treat them that way. Rather than shrugging your shoulders when a bug occurs—"oh yeah, another bug, that's what happens in software"—be shocked and dismayed. "That's our fourth bug this month! Why do we have so many bugs?"

I'm not so naïve as to suggest that the power of belief will eliminate bugs, but if you're already most of the way there, the shift in attitude will help you make the leap from reducing bugs to nearly eliminating bugs. You'll spend more energy on discovering and fixing root causes.

For this reason, I recommend that new XP teams not install a bug database. If you're only generating a bug or two per month, you don't need a database to keep track of your bugs. You can just process them as they come in. Explicitly not providing a database helps create the attitude that bugs are an abnormality. It also removes the temptation to use the database as a substitute for direct, personal communication.

Depending on your industry, you may need a formal way to track defects. In that case, a database may be appropriate. However, I never assume that a database will be necessary until the requirements of the situation prove that it is. Even then, I look for ways to use our existing process to meet the regulatory or organizational requirements. Although some shudder at the informality, archiving red bug cards in a drawer may suffice.

Although you may be able to eliminate your bug database, you still need to be able to reproduce bugs. You may need screenshots, core dumps, and so forth. If you fix bugs as soon as they come in, you may be able to work directly out of your email inbox. Otherwise, you can keep this information in the same place that you keep other requirements details. (See section X.Y, "Incremental Requirements".)

Questions

If novices can do this, what's stopping management from firing everybody and hiring novices?

All else remaining equal, experienced developers will always produce better results, more quickly, than novices. If you have the option to bring high-quality people into your team, do it.

The point is that these practices are within the grasp of average teams—even teams of novices, as long as they have an experienced coach. They won't achieve zero bugs, but they are likely to achieve better results than they would have otherwise.

How do we prevent security defects and other challenging bugs?

This approach only prevents bugs that you think to prevent. Security, concurrency, and other difficult problem domains may introduce defects that you never considered.

In this situation, using exploratory testing to test and fix your process is even more important. You may need to hire specialist testers, such as a security expert, to probe for problems and teach the team how to prevent such problems in the future.

How long should we spend working on a bug before we convert it into a story?

Ally

Slack, §X.Y

It depends on how much slack you have left in the iteration. Early in the iteration, when there's still a lot of slack, I might spend as much as four pair-hours on a defect. Later, when there's less slack, I might only spend ten minutes on it.

Bugs are usually harder to find than to fix, so enlist the help of the testers. The fix often takes mere minutes once you've isolated it.

If our guideline is to fix bugs as soon as we find them, won't we have the unconscious temptation to overlook bugs?

Perhaps. This is one reason we pair: pairing helps us maintain our team discipline. If you find yourself succumbing to the temptation to ignore a bug, write it on a story card rather than letting it slide by. Let the rest of the team know about the bug and ask somebody to volunteer to fix it.

If we don't find any bugs, how do we know that our testers are doing exploratory testing correctly?

It's a bit of conundrum: the team is supposed to prevent bugs from occurring in "done done" stories, so exploratory testing shouldn't find anything. Yet if exploratory testing doesn't find anything, you could be testing the wrong things.

If bugs are particularly devastating for your project, ask an independent testing group to test a few of your iteration releases. If they don't find anything surprising, then you can have confidence in your exploratory testing approach.

This is probably overkill for most teams. In practice, if you're following the practices and your testers haven't found anything, you can comfortably release your software. Re-evaluate your approach if your stakeholders or customers find a significant bug.

We have a large amount of legacy code. How can we adopt this policy without going mad?

Most legacy code doesn't have any tests and is chock full of bugs. You can dig your way out of this hole, but it will take a lot of time and effort. See section X.Y, "Legacy Code" for details.

Results

When you produce nearly zero bugs, you are confident in the quality of your software. You're comfortable releasing your software to production without further testing at the end of any iteration. Stakeholders, customers, and users rarely encounter unpleasant surprises, and you spend your time producing great software instead of fighting fires.

Contraindications

"No Bugs" depends on the support and structure of all of XP. To achieve these results, you need to practice nearly all of the XP practices rigorously.

• All of the "Thinking" practices are necessary (Pair Programing, Energized Work, Informative Workspace, Root-Cause Analysis, and Retrospectives); they help you improve your process and help programmers notice mistakes as they code.

• All of the "Collaborating" practices except "Reporting" are necessary (Trust, Sit Together, Real Customer Involvement, Ubiquitous Language, Stand-up Meetings, Coding Standards, and Iteration Demo); most help prevent requirements defects, and the remainder help programmers coordinate with each other.

• All of the "Releasing" practices except "Documentation" are necessary ("Done Done", No Bugs, Version Control, Ten-Minute Build, Continuous Integration, and Collective Code Ownership); most help keep the code organized and clean. "Done Done" helps prevent inadvertent omissions.

• All of the "Planning" practices except "Risk Management" are necessary (Vision, Release Planning, The Planning Game, Iteration Planning, Slack, Stories, and Estimating); they provide structure and support for the other practices.

• All of the "Developing" practices except "Spike Solutions" are necessary (Test-Driven Development, Refactoring, Simple Design, Incremental Design and Architecture, Performance Optimization, Customer Reviews, Customer Testing, Exploratory Testing); they improve design quality, reduce requirements defects, and provide a way for testers to be involved in defect prevention as well as defect detection.

If you aren't using all of these practices, don't expect dramatic reductions in defects. Conversely, if you have a project that's in XP's sweet spot (see section X.Y, "Is XP Right For Us") and you're using all of the XP practices, more than a few bugs per month may indicate a problem with your approach to XP. You need time to learn the practices, of course, but if you aren't seeing improvements in your bug rates within a few months, consider asking for help (see section X.Y, "Mentor").

Alternatives

Ally

"Done Done", §X.Y

You can also reduce bugs by using more and higher quality testing (including inspection or automated analysis) to find and fix a higher percentage of bugs. However, testers will need some time to review and test your code, which will prevent you from being "done done" and ready to ship at the end of each iteration.

Further Reading

"Embedded Agile Project By the Numbers with Newbies" [Van Schooenderwoert] expands on the embedded C project described in the introduction.

---

And here's the draft we initially put up for review. This was the "final" draft at the time. This would have gone in the final book if it hadn't been for the feedback.

---

No Bug Database

Audience

Whole Team

Fix bugs now.

Here's a crazy idea. Get rid of your bug database. No, I'm not advocating that you ignore problems. Exactly the opposite. Hear me out.

Bug databases are often a black hole of to-dos, packed with hundreds of questions, feature requests, and genuine defects. Beyond about ten items, who has the time or energy to slog through that mess? You don't have to keep that weight around your neck. You already have a better way to handle bugs and feature requests.

A Better Way to Fix Bugs

When you find a bug, fix it.

What if you had a zero-tolerance policy toward bugs? Programmers, what if you fixed bugs as soon as you found them, rather than filing bug reports? You don't have to worry about who originally owned the code (section X.Y, "Shared Code"). You can even do some root-cause analysis (section X.Y, "Root Cause Analysis") and refactor the code (section X.Y, "Refactoring") to make similar bugs impossible.

The same idea goes for customers and testers. When you find or hear about a bug, what if you took ownership of it? What if you figured out how to reproduce it, found a free programmer, and helped him fix it?

A Better Way to Track Requests

Of course, sometimes you're in the middle of an important task and need to finish it before fixing the bug. My preferred tracking mechanism is for the navigator to write the problem on our to-do list and then come back to it as soon as we get to a stopping point.

If you have too little time to fix bugs as they come up, you may need more slack (section X.Y, "Slack") in your schedule.

Some bugs are too big to identify and fix immediately. Others are feature requests in disguise. For these bugs, you need to track them for future scheduling. XP teams already have a way to prioritize and schedule work: stories (section X.Y, "Stories") and release planning (section X.Y, "Release Planning").

When you find something that you can't fix right away, create a story card for it. Add it to the stack of pending stories as usual, then let the product manager and other on-site customers schedule it as with any other story.

How Can This Possibly Work?

Can bug tracking really be this simple? How is it possible to eliminate a bug database? There are so many teams with hundreds or even thousands of items in their defect tracking system. If it were so simple, wouldn't they be doing it, too?

The more quickly you fix a bug, the cheaper it is to fix.

The key is high code quality—leading to fewer bugs in the first place—and rapid response to defects. The longer a defect sits in a system, the more expensive it is to fix [Boehm], and, we believe, to analyze for feedback. Ironically, the existence of a bug database is an invitation to ignore problems—to write them down so they'll become somebody else's problem—rather than fixing them. This raises costs.

Eliminating the bug database makes defect elimination everyone's responsibility. When you find a bug, you can't just take 30 seconds to type in an explanation... you have to go fix it. By doing so, you've shrunk the time between bug insertion and bug fix, which reduces the cost of fixing defects and helps keep code quality high by letting you learn from mistakes as soon as possible. It's a virtuous cycle that decreases the number of defects, making rapid response easier, which increases code quality, which in turn decreases the number of defects.

Success with this approach requires discipline and a passionate desire to eliminate bugs as soon as they occur. Your whole team must share this goal. If people take an "out of sight, out of mind" attitude instead, it won't work.

Questions

Do we really fix all bugs right away?

Immediately fix the bugs that need fixing. The on-site customers may decide that some bugs aren't important enough to fix now, if ever—usually the ones that are low impact and expensive to fix. This is a sensible quality/time tradeoff, albeit one to make with care.

This is silly. Has anybody really done this?

It seems pretty radical, doesn't it? Think about it, though: what's the difference between fixing bugs when you find them and batching them up to fix at the end of the project? You have to fix those bugs eventually, and it's cheaper to fix bugs sooner, so why wait? You'll reduce the chance of other people running into the same bugs (and trawling through your reporting system, filing duplicate reports) and have the chance to identify and eliminate the root causes for these bugs too.

[Schooenderwoert] tells a story of an embedded, multi-threaded C project composed primarily of novice developers (emphasis added):

The GMS team delivered this product after three years of development, having encountered a total of 51 defects during that time. The open bug list never had more than two items at a time. Productivity was measured at almost 3 times the level for comparable embedded software teams. The first field test units were delivered approximately six months into development. After that point, the software team supported the other engineering disciplines while continuing to do software enhancements.

If they can do it, so can you.

How long should we spend working on a bug before make it into a story?

It depends on how much slack you have left in the iteration. Early in the iteration, when there's still a lot of slack, I might spend as much as four pair-hours on a defect. Later, when there's less, I might only spend ten minutes on it.

Bugs are usually harder to find than to fix, so enlist the help of the testers. The fix often takes mere minutes once you've isolated it.

If we're supposed to fix bugs as soon as we find them, won't we be tempted to ignore them?

Perhaps. This is why we pair: pairing helps us maintain our team discipline. If you find yourself succumbing to the temptation to ignore a bug, write it on a story or task card rather than letting it slide by. Let the rest of the team know about the bug and ask somebody to volunteer to fix it.

We already have a bug database. How should we handle it?

You can stop using the bug database for new issues and just use it for old ones. Alternatively, customers and testers can spend some time going through the database, eliminating duplicates and unimportant issues, and turning the rest into story cards.

Stakeholders use our bug database. What should we tell them?

You may wish to leave that database in place. A stakeholder-facing bug database or issue tracker can be a convenient way to gather stakeholder feedback. Every day or so, have the testers and on-site customers sort through the outstanding issues. Give any real bugs the zero-tolerance treatment by fixing them right away. Discuss the remaining issues with stakeholders and turn them into stories as appropriate. Whittle the outstanding issues down to zero every time. Don't let them pile up.

Our on-site customers like to use a bug database for holding more information about stories. Is that okay?

Sure. Story cards are just for prioritization; customers often need to hold additional information somewhere. Some teams use a bug database or issue tracker. Others use a Wiki and still others use spreadsheets and PowerPoint. The customers should use whatever they're comfortable with—just be sure to use it as an aid to planning, not a replacement for it, and don't use it to track actual bugs.

If we don't have a bug database, won't people report the same issues over and over?

In my experience, they do that even if you do have a bug database. Of course, if you fix the bug right away and release to production frequently (section X.Y, "Release Planning"), there won't be anything to report.

Contraindications

All you need in order to follow this practice is a burning desire to eliminate bugs as soon as they appear. Although it helps if your team doesn't generate very many bugs, that isn't necessary.

If your team generates a lot of bugs, you will probably need to fix them eventually. Why not now? Yes, your velocity will go down because you're fixing bugs rather than adding features. That's okay—you'll probably release a little sooner overall and it's better to have a known, predictable hit to your velocity now than it is to have an indeterminate number of bug-fix weeks before you can release.

To succeed without a bug database, the team has to be disciplined about fixing bugs rather than ignoring them. Pair programming (section X.Y, "Pair Programming") helps reduce the temptation to ignore problems. The XP development practices (section X.Y, "Developing") also help by decreasing the number of bugs your team creates in the first place. A whole team (section X.Y, "Whole Team") that sits together (section X.Y, "Sit Together") is also helpful, as it allows you to squash tough bugs more quickly by attacking them as a team.

Alternatives

You can use a bug database if you want. Ultimately, the question of whether to fix a bug now, later, or never is the product manager's responsibility. Every issue that affects the schedule and scope of the project in a significant way (that is, by more than a few hours) needs to make it to a story card somehow.

If you use a dedicated bug database, you need some mechanism for getting stories out of the bug database and into the release plan (section X.Y, "Release Planning"). Try not to let the bugs pile up, as that will give you a false impression of your velocity and what you can release.

Further Reading

"Embedded Agile Project By the Numbers with Newbies" [Schooenderwoert] reports on the author's experience in more detail.

---

Thank goodness for feedback.