Voluntary Technical Debt
September 9, 2006
I've talked a lot about the evils of design debt. It's all true: design debt (also known as "technical debt") leads to ruin if left unchecked... and most teams aren't even aware of the concept, let alone making any attempts to pay it off.
There's a flip side to technical debt, though. A savvy team can use technical debt as a tool to accomplish more than they otherwise would. Be careful, though: it's sort of like handling sensitive explosives. Let me tell a story:
I've mentioned before that I'm working with my friend Dave on CardMeeting, a nifty tool for simultaneous collaboration. I don't think I've said much, though, about how we've come so far so quickly.
In late June or early July--I don't remember exactly when--Dave and I were talking about agile planning tools. I was lamenting their focus on numbers and spreadsheets. Agile planning is about communication: getting stakeholders and visionaries together to make difficult compromises and tradeoffs. (See Cards, Table, and Whiteboard.) It's a fundamentally human activity. As the manifesto says, "Individuals and interactions over processes and tools."
(That phrase, by the way, doesn't mean that processes and tools are worthless; just that individuals and interactions are more important.)
To make a long story short, Dave and I decided that we could make a tool that focused on individuals and interactions. The result was CardMeeting: a freeform tool that is all about people and giving them a way to interact. When you get a bunch of people around a table to plan with index cards, all kinds of good interpersonal dynamics occur. CardMeeting a tool for achieving some of these same results when you can't physically be together. It's meant to be used along with a conference call.
Anyway, Dave and I decided that this was a great idea: it would be particularly useful for distributed teams--something the agile community has struggled with for quite a while. Distributed teams aren't the ideal situation for agile development, but they're fairly common and only going to become more so.
Thanks to a bit of luck and a lot of hard work on Dave's part, we got an initial spike to work. I still remember Dave's first demo: it was just an index card in the center of the screen that we could zoom in and out. No text, no editing, no dragging--nothing. Just a perfectly realized 3x5 index card. (We still use the same model for our index cards today.) But that was enough to prove that we could do some simple 2-D work.
Okay, so what does all of this have to do with technical debt? Actually, everything. What we should have done at this point is archive our spike and start over using proper test-driven development. But we didn't. Instead, we kept going, trying more and more ideas, getting rudimentary networking in place, making the cards editable... until we didn't have a spike any more. We had a nascent application. One without tests. In other words, we already had a fairly sizable amount of technical debt and the application was only a week or two old.
I saw this happening, of course. Dave and I had talked about throwing the spike away and starting over with proper TDD. But by this point, it was clear that we were going to have something interesting to show in time for Agile 2006, which was only a few weeks away. The more we could show at Agile 2006, the more excitement we could generate. We decided to press forward. We voluntarily accepted the technical debt.
Dave is a wizard programmer. He used to work in the game industry and now does outsource work for companies needing heavy-duty enterprise Java. He chews up new technologies for breakfast. A week before the conference, he had finished all of the basic concepts: editing cards, creating new cards (with a handy icon), moving them around, zooming in and out, and the hardest: simultaneous collaboration over the Internet, or as Dave likes to say, multiplayer cards.
At that point, we were good to go. We created a CardMeeting logo and I ordered a bunch of custom-printed packs of index cards to hand out at the conference. (They were delivered on the last business day before I left.)
Of course, we still had a week to go. We had the minimum necessary to show off CardMeeting, but why stop there? A day before the conference, Dave implemented persistent meetings: cards would stick around forever, even if Dave reset the server. The next day, he added the ability to see cards move around smoothly as other people dragged them, rather than jumping from place to place. The next day, the ability to delete cards. The next, colored cards (a popular request at the conference).
The conference was a huge success. Bob Payne interviewed me for his Agile Toolkit podcast. Esther Derby loved the idea and blogged about it. Many other people had a chance to try it out and reactions were almost universally positive.
After the conference, we were flush with our success. Only one last feature stood between us and a product that would actually be useful to real teams: private meetings. A week later, we implemented them! We were out of "technical demo" status and hosting a real, useful alpha-quality service.
Elapsed time: a month, maybe two, from concept to useful product. Not bad.
It's now a month later and progress has slowed to a crawl. In the first month, we were adding significant new features every week, sometimes every day. What happened?
In two words: technical debt. To make progress so quickly, we cut a lot of corners. We didn't implement any tests. We only programmed for the best-case networking scenarios. We let bugs creep in.
All of these things would have been good things to fix. In particular, implementing tests would actually have made us faster, and it would have only taken a week or two at most to figure out how to use TDD in our UI- and network-intensive environment. But--here's the rub--even a week or two delay would have meant no Agile 2006 demo.
So, as a result, we've spent the last month paying back our technical debt. We've been fixing bugs, improving our networking, putting together a proper build and staging environment, and implement a proper test framework. It's taken a month so far and we have quite a bit left to do.
The thing about technical debt is that you aren't just deferring costs. You pay interest on those costs... even when you take on technical debt for as little time as a month. I can't say how much time the technical debt is costing us, but I would guess that it's anywhere from two to four weeks of extra work.
If we hadn't taken on the technical debt, we would have missed our demo opportunity. We might have saved a month of work if we had paid down technical debt right away. That's no small amount when you're only talking about six weeks of work to begin with. But saving a month wasn't worth it because we would have missed our demo opportunity.
So, instead, we took on technical debt and now we're paying the cost. It was a choice we made with our eyes open, and I'm glad we did it.