James Shore

I gave a presentation on "The Secrets of Agile Success" at the Dallas chapter of the APLN on May 12th. Susan Fojtasek recorded the session and posted it online. It was an interactive session with lots of discussion and audience stories. I thought it went really well. It's about 90 minutes long, so if you have a long commute or flight, take a look.

InfoQ just posted a video interview with me. Deborah Hartmann interviewed me at Agile 2007 last year just before the book came out. It came out great, and InfoQ also included a transcript for those of you who prefer reading to watching.

This interview joins a few other good recent interviews: Bob Payne's Agile Toolkit interview, also conducted at Agile 2007, and Clarke Ching's Agile Thinkers interview.

My understanding of success has evolved over the years. At first, I thought of it as a pyramid of levels, a la the CMM:

• Level 1: The software works
• Level 2: It's maintainable
• Level 3: It provides value
• Level 4: It provides more value than it costs
• Level 5: It provides the most value possible with the resources used

Later, I realized that there are actually multiple aspects to success:

• Organizational success is necessary for a product continue to receive funding.
• Technical success is needed to keep maintenance costs under control
• Personal success is required to receive support from stakeholders, prevent sabotage, and gain team member commitment.

Recently, Jurgen Appelo pointed me at an essay he wrote that discusses (among other things) the nature of success. He takes a completely different approach. In Jurgen's view, success is the absence of failure. One thought that occurred to me as I read is that we could measure success as the length of time a product is in production. An intriguing idea.

It's a fascinating essay, long, but worth the read. Take a look.

A reader recently asked:

I have learned a bit about Scrum and Crystal and found that things are looser and they do not prescribe practices as XP seems to and that they want you to work toward finding the practices that work for your team. An example is Mike Cohn saying that he does not like to prescribe practices, like TDD which he believes in, but wants the team to discover for themselves their need of it as they do root-cause analysis and retrospectives. Does this go against y'alls advice on trying to do all the practices and later adjusting things to meet your needs as a project? Or is it something different?

[For example, see] Mike Cohn's thoughts about Scrum and XP. In particular starting with Scrum and creating your own XP.

My response:

I agree with Mike Cohn's description of the differences between XP and Scrum. I would emphasize that Scrum and XP are very similar, with the exception that XP includes more "how to" practices. There's also a bit of an focus difference; I feel like Scrum's focus is "let's create a self-organizing team" and XP's focus is "let's create excellent software." In other words, when I talk to Scrum people, they tend to talk more about self-organization and enabling the team; when I talk to XP people, they tend to talk more about software and delivering value. But despite that difference in attitude, the actual practices are very similar.

Mike's closing statement, that Scrum brings big benefits by itself, matches my experience. Scrum is also easier and less threatening than XP, so I see a lot more people starting out with Scrum. On the downside, the teams that start with Scrum tend to struggle more than the teams that start with XP. The XP teams experience more pain starting out, but then get to a high performance state within the first year. The Scrum teams I know have a much longer ramp-up time, generally having code quality problems for several years. Actually, I have yet to meet one that wasn't having code quality problems. None have gotten to the same high performance result that good XP teams do.

Granted, these statements are based on my experience, and my experience is formed by the people I meet at conferences and the companies who hire me. High performance teams generally don't hire me, so it could be that I'm not meeting the high performance Scrum teams.

Nonetheless, my experience is that Scrum teams have an initial success and then struggle. XP teams struggle to get started, but once they figure it out they have more success. Sometimes they don't figure it out, fail, and give up. With Scrum, teams seem less likely to realize that haven't figured it out; I've met teams who said they were using Scrum but weren't doing anything of the sort; they were just using the terminology. That happens with XP, too, but to a lesser extent, and usually of the form "we're doing XP but not..."

To boil it down to simple statements (perhaps too simple):

• Scrum: easy to adopt, very hard to master, fails quietly. You're more likely to successfully adopt Scrum, but the benefits won't extend to your codebase and you'll struggle with legacy code for many years. If you're missing pieces, you may not realize it.

• XP: hard to adopt, easier to master, fails noisily. You're less likely to successfully adopt XP, but you'll be well positioned for long-term success and mastery. If you're missing pieces, you'll probably be able to tell.

I have one complaint about Mike's essay. XP doesn't prescribe engineering practices, as Mike says; it includes them. For teams new to agile development, I do recommend that you use all of the practices from the start. However, a team that knows what it's doing will change its practice of XP over time. No two experienced XP teams practice XP in the exact same way, and there's no requirement that you do exactly what the books say. For example, one long-running XP team I know has evolved away from using iterations entirely, but I'd still think of them as practicing XP.

However, in order to know what to change, you have to know how the techniques work in practice. That's why I recommend starting out with all of the practices. Otherwise, it's like deciding to be an avant-guarde violinist without first mastering the violin.

Should you start with Scrum or XP? I recommend starting with XP, but Mike's right--you won't succeed if you force the practices down people's throats. The team has to agree to try them. If they're not ready to do that, you might be better off starting with Scrum. Then you can use that success, along with some of the code problems you'll encounter, to guide them to XP.

Finally, if you have the opportunity to get a coach to help you adopt your method, I recommend that you do. A good coach will help the transition go a lot more smoothly.

I haven't been this excited about a programming language since I tried to invent my own.

Programming in Subtext

Subtext is a programming language invented by Jonathan Edwards. It's based on schematic tables (see figure) which display logic and computation in two dimensions. In contrast, textual languages show just one dimension (a stream of characters), and it's usually computation. Introducing conditionals requires you to puzzle out control flow.

Subtext makes logic and computation obvious by placing them on two axes in a table. To understand how this works, I recommend watching Jonathan's video demonstrating Subtext. (If you saw the video for the original version of Subtext, watch the new one--it's come a long way. I didn't even recognize it.)

Subtext nails a bunch of things that I want. To explain, I'll need to digress for a moment.

Example-Driven Development

My background is in commercial software development in a wide variety of industries. I'm passionate about seeing software lead to real-world successes, which involves organizational success (return on investment, to oversimplify), technical success (maintainable code), and personal success for the systems' community. I've dedicated my career to improving my understanding of those three components of software development. I still have a long way to go.

Along this journey, I discovered Agile methods and I've became fairly proficient with them. Two Agile techniques that I've found to be particularly powerful are Test-Driven-Development (using a tool like JUnit) and Example-Driven Requirements (using a tool like Fit).

A Fit document

JUnit and its brethren are great. TDD works well. Example-driven requirements also works well, but I've become increasingly dissatisfied with Fit. Fit, in case you haven't heard of it, is a tool created by Ward Cunningham and maintained, in a laissez-faire sort of way, by me. It allows you to provide domain-level examples in HTML tables, then automatically test those tables via fixtures that execute the table against your production code.

The great thing about Fit is that it enables example-driven requirements. This is incredibly powerful and useful, and Fit is the only tool I know of that focuses on this. (Most clones of Fit get it wrong, focusing on test automation rather than customer collaboration.)

The problem with Fit is that it's a maintenance burden. You have to write your examples in HTML and none of the existing tools (not even MS Word) make editing and revising HTML tables easy. You have to create fixtures. When done well, fixtures are small and simple, but they're easy to get wrong. You have to deal with Fit's primitive and difficult-to-use tool support. (Okay, I could fix this one, and should.) And, to top it off, almost everybody misunderstands and misuses Fit.

Okay, back to Subtext and Jonathan Edwards. TDD and example-driven requirements (EDR) can both be considered cases of example-driven development. In other words, your development efforts are focused around creating examples and making them work. The examples form sort of an incomplete specification of your work that are fleshed out along with your program. TDD operates at the level of programmer intent and EDR operates at the level of customer intent.

Well, Jonathan shares a passion for examples in programming. So it's no surprise that Subtext looks like it will support the Agile conception of example-driven development quite well. The best thing about it is that it looks like it will support TDD, example-driven requirements, and software development within a single paradigm. Brilliant.

Critique

In addition to its support for example-driven development, I like Subtext's focus on making programmers' intent more obvious. The schematic tables are a great way to show what happens computation is mixed with complex conditionals. I particularly liked Subtext's ability to automatically discover gaps and overlaps in conditions. A minor nit: I think Jonathan could decrease the verbosity of his language by allowing more complex statements on each line, perhaps allowing a click to change the view between the simple, linked form shown in the video and more complex compound statements. For example, the Fibonacci example could show the entire >= 2 case in one statement: fib(in - 1) + fib(in - 2). I think I'd actually find that more readable than the current display.

A little light reading

Given that Subtext is a work in progress, I liked almost everything I saw in Jonathan's video. The one thing I disliked was towards the end, as he was talking about polymorphism. I got a sense of disdain towards object-oriented programming. That's not uncommon in the academic world--OOP is anything but well-defined from a formal perspective.

But although OOP is messy, it has an important characteristic that Jonathan also values: it makes programming languages more usable. Objects are an important way of organizing large programs so that the programmer can ignore irrelevant details. Although polymorphism makes control flow more complex, as Jonathan observes, it also enables design techniques that make programs easier to understand.

In a small program, an experienced programmer relies on well-named functions/methods with carefully-constructed semantics. When that's the case, she may use the context of the program to infer the behavior of those functions. For example, if you saw that a mathematical calculation called an "exponent" function, you could guess that it calculated the exponent of a number and move on, not bothering to look at the body of the function unless it was directly related to your work.

Classes and methods perform a similar service in OOP. In a large program, an experienced programmer also relies on well-named classes with carefully-constructed responsibilities. The noun-verb combination of classes and methods allows her to infer the behavior of those classes. For example, if you saw a call to monster.attack(character), you could guess that it calculated attack and damage probability and adjusted character's hit points accordingly.

Subtext's damage calculation gets complicated

Subtext could face challenges dealing with larger, more complicated systems. You can see in the figure how even minor complications in the damage calculation start to make the table unwieldy. Subtext provides a focusing mechanism to hide unnecessary details, but I'm skeptical of how useful the current incarnation is for this purpose. I think it will work well for simple algorithms, but not for the enormously complex, interrelated computations that large systems deal with.

Those enormously complex, interrelated computations are exactly what object-oriented programming languages are good at managing. (With good design, at least, which is by no means a given.)

I'd like to see Subtext give objects a more prominent role in the language. It's hard to tell from the video exactly what's going on, but it looks like the programmers' use of objects is largely limited to defining types and subtypes. I'd like to see more. Specifically, I'd like to see support for named methods attached to objects. They could be expanded and edited in-place, or they could be edited in the context of the object.

Simplifying damage calculation with polymorphic methods

For example, a lot of the complexity of the damage calculation comes from the polymorphic 'power' constants. Magic attacks have a power of 5, melee attacks have a power of 4, and ranged attacks have a power of 3 (not shown in the figure). If Subtext showed that polymorphism as a reference to a method, then the diagram would be simpler. The effect of defense could be similarly collapsed into a 'damage' method.

To continue the example, if the programmer wished, he could open up the 'Attack' class and see 'power' and 'damage' defined there. Perhaps subclasses and/or methods could be shown as columns. That would allow him to contemplate the relationships between the classes and their methods--a common need--and make changes to them together.

This is actually a fairly minor tweak to an impressive system, and it's quite possible that Jonathan has already thought of or even included these improvements. If he hasn't, I hope he'll consider them. Either way, Subtext is impressive. I want it.

(Thanks to Keith Braithwaite for the link.)

Every so often, I describe what I do to make this site a reality. For the four of you who like to watch me gaze at my navel, I have good news: today is one of those days. For the rest of you, I'm sorry. It's one of those days. You can move on. (I won't be offended. You can even give this essay a nasty rating if you want. I have skin like leather, and I do pay attention to the ratings.)

Okay, here goes.

Production

I use a MacBook to compose all of my essays. My old Toshiba laptop (which was running Windows) died and I decided to see if life was better on the other side. I have come to like Mac OS X, after some initial frustrations, but I'm not a fanatic or anything.

I write the body of my entries in hand-crafted HTML using Smultron, a text editor. Smultron is not a great text editor, but I've grown used to it and haven't felt compelled to seek out something better. I also use OmniGraffle when I need to make pretty diagrams. Mmm... lickable.

Everything required to make the site go is stored locally, on my MacBook. I deploy the site to a local copy of Apache for testing, then to my web host when I'm ready to go live. I use a Ruby Rakefile to build the site, and then rsync the whole mess to the server. The exact same site goes on the server as on the local test site; all that changes is the rsync destination.

I am absolutely paranoid about backups and revision control. There's years of work here. That's why all of my essays are written in a text editor and stored on my laptop. Most blogging software I know of saves your work in a database on the web server. Easy, but not fault tolerant. I plan to be writing for the next ten, twenty, thirty years... minimum. My stuff has to be easily scripted, version-controlled, backed up, and trivial to deploy to another web host when (not if) my current host goes under.

The source for my site (including rakefiles) is versioned by Subversion. The repository is local, on my laptop. The laptop is backed up several times over: first, the entire site is on the web host, of course. Most of my files are also backed up automatically to my Mirra continuous backup server, running on my wireless network. The entire computer is backed up with Time Machine every hour. (Time Machine is better than Mirra, so when the Mirra server dies, I won't replace it. Mirra seems to have been killed by Seagate anyway.) I also make a bootable whole-disk backup using SuperDuper! and store it off-site.

Design

The biggest challenge for me in making the site was my complete lack of web design skills. (Software engineering, yes. Graphic design, no.) Sitepoint got me started and A List Apart carried me through. I borrowed liberally from Jeffrey Zeldman in creating the side menu, as permitted by comments in his CSS.

For the color scheme, Jason Beaird's "Color for Coders" article helped me get started and Visibone's Color Laboratory allowed me to pick web-safe colors. (I have no idea if we're supposed to limit ourselves to 216 "web-safe" colors any more, but it seemed like a safe bet. I cheated a bit for quotes, though.)

I also took advantage of some royalty-free icons. The RSS feed icon () came from Feed Icons; the print icon () came from graphicPUSH; the star icon () came from 1ClipArt; and the spinner () came from Andrew Davidson.

The nicely self-contained JS-Kit handles my comments and ratings. JS-Kit is very straightforward, but I do have a few small (very small) lessons learned that might be useful.

Finally, lots of trial and error got me to the result I have today. W3Schools' CSS Reference was invaluable for making it work. Eric Meyer's "Going to Print" article provided the finishing touch for my print stylesheet by showing me how to automatically insert URLs after printed links.

During this process, I learned how badly Internet Explorer 6 supports CSS. Rather than struggle with IE's problems, I settled for graceful degradation. IE will work, but Firefox renders the site better, will give you URLs in the print results, and has a more responsive "print" button.

Hosting

The site is hosted by TextDrive (now owned by Joyent) and runs on the Apache web server. I use mod_rewrite to make sure public-facing URLs are implementation-independent and to ensure that you can link to a page forever (for sufficiently small values of forever). Domain Discover makes sure that my domains (jamesshore.com and titanium-it.com) point to the right place. Mint handles my analytics, along with the plug-ins Fresh View, Trends, Download Counter, Doorbell, Outbound, and Secret Crush.

The site is rendered by the ultra-minimalistic Blosxom. Blosxom was the only blogging software I could find that actually allowed me to store entries locally, in files that I can back up and put in Subversion, rather than on a database on the server. Blosxom runs on Perl 5. Blosxom does almost nothing by itself, so I use these plug-ins to help out: blok, meta, default_flavour, prefs, breadcrumbs, directorybrowse, entries_index_tagged, interpolate_fancy, menu, plain_text, postheadprefoot, readme, static_file, urltranslate, and wordcount.

I run my whole site with Blosxom, not just the blog portion. Getting this working was quite a headache and I had to make some custom mods to several of the plug-ins as well as Blosxom itself. It's been three years since I got it set up the way I like it, though, and the site has been pretty much hands-off and trouble-free ever since.

Colophonem adidi.

(I ran across that little gem while using Google to define "colophon." It means "I have put the finishing touch to it.")

I recently received the following letter from a reader, "Andrea," who wishes to remain anonymous. Her brother has been volunteering for a well-known U.S. service organization. Here's Andrea:

Hi James,

I was a big fan of your change-diary, and recently I've been getting letters from my little brother who is [...] in [...]. The parallels between what he is dealing with his village and what you (and most of us in corporate it) dealt with are pretty incredible to me. I've forwarded his most recent email with personal/unrelated parts snipped out. Hope you find it an interesting read.

I found this letter fascinating and got Andrea's permission to share it with all of you, under the condition of anonymity. I hope you find it as interesting as I did.

*snip*

yeah i got great news from the funding agency, our project has been approved, we are just waiting for the official letter, and final agreements, etc. i am incredibly relieved, but not all that happy. i feel like the village simply assumed that this was gonna happen. they think if u write a proposal, money automatically comes. i think most, if not all of them, have no idea how hard it was to get this approved, and how lucky we are and greatful we should be. to them its like they just expect these handouts. so in a way i dont know if it would be worse to not meet their unrealistic expectations or, as in this case, actually live up to them. now they still have no idea how fortunate we are.

job-wise, things in the village are fine. this project will be coming through soon, and i can keep busy with school things in the mean time. im trying to start a garden, but its tough to do on my own. mainly b.c i cant seem to put enough days together where im not busy, so i can actually do it. some people say they want to help, but its just one of those things that just hasnt materialized yet. theres not really any good land on our property, so going to adjacent properties opens up a whole can of worms with ownership, etc. so i kinda get the run-around when i attempt that. its kinda like people dont want me doing things that are more or less personal projects. they think i should be doing things for the village at all times, what they dont realize though is that there is only so much i can do for the village in a given time, ideas/projects take time to develop. also there is only so much i can do on my own. some help would be nice. for instance i was talking to the preachers son yesterday about my plans to build a canoe (which should happen next week w. the help of a carpenter, which im excited about!) and his reply was "well what about water tanks?" then i explained to him that theres nothing we can do about tanks until the money comes. then he says "well what about the beach house?" this was something that a chief constructed on his land and made no mention of it being for tourists or anything and hasn't approached me about it at all. as if im supposed to magically make tourists appear or something. i dont even think the chief built it for tourists, i think he just built it for his family. its shit like that i have to deal with all the time. people see me doing something positive, maybe more or less personal, but positive nonetheless, and they want to know why im not out curing cancer in the village instead of building my canoe. i should have told him that if his father didnt suck the village dry of all its money, half the problems wouldnt be here.

what ive noticed though, that others may not seem to realize is that i somewhat lead by example. the reason i push for the personal projects is b.c they can blossom into village projects. the past few months ive been making a fuss about there being no fishing boats/canoes in the village and wanting to build one. this has been mostly met with opposition, them thinking the american doesnt know how to swim or fish and will drown at sea, so they just brush it off. that and they arent as simple/cheap to build as u think. but then last week a member of my village committee bought a canoe, this week my neighbor is building one, and after he is done, i should get my chance to build mine with the carpenter. these havent been around for years, and now they are coming back, i think in large part because i have shown determination to build one. the same thing has kinda happened with gardening. even though mine still doesnt exist, ive been talking about gardens and fruits n veggies since day one, and ive seen more gardens spring up around the village since ive been here. see, even my personal ideas still have a village purpose. it kinda gives people the push they need to go out and do it.

socially im a bit more removed from the village ever since the drama from last month. i dont really hang out as much in the evenings, and i dont try as hard to get to know people. i kinda just stay inside and read more where i used to try to make conversation, visit people at their houses, etc. the closer ive gotten to these people, the more they try to hit me up for money and free shit, and being that we are closer now, its much harder to say no. my girlfriends step father is probably the worst one. so its kinda caused me to be a little more removed. not totally, i still socialize and get along fine w. people, but im just not going the extra mile anymore. i got burned once by that and im not doing it again. that and comments about not seeing any water tanks in the village yet or other tangible results are becoming more frequent. its hard to ignore, but i try my best. i wish everyone there knew how much i bust my ass for their village. ive thrown out a ton of smaller project ideas that somehow or another dont materialize. like a talent show for the youth, the vet coming out to desex dogs, another volunteer that desexes cows, the red cross visiting to deliver medical supplies and give health talks, pen pals through my old schools, establishing a pre-school in the inland part of village, and other things that i cant name off the top of my head. some of these ideas might still happen, but all have already been attempted or discussed, but for one reason or another never come to fruition. like the womens committee wants the red cross, but they say they are always too busy with other things like mat weaving, holiday/social event prepartations, giving their money away to the pastor. things like that. oh well, we do have another proposal pending for a school fence that would enable the school to plant fruit trees within the school compound and improve the play area. (u have to keep the pigs out to do any of this). and i have talked to the member of parliament in our district and the dept. of fisheries in the city about deepening the channel off our shore so we can get a fishing boat in and out. thats as far as i can take that issue. now its up to the member of parliament to talk with the head of the port authority to put things into action. i cant do any more about that right now, but i still get asked by villagers "when am i gonna dig out the channel?" its questions like that i get a lot that sometimes make me want to smack people.

*snip*

so one more thing. could u get dad to send some fish hooks and maybe some fishing line. the basic eagle claw hooks would be good, but if he can send some heavier duty hooks and line that would be great too. i should be out to sea with the big boys next month. fishing is actually one thing ive really enjoyed here, although its mainly only spear fishing, and i dont do it nearly as often as i like, the men i go fishing with are good guys. i like them. i even gave my broken fins (fins i bough brand new in richmond from a dive shop and they broke the first day) to one of the guys and hes thrilled. im sure hell find a way to patch them up and use em.

well hope this updates u pretty well. love u all. miss u. hows cali?

I've just finished using JS-Kit to add comments and ratings to my site. It's a great tool--simple, straightforward, and fits into my world rather than demanding that I fit into its world. Just what I like.

JS-Kit works pretty much as advertised: paste in a few lines of HTML, and you have comments or ratings. For example, here's the code I use to display ratings on every post:

<div class="js-kit-rating" title="$title" path="$path/$fn.html" permalink="$url$path/$fn.html"></div>
<script src="http://js-kit.com/ratings.js" type="text/javascript"></script>

("$title" and similar variables are part of the template engine I use for this site.)

The <div> determines where the rating or comments show up in your HTML. It's optional--if you don't put in the <div>, the rating/comments appear at the point that you load the script. If you leave it out, it will show up wherever you load the js-kit script. This caused me a few problems, as I'll explain.

That's the basics. I wanted a bit more, and some of it took a little figuring out. None of it's rocket science, but I thought I'd put up what I learned in case it's helpful to someone else down the road:

Display a "Loading" Message

You can put whatever you want in the <div> and it will be erased when the comments are loaded. I use it to show a "loading" spinner, just in case the user's network connection is slow or the JS-Kit service goes down.

<div class="js-kit-comments" path="$path/$fn.html" permalink="$url$path/$fn.html">
  <p style="font-variant: small-caps">Loading comments...</p>
  <p><img src="http://jamesshore.com/images/spinner.gif" alt="" /></p>
</div>

Hide Bogus Comments and Ratings

If you put the JS-Kit scripts in the footer of every page (as I do), a page with no comments will have a spurious "leave a comment" link at the bottom of the page. That's because you won't have any JS-Kit <div> in the rest of the page, so JS-Kit (in an attempt to be a little too easy to use) will place the comment, or rating, at the location that you load the script.

This made me scratch my head a bit until I realized what was going on. Once I understood the problem, it was easy to solve. I just put the scripts in my own <div> and gave it the style display: none in my CSS. I also made sure that the extra comment box linked to a non-existant comment path.

HTML:

<div id="js-kit-loader">
  <div class="js-kit-comments" path="/null"></div>
  <script src="http://js-kit.com/comments.js" type="text/javascript"></script>
  <script src="http://js-kit.com/comments-count.js" type="text/javascript"></script>
  <script src="http://js-kit.com/ratings.js" type="text/javascript"></script>
</div>

CSS:

#js-kit-loader {
  display: none;
}

Liquid Comment Box

The default comment box is small. For me, small comment boxes make it hard for me to write thoughtful comments. I want my readers to be able to create masterworks--or at least, be comfortable writing more than a few lines. A big-ass text entry box was essential.

This was the hardest thing to figure out. The JS-Kit FAQ describes how to make the text box bigger, but code they provided only works for hard-coded sizes. I wanted the text box to be "liquid:" to grow and shrink with the size of the browser window. I eventually figured it out by looking at the default comment template in the JS-Kit customization documentation. The final fix only required a few lines of CSS.

.js-CreateCommentBg {
  width: 100%;
}

.js-CreateComment textarea {
  width: 100%;
  height: 20em;
}

I've only been using JS-Kit for a few days, but so far I'm quite happy with it. There's a bit of risk associated with letting another site control all of the comments on this site, but for my purposes, that's acceptable. If you can accept that risk and you're looking for a nice easy way to include comments and/or ratings on your site, I recommend it.

Update: They're enabled! Thanks to Jimmy Bogard for delivering a knockout argument.

I've avoided enabling comments on this blog up until now, for two reasons: a) It seemed like a lot of work to implement on my semi-homegrown platform; and b) I didn't want the hassle of moderation and spam filtering.

Well, I recently discovered JS-Kit, which makes adding comments ridiculously easy. It also includes a moderation engine and spam filtering. So I thought I'd give it a try.

What do you think? Should I enable comments? Do you have another comment engine that you prefer over JS-Kit? Let me know in the comments below. (RSS subscribers, you'll probably need to click through.)

For every complex problem there is an answer that is clear, simple, and wrong. --H. L. Mencken

How do we get people to do what we want?

A man went fishing one day. He looked over the side of his boat and saw a snake with a frog in its mouth. Feeling sorry for the frog, he reached down, gently took the frog from the snake, and set the frog free. But then he felt sorry for the snake. He looked around the boat, but he had no food. All he had was a bottle of bourbon. So he opened the bottle and gave the snake a few shots. The snake went off happy, the frog was happy, and the man was happy to have performed such good deeds. He thought everything was great until about ten minutes passed and he heard something knock against the side of the boat. With stunned disbelief, the fisherman looked down and saw the snake was back with two frogs!

"What Gets Rewarded Gets Done," by Michael LeBoeuf

This story, and others like it, has its roots in experiments conducted by B. F. Skinner. Skinner was an inventive psychologist who constructed ingenious boxes--"Skinner Boxes"--that held a pigeon and a lever. Peck the lever, and the pigeon is fed. Don't, and the pigeon goes hungry. The pigeons quickly learned to peck the lever.

Through this and similar experiments, Skinner discovered operant conditioning, in which behaviors can be influenced by providing or withholding incentives. The popular conclusion: What gets rewarded, gets done. Pop behavioralism has been embedded in American culture ever since. We just love the idea that people's actions can be traced back to the promise of some reward. (My biggest peeve: being accused of doing things just to sell books. Hey, bub, have you seen the size of my royalties? Me neither--my microscope isn't powerful enough.)

So, is that the stunning truth at the center of the pigeon story? That rewards influence behavior? Actually, no. The actual truth is much more profound. Here it is:

People Aren't Pigeons.

Or, to be more specific, human motivation is more complex than pigeon motivation. Yet popular culture has boiled all that complexity down into one easy conclusion: "what gets rewarded get done." Clear, simple... and wrong.

It's not totally wrong, mind you. What gets rewarded does get done. It's just that there's more to it than that.

Let me tell you some more stories. These are the results of rigorous scientific studies--not anecdotes--as reported by Alfie Kohn in his book Punished by Rewards.

In 1961, a graduate student at the University of Kentucky found something she didn't expect. ...She brought 72 nine-year-old boys into her laboratory one at a time and challenged them to tell [two] faces apart. Some of the boys were paid when they succeeded; others were simply told each time whether or not they were correct.

Miller expected that the boys would do a better job when there was money at stake. Instead, she found that those who were trying to earn the reward made a lot more mistakes than those who weren't. It didn't matter how much they were paid (one cent or fifty cents) or whether they were highly motivated achievers (as measured by a personality test).

...The following year, another graduate student... This time it was undergraduates, 128 of them in all, who were brought into a lab individually. ...As with Millers' experiment, some of the students were informed that they could earn anywhere from $5 to $20--quite a lot of money in 1962--if they succeeded; others weren't promised anything. Even though the subjects were older and the assignment quite different, Glucksberg's results echoed Miller's: when the task was more challenging, those who were working for the financial incentive took nearly 50 percent longer to solve the problem.

...In the early 1970s, a batch of new reports came out that showed the early reports were no flukes. ...It turned out that the children who received candy or the promise of candy got fewer [tests] right than those who received nothing more than information about how well they were doing--a result that led her [Janet Spence] to make the comment... "[Rewards] have effects that interfere with performance in ways that we are only beginning to understand."

Kohn's book is littered with these research stories. Here are a few more good ones:

Each [college student] was asked to work on an interesting spatial-relations puzzle. Half were promised money; the other half weren't. Then the experimenter announced that it would be a few minutes before the next phase of the study got started. The subject was left alone in a room to wait, where he or she could continue playing with the puzzle, read a magazine, or daydream.

Actually, this was the next phase of the study; the subjects were secretly watched to see how long they worked on the puzzle when they had a choice. Those who had been paid... spent less time on it than those who hadn't been paid. It appeared that working for a reward made people less interested in the task. Or, as Deci put it, "money may work to 'buy off' one's intrinsic motivation for an activity."

[Other experimenters (Mark Lepper and his colleagues)] gave fifty-one preschoolers a chance to draw with Magic Markers--something that most children of that age find very appealing. Some of them, however, were told that if they drew pictures they would each receive a special, personalized certificate, decorated with a red ribbon and a gold star. Between a week and two weeks later... those who had been [promised a reward] now seemed to be less interested in drawing with Magic Markers than the other children were--and less interested than they themselves had been before the reward was offered.

It turns out that there are two types of motivation: extrinsic motivation and intrinsic motivation. Extrinsic motivation is the type that comes from without: driven by rewards, praise, punishment, or threats. It's short-lived and fickle. What gets rewarded gets done, yes, but it's done sloppily, cheaply, and always in search of a greater reward. Further, extrinsic motivators tend to destroy whatever intrinsic motivation was already in place.

Intrinsic motivation comes from within. It's what keeps programmers working late into the night, staring at the screen until stomach and bladder demand relief. It's what gives us open-source projects--not the big corporate ones, but the vast majority, the ones created by a programmer or three in their spare time. Intrinsic motivation is powerful and long-lasting.

So, what motivates programmers? Intrinsically motivates them? We've known that for years. Steve McConnell provides a list in his book Rapid Development. The list is a bit old now (sources from 1978 and 1981), but it still rings true. Here are the top ten:

1. Achievement
2. Possibility for growth
3. Work itself
4. Personal life
5. Technical-supervision opportunity
6. Advancement
7. Interpersonal relations, peers
8. Recognition
9. Salary
10. Responsibility

The key to applying this list is not to try to manipulate your programmers (and other team members) by selecting items to dangle like carrots. Instead, make these things part of your workplace, to the degree that you can, and make them available unconditionally. Then hire people who love to work in the type of environment you have.

"What gets rewarded gets done." It's true... I'm not denying it. As Alfie Kohn says, "Do rewards motivate people? Absolutely. They motivate people to get rewards." But rewards are extrinsic motivators. As an answer to the problem of motivation, they're clear, simple, and wrong. What gets rewarded gets done, yes, but it won't be done well. To get the best results, rely on intrinsic motivation. Find out what intrinsically motivates people on your projects and make that an unconditional part of your work environment.

I'm a big fan of continuous, hands-free backup. I've been using a Mirra for years for that purpose. Mirra's UI is a bit clunky, though, and it doesn't back up the whole disk. That turned out to be a huge problem when my hard drive crashed a few years ago. I've been looking for a replacement for a while now. Unfortunately, there hasn't been anything as simple and hands-off as Mirra.

Until Time Machine, that is. Time Machine is the automated backup utility built into the latest Mac OS release. With the 10.5.2 release, they added support for network backup if you have an Airport Extreme base station. That was enough to convince me to upgrade, and I'm glad I did. It works like a charm.

Well, almost. That initial backup of 100GiB or more is a tad slow over a wireless network. Actually, saying it's "slow" is like saying "the surface of the sun is hot", or "code without tests is annoying." It's mind-numbingly slow. Painfully, tediously, stupendously slow. It took my wife's computer 48 hours to do the initial backup. And then (because I was futzing with hard drive configurations) I had to wipe it and start over. Twice. That's when I decided to find a better way.

I wanted to speed up the process by doing the initial backup locally, not over the network, but that didn't work. For some reason best known to Apple (and completely obscure to me), Time Machine backups are stored in a disk image when you use a network, but in simple directory when attach the disk locally. You can't just move a backup disk to the network and expect it to work.

Luckily, I figured a way around this restriction. You can conduct your initial backup over a fast USB or Firewire connection, then move the backup drive to your Airport Extreme. Here's how.

(This isn't as hard as it looks. I'm just being very detailed.)

1. Prep the Disk on the Network
   1. You'll need a backup drive (preferably erased), Mac OS 10.5.2, and an Airport Extreme running firmware 7.3.1. (Later versions might work, too, in both cases.)
   2. Hook your backup drive to your Mac, wait for it to show up in the Finder, then eject it. (This causes necessary hidden files to be written to the drive. You probably already have those files, but do this anyway, just in case.)
   3. Hook your backup drive up to your Airport Extreme.
   4. Navigate to the backup drive in the Finder. That makes it available to Time Machine.
   5. Open Time Machine preferences.
   6. Turn on the "Show Time Machine status in the menu bar" option. You'll use it later.
   7. Select "Change Disk." Your backup drive should be listed, with the name of your Airport Extreme in parentheses. Choose it.
   8. Select "Back Up Now" from the Time Machine menu bar icon.
   9. You should get a progress bar that says "Preparing..." This will take a while--mine took about five minutes. Wait for it.
   10. When the progress bar no longer says "Preparing..." and is showing data being copied, press the little X to cancel the backup. (Note: don't cancel while it says "Preparing!" This seems to get your backup into an unrecoverable state, and future backups will "Prepare" forever. You can fix that problem by deleting the backup image.)
   11. Turn off Time Machine. This will prevent backups from triggering inappropriately during the rest of this process. (Note: to avoid the indefinite "Preparing" bug I just mentioned, don't turn off time machine until after you cancel the backup!)
   12. In the finder, navigate to the root of your backup drive and open the Time Machine disk image. Mine was named Jim's Computer.sparsebundle.
   13. Navigate to the newly-opened image. It will have a single directory in it named Backups.backupdb. Delete it. That's right--you'll now have an seemingly-empty disk image. (In reality, there are some important hidden files left behind that make this trick work.)
   14. Empty your trash.
2. Do a Local Backup
   1. Eject the backup drive in the Finder, unplug it from your Airport Extreme, and plug it into your Mac.
   2. Select "Change Disk" in the Time Machine preferences and pick the backup drive again. This will automatically turn Time Machine on; turn it off again.
   3. Select "Back Up Now" from the Time Machine menu icon.
   4. Wait for the back up to complete. Rejoice as it takes hours, not days.
3. Move the Backup Drive to the Network
   1. Eject the backup drive in the Finder, unplug it from your Mac, and plug it into your Airport Extreme.
   2. Navigate to the backup drive in the Finder so it will show up in Time Machine.
   3. Select "Change Disk" in the Time Machine preferences and pick the backup drive again. This will automatically turn Time Machine on; you can leave it on this time.
   4. Select "Back Up Now" to prove that it works. It will spend a while "Preparing" (mine took about two minutes) and then you're done!

My son, Kiran Colin Shore, born 28 March 2008.

(His sister.)

O'Reilly has been offering a 35% discount on The Art of Agile Development. That discount ends in less than a week, on the 31st. It's the best deal I know of, so if you were planning to pick up some more copies of the book, now is a good time. To take advantage of the discount, go to the O'Reilly Store and use coupon code JVAGL. Again, that code expires on March 31st. (Sorry, it only works for US and UK customers.)

On another note, the bonus features on the book's site are still going strong. By the time I'm done, I'll have produced an agile pocket guide, a four-part epic haiku, and commented on every part of the book. (How do I get myself into these situations? Shouldn't I be kicking back and eating peeled grapes or something?) Anyway, although it may not be the most comprehensive book companion site yet produced, it's pretty substantial. If you haven't taken a look, go check it out.

Finally, a tip of the hat to Michael Denomy for his agile haiku. He's right--any day you get to use the word "detritus" has to be a good day.

Updated, May 21: I didn't receive enough responses to this, so no results are forthcoming. Sorry!

I just published the self-assessment quiz from The Art of Agile Development. I thought it would be interesting to collect a bunch of answers from people. It could show us where people tend to do well, and where they struggle.

This is far from a scientific survey--selection bias and all that--but it could be interesting nonetheless. Take the quiz and let me know how it turned out! Send me the five totals from the quiz, along with a little information about your team: how big it is, what method you use (it doesn't have to be an agile method), whether or not you're using a green-field or legacy codebase, and whether or not team members sit in the same room.

I'll publish the aggregated results here in a month or two. No personally-identifiable information will be shared. You can reach me at jshore@jamesshore.com. (Follow that link to create a pre-populated email message.)

What's more important, using agile practices or understanding agile principles and values? Both! But what if you had to pick? Consider this:

If you're happy, you smile. But did you know that, if you smile, you become happy? Somehow the act of smiling connects your brain to a feeling of happiness.

This is interesting to me because, like anybody, I get grumpy. If somebody saw that I was down, she might say, "Cheer up!" or, "Don't worry, be happy!" (Annoying, isn't it?) I already know I need to be happy. Sometimes I just can't seem to make it happen.

That's what makes the smile research so interesting. If somebody told me, "Try smiling for ten seconds... it seems silly, but it will make you feel better!," I might actually try it. Unlike, "be happy," it's something I can actually do. And, as it turns out, smiling actually does make you happier.

Practices or principles? I don't know. The principles are important and valuable. But if I could only have one, I think I'd choose conscientious and careful application of the practices, in hopes that the act of smiling would help me rediscover my good mood.