I said yesterday that

The skill sets [of developers and QA people] aren't simply unrelated—they are, to an extent, opposed.

 I think that deserves some explanation because for some reason this is not obvious even to people in IT.

The reason that developers and testers require fundamentally different skill sets is that they have fundamentally different responsibilities. What it comes down to is that it is a developer's job to make software work and it is a testers job to make "working" software break. In fact, I knew I had found the right QA Manager for XanGo when one of the first things he said to me was "hey, you wanna see something cool?", opened Excel, and demonstrated how to bring his computer to its knees in three easy steps. I know I'm not much of a tester because while I did think it was cool, I had forgotten those steps practically before he had finished his demonstration.

Separate is Better

When you are creating software, you want to take the quickest path to a functional application. Your emphasis is, correctly, on delivering an end product. You don't want to spend time thinking of every possible situation that can go wrong. You make your program as robust as it needs to be given its use and the importance it has to those using it.

When you are testing software, you want to take the quickest path you can find to break the software. You start with common uses and expand from that point to ensure that the finished software actually does all the things it is intended to do.

Is the skill set used for testing software useful for developers in creating it (and vice versa)? I get this question sometimes from people hoping to save the budget for a QA department. Their real question is "can't we just tell the programmers to do a better job testing their software?" The answer to that is "you save money, but only in the short term."

The thing is, we've found as an industry that testing works best if you have a group of people whose sole responsibility is to test software. This makes sense if you look at it as a case of separation of interests leading to a better combined result. This is why I consider the two skill sets ultimately opposed. When looking at a piece of software, is your intent to make it work or to break it? You really cannot do both at the same time and do them both well.

I'll take it one step further: developers are better off cultivating a perspective of getting their software out the door and testers are better off cultivating a perspective of preventing software from getting out the door until they cannot break it. Could someone, technically, develop both skill sets? Absolutely. I even know one developer who seems to have both built-in. I just don't see why anybody would want to cultivate both when companies are much better served having those functions separated.

Rarity and Value

I also stated yesterday that

QA is a wholly different skill set from development. A skill set, by the way, that is rarer (and hence more valuable).

I think this may be the more controversial statement because testers tend to be paid less than developers of comparable experience. I think this is unfortunate and based on being able to get away with it more than it represents true market value. Testers are often seen as Jr. Developers and that's unfortunate. I think this is possible because those who are truly good at testing are extremely rare. If you've ever had the chance to work with a top tester, though, you'll know what I'm talking about.

The best testers are meticulous to a degree that developers simply find unnatural. They test every edge condition and look for new ones wherever they can. They are almost obsessively detail oriented, even when they don't have to be. And they record everything they can think of.

Top testers improve the developers they work with. That's because a top tester will document fail points and develop hypotheses about why the errors occur in addition to where and under what conditions.

In addition, top testers analyze more than just the software they are testing. They'll assemble things like common fail points and patterns of error--mainly for their own benefit. Indeed, I've found that I sometimes have to dig to get their analysis because they sometimes don't realize the value of their findings to developers.

More subtly, they'll review common bug reports and feature requests and identify areas of improvement that users simply haven't thought of. This comes from their familiarity with the software they are testing. It isn't uncommon for the testers to know the software better than the developers who created it. This makes sense if you consider that a developer often visits any given piece of functionality in mostly unrelated chunks and a tester reviews it as a comprehensive whole.

How Much Testing?

My first experience with the benefits of a strong QA team came at Jenkon in the 90s. I was evaluating our software estimates because they tended to be off by a relatively constant margin. When I started tracking project estimates against actual delivery over time (I first had to start keeping that data. You'd be surprised how few people actually keep it beyond project completion), I found that thorough QA takes almost exactly as long as creating the software in the first place. For some reason, our estimators had it in their head that QA only took a quarter of the development time. Correcting this misperception was instrumental in reining in constant project overruns.

I haven't seen this ratio change since then. Every time someone thinks it has changed, it turns out that they are either skimping on QA or they're not counting all the actual QA time spent. It's been a couple of years since I had access to a decent QA team, though, so I could be off.

Technorati tags: , , , ,