mikeem em's Library tagged blog →  View Popular

Acceptance tests prove that the specific

component/feature is sane in the context of the fully deployed product

feature/component works as intended

Unit Tests (and by most measures, smoke and simulation tests) are under the ownership of the core developers.

verification that the product

This diagram is about cost - for each layer the code/feature passes through emanating from the developer, the cost to the team, and the difficulty in identification and resolution climbs.

The cost for a developer to find a bug with a unit test, and the cost to fix that bug introduced through new code/refactoring/etc, is essentially 1.

Essentially, it is in your best interest, as a developer, as a team, to encourage lots and lots of tests lower in the stacks shown here

Why? Because - as you move higher in the stack, that damned bug someone checked in is hidden behind layer upon layer of code

Now, your bug (our bug) has not only wasted your time, it’s holding up a release

God help you if your bug gets to level (F). This is called the “aversion level” because after a few of these sneak out, and the CEO of the company starts getting phone calls at 4am

Your goal is to avert bugs from reaching Level F. F stands for F’ed in the literal sense.

you have to give your teams time to test. You need to give them ample time to automate what is reasonable

need to be willing to not ship a component or feature that simply isn’t ready.

Letting preventable bugs get in the hands of users is not avoidable - but the risk can be mitigated, and many bugs that do end up in the hands of users are avoidable.

Years ago (1984 to be exact), the company I was working for bought a "quality" course to give to their employees. It was complete with workbooks, videos with 70's hairstyles, etc. One of the main things that stuck with me in that course was what they called the "1-10-100 rule". (Interestingly, I just failed in trying to google this rule. Maybe they predated the net by too far.) Anyway, the crux of this rule was equivalent to one of your main points. If the developer finds and fixes something before anybody else is involved, it costs "1". At the next level of testing, it costs "10", Then "100", "1000", and if the customer gets it, it could be up in the millions.

Fast forward a decade to 1994. The company I just started working for used exactly the same video course. The only things that had changed were that I had 10 years of experience since the last time I took the course, and the hairstyles in the videos looked even more ridiculous. When the instructor asked if there were any questions after going over the 1-10-100 rule, I raised my hand and asked "What does (the company) do do recognize and reward employees who find and fix things at the 1 or 10 level?" He seemed taken aback, and mumbled something about God and Country and the Right Thing To Do. I persisted: "Obviously, for (the company), it's good when their employees take this to heart, but how does the company prove that they like this to the employees?" More blathering from the instructor including "Look, OBVIOUSLY you will be rewarded better if you find and fix things earlier." to which I replied "Really? In my experience, companies recognize and reward employees who fix things at AT LEAST the 10000 level. Those are the employees whose names are known by the CEO of the company, and they get a pat on the back, and money, for putting out fires at the customer, even if they are the SOB who screwed it up in the first place."