I don't hire testers anymore and I'm still shipping zero-defect software

Downstream testing is a mistake. Modern companies (like Microsoft) do something different. Move toward strategies that embed quality early.

Photo by Ashim D’Silva on Unsplash

I haven’t hired a tester on any of my teams for at least 15 years. Even so, I insist on zero-defect software in production and won’t settle for anything less.

So, how do I get zero defects if I don’t have any testers?

To better understand this, we need to talk about why I don’t hire testers. In my mind there are really three reasons.

“Downstream” is too late

The first, and the most important, is simply that downstream testing is stupid. Inefficient at least, and useless at worst.

The very nature of a specialized “tester” role is to test software after it’s built.

Let’s think about the mechanics of downstream testing for a moment.

First, requirements and engineering specifications are created. Later, engineers turn those into working code. Somewhere along the line, acceptance criteria have hopefully been created but often this is poorly done. Late in the game, after code has been written, testing finally gets serious attention. It’s too late. The defects are already baked into the product.

What’s more, testing like this is so far removed from the original goal that it’s almost impossible to maintain consistency and relevance. Your value stream is inevitably broken — you end up with flawed software and a team that suffers through multiple break-fix cycles, trying to recapture that lost value.

I have a better way. The defects should be prevented, not removed. We want to engineer quality into the product — not add quality to a flawed product.

If you’re new, welcome to Customer Obsessed Engineering! Every week I publish a new article, direct to your mailbox if you’re a subscriber. As a free subscriber you can read about half of every article, plus all of my free articles.

Anytime you'd like to read more, you can upgrade to a paid subscription.

Engineers are the best testers

The second problem I have with “bolt-on testing” is who is writing the tests. After decades of observing the difference between downstream testing and engineers building the tests up front — I’ve decided engineers are way better at engineering tests.

Not only have I proven this empirically over dozens of projects and teams, it also adds up from my personal experience.

As an engineer, I’ve got more engineering experience than most testers, which means I can engineer more robust tests. I also understand where my code is likely complicated and where it’s straight-forward. I know what was difficult to implement and what was “off the shelf” and easy.

The fact is, engineers tend to be tougher on themselves than someone else. For every engineer that has really stress-tested and pushed the limits of their own code, I’ve seen half a dozen testers throw together a bare minimum flawed test and call it done.

As an engineer, I take pride in shipping a perfect feature, something that performs flawlessly with no bugs. I think most engineers are like this. I want to harness that self-critical nature by giving engineers the responsibility and the tools to build better.

Juniors shouldn’t be in charge of quality

Product quality is perhaps the most important thing to focus on. I don’t want juniors responsible for the quality of my product. My third problem is, most testers are junior.

There are essentially two career paths leading from tester:

1. Graduate to become an engineer. This is where the numbers are. It’s what I see happen most often. Testing is a relatively easy path into engineering, but it also means that most tester are juniors.

2. Become a senior quality engineer, perhaps with highly specialized skills in healthcare, finance, or safety-critical systems. This is a critical domain specific role, but it’s typically not who you’ll be working with in most downstream testing cases.

While the latter is incredibly important and valuable, it’s also specialized and rare. I find this is especially true in all cases where an organization chooses a downstream testing approach.

Putting the responsibility for engineering quality into a junior’s hands isn’t a great idea. We need to find a way to put that responsibility in the engineer’s hands and, even more important, shift the quality insertion point “left,” earlier in the product lifecycle — in the early engineering stage.

Could I ask a favor? Referrals help immensely — plus, you earn free premium access. If you could share this story with a friend I’d be very appreciative!

Embracing “shift-left”

If we aren’t going to rely on downstream, specialized testers to enforce product quality, what’s the alternative?

We have to shift the “quality insertion point” left, as early in product development as possible. That means injecting quality before code gets written whenever possible.

I routinely shift 80 or 90% of that effort left and remove nearly all downstream testing.

What remains is intrinsically “downstream.” In other words, it can only happen downstream — for instance, usability testing with real users, accessibility testing, and validation of subjective qualities like user experience. Mostly, these need to happen after there’s a working product to play with (although we can learn a lot from interactive, mocked-up demos — so, again, there’s still room to shift-left).

By intentionally changing where we want to insert quality, how we do it becomes clear.

Getting to zero-defect delivery

The most important mental shift is away from “testing” and into “defect-free coding.” Remember, we don’t want to concentrate on removing defects — we want the code to be free of all defects from the moment it is created. That starts with our acceptance criteria.

Too many teams focus on demos. A demo that shows off a feature is inconsequential without proving it passes its acceptance criteria.

The goal I establish with my team is that code should be proven defect-free before it’s merged into the product.

Acceptance criteria, after all, represent a contract. They define our delivery objectives. It only makes sense we ensure the contract has been fulfilled.

Approaching zero

Attaining zero-defect product delivery is not an overnight journey, but you can get there. Your first step is that mental shift to embed quality early. But there are specific strategies I rely on, which I’ll cover below.

To be clear, “zero-defect delivery” is a goal, a standard. No team is going to be perfectly zero-defect all the time. But, you can be so close to zero that the occasional bug that pop’s up is a rounding error. Your statistical average for defects can be a small enough fraction-of-one that it is “approaching zero.”

This means your typical sprint will be truly defect free. Features will flow without coming back to the team for corrective action. “Incident capture” will start to look a lot more like “feature requests.”

What is Quality Assurance?

Let’s think about what it takes to, “engineer quality into the product, not engineer defects out.”