The only way to deliver a zero-defect product
Testing is an intrinsic job of the engineering team. "Quality engineering" needs to bring value across the product lifecycle, not just write tests after-the-fact.
Welcome to those of you new to Customer Obsessed Engineering! Every week I publish a new article (plus occasional freebies, like this one), direct to your mailbox if you’re a subscriber. As a free subscriber you can read about half of each article, while paid subscribers can read all of it. For some, half is great! But if you'd like to read more, please consider a paid subscription!
I have to say it: I’m tired of seeing software testing handled as a specialized silo. I’m tired of seeing testing happen downstream, after the software has been written. I’m tired of seeing quality “injected” after a product has been built. It’s ineffective, and it devalues testing as an intrinsic component of software design.
Silos like this have their roots in industrial design. It dates back to when industrial process was re-crafted and applied to software. The problem is, it doesn’t work — and we knew that back when the idea was first introduced. Unfortunately, as an industry we forged ahead with a bad idea anyhow.
On an industrial assembly line, quality can be tested after the fact. In this setting we are inspecting for quality, not adding quality. For example, we’re checking to see if the weld joints on an airframe meet the tensile strength standards we expect. If the weld fails the product fails — and often the product is just thrown out. But in software, it results in a negative feedback loop — requirements are changed, and the team has to redo work to a new standard.
It’s inefficient, poorly thought out, promotes poor design up-front, and leads to poor quality in the long run. I don’t like being poor.
Which is why I say: I’m tired of seeing software testing handled as a specialized silo.
Testing is engineering’s job
Engineers are excellent at testing software, and even more so when they understand a product’s value proposition and technical design. I’m a huge advocate of Test Driven Development. I can’t even remember the last time I started a project without first thinking, “How am I going to test this? What’s my API look like?” TDD leads to better designs and better APIs.
Furthermore, the only truly zero-defect products that I’ve seen “coming off the assembly line” have been engineer tested. Not “downstream, siloed, QA-team tested,” but actually fully tested and delivered by the engineering team. This is the only path to creating zero defect software: The engineers must own product quality otherwise defects will be created.
Once defects are created, they are more expensive to engineer out of the system (dramatically more expensive, if they make it all the way to production).
Design is engineering’s job
We need intentional design very early in the Software Development Life Cycle (SDLC). If engineering is responsible for testing, they need to understand the acceptance criteria. If engineering understands the acceptance criteria, the product quality will be higher. In order to understand acceptance criteria, you have to own the design.
Design needs to include quality outcomes — in other words, the same engineers who are figuring out the technical implementation need to be figuring out how it’s going to be tested, and how it’s going to meet design goals.
It can’t be broken up and still be an effective, efficient process.
Testing as a path to engineering
Just about every test engineer I’ve met has been on a career path to software engineer. There have been a few that loved testing and chose to stay there, and that’s fine. There’s always room on my team for a skilled engineer that likes to focus on testing only — so long as they participate in the design process and support their teammates by collaborating on the tough testing problems.
But most have entered into the software testing path looking to “level up” to engineer, senior engineer, and someday full-stack engineer or architect. I think it makes a lot more sense to think about “test engineers” as “junior engineers.”
In this context, there is no such thing as a “test engineer.” There are just “engineers,” and some of them are going to have very specific skills (testing), with a mindset to learn more.
Quality engineering is about inspections, not testing
There is a path that focuses on quality assurance (we can call this “quality engineering” or “quality assurance”). It has absolutely nothing to do with writing software tests.
A quality engineer is focused on engineering quality into the product at all stages of the life cycle. This means:
Making sure that the customer is in agreement with feature specifications, and that the specifications are well stated.
Ensuring engineers are following good design standards.
Reviewing designs with the engineering team to ensure compliance and industry standards have been met.
Making sure the acceptance criteria have, in fact, been tested for.
Making sure the delivery pipeline enforces agreed-upon quality standards (such as quality gates, secret scanning, and penetration testing).
Inspecting against quality standards for documentation, user experience, and function.
Instituting systems that ensure defects are captured, remediated, and prevented from recurring.
Looking “around those corners,” trying to anticipate problems before they arise.
And yes, spending time with engineers to inspect their test strategies and ensure they are “fit for purpose.”
These are all activities rooted in inspection, not implementation. There’s a lot of work to be done here; staying on top of the product and all its artifacts, and making sure that every aspect of the project, every artifact, meets quality standards. But it’s not testing. It’s traditional quality assurance.
Specialized downstream testing
90% or more of testing should happen in the engineering team itself, usually by the engineers. That leaves room for some “specialist” downstream testing. That downstream effort should focus on challenging test outcomes that would otherwise be hard for the team to adopt.
For example, highly scalable performance testing. Testing performance at scale requires access to considerable infrastructure and often expensive tools. It probably makes sense to have a team that specializes in doing the final round of truly scalable performance testing — but even here, foundational performance testing can be done by the engineering team. Build out the basic framework, test the easy stuff.
Trust me, scaling it up will be easier, and the end product will be better.
Giving product quality the attention it deserves can only happen if we put due attention on test design and implementation. It can’t be an afterthought — and by moving testing into a downstream silo, that’s what it becomes: An afterthought. The right approach is to integrate test design into the up-front software design stage, be responsible for testing as an engineer, and take ownership of delivering a defect free product.