Solving the Defect Clustering Problem

It’s quite common for testers to encounter defect clustering and other quality issues as they build more complex products. How can we keep them from bringing down the quality? Let’s find out.

Often, defects are not evenly distributed in an application. The majority of quality issues can be caused by a small number of features in an application. This is called defect clustering. The reason for its occurrence may vary from legacy code being prone to breakage, to newer features that are undergoing frequent changes, to a particularly fickle 3rd-party integration. Whatever the reason, identifying the defect-prone areas with the right mindset and a suitable defect tracking software is a must-have.

Dealing with defects can be tricky. Often in pursuit of fixing defects, developers end up creating more defects when they make changes in the code. And when they try to fix those, they end up creating even more. Because of this multiplying effect, it’s often better to leave the code unchanged.

You have reason to believe that you’re dealing with defect clustering if:

Minimizing defect clustering would require the improvement initiative to focus on the specific software around which the majority of issues revolve, be it a product feature or a codebase. This certainly doesn’t mean that you should abandon everything else, but relocating few extra resources in the targeted technology can make a great difference.

For instance, if the majority of customer complaints revolve around a certain aspect of a product, focusing on the improvement initiative solely around that aspect can help a great deal. It might require you to borrow product managers or some developers from other projects and re-assigning them for a month or two to build up test coverage or automation around that feature.

Developers and testers often make the mistake of relying on their intuitions rather than facts. Cognitive biases often get the better of them, causing an error in judgment and forcing them to make mistakes. In the era in which defect tracking software programs are solving almost all problems, experts believe that cognitive biases are the reason why testers/developers end up asking themselves, “How did I miss this bug?”.
To avoid this, you should be a believer in data-driven quality improvement. Bleacher Report’s Quentin Thomas, while sharing his experience, explained how he managed to take a trove of defect data from his organization to prove that an old base, that was only being lightly maintained, was the source of the majority of bugs. Rather than suggesting the organization allocate more resources into coverage for that code-base, the data showed that maybe it needed to be abandoned altogether.

“The data gave us the ammo as QA to say “Hey, why don’t we just consider phasing it out, because it is causing us a lot of issues and that’s better than trying to spend all this time to test and analyze this stuff,” he says. “Sometimes getting rid of a service no one is maintaining is going to do a better job of improving quality than anything QA can do.”

This strengthens the point made earlier that sometimes, it’s better to leave the codebase unchanged or abandon an old one altogether rather than spending time and resources just to find out that the problem has gotten worse.