Compare Bugs? Oh No.

Question: I was doing a bug trend analysis yesterday to plan the future testing and found that the project had huge number of bugs when compared to the other projects. What should I usually look for in this case when there are huge number of bugs? The project is a technical change and am regressing to check the functional impact.


  1. Assume the bugs trends you drew up is based on a bug-database which is optimised (reviewed and triaged) for correctness and reliability.
  2. When you say “plan the future testing” assume you mean that you will use this (bug colonies for example) to prioritise testing and focus efforts which is logical. Please do this but keep your wits about you as far as the overall plan of testing is concerned and what it has to achieve. (That is don’t let the bug count bias or hijack your thought-process even if the alarm bell has been rung by a higher-up).
  3. It maybe foolhardy to compare the bug count across 2 or projects and draw conclusions in the manner you are doing. For all you now, the bug count in your project may be less than what it is if you are comparing apples & oranges. So ask yourself the questions “Are 2 projects ever the same even if the same functionality is developed by 2 teams” and “What am I comparing and am I justified in doing so”. All I am getting to is comparing might be a hole you are digging and standalone analysis of the application under test from different angles maybe more worth your while unless you have very strong justifications to compare.
  4. The large number of bugs found could have any number of causes being complexity of application mismatched by developer skills, inadequate developer skills, lack of induction in to the domain for the developers, requirements knowledge not articulated or gaps in communication, lack of effort, inadequate planning or time allocation, very skilled testing team ;)) and so on… Adequate thought into such aspects along with discussions with the critical team members is a precursor really.
  5. Assume you have also segregated counts of the kind of bugs raised by module, severity & type (functional/UI/config./data/etc.)
  6. Assume you will use the analysis to correct course of testing and help better its value for the project/organisation rather than it being a mundane process exercise for fattening your organisation’s process repository.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s