Giving Testing its due

Someone on Linkedin asked how to emphasise Quality in 3 steps in an organisation where it is apparently given the short shrift or not given the importance it deserves, I had this to say.

First up define in your head or clear the fog around the word ‘quality’. The word means so many things that I hesitate to use it in a generic manner. Alternatives I suggest for use is better functionality, easy to use software, fewer defects in production, fewer customer escalations, better test coverage, improving tester skills & so on….

Now in your current workplace if ‘quality’ is not given its due, my suggested course of action is as below.

Why is this is the case? Being in this place maybe you already know. Is it around people, process, tools, interpersonal relationships…? You need to understand the concerns why “quality” is not given its due. There should be no reason why it is not given importance because most people understand the value it brings. So try clear the cobwebs / misunderstanding / misgivings around it by first understanding what the malaise is and then go about systematically prescribing (rather reiterating) the virtues of how good testing can deliver the cure (emphasising all functions working together). Do this in a most strident manner, from the heart and justifying it with whatever you can find on the web (pre & post scenarios, debacles & what have you). Paint a holistic picture of the future scenario you visualise keeping in mind the organisation & the customer and you will be listened to. Be one-sided (whining selfishly about testing’s concerns) and you will be side-stepped and probably you will sink deeper into the hole than you already were.

Once you have the buy-in set about repairing what is broken within your function & its interplay with others. Maybe the processes, people, their skills, their confidence and so on. Collaborate and speak with the best you can find within and outside your company for the solutions. Ask for & arrange training, dig the web, mentor the guys.. Oh there are umpteen things to do but what, why & how much depends on your scenario…. Keep a check on progress..using metrics, trends, coverage analysis etc. intelligently & wisely.. drive & do not be driven by the numbers… check with the people who matter asking them for their feedback on progress.. make corrections.. as you near the horizon.. re-plan for new goals. Change is the only constant.. Evolve. Progress. Keep your ears on the ground & feet on the earth.

Empower people in your team to eat, drink and breathe ‘quality’. Every person in testing must be an ambassador of your team’s beliefs, vision and good practices. For everyone to speak the same language there has to be consensus within the team & for consensus it is important for leadership to do its role properly else the tail will wag while the mouth barks which is not what is needed.

Domain knowledge by chance

Should a tester acquire domain knowledge from outside to gain an advantage over others and give that supposed golden tinge to his resume?

If I have to answer this in one word I would say No.

The priority in my view goes to testing and technical (programming/scripting/tools) skills. So the tester should focus on what/how/how much to test so as to bring out information and unwanted defects faster and better than ever.

To become a better tester – he needs to listen to and learn from testing masters (one must listen to his inner self which becomes better with time if one consciously builds it and feeds it – one must never be a blind follower – Beware! the testing planet has groups bordering on cultism) be a part of testing forums, read and write about testing experiences, be open minded to changes and talk every night with oneself noting the rights & wrongs….. While he learns testing and gets better it would help if he understands the world that the developer lives in and picks up understanding on good coding practices, common development mistakes and gets acquainted with their thinking, what makes him proud or queasy and so on… The tester would benefit immensely by learning some scripting language and  gets the developer to mentor him and review his scripts. This bond would help not only him professionally but the team as a whole because the scripts can then be put to interesting uses like populating data, writing, reading logs and pick out errors and so on.. While getting on back-slapping terms with the developer the tester has to keep his pessimism and the reason of his existence uppermost in his mind and not fall into bad habits that the ‘smart’ developer might influence him into.

As far as the domain skills are concerned – when the tester gets into testing for a project – he must learn everything about the domain allotting time (through the supervisor of course) to it such that he does very good justice to it while performing the other testing activities. When learning the domain he must read the documents available and speak with the business users. He must make copious notes, share, brainstorm with colleagues, ask lots of questions, (learn mind-mapping) and so on… Good knowledge of the domain will ensure he tests by delving more deeply into areas that the business users would be looking critically at.

With experience the tester would have domain knowledge not really by choice but say because he worked with organisation(s) which catered to that segment/vertical or  he was allotted a project on purpose with a domain that he has worked with previously.  The collected expertise of the tester in domain and his big-picture knowledge opens the Business Analyst role also for Testers which is a new avenue to trudge… I think this is better than spending time on learning a domain by time-slotting it as against spending priority time on sharpening  testing / technical skills…

Grit

Determined at all costs!

I read this very good post on STP written by Bruce Butler and commented on it (as below) which raked up some emotion within me.

Your words in many places would appear to be rather caustic but I can understand the emotion behind it.

The barbs and the bitterness the job of reporting defects etc. brings about is sometimes hard to take in and I have been witness to my people ventilating these feelings choked as they are sometimes with red-shot eyes. I have also experienced it during my interactions with the development teams & their leaders. The atmosphere is dark & sombre but the job has to be done well by keeping calm & your maturity about you. If you have to hit out you simply must by finding more defects – drink the bitterness but pass the poison. It is nectar that we must try and eventually send to the customer.

The only consolation in words & visual imagery I would give my people & myself is that we are doing a service and someday the worth of finding the defects/ information/issues would be realised when the user smiles after using the product even though we may not be there to cherish it. I think we just need to knuckle down and press on… and peel the onions as you rightly said … even if we have to shed bitter tears!

The link to the STP article is here.

Compare Bugs? Oh No.

Question: I was doing a bug trend analysis yesterday to plan the future testing and found that the project had huge number of bugs when compared to the other projects. What should I usually look for in this case when there are huge number of bugs? The project is a technical change and am regressing to check the functional impact.

Answer:

1. Assume the bugs trends you drew up is based on a bug-database which is optimised (reviewed and triaged) for correctness and reliability.
2. When you say “plan the future testing” assume you mean that you will use this (bug colonies for example) to prioritise testing and focus efforts which is logical. Please do this but keep your wits about you as far as the overall plan of testing is concerned and what it has to achieve. (That is don’t let the bug count bias or hijack your thought-process even if the alarm bell has been rung by a higher-up).
3. It maybe foolhardy to compare the bug count across 2 or projects and draw conclusions in the manner you are doing. For all you now, the bug count in your project may be less than what it is if you are comparing apples & oranges. So ask yourself the questions “Are 2 projects ever the same even if the same functionality is developed by 2 teams” and “What am I comparing and am I justified in doing so”. All I am getting to is comparing might be a hole you are digging and standalone analysis of the application under test from different angles maybe more worth your while unless you have very strong justifications to compare.
4. The large number of bugs found could have any number of causes being complexity of application mismatched by developer skills, inadequate developer skills, lack of induction in to the domain for the developers, requirements knowledge not articulated or gaps in communication, lack of effort, inadequate planning or time allocation, very skilled testing team ) and so on… Adequate thought into such aspects along with discussions with the critical team members is a precursor really.
5. Assume you have also segregated counts of the kind of bugs raised by module, severity & type (functional/UI/config./data/etc.)
6. Assume you will use the analysis to correct course of testing and help better its value for the project/organisation rather than it being a mundane process exercise for fattening your organisation’s process repository.

Test Confidence

Someone on Linkedin asked a question on how-to derive a “Test Confidence” ratio? Here is how I responded…

I think a “Test Confidence” is best derived by experiencing the application first-hand over a period of time or soaking the expert opinion of someone or some people who have. Whether it is bugs or issues or plain-speak about application behaviour / functionality / performance / usability / other ‘ities’ what you/they feel based on tests that have been performed will be very helpful indeed. Maybe a questionnaire can be compiled with some rating system for the tested application. If this is answered by people who know the business that the application supports and have diligently traversed its length & breadth like a passionate tourist would in a city he has visited armed with atlases and guides’ in tow then you have information which is reliable and can both support & justify your cause.

This information is rarely captured and presented since management trust numbers being the easier to present n peek and graphs based on it are such pretty things.

Numbers/Metrics will have to be analysed and interpreted carefully.

If it is a decreasing count of defects across various phases of testing or across different cycles of the same phase (say System Testing) you will have to determine that the environment is the same (different browsers, versions, OS, versions, etc.), testers are the same or having the same skill, build has been more or less consistent in terms of features developed / delivered for testing and fixes have been brilliant and not caused any regression and there has been as much time spent actually on testing as planned by you. If this is the situation you are in – your job is done.

But as you know, it seldom is.

Metrics with weights for each factor mentioned above to make the eventual number less or more maybe the way to go if there is a lot of pressure on you to produce a ‘Test Confidence’ number. The risk is that the numbers more likely gets convoluted beyond a point and may stop telling the story you set out to. If this happens, then defending the numbers rather than providing an no-holds barred assessment of the application tested would I think jeopardize the existence of both the testing team and the organisation it belongs to!