Software Testing - discovering and exploring !

What is software testing like? Is it for folks who got rejected in development? Is it for those who are without a job and looking for a toehold? Or is it meant to be a stepping stone to perhaps, "greener" pastures in the world of development? These questions crop up every once in a while when interacting with software testers or potential testers.

The fact of the matter is that there will always be cross-functional movement. For example, there will be a few folks from software testing who will fantasize about software development and make attempts to move at an available opportunity. This isn't necessarily bad and there is no need to berate your selection process much. Of course, if folks are jumping ship in droves, your selection process for testers surely needs a close look. Once new folks come aboard the testing bandwagon, it is a matter of time before they figure out whether they are truly cut out to be testers or would rather "follow the herd" so to speak. I am not trying to indulge in berating development or show testing as a superior alternative to any other function.

Software testing, just like software development is a valuable piece of the whole that enables a great product to be produced. You cannot do without either of the functions. You could use the analogy of your own body to denote their significance. A hand is no more or less important than a leg or the brain more or less than the heart - there may be special situations where one may be used more; however, on the whole the entire body is complete and healthy when all its component parts work together.

Back to our original question - what is software testing like and what kind of folks would find testing interesting? Software testing is about discovery. Testing, more than any other function involved in producing software, is about discovery. Software professionals who love discovering and exploring are likely to feel at home in testing. Testers are constantly trying to discover what they do not know in the system. In the words of Philip Armour, "The challenge in testing systems is that testers are trying to develop a way to find out if they don’t know that they don’t know something. This is equivalent to a group of scientists trying to devise an experiment to reveal something they are not looking for. It is extremely difficult to do." He goes on to state that much of what we call as method or process in testing involves heuristic strategies.

We selectively test complex predicate logic, we create test cases that span the classes of inputs and outputs, we construct combinations of conditions, we press the system to its boundaries both internally and externally, we devise weird combinations of situations that might never occur in the real world, but which we think might expose a so-far unknown limitation in the system. None of these are guaranteed to throw a defect. In fact, nothing in testing is guaranteed, since we don’t really know what we are looking for. We are just looking for something that tells us we don’t know something. Often this is obvious, as when the system crashes; sometimes it is quite subtle.

If all this talk of discovery and exploration sounds too much, you might want to re-consider your choice of testing as a career and probably look at other options where things are clearly stated and probably involve such exciting tasks such as translating defined specs into a computer language following pre-defined coding standards and guidelines or regularly fixing issues in the code you or someone else wrote. Discovering is definitely not for everyone.
***
Join my community of professional testers to receive free updates by email. Use this link to add your email address to the community. Rest assured, I will neither spam nor share your email address with anyone else. Email subscriptions are managed by Google's FeedBurner service.

Share & Bookmark this blog entry

Image: Salvatore Vuono / FreeDigitalPhotos.net

Software Testing - Time machine

Looking back at some entries on this blog from an year ago.
  • A brief look at the theory that test automation might eliminate the need for manual testers: Read it here
  • A look at errors introduced in test automation vis-a-vis manual testers: Read it here
  • A look at the age-old question about QA vs QC: Read it here
  • And an exhaustive look at the question, "What is Quality?": Read it here 
***
Join my community of professional testers to receive free updates by email. Use this link to add your email address to the community. Rest assured, I will neither spam nor share your email address with anyone else. Email subscriptions are managed by Google's FeedBurner service.

Share & Bookmark this blog entry

Image: Salvatore Vuono / FreeDigitalPhotos.net

Software Testing: Theory on Defect Detection

Theory: A defect can only be discovered in an environment that contains the defect. 

This seems very obvious! So why even bother to mention it, let alone post a blog entry about it?  The motivation for this entry comes from a familiar situation that i am sure most testers would have encountered. 

Software testers devise extensive sets of tests and execute them prior to a product's release. These tests are usually executed on setups that testers have prepared in their lab environment. Many organizations may have invested significant sums of money to have the lab infrastructure in place. However, what generally tends to happen is that when this product is released, the customer reports issues pretty quickly. So, what happened? What happened to all the man-hours spent on testing and the investment on expensive lab equipment? Why did our in-house testing efforts not show up these defects that a customer seemed to find "easily"? What are we doing wrong?

This brings us to our theory, i.e. a defect can only be discovered in a system or environment that contains the defect. A defect that might show up in a customer environment may not manifest itself in a sanitized lab environment. Often our lab environment is setup and controlled based on our view of how the product is likely to be used. Within the confines of the boundaries we have defined, we execute our battery of tests and feel confident when the tests run without reporting issues. However, a customer's environment suffers from no such boundary constraints and does not find it hard to expose a defect. Defects may not necessarily be in the product itself. It could arise from the interactions of the product with its operating environment, dependencies, usage, etc.

Therefore, unless we are able to replicate in sufficient detail the customer environment and the likely real-world usage scenarios after the product is released, we will likely continue to see an increasing trend of customer reported issues.
***
Join my community of professional testers to receive free updates by email. Use this link to add your email address to the community. Rest assured, I will neither spam nor share your email address with anyone else. Email subscriptions are managed by Google's FeedBurner service.

Share & Bookmark this blog entry
Image: Salvatore Vuono / FreeDigitalPhotos.net

Software Testing and the hammer and nail approach

If all you have is a hammer, every problem looks like a nail.

It is sometimes a similar situation with software testing and specifically with test automation. This may not be much of an issue for single-product companies but when your organization produces a range of products and has multiple teams of testers handling these different products then the likelihood of hitting the hammer and nail problem arises.

One might say that the problem is prevalent more in a centralized testing structure than in a decentralized structure. The issue I am alluding to is the mandate to use a common test tool, mostly for automation since that is an area that everyone would like to see standardized but poses most challenges to standardization.

I have come across many instances where there is the attempt to identify a common tool that can satisfy all the requirements for automating various products being tested by different test teams. In some cases, the products would be as different as chalk and cheese. For example, there was this situation where we had a suite of products addressing different customer needs. This suite has a thick client application written in Java, a Win 32 application, a complex Ajax based web application, a few server products that you interact with using CLI and APIs, some middle ware products and few mobile applications.

Trying to find a common tool to handle all of the varied requirements for automating the different products could lead us to three possible ways of solving them.

1. One, you talk to a few tool vendors who would naturally promise the moon and claim that whatever tool they are selling can automate every kind of application that ever existed and will come into existence in the undetermined future. You could take this option like many folks do and purchase expensive tools and licenses and then mandate your testing teams to go use these (and only these) to automate in a standardized manner. Simple solution using a brute-force approach.

2. The second approach may be to take the compromise route. Here, you realize that a single tool may not be able to handle all the unique requirements of automating each product and go ahead to procure a tool that probably results in being the lowest common denominator solution. The tool ends up being  a good enough solution which essentially means that it is neither good nor enough from a testing perspective. However, the organization gets a standardized jack-of-all-trades kind of tool that all groups could use with varying degrees of effectiveness.

3. A third approach could be to take the time to understand the specific requirements and needs for automating each product, perform due diligence in identifying & then evaluating the tools that best fit the specific automation requirements before deciding on what tools to procure. This may result in having to procure more than one tool. If your products have similar automation requirements you may end up with needing just one tool but in cases where you have products with differing needs similar to what I stated as an example earlier, you might realize that more than one tool is needed to perform effective test automation.

Effectiveness, is the key here. Ultimately you automate not for the sake of automating but to support your testing campaign and deliver value. Trying to force teams to use a specific tool that does not fully support their needs is akin to the hammer and nail solution. When all you have is a hammer, then every problem is dealt with as you would a nail. Sometimes it is the right thing to do and in many cases it may not be the optimal approach to follow. In testing, such an approach could lead to teams bending their testing and automation practices around what the tool is capable of while ignoring other possibly important areas of the product which are harder to automate using the tool. The automation tool should not dictate how & what you test. It should truly be a tool (amongst other tools and techniques) in your arsenal enabling you to tackle the challenges of testing your software.
***
Join my community of professional testers to receive free updates by email. Use this link to add your email address to the community. Rest assured, I will neither spam nor share your email address with anyone. Subscriptions are managed by Google's FeedBurner service.

Share & Bookmark this blog entry

Image: Carlos Porto / FreeDigitalPhotos.net

Software testers, sharpen your saw

How would you list the day in the life of a software tester ? Get in to work and test software ? I realize this is an over-simplification, but i am sure most lists would place testing as the task that would take up most or all of a tester's time. After all isn't that why testers are paid? Software testers are normally expected to test and utilize their time optimally to find defects in the software. Test all day and find issues. We live in a society that values busyness and activity. It is easy to get caught up in the frenzy of running tests and trying to find issues. Before you think that i am trying to advocate testers to not do testing, let me clarify - testers must test ! that is their primary job responsibility. However, testing isn't all that a tester must do if he/she must remain relevant and valuable in the future.
At this point i would like to digress a bit to touch upon a concept that is mentioned in Stephen Covey's book, The 7 Habits of Highly Effective People. It is called "Sharpen the Saw". What this means is to preserve and enhance the greatest asset you have, which is … you

The book talks about having a balanced program for self-renewal in the four dimensions of your life: physical, social/emotional, mental and spiritual. Self-renewal enables you to create growth and change in your life. Sharpening the saw keeps you fresh so you can increase your capacity to produce and handle challenges. The book goes on to say that without this renewal, the body becomes weak, the mind mechanical, the emotions raw, the spirit insensitive, and the person selfish. 

This concept of sharpen the saw finds expression in another oft repeated tale about two wood-cutters who get down to cutting trees using their saws. One wood-cutter goes to work and relentlessly keeps at cutting wood. He spends a lot of time and effort to continuously cut wood. 

The other wood-cutter cuts some wood and then takes a little time off from cutting wood, to sharpen his saw. He then goes back to his task of cutting wood. He does this repeatedly. At the end of the day, it is observed that the second wood-cutter has cut more wood (increased productivity), is more relaxed (less stressed) and has both himself and his tools in good shape to handle another day's tasks. 

The saw of the first wood-cutter gradually grew blunt with increased use. The reaction of this wood-cutter was to increase his own effort at cutting wood hoping that the increased effort on his part would compensate for the reducing sharpness. Needless to state the obvious, the first wood-cutter ended up feeling burned-out and tired and probably surprised that all his efforts resulted in less  than optimal results. He did have a lot of busy-time but results did not match his level of activity.

Whats all this got to do with Software Testing ?  I am sure most of you would have figured out the connection and where we are heading. It is true that testers must test, but that isn't all that a tester should do. The best testers realize the principle of sharpening the saw. They must take the time to continually develop their skills and work on their creativity and thinking. These testers strive to constantly be abreast of developments in their area. Self-development need not be limited to areas that are directly connected to testing; do not hesitate to look at those areas that may not seem in any way related to testing. You never know where you might find ideas that can be implemented in your work.
***
Liked this entry? Join my community of professional testers to receive fresh updates by email. Use this link to add your email address to the community. Rest assured, I will neither spam nor share your email address with anyone else. Your email id will remain confidential. Subscriptions are handled by Google's FeedBurner service.

Share & Bookmark this blog entry

Emotions and feelings in testing software

Software testers generally look at the requirements to figure out how the product must behave. Often these requirements cover the functional and some non-functional attributes including performance, security, some elements of usability, etc. Tests are developed with expected results that align with these product requirements. So far so good. There is a clear line from the written down requirements to the tests.

As testers, as you proceed with executing your tests, there may be instances where you feel irritated or frustrated with aspects of the software under test. You might feel a range of emotions as you test the software. At such times, what do you do ? Do you listen to your feelings or do you go with the script, merely looking for expected behavior as described in the test cases that you are executing. If the test produces the expected behavior, while you have experienced conflicting emotions during testing, what would you do ? 

As a software tester, do you need to give importance to your emotions and feelings that you encounter during testing or do you have to leave these "softer" aspects of your self outside the door before beginning a test campaign ? Is testing purely based on logic and written down requirements alone ? Is there any value to listening to what your inner "voice" and feelings are trying to say as you test the software ?

In my view, testers need to listen to their feelings while testing. That said, some testing may require you to just follow the script and stop at checking what is stated as expected behavior. However, the good news is that most testing will find feelings and emotions to be an useful added aid to the test effort. At this juncture, it helps to remember three basic concepts which are listed below.
  1. Plain definition of a bug: A "bug" is something that will "bug" someone who matters. This someone could most likely be users of your software or someone significant enough to have an impact on your organization
  2. Not all software requirements are written down or even stated. There is a significant number of requirements that are either left unstated or assumed or implied
  3. In many cases customers may not really know all that they need from the software at the outset when requirements are being firmed up. This is especially true in the case of attributes of software such as usability
If your emotions are telling you something, listen to them. If you feel frustrated while using your software; or are feeling confused at the non-intuitive interfaces; or are tired of waiting for your software to respond or process information; or are unhappy with the workflow or any other aspect of the software and if these are not explicitly stated in your requirements, do not ignore them just because your written set of requirements do not say anything about these experiences. If you are bugged by something in your software, it is likely that users of your software could be bugged too and that can have a more serious implication.

Something with the software that frustrates or irritates you can very well irritate or frustrate your product's users. In the interests of making your software more user-friendly and intuitive, allow your feelings to ride along as you test. Report any issues you find or improvements you would like.

The other thing to remember is that issues raised based on what you feel regarding the software may not always go well with your counter-parts in development. Some issues may be accepted and fixed in the current release if these are deemed to cause significant inconvenience or loss of functionality for the users. Some issues may just be deferred to a future release and some issues may be closed as not being bugs. 

If you truly believe that a bug is valid and severe enough (a function of the frequency of occurrence and impact) to merit attention, you should have a discussion with your developers or someone who can make decisions on the bug's status. It can be a great help if you have the ability to compare your software and its behavior or interfaces with a similar application elsewhere - this could be a competitive product or a substitute-able product offering. If such an opportunity for comparison exists and you are able to show how your software is lacking, it can boost your case to have the issues fixed. Also, having a customer representative put their weight behind the issues you raise is a big push for fixing the issue. 

Ultimately, realize that some issues will go unfixed. However, your pursuit to ensure that your customers have the best possible experience while using your organization's product should continue and towards that end, keep an eye open for what your emotions and feelings are telling you as you use and interact with the software you are testing.

A great piece of software is not just the one that meets all the functional requirements, but one that goes beyond in anticipating user needs and potential pain-points and tries to address them.
***
Liked this entry? Join my community of professional testers to receive fresh updates by email. Use this link to add your email address to the community. Rest assured, I will neither spam nor share your email address with anyone else. Your email id will remain confidential. Subscriptions are handled by Google's FeedBurner service.

Share & Bookmark

Image: FreeDigitalPhotos.net

Does Agile development need specialists ?

"Agile is to software development what poetry is to literature. 
It is 
very difficult to do well, very easy to do poorly, 
and most people 
don't understand 
why a good poem is good and a bad poem isn't...”
- from the web

Transition from a traditional development model to an agile methodology is often met with some degree of skepticism and doubts by testers. Books and programs on Agile development tend to emphasize the need for multi-skilled generalists who can take up different functional roles as needed. This tends to cause specialist testers to worry about losing their identity in an agile world. Is there a need for specialists in agile or is the agile world inhabited by generalists, the proverbial jack-of-all-trades ?

To answer this question, look no further than your favorite team sport. Agile software development is a team process involving members from the different functional groups coming together as part of one team to produce software. No longer are they members of distinct teams such as development, testing, technical writing, i18n, l10n, etc. Back to our earlier question on whether agile needs specialists or is the new world full of generalists ?

Like a team sporting activity, for a team to be successful it cannot be - 1) all members of one particular type or specialize in one activity e.g. development or testing alone 2) all members who are generalists and knowing part of all functions but not specialists of any function. How would you rate the chances of your favorite sports team if it were comprised of either types of members – a) all players who specialize in just one area e.g. of cricket which I follow: a team of just batsmen or just bowlers b) all generalists e.g. again of cricket: a team of just all-rounders – it might be better than the first choice with only players of one type but still not the best choice. An agile team takes a step towards success when it comprises specialists from the different functions coming together and working together collaboratively to produce software. Each member of the agile team brings to the table their unique set of skills that influence the software's development and success of the project.  However, an additional requirement for these specialists that are part of the agile team is to be able and willing to share in some of their non-core tasks. For example, a tester who can debug defects and make small fixes if need be, a developer who can also document their feature or do some testing, etc.

There is truth to the statement that agile needs generalists. However, it is better to have specialists who can go beyond their functional domains to help towards the release rather than specialists who just restrict their involvement to their functional area of specialization. Ultimately, agile development is a team effort and testers like others on the team must own the release from the beginning without merely trying to police it at the end.
***
Liked this entry? Join my community of professional testers to receive fresh updates by email. Use this link to add your email address to the community. Rest assured, I will neither spam nor share your email address with anyone else. Your email id will remain confidential. Subscriptions are handled by Google's FeedBurner service.

Share & Bookmark

Software Testing - Time Machine: from an year ago

"The past is behind, learn from it,
The future is ahead, prepare for it,
The present is here, live it"
- Thomas Monson

In this entry we look at a few of the blog entries that were published on this blog about an year ago. These entries touched upon some of the very basics of Software Testing. I hope you still find these useful and interesting.


***
Liked this entry? Join my community of professional testers to receive fresh updates by email. Use this link to add your email address to the community. Rest assured, I will neither spam nor share your email address with anyone else. Your email id will remain confidential. Subscriptions are handled by Google's FeedBurner service.