Make Testing and Evaluation Your Best Friends
by Adele Sommers, Ph.D.

Once your company designs its products and services, how should you oversee whether your offerings are actually performing the way they're supposed to? Do you have a systematic method for ensuring that what you develop results in a smoothly functioning product or service?

To begin with, you would want to use a specification to describe what your product or service is intended to do, and have a way to continually compare your product or service against that specification to determine whether it actually 1) does what it's supposed to do, 2) does it correctly, and 3) as advertised. This article explores why testing and evaluation are so critical, and dispels two pervasive myths about testing.

The Importance of Testing and Evaluating Your Offerings

A great product launch can take an undesirable turn if there's no process to ensure that your product does exactly what you and your customers expect. Are the features muddy and maze-like? Do they require users to go around in circles trying to follow the instructions? Are there bugs and inconsistencies throughout? These are signs of trouble! Remember that your customers will have demanding expectations -- just as you should have as a consumer.

Related inquiries you could make include: Are your products and services confusion-free -- even if they carry out complicated tasks? Also, how well do they perform the actions they're supposed to perform?

This is where testing and evaluation techniques can become your best friends and your "secret sauce." The earlier in the life cycle this process can begin — specifically, in the requirements and design stages, when your initial ideas are still on the drawing board — the more successful your offerings will be at satisfying your customers’ needs and desires. This early starting point lets you build quality incrementally into your offerings, instead of trying to layer it on top as an afterthought, the way your competitors might.

Testing is no trivial undertaking, however. Below appear two myths about testing and evaluation that highlight the pitfalls of underestimating these efforts.

Myth #1: Testing Complex Products Is Very Straightforward

Just the opposite is true -- the more complex a product is, not only is it more difficult to use, it can actually become too complicated to test.

The reason for that phenomenon is: The more features you have, the more likely that there will be too many combinations of features to test. When something is too complicated to test, no quality assurance program in the world can manage it.

Are you wondering why that matters? Perhaps you're thinking, "Who cares how many features there are? If we just make sure that the main features work, the customers ought to be happy, right?"

To gain more insight into this assumption, I recommend a business novel called "Necessary But Not Sufficient" by Eli Goldratt. In it, the fictional characters explore the problem of complex systems and the pros and cons of how to design, test, and maintain them.

One main theme of this book exposes why an exceedingly competent software development team suddenly cannot figure out how to continue to maintain a highly successful but extremely complex software product. It was the direct result of customers asking for more and more features, and the development team accommodating them.

Every added feature set increased the combinations of functions and interactions within the system exponentially, rather than in a linear fashion. Eventually, no one could identify, much less test, all of the feature permutations.

That's the problem with complex systems. You can reach a point where there are too many sets of variables to test in your lifetime, much less within the time available to release the product!

But let's assume that you are dealing with something less complicated. In any case, if you believe that your offerings ought to be as robust and error-free as possible, then you should consider testing early in the game -- as early as the requirements and design phases -- to evaluate the degree to which the functions and features you envision are simple, elegant, intuitive, and correct.

Myth #2: "But It's Fine to Wait Until the End"!

Really? As you can imagine, waiting to catch all of the problems at the end is sort of like hoping to count all of the water droplets that flow over Niagara Falls. The effort can be overwhelming! Testing is lot more about prevention than about cure, which means doing much more in the early stages of the life cycle when most people don't even want to think about it. Instead, they're hoping to test everything just prior to releasing the system, which is like trying to put icing on a bug-riddled cake.

By the time the end rolls around, the chances of finding -- and successfully fixing -- all buried defects are remote. The expense of fixing a bug, for example, grows dramatically with any delay in either discovering it or correcting it. By many estimates, the costs are 10 to 100 or more times greater after deployment than if the team had found and resolved it much earlier in the life cycle.

Contributing to this price includes redoing anything that was mistakenly undone as a result of trying to fix the original problem. What does that mean? Well, frequently, any attempt to tweak one area of a system can accidentally "break" something else that was working correctly. This happens so often that there's a special category of testing designed just for detecting such mishaps, called regression testing.

In conclusion, please don't underhandedly transfer to your customers the primary burden of finding and reporting system problems. And whenever someone does report a bug, ask yourself the following:

• How could we have automatically detected this defect?
• Better yet, how could we have prevented it?

By making testing and evaluation your partners and allies, you will give your brand and offerings enormous competitive advantages. Related articles on this topic will explore specific techniques that you can use to achieve outstanding results.

~~~~~~~~~~~
About the Author

Adele Sommers, Ph.D. is author of “Straight Talk on Boosting Business Performance: 12 Ways to Profit from Hidden Potential.” To learn more about her book and sign up for more free tips like these, visit her site at www.LearnShareProsper.com

This article may be distributed freely on your Web site, as long as this entire article, including the links and full “About the Author” section, are unchanged. Please send a copy of, or link to, your “reprint” to Adele@LearnShareProsper.com.

Copyright 2007 Business Performance Inc., Adele Sommers, All Rights Reserved.

865 words

Return to the Free Articles index