Quality By Design - Part 1


I'm going to cure the world of software bugs and make everybody's lives much easier.

Bugs in software are bad. They cost more and more to fix as they are discovered later in the project life-cycle. Often bugs that were fixed previously arise again after changes to the software. Testing is often cut short due to pressing deadlines and software is released with bugs still present. Developers often do not exhaustively test their own code each time they make changes.

This article, in three parts, explains how some simple design and development techniques can be used to eliminate all those bug concerns and massively accelerate your productivity. The three techniques described are Automated Unit Testing, used heavily in Extreme Programming(XP); Design By Contract, and Response Matrices.

By using these techniques you can quite quickly double your productivity, and in some cases increase productivity four or even five fold.


This article is aimed at developers, team leaders, and project managers who wish to achieve much higher productivity from their development teams.

Q) Do you enjoy testing your code?
Q) Do you enjoy fixing all your bugs?
Q) Do you prefer to spend your time testing/debugging rather than creating and developing new software?

I suspect for most of you the answer to this is a big NO. However, if you use the techniques I describe in this set of articles you will be able to answer YES. The reasons being that:

a) testing will take seconds, not minutes, hours, or days;
b) you will be able to quickly and easily identify and remove bugs;
c) you will spend much more time developing, rather than debugging/fixing.

All sounds too good to be true doesn’t it?

Please read the situations described below. If you find them familiar to your project, then I will guarantee that the techniques I describe can significantly increase your productivity and make life much easier. Using these techniques it is common to double productivity, and in some cases I’ve seen four-fold improvements. The techniques are very simple, not at all complex, and certainly DO NOT turn your project into a science project. You’ll be astounded at how something so simple can have such an enormous impact on your productivity.

Are these situations familiar?

  1. Eliminating bugs at the end of the project takes longer than you originally anticipated.
  2. Bugs often re-surface after they have been fixed in a previous release.
  3. Fixing bugs often creates more bugs.
  4. A lot of time is wasted trying to integrate code written by many developers.
  5. Integration is a slow painful process.
  6. Testing time is chopped as the deadline approaches. You release a system that you know has bugs and are not 100% confidence with. You then start fire fighting.
  7. Any testing that does get done is manual, ad-hoc, takes too long, and some tests are missed out all together.
  8. Fire fighting starts after the code is released resulting in many patches being released. Project timescales slip.

If your project suffers from any of the above then you can significantly improve your productivity by using some very simple techniques. You will eliminate all the head aches mentioned above and work will become so much more enjoyable.

The Techniques

The techniques I will explain are:

Part 1: Automated Unit Testing, used heavily in Extreme Programming(XP).
Part 2: Design By Contract.
Part 3: Response Matrices.

Actual Experience Of The Techniques

Using Automated Unit Testing I’ve written tests that can test 3 years of man effort in around 20 minutes. These tests are written before the coding starts, during the coding, and after the coding. As an example, in the last six months I’ve not had one repeat bug, and have had a handful of minor bugs. My developers spend more time developing and hardly anytime fighting fire. The whole team enjoys their work.

Using Design By Contract I’ve delivered virtually bug free components that other developers have been able to integrate with seamlessly. This means my developers spend more time developing new functionality, rather than teaching others how to use what they have developed. It means other developers work quicker with easy to use components.

Using Response Matrices I’ve been able to drive out exhaustive sets of tests for the highest risk critical components to ensure that are 100% correct and robust. This has again ensured seamless integration and avoided halting streams of developers dependant on central components.

Automated Unit Testing

First, to clarify, I’m talking here about Unit Testing, not User Acceptance Testing or System Testing. Unit testing is the testing developers do as they are writing the code, and before they release the code for system testing. Thus, the Automated Unit Testing does not use tools like Mercury WinRunner, or Rational SQA Robot. You don't need flash tools to do this at all. In fact, you don't need anything other than your existing development environment.

If you are not doing this already, I will claim that implementing Automated Unit Testing alone will halve the time it takes to build quality software. Okay, so you need some convincing! Some key questions:

· How long do you spend testing the code that you write?
· How much time do other developers spend trying to integrate with your code?
· How much positive testing do you do, and how much negative testing do you do?

Lets say you have a very simple component with a few classes that have a couple of parameterised interfaces each. How do you test your component? Commonly as a developer, you will knock together a form with a few buttons and text boxes on it that run various tests. Lets say you have 4 tests per interface. That means opening the form, keying in 12 sets of values, clicking the button each time and checking the results.

Okay, lets say that takes about 15 minutes to test each time. It takes you one week to develop and debug the code, and on average you test it 6 times a day after you have made changes to the code. That means you spend 1 day a week testing the code you develop, and you’ve also run the risk of not testing it exhaustively and left some bugs in it.

That 1 day can be got down to 1 minute using Automated Unit Testing. To do this you simply write some code that tests the component using assertions. If you are using VB you could write something like this:

Dim oMyComp As CMyComp
Dim strResult As String

Set oMyComp = New CMyComp

strResult = oCMyComp.MethodOne(“parm1”, “parm2”)

Debug.Assert strResult = “{put expected result here}”

You can write many of these tests for both the happy flows (when things go right) and also the negative flows (when things go wrong).

When you run these tests, if the assertions fail it will drop into the code at the assertion. You can then step through and debug.

The Benefits of Automated Unit Testing

Okay, so that has saved 1 day in development time per week. I did say that you could halve the time. The time I am referring to here is from when you first start building the software, to when it is finally shipped into production. Given that you are writing much higher quality software, and never get any repeat bugs the number of releases drops. The debugging and fixing time drops. Because you spend less time fixing bugs, you spend more time on development. There are less iterations of the software – so the System Testing cycles drop, and thus so does the UAT testing.

The cost of a bug found in UAT costs significantly more than the cost of a bug found during development for exactly the reasons outlined above. The whole key here is to find them quickly in development and to make sure they do not happen again.

If after release you have to change the code (enhancing or extending it) you can use the existing tests you have already written to make sure the changes you made did not break anything. This testing takes seconds and is exhaustive every single time.

I discuss more about automated testing on my article Test Driven Development .

Going Further With Automated Unit Testing

If you want to get even better, try writing the tests before you write the code – you’ll be amazed at how quickly you can write quality software. Also, if you start thinking about the tests before you write the code, you will find flaws in the design before you have even implemented it. Writing the tests first also help you think about how you might write the implementation.

A golden rule is that if you can’t write a test for what you are about to build, then you shouldn’t be starting to build since you don’t know what you are building!

Extreme Programming(XP) makes a heavy investment in Automated Unit Testing and advocates the writing of the tests first. From the experience I’ve had I doubt I will ever write a piece of code without writing the tests first. It is just so much quicker and easier. You need to try it yourself to be convinced. Trust me and have a go!

Starting out on Automated Unit Testing

For new comers to Automated Unit Testing who wish to try out some of these techniques I would suggest the following track:

  1. Start by writing a simple automated unit test harnesses using assertions. These harnesses should have no screens, but simply run on start up. [E.g. In VB, just use an Active X EXE with a Sub Main that runs the tests.]
  2. Migrate your tests to use a front end unit test tool – xUnit is the industry standard in Extreme Programming(XP) circles.

What's in Part 2?

In Part 2 I will show you how you can use Design By Contract to build robust reliable software, and further reduce the risk of defects in your software.

All content © Byte-Vision Limited 1997 - 2004