The Purpose of a Test

I may not be an active Software Engineer, but I still have the heart of one.  And, as such, there are a couple of directives that I personally find very compelling:

1. Test often.
2. A successful test is one that finds a problem.

As an instructor, I tried to hammer this philosophy home, with varying degrees of success.  The first issue is that everyone has a tendency to grow tired of something you have to do all of the time....  (Oh, geeez, I've got to go give water to the birds AGAIN!)  Many programmers and software designers are much more attracted to the building of new things.  It is not nearly so glamorous to expend great amounts of time and effort in devising and then employing test after test to see if that creation is working correctly. 

Hey!  I want water - NOW!

Then, I had the gall to tell everyone that it isn't necessarily a good thing if the tests you design fail to uncover any problems!  If you have ever written code to create a computer program or an app (or whatever), you know exactly how difficult it is to get a program to run in the first place.  The suggestion that successful tests cause a program to break is usually not received well!  How terribly rude of me.

Why Testing for Failure is a Good Thing
In the world of software development, it is my belief that the last person you want to discover a problem with the code is the client who is trying to use the software to do their task.  They are the least likely person to be able to fix the problem once it arises and they are most likely to move on to a competitor's product, if they can, as the solution. 

What's worse?  What if they can't move on to another product (or they don't see the problem in the first place)?  What sort of harm might this cause?  This is a bit easier to measure if we are talking about programs for air traffic controllers or automated tools that deliver doses of radiation to cancer patients.  But, don't discount the ripple effects that come from things not working as they were intended because they were not tested for failure!  Case in point, an email program that reports that emails were sent correctly to customers - but were instead deleted.  Now the customers begin to wonder if the business really cares about their customer base.  The business owner wonders if there really is any demand for the product.  The ever fragile trust between the two is tested.  How do we measure that potential damage?

To the dismay of my students, I often employed this strategy with the exams I gave them to assess their learning.  It would be fair to say that most of my tests were considered to be difficult (at the least).  What was I doing?  Yes, I wanted them to have success.  But, my viewpoint was that they would see more success if we could identify some of what they still did not understand.  Granted, this approach was hard on all of us.  I didn't like that the scores made them unhappy and I really didn't want it to be about that.  But, there was no avoiding the fact that there was value in the exercise.  In the end, it wasn't an issue that solid communication couldn't solve.

The green carts in front could have used some testing.

Real Life Testing Needs
Here is a case for testing that applies to the farm.  The two green carts at the front in the picture above are likely familiar to anyone who farms at our scale or to people who have gone to any number of nurseries to buy plants for a garden.  The green payload area is actually pretty sturdy.  The design of the overall cart is pretty good.  In fact, I bet the prototypes tested out (if they tested them at all) pretty well.  But, they clearly stopped testing once these things went into mass production.

What is wrong with these carts you ask?  After all, these two have been with us for nearly 15 years.  They must be successful, right?

First, you should note the other carts in the picture.  The smaller cart is a left-over from our pre-farm days, so it doesn't really count in the discussion.  But, the black cart is something we have purchased in the time SINCE.  So, why did we move on?

The nice 'pneumatic' wheels they highly touted for these carts have a tendency to break on the single weld at the axle.  Of the three carts of this type we have, there are NO original wheels remaining.  We've had to replace them all.  Some of them were replaced more than once until we could find a better manufacturer for replacement wheels.  To top it off, we know we are not the only clients who have dealt with this flaw.

The Gingko may have failed the late freeze test this year.

No Test Means No Problem?
Mother Nature is always testing for failure, creating stresses that remove the weak and favor the strong.  Gingko trees do not tend to take well to late frosts and freezes.  If the tree is strong and healthy, it can pull on reserves to send out new leaves.  We're not sure our Gingko will do that because it has been under stress during recent wet years.  We shall see.

Humans, on the other hand, seem to prefer to have their testing done in real time - rather than employing some patience and a bit of their smarts to run tests BEFORE the consequences are dire.  Even sadder is our tendency to ignore test results that point to a problem without addressing them OR at the least, determining that the likelihood of that problem occurring is very, very low.  I have a hard time understanding the apparent preference for surprises that we could have prepared for - yet we all seem to do it at some level.

In the end, I think the true Software Engineers in this world have it right:

1. Test often.
2. A successful test is one that finds a problem.

And, now, I will go test my limits for giving water to the poultry.  Have a good day everyone!