The Purpose of Testing

I may not be an active Software Engineer, but I still have the heart of one.  And, as such, there are a couple of directives that I personally find very compelling:

1. Test often.
2. A successful test is one that finds a problem.

As an instructor, I tried to hammer this philosophy home, with varying degrees of success.  

The first hurdle is that everyone has a tendency to grow tired of something you have to do all of the time....  (Oh, geeez, I've got to go give water to the birds AGAIN!)  Many programmers and software designers are much more attracted to the building of new things.  It is not nearly so glamorous to expend great amounts of time and effort in devising and then employing test after test to see if an existing creation is working correctly. 

Hey!  I want water - NOW!

You see, I wasn't clueless as an instructor and I am not clueless now.  To most people, it's more fun to write program code than it is to write tests to break that code.  Just like it is more fun to write new blog posts than it is to edit them.  (Edit often and successful editing uncovers writing errors? Now there's a thought!)

But I still had the gall to tell people to test often.  And then I had the gall to tell everyone that it isn't necessarily a good thing if the tests you design fail to uncover any problems!  If you have ever written code to create a computer program or an app (or whatever), you know exactly how difficult it is to get a program to run in the first place.  The suggestion that successful tests cause a program to break is usually not received well!  

How terribly rude of me.

We continue to test our clothes dryer

Why Testing for Failure is a Good Thing

In the world of software development, it is my belief that the last person you want to discover a problem with the code is the end-use client who is trying to use the software to do their task.  They are the least likely person to be able to fix the problem once it arises and they are most likely to move on to a competitor's product, if they can, as the solution. 

What's worse?  What if they can't move on to another product (or they don't see the problem in the first place)?  What sort of harm might this cause?  

This is a bit easier to see the impact if we are talking about programs for air traffic controllers or automated tools that deliver doses of radiation to cancer patients.  The discovery of a software error while these tools are being used in industry can result in injury and death.  But, don't discount the ripple effects that occur when a less "critical system" fails because it was not properly tested.

One case in point,  consider an email program that reported emails were sent correctly to customers - but were instead deleted or missent (this is an actual case).  How do we measure the potential damage that comes from this failure?  Perhaps the majority of the lost communications were not terribly critical, but the damage that was done to the reputations of the senders and recipients of these emails could be significant.  For example, one person thinks they sent something and they've never gotten a reply to their questions - what might they be thinking about the people who did not respond?  Could it negatively impact their interactions in the future?

To the dismay of my students, I often employed the strategy of testing for failure with the exams I gave them to assess their learning.  It would be fair to say that most of my tests were considered to be difficult (at the least).  

Yes, I wanted them to have success.  But, my viewpoint was that they would see more success if we could identify some of what they still did not understand.  Granted, this approach was hard on all of us.  I didn't like that the scores made them unhappy and I really didn't want it to be about that.  But, there was no avoiding the fact that there was value in the exercise.  In the end, it wasn't an issue that solid communication couldn't solve.

The green carts in front could have used some testing.

Real Life Testing Needs

Here is a case for testing that applies to the farm.  The two green carts at the front in the picture shown above are likely familiar to anyone who farms at our scale or to people who have gone to nurseries to buy plants for a garden.  The green payload area is actually pretty sturdy.  The design of the overall cart is pretty good.  I bet the prototypes tested out (if they tested them at all) pretty well.  But, they clearly stopped testing once these things went into mass production.

What is wrong with these carts you ask?  After all, these two have been with us for nearly 18 years.  They must be successful, right?

First, you should note the other carts in the picture.  The smaller cart is a left-over from our pre-farm days, so it doesn't really count in the discussion.  But, the black cart is something we have purchased in the time SINCE.  We went and found ourselves a better cart.  

So, why did we move on?

The nice 'pneumatic' wheels that were highly touted for the green carts have a tendency to break on the single weld at the axle.  Of the three carts of this type we have, there are NO original wheels remaining.  We've had to replace them all.  Some of them were replaced more than once until we could find a better manufacturer for replacement wheels.  To top it off, we know we are not the only clients who have dealt with this flaw.

No Test Means No Problem?

Mother Nature is always testing for failure, creating stresses that remove the weak and favor the strong.  Gingko trees do not tend to take well to late frosts and freezes.  If the tree is strong and healthy, it can pull on reserves to send out new leaves.  This year, the Gingko did not succumb to the temptation to begin to bud out prior to the recent cold weather.  But, in 2020, it was sorely tested and it actually survived. 

Humans seem to prefer to have their testing done in real time and in real life - just like Mother Nature.  We do this, rather than employing some patience and some intelligence to run tests BEFORE the consequences are dire.  It gets even sadder when I consider the tendency to ignore test results that point to a problem and then we proceed without addressing the problem.  We could, at the least, determine that the likelihood of that problem occurring is very, very low before proceeding.  But we don't. 

I have a hard time understanding the apparent preference for surprises that we could have prepared for if only we had tested for them.

In the end, I think the true Software Engineers in this world have it right:

1. Test often.
2. A successful test is one that finds a problem.

And, now, I will go test my limits for giving water to the poultry.  Have a good day everyone!

Resistors are Futile - Friday Faux Real Stories

 

Welcome to another Faux Real Story on the Genuine Faux Farm blog!  This week, we're going to take you back to a time when the farmer was not a farmer.  Instead, he was a programmer who worked with software that was used to design printed circuit boards.

It was my first full-time job as a computing professional after graduating with my Bachelor's degree in Computer Science and Mathematics.  I had been hired to be part of the Government Avionics Division of Rockwell-Collins in Cedar Rapids, Iowa.  I still vividly remember arriving for my first day at the south entrance of Building 105 and being met by my immediate supervisor, Jerry.  As we walked past some of the entrance office area, that included (believe it or not) a pharmacy and travel agency, and entered the main part of the building, Jerry said, "Remember how to get back to this hallway.  Then you can always find a way back out of the building."

Jerry was not kidding.  Building 105 was actually a complex of buildings and I could see a pinprick of light that represented the far end of the wide hallway we were in.  This particular hallway was wide enough that managers in power suits could been seen walking one way and a forklift carrying a pallet could be going the other way.

Recent satellite image of complex including Building 105

There were various offices tucked into corners and there were computer labs, development labs, storage areas and assembly lines (mostly in the western portion).  It was incredibly easy to take a few turns and no longer know which way was up - much less out.

My job was to help maintain an older CAD (Computer Assisted Design) system that was used to design printed circuit boards.  Things that might look a bit like this:

The whole point of a CAD system was to give a designer a bit more flexibility in moving and visualizing components and making connections as they considered design options.  They could select from a long list of possible components - various capacitors, chips, jumpers and resistors - and position them on the board.  Some programs could be run to help route the circuits and generate the documentation required by the government.

These designs could then be implemented on the manufacturing floor using the various types of equipment.  Some of the newer automated equipment of the time included Surface Mount Technology (SMT) and Wave Soldering.  

One of my projects was to gather the coordinates and placement of simple components (capacitors and resistors) and put them into a computerized file format so that a new "chip shooter" machine could automatically place these components onto circuit boards.  The circuit board already had holes drilled to receive these components at their proper X,Y location.  It would move the board around until it lined up for the "shooter" to put the component into that spot.  This new equipment probably could place somewhere in the neighborhood of eight to fifteen components per second on a given printed circuit board.  In other words, it moved pretty fast.

Here is an example of a chip shooter in action.  This type keeps the board in place and moves the "shooter" around - it's probably for higher precision components that need a bit more care.  The one I saw moved the board to a "shooter" that was stationary.

What I recall seeing was a bit more like this one.  Go to the 57 second mark to see it in action.

In any event, my job was to identify items that could be loaded by the chip shooter in a printed circuit board design.  Then, the program I wrote would take the X,Y locations and the orientation (angle and depth) and place them into a format this piece of equipment could read.  

On the day of the big "first test," I went to the manufacturing lines where I was greeted kindly enough, but in a typical way line workers might react to one of the "educated pups" that worked on those @#$&(! computers.  The good news is that I didn't act like many of those pups tended to - but that still didn't mean they couldn't have some fun on my behalf.

One guy looked at me and asked where my helmet and safety glasses were.  Of course, I had none because no one had informed me that I needed them.  They found me a helmet that was a bit too big and some glasses before we loaded up the file to watch the machine work.  I think they were expecting me to ask why I needed the safety equipment - but I already had an inkling that something was up.

The machine took a while to set up, but once it was ready it pulled in a circuit board and hammered in a series of components and then it moved the board out of the way.  For good measure, it then SPIT another component out that bounced off the floor and flew off to the side.  While it was unlikely that it would have shot up at an angle to hit me in the face, it was not impossible.  Hence, the explanation for the safety equipment.

I am certain they could have stopped the machine then.  But, they decided this was too much fun and let it run a few more boards before shutting it down.  Each board was loaded up properly and then an extra resistor would get sent to the floor below - Zzzziiiing!

Of course, I got some good-ribbing after that because there was a problem with the data I had delivered to them.  I apparently took it well enough because I was invited to eat lunch with them afterwards.

So - what had happened?

Well, a CAD system allows the designer to grab an image that is a component and place it on the screen.  ANYWHERE on the screen.  This includes places that are not actually ON the circuit board.  So, the designer could pick a component, put it on the edge of the screen so they can grab it later and move it into place.  

Except this particular design had an extra component that the designer had left on the side of the screen - either by accident or on purpose.  My program grabbed every resistor and capacitor that was in the design file for any given circuit board.  It very diligently recorded ALL of them, including the one that was set off to the side of the board.

So, the chip shooter was also diligently following the instructions and trying to put this extra resistor where it belonged - off to the side of the board and onto the floor. 

I will say this, resistors are futile when you try to put them off the edge of a board.  But they make for a good story.