Showing posts with label Quality Assurance. Show all posts
Showing posts with label Quality Assurance. Show all posts

Tuesday, December 3, 2024

Repeatability

Have you heard the story of a guy that traded a paperclip for a house? It didn't happen that simply. He traded the paperclip for something else and continued trading until he eventually had a much more expensive item. When I think about what he did, I imagine he took advantage of having something that someone else needed to the point where they would trade something more valuable for the less valuable item he had. I also doubt the process he followed is very repeatable. While someone could take some of the ideas, trades for different items would need to be substituted. Furthermore, I doubt many would have his same success.

This leads me to my topic of interest today and that is repeatability. When software engineers create a program, they require a team of quality assurance (QA) engineers to test their code to look for problems. The worst kinds of problems to find are non-repeatable ones. If the program runs 9 times out of 10, it becomes the QA engineer's responsibility to figure out the exact repeatable steps to find the one time it doesn't work.

One area where this becomes increasingly difficult is when using a program that creates random numbers. The whole point of random numbers is to make things unpredictable. This is why most random number programs have two calls. The primary call provides a random number. To make sure it is repeatable, it will always provide numbers in the same order. If you ask for a single digit random number, it might first provide a 3 and the next time you call the function it returns a 5. Stopping the program and restarting it give you the same numbers in order. This is very helpful when trying to locate software bugs. The important thing to remember is to also use the second call or randomize function. This tells the program to not use the same order but instead to mix it up.

Why is this important? Several years ago I noticed that the software provided by the governing body for ski racing forgot to include the randomize function in their software. My son attended a race at Jackson Hole and had yet to complete a Super-G race. He got lumped in with all the other first-timers. The weekend had 3 individual races where all the first-timers should have been given random starting orders within their group. Starting near the front has its advantages and so one would think my son would have his start position changed randomly. Nope the 10 to 15 racers all started in the same order for each of the 3 independent races. The developer of the software never randomized the start order as he or she should have. I tried to protest but ski coaches and race organizers don't understand the nuances of software development and accepted as fact that the coding had been done correctly.

When looking for software bugs, repeatability is important. It makes it a lot easier to find problems and fix them. Just remember to add in that randomize function at the end. Otherwise you are not really creating random numbers and that is one area hackers exploit to break into systems.

Tuesday, November 16, 2010

Finding Bugs

Recently I purchased a new Ham radio. It isn't anything special, just an inexpensive hand-held unit used to help with local events or emergencies. This new radio has the ability to program 128 memory channels using a personal computer. The application runs on Windows and presents you with a spreadsheet-style interface. You fill out specific parameters for each of the memory locations and then upload the information to the radio through a USB cable. This is much simpler than entering all of the information by hand using the radio's cumbersome menu system.

I don't think the software to program my radio was ever fully tested. It took me a good hour to figure out how to use it. Now that I have figured it out, it is really simple to use. The problem was that you have to do things in a certain order. If you don't then the software freezes or refuses to communicate to the radio. It was frustrating while I was figuring it out and I almost sent back the radio.

Testing is very important and something that I have to do on a software project I am working on in my spare time. Since I spend most of my evenings away from home, I have three or four hours a night to work on the software and it is coming a long nicely. When I was home in Utah this weekend, I took the opportunity to show it to my wife. I noticed several problems during the course of my demo and was able to get them fixed immediately.

Most commercial software companies have formalized methods they use to ensure their products live up to expected levels of quality. One method is using someone who didn't write the software to test it. For smaller organizations, such as a guy working on software in his basement, garage, or boat, the demo to spouse or kids can serve a similar purpose. If only the company that wrote the software for my new Ham radio had been so careful. Then I wouldn't have blown a Friday evening trying to figure out how to get their software to work.