Archive for April, 2013

With fifth dimension testing, we use techniques more commonly employed by the “bad guys.” For example, we may execute techniques such as: Fuzzing Response modification using genetic algorithms Input breakdown Overflows Underflows This approach allows for evolution of our test collection. We can automate a significant number of these tests if we have a comprehensive, […]

            Extreme testing occurs when we deliberately “torture” both the hardware and software to see what happens under undesirable conditions. Some examples of extreme testing include: Random voltages within the allowable voltage boundaries Voltage slews Deliberately introduced random noise on the data bus Extremely high bus loading (over 80% and sometimes over 90%) to see […]

        Stochastic testing occurs when we allow a reasonably well-seasoned test engineer to go with their “gut” and feel their way about the product’s performance. During the development of numerous embedded automotive products, we have seen stochastic testing elicit roughly the same amount of test failures as combinatorial testing. We are not recommending that stochastic testing […]

            We know that a very simple product or system can generate a vast number of potential test cases. With more complex systems, this number becomes astronomical. This is the result of a factorial calculation! One technique we use to get around this problem originates with designed experiments. Many designed experiments are based on orthogonal […]

            We have discussed compliance testing earlier. This is known as testing to requirement. These requirements can be taken directly from a customer specification (when we have one) or derived internally from a requirements review or even both. Compliance testing is the primary method we use to ensure that we are meeting all specifications and […]

To perform embedded software testing, we recommend five phases of testing. These phases may take place concurrently and are as follows: Compliance testing Combinatorial testing Stochastic (or exploratory) testing Extreme testing Attack mode Each of these methods provides some view of the product the other method does not. Of course, testing the product is not […]

We do not need to suffer the pains of the risk due to spot checking.  We have a number of alternatives. For example: Modeling and simulation Iterative testing throughout the development Launch AND continue test and verification Launch and continue test and verification reduce volume of customers Relying upon modeling and other forms of simulation […]

We should guard against our organization becoming enamored of the idea of “spot checking”.  Spot checking, if done at all, should be based upon facts, calculations and historical record and not due to the logistical inconvenience of a hard date.  Spot checking is when we perform a cursory review of the product. Some folks call […]

We have briefly discussed why verification is important to the product quality. Verification does not just address the product quality. Our project work requires verification as well. When we take on a project, we should have the scope articulated in a way that we can confirm that the project did indeed fulfill the objective.  As […]

Requirements management and configuration management are required for anything that even closely resembles effective testing.  Experience suggests failing in these two areas unnecessarily complicates the product verification activities, and we will show some of those traumas in the next few posts.  An iterative and incremental product development process calls for reviews throughout the development process.  […]