Regression Testing

By Jon M. Quigley

 

I saw a LinkedIn post yesterday about scope of testing during times of compressed schedule. The position was to test what is new in the software, and of that new, what is the most important, perhaps meaning what if it goes wrong, would be the worst for the client or customer.  Generally this is probably a good idea. However, there are some drawbacks to this approach.  This means no regression testing. Regression testing is testing of the old software features when we add new software features to the product.

Testing as above, is predicated on the belief that those things that we have changed or added, have no implication or impact on those features and functions that were already in place prior to this last iteration of the software. That may not be true.  If we make changes to some software module that is used by other functions, we may miss testing a change in … Continue reading

Why formalize root cause analysis?

Why formalize root cause analysis?

 

There are many approaches to determining the root of our problems.  In the automotive world, there are two typical approaches, the 8D or the 8 Discipline, or the A3 (named for the paper size).  We will not go into the details of either of these approaches but you can find the templates for these by clicking the links above.

There are some benefits to a formalized root cause analysis.  We can think of 6 reasons as listed below:

Controls jumping to conclusions and just working on the symptom Repeatable Coordinates effort / focus Documents actions so we know what have tried and what is next Traceable for future events

 

 

Controls jumping to conclusions and just working on the symptom

A formalized approach to root cause analysis in such a way that will keep us from jumping to conclusions about the nature of the failure.   Those that have read this … Continue reading

Independent Test and Verification

We are composing a glossary of test terms and we take one from that work today and discuss here.

A development organization can be structured in many ways. The development and testing can be essentially one department under one management structure, or these two areas, development and test can be separated each with a respective hierarchy.  Like most things, there are benefits and drawbacks for each approach.  The trick is to map the biggest benefit to your organization, or reduce the negative effects on your organization.  All of this means the selection for organizational structure is likely not best left to an arbitrary decision.

Communication

I am sure we see the obvious benefit of having the test personnel integrated with the development staff.  Those who have been in development for awhile no doubt understand the communications challenges that can come with separating groups when interaction between those groups is paramount for project success.  So, the benefit of these two disciplines … Continue reading

Drop in Replacement

Today we discuss interchangeability of parts.  This may sound trivial, but you probably would not even consider replacing your food processor blade with your lawnmower blade.  It would be obvious that these are not interchangeable.  However, there are times when a part needs to be replaced or design is reworked altering the composition, that is the constituent parts of the product or design.

Let’s take an example and walked through it.  Consider the ignition key switch to your car.  It has a specific shape, mounts and attached with specific mechanisms, and performs or initiates many actions of the vehicle.  This is often referred to as form, fit and function.  Let’s consider the old ignition switch is going out of production from your supplier.  We would need to find another solution.  As an engineer we would explore alternative or replacement parts.  If we want to minimize costs, we would look for another ignition switch that forms, fits and functions identically to … Continue reading

Independent Test and Verification

Independent Test and Verification

We are composing a glossary of testing terms and we take one from that work today and discuss here.   We welcome any and all that would like to contribute.

Today we discuss development and independent test and verification.  A development organization can be structured in many ways. The development and testing can be essentially one department under one management structure, or these two areas, development and testing can be separated each with a respective hierarchy.  Like most things, there are benefits and drawbacks for each approach.  The trick is to map the biggest benefit to your organization, or reduce the negative effects on your organization.  All of this means the selection for organizational structure is likely not best left to an arbitrary decision.

Communication

I am sure we see the obvious benefit of having the test personnel integrated with the development staff.  Those who have been in development for awhile no doubt understand the … Continue reading

Off Topic – A note to a son.

You are in college now, and you see people that are likely smart as far as youth will provide.  Everybody can have strong “opinions” and perhaps their track record in high school has been one of success after success or being the person known to be knowledgeable – as far as high school topics go.  University work is much different as I am sure you know.

I have been ruminating about engineers and what I think makes a good one. It is not the know it all’s – and many people besides engineers fall into that category (have this malady). People are so certain of what they know to be true, but is likely not at all true, or only true for a very finite or specific situation.  Engineering is as much about creativity as science.  It is about devising experiments for those situations in which it is not so easy to understand or calculate due to complexity, or wide … Continue reading

Testing – End-To-End

There have been some twitter discussions going on about the validity of the term end-to-end (E2E) testing. I have been around this concept for many years and still see the validity in the term and the approach. To that end, I will describe as quick and briefly as I can since this is a blog post and not a book.

The various devices being developed (Device Under Test)

Vehicles are comprised of a myriad of subsystems. In this case we are going to look at the vehicle electrical / electronic architecture that consists of embedded electronic control units (ECU). These are component such as:

Instrument cluster Engine ECU Transmission ECU Antilocking Brakes System (ECU) Door control modules Telemetry systems

Each of these consists of many analog and digital inputs and outputs, along with data link communications. Each have specific expectations under certain conditions and the developers will work to deliver these algorithms and software that will ensure the system works … Continue reading

Measurement effects and analysis on personnel and organizations

by: Shawn P. Quigley

Whereas we have discussed some of the possible flaws in measurements we can all still agree that they are needed to provide both improvement in processes and the organization. However, other aspects of obtaining data for the production of quantifiable information: trend analysis and process evaluation, is the human factor both workers and management. As in so many of our conversations we look at the affect it has on the people who are essentially being evaluated by the information gathered for these measures. An issue we will discuss later in this post, but first let us look at the management aspect of this equation.

As a quality analysis person data may seem to be clear most of the time, but as a management person how do you gauge the data which is being received? Do you understand its’ meaning? Do you look at the outliers to forecast or do you think they are just noise to … Continue reading

Analysis of Measurements

by: Shawn P. Quigley & Jon M Quigley

Measurements and Bias

Solely by the process of observing something we can alter the thing which is being observed. This is a known as the observer-expectancy effect.  This effect is born out of conscience and subconscious biases of the observer.  In the case of observing people, we have noted in earlier blog posts that the act of observation, taking an interest, may alter the outcome or performance as well (consider The Hawthorne study).  To make an effective measurement we must work to account for these impacts.  We also must know the goal we have set for collecting the information, that is the measurement is context based.  Having a specific goal, informs the type of data and methods of data collection.  Both of these are rife with opportunity for bias to creep into the measurements,  and delude our team.  This bias can creep in not only, what we identify … Continue reading

Verification and Validation

Verification and Validation

The definition for verification and validation can be found at[1]:

 

We must express some disagreement with the activities associated with the individual areas. For example, testing is not limited to Validation. Testing is also a function of verification as we will use these techniques to understand if the instantiation of the product meets the specifications.  Testing will be part of the quality assurance activities for the product – are we building what we said we would – through iterations of the product.  Big bang is dead – meaning build everything at the beginning no longer happens. That method only works in products of the smallest scope.  Rather, we will increment the feature content, verify the product build matches requirements delivered, log bugs and get ready for the next round. The next round fixes bugs, adds more of the requirements to the product content verify and so forth.

Likewise for Validation, there are things … Continue reading

Single Sign On provided by vBSSO