Regression Testing

By Jon M. Quigley

 

I saw a LinkedIn post yesterday about scope of testing during times of compressed schedule. The position was to test what is new in the software, and of that new, what is the most important, perhaps meaning what if it goes wrong, would be the worst for the client or customer.  Generally this is probably a good idea. However, there are some drawbacks to this approach.  This means no regression testing. Regression testing is testing of the old software features when we add new software features to the product.

Testing as above, is predicated on the belief that those things that we have changed or added, have no implication or impact on those features and functions that were already in place prior to this last iteration of the software. That may not be true.  If we make changes to some software module that is used by other functions, we may miss testing a change in … Continue reading

Common Cause and Special Cause Variation

Variation!

Though sometimes we may refuse to recognize it, the world is a full of variation, even in the things we think or believe are constant. For example, my wife has been known to say, “you always do…” or “you never do…”, to which I retort, I am human and I am not that repeatable.  I say that, but it is not just humans that are not so repeatable.  Everything has variation, and understanding this variation, is important for product development, project management, manufacturing, product testing and so much more.  To master this work, we need to understand this variation as completely as possible.

Common and Special Cause Variation

Shewart is credited for developing the concepts of special and common cause [1].  Special cause variation, are variations that are outside of the expected (intermittent) range of possibilities. Common cause variation, are the variation expected, we know about these, these are predictable, provided we have put some effort into learning about … Continue reading

Poke Yoke

Poke Yoke

Jon M Quigley

There are a set of tools and techniques that come with developing products for the automotive industry and are part of the Advanced Product Quality Planning for the product.  We have written about APQP or some years and have decades of experience in this approach to product development.  In general, the phases of the project are described as:

Voice of Customer Product Development Process Development Product Validation Process Validation Launch Feedback

 

 

We have a discussion board that supports questions you may have regarding APQP.  Today, we will ruminate on the technique known as Poke Yoke, which I heard an Supplier Quality Assurance professional refer to as “goof proofing”.  While largely benefiting manufacturing, these considerations and implementation is achieved in the development work, otherwise we end up doing considerable rework of the developed product to optimize manufacturing or field service work. Rather, we will use design for manufacturing (DFM) or design for … Continue reading

Missing Requirements

Missing Requirements

Sometimes the best way to convey the challenges in product development, is to show some of the reasons things can go wrong.  To that end I am going to regale you with a tale of requirements gone astray.

Wearing Many Hats

I worked at a small company a long time ago. I am glad I did, as I wore many hats and learned something new every day while having this job at this small company.  This particular instance, we were to design and build an embedded control system for a large manufacturer, and the requirements were listed on a short sheet of paper as a bulleted list of the objective the constraints, and the general performance expected.

Product Development Input

The primary source of contact and the requirement was the head of marketing. This company generally had short communication and hierarchy chains.  However, the marketing person was essentially a customer advocate and not really an engineer.  … Continue reading

Why formalize root cause analysis?

Why formalize root cause analysis?

 

There are many approaches to determining the root of our problems.  In the automotive world, there are two typical approaches, the 8D or the 8 Discipline, or the A3 (named for the paper size).  We will not go into the details of either of these approaches but you can find the templates for these by clicking the links above.

There are some benefits to a formalized root cause analysis.  We can think of 6 reasons as listed below:

Controls jumping to conclusions and just working on the symptom Repeatable Coordinates effort / focus Documents actions so we know what have tried and what is next Traceable for future events

 

 

Controls jumping to conclusions and just working on the symptom

A formalized approach to root cause analysis in such a way that will keep us from jumping to conclusions about the nature of the failure.   Those that have read this … Continue reading

Independent Test and Verification

Independent Test and Verification

We are composing a glossary of testing terms and we take one from that work today and discuss here.   We welcome any and all that would like to contribute.

Today we discuss development and independent test and verification.  A development organization can be structured in many ways. The development and testing can be essentially one department under one management structure, or these two areas, development and testing can be separated each with a respective hierarchy.  Like most things, there are benefits and drawbacks for each approach.  The trick is to map the biggest benefit to your organization, or reduce the negative effects on your organization.  All of this means the selection for organizational structure is likely not best left to an arbitrary decision.

Communication

I am sure we see the obvious benefit of having the test personnel integrated with the development staff.  Those who have been in development for awhile no doubt understand the … Continue reading

Task Dependencies and Probability

Flip a coin, heads or tails. The probability that it will come up heads is 50%, after all there are only two sides.  Flip that same coin again, the same probability, 50%.  However, if we say that success is two successive heads (or tails) that is different. The probability of two consecutive heads, is the product of the two coin tosses (50% x 50%) or 25%.

I’m sure we understand task dependencies.  This is fundamental to project management and schedule.  Dependencies are the connections between tasks for example, I must put socks on before I put my shoes on (assuming I wear socks, which I generally do).  Of course, this is not the only type of dependency (finish-start). There are four actually;

Finish-start Start-start Start-finish Finish-finish

 

 

Let us focus on finish-start for a bit. Specifically, lets talk about a portion of the resulting schedule and the task consequences on the entire project.

Each task … Continue reading

Testing – End-To-End

There have been some twitter discussions going on about the validity of the term end-to-end (E2E) testing. I have been around this concept for many years and still see the validity in the term and the approach. To that end, I will describe as quick and briefly as I can since this is a blog post and not a book.

The various devices being developed (Device Under Test)

Vehicles are comprised of a myriad of subsystems. In this case we are going to look at the vehicle electrical / electronic architecture that consists of embedded electronic control units (ECU). These are component such as:

Instrument cluster Engine ECU Transmission ECU Antilocking Brakes System (ECU) Door control modules Telemetry systems

Each of these consists of many analog and digital inputs and outputs, along with data link communications. Each have specific expectations under certain conditions and the developers will work to deliver these algorithms and software that will ensure the system works … Continue reading

The Process vs. The Product

I read the article Product Beats the Process and  felt compelled to respond

Product Beats Process

The author, Jeff Morris Jr. is right, the populous or customers at large probably do not care about how we arrived at the product. They do not care about the process. They do not care about the creative problem solving that went into bringing the product to life.  They do not care about specific Jira or defect report tickets.  They do not care about long hours, personality clashes, and design trade off curves.  However, I disagree when the author says they only care about how it feels in their hands and nothing else.  This disagreement is not based upon some statistical analysis so, perhaps he is correct. It is however, based upon my personal priorities and experience, and I find it difficult to believe that I am a singularity.

What matters is that we build products that make customers smile and realize … Continue reading

Measurement effects and analysis on personnel and organizations

by: Shawn P. Quigley

Whereas we have discussed some of the possible flaws in measurements we can all still agree that they are needed to provide both improvement in processes and the organization. However, other aspects of obtaining data for the production of quantifiable information: trend analysis and process evaluation, is the human factor both workers and management. As in so many of our conversations we look at the affect it has on the people who are essentially being evaluated by the information gathered for these measures. An issue we will discuss later in this post, but first let us look at the management aspect of this equation.

As a quality analysis person data may seem to be clear most of the time, but as a management person how do you gauge the data which is being received? Do you understand its’ meaning? Do you look at the outliers to forecast or do you think they are just noise to … Continue reading

Single Sign On provided by vBSSO