Why formalize root cause analysis?
There are many approaches to determining the root of our problems. In the automotive world, there are two typical approaches, the 8D or the 8 Discipline, or the A3 (named for the paper size). We will not go into the details of either of these approaches but you can find the templates for these by clicking the links above.
There are some benefits to a formalized root cause analysis. We can think of 6 reasons as listed below:
Controls jumping to conclusions and just working on the symptom Repeatable Coordinates effort / focus Documents actions so we know what have tried and what is next Traceable for future events
Controls jumping to conclusions and just working on the symptom
A formalized approach to root cause analysis in such a way that will keep us from jumping to conclusions about the nature of the failure. Those that have read this blog for … Continue reading
We are composing a glossary of test terms and we take one from that work today and discuss here.
A development organization can be structured in many ways. The development and testing can be essentially one department under one management structure, or these two areas, development and test can be separated each with a respective hierarchy. Like most things, there are benefits and drawbacks for each approach. The trick is to map the biggest benefit to your organization, or reduce the negative effects on your organization. All of this means the selection for organizational structure is likely not best left to an arbitrary decision.
I am sure we see the obvious benefit of having the test personnel integrated with the development staff. Those who have been in development for awhile no doubt understand the communications challenges that can come with separating groups when interaction between those groups is paramount for project success. So, the benefit of these two disciplines … Continue reading
Independent Test and Verification
We are composing a glossary of testing terms and we take one from that work today and discuss here. We welcome any and all that would like to contribute.
Today we discuss development and independent test and verification. A development organization can be structured in many ways. The development and testing can be essentially one department under one management structure, or these two areas, development and testing can be separated each with a respective hierarchy. Like most things, there are benefits and drawbacks for each approach. The trick is to map the biggest benefit to your organization, or reduce the negative effects on your organization. All of this means the selection for organizational structure is likely not best left to an arbitrary decision.
I am sure we see the obvious benefit of having the test personnel integrated with the development staff. Those who have been in development for awhile no doubt understand the communications … Continue reading
There have been some twitter discussions going on about the validity of the term end-to-end (E2E) testing. I have been around this concept for many years and still see the validity in the term and the approach. To that end, I will describe as quick and briefly as I can since this is a blog post and not a book.
The various devices being developed (Device Under Test)
Vehicles are comprised of a myriad of subsystems. In this case we are going to look at the vehicle electrical / electronic architecture that consists of embedded electronic control units (ECU). These are component such as:
Instrument cluster Engine ECU Transmission ECU Antilocking Brakes System (ECU) Door control modules Telemetry systems
Each of these consists of many analog and digital inputs and outputs, along with data link communications. Each have specific expectations under certain conditions and the developers will work to deliver these algorithms and software that will ensure the system works … Continue reading
by: Shawn P. Quigley
Whereas we have discussed some of the possible flaws in measurements we can all still agree that they are needed to provide both improvement in processes and the organization. However, other aspects of obtaining data for the production of quantifiable information: trend analysis and process evaluation, is the human factor both workers and management. As in so many of our conversations we look at the affect it has on the people who are essentially being evaluated by the information gathered for these measures. An issue we will discuss later in this post, but first let us look at the management aspect of this equation.
As a quality analysis person data may seem to be clear most of the time, but as a management person how do you gauge the data which is being received? Do you understand its’ meaning? Do you look at the outliers to forecast or do you think they are just noise to … Continue reading
by: Shawn P. Quigley & Jon M Quigley
Measurements and Bias
Solely by the process of observing something we can alter the thing which is being observed. This is a known as the observer-expectancy effect. This effect is born out of conscience and subconscious biases of the observer. In the case of observing people, we have noted in earlier blog posts that the act of observation, taking an interest, may alter the outcome or performance as well (consider The Hawthorne study). To make an effective measurement we must work to account for these impacts. We also must know the goal we have set for collecting the information, that is the measurement is context based. Having a specific goal, informs the type of data and methods of data collection. Both of these are rife with opportunity for bias to creep into the measurements, and delude our team. This bias can creep in not only, what we identify … Continue reading
Verification and Validation
The definition for verification and validation can be found at:
We must express some disagreement with the activities associated with the individual areas. For example, testing is not limited to Validation. Testing is also a function of verification as we will use these techniques to understand if the instantiation of the product meets the specifications. Testing will be part of the quality assurance activities for the product – are we building what we said we would – through iterations of the product. Big bang is dead – meaning build everything at the beginning no longer happens. That method only works in products of the smallest scope. Rather, we will increment the feature content, verify the product build matches requirements delivered, log bugs and get ready for the next round. The next round fixes bugs, adds more of the requirements to the product content verify and so forth.
Likewise for Validation, there are things other … Continue reading
Testing Complex and Embedded Systems What set of conditions could cause this event to occur?
When we have elicited all we can from the customer about fault information, it is time to proceed further in our analysis. This next step requires investigation of the design to understand how the symptom of failure described could happen by breaking down the hardware and software and the interactions within them to understand the improper behavior of the features to the customer. If the investigator is in the automotive, pharmacy, or food industries, they can resort to an immediate perusal of the Design Failure Mode and Effects Analysis (DFMEA) and the Process Failure Mode and Effects Analysis (PFMEA). If our investigator is lucky, they may find pointers to the cause of the issue in these documents.
To be successful, we need to perform a rigorous and systematic critique of the design—with enough follow up to ensure that any correctable issues have been resolved. Usually, … Continue reading
The Emissions Tango
There is so much to learn from this case for those who develop products for a living and automotive products in particular. Understanding the impact of concept selection and testing of the product on the project success and product quality is important. The early decisions we make regarding the development of the product will stay with us long after launch or we may find ourselves reworking the entire product.
Trade – offs
Most every design solution requires trade-off considerations. For example, we sacrifice weight for reliability; or we may increase the cost to obtain a longer battery life. Making this trade-off decision requires forethought and systems level thinking regarding the consequences and risks to product development and production. In the case of the VW diesel situation, the trade off was NOx and CO2. 
The starting point of the diesel matter was, in hindsight, the strategic decision by Volkswagen in 2005 to start a … Continue reading
Below is an excerpt of our book Testing of Complex and Embedded Systems
Pries, K., & Quigley, J. (2011). Chapter 4. In Testing Complex and Embedded Systems (pp. 33-35). Boca Raton, FL: CRC Press.
Basic Principle of Testing
The verification and test group is there to provide some critical and unbiased review of the product. This is used to understand the real quality of the product and make adjustments as to improve that quality. When we find a bug or defect, we are in a position to consider whether it gets corrected before the product is shipped to the customer. Without this work, the first opportunity to ascertain the product quality would be the customer.
“Concentrate on what cannot lie–the evidence.” ~ Grissom to Warrick in the CSI Pilot
When we test, we are in a position to provide evidence-based results and conclusions to our product development team. In the famous Kalama Sutra, Siddhartha Gautama–the historical Buddha–explains to the … Continue reading