We are composing a glossary of test terms and we take one from that work today and discuss here.
A development organization can be structured in many ways. The development and testing can be essentially one department under one management structure, or these two areas, development and test can be separated each with a respective hierarchy. Like most things, there are benefits and drawbacks for each approach. The trick is to map the biggest benefit to your organization, or reduce the negative effects on your organization. All of this means the selection for organizational structure is likely not best left to an arbitrary decision.
I am sure we see the obvious benefit of having the test personnel integrated with the development staff. Those who have been in development for awhile no doubt understand the communications challenges that can come with separating groups when interaction between those groups is paramount for project success. So, the benefit of these two disciplines … Continue reading
Today we discuss interchangeability of parts. This may sound trivial, but you probably would not even consider replacing your food processor blade with your lawnmower blade. It would be obvious that these are not interchangeable. However, there are times when a part needs to be replaced or design is reworked altering the composition, that is the constituent parts of the product or design.
Let’s take an example and walked through it. Consider the ignition key switch to your car. It has a specific shape, mounts and attached with specific mechanisms, and performs or initiates many actions of the vehicle. This is often referred to as form, fit and function. Let’s consider the old ignition switch is going out of production from your supplier. We would need to find another solution. As an engineer we would explore alternative or replacement parts. If we want to minimize costs, we would look for another ignition switch that forms, fits and functions identically to … Continue reading
Independent Test and Verification
We are composing a glossary of testing terms and we take one from that work today and discuss here. We welcome any and all that would like to contribute.
Today we discuss development and independent test and verification. A development organization can be structured in many ways. The development and testing can be essentially one department under one management structure, or these two areas, development and testing can be separated each with a respective hierarchy. Like most things, there are benefits and drawbacks for each approach. The trick is to map the biggest benefit to your organization, or reduce the negative effects on your organization. All of this means the selection for organizational structure is likely not best left to an arbitrary decision.
I am sure we see the obvious benefit of having the test personnel integrated with the development staff. Those who have been in development for awhile no doubt understand the … Continue reading
You are in college now, and you see people that are likely smart as far as youth will provide. Everybody can have strong “opinions” and perhaps their track record in high school has been one of success after success or being the person known to be knowledgeable – as far as high school topics go. University work is much different as I am sure you know.
I have been ruminating about engineers and what I think makes a good one. It is not the know it all’s – and many people besides engineers fall into that category (have this malady). People are so certain of what they know to be true, but is likely not at all true, or only true for a very finite or specific situation. Engineering is as much about creativity as science. It is about devising experiments for those situations in which it is not so easy to understand or calculate due to complexity, or wide … Continue reading
There have been some twitter discussions going on about the validity of the term end-to-end (E2E) testing. I have been around this concept for many years and still see the validity in the term and the approach. To that end, I will describe as quick and briefly as I can since this is a blog post and not a book.
The various devices being developed (Device Under Test)
Vehicles are comprised of a myriad of subsystems. In this case we are going to look at the vehicle electrical / electronic architecture that consists of embedded electronic control units (ECU). These are component such as:
Instrument cluster Engine ECU Transmission ECU Antilocking Brakes System (ECU) Door control modules Telemetry systems
Each of these consists of many analog and digital inputs and outputs, along with data link communications. Each have specific expectations under certain conditions and the developers will work to deliver these algorithms and software that will ensure the system works … Continue reading
by: Shawn P. Quigley & Jon M Quigley
Measurements and Bias
Solely by the process of observing something we can alter the thing which is being observed. This is a known as the observer-expectancy effect. This effect is born out of conscience and subconscious biases of the observer. In the case of observing people, we have noted in earlier blog posts that the act of observation, taking an interest, may alter the outcome or performance as well (consider The Hawthorne study). To make an effective measurement we must work to account for these impacts. We also must know the goal we have set for collecting the information, that is the measurement is context based. Having a specific goal, informs the type of data and methods of data collection. Both of these are rife with opportunity for bias to creep into the measurements, and delude our team. This bias can creep in not only, what we identify … Continue reading
Verification and Validation
The definition for verification and validation can be found at:
We must express some disagreement with the activities associated with the individual areas. For example, testing is not limited to Validation. Testing is also a function of verification as we will use these techniques to understand if the instantiation of the product meets the specifications. Testing will be part of the quality assurance activities for the product – are we building what we said we would – through iterations of the product. Big bang is dead – meaning build everything at the beginning no longer happens. That method only works in products of the smallest scope. Rather, we will increment the feature content, verify the product build matches requirements delivered, log bugs and get ready for the next round. The next round fixes bugs, adds more of the requirements to the product content verify and so forth.
Likewise for Validation, there are things … Continue reading
Testing Complex and Embedded Systems What set of conditions could cause this event to occur?
When we have elicited all we can from the customer about fault information, it is time to proceed further in our analysis. This next step requires investigation of the design to understand how the symptom of failure described could happen by breaking down the hardware and software and the interactions within them to understand the improper behavior of the features to the customer. If the investigator is in the automotive, pharmacy, or food industries, they can resort to an immediate perusal of the Design Failure Mode and Effects Analysis (DFMEA) and the Process Failure Mode and Effects Analysis (PFMEA). If our investigator is lucky, they may find pointers to the cause of the issue in these documents.
To be successful, we need to perform a rigorous and systematic critique of the design—with enough follow up to ensure that any correctable issues have been resolved. Usually, … Continue reading
No matter the industry or domain, there will be times when we need to quickly make a management or technical decision. Decision matrix allows us to compare a range of solutions to a defined set of criterion. A quick and easy tool to accomplish this assessment is the decision matrix, of which there are two varieties, the non-weighted decision matrix and the weighted decision matrix. The non-weighted is the quickest though the two are not much different. We will, in a later blog post, discuss the weighted version often referred to as the Pugh Matrix.
To create the table, we need a few of things. First we need to know what we are trying to do, our objective. Second, we will need to generate approaches that we think will allow us to reach those objectives. Thirdly, we will then need to identify desirable attributes of a successful implementation. Finally, we will compare the approaches to the desired attributes and select … Continue reading
Testing and Repeat-ability
Repeat-ability of testing results is important to establishing cause and corrective actions. If it is not possible to repeat the sequence of events leading to a failure, it is not possible to replicate and therefore difficult solve the cause of the fault or failure. The steps that evoked the problem are necessary to replicating the problem – only sometimes that seemingly does not work.
Testing and Configuration Management
Things like a specific system configuration may be implicated in the failure, testing to find a failure on one configuration after having found the problem on another configuration, does not mean the failure does not exist. This is another link between testing and configuration management.
Repeating the test with some fidelity requires documenting the test steps with some diligence as well as trace-ability of the test specimen, all parts are not created equal. When it comes to software or embedded products the latter is very true as it is … Continue reading