Home »

Tag Archives: testing

Regression Testing

   admin
   Posted on : Jan 09, 2019
   Categories : Business, Quality, testing | Tags: , , , , | Leave A Comment »

By Jon M. Quigley

 

I saw a LinkedIn post yesterday about scope of testing during times of compressed schedule. The position was to test what is new in the software, and of that new, what is the most important, perhaps meaning what if it goes wrong, would be the worst for the client or customer.  Generally this is probably a good idea. However, there are some drawbacks to this approach.  This means no regression testing. Regression testing is testing of the old software features when we add new software features to the product.

Testing as above, is predicated on the belief that those things that we have changed or added, have no implication or impact on those features and functions that were already in place prior to this last iteration of the software. That may not be true.  If we make changes to some software module that is used by other functions, we may miss testing a change in … Continue reading →

Posted in Business, Quality, testing | Tagged , , , , | Leave a reply

Why formalize root cause analysis?

   admin
   Posted on : Apr 30, 2018
   Categories : Quality, root cause analysis, testing, Verifcation | Tags: , , , | Leave A Comment »

Why formalize root cause analysis?

 

There are many approaches to determining the root of our problems.  In the automotive world, there are two typical approaches, the 8D or the 8 Discipline, or the A3 (named for the paper size).  We will not go into the details of either of these approaches but you can find the templates for these by clicking the links above.

There are some benefits to a formalized root cause analysis.  We can think of 6 reasons as listed below:

Controls jumping to conclusions and just working on the symptom Repeatable Coordinates effort / focus Documents actions so we know what have tried and what is next Traceable for future events

 

 

Controls jumping to conclusions and just working on the symptom

A formalized approach to root cause analysis in such a way that will keep us from jumping to conclusions about the nature of the failure.   Those that have read this blog for … Continue reading →

Posted in Quality, root cause analysis, testing, Verifcation | Tagged , , , | Leave a reply

Independent Test and Verification

   admin
   Posted on : Mar 28, 2018
   Categories : APQP, testing, Verifcation, verification | Tags: , | Leave A Comment »

We are composing a glossary of test terms and we take one from that work today and discuss here.

A development organization can be structured in many ways. The development and testing can be essentially one department under one management structure, or these two areas, development and test can be separated each with a respective hierarchy.  Like most things, there are benefits and drawbacks for each approach.  The trick is to map the biggest benefit to your organization, or reduce the negative effects on your organization.  All of this means the selection for organizational structure is likely not best left to an arbitrary decision.

Communication

I am sure we see the obvious benefit of having the test personnel integrated with the development staff.  Those who have been in development for awhile no doubt understand the communications challenges that can come with separating groups when interaction between those groups is paramount for project success.  So, the benefit of these two disciplines … Continue reading →

Posted in APQP, testing, Verifcation, verification | Tagged , | Leave a reply

Testing – End-To-End

   admin
   Posted on : Oct 10, 2017
   Categories : prototype, Quality, root cause analysis, testing, Verifcation, verification | Tags: , , | Leave A Comment »

There have been some twitter discussions going on about the validity of the term end-to-end (E2E) testing. I have been around this concept for many years and still see the validity in the term and the approach. To that end, I will describe as quick and briefly as I can since this is a blog post and not a book.

The various devices being developed (Device Under Test)

Vehicles are comprised of a myriad of subsystems. In this case we are going to look at the vehicle electrical / electronic architecture that consists of embedded electronic control units (ECU). These are component such as:

Instrument cluster Engine ECU Transmission ECU Antilocking Brakes System (ECU) Door control modules Telemetry systems

Each of these consists of many analog and digital inputs and outputs, along with data link communications. Each have specific expectations under certain conditions and the developers will work to deliver these algorithms and software that will ensure the system works … Continue reading →

Posted in prototype, Quality, root cause analysis, testing, Verifcation, verification | Tagged , , | Leave a reply

Verification and Validation

   admin
   Posted on : Jan 26, 2017
   Categories : Business, process improvement, project management, Quality, testing, validation, Verifcation, verification | Tags: , , , | Leave A Comment »

Verification and Validation

The definition for verification and validation can be found at[1]:

 

We must express some disagreement with the activities associated with the individual areas. For example, testing is not limited to Validation. Testing is also a function of verification as we will use these techniques to understand if the instantiation of the product meets the specifications.  Testing will be part of the quality assurance activities for the product – are we building what we said we would – through iterations of the product.  Big bang is dead – meaning build everything at the beginning no longer happens. That method only works in products of the smallest scope.  Rather, we will increment the feature content, verify the product build matches requirements delivered, log bugs and get ready for the next round. The next round fixes bugs, adds more of the requirements to the product content verify and so forth.

Likewise for Validation, there are things other … Continue reading →

Posted in Business, process improvement, project management, Quality, testing, validation, Verifcation, verification | Tagged , , , | Leave a reply

The Un-repeatable Fault

   admin
   Posted on : Jan 18, 2017
   Categories : APQP, Quality, testing, Verifcation, verification | Tags: , , | Leave A Comment »

Testing Complex and Embedded Systems What set of conditions could cause this event to occur?

When we have elicited all we can from the customer about fault information, it is time to proceed further in our analysis. This next step requires investigation of the design to understand how the symptom of failure described could happen by breaking down the hardware and software and the interactions within them to understand the improper behavior of the features to the customer. If the investigator is in the automotive, pharmacy, or food industries, they can resort to an immediate perusal of the Design Failure Mode and Effects Analysis (DFMEA) and the Process Failure Mode and Effects Analysis (PFMEA). If our investigator is lucky, they may find pointers to the cause of the issue in these documents.

To be successful, we need to perform a rigorous and systematic critique of the design—with enough follow up to ensure that any correctable issues have been resolved. Usually, … Continue reading →

Posted in APQP, Quality, testing, Verifcation, verification | Tagged , , | Leave a reply

Project Closure

   admin
   Posted on : May 02, 2016
   Categories : Business, management, organization development, product development, project management, testing | Tags: , , , , | Leave A Comment »

Project Closure and Team Disband

One of my favorites and often poorly executed part of a project is the project closure.  Project closure is not just a simple wipe our hands together and smile, then proclaim that we are finished with the project. It is not even handing the customer our results and patting everybody on the back – and on to our congratulatory soda.  The roots of the closure of the project happen in the very beginning of the project when we establish the scope of the project.  If we are under contract the seeds we sow in the contract will be the harvest we reap during contract closure.

Project Contract Closure

To effectively close the project out, we need to have identified the project scope in a tangible way.  That requires a way of tracing the top level scope (customer expectation) to the details of the project, which in the product development world, is typically contained in the … Continue reading →

Posted in Business, management, organization development, product development, project management, testing | Tagged , , , , | Leave a reply

Regression Testing and Scope

   admin
   Posted on : Feb 19, 2016
   Categories : Business, testing | Tags: , , , , , | Leave A Comment »

Regression Testing and Scope

A brief exchange on twitter on the topic of regression testing leads me to this post.  Twitter, is not the place to really discuss this topic as there are too few characters and there are many factors that go into determining the scope of regression test.  Regression testing is the testing of portions of the product that were built previous to this iteration.  The testing is to ensure that in the addition of the new features to the product we have not inadvertently broken the product.  A good definition of regression testing can be found from ISTQB.

 

Risk Aversion 

One of the things we will consider in determining the scope of our regression testing will be the level at which our company is comfortable with the risk associated with the product in the field.  If our company makes medical or perhaps automotive or aerospace products, we may generally be risk averse since the consequences of our … Continue reading →

Posted in Business, testing | Tagged , , , , , | Leave a reply

Poor Excuse for Not Automating Testing

   admin
   Posted on : Feb 05, 2016
   Categories : Business, testing | Tags: , , , | 1 Comment »

Poor Excuse for Not Automating Testing

Recently I came across and participated in a social media exchange that proposed that automating product testing (software) was really not helpful.  Their assertion was backed with comments about personnel new to testing will be unable to learn how to test.

Testing and System Complexity

System and software complexity, the number of interactions, permutations (configurations) and feature content, and failure mode consequences can make comprehensive testing via solely manual techniques very improbable.  The only time solely manual methods may work, is for very simple and small scale products.  The number of test cases for a moderately complex system can be in the thousands.

Testing and Repeatability

Done correctly, automating testing helps ensure repeatability of testing.  This is important when we find a defect and we wish to trace the root cause of the problem, the exact set of steps of the test that evoked the defect.  Of course, Continue reading →

Posted in Business, testing | Tagged , , , | 1 Reply

Assumptions and Models in Product Development

   admin
   Posted on : Nov 17, 2015
   Categories : process improvement, product development, project management, prototype | Tags: , , , | Leave A Comment »

We have recently posted how assumptions, left unquestioned can damage a project. It is similarly true for the product when we use models and simulation to generate our product.  In the course of building these models, we will know some things for certain.  Some attributes of the model we may think we know for certain but may in fact not be valid.

That is why we couple testing with our simulation and models. We will confirm those things we believe to be true and the many things we hope to be true, and probably uncover things we did not even consider to be true through testing.  Those attributes that are connected to the desired outcome, we must understand so we can accurately predict the outcome.  Below find a graphic that shows the process for building and understanding the model[1]:

 

 

Assumptions are deadly when it comes to models.  If we believe our simulations and the many assumptions … Continue reading →

Posted in process improvement, product development, project management, prototype | Tagged , , , | Leave a reply
Single Sign On provided by vBSSO