Having the plan is only partially helpful. The list of test cases and the expected rate of accomplishment allows us to refine our estimates as we progress through the testing. We will be in a position to provide the project manager and stakeholders with a better “ETA” (Estimated Time of Arrival) just like the GPS informs us as we progress toward our final destination.
We know must execute to see if the planned rate of accomplishment and expected conclusion date are viable. We will the monitor and track the progress of the team. Here is where we take another play from the agile playbook. Each morning we have a short 15 minute meeting with the verification team members. Filling a role comparable to a scrum master, we ask:
What did you do yesterday? What do you have planned for today? What obstacles are you encountering?
Notes are taken especially when it comes to obstacles. We can ill afford to have … Continue reading
Testing Complex and Embedded Systems What set of conditions could cause this event to occur?
When we have elicited all we can from the customer about fault information, it is time to proceed further in our analysis. This next step requires investigation of the design to understand how the symptom of failure described could happen by breaking down the hardware and software and the interactions within them to understand the improper behavior of the features to the customer. If the investigator is in the automotive, pharmacy, or food industries, they can resort to an immediate perusal of the Design Failure Mode and Effects Analysis (DFMEA) and the Process Failure Mode and Effects Analysis (PFMEA). If our investigator is lucky, they may find pointers to the cause of the issue in these documents.
To be successful, we need to perform a rigorous and systematic critique of the design—with enough follow up to ensure that any correctable issues have been resolved. Usually, … Continue reading
Not all television is not mind numbing. I enjoy The History Channel and many other similar channels as these are not exactly learning opportunities but close. However, my son turned us on to a show called House on Netflix and it is very interesting.
House (also called House, M.D.) is an American television medical drama that originally ran on the Fox network for eight seasons, from November 16, 2004 to May 21, 2012. The series’ main character is Dr. Gregory House (Hugh Laurie), a pain medication-dependent, unconventional, misanthropic medical genius who leads a team of diagnosticians at the fictional Princeton–Plainsboro Teaching Hospital (PPTH) in New Jersey.
All of you engineers, test engineers, and project management people out there could benefit watching this show. In this show, the premier diagnostician – House works to solve complex physiological and life threatening conundrums with his patients. He does this by considering the symptoms (failure mode) and then drawing … Continue reading
Software Structure Defined
Non-functional requirements such as software extensibility can be very difficult to document as we likely do not know all of the future features or growth we can anticipate for the product as it matures. Poorly managed, the code may descend into what is sometimes referred to as spaghetti code. Instead of the document focusing solely upon the features, we can put some requirements for the structure in place before the coding even begins. A way to do this is through the use of a software design description document. This document may be especially important the first time working with the vendor or if there are some questions about the maturity level of the software department. It may not be necessary to create a software design description document for every project. An organization of a sufficient maturity level may not require such an artifact. We can glean some understanding of an organization’s capability through reviews of … Continue reading
We will continue our review of CMMI and requirements management practices. As we have seen in the earlier posts, managing the requirements is necessary for efficient development and doing so has positive impacts on the project as well. Specifically, the project benefits when the organization stands behind attaining the requirements, and is in for a rough road when this commitment to the requirements is in fact not there. At the very least if the organization is not backing the project, the project manager will understand the risks this presents to the project. The project also benefits from controlling the changes to the requirements. We are going to learn as we do the work and that learning will be mechanisms for product updates.
In this post, we are discussing being able to trace the top level customer needs through the system and to the detailed level requirements for the product or system. We reduce the turbulence the project encounters when … Continue reading
Continuous Deliver and Embedded Automotive
I have worked on projects that employed continuous delivery for embedded products. The embedded product was an automotive component. The core of the software (the operating system) was specified using conventional approach. This operating system consisted of the maximum model requirements for this globally used component. The component looked and acted differently depending upon market segment and geographic location, for example, there were two different European incarnations and more than two for the markets of the United States. This was not just some parameter setting or configuration differences, but whole scale differences in feature content and manner of execution.
As we indicated, the operating system, though we called it an execution engine, was specified and developed using an incremental approach. This incremental approach was based upon a prioritized set of features. That prioritization was based on what it would take to produce a vehicle that would operate safely allowing for testing of key features. In the … Continue reading
Project Closure and Team Disband
One of my favorites and often poorly executed part of a project is the project closure. Project closure is not just a simple wipe our hands together and smile, then proclaim that we are finished with the project. It is not even handing the customer our results and patting everybody on the back – and on to our congratulatory soda. The roots of the closure of the project happen in the very beginning of the project when we establish the scope of the project. If we are under contract the seeds we sow in the contract will be the harvest we reap during contract closure.
Project Contract Closure
To effectively close the project out, we need to have identified the project scope in a tangible way. That requires a way of tracing the top level scope (customer expectation) to the details of the project, which in the product development world, is typically contained in the … Continue reading
The Emissions Tango
There is so much to learn from this case for those who develop products for a living and automotive products in particular. Understanding the impact of concept selection and testing of the product on the project success and product quality is important. The early decisions we make regarding the development of the product will stay with us long after launch or we may find ourselves reworking the entire product.
Trade – offs
Most every design solution requires trade-off considerations. For example, we sacrifice weight for reliability; or we may increase the cost to obtain a longer battery life. Making this trade-off decision requires forethought and systems level thinking regarding the consequences and risks to product development and production. In the case of the VW diesel situation, the trade off was NOx and CO2. 
The starting point of the diesel matter was, in hindsight, the strategic decision by Volkswagen in 2005 to start a … Continue reading
Regression Testing and Scope
A brief exchange on twitter on the topic of regression testing leads me to this post. Twitter, is not the place to really discuss this topic as there are too few characters and there are many factors that go into determining the scope of regression test. Regression testing is the testing of portions of the product that were built previous to this iteration. The testing is to ensure that in the addition of the new features to the product we have not inadvertently broken the product. A good definition of regression testing can be found from ISTQB.
One of the things we will consider in determining the scope of our regression testing will be the level at which our company is comfortable with the risk associated with the product in the field. If our company makes medical or perhaps automotive or aerospace products, we may generally be risk averse since the consequences of our … Continue reading
We can use a decision matrix to help determine the best test strategy. In this instance, the decision matrix is comparing what we believe to be vehicle testing success criterion (such as the fidelity of the test results and ability to duplicate, the speed at which we can test and meeting critical dependencies) against a number of possible solutions. The highest scoring approach represents the best approach given the constraints.