Archive for June, 2015

Testing, failures and inability to duplicate.

Posted on: June 12th, 2015 by admin 1 Comment

Testing and Repeat-ability 

Repeat-ability of testing results is important to establishing cause and corrective actions. If it is not possible to repeat the sequence of events leading to a failure, it is not possible to replicate and therefore difficult solve the cause of the fault or failure.  The steps that evoked the problem are necessary to replicating the problem – only sometimes that seemingly does not work.

Testing and Configuration Management

Things like a specific system configuration may be implicated in the failure, testing to find a failure on one configuration after having found the problem on another configuration, does not mean the failure does not exist.  This is another link between testing and configuration management.

Testing the product

Testing the product

Repeating the test with some fidelity requires documenting the test steps with some diligence as well as trace-ability of the test specimen, all parts are not created equal.  When it comes to software or embedded products the latter is very true as it is impossible to look at compiled code or a microchip and understand what level of software is contained within the product. All of this traces back to configuration management.  

 

Exploration is like spelunking.

Exploration is like spelunking.

 

Exploratory Testing

The test cases can be another story.  It is a little difficult with exploratory testing, or when things just happen to go wrong while manipulating (playing with) the application. Often in these situations we have not created a test case with the steps prior to execution, rather we are spelunking the product.  Exactly repeating the steps including timing of and between steps.  These factors can be important contributors, sometimes timing is an ingredient to fault replication.  Thus reproducing the failure by recall is difficult. The ability to record the steps can help with that even for exploratory testing. Generally the longer the test sequence, the more difficult to recall the sequence though well documented test cases can help here.

 Memory set up

Initialization and Variables have Testing Impact

Perhaps the bug that was momentarily demonstrated is actually dependent upon the prior test case execution, or the original state of the product (memory location). In embedded products for example, we may not have defined the starting point for the contents of the accessed memory locations, but later use that memory content.  In this case the phenomenon will occur once and will seem to not be repeatable.  However, this would perhaps be repeatable on other instances of the product the first time.  Once we find this undefined memory location and define it in the software (flag register or semaphore) the first time, we can cure the fault.  For the right type of software product an ICE (In Circuit Emulator) or simulator may be helpful, as these tools allow you to set break points and peek into memory locations.

Testing and Risk

Reporting based upon level of risk is a great idea, though I am not sure if we should neglect to report this fault just because we cannot replicate and it appears to be a low risk failure impact.  I think it may be more prudent to report the failure so there is a record of the event occurring.  If the symptom occurs in the field, we will then not be in a position to say “we have never seen that failure”.  Maybe we can learn something more from the field reports that will help us replicate and fix if necessary.  

Summary

There are many reasons why a failure may not be immediately reproducible. That does not mean the failure or fault does not exist. 

  • Record the steps via test cases
  • Record (best you can) the impromptu or exploratory testing
  • Check the configuration of the product against intended or expected
  • Consider variables in the software associated with the feature
  • Consider there could be a timing element associated that is unpredictable (or a background feature timing / collision)
  • Log / report the failure in the fault tracking tool along with as much information as possible (configuration, test case, sequence prior, etc)

 

Configuration Management Book

Posted on: June 11th, 2015 by admin No Comments

 

New Configuration Management book

New Configuration Management book

 

Early Reviews

Just finished reading Configuration Management: Theory, Practice and Application, and I must say this is the most comprehensive document I have ever read on the subject. It addresses theory, practice, and application, with many real-world examples of what happens when the principles of product/configuration management are not followed!
Bill Dawson, SVP Product Management, Volvo Group

… the definitive guide to configuration management. No configuration manager should be caught without a copy of this book. … explores not only the technical aspects of the field, but the practical applications of configuration management in a wide variety of industries not just the technology field.
Joe Townsend, Configuration Manager, Indiana Public Retirement System

… captures the essence and significance of configuration management. It provides the framework, tools, and practices that will enable any enterprise to successfully create and manage a framework for CM in a mass customization world. … A much-needed book and discussion at a critical time in the age of mass customization!
Sundar Ananthasivan, Director of Global Engineering and Product Development—Couplings, Rexnord LLC

Finally a universal CM guidance which explains, by using practical examples, why configuration management is not only beneficial to the government area but adds tangible value across all market segments.
Dirk Wessel, Chief CM Section, NATO Communications and Information Agency

Great book, lots of information, easy read, and applicable …. Many of the concepts apply to the everyday world and not just the corporate or government worlds.
Steve Nissen, Software Quality Assurance, Ball Aerospace & Technologies Corporation

Well written and laid out to make it understandable, even to inexperienced CM practitioners … a recommended read.
Sunil Mavadia, CM Department Manager, Digital Globe, LLC

Configuration Management: Theory, Practice, and Application details a comprehensive approach to configuration management from a variety of product development perspectives, including embedded and IT. It provides authoritative advice on how to extend products for a variety of markets due to configuration options.

The book also describes the importance of configuration management to other parts of the organization. It supplies an overview of configuration management and its process elements to provide readers with a contextual understanding of the theory, practice, and application of CM.

Explaining what a configuration item is and what it implies, the book illustrates the interplay of configuration and data management with all enterprise resources during each phase of a product life cycle. It also demonstrates the interrelationship of CM to functional resources.

Shedding light on current practice, the book describes CM baselines, configuration identification, management baseline changes, and acceptance criteria for end products. It also considers testing, inspection and evaluation, related CM standards, and reference data. Coverage includes the product life cycle, the supporting enterprise infrastructure, functional resources, product management, CM elements, data types, and control requirements.

Providing a systems perspective of the various elements of configuration and data management, the book explains how they relate to the enterprise and details proven risk management solutions for when things go wrong.

Features of the book:

  • Explains what a configuration item is and what it implies
  • Supplies an overview of configuration management and its process elements
  • Describes configuration management baselines, configuration identification, management baseline changes, and acceptance criteria for end products
  • Presents risk management solutions for when things go wrong
  • Provides a systems perspective of the various elements of configuration and data management—explaining how they relate to the enterprise
  • Provides critical link between product testing, quality, and configuration management

Catalog Copy

This is the first book on configuration management that takes readers on a journey that traces the management theory behind the things we produce and consume that starts in antiquity and continues through the evolutionary development to their current implementation. Understanding this back story is critical to grasping the implications of decisions made in the first 60 days of any development program, as these decision impact 90 percent of the downstream developmental costs. The book is resplendent with original artwork and real-world examples of how configurations are managed across all market segments—uniting theory with practice and application.

Get a copy at the Value Transformation secured server book store by clicking this link.

Intention and Follow Through

Posted on: June 10th, 2015 by admin No Comments

Civil War

The Civil War

It was the United States Civil war, the battle was Gettysburg.  There was no doubt, General Meade had the intention to defeat Robert E. Lee.  In the end General Meade was victorious and his congratulatory order to the troops was also sent by telegraph to the War Department (and would be read by Abraham Lincoln). The crux of the message was the invaders had been driven from our soil.

Later, President Lincoln would meet with General Mead:

Do you know, General, what your attitude towards Lee after the battle of Gettysburg reminded me of?” Meade replied: “No, Mr. President – what is it?” Lincoln said: “I’ll be hanged if I could think of anything else but an old woman trying to shoo her geese across a creek[1].

Follow Through

In the President’s mind, Meade had left the job unfinished.  He had not pursued the rebels after the battle.  General Mead had good intentions, he performed well, but the fact was he did not follow through or in this case follow the rebel army and finish the job.

Intention and Follow Through

This same sort of thinking can permeate a project management organization as well.  In the best of circumstances the project manager and team revel in the immediate win and may not see there is much more to be made of the situation.  Under the worst conditions, the project flails and is unable to cobble anything that looks like a victory due lack of follow through.  This applies to more than just project management.

I meet plenty of people that want write a book, but lament finding he time or start and are unable to follow through.  These become the bane of any effort.  

Any endeavor is well advised to follow through. Good intentions or half done often will cause a project to flail or bring a project to bitter consequences and failure.  If it is important for project success, stubbornly, doggedly pursue to the desired end.



[1] Michael Burlingame, Abraham Lincoln: A Life, Volume II, p. 512.

Concurrent Engineering Problem Part 2

Posted on: June 5th, 2015 by admin No Comments

Concurrent engineering problem take many forms

From our last blog, we have learned that of an organization that has concurrent engineering difficulty, specifically coordinating the design work.  We will further explore this situation.  One of the subsystem groups decides to improve the coordination effort internal to that specific department. For example, System 1 chooses to focus on their handling of the design coordination internal to that department. This is not a bad idea, as there are some inefficiencies and losses due to ineffectual coordination of the various subsystem designs. 

 

Internal to department areas of improvement

Internal to department areas of improvement

 

There is another problem area, and that is the coordination of the systems that comprise the entirety of the product.  Below is a graphic that illustrates this other area lacking coordination.

Cross department areas of improvement.

Cross department areas of improvement.

 

Where to start?

The trick is to understand the best area upon which to focus the finite resources.  A short sample period, a portion of one large project, provides and example of the costs due to errant or insufficient control over development of the design.  Based upon these numbers, it could be the best situation is to work the interfaces between the systems rather than those issues internal to each system area.

Costs to the company for internal department and cross department due to coordination problem.

Costs to the company for internal department and cross department due to coordination problem.

Perhaps the reason for focusing on the internal was the result of spheres of control interests (political).  For example, System 1 (department) believes they have more complete control over the outcome and therefore the probability of success is greatly improved. If the larger problem were to be undertaken, that requires involvement in management structures and coordination from System 2, System 3 and so on.  There is some thing to be said for the probability of the endeavor being successful. An entirely internal affair is likely to have a significantly higher probability of success.  On the other hand, to show the largest improvement to the bottom line it is prudent to prioritize the work and solve the largest loss.   Treating a black eye while the patient hemorrhages profusely is not the likely road to success.

Consider the entire system and the losses. Even if probability of failure were factored in (multiply risk X money at stake) the outcome is so one sided that result would likely favor taking on the bigger challenge.  

 

 

 

Concurrent Engineering Problem

Posted on: June 4th, 2015 by admin No Comments

Once upon a time

There once was a company, with a systemic problem with concurrent engineering and change management.  This was a complex organization, with many functional areas.  Each functional area, had sub-function divisions.  This type of organizational structure is often referred to as a functional organization with the associated hierarchy.  These various functional areas design and produce sundry parts that will be assembled together to produce a complex electro-mechanical and embedded software system.  A partial drawing of the structure is provided below:

Organization Structure

Organization Structure

 

Concurrent Engineering Requires Coordination 

 Within the systems level, there are sub-system levels.  These subsystem levels are departments within the systems department. Each sub-system focuses on a unique aspect of the entire system.  There work has some commonality, for example system 1 could be the electrical / electronics engineering discipline.  Work within this department is coordinated to meet that of the entire system.  Generally speaking, these departments are often co-located, under those circumstances internal department communication can be easer.  

The concurrent engineering problem this organization has, is keeping the end target system (or developmental iteration of the) coordinated.  This is done to produce a functioning product with the system deliveries from the various departments, and is a function of configuration management, project management and change management.

This difficulty in coordinating the design has produced costly rework and late deliveries as the parts that do not work together as expected are then re-designed, reworked, released and built.  These problems occur in two areas.  We will write more on that in the next blog.

Risk Management Failures

Posted on: June 1st, 2015 by admin No Comments

Have you seen these risks in your projects?

  1. The project that selects a scope that does not match the constraints (cost, quality and delivery).
  2. The project strategy that dooms the organization to cost over runs, surprises the organization with late delivery that would have been easily predicted.
  3. The ineffectual risk register (or non-existing).
  4. The team that sees the incoming disasters possible but does not let you know until a risk becomes a certainty (then reports to you there is a risk).

All of these occurrences are common place, and should not be.  Companies loose money, customers and overly stress employees by placing ridiculous demands to turn the tide of the project when the problem could very easily been avoided or at the least remediated.  The first to leave are those that often your best talent.

Avoid these problems, learn early and adapt. This workshop will help you get there.  There is another offering on the weekend if that is easier or more preferable.   

Check them out! I look forward to seeing you there.

Subscribe to our Newsletter

Subscribe to our Newsletter
Name
Email *
Single Sign On provided by vBSSO