Archive for January, 2017

Analysis of Measurements

Posted on: January 27th, 2017 by admin No Comments

by: Shawn P. Quigley & Jon M Quigley

Measurements and Bias

Solely by the process of observing something we can alter the thing which is being observed. This is a known as the observer-expectancy effect.  This effect is born out of conscience and subconscious biases of the observer.  In the case of observing people, we have noted in earlier blog posts that the act of observation, taking an interest, may alter the outcome or performance as well (consider The Hawthorne study).  To make an effective measurement we must work to account for these impacts.  We also must know the goal we have set for collecting the information, that is the measurement is context based.  Having a specific goal, informs the type of data and methods of data collection.  Both of these are rife with opportunity for bias to creep into the measurements,  and delude our team.  This bias can creep in not only, what we identify as needed to measure, when the measurements are taken, but also in the interpretation of this information.  Our people who we are taking the measurements know what the measurement could mean for them; we again taint the validity of the information we are attempting to collect. Taking just these few issues into account it would seem that using functional measurements does not sound like a good idea.

Measurements Facilitate Understanding

Before we can take an effective measure of something to facilitate any form of change and/or improvement we must first full understand what the current processes, procedures, and personnel skills sets we are looking to improve.  This understanding where we are alone will likely require measurements.  This is the first step of any form of change management; knowing your actual starting point. Knowing from where one is starting is harder than most people would assume. Because preconceived notions of being better or just a lack of understanding of the work-around that are being employed to make the processes and procedures seem functional are rarely know by the people who desire a measurement to determine effective improvements.

It seems quite obvious as to why taking functional and operational measurements is necessary. It allows for trend forecasting, process improvement, and personal evaluation. When collecting information for trend analysis the process is mainly to record what is actually occurring. This differs from process improvement and personnel evaluation data collection in that some quantitative component must be involved to allow comparison.  It is this very quantitative component that drives the information collected and possibly the associated personnel behavior.

So Much Trouble – Measurements

This post would seem more to countermand any form of measuring: data collection, due to all the negatives brought forward. This is quite the contrary, knowing where a bias or gap in obtaining useful data for analysis is a must before starting on such an endeavor. Most processes or methods for gathering data for measuring something are costly and if employed improperly can lead to even more costly mistakes. This is the reason we start this segment on measures and analysis discussing these pitfalls.

Verification and Validation

Posted on: January 26th, 2017 by admin No Comments

Verification and Validation

The definition for verification and validation can be found at[1]:

Verification and Validation comparison

Verification and Validation comparison


We must express some disagreement with the activities associated with the individual areas. For example, testing is not limited to Validation. Testing is also a function of verification as we will use these techniques to understand if the instantiation of the product meets the specifications.  Testing will be part of the quality assurance activities for the product – are we building what we said we would – through iterations of the product.  Big bang is dead – meaning build everything at the beginning no longer happens. That method only works in products of the smallest scope.  Rather, we will increment the feature content, verify the product build matches requirements delivered, log bugs and get ready for the next round. The next round fixes bugs, adds more of the requirements to the product content verify and so forth.

Likewise for Validation, there are things other than testing.  For example, we will build models with which our proposed customers can interact to help refine the feature content and performance. We can build prototype parts and provide these to the customer as a means for evaluation.  This evaluation will provide us with insight to customer use that will then become requirements in the product specification.  This evaluation will either mimic or be the actual proposed use of the product. This will be scenario based (analogous to live fire exercises) and will help set the requirements if early enough in the development cycle.

Agile Practices Applied to Line Management – Monitor and Control

Posted on: January 25th, 2017 by admin No Comments

Having the plan is only partially helpful.  The list of test cases and the expected rate of accomplishment allows us to refine our estimates as we progress through the testing.  We will be in a position to provide the project manager and stakeholders with a better “ETA” (Estimated Time of Arrival) just like the GPS informs us as we progress toward our final destination.

We know must execute to see if the planned rate of accomplishment and expected conclusion date are viable.  We will the monitor and track the progress of the team.  Here is where we take another play from the agile playbook.  Each morning we have a short 15 minute meeting with the verification team members.  Filling a role comparable to a scrum master, we ask:

  1. What did you do yesterday?
  2. What do you have planned for today?
  3. What obstacles are you encountering?

Notes are taken especially when it comes to obstacles. We can ill afford to have the work stalled so when obstacles arise, we set about resolving them immediately.  In some instances, the work will be diverted to another set of test cases to be conducted while the obstacle is removed.  In that way we keep the rate of progress going.

At the end of the meeting I prompt all who have not updated the test case tracking sheet to do so.  Specifically, the updates are the number of test cases executed for a specific specification (see the first blog post of this series).

Comparison of Plan to Actual Performance

We now have our desired rate of accomplishment against our actual rate of accomplishment, but we are not finished yet.  This tells us little about the product other than our rate of testing. Check back for the next installment.


The Un-repeatable Fault

Posted on: January 18th, 2017 by admin No Comments

Testing of Complex and Embedded Systems

Testing of Complex and Embedded Systems

Testing Complex and Embedded Systems

What set of conditions could cause this event to occur?

When we have elicited all we can from the customer about fault information, it is time to proceed further in our analysis. This next step requires investigation of the design to understand how the symptom of failure described could happen by breaking down the hardware and software and the interactions within them to understand the improper behavior of the features to the customer. If the investigator is in the automotive, pharmacy, or food industries, they can resort to an immediate perusal of the Design Failure Mode and Effects Analysis (DFMEA) and the Process Failure Mode and Effects Analysis (PFMEA). If our investigator is lucky, they may find pointers to the cause of the issue in these documents.

To be successful, we need to perform a rigorous and systematic critique of the design—with enough follow up to ensure that any correctable issues have been resolved. Usually, this approach means that we trace the symptom–usually an output–until we discern potential causes. Note that this approach is very close to a logical fallacy called affirming the consequent, where we attempt to find a given antecedent (cause) for a specific consequent (effect). The reason this approach causes problems is the effect may derive from more than one cause. However, we are suggesting that we compile a list of candidate causes. These possibilities are prioritized for which is the most likely when we think have enough information to do so. Alternatively, we can use our candidate cause list and induce the observed failures in a controlled environment to test the theory of the root cause. Even with this testing, our conclusions remain vulnerable to error, since a demonstration of the failure is not necessarily a demonstration of the cause. One method to try and deal with this fallacy is used with electronic parts and has the following steps:

1. Reproduce the observed failure if possible (let’s assume success)
2. Hypothesize an electronic or mechanical cause for the failure
3. Open the unit and test the hypothesis
4. If the hypothesis appears to be correct, then repair the part
5. Attempt to reproduce the observed failure
6. If we fail to reproduce the failure, we can have some confidence that we did indeed discover the cause of the failure

Another potential solution is to have failed material sent back for analysis. However, we are limited as to what we can do with the failed material unless the failure is a hardware failure. If the product is part of a larger system; then removing the product from the system may remove the stimuli from the “failing” component. If the failure is a hard failure, then review of the failing part and the nature of the failure provide evidence to the source of the causing element.

If issue not seen the fault, assume you haven’t found the trigger

It is not time to give up or say “there is no problem”. Customers never want to hear their suppliers tell them it is all in their heads.  Time with the customer in the application analysis may be helpful.  Finding the scenario where the problem seems to be more common place and traveling to investigate the problem where it exists are options in determining the cause. Do not forget that some problems can be related to geography; in other words, we are talking temperature, humidity, rough roads, electromagnetic interference, and other environmental noise. We may even have to resort to a systematic replacement of components to find the guilty part, a task made even more difficult if the `part’ is actually software.

Pries, K. H., & Quigley, J. M. (2011). Testing complex and embedded systems. Boca Raton: CRC Press.  page 40-41

Product Life-Cycle – The Development Phase

Posted on: January 13th, 2017 by admin No Comments

In the development phase of the product development life cycle, we are generating ideas for the product. We see opportunities in the market place and wish to explore if we can capitalize (not a vulgar word) upon these opportunities.  Perhaps a new technology has become available to us.  Our organization will want to investigate the application of that technology toward existing businesses. Maybe it is a new market. Perhaps the organization is looking from an entirely new application perspective. At any rate this phase starts off by idea generation and rigorous critique of those ideas to guide the development to the product (or eliminate the idea of the product).

This critique is facilitated via prototyping and reviews of designs. Concurrently with the development of the product we will develop our manufacturing capabilities and critique that development work as well.  All of which, will guide our design or final product instantiation and manufacturing processes.

We will also need to get some information on the product from our would-be customers. These are the people who will make it possible to generate a business case that is beyond mythical and more tangible.  The business case will help us understand the design scope and specific incarnations to achieve our desired profitability margin.  This profit calculation will be based upon:

1           the volume of customers testing indicate we can expect

2           the cost to make and deliver the product (includes manufacturing and delivery)

3           the selling price of the product

The selling price is also derived from the volume calculation as there will be some volume implication due to the customer sale price (higher price can reduce number of customers).  We will garner as much data as we can via market testing to arrive at this optimum selling price.  Ultimately at the end of development we have a product that we will move on to the next step – introducing the product to the market place in earnest (as opposed to learning the desired particulars of the end product from prospective customers)

Maslow and the Learning Organization

Posted on: January 12th, 2017 by admin No Comments

By Shawn P. Quigley

Maslow and Motivation

In our previous discussions we have referred to Maslow’s Theory of Human Motivation (Hierarchy of Needs) and how this relates to work place motivation. To best continue our discussion we must first review some of the tenets of Maslow’s theory in more detail and dispel the misconception that Maslow set the hierarchy in the form of a triangle to convey that one need must be fully satisfied before another can become predominate or pre-potent as coined by Maslow in his 1943 paper published in the Psychological Review. (Maslow, 1943) Maslow discusses that if the individual feels a need more than another it will be the main driver and this can only be determined by that individual.

Relation of Needs

In his article Maslow states that there are at least five goals or basic needs: Physiological, Safety, Love, Esteem, and Self-actualization. (Maslow, 1943) However, he also states that these are only part of what determines the behavior of individuals. There are biological, cultural, and situational drivers as well. Maslow further stated that the order in which these basic needs are structured is determined by the individual, their experiences, and the level of prepotency assigned by the individual. The example he provided was that of an individual that has never felt hungry would not have that as a physiological motivator or a predominate need. And thus that basic need would not come into play for the individual in question.

 Application of Theory

At this point you are probably asking, “How this relates to work place motivation?” Using the basic needs as a guide we can see how being involved with an organization or group could provide a feeling of safety, self-esteem, and belonging; a lower grouping for both esteem and love. On the same hand it could detract from these needs if the environment provided is unsafe, degrading, and lacking a sense of unity or belonging. Many organizations strive to provide a safe work environment, but there is more to safety; as a basic need, than just not having a potentially hazardous environment. In fact there are numerous jobs that are not safe, yet the individual feels this need is met through its potency compared to other needs. Having provided is as an example we can surmise that the situation and/or environment plays a major role in what needs the individual sees as predominate and the only way for supervisory or management personnel to know this re-sequencing of the individual’s needs is through positive interaction.

Interpretation of Needs

Now we should discuss what positive interaction is and some common short comings of supervisors and management in this area. It is not uncommon for supervisors and managers to assume that they have determined the underlying motivators; needs, of their people through mere observation. To address this issue we will again look to something Maslow said, “The person who thinks he is hungry may actually be seeking more for comfort.” (Maslow, 1943) If an individual state that they are hungry and supervisor may assume that is their need the true point may be being missed. Probably the most misinterpreted signal is that of monetary gain because determining what need it actually equates to can be exceptionally difficult and it more than likely relates to several needs at the same time. However, only one need usually has predominance at any given time. The key in that sentence is “at a time” because the situation can and most likely does change which need takes led. An example of this would be if an individual is asking for more money, but their actual need is esteem it could possibly be satisfied through being offered a position as team leader or recognition for their abilities and contribution to the organization. Again the need of the individual asking for more money could be something totally different, such as safety. The only way to determine the actual need in any situation is through open and honest dialogue; the open Mental Model.

Lack of Dynamic Tension

As with most things if it were this simple everyone would be doing it, but there is more. If needs are constantly satisfied and the individual develops a sense of being owed this satisfaction or become disenchanted due to a lack of tension between actual and desired states they can no longer be used as motivators and when used produce only negative connotations for both the individual and the organization. There needs to be some form of tension between the actual and desired state; tension, for a need to take predominance. It is this predominance that makes the need hold value to the individual and its’ use as a motivator. This is not to say that organizations should not attempt to satisfied their employees’ needs, but to say that they must constantly show them how doing what is needed for the organization will assist them in maintaining their base needs and working on those higher order needs that can rarely be satisfied, such as self-actualization: Personal Mastery.


In summary we have reviewed Maslow’s Theory of Human Motivation (Theory of Needs), discussed how needs can be situational, what allows and need to be a motivator or a detractor, and related this discussion to two of the principles of the Learning Organization. It is the basic understanding of these principles and more that will enable an organization, project, and/or team to meet their goals while helping the individuals that make them up to obtain their needs and develop.


Maslow, A. H. (1943). A Theory of Human Motivation. Psychological Review(50), 370 – 396.

Subscribe to our Newsletter

Subscribe to our Newsletter
Email *
Single Sign On provided by vBSSO