Archive for May, 2016

Quality requires more than Fix Your Own Sandbox

Posted on: May 31st, 2016 by admin No Comments

Recent events have prompted us to preempt our CMMI requirements management series for this waste of company resources that we can only attribute to an overly politicized work environment and fear.  The downside of functional or siloed organizations is demonstrated in the sentiment “fix your own sandbox”.

Complications of the Organization

In general, the work of our organization is not getting less complicated.  For an organization to be operating at peak capability and delivering a quality product or service, may require numerous exchanges between a variety of departments (at least if you are functionally structured) and people.

Fix Your Own Sandbox (Quality)

I have heard, at least once from managers regarding quality and process improvement, that we should focus on our own part of the sandbox and fix only those things that are directly under our control.  That is all well and good, perhaps, when the quality concern or dysfunction is within our department only.  However, what if the problems originate or are more born out of the exchanges between departments or those problems are costing us more than our department internal issues?

Consider the department that has an internal problem that is costing them thousands of dollars. They also have a problem as a function of their interactions with other departments that are costing millions of dollars. Under the fix your own sandbox rule, we would neglect the hemorrhage of millions of dollars, opting to fix a problem that is costing the company only a few thousand dollars.  This is how companies go out of business.

Cost of Poor Quality

We may think we are doing the right thing in not recognizing and solving these imperfections that are costing our company BIG money, perhaps due to political sensitivities, but what we are really doing is wasting time, wasting money, destroying morale and eroding our company’s ability to effectively compete.

Quality and Long Term ability to Compete

If you want your company to be able to compete and thrive long term, then you must confront and solve the problems.  We use tools like Pareto, histograms, and scatter plots to and understand prioritize the work. We will also need a systemic and organizational approach, one that does not promote the “fix your own silo” approach.  The approach should be something like a Total Quality Management approach that considers the entire organization producing value for the customer, including the internal customers and culminating in the most important customer, the one that keeps your company thriving.  Call it whatever you want to, but ensure the functional barriers are eliminated and the organization has a continuous process improvement approach without fear.  It is almost always better to solve the multi-million dollar quality problem that is sucking the life out of your organization like a leech over solving a problem that is costing little.  It is an example of squinting at a gnat and swallowing and elephant. This approach is analogous to a doctor prioritizing a small scratch when the patient has an artery has been compromised and hemorrhaging.  This fear of taking on the challenge due to political discomfort does not reflect an approach that the leadership or management should endorse lest they watch the company disappear.

CMMI, Requirements and Traceability

Posted on: May 29th, 2016 by admin 1 Comment

We will continue our review of CMMI and requirements management practices.  As we have seen in the earlier posts, managing the requirements is necessary for efficient development and doing so has positive impacts on the project as well.  Specifically, the project benefits when the organization stands behind attaining the requirements, and is in for a rough road when this commitment to the requirements is in fact not there.  At the very least if the organization is not backing the project, the project manager will understand the risks this presents to the project.  The project also benefits from controlling the changes to the requirements.  We are going to learn as we do the work and that learning will be mechanisms for product updates. 

 Customer need and Trace-ability

In this post, we are discussing being able to trace the top level customer needs through the system and to the detailed level requirements for the product or system.  We reduce the turbulence the project encounters when we connect requirements to specific scope attainment.  In the end, when the project goes to close out contractual obligations, it will be necessary to trace what is actually delivered to this customer need.  However, even before that, this traceability will come in handy for the test and verification personnel as it will be easier to ascertain the severity of failures found during testing.  This is important for the severity part of the failure report and is a prioritization mechanism for what faults to address first and which ones can be delayed in resolution or perhaps acceptable.

The degree of difficulty we may have closing the project, will be influenced by the degree of diligence we have taken in building the requirements along with a way of tracing the connections from the top level scope, that is the customer needs, and the minutia of details that are commensurate with the product or system under development.  If you are the customer, you should expect to see this table or matrix.  Without this table, you are unsure if the project is actually in a position to be closed.  Experience also suggests that customers miss this need and close the contract without knowing what part of the work is actually delivered.


CMMI and Manage Changes to the Requirements

Posted on: May 27th, 2016 by admin No Comments

If commitment to the requirements is a significant source of failure, it is followed close behind by the management of changes or additional requirements that come from doing the work.  Though many project managers may believe that once a project is underway, there shall be no changes; that is a myopic approach.  Change happens.  As we conduct our project we will learn as we learn we will update the expectation and details of our product via specifications and requirements documentation.  Our project life-cycle layout will facilitate this incremental and iterative approach to the product which will include the requirements. For example, in the automotive world, we typically have a project life-cycle that produces a variety of prototypes that increase in sophistication as we make progress and approach the final production iteration of the product.  We may have a first part that may be largely a mock up from which we will learn, update the specifications and then move to the next level of refinement.  As we produce a part, test and otherwise explore the part, we will learn things that the next iteration of the product will need to have incorporated.  We will update drawings, specifications, and other documentation and update the revision level and release these for the next round of work.  All of which will fall under our configuration and change management for the project.

We do not just increment these changes into the product, via some email or telephone call. We will review the consequences and impact of these changes and go through some type of approval process. Once we understand the impact, we will either choose to incorporate the change (approve) or reject the change (disapprove).  The CMMI standard describes the situation in which our requirements are well under control.  According to the specific practices, the typical work products are[1]:

  1. Requirements status
  2. Requirements database
  3. Requirements decision database

With this approach we are able to trace the requirements and how these change over the course of the project and product development.  This can provide us with some indication as to the volatility of the requirements.  The ability to trace the requirements to scope and vice-versa provides us with a way to understand the consequences of a failure from testing. We are then able to trace what is found during testing all the way back to a specific part of the scope and make an informed decision on the severity of the impact, which leads us to the traceability portion of requirements management, coming up next.

[1] Chrissis, M. B., Konrad, M., & Shrum, S. (2003). CMMI: Guidelines for process integration and product improvement. Boston: Addison-Wesley. page 490

CMMI Obtain Commitment to the Requirements

Posted on: May 25th, 2016 by admin No Comments

Knowing and Committing to Requirements

Once we have identified and understood the requirements, we then must gain organizational support for delivering to meet the customer demand.  It does not matter if we are a supplier or taking on a project internal to our company, we will need people to stand behind the endeavor – commitment.

Commitment is more Difficult than it seems 

Experience suggests commitment is more difficult than it looks.  Organizations make the commitment often without really understanding what that means, or knowing the present level of commitment to which they are presently subjected.  The result is an over-taxing or more hours booked than available talent and resources.  Additionally, it is difficult to make the commitment for the individual, even when we assign that talent to the project. In other words, having the talent assigned to the project is not the same thing as having the talent and resources committed to the project.  In the model we can expect the typical work products to be[1]:

  1. Requirements impact assessment
  2. Documented commitments to requirements and requirements changes

 Intersection of Engineering and Project Management

Again, here is another touch point of engineering, product development, and project management. Part of the monitoring commitments falls to the project management defined areas of CMMI.  This process area is responsible for monitoring the commitments and compare to the project plan. This is done via metrics identified early in the project.  Metrics are not some subjective green smiley face that can be easily duped or spun and requires a certain amount of objectivity and diligence to understand the situation.  To ensure we are working according to plan, our project activities will include some specific reviews of progress and compare that to the commitments, taking the requisite actions when there is a gap between the intended and the actual.  In general, we will produce records of our commitment reviews such work items as[2]:

  1. Regularly review commitments both internal and external
  2. Identify commitments that have not been satisfied or are at significant risk of not being satisfied
  3. Document the results of the commitment review

Summary of Obtain Commitment and Requirements

We can see how these two subsystems; requirements management and project management intersect and interact.  We also can see that taking on the project requires we are willing to place the resources of the company, that is talent and equipment to bear on behalf of the project.  Without the backing of the organization, this project may consume valuable resources and produce nothing of benefit.  Experience suggests this happens more often than we care to admit.

[1] Chrissis, M. B., Konrad, M., & Shrum, S. (2003). CMMI: Guidelines for process integration and product improvement. Boston: Addison-Wesley. page 489

[2] Chrissis, M. B., Konrad, M., & Shrum, S. (2003). CMMI: Guidelines for process integration and product improvement. Boston: Addison-Wesley. page 395

CMMI and Understanding Requirements

Posted on: May 24th, 2016 by admin No Comments

CMMI and Understanding Requirements

We have recently been involved in a LinkedIn discussion about understanding requirements.  We have had several quick blog posts on requirements over the years.  For example, we have written about the connection of requirements and project management. We have also discussed how requirements grow over the course of the development of the product.  We have also reviewed the connection between requirements and testing of the product.  There is more to testing than to requirements, but at least a portion of the testing should compare the product against the requirements.  Those blog posts were just a quick demonstration to help show project managers how this process area and the specific organizations capability can help or hinder the project progress.  Not only hinder but put the project at risk altogether of being able to achieve the objectives of the project.

Disparaging Requirements is NOT the Road to Success

The LinkedIn conversation spoke of requirements in a disparaging way.  My experience of  generating and handling are not quite so negative.  Gathering and understanding the requirements can take time but it is time well spent. The alternative is to not take this time to develop an understanding which will lead to constant rewriting and reworking, important items missing and trivial items added to the requirements documentation.  It is often not possible to understand the entire scope of the product needs in the beginning.  That is why we prioritize those aspects of the product that are most prized by the customer.  To that end, we must know the significant players or mechanisms from which we will get the product information we need.  This is important for the initial exploration of the product  as well as throughout the product development and is a significant input for the project manager.  We will also need to know when we have a something that bears sufficient weight to be defined as a requirement.  In general, we will produce such work items as[1]:

  1. Lists of criteria for distinguishing appropriate providers
  2. Criteria for evaluation and acceptance of requirements
  3. Results of analysis against criteria
  4. An agreed to set of requirements



Requirements Understanding

At the end of this collecting, understanding, and vetting, we will have some agreed to set of requirements from which we can begin our work.  This is not the end of the requirements work that will be performed.  This is just the first sets of steps in our requirements work.  At no point in the CMMI documentation does it suggest that the requirements gathering and documentation a single point event.  As we document and build an instantiation of the product we can expect to learn about the product and our way of working that will influence the next iteration of the product.  We will then use what was learned to update the requirements documentation in a controlled way.  More on that in a future post.

[1] Chrissis, M. B., Konrad, M., & Shrum, S. (2003). CMMI: Guidelines for process integration and product improvement. Boston: Addison-Wesley. page 488

Project Management: Interacting Subsystems

Posted on: May 23rd, 2016 by admin No Comments

I have a mental exercise for you project management type people out there.  In this exercise, we are going to explore the possible interactions and results of the various knowledge management areas as defined by the Project Management Institute or PMI.  In this exercise, we will start with an Ishikawa or fishbone diagram.  In this exercise, we will label the diagram differently than the typical 5 or 6 M approach.  For this exercise, we will label the diagram with the 10 knowledge management areas[1]:

  1. Integration Management
  2. Scope Management
  3. Time Management
  4. Cost Management
  5. Quality Management
  6. Human Resource Management
  7. Communications Management
  8. Risk Management
  9. Procurement Management
  10. Stakeholder Management


Collection of Sub-Systems

Project Management a Collection of Sub-Systems


Now, visualize a project management failure, and put it at the head of the Ishikawa diagram as the symptom for which you want to uncover the reasons, for example, late delivery to the schedule as the symptom.  Now see how many of items you can generate, at least one from each knowledge management area that will perhaps give you the symptom of late schedule.  You will likely be surprised just how many things of the knowledge management areas can have things go wrong and produce a failed schedule.  Try this for other project failures and you will see that each of these are connected to one another. We are really working with a system that is a collection of subsystems and we are balancing the performance of each subsystem against the target or objective.

You will likely produce many items from the other knowledge management areas that will allow the project to be late to the schedule, even though the schedule is defined as being part of time management.  Poor scope management, for example, can have in impact on our ability to produce and effective schedule. Insufficient talent (human resources) may also have a similar impact on the schedule.

Try it, and let me know what you discover!


[1] A guide to the project management body of knowledge (PMBOK Guide). (2008). Newtown Square, PA: Project Management Institute.



Words Mean Things.

Posted on: May 21st, 2016 by admin No Comments

I recently taught a class in preparation for the Project Management Institute, Project Management Professional certification exam.  We were going through example questions and the verbiage was very specific and pointed to a specific answer from the four choices available.  When the answer was not what was selected there was an obvious level of disappointment from at least one of the participants regarding the words in the question and answers.  The words used in the questions were very specific and provided clues to the answers based upon the terminology as the basis for the certification.  Meaning, the words have very specific meaning, even outside of the project management context. However, to ensure we all know what we mean when we say things, a common dictionary can help.  The term “process group” should bring very specific things to the forefront of your mind, and this is different than the word process.  The reason for this specific language is  to facilitate communication and reduce the probability of a misunderstanding.  The choice of words is not just semantics.  This common lexicon helps in developing a shared understanding of the item under discussion.  This shared understanding of what is said is then able to critique or seek alternative perspectives that are bound in this common language.  These specific words may be a pain to learn this lexicon or the principles, but it is absolutely necessary to enable the building of the project organization.

DevOps and Continuous Delirium

Posted on: May 18th, 2016 by admin No Comments

This post was inspired by David Greer who presented us with the topic of devops and continuous delirium.

When it comes to devops, continuous delirium has two connotations as in wildly excited, and incoherent and bewildering.  When done appropriately, the development personnel and our customer will be wildly excited to be a part of such a wonderful and productive project.  When done poorly or inappropriately, our work resembles a sense of random and non-sequitur moments that leave the team and customer feeling like they have been thumped on the head.

Constant Builds and Poor Configuration Management

Constant builds of software with insufficient configuration and build management leaves everybody bewildered and wondering about the build and the constituent parts that actually comprise the build, not just what we suppose it to be.  Defects come, then are solved and in a subsequent build we find a defect very similar to the previous defect. Upon investigation, we will find that it is, in fact, the same defect. Cure this malady with some rational approach to the build and configuration management process. This can be establishing check-in and check-out procedures for software module as a tracking sheet that identifies the revision of the sub-modules for the entire software.  At the higher level, the compiled product iteration should have its own corresponding designator.  Ultimately, we need to be able to reliably go back to earlier revisions and understand the contents therein.

Who is Responsible for What?

Poor understanding of the work, we do not know who is doing what; neither do we know what constitutes successful delivery and pandemonium reigns.  Consider the example where a person is assigned a set of tasks or objectives and they work toward that end.  Person one’s output or deliverable goes to the next in the chain of development.  Upon delivery of the output, the receiver notes that many questions are unanswered. These questions require answering before the receiver can start their part of the work.  Invariably this will require rework.  Cure this malady by understanding what constitutes appropriate quality of the proposed delivery as well as periodically review the state of the objective with the receiving or depending entity.  There can even be metrics associated with the quality of the delivery that are periodically reviewed to understand if the work is on time and target (for example, the number of features documented, the total number of features, number features delivered, the number of test cases).  Do not wait until all of the work on the subject has been theoretically completed before reviewing.  Rather define the work and specific critical metrics and review as the development progresses.

Clean Compile is not Considered Testing

A clean compile daily is not to be considered testing, though some may have you believe that to be the case, neither should testing be relegated to the end of the development, this is another failure similar to the one above.  Often testing is treated as the final act of the delivery. On top of this, it is scheduled against the delivery as if there will be no defects found. I suppose if you do not look, you will not find defects, but that is not a productive solution. The problem comes when your customer uses the product. Then the defects are discovered and the ability to correct is much more complicated and costly. If the application is sufficiently critical, there may be legal ramifications.  Rather than relegate testing to the final stages, integrate the testing with the development.   Use numerous complimentary approaches such test to requirements, combination of stimulus testing, extreme testing and exploratory testingWhere testing to specification is concerned, invest in automation freeing the testing talent to perform exploratory testing on the product.  This testing will be outside the specifications but in the practical application of the product by the customer. Often, the things we believe the customer would never do are the very things they do and when they do, there can be consequences we would rather not the customer experience. Define key test metrics, for example, the number of defects found per iteration, the distribution of the severity of those defects is also a key metric.  Defects that are a minor blemish are not the same thing as a defect that can cause harm. An understanding the defect arrival rate and the severity of those defects make some level of prediction possible.  We would expect to see the number of severe defects (that is defects that can cause harm or monetary loss) to abate over time if our development and test work are successful.  This provides for the prediction of launch based upon the product quality.

Predicting the Future

Planning horizon is really the duration into the future we can reasonably and reliably understand, predict and control.  We delude ourselves when we plan projects out year or more into the future. The further into the future the less we are able to predict.  When we treat a plan as if we know everything associated with this project for the next 12 to 18 months we are suffering from delirium.  In fact, one study shows that 90% of successful projects were less than 12 months long and 47% were less than 6 months in duration[1].  Break the project down into smaller and incrementally approach the plan and the work.

Insufficient Customer Involvement

Customer (user) involvement is essential to meeting the customer target and in my opinion is a significant source of agile successes.  Getting the product or a facsimile (model, prototype or iteration) of the product in the customer’s hands for exploration as this provides the feedback loop to the developers.  Many may see this as a problem as this opens the door for change but that would be a very myopic way of thinking indeed.  Rather than being guided to what is required, we assume we (customer and developers) know all we need to know about the proposed product.  Rather than try to deliver one incarnation of the product to the customer at some future date, break the delivery down into iterations that will meet the prioritized customer expectation, refining and incrementing the feature content as progress on the product is made.  According to a study from The Standish Group, number one of the top 10 success factors was User Involvement[2].

Uncontrolled, Uncoordinated Changes

Product Change Management goes hand in hand with customer involvement. We can choose to not involve the customer, but this will be done at our own peril.  Waiting to give the customer the final product often uncovers questions that we should have been asking all along. Since we know to involve the customer, we also know we are going to require a consistent and capable way of managing the changes.  Change itself is not a problem within the development, unmanaged change is the problem.  There are ways to manage any costs associated with altering the product design over time. However, if our planning horizon and work duration are not excessively long, budgeting becomes easier to manage.  Ultimately a connection between the budget and the delivery must be known and a good mechanism for this is Earned Value Management techniques defined in conventional project management.

Systems Thinking

Collaboration (systems thinking) is required for all but the most remedial of software development work.  Streamlining delivery to the customer requires clear and quick communications channels throughout the devop chain.  Everybody from the customer, through the developer and testing personnel, should work like a well-oiled machine by free-flowing communication.  If our enterprise produces a manufactured product from operations, then this manufacturing group would also be required to participate in the development of the product.  In this way, we shorten the development cycle and obtain the most from our talent as we strive for clarity (rather than delirium) and constant improvement.


Save yourself from the bad case of continuous delirium.  Instead, pay close attention to the fundamentals and your customer will remain in a continuous state of delight, wildly excited by the growth and quality of the product.  The team will likewise be in such a state, as the frequently ridiculous requests that are required to save the project will be reduced greatly.


[1] Thomas, M. 2001. “IT Projects Sink or Swim.” British Computer Society Review.

[2] Johnson, Jim et al, 1998.  ChAOS: A Recipe for Success, 1998 Published Report, The Standish Group

Continuous Delivery

Posted on: May 13th, 2016 by admin No Comments

Continuous Deliver and Embedded Automotive

I have worked on projects that employed continuous delivery for embedded products. The embedded product was an automotive component.  The core of the software (the operating system) was specified using conventional approach. This operating system consisted of the maximum model requirements for this globally used component. The component looked and acted differently depending upon market segment and geographic location, for example, there were two different European incarnations and more than two for the markets of the United States.  This was not just some parameter setting or configuration differences, but whole scale differences in feature content and manner of execution.

As we indicated, the operating system, though we called it an execution engine, was specified and developed using an incremental approach. This incremental approach was based upon a prioritized set of features. That prioritization was based on what it would take to produce a vehicle that would operate safely allowing for testing of key features.  In the past, this component would have had the entirety of the software outsourced with the exception of parameter settings for each geographic / market segment. The desire was to make common use of hardware, and core software and allow for ease of adaptation based on the local application requirements.

 Continuous Delivery, Prioritization, and Specifications

The continuous delivery started with the specifications for each unique incarnation of the product. These were feature specification for the product and there were more than 150 of these unique features and each feature had a number of variations.  The writing of these unique feature specifications was likewise prioritized based on the most important features to get a vehicle moving safely, and synchronized with the supplier operating system delivery.  This team consisted of 3 members; vehicle system experts and component experts. This team wrote specifications and reviewed with the system portion of the team in small iterations. For example, the top 5 feature specifications that supported the objective of a vehicle that was safe to drive.  Once written, these functional specifications would be reviewed with the systems engineering portion of the team, and upon passing the review, these feature requirements documents would be given a version number and sent to the second team that performed the development and testing of the product (more on that later).  There was a continuous delivery of feature specifications, a few at a time, to the developers.


Continuous Delivery

Continuous Delivery


Continuous Delivery and Execution

The second team consisted of five people. Those were two developers for two different product versions, as well test and verification person for the products, as well as the person who was writing the specifications.  In fact, this person led the team using an agile approach.  The difference between these two products was not parameters only.  As the project progresses, the specifications and features of the two products would diverge. However, in the early phases, some of the specifications were common or described parameter setting and performance differences.  Since the team lead (scrum master) for this project was intimate with the development of the specifications, they were in a position to answer questions regarding the development.  This input, as well as that from the testing, are then fed back to the specifications for updates as the iterations progress.

After testing, and providing there were no safety ramifications, the iteration of software was provided to people within the organization that represented the customer and drove the vehicles to get constructive input to further guide and refine the development.

Continuous Delivery Summary

There were quick writing of the feature specifications just in time for the developers to code that part of the system. The developers would deliver iterations of the product that contained the latest specification content to the test personnel. The test personnel would test the product against the specifications (among other things) and review the results with team members to ascertain if what was found was a failure in the product or a defect in the specification.  The defect would then be reported assigning the appropriate area for the corrective action (update the specification or the software or even both when necessary).

Continuous delivery requires prioritizing the most important and then bringing a laser-like focus to those things.  We will continuously do the upstream work (specifications) and we will continuously build, test, and update the product.  This produces a product that can be demonstrated from which we can learn and take appropriate actions to bring the design to fruition effectively and efficiently.  Besides that, we are able to gather data on our execution and will be in a better position to predict the outcome of the work.

Learning and M&M?

Posted on: May 10th, 2016 by admin No Comments

Learning and Morbidity & Mortality

I have been watching a hospital type show.  That show demonstrated something called a Morbidity and Mortality (M&M) Conference and it occurred to me that some adaptation of this approach would help organizations to bring the learning from the work to the entire company.

Learning and Conventional Projects

We have written considerable on the learning organization and how to distribute what is learned throughout the organization.  Writing down lessons learned in books, as conventional project management describes, is only so successful.  Recovery of those lesson gems can be difficult requiring somebody to read a multitude of lessons that do not apply before stumbling on that piece of jade or diamond that may help.  We can make this easier if we have digitized our lessons learned book.  However, we really still have the same problem even if we make use of metadata and tags to help the search.

Learning and Agile

The agile approach to lessons learned is a bit more intimate.  Rather than documenting our learning into a book, we opt to work through the learning together in the retrospective.  This allows all team members to contribute, discover and learn from the variety of perspectives and bits of information and knowledge within the individual team members.  Part of the problem is the amount of time spent using this approach, as in rather little, and some lessons are not so quickly learned.  Sometimes we have to determine what to measure, how to measure then analyze the results.  This can take more time than that allocated during retrospectives.  Additionally, what is learned really goes no farther than the team, scrum master, and perhaps the product owner. This does not distribute the lessons learned throughout the organization.

What we can learn from M&M

Now we come back to the medical approach of Morbidity and Mortality Conference.  In this event, as described in the link, includes more than just the immediate team, but also is not focused entirely on researching the problem and propagation via written work.  We have the entire team, and it is often presented not by the most senior personnel.  This gives the newer members of the team to present the material and field questions.  Doesn’t this quote from the article demonstrate what we really want from our organization?[1]

The idea is that the discussion informs the whole department, and everyone does not have to make the same mistake. In theory, the complication might be averted the next time.


We think perhaps the best solution is not to short cut the time with people by relying entirely on the written word that is cumbersome to search.  We also submit, improving one agile team is interesting, but the entire organization learning from these events, now that represents a larger scale of improvement.  We also believe that more time really understanding the failure is often required than what is often allocated in agile

[1] Available at: Accessed May 4, 2016.

Subscribe to our Newsletter

Subscribe to our Newsletter
Email *
Single Sign On provided by vBSSO