Testing, Trade-off and the Emissions Tango

The Emissions Tango

There is so much to learn from this case for those who develop products for a living and automotive products in particular.  Understanding the impact of concept selection and testing of the product on the project success and product quality is important.  The early decisions we make regarding the development of the product will stay with us long after launch or we may find ourselves reworking the entire product.

Trade – offs

Most every design solution requires trade-off considerations.  For example, we sacrifice weight for reliability; or we may increase the cost to obtain a longer battery life.  Making this trade-off decision requires forethought and systems level thinking regarding the consequences and risks to product development and production.  In the case of the VW diesel situation, the trade off was NOx and CO2. [1]

The starting point of the diesel matter was, in hindsight, the strategic decision by Volkswagen in 2005 to start a major diesel campaign in the United States and to facilitate a breakthrough for this technology which at the time was already very popular in Europe. For this purpose, the Company decided to develop a new diesel powertrain unit with the EA 189 type diesel engine that features high performance and cost-efficient production.

The U.S. emissions limits for emissions of pollutants are strict. Under the strictest standard in the U.S. at the time, only 31 mg/km of nitrogen oxides (NOx) were allowed to be emitted, about six times less than under the EU5 standard applicable in Europe at that time. When designing modern diesel engines, technicians and engineers face the challenge that any measure to reduce nitrogen oxides has a knock-on effect with regard to other parameters (e.g. CO2).

The trade must be characterized and understood before making final decisions regarding the product.  This is often done theoretically and through the use of design of experiments (DOE). This data is used to generate limit curves and trade-off curves that demonstrate the relationship between the two variables.  Once we understand this relationship perfectly, we can make informed decisions as to the proposed design solutions ability to meet the demands

Bench Testing

How many times have products worked on the bench only to be a disappointment in the field?  Testing the product on the bench should happen early allowing the design team to learn about the product.  However, the bench is not the real world and hanging our hat on the bench as the definitive answer regarding the product’s suitability for the field is dangerous.  To get the most out of bench testing, there must be some correlation to the real world application.  The closer the bench test replicates the real world, the better.

In the ensuing period, in order to resolve this conflicting objective satisfactorily within the timeframe and budget of the EA189 project, according to the current state of knowledge, a group of persons – whose identity is still being determined – at levels below the Group’s Management Board in the powertrain development division, decided to modify the engine management software. With this software modification, emissions values were generated in bench testing that differed substantially from those under real driving conditions.

Bench testing informs the development staff if the design is close to workable, and not the final proof that the development work is out of the woods.  Most automotive manufacturers will build versions of the vehicles and log miles on them to understand the true performance in the real world.  In the case of heavy vehicle manufacturers, there are often customers with a strong relationship with the brand that get the product early to get some feedback from the customer and log miles of actual work to understand the suitability and reliability of the product. In the case of emissions, that would not work as emissions are not the type of feature the customer would likely notice.  However, it is possible to recover these vehicles after some driving time to understand the performance after the fact.

Hindsight and Defeat

At the end, the explanation for the technical causes for the irregularities, from the perspective of the Management Board member’s, amounts to a defeat of the emissions system according to the article.

At the end of August 2015, Volkswagen technicians gave a full explanation of the technical causes for the irregularities discovered regarding the emission of nitrogen oxides in the U.S. to lawyers from the Volkswagen Legal Department as well as to the U.S. attorneys from Kirkland & Ellis. These detailed explanations led to the Management Board member’s realization that the modification of the engine management software constituted a prohibited defeat device under U.S. law. It was then decided to communicate this information transparently to CARB and EPA. This occurred during a meeting with the U.S. authorities on 3 September 2015. Mr. Winterkorn was informed accordingly in a note dated 4 September 2015.

Let’s take a moment to revisit the bench testing.  If the altered software constitutes a defeat of the emissions system, how was it possible for the system to pass bench testing?  It could be that the bench testing did not sample or include the variables that are associated with emissions.  There are likely other explanations, but it should be clear that the testing is insufficient or the results were manipulated or interpreted errantly.

Conclusion

Product development can be very complicated and complex.  To hit the target requires selecting the optimum strategy to achieve the results desired.  That is matching money, benefit, scope and time for the project. Selecting what appears to be the easiest or cheapest solution may, in fact, be more costly than planned when you add the quality and non-conformance costs to the product.

I have seen companies select what amounts to the riskiest strategy given the constraints for the project. This is especially true for projects that have fixed dates such as vehicle emission projects.  In addition to selecting the best concept, there must be a constant sampling of the proposed design solution, comparing that performance against the performance expected from the final product. Lastly, the product testing must have some basis in fact or real situations.  In this case, the discrepancy between the bench test and the testing conducted by other entities that demonstrated non-compliance.  This discrepancy does mean the testers cheated or there was some manipulation of the system, but it does mean it is time to explore how the company tests the product and the test department’s approach to quality.  Testing is often the last opportunity for discovering these types of issues.

Post by admin