Part 1: Beware of System Diagnostics Side-Effects

With the increasing emphasis on utilizing existing technology as much as possible, we have recently seen a shift towards the use of Commercial Off The Shelf (COTS) products. In fact, virtually all DoD contracts today call for COTS components as the first choice for design wherever practical and to the extent that they meet specified requirements. The objective is, of course, to make development and production more affordable-a goal that, at first glance, would appear to be unequivocally admirable. Unfortunately, prescribing this “pill” could do you more harm than good if it was not expressly intended for system use. Quick solutions to lowering development costs may, from an ownership (customer) standpoint, result in even more costly support scenarios. Our goal is to shed some light on this hasty and often unnecessary practice and to help reduce the total costs of development, production and support.

System performance, as defined in DoD Directive 5000.2, “Defense Acquisition Program Procedures” is composed of not only operational characteristics but support characteristics as well. DoD 5000.2 (Para. C.. DEFINITIONS, 7. Performance.) further states that “The support characteristics of the system include both supportability aspects of the design and the support elements necessary for system operation.” Often, this co-performance support angle is overlooked in favor of immediate cost-cutting measures that can be more quickly realized during the operational portion of the design equation. In today’s competitive environment, it is too often a consideration to cut design costs at the expense of diagnostic effectiveness; even though the out-briefing of the “Desert Storm” conflict recognized that more and better diagnostics are essential to success in the modern battlefield.

COTS components, more often than not, lack the necessary diagnostic hooks. LSI components often lack the fault coverage required to satisfy the high percentage fault detection or isolation requirements demanded by modern systemsætypically in the realm of 95% detection of Mission Critical failures. Let’s not forget that the use of COTS implies, by definition, a bottom-up methodology of designing. It may successfully “fit the bill” operationally, but could easily fall short in the system diagnostics and support arena. The areas in which the benefits of COTS are most often realized are those efforts which are price-conscious, short-term and multi-application. This is typically the case when a simple generic alternative is sought. What must never be overlooked, however, is that the systems integrator is ultimately accountable for any support deficiencies-as well as a host of other costly related liabilities-and taking the wrong pill could result in disastrous side-effects.

The use of COTS produces becomes a far more viable alternative when it is incorporated into a top-down systems diagnostics development process. The primary objective should be to determine all the necessary functional allocations and the access that would be required for effective diagnostics before the system is synthesized into the hardware design. Ideally, one would like to know complete system and environmental diagnostic information before the functionality of the system is validated (frozen). This can only be achieved when the system diagnostic requirements and overall diagnostic strategy provide a basis for determining what types of diagnostic hooks are needed from the COTS components. Implementing a top-down diagnostic/operational co-development process that is targeted towards system diagnostics can yield a system which is truly a support jewel.

Part 2: Defining the Tools that Meet the Challenge

Proper integration of COTS components within the overall system diagnostic strategy must be proven effective and simple. The effort of integration must take into consideration the many hierarchical elements of the system and must use a diagnostic strategy that seamlessly crosses such hierarchies and disciplines while employing a variety of diagnostic philosophies.

Proper integration requires a tool of a different breed in order to tackle such a wide-ranging task. It must be easy to operate for many different levels of personnel within a company, and should be able to integrate similar data from other companies (COTS suppliers for one). It must also have a proven diagnostic engine capable of achieving unambiguous isolation at any analytical level and in any diagnostic scenario. This tool must therefore, at a minimum, offer the following:

Easy-to-use CAD/CAE interface

This provides the engineers with an understandable interface for developing/importing the schematics required within their department.

System-Level Drawing Capability

Before any analysis should be done at lower levels, there must be a well-defined system concept which creates the requirements for the lower level diagnostics. Also, program and project managers often need a picture to be able to convey the current state of the development effort.

Cross-Platform Compatibility

By storing all data in a cross-platform compatible format, it may be utilized and distributed between companies with mixed PC and UNIX workstations. Without this capability, certain COTS suppliers may not be able to produce data in a format which is useable by the integrator.

Proven Diagnostics

There must be a proven diagnostic engine capable of analysis at any level of the design, and at any time so that the tool itself does not create waiting periods. Engineers must be able to receive immediate and continual feedback about the requirements to date.

Links to other software tools

Without a mechanism to load/save data to and from other tools, much of a company’s investment might be locked up in other software. Capabilities such as support for the EDIF format, the ability to retrieve schematic symbols from a vector file, and the ability to import spreadsheet data from an ASCII file greatly reduce the chances of the data flow being bottled up in another tool.

With all these features in mind, effective usage of the tool requires that it play a dynamic role throughout the life of the product. Initially, the diagnostic characteristics of the system must depict choices being made with COTS components before the functionality of the design is frozen. It must also be capable of recognizing the diagnostic effectiveness currently achieved, along with the results of possible alternatives.

It is also too often forgotten that, being ultimately and intimately involved with the actual testing of a design, the Test Engineer can make a positive impact by being included in this process rather than relegated to a tail-end effort. For instance, certain test points might be useless due to the type of tests they require. Furthermore, a Test Engineer can often, with the use of the right tool, combine Failure Mode testing (in which diagnostics are based upon specific failure symptoms) with Functional testing (in which diagnostics asses the correct functionality of a device) to substantially reduce the overall costs of meeting the requirements.

Part 3: Integration using eXpress and STAT

In the previous parts 1 and 2, we have discussed the problems of COTS integration and the type of tools that could ease the process. We must have a tool which works from the top down, or we suffer constant delays from waiting for low-level design data-which even once we have it, the system diagnostics will be constrained by the limited scope of the low-level analysis. That is, a bottom-up approach often impedes the communication between system integrator and COTS supplier that is needed to achieve system-level performance in line with the expected or even warranted low-level diagnostics.

To tackle the top-down requirements, eXpress maintains a dynamic link between different levels of designs. This allows two things to happen. First, it allows the system to be analyzed as changes are made during the development progresses; and second, it allows low-level design changes to be monitored at the system level. By providing both these capabilities, eXpress allows requirements to be easily passed down from the system level, and the changes they effect can be immediately seen in their system perspective.

Another main concern precluding acceptance of any tool is that it be able to produce understandable results by both engineers and management alike. To achieve this, eXpress provides several options. First of all, indicators that represent low-level signal flow may be turned off to reduce “visual clutter,” thus increasing the readability of content-heavy, system-level diagrams. Secondly, eXpress provides a vast number of symbols as well as an import capability so that the elements within a design do not simply appear as boxes, but rather indicate the function performed. Finally, all elements within eXpress are fully reusable to reduce development time and provide engineers with a set of components previously created and validated to eliminate re-entry errors.

To achieve true integration with data supplied by COTS suppliers (or any other subcontracted companies or vendors), eXpress uses a cross-platform/binary-compatible data structure for all of its files. This allows different companies, divisions or work groups to provide Part Libraries, Symbol Libraries, and even complete designs regardless of the hardware used. In fact, in a fully networked environment, SUN UNIX, HP UNIX and PC-based workstations may all access the same files without extra conversion. Existing data from previous efforts residing in other CAD/CAE tools can also be translated through the use of eXpress’s import feature which can handle numerous formats, including EDIF-a netlist standard common to most electronic CAD/CAE tools. Also, a DXF import function allows vector-based symbols to be imported from tools such as AutoCAD®, CorelDraw, and Visio.

Finally, what is probably the most challenging aspect of a successful COTS integration is the ability to co-develop the diagnostic strategy along with the product design. eXpress and STAT together provide the ability to perform analysis at any level, whether it be the system, subsystem, or a combination thereof. By reducing STAT’s job through extracting only the pertinent information to a given analysis, eXpress also reduces analysis timeætime better spent determining how additional tests can be implemented and whether the COTS hardware can provide it. In fact, to help the Test Engineer determine the most effective use of information available at a given Test Point, eXpress provides the unique capability of being able to mix functional mode and failure mode test information.

By utilizing eXpress and STAT in a top-down environment in which diagnostic and operational features are co-developed, the systems integrator can safely choose to incorporate COTS components without unknowingly assuming hidden risks, costs and liabilities. eXpress and STAT perform their best in making the COTS pill safe to swallow.