Testing of Complex and Embedded Systems

Independent Test and Verification

We are composing a glossary of testing terms and we take one from that work today and discuss here.   We welcome any and all that would like to contribute.

Today we discuss development and independent test and verification.  A development organization can be structured in many ways. The development and testing can be essentially one department under one management structure, or these two areas, development and testing can be separated each with a respective hierarchy.  Like most things, there are benefits and drawbacks for each approach.  The trick is to map the biggest benefit to your organization, or reduce the negative effects on your organization.  All of this means the selection for organizational structure is likely not best left to an arbitrary decision.

Communication

I am sure we see the obvious benefit of having the test personnel integrated with the development staff.  Those who have been in development for awhile no doubt understand the communications challenges that can come with separating groups when interaction between those groups is paramount for project success.  So, the benefit of these two disciplines is one less barrier to communication. However, there is more to it than that.  For example, consider a company that develops manufactures sells and supports the product after launch, like perhaps an automobile.  When communication between the development group and the test group takes on a more informal approach (verbal) we lose some of the ability to propagate the details of the product.  In my experience, the veracity of the documentation has a big impact on the depending activities of such an organization. For example, design documentation can serve as the core for service and troubleshooting of the customer’s product.  I have personally witnessed where an increased or sole reliance upon informal communication rather than technical documentation, meant the technical documentation was not promptly (or ever) updated to meet the actual product when informal communication is largely or solely the vehicle for this exchange. Those that get the formal documentation, are receiving outdated and errant information from which aftermarket product documentation and troubleshooting manuals may be derived.  In an instance like this, we exchange the improve quick interactions between the developers and the testing group, but it can come with consequences to those downstream.

Familiarity – not via Communication

Integration of the development and the testing groups comes with another potential downside.  We are all on the same team, we can suffer from group think. That is, we do not see the blind spots. We are on the same team, the level of critique from the testing group may suffer.  This relationship between the developers should be cordial, but not so familiar that the testing is soft on the product for fear the developers will take the feedback poorly.  I liken it to yin and yang or offense and defense in a sport (football).   I have seen the level of formalism descend to the occasional excel sheet tracking what is found and those depending activities – product documentation and service information will be built upon design documentation that has only a modicum of relation with the actual design incarnate.  The customer suffers and moreover, understanding the way our product works moves toward word of mouth, and therefore depends upon who you talk to for the answer.

Specialty Division

Creating both a development team and a testing team provides time, space and focus for specialization.  That is, the test team can develop their processes, tools and talents to meet the specific demands, and so too can the development group. We do not have development personnel burdened or distracted by the test activities within the group and vice versa for the test group.

Fault Reports and Escalation

Testing finds defects or non-conformance’s in the product.  The splitting of the development from the testing can put both disciplines on an equal playing field.  For example, consider the development manager that has test personnel also within the staff.  During a project, decisions must be made regarding the product suitability for launch.  The launch of the product is approaching, and testing finds some things pending evaluation but really has not concluded their work.  The manager will likely feel compelled to launch the product and there is no entity designated to defend an alternative viewpoint.  What do you think the probability of launching the product is, when the only person to make that decision is responsible for both the development and the testing in their hands?  With another hierarchical structure in place, there is a counter argument and discussion that can take place regarding any latent reliability and perhaps legal risks that may come our way.

Conclusion

There are many things to consider when structuring your organization.  The organization’s culture, the level of risk toleration based upon the product, company and industry and many other nuances.  It is not as easy as just keep the existing structure or “spin” off the testing from the development.