[ Pobierz całość w formacie PDF ]
.Back now to the other side of the problem.That is that the provider pro-duced exactly what was supposed to be produced and the problem is yours.Once again, you have several options.First, you can change the specification inthe subcontract or the Work Order and have the job redone.Second, you canfind another source (if this is a purchased rather than a developed product),buy that part, and continue.A quick note here you will likely be responsiblefor a thing called liquidated damages (see glossary) if you choose this route.In other words, you ordered 5,000 widgets against your incorrect requirementand caused a vendor to gear up for that production.You only bought onehundred, so the vendor has a bunch of these things (or at least the cost) alreadybuilt.You are likely to be hit for the costs to make the vendor whole again.Is there any wonder it is so necessary to make absolutely sure the requirementsare traceable and correct?59 UNIT TEST59a (NO) Each Unit1 Test does not correctly reflect the requirement.Each unit test does not correctly reflect the requirement when each elementof the unit test is not directly traceable to each element of the unit requirement(Specification). 142 BLUEPRI NT FOR PROJECT RECOVERYRECOVERYEach testable unit should have its own requirement (Specification).This istrue whether the unit will be built internally or by subcontract.The subcontractwill contain procurement data in excess of the requirement (Specification) butwill otherwise be the same.The requirement (Specification) should be the basis of a Test Plan for theunit.The Unit Test Plan must define the:R' Test performance figures, standards, etc., that the unit must meetR' Conditions under which the test will be runR' Support equipmentR' Test equipmentR' Conditions for acceptance of the testR' Process for documenting and resolving test discrepanciesR' The details and step-by-step process of the test itselfTest Plans vary widely in their composition and content, but the definitionsabove are consistent throughout all plans.Additional Resources:Hardware Plans Software PlansMIL-STD-1519/1 ISO/IEC 12207MIL-STD-2076 MIL-STD-498MIL-T-18303B MIL-STD-2165IEEE-STD 416 DI-ATTS-80002MIL-STD-483 DOD-STD-2168MIL-STD-499DI-ATTS-80005DI-ATTS-81270DI-ATTS-81273DI-NDTI-81284DI-NDTI-81307DI-SDMP-81475(continues) RECOVERI NG FROM TECHNI CAL PROBLEMS 143DI-ATTS-80002DI-ATTS-80005DI-TMSS-80007DI-QCIC-8020459b (NO) Each design element that applies to the routine/module/subsystem does not have its own test case.Each design element that applies to the routine/module/subsystem does nothave its own test case when there is no direct correlation between the require-ment and the elements tested in the unit, subsystem, or system test.RECOVERYReturn, once again, to the Requirements Traceability Matrix (RTM) and fol-low the columns to the right.There should be entries in the testing columnsshowing where the requirement is tested and proved.If this does not ring trueor if you do not have an RTM, this is the time to build one.Refer to Attachment7, Requirements Traceability Matrix.In general, your RTM should look similar to Table 5-13.Once the RTM is complete and the test reference data is appropriately en-tered, it should expose the holes in the test string.T a b l e 5 - 1 3 R e q u i r e me n t s T r a c e a b i l i t y Ma t r i x ( R T M)Unit SystemSOW/ WBS S/C SOW/ Test TestSpec Para Requirement Number Spec Para Number Para MonitorSOW4.3.1 Security 06-03-02 N/A T-0304 4.4.1 SmithSpec3.2.1 System weight 02-04-03 3.4.6 T-0045 3.4.1 Jonesshall be lessthan 10,000pounds 144 BLUEPRI NT FOR PROJECT RECOVERY59c (NO) Unit Test findings were not reviewed for completeness andnot forwarded to be incorporated into Subsystem Tests andthe System Test.If this is the condition with which you are faced at the subsystem or systemlevel, you have a lot of work to do.The initial requirements must be provedsomewhere in the test process.RECOVERYThe quick and dirty method of recovery is to lay out the initial requirementsas stipulated in the specification and then identify where the requirement wasproved in the System Test.If you can prove that the requirements were incorpo-rated and your customer accepts the process, you are in luck.If not, you will berelegated to the complete documentation method.The complete documentation method requires starting with the Require-ments Traceability Matrix (RTM) and flowing each requirement into the unitand subsystem that will be built as a part of the system.The requirements mustfirst be flowed down from the requirements document (contract) to the speci-fication for each unit and subsystem and then flowed up in the Unit Test Plan,the Subsystem Test Plan and the System Test Plan.This methodology is espe-cially critical in the development of a secure system where security qualificationmust be proved and certified at every level of development.That philosophyreally should ensure whether the system is secure or not; then, there is no ques-tion.If you do not have an RTM, refer to Cause Description 51a All CriticalSuccess Factors (CSFs) such as MTTR, MTBF, etc., have been documented andunderstood and Attachment 7 for suggestions for how to develop your RTM.If you do not have a Unit Test Plan (UTP), refer to Cause Description 59aEach Unit Test correctly reflects the requirement for suggestions for how todevelop your UTP.If you do not have a System Test Plan (STP) refer to Cause Description 60c(NO) The System Test has not tested all elements of the system concurrently for suggestions for how to develop your STP.59d (NO) All Problem Test Reports (PTRs) were not captured,dispositioned, or worked off.All (PTRs) were not captured, dispositioned, or worked off when there is notcomplete accountability for every error that occurred during test conduct. RECOVERI NG FROM TECHNI CAL PROBLEMS 145Those errors were not assigned to responsible individuals for correction andthe results were not worked into the system.The System Test as written wassubsequently not run without error.RECOVERYEvery test run should have PTR forms available to capture any anomaliesthat occur during the conduct of the test.Further, there must be a PTR log inwhich to record the PTRs and account for each and every one.Usually, a se-quence number is assigned to a PTR preceded by a unique Alpha that relates tothe test.For example, the first PTR for the System Test could be numbered asST-001.If you do not have a PTR system, consider using the information to create aform for your own use:R' PTR No.R' PriorityR' SystemR' SubsystemR' Test ConductorR' Test Title Run No [ Pobierz całość w formacie PDF ]
zanotowane.pl doc.pisz.pl pdf.pisz.pl centka.pev.pl
.Back now to the other side of the problem.That is that the provider pro-duced exactly what was supposed to be produced and the problem is yours.Once again, you have several options.First, you can change the specification inthe subcontract or the Work Order and have the job redone.Second, you canfind another source (if this is a purchased rather than a developed product),buy that part, and continue.A quick note here you will likely be responsiblefor a thing called liquidated damages (see glossary) if you choose this route.In other words, you ordered 5,000 widgets against your incorrect requirementand caused a vendor to gear up for that production.You only bought onehundred, so the vendor has a bunch of these things (or at least the cost) alreadybuilt.You are likely to be hit for the costs to make the vendor whole again.Is there any wonder it is so necessary to make absolutely sure the requirementsare traceable and correct?59 UNIT TEST59a (NO) Each Unit1 Test does not correctly reflect the requirement.Each unit test does not correctly reflect the requirement when each elementof the unit test is not directly traceable to each element of the unit requirement(Specification). 142 BLUEPRI NT FOR PROJECT RECOVERYRECOVERYEach testable unit should have its own requirement (Specification).This istrue whether the unit will be built internally or by subcontract.The subcontractwill contain procurement data in excess of the requirement (Specification) butwill otherwise be the same.The requirement (Specification) should be the basis of a Test Plan for theunit.The Unit Test Plan must define the:R' Test performance figures, standards, etc., that the unit must meetR' Conditions under which the test will be runR' Support equipmentR' Test equipmentR' Conditions for acceptance of the testR' Process for documenting and resolving test discrepanciesR' The details and step-by-step process of the test itselfTest Plans vary widely in their composition and content, but the definitionsabove are consistent throughout all plans.Additional Resources:Hardware Plans Software PlansMIL-STD-1519/1 ISO/IEC 12207MIL-STD-2076 MIL-STD-498MIL-T-18303B MIL-STD-2165IEEE-STD 416 DI-ATTS-80002MIL-STD-483 DOD-STD-2168MIL-STD-499DI-ATTS-80005DI-ATTS-81270DI-ATTS-81273DI-NDTI-81284DI-NDTI-81307DI-SDMP-81475(continues) RECOVERI NG FROM TECHNI CAL PROBLEMS 143DI-ATTS-80002DI-ATTS-80005DI-TMSS-80007DI-QCIC-8020459b (NO) Each design element that applies to the routine/module/subsystem does not have its own test case.Each design element that applies to the routine/module/subsystem does nothave its own test case when there is no direct correlation between the require-ment and the elements tested in the unit, subsystem, or system test.RECOVERYReturn, once again, to the Requirements Traceability Matrix (RTM) and fol-low the columns to the right.There should be entries in the testing columnsshowing where the requirement is tested and proved.If this does not ring trueor if you do not have an RTM, this is the time to build one.Refer to Attachment7, Requirements Traceability Matrix.In general, your RTM should look similar to Table 5-13.Once the RTM is complete and the test reference data is appropriately en-tered, it should expose the holes in the test string.T a b l e 5 - 1 3 R e q u i r e me n t s T r a c e a b i l i t y Ma t r i x ( R T M)Unit SystemSOW/ WBS S/C SOW/ Test TestSpec Para Requirement Number Spec Para Number Para MonitorSOW4.3.1 Security 06-03-02 N/A T-0304 4.4.1 SmithSpec3.2.1 System weight 02-04-03 3.4.6 T-0045 3.4.1 Jonesshall be lessthan 10,000pounds 144 BLUEPRI NT FOR PROJECT RECOVERY59c (NO) Unit Test findings were not reviewed for completeness andnot forwarded to be incorporated into Subsystem Tests andthe System Test.If this is the condition with which you are faced at the subsystem or systemlevel, you have a lot of work to do.The initial requirements must be provedsomewhere in the test process.RECOVERYThe quick and dirty method of recovery is to lay out the initial requirementsas stipulated in the specification and then identify where the requirement wasproved in the System Test.If you can prove that the requirements were incorpo-rated and your customer accepts the process, you are in luck.If not, you will berelegated to the complete documentation method.The complete documentation method requires starting with the Require-ments Traceability Matrix (RTM) and flowing each requirement into the unitand subsystem that will be built as a part of the system.The requirements mustfirst be flowed down from the requirements document (contract) to the speci-fication for each unit and subsystem and then flowed up in the Unit Test Plan,the Subsystem Test Plan and the System Test Plan.This methodology is espe-cially critical in the development of a secure system where security qualificationmust be proved and certified at every level of development.That philosophyreally should ensure whether the system is secure or not; then, there is no ques-tion.If you do not have an RTM, refer to Cause Description 51a All CriticalSuccess Factors (CSFs) such as MTTR, MTBF, etc., have been documented andunderstood and Attachment 7 for suggestions for how to develop your RTM.If you do not have a Unit Test Plan (UTP), refer to Cause Description 59aEach Unit Test correctly reflects the requirement for suggestions for how todevelop your UTP.If you do not have a System Test Plan (STP) refer to Cause Description 60c(NO) The System Test has not tested all elements of the system concurrently for suggestions for how to develop your STP.59d (NO) All Problem Test Reports (PTRs) were not captured,dispositioned, or worked off.All (PTRs) were not captured, dispositioned, or worked off when there is notcomplete accountability for every error that occurred during test conduct. RECOVERI NG FROM TECHNI CAL PROBLEMS 145Those errors were not assigned to responsible individuals for correction andthe results were not worked into the system.The System Test as written wassubsequently not run without error.RECOVERYEvery test run should have PTR forms available to capture any anomaliesthat occur during the conduct of the test.Further, there must be a PTR log inwhich to record the PTRs and account for each and every one.Usually, a se-quence number is assigned to a PTR preceded by a unique Alpha that relates tothe test.For example, the first PTR for the System Test could be numbered asST-001.If you do not have a PTR system, consider using the information to create aform for your own use:R' PTR No.R' PriorityR' SystemR' SubsystemR' Test ConductorR' Test Title Run No [ Pobierz całość w formacie PDF ]