ASOKAN

• Jun 6th, 2005

 

# Purpose 
 
1. product 
 
To satisfy some contractual requirement. 
2. tool 
 
To further the testing process itself. 
 
1. schedule 
2. map 
 
Why distinguish the two? Change is inevitable; deal with it: external forces often dictate changes to the test plan. To be a useful tool, the test plan has to evolve to meet a changing environment. 
 
# Goals 
 
1. Address product risks 
2. Address project risks 
 
# Success/completion Criteria 
 
1. Test until time runs out 
2. Fraction of code/functions/features covered 
3. Test until rate of failure falls below some threshold 
 
(failures/hour of testing). 
4. Number of detected failures as fraction of estimated total 
 
failures (based on historical data). 
5. Fraction of test cases passed 
6. Ratio of failures reported to fixed 
7. Fraction of total defects removed 
 
# What is a plan 
 
1. Schedule (W5H2) 
 
The Who, What, Where, When, Why, How, How Much of a project. 
 
1. Milestones 
 
Points of time in the plan at which significant things are supposed to happen; Milestones divide the project into phases. 
 
1. Exit Criteria - what determines whether the Milestone has been 
 
successfully passed. Must be ambiguous and binary (go/no-go). These can be in terms of: 
 
1. Deliverables - what will be delivered at the Milestone, hence 
 
what is produce by work leading up to the milestone. 
2. Dates - a milestone may simply be a date at which something 
 
must happen, such as a ship date or marketing event. 
 
Note: it is not possible to fix both deliverables and dates; one must be derived from another. For example, if there are certain products (deliverables) that must be delivered to meet the Milestone, then the Milestone is actually passed when the deliverables are completed; any date attached to the Milestone is an estimate and subject to change. 
 
2. Tasks - distinct activities, assigned to people, that must be 
 
performed to perform the work in a phase. 
 
2. Map 
 
Like a roadmap or architect's drawing. This approach inventories the functions to be tested and the tests to be performed. The idea is to prioritize testing activities so that the most important things get done in the presence of schedule pressure. 
 
# Plan Contents 
 
The following are distilled from Rex Black. 
 
1. Overview, defining test objectives 
2. Bounds 
 
1. Scope - what areas/kinds of tests will/will not be addressed 
2. Definitions 
3. Setting - environment in which the plan is carried out 
 
3. Quality risks - risks associated with product features 
4. Plan 
 
1. Phases 
 
1. entry criteria 
2. exit criteria 
3. stopping criteria 
 
2. Milestones 
 
1. deliverables (estimated, if date fixed) 
2. date (estimated, if deliverables fixed) 
3. exit criteria (same as phase exit criteria) 
 
3. Tasks - what will be done in each phase 
 
1. reviews to perform 
2. tests to perform 
3. other tasks (test implementation, lab preparation, etc 
 
4. Test Environment/Configuration 
5. Resources 
6. Test and Failure tracking/management 
7. Project Risks and Contingencies 
 
Not product (quality) risks. 
8. Approvals 
9. Change history 
10. References 
 
# Phases 
 
Tests are performed by different roles, as appropriate to their expertise. Tests can be grouped into phases according to their purpose. Phases and roles often correspond: 
 
1. Performed by developers: 
 
1. Unit Test 
2. Integration Test 
 
2. Performed by testers: 
 
1. Functional Test 
2. System Test 
3. Regression tests 
 
3. Performed by customers: 
 
1. Acceptance Test 
 
4. Performed by other groups 
 
1. Performance analysis 
2. Stress tests 
 
# Entry/Exit/Stopping Criteria 
 
Does it make sense to begin System testing if Function tests fail? Can you even begin Function Testing if Integration tests fail? What happens if more tests fail than pass? 
 
1. Entry Criteria: specify what conditions must be true before a test 
 
phase can begin. Example: developers must deliver a fully integrated product that passes the ``smoke'' test before Functional testing can begin. 
2. Exit Criteria: specify what conditions must be trure before a test 
 
phase can be considered complete. Example: System testing is complete when two weeks elapse with no ``show stopper'' failures discovered. 
3. Stopping Criteria: specify conditions that indicate a test phase 
 
should be suspended. Example: Functional testing should be suspended if any tests reveal ``show stopper'' failures. 
 
It's possible for phases to overlap. We might specify entry criteria such that System testing of a function can begin as soon as Functional tests for that function pass. 

C.Chitra Naidu

• Jul 15th, 2005

 

We can prioritize testing tasks within a project by determining the risk factor of each and every module on the basis of impact and likelihood and then start testing the module having the highest risk factor.