Answer
# 1 |
Characteristics of Effective Test Metrics
Ideally, identifying test metrics takes place at the
beginning of the project, so incorporation into the
appropriate activities is easy. The test metrics you wish
to collect need to be:
· quantifiable,
· easy to collect,
· simple,
· meaningful,
· non-threatening.
Quantifiable Measurements
To ensure consistent comparison of findings, the method of
measurement needs to be standard, concise, and quantifiable.
For example, to determine the density of defects, you need
to identify what metrics provide this information and a
standard of measurement. For example, the test metric to
gather is the number of defects and the method of
measurement is lines of code (loc), (i.e., “x” number of
defects per “y” loc).
Definitions must be clear and concise. For example, the
definition of defect must state what constitutes a defect
and the definition of lines of code must state the number of
lines of code to be used as the standard of measure, (e.g.,
1000). The definitions must also provide any other
information necessary to ensure consistency, (e.g., if the
lines of code are commented or not commented).
Easy to Collect
The information collection process must not take too much of
the collector’s time, or the information will not be
collected. The amount of test metrics gathered from any one
group needs to be kept at a minimum, collecting only that
which is most useful. Whenever possible, automate the data
collection process.
Simple Information
The information collected should be simple to gather. If it
is hard for the collector to determine what to measure or
report, the information is likely to be inaccurate.
Meaningful Purpose
The information gathered must have a specific purpose, (or
purposes). For example, the information will be used to
determine the number of defects and time used for each
testing phase, in order to determine the most cost effective
ways to minimize errors.
The information to collect must be understandable and viewed
as relevant to the collector, or the information will not be
collected. For example, to make the information in the
previous example relevant, explain that the findings will
highlight the testing methods that work and methods that
don’t work, so that employee effort is focused on productive
activities.
Non-Threatening Use
Avoid using test metrics for employee evaluation purposes.
Collection of information that is perceived as a threat to
the employee’s job status is frequently reported
inaccurately or incompletely.
Methods for Identifying Test Metrics
Start the process of identifying test metrics by listing the
problems to be solved and objectives first. Then determine
the items to measure and the standards of measurement to
use, to achieve the objectives.
Various methods can be used to complete the test metrics
identification process, (e.g., brainstorming, use of a
committee composed of representatives from management and
the groups that will help with the collection process).
Documenting Test Metrics Requirements
A template and sample illustrate how to document the
objective, the test metrics to gather, how the metrics
provide the necessary information, and definitions of the
test metrics.
Test metrics sample document:
TEST METRICS REQUIREMENTS
Objective
To determine error trends and causes based on defects found
during system testing, in order to identify the most common
defects and minimize such defects on the current and future
projects.
Metrics to Collect
Number of defects
Source of each defect
Source of each coding defect
How to Determine
Divide the number of defects for each source of defect type
by the total number of defects.
For coding defects, divide the number of defects for each
subtype by the total number of coding defects.
Definitions
Defect
A flaw which causes an incorrect or incomplete functioning
of the system, (i.e., only defects identified as real
problems (RP); cosmetic and usability errors are not included).
Source of Defect, specified using the following source of
defect codes:
The area in which the error occurred as specified using the
source of defect codes.
Functional Specification (FS)
Coded per the functional specifications, but the function
described in the specifications is incorrect, unclear or
incomplete.
Design (DSN)
Coded per the design specs, but the design is incorrect,
unclear, or incomplete.
Architecture (ARC)
Selected architectural components do not work together as
expected or planned.
Human Interface (HI)
Coded per specifications, but the human interface, (i.e.,
screens, reports, input documents) defined in the
specifications is incorrect, unclear, or incomplete.
Database Design (DB)
Unexpected results achieved due to the database design,
(e.g., incorrect primary key definition, incorrect data type).
Code Errors (CD), specified further using the following
subtypes:
Not coded according to the detailed design specifications.
Logic Errors (CD-LE)
The logic is incorrect, unclear, or incomplete.
Computation Errors (CD-CE)
A calculation is incorrect, unclear, or incomplete.
Program Unit Interfaces (CD-PI)
When control is received or given to an entity outside the
program unit, a procedure is incorrect, unclear, or
incomplete, which results in the incorrect transfer of data.
Data Handling (CD-DH)
A data handling mechanism, such as a data declaration or
data structure, is incorrect, unclear, or incomplete.
Error Checking (CD-EC)
An error checking procedure is incorrect, unclear, or
incomplete.
Collection Mechanism
Programmers currently specify the source of the error on
the fault report when they correct errors identified by
system test. Add subtype for coding errors to the fault
report form.
|
| Hi
|
To Refer this Site to Your Friends 
Site Map
