Answer / hi

Characteristics of Effective Test Metrics



Ideally, identifying test metrics takes place at the
beginning of the project, so incorporation into the
appropriate activities is easy. The test metrics you wish
to collect need to be:



· quantifiable,

· easy to collect,

· simple,

· meaningful,

· non-threatening.



Quantifiable Measurements



To ensure consistent comparison of findings, the method of
measurement needs to be standard, concise, and quantifiable.
For example, to determine the density of defects, you need
to identify what metrics provide this information and a
standard of measurement. For example, the test metric to
gather is the number of defects and the method of
measurement is lines of code (loc), (i.e., “x” number of
defects per “y” loc).



Definitions must be clear and concise. For example, the
definition of defect must state what constitutes a defect
and the definition of lines of code must state the number of
lines of code to be used as the standard of measure, (e.g.,
1000). The definitions must also provide any other
information necessary to ensure consistency, (e.g., if the
lines of code are commented or not commented).



Easy to Collect



The information collection process must not take too much of
the collector’s time, or the information will not be
collected. The amount of test metrics gathered from any one
group needs to be kept at a minimum, collecting only that
which is most useful. Whenever possible, automate the data
collection process.



Simple Information



The information collected should be simple to gather. If it
is hard for the collector to determine what to measure or
report, the information is likely to be inaccurate.



Meaningful Purpose



The information gathered must have a specific purpose, (or
purposes). For example, the information will be used to
determine the number of defects and time used for each
testing phase, in order to determine the most cost effective
ways to minimize errors.



The information to collect must be understandable and viewed
as relevant to the collector, or the information will not be
collected. For example, to make the information in the
previous example relevant, explain that the findings will
highlight the testing methods that work and methods that
don’t work, so that employee effort is focused on productive
activities.



Non-Threatening Use



Avoid using test metrics for employee evaluation purposes.
Collection of information that is perceived as a threat to
the employee’s job status is frequently reported
inaccurately or incompletely.



Methods for Identifying Test Metrics



Start the process of identifying test metrics by listing the
problems to be solved and objectives first. Then determine
the items to measure and the standards of measurement to
use, to achieve the objectives.



Various methods can be used to complete the test metrics
identification process, (e.g., brainstorming, use of a
committee composed of representatives from management and
the groups that will help with the collection process).



Documenting Test Metrics Requirements



A template and sample illustrate how to document the
objective, the test metrics to gather, how the metrics
provide the necessary information, and definitions of the
test metrics.

Test metrics sample document:
TEST METRICS REQUIREMENTS

Objective
To determine error trends and causes based on defects found
during system testing, in order to identify the most common
defects and minimize such defects on the current and future
projects.

Metrics to Collect
Number of defects

Source of each defect

Source of each coding defect

How to Determine
Divide the number of defects for each source of defect type
by the total number of defects.

For coding defects, divide the number of defects for each
subtype by the total number of coding defects.

Definitions
Defect
A flaw which causes an incorrect or incomplete functioning
of the system, (i.e., only defects identified as real
problems (RP); cosmetic and usability errors are not included).

Source of Defect, specified using the following source of
defect codes:
The area in which the error occurred as specified using the
source of defect codes.

Functional Specification (FS)
Coded per the functional specifications, but the function
described in the specifications is incorrect, unclear or
incomplete.

Design (DSN)
Coded per the design specs, but the design is incorrect,
unclear, or incomplete.

Architecture (ARC)
Selected architectural components do not work together as
expected or planned.

Human Interface (HI)
Coded per specifications, but the human interface, (i.e.,
screens, reports, input documents) defined in the
specifications is incorrect, unclear, or incomplete.

Database Design (DB)
Unexpected results achieved due to the database design,
(e.g., incorrect primary key definition, incorrect data type).

Code Errors (CD), specified further using the following
subtypes:
Not coded according to the detailed design specifications.

Logic Errors (CD-LE)
The logic is incorrect, unclear, or incomplete.

Computation Errors (CD-CE)
A calculation is incorrect, unclear, or incomplete.

Program Unit Interfaces (CD-PI)
When control is received or given to an entity outside the
program unit, a procedure is incorrect, unclear, or
incomplete, which results in the incorrect transfer of data.

Data Handling (CD-DH)
A data handling mechanism, such as a data declaration or
data structure, is incorrect, unclear, or incomplete.

Error Checking (CD-EC)
An error checking procedure is incorrect, unclear, or
incomplete.

Collection Mechanism
Programmers currently specify the source of the error on
the fault report when they correct errors identified by
system test. Add subtype for coding errors to the fault
report form.

Answer / sathya

Metrics are the derived value from the base measures.

Base measures are nothing
but "effort,defects,schedule,requirements,test
cases,project size."

Using the base measures, you can track the project's status
quantitatively(thro' different metrics).

Here you can refer this example(for calculating the
metrics "productivity") :

Formulae: Productivity = Project size/ effort(in person
days)

so for e.g. if the project size is 300 FP(functional
points) & the actual effort in person days is 30, then the
productivity will be 10.(i.e.300/30)

Answer / ganesh

A metric is a mathematical number that shows a relationship
between two variables.
Software Metrics are measure that are used to quantify the
software, software development resource and software
development process.

test cases on calculator

3 Answers   BMC, MNC,

---

special bug you posted in your project

3 Answers   HCL,

---

compatibility testing?

2 Answers   NDS,

---

Draw Backs of automated testing?

6 Answers   Wipro,

---

How you create Test Cases Based on Test Plan ? Is there any approach following for creation of test cases?

4 Answers   US Technology,

---

---

What is Forward and Backward compatibility? How will you write test cases for compatibility testing?

1 Answers  

---

What is the difference between Use Case and test case?

4 Answers  

---

hi plz send me top 30 FAQ's & answers on MANUAL TESTING.... regards Sonia

3 Answers  

---

Does anyone have any good examples of some acceptance criteria for non-functional requirements?

0 Answers   IBM,

---

Hello Friends,I want some clearence regarding. Test report and Final report.

0 Answers  

---

write the test cases that you have scheduled an interview and you came for the same and you are about to take the interview.

1 Answers   TCS,

---

Describe to me when you would consider employing a failure mode and effect analysis.

1 Answers  

---