The aim of collecting quality metrics is to use the data for:
- improving the test process,
- learning how our team improved its efficiency and productivity, rather than to just show fancy reports.
This includes finding tangible answers to the questions:
- How bad are the bugs?
- How many bugs found were fixed? reopened? closed? deferred?
- How many bugs did the test team did not find?
- How good were the tests?
- Was the test effort adequate? Could we have fit more testing in this release?
Good answers to these questions need measurement. This post includes metrics to answer above question.
1. % of high / critical defects
Any experienced tester can find 20 bugs without even knowing the application. The key is to make sure we focus on what really matters to our customer. This metric ensures that you’re focusing on the critical ones.
% of high defects= Number of high priority bug / Number of bug reported
% of critical defects= Number of critical bug / Number of bug reported
2. % of escaped defects
Escaped defects are those that are missed by the tester and found by the end user. Although they should not be a cause for panic if discovered by the customer, the team must investigate and learn from them to improve and prevent them from recurring.
% of escaped defects= Number of high customer complaint bug after release / Number of bug reported before release
3. % of rejected defects
These are defects found in the product but not accepted by the developer as defects.
A large number of rejected defects indicates whether the developer and tester are on the same page about the feature’s functionality and its purpose.
% of rejected defects= Number of invalid bug / Number of bug reported
4. Defect removal efficiency
Defect removal efficiency is the extent to which the development team is able to handle and remove the valid defects reported by the test team.
Defect removal efficiency= Number of bug fixed / Number of bug reported
5. Mean Time to Repair
This metric demonstrates the amount of time needed to effectively fixed the defect.
Mean Time to Repair= Number of bug fixed / Total fixing time