Test Configuration & Measurement Overview

Test Measurement Overview

• The objective is to define Test Configuration & Measurement in Software Testing
• The objective is also to explain the process and discuss factors involved in Test Configuration & Measurement

Contents

• Introduction to Test Configuration and Measurement
• Test Configuration
• Test Measurement
• Overall Process

Introduction to Test Configuration & Measurement

• It ensures the right software is developed
• It serves to determine if we can improve something that cannot be measured
• Software version indicates change
• The software version gives an indication of its purpose

Test Configuration

• It is a Software Configuration Management (SCM)
• SCM is the task of tracking and controlling changes in the software
• SCM practices include Version Control and the establishment of baselines
• SCM can determine what was change and who changed it in the event of issues
• SCM can determine how to replicate it across many hosts when working well
• Latest Standards: IEEE 828-2012
• Revision year: 1983, 1990, 1998, 2005, 2012

Factors of running Test Configuration

• Multiple people are working on software that is continually being updated
• Software should be able to run on various machines and Operating Systems.
• There can be multiple versions and branches, authors in a software project and geographically distributed teams working concurrently
• Helps to develop coordination among stakeholders.
• Changes in user requirement, policy, budget and schedule need to be accommodated
• SCM process is also beneficial to control the costs involved in making changes to a system

Software Branching

• A branch is a copy of code line managed in a version control system (VCS)
• Branching helps software development teams work in parallel
• Branches separates in-progress work from tested and stable code
• Codebase in a VCS is referred to as the trunk, baseline, master, or mainline

Test Measurement

• There are two types of measurement in science
  • Qualitative: Looks for pattern in non-numerical data (descriptive: taste & color of apple)
  • Quantitative: Involves running statistical analysis on data that has numerical values (# of apple)
• Quantitative analysis is brought into software testing for measurement
• To measure, metrics are used
  • Base Metrics (#)
  • Calculated Metrics (%).

Base Metrics (Direct Measure)

• Raw data gathered by a Test Analyst throughout the testing effort
• These metrics are used to provide project status reports to the Test Lead and Project Manager
• They feed into the formulas used to derive Calculated Metrics.
  • E.g. : # of Test Cases, # of Test Cases Executed

Calculated Metrics (Indirect Measure)

• Base Metrics data is converted into more useful information.
• Generally, these metrics are the responsibility of the Test Lead
• Can be tracked at many different levels such as module, tester, or project
  • E.g. : % Complete, % Test Coverage

Test Measurements Factors

• Need for a decision for next phase of activities
• Need of evidence of claims or prediction
• Need to understand the type of improvement required
• Need decision or process or technology change

Metrics Life Cycle

• Analysis includes the identification the Metrics and defining the identified Metrics.
• Communication explains the need of metric to stakeholder and testing team, and educates the testing team about the data points need to be captured for processing the metric
• Evaluation covers capturing & verifying the data, and calculating the metric(s) value using the data captured
• Reporting covers developing the report with effective conclusion, distributing report to the stakeholder and respective representative, and taking feedback from the stakeholder