METRICS MODULE 1.pdf

SWE 2020 - Software Metrics Dr. Krithika L B, SCORE,VIT Course Objectives 1. To understand data analysis metrics and models to assess software products. 2. To emphasize the use of software product and quality metrics. 3. T...

SWE 2020 - Software Metrics Dr. Krithika L B, SCORE,VIT Course Objectives 1. To understand data analysis metrics and models to assess software products. 2. To emphasize the use of software product and quality metrics. 3. To study various metrics models in the applications of software design and production  1. To understand the challenges and difficulties of Course  applying software metrics. Outcomes  2. Choose appropriate metrics to collect data and use them to make predictions.  3. Carry out data analysis and visualization  4. Capture a key aspect of software size.  5. Capture a key aspect of software structure.  6. Identify a variety of quality models and evaluation techniques.  7. Make decisions for software project risk assessment and prediction.  8. Apply and evaluate the data analysis methods to validate decisions Course Overview Module 1: Basics of Measurements Module 2: Software Metrics Data Collection Module 3: Measuring Internal Product size and structure Module 4: External Product Attributes Module 5: Metrics for Decision Support Module 6: Applications of Software Metrics in industry Text Book and Reference Books Mode of Evaluation 6 Digital Assignment 1  10 Marks Quiz 1  10 Marks Quiz 2  10 Marks CAT1  15 Marks CAT2  15 Marks FAT  40 Marks Module-1 Basics of Measurement Measurement in Software Engineering - Scope and basics of Software Measurement - A Goal-Based Framework for Software Measurement- Applying the Framework - Software Measurement Validation. Measurement in everyday life Measurement helps us to: Understand our world Interact with our surroundings Improve our lives Key Points Metric - A quantifiable characteristic of software. Measurement - The process of mapping from real world attributes to a mathematical representation. Model - A mathematical relationship between metrics.  Why Measure?  The primary purpose of a measurement program should be to helping project managers make project decisions.  You can’t manage what you can’t measure!  If you are not measuring, then how are you managing? What is measurement? A process by which numbers or symbols are assigned to attributes of entities in the real world in such a way as to describe them according to clearly defined rules Entityobject(person in a room) or an event (Journey or testing phase of a software project) Attributeproperty of an entity(color of room , cost of journey, elapsed time of testing phase) Why measuring software? Measure help us for following three activities: Understand what happens during development and maintenance. Control what happened on the project, predict what happens and make changes to meet our goal. Based on the design quality improve the process and product. What is Software Metrics? Provides values and units for describing software attributes Motivation for Metrics  Estimate the cost & schedule of future projects  Evaluate the productivity impacts of new tools and techniques  Establish productivity trends over time  Improve software quality  Forecast future staffing needs  Anticipate and reduce future maintenance needs  Measurements in Software Engineering SE activities include managing, costing, planning, modeling, analyzing, specifying, designing, implementing, testing and maintaining. Engineering approach-each activity is understood and controlled. SE focuses on implementing the software in a controlled and scientific way. Chapter1 Need to measure in SE Best software engineers measure the characteristics of the software to ensure: Whether requirements are consistent and complete Whether design is of high quality Whether the code is ready to be tested Effective Project managers measure attributes of product and process to tell, When the software will be ready for delivery and whether the budget will be exceeded Informed customers measure aspects of final product to determine if it meets the requirements and is of sufficient quality Maintainers must be able to measure the current product to see what should be,upgraded and improved Types of measurement –Direct & Indirect  Direct Measurement  measured without the involvement of any other entity or attribute.  Example:  Length of source code by LOC  Duration of testing process (measured by elapsed time in hours)  Number of defects discovered during the testing process (measured by counting defects)  Time a programmer spends on a project (how many weeks)  Indirect Measurement  measured in terms of any other entity or attribute.  Example:  Programmer Productivity=LOC produced / Person months of effort.  Requirement Stability=No. of initial requirements / Total no. of requirements.  System spoilage=Effort spent for fixing faults / Total project effort  Some measures used in SE Entity – object Attribute – property Measure – Map between entity and attribute  Software Metrics  Software metrics is a standard of measure that contains many activities which involve some degree of measurement. It can be classified into three categories:  Project metrics describe the project characteristics and execution. Examples include the number of software developers, the staffing pattern over the life cycle of the software, cost, schedule, and productivity.  Process metrics can be used to improve software development and maintenance. Examples include the effectiveness of defect removal during development, the pattern of testing defect arrival, and the response time of the fix process.  Product metrics describe the characteristics of the product such as size, complexity, design features, performance, and quality level.  Benefits of Software metrics  It makes the better control, planning and clear visibility.  It helps to increase the production and quality.  can find out the errors which creates problem on the first level of development life cycle.  can better control and examine this process of development life cycle.  can take idea that how much cost and size would be this software.  can analyze the problem which occurred in development process and helpful in better decision making. Components of a measurement system Components of a measurement system: – m = – Attribute is what is being measured (e.g., size of a program) – Scale is the standard and scope of measurement (e.g., nominal,ordinal, ratio scale, etc.) – Unit is the physical meaning of scale (e.g., a positive integer, a symbol, etc.) Determining the value of an attribute of an entity. Determining the class of entities to which the measurement relates.  The Representational Theory of Measurement Chapter2 The Representational Theory of Measurement Empirical Relations: Empirical relation system (C,R), where C is the set of entities and R is the set of empirical relationship Eg: X is taller than Y here “taller than “ is binary empirical relation on entities X and Y A and B are much taller than Y “much taller than “ is empirical relation A is tall “tall” is empirical relation Mapping technique used to map these empirical relations to the mathematical world Likert Scale Here, the users will be given a statement upon which they have to agree or disagree Forced Ranking:- Order the given alternatives from 1 (best) to n (worst). Mapping technique used to map these empirical relations to the mathematical world Verbal Frequency Scale Ordinal Scale Here, the users will be given a list of alternatives and they have to select one Mapping technique used to map these empirical relations to the mathematical world Comparative Scale Here, the user has to give a number by comparing the different options. Numerical Scale Here, the user has to give a number according to its importance.  The Rules of Mapping To perform the mapping, we have to specify domain, range as well as the rules to perform the mapping. For example Domain - Real world Range − Mathematical world such as integers, real number, etc. Rules − For measuring the height, shoes to be worn or not Similarly, in case of software measurement, the checklist of the statement to be included in the lines of code to be specified. The Representational Condition of Measurement Measurement mapping (M) must map entities into numbers, and empirical relations into numerical relations in such a way that the empirical relations preserve and are preserved by numerical relations. For example: The empirical relation ‘taller than’ is mapped to the numerical relation ‘>’ X is taller than Y if and only if M(X) > M(Y) Since, there can be many relations on a given set, the representational condition also has implications for each of these relations. For the unary relation ‘is tall’, we might have the numerical relation as, X > 50 The representational condition requires that for any measure M, X is tall if and only if M(X) > 50 The Representational Condition of Measurement The Representational Condition of Measurement Empirical relations that hold: X is “taller than” YM(X)>M(Y) Y “is tall” M(Y)=72 >70 X is “much taller than” Z M(X)=84>57=M(Z)+15 Y is “much taller than” Z M(Y)=72>57=M(Z)+15 Z is higher than X when sitting M(X)=84 on Y’s shoulders M(Y)=72 M(Z)=42 0.7M(Z)+0.8M(Y) =87>84=M(X)  Real, Empirical & Formal Worlds Measurement Real World Empirical World Scales & Units Modeling and verification Formal World Mapping Real, Empirical & Formal Worlds Measurement Height Entity A 199 Entity B 108 Scale: ratio Unit:cm M(A)>M(B) Modeling & Verification Mapping Measurement Scale and types  Measurement Scale and types Measurement scales are the mappings used for representing the empirical relation system. Eg: It is mainly of 5 types: Nominal Scale Ordinal Scale Interval Scale Ratio Scale Absolute Scale Nominal Scale It places the elements in a classification scheme. The classes will not be ordered. Each and every entity should be placed in a particular class or category based on the value of the attribute. It has two major characteristics − The empirical relation system consists only of different classes; there is no notion of ordering among the classes. It does not have numerical value, no order can’t perform arithmetic operations on them, like addition or subtraction, or logical operations like “equal to” or “greater than” on them. Any distinct numbering or symbolic representation of the classes is an acceptable measure, but there is no notion of magnitude associated with the numbers or symbols.  Examples of Nominal Variables Eye color (Blue, Green, Brown, Hazel). Type of house (Bungalow, Duplex, Ranchi). Gender (Male, Female). Type of pet (Dog, Cat, Rodent, Fish, Bird). Genotype ( AA, Aa, or aa). naming; participants respond with words; does not put participants in any particular mathematical order The Nominal Scale The nominal scale, sometimes called the qualitative type, places non- numerical data into categories or classifications. For example: Placing cats into breed typea Persian is a breed of cat. Placing cities into states Vellore is a city in Tamilnadu. Surveying people to find out if men or women have higher self-esteem. Qualitative variables are measured on the nominal scale. A pie chart displays groups of nominal variables (i.e. categories) Survey on how people travelled Ordinal Scale It places the elements in an ordered classification scheme. It has the following characteristics, The empirical relation system consists of classes that are ordered with respect to the attribute. Any mapping that preserves the ordering is acceptable. The numbers represent ranking only. Hence, addition, subtraction, and other arithmetic operations have no meaning. For example, first, second, third, or fourth are all ordinal numbers. Ordinal data You don’t have to have the exact words “first, second, third…etc.” Instead, you can have different rating scales, like “Hot, hotter, hottest” or “Agree, strongly agree, disagree.” The intervals between the values are not equal. We know that a list of cardinal numbers like 1, 5, 10 have a set value between them (in this case, 5) but with ordinal data you just don’t know. For example, in a marathon you might have first, second and third place. But if you don’t know the exact finishing times, you don’t know what the interval between first and second, or second and third is. Ordinal scale Ordinal scales are made up of ordinal data. Deals with ordered variables Some examples of ordinal scales: High school class rankings: 1st, 2nd, 3rd etc.. Social economic class: working, middle, upper. The Likert Scale: agree, strongly agree, disagree etc.. ordering; measurements place participants in order from high to low The ordinal scale classifies according to rank. Ordinal Scale Examples conduct a survey and ask Creating the order of preference people to rate their level of results in the movies being ordered on an ordinal scale: satisfaction with the choice of Order five movies from your most the following responses: favorite to your least favorite: Extremely satisfied. The Matrix. Satisfied. Jaws. Neither satisfied nor dissatisfied. Children of Men. Dissatisfied. The Sound of Music. Extremely dissatisfied. All Good Things. Interval Scale This scale captures the information about the size of the intervals that separate the classification. Hence, it is more powerful than the nominal scale and the ordinal scale. It has the following characteristics − It preserves order like the ordinal scale. It preserves the differences but not the ratio. Addition and subtraction can be performed on this scale but not multiplication or division. If an attribute is measurable on an interval scale, and M and M’ are mappings that satisfy the representation condition, then we can always find two numbers a and b such that, M = aM’ + b Interval scale examples An interval scale has ordered numbers with meaningful divisions. Temperature is on the interval scale: a difference of 10 degrees between 90 and 100 means the same as 10 degrees between 150 and 160. equal interval without an absolute zero (ex: multiple choice achievement tests) Ratio Scale An empirical relation exists to capture ratios. It has the following characteristics − It is a measurement mapping that preserves ordering, the size of intervals between the entities and the ratio between the entities. There is a zero element, representing total lack of the attributes. Must start at zero and increase at equal intervals, known as units. All arithmetic operations can be applied. Here, mapping will be of the form M = aM’ Where ‘a’ is a positive scalar. Ratio vs Interval scale A ratio scale has all the properties of an interval scale. Ratio data on the ratio scale has measurable intervals. For example, the difference between a height of six feet and five feet is the same as the interval between two feet and three feet. Where the ratio scale differs from the interval scale is that it also has a meaningful zero. The zero in a ratio scale means that something doesn’t exist. For example, the zero in the Kelvin temperature scale means that heat does not exist at zero Example Weight At 0 pounds, you would weight nothing and therefore wouldn’t exist. Height If you were 0″, you would have no height. Sales figures Sales of zero means that you sold nothing and so sales didn’t exist. Compare that to a temperature of zero, which while it exists, it doesn’t mean anything in particular (although admittedly, in the Celsius scale it’s the freezing point for water). The Ratio Scale and Ratios As the “0” in the ratio scale means the complete absence of anything, there are no negative numbers on this scale a count of how many tests you took last semester could be zero if you didn’t take any tests. However, if you took two exams this semester and four the last semester, you could say that the frequency of your test taking this semester was half what it was last semester. Absolute Scale On this scale, there will be only one possible measure for an attribute. Hence, the only possible transformation will be the identity transformation. It has the following characteristics − The measurement is made by counting the number of elements in the entity set. The attribute always takes the form “number of occurrences of x in the entity”. There is only one possible measurement mapping, namely the actual count. All arithmetic operations can be performed on the resulting count. Scale types Nominal: just classification Ordinal: linear ordering (>) Interval: like ordinal, but interval between values is the same (so average has a meaning) Ratio: like interval, but there is a 0 (zero) (so A can be twice B) Absolute: counting number of occurrences Summary of Measurement scales and statistics Examples Examples Objective, multiple choice achievement tests are usually assumed to measure at what level? interval If we measure in such a way that we find out which subject is most honest, which is the next most honest, and so on, we are measuring at what scale of measurement? ordinal The number of minutes of overtime work that employees perform is an example of what scale of measurement? ratio Weight measured in pounds is an example of which scale of measurement? ratio  A GOAL-BASED FRAMEWORK FOR SOFTWARE MEASUREMENT Chapter3 A GOAL-BASED FRAMEWORK FOR SOFTWARE MEASUREMENT A conceptual framework for the diverse software measurement activities that contribute to organization’s software practices: It has three principles: Classifying the entities to be examined Determining relevant measurement goals Identifying the level of maturity that your organization has reached Goal Question Metric(GQM) Process Determining what to measure – Identifying entities – Classifying entities to be examined – Determining relevant goal Determining how to measure – Inquire about metrics – Assign metrics Thursday, May 9, 2019 GQM  hierarchical model that follows a top-down approach where first the goals are specified, then questions are written and collected, and finally, metrics are associated with each question GQM  Goal – improve Products, Processes or Resources (Conceptual level)  Question – should be specific to the goal (Operational level)  Metric – quantitative way in which data can be collected to address the question you are asking (Quantitative level)  What is the purpose of GQM? To ensure that you use metrics effectively Example  You can make sure that they are measuring aspects of your project that are valuable to you and your team. That method is called goal, question, metric. Goal, question, metric, or GQM is a method of choosing metrics for your software projects created by Victor Basili in the University of Maryland in 1994.  The basic idea behind GQM, according to Basili himself, is that for an organization to measure in a purposeful way, it must first specify the goals for itself and its projects. Then it must trace those goals to the data that are intended to define those goals operationally, and finally, provide a framework for interpreting the data with respect to the stated goals.  Basically, in order to make sure that you're using the right metrics for your project you have to know the concept that you want to measure first, also known as your goal. Once you know that you have to figure out what defines that goal, also known as your question. Then you have to find a way to calculate that definition in a way that you can interpret to inform your progress towards the goal, this is known as your metric.  Essentially, you could choose your goal to improve one of these three things.  You could choose to improve your product like your code, or your documentation. You could choose to improve your processes, such as your testing or elicitation processes. What is the purpose of GQM?  To ensure that you are building the right product.  To ensure that you use metrics effectively.  To make sure that metrics use the correct data.  To make sure that you ask the client the right questions.  The answer is , to ensure that you use metrics effectively.  GQM is a framework for using metrics that focuses on making sure that you use the best metrics for answering the question you want to ask about your project. The next thing to consider after each goal are the questions you want to ask about the goals. Basically, what questions do you need to ask that will tell you whether each goal is being met. This depends on the individual goal. But the general idea is that your questions should be specific enough so that the answer to the question addresses the goal.  And how do you find that answer? That's where the metric part comes in.  To start at the beginning, how do you set a goal? Well, according to Basili, goals are applied to objects. These objects are products, processes, and resources.  A metric is a quantitative way in which data can be collective to address the questions you're asking.  For example, this could be number of product downloads, sales numbers, processing time, or user stories completed. Subjective metrics are those that depend on interpretation, such as product ratings by users, developer morale, or code quality. Let's look at an example of GQM. Say, you're developing product and you want to know whether your product is done right. Product quality then is your goal. Here are a few questions that could help you address this goal.  How much of my product's code is covered by unit tests?  How many defects are in my product?  Once you've asked your questions, the next step is to identify your metrics.  So, if we were to take the question of how many defects are in my product? We could approach this in a few ways.  One metric could be the number of times a user store is rejected during acceptance testing before it's finally accepted. This could be represented as a percentage of rejections to acceptances across the whole project.  Another metric that addresses this question is the number of bugs that are placed in your product backlog at any given time. Identifying entities Process Products Resources Thursday, May 9, 2019  Classifying Software Measures In software there are three such classes  Processes: Collection of software-related activities eg: timescale  Products: Artifacts, deliverables or documents that result from a process activity Eg: prototype  Resources: Entities required by a process activity. Internal and External Attributes External Internal  Usability  Size, Effort, Cost  Integrity  Code Complexity  Efficiency  Functionality  Testability  Modularity  Reusability  Redundancy  Portability  Syntactic Correctness  Interoperability  Reuse It is assumed that good internal structure leads to a good external quality. Processes Products Questions about our software- Any artifact or document produced development activities and during the software life cycle can processes that measurement can be measured and assessed help us to answer. External product attributes depend on both product behavior and how long it takes for a process to environment, each attribute complete, measure should take these how much it will cost, characteristics into account. whether it is effective or efficient, Internal product attributes are and sometimes easy to measure eg: size and some attributes are difficult to how it compares with other measure processes that we could’ve chosen. Eg: code complexity  Resources The resources include any input for software production. Resources include personnel, materials, tools and methods. Resources are measured to determine their magnitude, cost and quality. Cost is often measured across all types of resources, so that managers can see how the cost of inputs affects the cost of the outputs. Resource measure combine a process measure(input) with a product measure(output). Resources Personnel (individual or team), materials (including office supplies), tools, methods. Resource measurement may show what resource to blame for poor quality. Cost measured across all types or resources. Productivity: amount_of_output/effort_input. Combines resource measure (input) with the product measure (output). Components of software measurement Goal-Question-Metric Paradigm Goal‐Question‐Metric Many metrics begin by what is convenient or easy to measure rather than what is needed such program not useful for developer and maintainer. GQM involves three steps: List the major goals of the development project Derive questions from each goal that must be answered to determine if the goals are being met Decide what must be measured in order to answer the question adequately. Example for GQM Templates for goal definition Purpose: To (characterize, evaluate, predict…) the (process, model, metric…) in order to (understand, assess, manage, learn, improve…) Eg: To evaluate the maintenance process inorder to improve it Perspective Examine the (cost, correctness, defects, changes…) from the viewpoint of (developer, manager, customer…) Examine the cost from the viewpoint of manager Environment The environment consists of process factors, people factors, methods, tools, etc. The maintenance staff are poorly motivated programmers who have limited access to tools GQM example You are manager for a software development team and you have to decide upon the release time of your product. Construct a GQM tree related to this. Iteration 1 Final iteration Measurement and Process Improvement Measurement can help us to answer about the effectiveness of techniques or tools, and productivity , quality of products and more. It allows manager and developer to monitor the effects of activities and changes on development. So that we can control the final outcome. Its useful for Understanding, Establishing a baseline Assessing and predicting Overview of maturity levels SEI’s Process Maturity Levels Combining GQM with process maturity High level goals Improving productivity Improving quality Reducing risk Goal: Improving productivity has several sub goals affecting resources. Ensuring adequate managerial skills. Ensuring adequate staff skills. Ensuring adequate host software engineering technology. To improve productivity using products can mean Identifying problems early in the life cycle Using appropriate technology Reusing previously built products. Measurements for the question based on maturity levelQuestion: Is the set of requirements maintainable? Process Status Measurement Comments Maturity Level Level1 Project have ill- Measuring requirements at Count the number of requirements defined this level is difficult and changes to those requirement requirements to establish a baseline Level 2 Requirements are Can collect additional Type of each requirement well defined information (database, interface, performance requirements and so) and number of changes to each type Level3 Visibility into A richer type of Eg: Noting the effects on each process is measurement: measuring change on related components. improved and the traceability of each intermediate requirement to the activities are corresponding design , code defined with entry and test components and exit criteria Conclusion Goal and question analysis is the same but More mature your process more Thursday, May 9, 2019 metric recommendations varies with maturity B.Prabadevi, SITE mature your measurements 102