Ch6 QA and Testing.pdf

Chapter 6 - Quality Assurance and Testing 37 10/12/2014 Chapter 24 Quality management 1  subjective is Software quality management ✧ Concerned with ensuring that the required level of quality is achieved in...

Chapter 6 - Quality Assurance and Testing 37 10/12/2014 Chapter 24 Quality management 1  subjective is Software quality management ✧ Concerned with ensuring that the required level of quality is achieved in a software product. ✧ Three principal concerns: p t.MS I ▪ At the organizational level, quality management is concerned with establishing a framework of organizational processes and standards that will lead to high-quality software (define the software development process and applied standards). ▪ At the project level, quality management involves the application of specific quality processes and checking that these planned processes have been followed (Project output meet the standards that applicable to the project). ▪ At the project level, quality management is also concerned with establishing a quality plan for a project. The quality plan should set out nonfunctional the quality goals for the project and define what processes and requirement standards are to be used. 10/12/2014 Chapter 24 Quality management 2 Quality management activities ✧ Quality management provides an independent check on the software development process. ✧ The quality management process checks the project deliverables to ensure that they are consistent with WI organizational standards and goals ✧ The quality team should be independent from the development team so that they can take an objective view of the software. This allows them to report on software quality without being influenced by software development issues. theyshould have organizationwide responsibility Quality management have 2parts Qualityassuranceprocesses and standards that should lead to highquality products Qualitycontrolapplication of these quality processes to weedoutproducts that arenotoftherequired levelofqua 10/12/2014 Chapter 24 Quality management 3  Quality qualitymanagement plan activities ✧ Quality management introduction provides an of independent the check on products Product the software development process. ✧ The quality managementAyescription process checks the project Product plan the critical releasethat deliverables to ensure they dates andare consistent responsibilities with for the Product organizational standards Process descriptions and goals the development and service processes ✧ The quality team should be independent from the development Quality goalsidentification and justification team so that they can take of critical productan objective attributes quality view of the software. This allows them to report on and risk management Riskssoftware quality without thekeyrisksbeing influenced by software that might affect product quality and the actions development issues. theyshould have organizationwide responsibility 10/12/2014 Chapter 24 Quality management 3 Quality management and software development quality checkingand development are interleaved The quality management process checks the project deliverables to ensure that they are consistent with organizational standards and goals The quality management team should be independent and report to management above the project manager level. They should ensure that the organization standards and procedures are not compromised with budget and schedules. The quality plan should set out the desired software qualities and how they are to be assessed. 10/12/2014 Chapter 24 Quality management 4  inlarge systems teams are more likely tochange sothey can use the documetations to Scope of quality management understand the system ✧ Quality management is particularly important for large, complex systems. The quality documentation is a record of progress and supports continuity of development as the development team changes. ✧ For smaller systems, quality management needs less documentation and should focus on establishing a quality culture. ✧ Techniques have to evolve when agile development is used. because agile focuses on minimal documentation which contradict with this 10/12/2014 Chapter 24 Quality management 5  Software quality 10/12/2014 Chapter 24 Quality management 6 Software quality ✧ Quality, simplistically, means that a product should meet its specification. ✧ This is problematical for software systems ▪ There is a tension between customer quality requirements (efficiency, reliability, etc.) and developer quality requirements (maintainability, reusability, etc.); ▪ Some quality requirements are difficult to specify in an unambiguous way; 6,5 ▪ Software specifications are usually incomplete and often inconsistent. ✧ The focus may be ‘fitness for purpose’. ▪ An evaluation of how well a product or service fulfills a customer’s desires based on the organization’s goals or reason for existence. In short, it is the ability of an organization or team to fulfill its mission. 10/12/2014 Chapter 24 Quality management 7 Non-functional characteristics ✧ The subjective quality of a software system is largely based on its non-functional characteristics. ✧ This reflects practical user experience – if the software’s functionality is not what is expected, then users will often just work around this and find other ways to do what they want to do. ✧ However, if the software is unreliable or too slow, then it is practically impossible for them to achieve their goals. 10/12/2014 Chapter 24 Quality management 8 Quality conflicts ✧ It is not possible for any system to be optimized for all of these attributes – for example, improving security may lead to loss of performance. ✧ The quality plan should therefore define the most important quality attributes for the software that is being developed. d not everything isequally important ✧ The plan should also include a definition of the quality g assessment process, an agreed way of assessing whether some quality, such as maintainability or robustness, is present in the product. 10/12/2014 Chapter 24 Quality management 9 Process and product quality ✧ The quality of a developed product is influenced by the quality of the production process. ✧ This is important in software development as some product quality attributes are hard to assess. ✧ However, there is a very complex and poorly understood relationship between software processes and product quality. ▪ The application of individual skills and experience is particularly important in software development; ▪ External factors such as the novelty of an application or the need for an accelerated development schedule may impair product quality. good processes are morelikely to likely to leadto is good quality software but does notguaranteeit 10/12/2014 Chapter 24 Quality management 10 inspections benefits in testingerrors can hideothererrors o if you're using inspection you're not Xecuting the program so thiswonthappen incompleteversions of Reviews and inspections a system can e inspected without additional costs n testing youmight haveto create additional inspection can consider broader quality attributes f a Program non functional attributes 10/12/2014 Chapter 24 Quality management 11  thepurpose is to improve formal informal software quality Reviews and inspections not to assess the performance of people in the development team ✧ A group examines part or all of a process or system and its documentation to find potential problems/issues. 85 ✧ Software or documents may be 'signed off' at a review which signifies that progress to the next development stage has been approved by management. ✧ There are different types of review with different objectives code ▪ Inspections for defect removal (product); documentation ▪ Reviews for progress assessment (product and process); ▪ Quality reviews (product and standards). 10/12/2014 Chapter 24 Quality management 12  check the consistency and completeness of the documents or code under Quality reviews review ✧ A group of people carefully examine part or all of a software system and its associated documentation. ✧ Code, designs, specifications, test plans, standards, etc. can all be reviewed. ✧ Software or documents may be 'signed off' at a review which signifies that progress to the next development stage has been approved by management. 10/12/2014 Chapter 24 Quality management 13  reviewteams usually are 34 people meeting can be held online The software review process atndnace sggggga.IE short two hours at post d tr anauthor of the group document walkthrought thedocument with the afterthe before the review team meeting meeting 10/12/2014 Chapter 24 Quality management 14  different places Distributed reviews if the Group are in ✧ The processes suggested for reviews assume that the review team has a face-to-face meeting to discuss the software or documents that they are reviewing. ✧ However, project teams are now often distributed, sometimes across countries or continents, so it is impractical for team members to meet face to face. ✧ Remote Reviewing can be supported using shared documents where each review team member can annotate the document with their comments. 10/12/2014 Chapter 24 Quality management 15 Program inspections informal ✧ These are peer reviews where engineers examine the source of a system with the aim of discovering anomalies and defects. I can check only one method not a whole system ✧ Inspections do not require execution of a system so may be used before implementation. ✧ They may be applied to any representation of the system (requirements, design,configuration data, test data, etc.). ✧ They have been shown to be an effective technique for discovering program errors. inspections may bepart of the software verification and validation process 10/12/2014 Chapter 24 Quality management 16 Inspection checklists ✧ Checklist of common errors - should be used to drive the inspection. ✧ Error checklists are programming language dependent and reflect the characteristic errors that are likely to arise in the language. You usedifferent fordifferent programming languages checklists ✧ Examples: Initialisation, Constant naming, loop termination, array bounds, etc. the weaker the type checking the larger the checklist d will be like Python 10/12/2014 Chapter 24 Quality management 17 An inspection checklist (a) Fault class Inspection check Data faults ∙ Are all program variables initialized before their values are used? ∙ Have all constants been named? ∙ Should the upper bound of arrays be equal to the size of the array or Size -1? ∙ If character strings are used, is a delimiter explicitly assigned? ∙ Is there any possibility of buffer overflow?type varible f P Control faults ∙ For each conditional statement, is the condition correct? ∙ Is each loop certain to terminate? ∙ Are compound statements correctly bracketed? ∙ In case statements, are all possible cases accounted for? ∙ If a break is required after each case in case statements, has it been included? Input/output faults ∙ Are all input variables used? ∙ Are all output variables assigned a value before they are output? ∙ Can unexpected inputs cause corruption? 10/12/2014 Chapter 24 Quality management 18  An inspection checklist (b) Fault class Inspection check Interface faults ∙ Do all function and method calls have the correct number of parameters? ∙ Do formal and actual parameter types match? ∙ Are the parameters in the right order? ∙ If components access shared memory, do they have the same model of the shared memory structure? Storage management ∙ If a linked structure is modified, have all links been faults correctly reassigned? ∙ If dynamic storage is used, has space been allocated correctly? ∙ Is space explicitly deallocated after it is no longer required? Exception management ∙ Have all possible error conditions been taken into faults account? 10/12/2014 Chapter 24 Quality management 19  Software Testing 30/10/2014 Chapter 8 Software Testing 20 Program testing ✧ Testing is intended to show that a program does what it is errors intended to do and to discover program defects before it is put into use. ✧ When you test a software, you execute a program using artificial data. I ✧ You check the results of the test for errors, anomalies or information about the program’s non-functional attributes. ✧ Can reveal the presence of errors NOT their absence. 30/10/2014 Chapter 8 Software Testing 21 Validation and defect testing ✧ Validation testing first goal ▪ You expect the system to perform correctly using a given set of test cases that reflect the system’s expected use. ✧ Defect testing second goal ▪ The test cases are designed to expose defects. The test cases in 1 defect testing can be deliberately unclear and need not reflect how the system is normally used. errors in special cases not the normal Flo 30/10/2014 Chapter 8 Software Testing 22  An input-output model of program testing Validation Are we buildingthe right product get ensurethatthe software eats thecustomer's expectations rfication Are we building the roductright Process of checking that he software meets it stated functional and nonfunctional requirements 30/10/2014 Chapter 8 Software Testing 23 Inspections and testing focus on the source code ✧ Software inspections Concerned with analysis of the static system representation to discover problems (static verification) ▪ May be supplement by tool-based document and code analysis. ✧ Software testing Concerned with exercising and observing product behaviour (dynamic verification) ▪ The system is executed with test data and its operational behaviour is observed. 30/10/2014 Chapter 8 Software Testing 24 Inspections and testing 30/10/2014 Chapter 8 Software Testing 25 Stages of testing after some ✧ Development testing, where the system is testeddiagrams is during development to discover bugs and defects.finished ✧ Release testing, where a separate testing team test a complete version of the system before it is released to users. theaim is tocheck thatthesystem meets the requirements of the system stakehol ✧ User testing, where users or potential users of a system test the system in their own environment. Acceptance testing is onetype ofuser testing the feasting process usually involves of manualand automated testing a mixture tester runs the program with sometestdata and compares theresults to their expectatit tests are encoded in a program thatis run eachtimethesystem automated is faster than manual testing 30/10/2014 Chapter 8 Software Testing 26  Development testing 30/10/2014 Chapter 8 Software Testing 27  the tester is usually the programmer Development testing ✧ Development testing includes all testing activities that are carried out by the team developing the system. ▪ Unit testing, where individual program units or object classes are tested. Unit testing should focus on testing the functionality of objects or methods. ▪ Component testing, where several individual units are integrated to create composite components. Component testing should focus on testing component interfaces. ▪ System testing, where some or all of the components in a system are integrated and the system is tested as a whole. System testing should focus on testing component interactions. 30/10/2014 Chapter 8 Software Testing 28 Unit testing ✧ Unit testing is the process of testing individual components in isolation. ✧ It is a defect testing process. ✧ Units may be: ▪ Individual functions or methods within an object ▪ Object classes with several attributes and methods ▪ Composite components with defined interfaces used to access their functionality. Kinds of test cases reflect normal operationshould show thatthe component works common problems 30/10/2014 Chapter 8 Software Testing 29  component testing Object class testing functions methods are the ✧ Complete test coverage of a class involves simplest type of components ▪ Testing all operations associated with an object ▪ Setting and interrogating all object attributes ▪ Exercising the object in all possible states. ✧ Inheritance makes it more difficult to design object class tests as the information to be tested is not localised. 30/10/2014 Chapter 8 Software Testing 30 Testing policies ✧ Exhaustive system testing is impossible so testing policies which define the required system test coverage may be developed. ✧ Testing, should be based on a subset of possible test cases chosen by the software companies according to its policies. ✧ Examples of testing policies: ▪ All system functions that are accessed through menus should be tested. ▪ Combinations of functions (e.g. text formatting) that are accessed through the same menu must be tested. ▪ Where user input is provided, all functions must be tested with both correct and incorrect input. 30/10/2014 Chapter 8 Software Testing 31  Release testing 30/10/2014 Chapter 8 Software Testing 32 Requirements based testing ✧ Requirements-based testing involves examining each requirement and developing a test or tests for it. ✧ Mentcare system requirements (A Case): ▪ If a patient is known to be allergic to any particular medication, then prescription of that medication shall result in a warning message being issued to the system user. ▪ If a prescriber chooses to ignore an allergy warning, they shall provide a reason why this has been ignored. 30/10/2014 Chapter 8 Software Testing 33 Requirements tests (for Mentcare Case) ✧ Set up a patient record with no known allergies. Prescribe medication for allergies that are known to exist. Check that a warning message is not issued by the system. ✧ Set up a patient record with a known allergy. Prescribe the medication to that the patient is allergic to, and check that the warning is issued by the system. ✧ Set up a patient record in which allergies to two or more drugs are recorded. Prescribe both of these drugs separately and check that the correct warning for each drug is issued. ✧ Prescribe two drugs that the patient is allergic to. Check that two warnings are correctly issued. ✧ Prescribe a drug that issues a warning and overrule that warning. Check that the system requires the user to provide information explaining why the warning was overruled. 30/10/2014 Chapter 8 Software Testing 34 Performance testing ✧ Part of release testing may involve testing the emergent properties of a system, such as performance and reliability. nonfunctional requirement ✧ Tests should reflect the profile of use of the system. ✧ Performance tests usually involve planning a series of tests where the load is steadily increased until the system performance becomes unacceptable. ✧ Stress testing is a form of performance testing where the system is deliberately overloaded to test its failure behaviour. a lot of users 30/10/2014 Chapter 8 Software Testing 35  Release testing vs. System testing There are two important distinctions between release testing and system testing during the development process: 1. A separate team that has not been involved in the system development should be responsible for release testing. 2. System testing by the development team should focus on discovering bugs in the system (defect testing). The objective of release testing is to check that the system meets its requirements and is good enough for external use. Primary goal of the release testing process is to convince thesupplier of the system th is good enough for use 30/10/2014 Chapter 8 Software Testing 36  User testing 30/10/2014 Chapter 8 Software Testing 37 User testing informal Process essential ✧ User or customer testing is a stage in the testing process in which users or customers provide input and advice on system testing. 41 ✧ User testing is essential, even when comprehensive system and release testing have been carried out. ▪ The reason for this is that influences from the user’s working environment have a major effect on the reliability, performance, WE usability and robustness of a system. These cannot be replicated in a testing environment. 30/10/2014 Chapter 8 Software Testing 38  the first test is alpha testing Types of user testing then beta testing ✧ Alpha testing ▪ Users of the software work with the development team to test the software at the developer’s site. ✧ Beta testing at the user site ▪ A release of the software is made available to users to allow them to experiment and to raise problems that they discover with the system developers. ✧ Acceptance testing ▪ Customers test a system to decide whether or not it is ready to be accepted from the system developers and deployed in the customer environment. Primarily for custom systems. 30/10/2014 Chapter 8 Software Testing 39  The acceptance testing process I t d d acceptancecriteria deciding on acceptance if there is meetingbetw some developers Should theresources time test should decided bepart of aim to functional Problems thennegotiate an hesystemcontract budgetrisks thecustome and nonfunctional to if the system is goodenough to used be 30/10/2014 Chapter 8 Software Testing 40  Stages in the acceptance testing process ✧ Define acceptance criteria ✧ Plan acceptance testing ✧ Derive acceptance tests ✧ Run acceptance tests ✧ Negotiate test results ✧ Accept/Reject system customer wantsand theimportant functions that not in the system will never acceptthat it is 30/10/2014 Chapter 8 Software Testing 41