Software Product Evaluation PDF

IT2002 Evaluation of Products for Conformance…………………………………… 4. Reevaluate the plans for conformance with contract requirements after the plans have been corrected. A substantial part of...

IT2002 Evaluation of Products for Conformance…………………………………… 4. Reevaluate the plans for conformance with contract requirements after the plans have been corrected. A substantial part of the SQA function’s efforts is devoted to evaluating the software processes’ products. Software products include software The Evaluation of Project’s Software Products for Conformance……… packages/systems and software services. In contrast, relevant documentation includes various development and operation reports, such The SQA function’s objective is to evaluate the conformance of a project’s as design reports, test reports, and periodical customer services’ reports. development products with the project contract and relevant regulations and conventions. Types of Software Product Evaluation Activities………………….……… The Evaluation Process The evaluation of the project plans for conformance The evaluation process to be performed by the SQA function includes the The evaluation of the project’s software products for conformance following: The evaluation of the project's products for acceptability by the 1. Identify project development products and related documentation, as customer listed in the project contract. The evaluation of the project’s operation phase products for 2. Evaluate the project’s products, software,related documents for conformance conformance with contract requirements and relevant regulations and The evaluation of the software product by measurement conventions. The software’s evaluation shall be performed mainly by examining review reports and test results and participating in reviews The Evaluation of Project Plans for Conformance………………………… and tests. 3. Reevaluate the software, and related documentation for conformance The objective of the SQA function is to evaluate the project plans for with contract requirements after the software and documentation conformance with the project contract. In other words, to verify that the have been corrected. contract requirements are fully covered in the plans. The evaluation should be concluded with a list of nonconformity findings to be corrected by the The Evaluation of Project Products for Acceptability by the Customer project team. The success of the software product delivery stage, including the related The Evaluation Process documentation package, depends on the development team’s product The evaluation process to be performed by the SQA function includes the quality.In other words, on the degree, the contract requirements, and the following: acquirer’s documented expectations have been fulfilled. 1. Identify the required plans in the contract document. 2. Evaluate the plan items for conformance with contract requirements The Evaluation Process and relevant regulations and conventions. The evaluation will yield The evaluation process to be performed by the SQA function includes the lists of nonconformities: following: Contract requirements that were not covered by the plans 1. Derive criteria for acceptance of the software product and the related Contract requirements that were incorrectly defined in the documentation from the contract, project documentation, SQA plans reports, joint project committee minutes, and so on. Plan items that are not required by contract, relevant 2. Evaluate the degree to which the project products, software, and regulations, or conventions related documents successfully fulfill the acceptance criteria, and 3. Evaluate the plans for consistency and list items that are not conform to contract requirements, and relevant regulations and consistent. conventions. 06 Handout 1 *Property of STI  [email protected] Page 1 of 6  IT2002 3. Reevaluate the software and related documentation to fulfill 4. Reapply the measurements and determine the effectiveness of the acceptance criteria after the software and documentation have been improvements. corrected. 5. Activities 3 and 4 may be repeated until measurement results are satisfactory. Evaluation of Project’s Operation Phase Products for Conformance….. Review…………………………………………………………………………….. The function’s objective is to evaluate the degree of conformance of products from a project’s operation phase with the project contract and A review is a process for evaluating a documented software project product. relevant regulations and conventions. A group of professionals and stakeholders raise comments regarding the contents of a document presented to them beforehand. The reviewers may The Evaluation Process have the authority to approve the document’s contents and approve the The evaluation process to be performed by the SQA function includes the project advancing to the next stage. following: 1. Identify products from the project operation phase and related Methodologies of Review……………………………………………………… documentation, as listed in the project contract. 2. Evaluate the project’s operation phase products and related Formal Design Reviews documents for conformance with contract requirements and relevant Peer Reviews regulations and conventions. Evaluation of the user support and Expert Opinions maintenance services shall be performed mainly by examining software operation service performance reports and direct Formal Design Reviews………………………………………………...……… observation of services. Formal design reviews, also called design reviews (DRs), differ from all The Evaluation of Software Product by Measurement………………….... other review methods. Theyare the only type of review that is a requisite for approving the development product. Without this approval, the development Quantitative criteria of software products provided by software product team cannot continue to the next phase of development. Formal design metrics are very important tools for evaluation. These metrics, in many reviews may be conducted at any development milestone requiring cases, are part of standards and procedures: general, organizational, or completion of a document, whether the document is a requirement specific procedures developed for a software projector and its services’ specification or an installation plan. operation. Participants in a Design Review The Required Process Software product measurements to be performed by the SQA function All DRs are conducted by a review leader and review team. The choice of include the following activities: appropriate participants is of special importance because of their authority 1. Defining a set of product metrics according to relevant general and to approve or disapprove the development product. project-specific procedures, where each metric represents a product Review Leader: Appropriate candidates for review team leadership quality attribute correctly: for example, the response time of the include the development department manager, chief software software program to a specific query engineer, a different project’s leader, the software quality assurance 2. Implementation of the metrics that may result in a list of gaps between unit head, in certain circumstances, and the customer’s chief the results of measurements and the expected quality result software engineer. The review leader’s appointment is expected to 3. Development of software or hardware improvements to close the be performed by a person of higher seniority than the project leader. gaps 06 Handout 1 *Property of STI  [email protected] Page 2 of 6  IT2002 Review Team: The entire review team should be selected from o Having undergone training as a moderator is also required among the senior members of the project team, together with – for inspections. appropriate senior professionals assigned to other projects and The Author: The author is, with no exception, a participant in each departments, customer/user representatives, and software type of peer review. Examples are programmer or designer who is development consultants. Non-project staff should make up the responsible for producing the program/ document that is being majority of the review team. inspected. They are also responsible for the work product and for correcting any defects. Peer Reviews………………………………………………...…………...……… Specialized Professionals: The recommended professionals are: o A Designer: They are the system analyst responsible for The major difference between formal design reviews and peer review analyzing and designingthe software system reviewed. methods is rooted in the participants’ level of authority. While most o A Coder or Implementer: A professional who is thoroughly participants in DRs hold superior positions to the project leader, participants acquainted with coding tasks, preferably the designated in peer reviews are, as expected, the project leader’s equals, members of coding team leader. This inspector is expected to contribute the leader’s department, and other units. The other major difference lies in his/her expertise in detecting defects that could lead to the degree of authority. DRs are authorized to approve the design document coding errors and subsequent software implementation so that work on the next stage of the project may begin. This authority is not difficulties. granted to the peer reviews, whose main objectives lie in detecting errors o A Tester: This experienced professional, preferably the and deviations from standards. leader of the assigned testing team, focuses identifying design errors usually detected during the testing phase. Participants of Peer Reviews Walkthrough Participants: The participants are led through the material in The level of formality differentiates a walkthrough from an inspection. While one of two formats: the presentation is made without interruptions and the inspection team members are required to prepare for the inspection comments are made at the end, or comments are made throughout. session, walkthrough participants are not requested to make any Review Leader (Coordinator): Candidates for the coordinator meaningful preparations. A recommended peer review team includes: position should have traits similar to those of the inspection Inspection participants moderator. They ensure that the review process is followed and that Walkthrough participants the other reviewers perform their responsibilities throughout the review process. Inspection Participants: The number of participants in an inspection range The Author: The author is, with no exception, a participant. In many from a minimum of three to a maximum of six. The minimum number of cases, he/she serves as the coordinator. They provide the rest of participants is derived from the required roles to be assigned. The the reference material that makes up the inspection package participants in an inspection are all assigned at least one role. The roles in distributed to each inspector. an inspection are: Specialized Professionals: The recommended professionals are: Review Leader (Moderator): Leader of the inspection responsible o A Standards Enforcer: This team member, who for planning the inspection and coordinating it. Candidates for this specializes in developing standards and procedures, is position must: assigned to locate deviations from the standards and o Be well-versed in the development of projects of the current procedures. type and be familiar with its technologies. o A Maintenance Expert: The maintenance expert is called o Come from outside the project team. upon to focus on maintainability, flexibility, and testability o Display proven experience in coordination and leadership issues and to detect design defects capable of impeding of professional meetings. 06 Handout 1 *Property of STI  [email protected] Page 3 of 6  IT2002 the correction of bugs or the performance of future 3. System Testing: A level of the software testing process where a changes. complete, integrated system is tested. The purpose of this test is to o A User Representative: Participation of an internal or evaluate the system’s compliance with the specified requirements. external user representative in the walkthrough team 4. Acceptance Testing: A level of the software testing process where contributes to the review’s validity. He/she examines the a system is tested for acceptability. The purpose of this test is to software system from the point of view of the evaluate the system’s compliance with the business requirements user/consumer, rather than the designer/supplier. and assess whether it is acceptable for delivery. Expert Opinions………………………………………………...…………...… Software Testing Strategies…………………………………………………... Expert opinions, prepared by outside experts, support quality evaluation by The following software testing strategies are: introducing additional capabilities to the internal review staff and reinforcing Incremental Testing versus Big Bang Testing the organization’s internal quality assurance activities. Outside experts Black Box Testing and White Box Testing transmit their expertise either by: Automated Testing versus Manual Testing Preparing an expert judgment about a document or a code section, Alpha and Beta Site Testing or by Participating as a member of an internal design review, inspection, Incremental Testing versus Big Bang Testing……………………………... or walkthrough team. The two (2) contrasting testing strategies are defined and compared: Software Testing………………………………………………………………… Incremental Testing - tests the software in steps – software modules as Software testing is an activity in which a system or component is executed they are completed (unit tests), followed by groups of modules composed under specified conditions. The results are observed or recorded, and of tested modules integrated with newly completed modules (integration evaluation is made of some aspect of the system or component. Software tests). Once the entire package is complete, it is tested as a whole (system testing is a process to evaluate the functionality of a software application to test). find whether the developed software met the specified requirements or not and to identify the defects to ensure that the product is defect-free to Big Bang Testing - tests the software as a whole once the completed produce a quality product. package is available. In big bang testingidentifying of errors in the entire software package is very difficult. As a result, performing perfect correction Software Testing Levels………….………………//…………………………... of an error in this context is frequently laborious. In addition, estimates of the required testing resources and testing schedule tend to be rather fuzzy. Software testing levels are the different stages of the software development lifecycle where testing is conducted. There are four levels of In contrast, the incremental testing, as it is performed on relatively small software testing: software units, yields higher percentages of identified errors and facilitates 1. Unit Testing: A level of the software testing process where correction. Thus, usually, incremental testing is more effective and more individual units of software are tested. The purpose is to validate efficient. As a result, it is generally accepted that incremental testing should that each unit of the software performs as designed. be preferred. 2. Integration Testing: A level of the software testing process where individual units are combined and tested as a group. The purpose of this level of testing is to expose faults in the interaction between integrated units. 06 Handout 1 *Property of STI  [email protected] Page 4 of 6  IT2002 Black Box Testing and White Box Testing………………………...………... Requires vast resources, much more than those required for black box testing Black Box Testing is also called “structural testing”. It identifies bugs only Cannot perform important test classes. according to software malfunctioning as they are revealed in its erroneous outputs. In cases that the outputs are found to be correct, black box testing Automated Testing versus Manual Testing……………………...………... disregards the internal path of calculations and processing is performed. Manual Testing is the process of testing software by hand to learn more Advantages of Black Box Testing about it, to find what is and isn’t working. This usually includes verifying all The following are the main advantages of black box testing: the features specified in requirements documents and includes the testers Less testing resources are required compared with white box trying the software with the perspective of their end user’s in mind. Manual testing. test plans vary from fully scripted test cases, giving testers detailed steps The needed resources for black box testing may be further reduced and expected results to high-level guides that steer exploratory testing by conducting automatic black box testing. sessions. The tester may carry out almost all test classes; some are not available with white box testing. Automated Testing is the process of testing the software using an automation tool to find the defects. In this process, testers execute the test Disadvantages of Black Box Testing scripts and automatically generate the test resultsusing automation tools. The following are the main disadvantages of black box testing: Test automation represents an additional step in the integration of Inability to directly test software and identify errors computerized tools in the process of software development. These tools are Lacks control of line coverage utilized in the execution of the tests, and in the management and control of Allows for identification of coincidental aggregation of several errors the testing process. as correct Lacks possibilities to test the quality of coding work Alpha and Beta Site Testing………………………...…………………….…... White Box Testing examines internal calculation paths in order to identify Alpha Site Testing are tests performed by potential users at the bugs. Although the term “white” is meant to indicate its being a contrary developer’s site on a new software package. By applying the new software alternative to black box testing, the method’s other name, “glass box to the specific requirements of his organization, the customer tends to testing,” better expresses its basic characteristic,investigating the examine the package from angles not expected by the testing team. The correctness of code structure. errors identified by alpha site tests are expected to include errors that only real users can reveal and thus should be reported to the developer. Advantages of White Box Testing The following are the main advantages of white box testing: Beta Site Testing is much more commonly applied than alpha site testing. Enables directly checking processing paths and algorithms The beta site test process can be described as follows: Once an advanced Provides line coverage follow-up that delivers lists of lines of code version of the software package is available, the developer offers it free-of- not yet been executed charge to one or more potential users. Participants in beta site testing are often users of previously released packages, sophisticated software Capable of testing the quality of coding work professionals, and the like. As beta site testing is considered to be a valuable tool, some developers involve hundreds and even thousands of Disadvantages of Black Box Testing participants in the process. The following are the main disadvantages of white box testing: 06 Handout 1 *Property of STI  [email protected] Page 5 of 6  IT2002 Advantages of Alpha and Beta Disadvantages of Alpha and Site Testing Beta Site Testing Identification of unexpected errors Lack of systematic testing A wider population in search or Low-quality error reports errors Low costs Much effort is required to examine reports References: Automated Testing versus Manual Testing (n.d.). Citing sources. Retrieved from https://www.softwaretestingmaterial.com Galin, D. (2018). Software quality assurance – Concepts and practice: IEEE Computer Society, Inc. Inspection Participants (n.d.). Citing sources. Retrieved from http://www.qasigma.com Software Testing Level (n.d.). Citing sources. Retrieved from http://softwaretestingfundamentals.com 06 Handout 1 *Property of STI  [email protected] Page 6 of 6