ELEC5618 Notes.pdf

Week 2 Software Quality Software quality is the degree to which a product meets established requirements, which accurately represent stakeholder needs, wants and expectations. Software Failures Classification of software error causes: 1. Faulty definition of requirements (by the client) ● Requireme...

Week 2 Software Quality Software quality is the degree to which a product meets established requirements, which accurately represent stakeholder needs, wants and expectations. Software Failures Classification of software error causes: 1. Faulty definition of requirements (by the client) ● Requirements defined wrongly or erroneously ● Requirements incomplete ● Lack of essential requirements ● Unnecessary requirements 2. Client-developer communication failures ● Client’s instructions misunderstood by developer ● Client’s requirement changes misunderstood by developer ● Lack of attention to client messages and responses ● Client’s responses to design issues misunderstood by developer 3. Deliberate deviations from software requirements (by the developer) ● Reusing or recycling software modules from previous projects ● Omitting required functionality due to time and budget pressures ● Effecting improvements with managerial and client approval 4. Logical design errors ● Erroneous algorithm listed in software requirements ● Erroneous flowchart and sequencing for process definitions in requirements ● Erroneous definition of boundary conditions ● Omission of required states and operations 5. Coding errors ● Errors in programming languages ● Errors in data selection 6. Noncompliance with documentation and coding instructions ● Difficulties in understanding the software ● Inadequate handling of detected bugs ● Noncomplying team members ● Noncomplying code ● Failure to test noncomplying code 7. Disintegrated testing process ● Test plans incomplete ● Failure to document and report detected bugs, errors and faults ● Failure to promptly correct detected bugs, errors and faults ● Failure to test corrections and patches of detected bugs, errors and faults 8. User interface and procedure errors ● Failure to examine user interface and procedure ● Incorrect procedures 9. Documentation errors ● Erroneous explanations in documentation ● Erroneous instructions in documentation ● Omission of software functions in documentation ● Nonexistent software functions listed in documentation Week 3 Software Quality Activities Software Quality Planning A project-level quality plan is produced to identify relevant quality standards and regulations. The plan: ● defines quality requirements and goals to be achieved ● selects applicable software quality procedures for the project ● supports the introduction of new procedures ● supports the utilisation of tools for Software Quality Assurance Software Quality Assurance The definition and evaluation of software processes to ensure relevant quality standards are met. The definitions and evaluations: ● should be free from technical, managerial and financial pressures ● support systematic planning and implementation ● fulfil technical requirements and stakeholder’s considerations ● consider external factors such as schedule and budget Software Quality Control A set of procedures to ensure that the software meets its quality goals. The procedures: ● ensure that the software is free from high defect rates ● support systematic checking - checks are carried out according to a fixed plan ● support functional and non-functional requirements ● lead to verification, validation and software testing IEEE 730-2014 Standard This standard defines the requirements for initiating, planning, controlling, and executing the Software Quality Assurance processes of a software development or maintenance project. Its purpose is to produce a justified statement of confidence that a software product meets its established requirements. It is applicable to both standalone software products and software products part of a system. However, it is limited in that it does not impose constraints on the implementation or performance of activities and tasks. It aids in software process improvement efforts and ensures that consistency is maintained for quality management systems. ISO/IEC 25010:2011 Standard This standard defines the necessary and desired quality characteristics required in a software product. It focuses on the representation of stakeholders’ needs. It consists of two models of software quality: Quality In Use (Category A) ● ● ● ● Relates to the outcome of interaction when a product is used in a particular context of use Applicable to human-computer system Computer systems in use Software products in use Effectiveness Efficiency Satisfaction Freedom from risk Context coverage System/software product quality (Category B) ● ● Relates to the static properties of software and dynamic properties of the computer system Applicable to computer systems and software products Functional Suitability Reliability Performance Efficiency Operability Security Compatibility Maintainability Portability Week 4 Software Requirements Specification Definition and Structure A Software Requirements Specification is a document that clearly defines the system under development. Its intended audience is the owner, development team, and end users of the system. The document defines: ● the information structure (data model) ● what to do with the data (functional and information flow model) ● how to interact with the system (behavioural model) The document should cover: ● system needs ● a high-level description of the system functionality ● goals of the system ● definition of terms ● definition of acronyms The references section should be used for defining existing systems, procedures and requirements. The overview section should provide a brief description of how the document is organised and support enhanced readability. The general description section should include: ● Product perspective ○ A high-level diagram that shows the major components of the overall system, its interconnections, and external interfaces ● Product functions ○ A list of the major functions of the system ● User characteristics ○ A list of users anticipated to be using the product ● Operating environment ○ The hardware platform and operating system to be used ● General constraints ● ○ A list of hardware, software, and network constraints Assumptions and dependencies ○ A list of assumptions made and factors considered Key Characteristics Key characteristics of a Software Requirements Specification: Complete Traceable Precise Understandable Unambiguous Organised Verifiable Correct Consistent Concise Modifiable Usable Benefits Benefits of having a Software Requirements Specification: ● Able to understand and specify requirements ● Able to provide precise statements about the functions of the system ● Able to provide a basis for costs and schedules estimations ● Able to provide a baseline for validation and verification ● Able to mitigate the problem of language barriers ● Able to mitigate difficulties in understanding and reading documents ● Able to prevent requirements confusion ● Able to prevent mixing of functional and non-functional requirements ● Able to prevent the combination of several different requirements ● Able to reduce development efforts ● Able to serve as a basis for enhancement IEEE 830-2009 Standard This standard defines the contents and characteristics of a good Software Requirements Specification. Contents of a good Software Requirements Specification: 1. Introduction 1.1. Purpose 1.2. Scope 1.3. Definitions, acronyms and abbreviations 1.4. References 1.5. Overview 2. 3. 4. Overall Description 2.1. Product perspective 2.2. Product functions 2.3. User characteristics 2.4. Constraints 2.5. Assumptions and dependencies 2.6. Apportioning of requirements Specific Requirements 3.1. External interfaces 3.2. Functions 3.3. Performance requirements 3.4. Logical database requirements 3.5. Design constraints 3.6. Software system quality attributes 3.7. Organising the specific requirements Supporting Information 4.1. Table of contents 4.2. Index 4.3. Appendices Characteristics of a good Software Requirements Specification: Unambiguous Complete Modifiable Traceable Verifiable Consistent Usable - operation and maintenance phases Example of Functional Requirements The following is a sample list of functional requirements in the Uber app’s Software Requirements Specification: ● The system requires access to mobile phone ● The system will be synced to the mobile phones ● The phone app will provide the required information ● The system will ensure customers can access rides ● The system will ensure customers information are in the databases ● The system will accept payments ● The system will incorporate driver ratings ● The phone app will record customer data ● Customer data is automatically retrieved ● Customers can cancel rides ● Customers can pay for rides ● Customers can provide feedback ● The system will be able to add or delete customer details Use Cases Use cases provide a description of how a user uses a system to accomplish a certain goal. Use case template Goal Main goal of the use case Scope & Level Scope and level of the use case Pre-condition Conditions that must be met before the use case starts Success end condition Successful condition with the use case implementation Failed end condition Failed condition with the use case implementation Actors Users that initiate the use case Trigger Anything that starts a train of actions Description A list of steps/actions to be taken Week 5 IEEE Standard 1012 Definition This standard defines verification and validation activities in the entire software life cycle. It deals with processes applied to determine whether a software product conforms to its specifications (verification) and whether it satisfies the objectives of its intended use (validation). It provides an objective assessment of the product and processes to illustrate whether these are correct, complete, accurate, and consistent, and also meet with relevant specifications and satisfy their intended use and user needs. Basic Concepts of IEEE Standard 1012 ● ● Broadly defines verification and validation activities Defines integrity levels for system performance issues ○ Major - a function affects important system performance ○ Moderate - a function affects system performance but an alternate method of operation is available ● ● ● ○ Low - a function affects system performance only by inconveniencing the user Defines minimum verification and validation tasks ○ What minimum tasks are required for each integrity level? Provides detailed criteria for verification and validation tasks ○ What is the minimum criteria for correctness, consistency, completeness, accuracy, readability, and testability? Allows for compliance and compatibility to international standards ○ Verification and validation processes are defined to conform to the international software life cycle standards as well as the entire group of IEEE typical software engineering standards Processes Activities Tasks General/Common Concept definition System requirements System Requirements analysis Procedures documentation Software Design Project cooperation Hardware Implementation Acceptance criteria Transition Schedules Operations Maintenance Disposal/Improvement Verification Verification is a process of evaluating a software product or component with the purpose of checking whether the software system or component meets its specification. It involves checking whether the software product or component is correctly and fully implemented based on the presented specifications at the beginning of the development phase of a project. Verification tasks may include: ● Contract verification ○ System requirements ○ Procedures for change management ○ Procedures for project cooperation ○ Acceptance criteria ○ Documentation of procedures in accordance with requirements ● Reporting Validation Validation is a process of evaluating a software product or component with the purpose of checking whether its specification captures the client’s needs or intended use. It improves customer satisfaction with the software product. Systematic follow-up activities are required for validation and may include: ● Data collection ● Number of code errors ● Analysis of the error data ● Comparative analysis of the severity of errors Overview of ExpliSAT ExpliSAT is an IBM formal verification tool for C/C++ software. It verifies that: ● the program satisfied a wide set of correctness properties and embedded assertions ● no feasible execution path can lead to a violation of an assertion or built-in check ● the software code satisfies various arithmetic properties Test cases can be compiled together with the program code and examined using a debugger. It is used in embedded software projects for unit testing of critical pieces of code. Software Validation for Publishing in the IBM Cloud IBM Cloud includes a test environment where you can add your software. Configuration details for your software will need to be finalised, including ● Version details ● Prerequisites ● Constraints After successful validation, your software will become available in the IBM Cloud. Week 6 Software Testing Software testing is an important means of assessing a software to determine its quality. It is a process used to verify that the software satisfies the specific requirements and to detect errors. The amount of time required for software testing increases as maintenance and growth of an existing software increases. The software is systematically exercised in controlled circumstances to affirm its quality. Software Testing Measurement Software testing is measured according to attributes or product characteristics in need. Software testing measurement can be compared to levels of software testing result: ● Level 1 - did you run the planned tests? ● Level 2 - what are the test results? ● Level 3 - did you apply the test results to improve the code? ● Level 4 - what is the return on investment for the amount spent on testing? Metrics and measurements can also be applied to measure the effectiveness of software testing on software. Metrics can be based on quality characteristics such as functionality, reliability, usability, efficiency, maintainability, security, portability, etc. ISO/IEC/IEEE 29119 Standard This standard aims to resolve conflicts in definitions and procedures whilst providing standards for organisational test policy and strategy, project test management, non-functional testing, and system and acceptance testing techniques. It is applicable throughout the development and maintenance of software products. Its scope and structure is as follows: I. Concepts and vocabulary ○ Software testing concepts and vocabulary for enhanced understandability and readability II. Test processes ○ Organisational test processes ○ Test management processes ○ Fundamental test processes III. Test documentation ○ Organisational test policy and strategy ○ Project test plan and completion report IV. ○ Testing contents ○ Appendices (examples of documents at each testing level) Testing techniques and coverage ○ Static (e.g. reviews, inspections) ○ Dynamic (e.g. white box and black box) ○ Non-functional ○ Test measurement ○ Appendices White Box Testing White box testing emphasises on the internal structure of the software. The tester has full knowledge of and access to the code being tested. Test cases are selected based on the implementation of the software’s entities. Expected results are evaluated on coverage criteria such as: ● Path coverage ● Branch coverage ● Statement coverage For a codebase, the following may be verified: ● All tests cases pass ● All paths have been traversed (path coverage) ● Branches of each condition have been executed (branch coverage) ● All code statements have been executed (statement coverage) Black Box Testing Black box testing emphasises on input domain, expected behaviour, specifications, and knowledge of the software. The tester does not have access to the code being tested and must carry out testing based on the expected behaviour and functionality of the software. Based on the software specifications and knowledge of the expected behaviour, a tester may: ● Design test cases ○ Based on particular data to test with ● Determine whether a function or sub-function of the software is working ○ Based on the expected test result ● Identify the functions and sub-functions of the software under test ○ Based on what the software can do ● Identify the data attributes and execution conditions associated with the relevant test associated with the software ● Test each function and see whether the function does what it is supposed to do or not Apple’s Basic Software Testing Recommendations ● ● ● ● ● ● ● ● ● Importance ○ Testing is important because it protects customers’ and employees’ experience Continuous efforts ○ Testing should be carried out all year-round ○ Testing should be carried out across all scenarios Dedicated testing team ○ A dedicated testing team should exist to evaluate functionalities of software List all specific use cases Cross-functional groups participation and testing prioritisation Beta testing ○ Beta testing allows necessary data to be captured ○ Beta testing allows feedback to be submitted, thereby helping discover bugs Beta-testing program ○ A beta-testing program would allow the latest pre-release software versions to be evaluated Organise and track ongoing testing ○ A comprehensive spreadsheet of all use cases should be used Consider methodological testing throughout beta releases for enhanced security, employee productivity, and operational integrity Week 7 Software Test Plan Definition and Characteristics A software test plan is a document that describes the objectives, scope, approach and focus of software testing. It: ● ● ● ● ● validates the acceptability of a software product focuses on the intended audience enhances understanding of why and how the software product is validated covers all requirements specifications - both functional and non-functional describes: ○ the items to be tested ○ the level that each item will be tested ○ the sequence of testing the items ○ the test strategy ○ the test environment ● ● should be organisation wide - applicable to all software developments in the organisation should start at the same time the requirements documentation and project plan are being developed Key Contents of a Software Test Plan I. II. III. IV. V. VI. VII. VIII. IX. X. XI. XII. XIII. XIV. XV. XVI. XVII. XVIII. XIX. XX. Test plan identifier References Introduction/Background Test items Software risk issues Features to be tested Features not to be tested Approach Item pass/fail criteria Suspension criteria and resumption requirements Test deliverables Remaining test tasks Environmental needs Staffing and training needs Roles Responsibilities Schedule Planning risks and contingencies Approvals Glossary Supporting Standards for Software Test Plan ● ● ● IEEE 829 Standard: Software and System Test Documentation IEEE 1008 Standard: Software Unit Testing IEEE 1012-2016 Standard: System, Software, and Hardware Verification and Validation Roles and Responsibilities Team Roles Management Team ● ● Project Leader Software Quality Assurance Leader Testing Team ● ● Test Planner and Controllers Testers Business Team ● ● Business Analysts Business Representatives Testing Support Team ● Support Programmers External Support Team ● Technical Support Integration Testing Integration testing involves combining two or more components into a larger structure for testing. It is widely recommended for software testing programs. Integration tests can be manual and automated, and are derived from specifications. Approaches to integration testing include: ● Top-down testing ● Bottom-up testing ● A combination of both top-down and bottom-up testing Example of top-down testing Example of bottom-up testing Integration Testing in a Microservice Landscape Microservice architectures support the introduction of new features and bug fixes for their services without depending on other services. Integration testing can be carried out via parallel testing. This allows the testing of any service without affecting production, making it easier to identify and contain bugs. Week 8 Finite State Machines Definition A finite state machine is a model for describing the dynamic behaviour of an object over a period of time. Finite State Machine Elements ● States ○ A set of object values ○ A period of time an object waits for an event to occur ○ A period of time an object performs an activity ● ● ● Transitions ○ The response of an object in the state to the occurrence of an event ○ Consists of: ■ Event trigger ● e.g. an event that enables a transition ■ Guard condition ● e.g. a Boolean expression ■ Effect ● e.g. an action ■ Target state Inputs ○ A finite set of values ○ Largely dependent on the states ○ Can consist of conditions Outputs ○ Largely dependent on the states and inputs ○ Correspondence to the inputs’ conditions ○ Results associated with states Representations of Finite State Machines Finite state machines can be represented in multiple formats: Number of states Recommended format Small number of states ● ● ● Graphical representation Nodes and arrows Easy to create Large number of states ● ● ● ● ● Tabular representation Transition and output relations Set of inputs and states are impact Easy to process by tools Example: Current State Large number of states, few transitions ● ● Input Next State Transition lists Example: {(input_A, output_A, state_A), ... (input_i, output_i, state_i)} Key Limitations of Finite State Machines ● ● ● Output Subject to changes when designs change Unforeseen problems from states, transitions, inputs, and outputs Heavy reliance on representations and operations ● ● States may be represented incorrectly ○ Missing states ■ e.g. lack of initial states ○ Incorrect States ■ e.g. specifying incorrect behaviour ○ Extra states ■ e.g. wrong combination of inputs Transitions may be represented incorrectly ○ Missing transitions ■ e.g. unconnected sequence of states ○ Incorrect transitions ■ e.g. incorrectly connecting two states together ○ Extra transitions ■ e.g. transitions that do not correspond with the product Markov Chains Markov chains are finite state machines with stochastic transitions described with probability values. The probability values represent the probability of a transition: ● from state i at time n ● to state j at time n+1 The sum of probabilities of all outgoing transactions from a state should be 1. The probability values are estimated: ● initially, based on expert knowledge of the application ● based on previously collected statistics ● using a combination of both expert knowledge and collected statistics Microsoft’s use of Finite State Machines Microsoft used finite state machines to model asynchronous callbacks in software systems. They did this because of the millions of lines of source code in separate modules that must interact. In doing so, they improved the performance and reliability of their software systems. Week 9 Software Quality Engineering in Agile Environments The Agile Software Development Framework: ● supports constant dialogue between “possible” and “desirable” ● supports small releases, simple designs and constant refactoring ● supports pair programming, content ownership, and continuous integration in Software Quality Engineering. When the Agile Software Development Framework is used, Software Quality Engineering objectives may be achieved in a more cooperative and collaborative manner. Principles of Agile Software Development Principle Description Highest priority ● Customer satisfaction is ensured with early and continuous delivery of valuable software Change management ● Change should be possible even at the later stages of software development This would provide a competitive advantage ● Time management ● Shorter timescales are preferred for frequent delivery of working software Collaboration ● Business people and developers should be able to work together everyday throughout the project Trusted project building ● The project should be built around motivated individuals Information delivery ● Information should be conveyed effectively Progress measurement ● Progress on tasks should be easily measured with working software (e.g. Jira, Trello) Sustainable development ● Project sponsors, developers, and users should be able to maintain indefinite constant pace throughout the project Continuous attention ● ● Technical excellence should be ensured with good design Agility should be enhanced with good design Simplicity ● Essential tasks are supported by keeping things simple Self-organised teams ● Teams should utilise best architectures, requirements, and designs Continuous improvement ● ● ● Reflection should be carried out on how to become more effective The team will become more effective Behaviours in the project will be adjusted accordingly Software Development Methodologies that share the principles ● ● ● ● ● ● ● ● ● Scrum Agile modelling Rapid application development (RAD) Extreme programming (XP) Crystal Feature-driven development (FDD) Dynamic system development method (DSDM) Kanban Lean software development Oracle Agile Product Lifecycle Management Oracle Agile PLM is a solution that helps businesses manage their product value chain. It supports cross-functional teams, allowing them to work together on co-related assignments through an integrated framework and product data insights. It also supports: ● agile product governance ● agile product collaboration ● agile product cost ● Oracle product lifecycle analytics Agile Development Agile software development is a set of principles for software development. Agile software development values: ● individuals and interactions over processes and tools ● working software over comprehensive documentation ● customer collaboration over contract negotiation ● responding to change over following a plan Scrum Model Team ● ● ● Product Owner Scrum Master Developers/Programmers Events/Activities ● ● ● ● ● Spring Planning Daily Scrum Sprint Review Sprint Retrospective Sprint Artefacts ● ● ● Product Backlog Sprint Backlog Burndown Steps 1. 2. 3. 4. 5. 6. 7. Form the team Sprint planning Create the product backlog Start a sprint Track progress Complete a sprint Repeat