Software Quality and Testing Revisions Document PDF

Software quality and testing ============================ Critical awareness/evaluation/assessment to compare and contrast, make judgement calls and justify them with evidence. Definitions (these are my definitions inspired by other references). **Software:** "A list of instructions executed on...

Software quality and testing ============================ Critical awareness/evaluation/assessment to compare and contrast, make judgement calls and justify them with evidence. Definitions (these are my definitions inspired by other references). **Software:** "A list of instructions executed on a computer in order to fulfil a pre-determined purpose" **Software Testing:**\ \ "Process of evaluating and verifying that a piece of software fulfils its pre-determined purpose" **Errors:**\ \ People make these. These may cause many faults. **Faults:**\ \ These are caused by errors in software. These may cause many failures or none. **Fail:**\ \ Faults can (not always) cause software to fail. **Failure:**\ \ Observed departure of the software from its expected behaviour. This is dependant on the expectations of the software. **Quality:\ **\ "Meets and exceeds requirements" Quality assigned to software though is more like "Fir for purpose" and there are many measurables:\ \ A diagram of quality control Description automatically generated **A good test:** "A good test case is one that has a high probability of detecting an as-yet undiscovered error". During the software engineering lifecycle testing/reviewing can be done throughout at every stage. **Testing (narrow):** "Dynamic execution of the software completed towards the end of the process" **Reviewing:** "Can be done early on the in process and is a static examination of an output" Structural testing ------------------ For a test strategy we need to know what to test and how to test it in ***[each]*** stage of development. Noting any constraints. **Test plan:**\ \ "A good test plan helps organise and manage the testing effort" Communicates the test approach and responsibilities and specifies the tests to run. Test plans can be reviewed. **What is in a test plan:**  **Test case:** "An aspect of the software that needs to be tested -- purpose of testing" These should be written for invalid and unexpected input conditions **Test Data:** "For each test that is run, the planned inputs, starting conditions and expected outputs " **Stages of testing:\ \ **A screenshot of a computer Description automatically generated You cannot test everything all testing is sampling. **Structural (white-box or glass-box testing):\ \ **Tests come from analysis of the structure of the software. The internal code structure is visible. Structural tests can be based on control flow and/or data flow using a directed graph for control flow analysis. The goal of structural testing is to test as much of the program as possible. The subject of knowing how much testing is enough is assessed through the concept of coverage. **Functional (black-box spec-based testing):** Tests come from the requirements specification of the software. The internal code structure can be unknown. Directed graph -------------- Set of nodes connected by edges.\ \ Nodes (spots/circles) represent a set of sequential statements (block of code can be entire statements or fragments of a statement)\ \ Edges (lines with arrows) represent transfer of processing control or the flow of control.  **Path:** Each path through a directed graph is a unique route from the beginning of the program to the end, however if there are iterations (loop) each iteration could be considered a different path. **Test Coverage:** The minimal set of test cases that satisfy the coverage criteria. Example "a set of test cases the ensure that all statements in a unit of code have been executed" Testing Techniques ------------------ There is no perfect testing technique, any technique should be justified when used in order to accommodate the techniques flaws.\ \ A testing technique is how we come up with test cases , with each different technique having different testing coverage. Statement testing -- Design a test that executes every source statement in the software at least once.\ \ if (mark \> 40){\ indicator = "Passed"\ }\ \ In the above example if the input mark is 47 this would be sufficient for statement testing. Branch testing -- Design test cases so that each decision has both TRUE and FALSE outcomes executed at least once.\ \ if (examMark \>= 36 && courseworkMark \>=36 && (examMark + courseworkMark) / 2 \> 40 ){ indicator = "Passed" }\ else\ {\ indicator = "failed"\ }\ \ In this example you would need 2 tests to be sufficient for branch testing\ \ test 1 examMark 48, courseworkMark 63 == TRUE\ test 2 examMark 30, courseworkMark 90 == FALSE Condition testing -- Design test cases so that each condition in a decision in the program has both TRUE and FALSE outcomes executed at least once (if possible).\ \ \ if ((A \|\| B) && C){ statement }\ else\ { statement } **A** **B** **C** ------- --------------- --------------- TRUE Not evaluated FALSE FALSE TRUE TRUE FALSE FALSE Not evaluated In this example the above 3 test would be sufficient for condition testing coverage.\ \ if (value == 1){\ display = 1\ }\ else if(value == 2){\ display = 2\ }\ \ This is also condition testing for this there is two tests value = 1 and value = 2 Decision/Condition testing -- design test cases for the program so that each condition in a decision in the program has both TRUE and FALSE outcomes executed at least once (if possible) and each decision has both TRUE and FALSE outcomes executed. Multiple condition testing -- Design test cases for the program so that all possible combination of TRUE and FALSE condition outcomes in each decision are invoked at least once. DO WHILE (employee.salary != 0 && employee.number = 0 \|\| records.all){\ employee.read();\ }\ \ Path testing -- Design test cases that cause each path through the program to be executed at least once. Path -- unique rout from the start to the end of the program. Example\ \ A screen shot of a computer code Description automatically generated\ \  Example 2\ \ A screen shot of a computer Description automatically generated There is an issue with loops ---------------------------- The issue with loops is that each iteration through a loop is technically a different path.\ \ \ \ Where N is the maximum number of iterations. Issue here is that N could be infinite. Data flow -- Classify each occurrence of a variable in a program as one of these: d- Definition. The variable is assigned a value.\ u- Use. The variable is used in output or in an expression or in a condition. OR Design test cases so that each d-u pair is exercised at least once. A d-u pair is a pair off program statements one off the being the d-definition of the variable and the other being th u-use of the variable. Each test case is chosen to exercise the path through the unit that executes this d-u pair of statements. Example:\ A screen shot of a computer Description automatically generated d-u pair (factorial) D U ---------------------- --- --- 1 6 4 2 8 4 3 6 6 4 8 6 Uses of data flow testing -- analyse the data flow for each variable, choose test cases that exercise all d-u pairs for each variable. Advantages of data flow testing -- can find: a var that is declared but not used, a variable that is used but never declared, a variable that is defined multiple times before it is used deallocating a variable before it is used. Weaknesses of data flow testing -- doesn't deal with compound variables like arrays, there might be a more than one path between a d-u pair (this is addressed by a stronger form of data flow testing known as all-d u-paths). All techniques have strengths and weaknesses. Unit testing -- derive tests from analysis of the unit specification using some functional testing. Functional testing ------------------ Derive test cases from analysis of the requirements specification. An issue with this is it depends on the quality of the requirements specification clarity completeness, consistency accuracy and testability. This can be done before any coding and can show errors in the design before you start coding. ### Equivalence partitioning (EV) Partition -- possible set of inputs into subsets which the application processes in the exactly the same way, according to the requirements specification. Select at least one test case from the partition and one from outside the partition (if appropriate). The assumption is that any representative value from a partition is as likely to reveal a fault as any other from that partition. It's based on an assumption that all elements within a partition are the same but are they, there is also no set in stone way to pick partitions. ### Boundary value analysis (BVA) Select data on the edge (boundary) of each EV Ideally at least on value on either side of a boundary. This is done as functionality errors are more likely to be at the boundaries, this simply comes from experience of coding it before, setting boundaries in code are often conditional statements e.g., if(value \> 0) or if(value \