Software Testing: Why Test Programs PDF

TESTING why do test the software  Overview Introduction Why testing is needed Failure, fault, and error. Defect masking? Testing and debugging. ISO 9126 System reliability Test oracle. Who test programs? 2  Al...

TESTING why do test the software  Overview Introduction Why testing is needed Failure, fault, and error. Defect masking? Testing and debugging. ISO 9126 System reliability Test oracle. Who test programs? 2  All in one question: Suppose you are being asked to lead the team to test the software that controls a new X-ray machine. Would you take that job? Would you take it if you could name your own price? What if the contract says you’ll be charged with murder in case a patient dies because of a mal- functioning of the software? 3  Workflow Software testing - What is Scope of Software testing? - June 4, 1996 an unmanned Ariane 5 rocket was launched. - At 39 seconds after launch, when the rocket reached an altitude of 2.5 miles, a self-destruct mechanism was activated. - Self-destruction was triggered automatically because aerodynamic forces were ripping the boosters - The rocket was on its first voyage, after a decade of development costing $7 billion. The destroyed rocket and its cargo were valued at $500 million. - One bug, one crash : ▪ Steering was controlled by the on-board computer, which mistakenly thought the rocket needed a course change because of numbers coming from the inertial guidance system ▪ It turned out that the cause of the failure was a software error. ▪ A 64 bit floating point number was converted to a 16 bit signed integer. The number was larger than 32,768, the largest integer storeable in a 16bit signed int, and thus the conversion failed  Workflow Software testing What is Scope of Software testing? → A primary purpose of testing is to detect software failures so that defects may be corrected. → Software testing can also be stated as the process of validating and verifying that a software program/application/product: meets the business and technical requirements that guided its design and Testing can never be used to show development; the absence of works as expected errors, but can be implemented with the same only their presence characteristics.  Workflow Software testing - Levels of Testing Levels of software testing: Module Test Sofware Integration Test Software Test System Integration Test System Test  Workflow Module Test The purpose: - verify, that the software modules Tessy Environment properly reflect the software specification and design. Target: - 100% C1 coverage - 100% coverage on requirements  Workflow Integration The purpose ▪ integrate the software modules into larger assemblies, producing integrated software consistent with the software design/arhitecture In other words: ▪ Phase when all the project developers meet. ▪ Put all modules and applications together and see if they work.  Workflow Integration Test The purpose: Phase when all the interactions between the interfaces of integrated modules are tested SW Integration Test objectives failures within the interaction and communication of the integrated software modules problems with the resource management runtime failures  Workflow Software Test Determine if the software meets all of the specifications defined by Software Requirements The objective is to provide confidence that the delivered software meets the business requirements of both sponsors and users. The Software Test may also act as the final quality gateway, where any quality defects not previously detected may be uncovered. Once Software Test is completed successfully, the software is integrated with the hardware and tested together  Failure, Fault, Error A failure occurs when the observed behaviour differs from the expected one. Note that the reference is the expected behaviour, not the specification, since even the spec could be false. An error is the part of the system state which may lead to a failure. A fault is the cause of an error. A software fault lies in software, a hardware fault lies in hardware. Testing | HRO-E-DQ-SWT2 | Craiova, in January 2012 11  System reliability Hardware reliability probability a hardware component fails Software reliability probability a software component will produce an incorrect output software does not wear out software can continue to operate after a bad result Operator reliability probability system user makes an error 12  Failure, Fault, Error We tend to believe that well written, well tested, safety critical software never fails. Experience proves otherwise. Software fails even when it is produced on schedule within budget and meets the customer’s specified software requirements The Aegis radar system, for example, met all requirements that the developer and the customer had set for it. The system designer’s did not take into account the users of the software nor the conditions in which it would be used. The system was a success in terms of budget, schedule, and requirements satisfaction The user interface to the system was a primary factor in the Vincennes shooting down an Iranian commercial airliner killing 263 people. Testing | HRO-E-DQ-SWT2 | Craiova, in January 2012 13  Failure probability If there are two independent components in a system and the operation of the system depends on them both then P(S) = P(A) + P(B) If the components are replicated then the probability of failure is P(S) = P(A)n meaning that all components fail at once 14  Failure, Fault, Error Definition: Fault : An incorrect step, process, or data definition in a computer program which causes the program to perform in an unintended or unanticipated manner. A fault is a structural imperfection in a software system that may lead to the system's eventually failing It is an inherent weakness of the design or implementation which might result in a failure. Fault avoidance – using techniques and procedures which aim to avoid the introduction of faults during any phase of the safety lifecycle of the safety-related system Fault tolerance – the ability of a functional unit to continue to perform a required function in the presence of faults or errors 15  Failure, Fault, Error Error : A discrepancy between a computed, observed, or measured value or condition and the true, specified, or theoretically correct value or condition. An error is the part of the system state which may lead to a failure. E.g.: Explosion of the Ariane 5 On June 4, 1996 an unmanned Ariane 5 rocket launched by the European Space Agency exploded just forty seconds after lift-off. The rocket was on its first voyage, after a decade of development costing $7 billion. The destroyed rocket and its cargo were valued at $500 million. A board of inquiry investigated the causes of the explosion and in two weeks issued a report. It turned out that the cause of the failure was a software error in the inertial reference system. Specifically a 64 bit floating point number relating to the horizontal velocity of the rocket with respect to the platform was converted to a 16 bit signed integer. The number was larger than 32,768, the largest integer storeable in a 16 bit signed integer, and thus the conversion failed. 16  Defect masking It is the defect which is hiding the other defect, which is not detected. If there is a existing defect which is not caused (found) to reproduce another defect. The other defect is masked with the previous defect. E.g.: You are testing the Help of any application. There is a defect in one link say Add Customer. This defect was not found by QA and it is went to the live. Then there is one defect residing in the Add Customer link say the link is not working on the Add Customer page (There is bug in help of adding new cell in the help in the Add Customer) This Add cell is a masked defect. It means it is been masked with the Add Customer defect 17  Testing vs. debugging Testing What should I do to try to break my program? Debugging What should I do to try to fix my program? http://www.cs.princeton.edu/courses/archive/spr10/cos217/lectures/06Debugging. pdf 18  Testing vs. debugging testing starts with known conditions, uses predefined procedures, and has predictable outcomes. Debugging starts from possibly unknowns initial conditions and the end cannot be predicted, except statistically testing should be designed and scheduled beforehand. The procedures for, and duration of debugging cannot be constrained testing is a proof of error or apparent correctness. Debugging is a deductive process testing proves programmer's failure. Debugging is the programmer's vindication testing should strive to be predictible, dull, constrained, rigid, and inhuman. Debugging demands intuitive leaps, experimentation, intelligence, and freedom testing is done by an outsider. Deugging is made by an insider. 19  Who test pogramms Some believe that programmers can't test their own code. On the face of it, this is false: programmers test their code all the time, and they do find bugs. Just not enough of them, which is why we need independent testers 20  Difficulties of testing Testing is seen as a beginner's job, often assigned to the least experienced team members Testing often done as an afterthought (if at all) Testing cannot be "conquered"; it is impossible to completely test a system example: Space Shuttle Columbia launch, 1981 programmer set 50ms delay to 80ms, led to 1/67 chance of launch failure top-down testing ignored by shuttle guy because when error is found, impossible to fix without redesign or expense 21 Why different levels are needed? 22  Test oracle An oracle is a mechanism used by software testers and software engineers for determining whether a test has passed or failed. It is used by comparing the output(s) of the system under test, for a given test case input, to the outputs that the oracle determines that product should have. Method postconditions are commonly used as automated oracles in automated class testing. Common oracles include: specifications and documentation other products (for instance, an oracle for a software program might be a second program that uses a different algorithm to evaluate the same mathematical expression as the product under test) an heuristic oracle that provides approximate results or exact results for a set of a few test inputs a statistical oracle that uses statistical characteristics, a consistency oracle that compares the results of one test execution to another for similarity a model-based oracle that uses the same model to generate and verify system 23 behavior  Test Oracles Test Input Programs that check the Software output of other programs Under Test Input and output may be difficult to capture Test Output Test Oracle Oracle may have to parse complex I/O Pass/Fail Software Development Lifecycle V Model – Major Roles 25 Software Development Lifecycle V Model – Important documents 26 Software quality (ISO 9126) 27  References Serial Peripheral Interface Bus, Wikipedia, http://en.wikipedia.org/wiki/Serial_Peripheral_Interface_Bus Serial Peripheral Interface, SPI, http://www.rpi.edu/dept/ecse/mps/SPI.pdf What is serial peripheral interface (SPI) http://whatis.techtarget.com/definition/0,,sid9_gci959922,00.html Overview and Use of the PICmicro Serial peripheral Interface, Microchip PICmicro MCU SPI Block Guide, Motorola (2003) Programming Philips ARM LPC microcontroler family. Philips semiconductors. (***programming SPI sample code) http://www.nxp.com/documents/application_note/AN10369.pdf Configuring and Using the SPI MC9328MX1, MC9328MXL, and MC9328MXS (2005) http://cache.freescale.com/files/32bit/doc/app_note/AN3020.pdf 28 #define ON 1 #define OFF 0 int TurnOnLight (bool state, int) { int previousState; if (previousState == 0){ If (state == ON) { Cout