Software Testing Introduction PDF

INTRODUCTION OF WEEK 1 - 2 SOFTWARE TESTING SOFTWARE APPLICATION TESTING https://www.youtube.com/watch?v=E2t5XbWwj7I&list=PLL34 mf651faM_nn8uKlnwbQPw5zSh_F84&ab_channel=SoftwareT estingMentor  Software Testing  1st...

INTRODUCTION OF WEEK 1 - 2 SOFTWARE TESTING SOFTWARE APPLICATION TESTING https://www.youtube.com/watch?v=E2t5XbWwj7I&list=PLL34 mf651faM_nn8uKlnwbQPw5zSh_F84&ab_channel=SoftwareT estingMentor  Software Testing  1st software quality assurance (SQA) tool that applied to control the software product’s quality.  Formal process carried out by a specialized testing team  Testing was conducted in the final stage of development  Testing : “Execution of a work product with intent to find a defect”  Primary role of software testing? The words and phrases stressed in the definition allow us to compare the INTRODUCTION key characteristics of software testing with those of other software quality assurance life cycle tools. This is a software testing definition:  Formal process  Specialized testing team in software Unit  Examined by running the programs on a computer  All associated test performed approved test procedures and  Approved test cases  The concept of software testing has evolved from simple program “checkout” to a broad set of activities that cover the entire software life-cycle. There are five distinct levels of testing that are given below: WHAT IS ❑Debug: It is defined as the successful correction of a failure. SOFTWARE ❑Demonstrate: The process of showing that major features TESTING? work with typical input. ❑Verify: The process of finding as many faults in the application under test (AUT) as possible. ❑Validate: The process of finding as many faults in requirements, design, and AUT. ❑Prevent: To avoid errors in development of requirements, design, and implementation by self-checking techniques, including “test before design.”  Testing is NOT: ✓The process of demonstrating that errors WHAT IS are not present. SOFTWARE ✓The process of showing that a program TESTING? performs its intended functions correctly. ✓The process of establishing confidence that a program does what it is supposed to do.. The Technical Case: ❑ Competent developers are not infallible. ❑ The implications of requirements are not always forseeable. WHAT IS TESTING ❑ The behavior of a system is not necessarily predictable from its components. NECESSARY? ❑ Languages, databases, user interfaces, and operating systems have bugs that can cause application failures. ❑ Reusable classes and objects must be trustworthy.  Software Testing Life Cycle (STLC): TESTING LIFE CYLE  Basic Principle of Testing  Define the expected output or result for each test case executed.  Developers must not test their own programs.  Inspect the results of each test completely and carefully.  Include test cases for invalid or unexpected conditions which are feasible during production. TESTING  Test the program to see if it does what it is not supposed to do as PRINCIPLES well as what it is supposed to do.  Avoid disposable test case unless the program itself is disposable.  Do not plan assuming that no error will be found.  The probability of locating more errors in any one module is directly proportional to the number of error already found in the module.  To make software testing effective and efficient we follow certain principles. These principles are stated below: ❑Testing should be based on user requirements: This is in order to uncover any defects that might cause the program or system to fail to meet the client’s requirements. ❑ Testing time and resources are limited: Avoid redundant tests. ❑Exhaustive testing is impossible: As stated by Myer, it is impossible to test everything due to huge data space and the large number of paths TESTING that a program flow might take. PRINCIPLES ❑Use effective resources to test: This represents the most suitable tools, procedures, and individuals to conduct the tests. The test team should use tools like: ▪ Deja Gnu: It is a testing frame work for interactive or batch oriented applications. It is designed for regression and embedded system testing. It runs on UNIX platform. It is a cross-platform operating system. ❑Test planning should be done early: This is because test planning can begin independently of coding and as soon as the client requirements are set.  To make software testing effective and efficient we follow certain principles. These principles are stated below: ❑Testing should begin “in small” and progress toward testing “in large”: The smallest programming units (or modules) should be tested first and then expanded to other parts of the system. ❑Testing should be conducted by an independent third party. TESTING ❑All tests should be traceable to customer requirements. PRINCIPLES ❑Assign best people for testing. Avoid programmers. ❑Test should be planned to show software defects and not their absence. ❑Prepare test reports including test cases and test results to summarize the results of testing. ❑Advance test planning is a must and should be updated in a timely manner.  WEEK 1 - 2 TESTING TOOLS SOFTWARE APPLICATION TESTING  ❑Developing software to test the software is called test automation/automated testing. In simple terms, automated testing is automating the manual testing process. ❑It is used to replace or supplement manual testing with a suite of testing tools. Automated testing tools assist software testers to evaluate the quality of the software by automating the mechanical aspects of the software INTRODUCTIONS ❑Automated testing tools vary in their underlying approach, quality, and ease of use. ❑Manual testing is used to document tests, produce test guides based on data queries, provide temporary structures to help run tests, and measure the results of the tests. ❑Manual testing is costly and time-consuming. Therefore, automated testing is used to cut down time and cost.  There are an abundance of software testing tools that exist. Some of the early tools are listed below: a. Mothora: It is an automated mutation testing tool-set developed at Purdue University. Using Mothora, the tester can create and execute test cases, measure test case adequacy, determine input-output correctness, locate and remove faults or bugs, and control and document the test. TESTING b. NuMega’s Bounds Checker, Rational’s Purify: They are TOOLS runtime checking and debugging aids. They can both check and protect against memory leaks and pointer problems. c. Ballista COTS Software Robustness Testing Harness [Ballista]: It is a full-scale automated robustness testing tool. It gives quantitative measures of robustness comparisons across operating systems. The goal is to automatically test and harden commercial off-the-shelf (COTS) software against robustness failures  Because testing is of two types: a. Static testing b. Dynamic testing TYPE OF The tools used during testing are named: a. Static testing tools. TESTING b. Dynamic testing tools. TOOLS Static testing tools seek to support the static testing process whereas dynamic testing tools support the dynamic testing process. Note that static testing is different from dynamic testing. We tabulate the differences between static and dynamic testing before discussing its tools.  Static Testing V/s Dynamic Testing. TYPE OF TESTING TOOLS  Testing tools can be classified into one of the two categories listed below: Static Test Tools: These tools do not involve actual input and output. Rather, they take a symbolic approach to testing, i.e., they do not test the actual execution of the software. These tools include the following: ▪ Flow analyzers: They ensure consistency in data flow from input to output. ▪ Path tests: They find unused code and code with contradictions. ▪ Coverage analyzers: It ensures that all logic paths are tested. TYPE OF ▪ Interface analyzers: It examines the effects of passing variables and data TESTING between modules. Dynamic Test Tools: These tools test the software system with “live” data. TOOLS Dynamic test tools include the following: ▪ Test driver: It inputs data into a module-under-test (MUT). ▪ Test beds: It simultaneously displays source code along with the program under execution. ▪ Emulators: The response facilities are used to emulate parts of the system not yet developed. ▪ Mutation analyzers: The errors are deliberately “fed” into the code in order to test fault tolerance of the system  1. Automated execution of test cases is faster than manual execution. This saves time. This time can also be utilized to develop additional test cases, thereby improving the coverage of testing. 2. Test automation can free test engineers from mundane tasks BENEFIT OF and make them focus on more creative tasks. TESTING 3. Automated tests can be more reliable. This is because manually running the tests may result in boredom and fatigue, more TOOLS chances of human error. While automated testing overcomes all these shortcomings. 4. Automation helps in immediate testing as it need not wait for the availability of test engineers. In fact, Automation = Lesser Person Dependence  5. Test cases for certain types of testing such as reliability testing, stress testing, and load and performance testing cannot be executed without automation. For example, if we want to study the behavior of a system with millions of users logged in, there is no way one can perform these tests without using automated tools. 6. Manual testing requires the presence of test engineers but automated tests can be run around the clock, in a 24 × 7 BENEFIT OF environment. 7. Tests, once automated, take comparatively far less resources to TESTING execute. A manual test suite requiring 10 persons to execute over 31 days, i.e., 31 × 10 = 310 man-days, may take just 10 man-days for TOOLS execution, if automated. Thus, a ratio of 1: 31 is achieved. 8. Automation produces a repository of different tests which helps us to train test engineers to increase their knowledge. 9. Automation does not end with developing programs for the test cases. Automation includes many other activities like selecting the right product build, generating the right test data, analyzing results, and so on.  1. An average automated test suite development is normally 3-5 times the cost of a complete manual test cycle. 2. Automation is too cumbersome. Who would automate? Who would train? Who would maintain? This complicates the matter. 3. In many organizations, test automation is not even a DISADVANTAGES discussion issue. OF TESTING 4. There are some organizations where there is practically TOOLS no awareness or only some awareness on test automation. 5. Automation is not a high priority item for management. It does not make much difference to many organizations. 6. Automation would require additional trained staff. There is no staff for the purpose of automation.  The main criteria for the selection of test tools are given below: CRITERIA OF 1. Meeting requirements SELECTION 2. Technology expectations TOOLS 3. Training/skills 4. Management aspects  Meeting requirements a. There are many tools available in the market today but rarely do they meet all the requirements of a given product or a given organization. Evaluating different tools for different requirements involves lot of effort, money, and time. A huge delay is involved in selecting and implanting test tools. b. Test tools may not provide backward or forward compatibility with CRITERIA OF the product-under-test (PUT). c. Test tools may not go through the same amount of evaluation for SELECTION new requirements. Some tools had Y2K-problems too. TOOLS d. Several test tools cannot distinguish between a product failure and a test failure. This increases analysis time and manual testing. The test tools may not provide the required amount of trouble- shooting/ debug/error messages to help in analysis. For example, in the case of GUI testing, the test tools may determine the results based on messages and screen coordinates at run-time. So, if the screen elements of the product are changed, it requires the test suite to be changed. The test tool must have some intelligence to proactively find out the changes that happened in the product and accordingly analyze the results.  Technology expectations a. In general, test tools may not allow test developers to extend/modify the functionality of the framework. It involves going back to the tool vendor with additional cost and effort. Very few tools available in the market provide source code for extending functionality or fixing some problems. Extensibility and customization are important expectations of a test tool. b. A good number of test tools require their libraries to be linked with product binaries. When these libraries are linked with the source code CRITERIA OF of the product, it is called the “instrumented code.” This causes portions of testing to be repeated after those libraries are removed, as the result of certain types of testing will be different and better when SELECTION those libraries are removed. For example, the instrumented code has a major impact on performance testing because the test tools introduce TOOLS additional code and there could be a delay in executing the additional code. c. Finally, test tools are not 100% cross-platform. They are supported only on some OS platforms and the scripts generated from these tools may not be compatible on other platforms. Moreover, many of the test tools are capable of testing only the product, not the impact of the product/test tool to the system or network. When there is an impact analysis of the product on the network or system, the first suspect is the test tool and it is uninstalled when such analysis starts.  Training/skills Test tools require plenty of training but very few vendors provide the training to the required level. Organization- level training is needed to deploy the test tools, as the CRITERIA OF users of the test suite are not only the test team but also the development team and other areas like SCM SELECTION (software configuration management). TOOLS Test tools expect the users to learn new language/scripts and may not use standard languages/scripts. This increases skill requirements for automation and increases the need for a learning curve inside the organization.  Management aspects A test tool increases the system requirement and requires the hardware and software to be upgraded. This increases the cost of the already-expensive test tool. When selecting the test tool, it is important to note the CRITERIA OF system requirements and the cost involved in upgrading the software and hardware needs to be included with the cost of SELECTION the tool. TOOLS Migrating from one test tool to another may be difficult and requires a lot of effort. Not only is this difficult as the test suite that is written cannot be used with other test tools but also because of the cost involved. The tools are expensive and unless management feels that the returns on investment (ROI) are justified, changing tools are generally not permitted.  Step 1. Identify your test suite requirements among the generic requirements discussed. Add other requirements, if any. Step 2. Make sure experiences discussed in previous sections are taken care of. Step 3. Collect the experiences of other organizations which used similar test tools. STEP OF Step 4. Keep a checklist of questions to ask the vendors on TOOLS cost/effort/support. SELECTION Step 5. Identify a list of tools that meet the above requirements and give priority for the tool that is available with the source code. Step 6. Evaluate and shortlist one/set of tools and train all the test developers on the tool. Step 7. Deploy the tool across test teams after training all the potential users of the tool. TESTING  WHICH ARE SUITABLE WITH THIS TOOLS COURSE?? Set x to 1 Display “MULA!” While (x < 400) Display X X=x * 5 Display “MALATOPS!” Line/St x While x=x * 5 Display / ep x