Week 09 - Testing CAB201 PDF

Testing CAB201 – Programming Principles School of Computer Science, Faculty of Science TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science ...

Testing CAB201 – Programming Principles School of Computer Science, Faculty of Science TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  ACKNOWLEDGEMENT OF TRADITIONAL OWNERS QUT acknowledges the Turrbal and Yugara, as the First Nations owners of the lands where QUT now stands. We pay respect to their Elders, lores, customs and creation spirits. We recognise that these lands have always been places of teaching, research and learning. ​ QUT acknowledges the important role Aboriginal and Torres Strait Islander people play within the QUT community.​ TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  ERROR HANDLING TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  Error Handling Unexpected things can happen during the execution of your program Users will type funny things into your program, either accidentally or intentionally Files will become partially corrupted during transmission and users will supply them as input to your program Users will run your program on files on a network or USB drive and the connection will fail / the USB drive will be pulled out partway through Users will run your program with input files that are orders of magnitude larger than you expected, leading to possible memory issues TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  Error Handling Given that these things will happen, you need to write programs in such a way that they can handle these things gracefully What constitutes graceful depends on the context For example, it is usually not a good thing for an entire GUI program to die with a cryptic error message because you tried opening the wrong type of file with it On the other hand, for a command-line tool that reads in a file and performs one operation on it, this may be acceptable TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  Traditional Error Handling Traditionally, error handling involved using special return values from methods to indicate that a problem occurred We still use this (e.g. int.TryParse); in some situations, it is the most appropriate approach This is the standard error handling approach used in languages like C Works very well when the program is simple, the call stack is mostly flat and the range of unexpected occurrences is limited TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  Issues with Traditional Error Handling Traditional error handling becomes clumsy when there is a gap between the place where the error occurs and the place where we can do something about it Often, error handling means leaving the code path we were previously on and going down an alternate code path This can be complicated when the call stack is deep It may result in large amounts of error handling code being added to our program, making it less readable Traditional error handling can result in a performance cost when error codes need to be checked and re-checked Error handling complicates the object model TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  EXCEPTIONS TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  Exceptions Exceptions are used as part of the normal error handling of an application They are still present during Release builds Conceptually: When an error occurs the call stack is unwound until code that can handle that error is found The handling code can then make a decision as to whether to handle the error (and how to handle it) or to allow the stack to continue unwinding until code that can handle it is found Has performance benefits compared with traditional error handling if used properly, but can be disastrous if misused TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  Exceptions Exceptions are undesirable events that are outside the "normal" behaviour of a program Intended behaviours should not be coded using exceptions Advantages: Error handling is kept separate from normal code Exceptions may be able to be recovered from Can be dealt with at the source or propagated up Easily able to distinguish between / deal with different kinds of exceptions (inheritance) Can occur in constructors TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  Exceptions in C# Exception classes inherit (directly or indirectly) from System.Exception You can create your own exception types by inheriting from Exception Code that raises an exception is said to throw the exception Can be thrown explicitly Can also occur naturally (e.g. divide by zero) A try / catch block is used to handle exceptions: If code in the try block results in an exception, the catch block will have an opportunity to handle it TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  Stack unwinding When an exception is thrown, the normal processing of that method is interrupted Usually the throwing of an exception means that normal processing of that method cannot continue (for example, a file failed to open and the rest of the method deals with reading from that file) If the code is inside a try block, the catch block will be checked to see if it applies to this exception If it does, the code in the catch block will execute If there is a finally block, the code in it will be executed no matter what (after the catch block code is, if the exception was caught) TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  Stack unwinding Otherwise, if the exception still has not been handled, the method will immediately exit and control will return to the calling method Note that, in this case, the method will not return anything and the calling method will proceed to propagate the exception in the same way as before – by looking at the outer enclosing try block, and if there isn't one (or the catch block does not handle the exception type) the exception will continue moving up until something finally handles it If no code handles an exception, the entire program will terminate TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  Visual: Exception handling and stack unwinding Main try{MethodA();} 1. Call to catch(Exception e) {…} MethodA MethodA 6. Exception handler found in MethodA try{MethodB();} 2. Call to catch(Exception e) {…} MethodB MethodB 5. No exception handler in MethodB MethodC(); 3. Call to MethodC MethodC 4. No exception throw new Exception(); handler in MethodC TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  Cleanup During stack unwinding, normal execution of code ceases, which means that calls to important cleanup methods (e.g. for deallocating resources, closing files, closing database / network connections) do not go through Two approaches can be used to avoid this problem: finally If an exception is raised inside a try block and there is a finally block attached, the code inside will run no matter what. From here we can issue appropriate cleanup calls. using If a resource is allocated in a using statement, it will be disposed when the code leaves that block, even during exception handling TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  Appropriate use of exceptions Code with exceptions is fast on the non-exceptional path and slow on the exceptional path This is a design decision within the language to make exceptions have as little impact on code as possible when they are not actively being thrown and propagated If unusual / invalid values are expected as a matter of course, consider using error codes (e.g. int.TryParse) instead of exceptions Choose the right kind of exception when throwing one, and include a useful message throw new ArgumentException("x must be positive"); TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  Presented by Dr Lawrence Buckingham INTRODUCTION TO TESTING TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  Review software development life cycle TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science 18  Software development (1) Requirement discovery: Collect user stories from customer to capture requirements. Analysis: Process user stories to identify nouns and verbs. Nouns will become classes or attributes in classes. Verbs will become methods. Identify other rules and constraints. Design: Assign attributes and methods to classes. Identify the programmatic interface of each class. Identify collaboration between classes. Capture the essence of the model in class diagrams or stubbed-out interfaces and classes in source code. TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  Software development (2) Implementation: Write the source code and compile the program. Testing: Verify that the program functions correctly and matches user expectations. i.e. the program meets requirements. Deployment: Deliver the finished system. Maintenance: Identify and fix defects. Implement new functionalities as customer requirements evolve. 90% of the system’s lifespan will be spent here. TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science 20  Requirements – functional vs non- functional Functional requirements specify what a system is supposed to do. Non-functional requirements specify how the system should do it. Some examples are listed below. Functional requirement Non-functional requirement A social media app should allow users to A social media app should be able to handle create a profile. 10,000 users at the same time. A banking app should allow users to transfer A banking app should keep user data secure. money between accounts. A weather app should display current A weather app should load quickly and update temperature, rainfall, and wind details. in a timely manner. TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  Testing Testing is a process in which we evaluate an application to verify that it conforms to requirements. We systematically ask “What might go wrong?”, and set up tests to check these things. Testing can help us identify and eliminate defects, and locate performance issues in our software. Testing also gives us an objective measure of how well the product meets requirements. If defects are discovered after the system is delivered to the customer, we can set up tests to reproduce the incorrect behaviour, and use those tests help us verify that errors have been corrected. Limitations: We’re limited by our ability to foresee possible error conditions. In a complex program it is impossible to exhaustively test all combinations of inputs. TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science 22  Alternatives to testing Before we look in detail at testing, remember that testing is not the only way to verify the correctness of a system. Some other approaches include: Static analysis: Analyse the source code to look for defects and vulnerabilities Software tools can help find memory leaks, buffer overruns, null pointer dereferences, and other problems. Model checking: Create a mathematical model of the system and use the model to verify that the system meets specification. Code reviews: Other developers review the code (often in a group setting) to look for problems. Code reviews can find logic errors, security issues, and performance problems. Pair programming: Two developers work together on the same source code. One person types code while the other reviews it in real time. Can help avoid logic errors and design problems. TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science 23  Kinds of testing (1) Unit testing: Tests individual units or components of the system in isolation. A unit is usually a single method or a single class. Usually done by developers. Ensures that each individual component is working correctly before being used with other components. There are two kinds of unit test: Black box tests exercise the specification and contract, but do not take implementation details into account. Glass box tests are developed by looking at the source code of the implementation to identify special cases which may cause problems. Integration testing: Tests how components work when used together. Ensures that collaborating components interact correctly. System testing: Tests the system as a whole. Make sure the system meets functional and non-functional requirements. TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  Kinds of testing (2) Acceptance testing: Determines whether the system meets customer or end user expectations. Usually done by customer or end user. Confirms that system is ready for deployment. Regression testing: Test to ensure that changes to code do not break existing functionality. That is, everything that worked before a change still works after the change. Performance testing: How well the system performs under different workloads and conditions. Confirms that system meets performance requirements. Security testing: Test for security flaws such as buffer overruns, SQL injection, cross-site scripting vulnerabilities. TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science 25  Test plan (general idea) We use a test plan to guide the testing process. The test plan is a document that outlines the objectives, scope, approach, and schedule of test activities. Describes how testing will be performed at a particular level, for example, unit testing or system testing. Provides a blueprint which allows testing to be carried out as a rigorous and controlled process. Sets out the objectives, i.e. what you’re trying to verify. Defines the scope, i.e. what will be covered, and what will not be covered. Includes details of the activities that will be carried out. TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science 26  Test plan (for unit testing) A table is often the best way to present a unit test plan. The table will have one row for each test, and usually include the following columns: 1. Test number – a unique identifier for each test. 2. Description – a brief summary of what the test is about. 3. Test categories – a list of tags or labels which you can assign to each test to help keep them organised. The labels should be fairly standardised and descriptive, e.g. “Boundary”, “Invalid”, “Valid”, “Regression”. They should give you an idea about the purpose of the associated test, but also be useful for grouping or filtering tests. 4. Setup – detailed instructions which say how to prepare for the test and what input data is used in the test. Input data should be concrete and reproducible, including actual numbers or input strings wherever possible. It is not satisfactory to say “Run the program with valid data”, or other similarly vague statements. 5. Expected outcome – a detailed and specific description of what the correct results should be. The expected outcome should be concrete and measurable, using actual numbers or output strings wherever relevant. Avoid imprecise statements such as “The correct value should be returned”. 6. Status – “pass” or “fail”. You may include more detail here if it is useful. TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science  Presented by Dr Lawrence Buckingham MSTEST TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science 28  Overview MSTest is a unit testing framework that is included with Visual Studio. It is used to automate the unit testing process. The framework provides a namespace Microsoft.VisualStudio.TestTools.UnitTesting. The namespace contains classes and attributes which can be used to code and execute unit tests. TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science 29  MSTest Attributes TestClass – indicates that a class contains unit tests. The framework will look for unit tests in the class. TestMethod – indicates that a method is a unit test. The framework will execute the method automatically. TestCategory – assigns a category to a unit test. Each unit test may belong to multiple categories. TestInitialize – indicates that a method must be executed before every test in the containing class starts. ExpectedException – indicates that we really want a particular type of exception to be thrown. Timeout – set the maximum time a test is allowed to run. DataTestMethod – indicate that the test wil be called multiple times, with different inputs. DataRow – supply input to a DataTestMethod. TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science 30  General structure of a unit test A test usually has three phases: Setup – lays the groundwork to allow a test to be executed. This is where we create any necessary objects, define the input, and expected output. Run – the unit under test is executed and we get the actual output. Assert – we use methods from the Assert class to insist that the required outcome has been achieved. Usually, we assert that the actual output is equal to the expected output. Sometimes we don’t have to, for example when we use ExpectedException to look for an exception to be thrown. TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science 31  Presented by Dr Lawrence Buckingham CASE STUDY 1 – INCOME TAX TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science 32  Concepts illustrated: This case study illustrates how to: Formulate a unit test plan for a function. Select suitable input values, and pre-compute the expected output for each. Use MSTest to automate the test process. Verify that correct output is produced by a method. Verify that an expected exception is thrown. Use data driven tests to process multiple tests with minimal effort. In the accompanying video, you will learn how to use the debugger with MSTest to help find errors. TEQSA Provider ID PRV12079 Australian University | CRICOS No.00213J CAB201 – Programming Principles School of Computer Science, Faculty of Science 33  Functional requirements Australian personal income tax is defined by a piecewise linear function. Excel formula (2022-2023 financial year): =IF([@Income]