Software Testing Fundamentals

• Software Testing checks if the software matches expected requirements and is defect-free.
• Testing involves using manual or automated tools to evaluate software/system components.
• The purpose of software testing is to identify errors, gaps, or missing requirements.
• Testing helps identify correctness, completeness, security, quality, and reliability to determine if software is useful for its intended purpose.

Testing Tasks

• Determine if documentation, processes, and programs work as expected.
• Testing can show the presence of bugs but not the absence of them.
• Companies can save millions per year via good testing techniques and QA processes.
• Early software testing uncovers problems before a product goes to market.
• Early detection of architectural flaws, poor design decisions, invalid/incorrect functionality, security vulnerabilities, and scalability issues is ideal.
• Faults are differences occur from expected behavior; they are caused by defects in software.
• When a fault is discovered, classify its urgency based on whether colors are not displayed correctly or a cash machine is dispensing incorrect amounts.
• Locating a programming fault involves debugging the program.
• The next step may to ignore it, realize the requirements are incorrect or fix it.
• Testing can reveal errors in coding, increase confidence in software, measure performance, and indicate quality.
• Testing is not a substitute for quality, but it reveals poor quality systems.

Testing Activities

• Software Verification
• Validation and Testing
• Black Box Testing
• White Box Testing

Testing Roles

• Tester
• Test Designer
• Test/QA Team Lead
• Test Build Manager
• Test Configuration Manager
• Test automater/automation developer
• Test administrator
• All aspects of a computer system should be tested
• Testing of aspects include translations, installation, fault logging & tracking, payment systems and backups!
• Requirements regarding reality, previous systems (feasibility analysis), self-consistency, and user feedback needs tested
• Analysis includes relationships in the problem and ability to meet all requirements with the model.

Design Considerations for Testing

• Determining if the system can be implemented within given time constraints
• Assessing if the design can be built within budget
• Implementation entails confirming each class works as specified and that the classes work together properly
• Documentation testing involves verifying user understanding and matching functionality of the system

Types of Software Testing

• Functional Testing
• Non-Functional Testing
• Maintenance Testing

Functional Testing Sub-Types

• Unit Testing
• Integration Testing
• Smoke Testing
• UAT (User Acceptance Testing)
• Localization
• Globalization
• Interoperability

Non-Functional Testing Sub-Types

• Performance
• Endurance
• Load
• Volume
• Scalability
• Usability

Maintenance Testing Sub-Types

• Regression
• Maintenance

Unit Testing Explained

• Checks if low-level modules work in isolation.
• Modules typically interact with other modules, which requires creating a framework to support them.
• Replace modules can be replaced with simple versions which are for easy coding, which return values to test sufficiently.
• Code is required to call the module and is called a test harness.

Test Harness Elements

• Test Execution Engine: Runs test cases automatically.
• Test Data Management: Provides necessary input data.
• Logging and Reporting: Captures test results and logs issues.
• Stubs and Drivers: Simulates missing components or dependencies.
• Configuration Management: Ensures consistency in test environments.
• PyTest test harness example: If a function adds two numbers, then the harness automates testing, logs results, and reports failures.

Integration Testing

• After unit tests, units are combined to form larger groups
• Integration tests see if components communicate correctly and provide correct behaviour
• Establish pre/post conditions/invariants
• Identify requirements that components must satisfy.

System Testing

• System testing tests components delivered to customer
• These tests are based against use cases and scenarios created during requirements.
• System testing validates the system's functionality, performance, and reliability across the entire system, including its components and interfaces.
• Focus involves checking system-wide issues, performance bottlenecks, and usability problems on real-world user conditions.

Integration Testing vs. System Testing

• Integration tests focus on component interactions, using simulated environments, to verify data exchange correctness
• Software errors often arise from not knowing the language (i.e., not understanding “break”), typographical errors, problem solving logic or poor results.
• Errors also arise from critical input values, data structure defects, or misinterpretation of specifications.
• When computing with floating point numbers or static code checking tools, errors can occur.

White Box vs Black Box Testing

• White box testing involves writing tests for internal aspects of the class, such as if statements and algorithms.
• White box testing involves code reviews or pair programming.
• Black box testing ignores internal implementation details
• Black box testing concentrates on the service that the modules offer and tests sequences of calls into the object.
• Testers should not have information on the internal system workings.
• Black box testing focuses on the behavior of the software from an external or end-user perspective.
• White box testing looks at branching points where decisions are made, like if statements, loops, or switch statements.
• Boundary conditions for variables, such as testing for a month value outside of range, should be tested.

Testing Common Faults

• "Bugs lurk in corners and congregate at boundaries"
• Boundary conditions are where the behaviour of the system changes dramatically from one state to another.
• Test cases should ensure "a" is set to values above and below boundary conditions.
• Modern methods utilize lots of small changes which ensures software is always runnable.
• Modern methods encourage automated testing.

Testing Realities

• Unit testing has limits of defects discovered.

Release Criteria

• Testing is never a substitute for developing software with a good process, to identify a program's divergence from requirements
• Not all faults need to be fixed.
• Automated testing simplifies testing and development.
• Companies separate testing team roles since developers are blind to problems.
• Testers often work under extreme pressure as the last line in the product release cycle.

Testing Roles Breakdown

• Tester: Executes tests and reports results.
• Test Designer: Creates test cases and scenarios.
• Test/QA Team Lead: Manages the testing team.
• Test Build Manager: Oversees the test build process.
• Test Configuration Manager: Manages test environments.
• Test automater/automation developer: Develops automation scripts.
• Test administrator: Manages test infrastructure.
• Correct faults need to be identified, recorded and tracked on which version of the software they occur.
• Fault tracking software helps automate things with things like Roundup and TestTrackPro

Release Criteria Factors

• Define criteria to check before agreeing to release, including defining success, providing customer value, matching company needs, and using SMART criteria (Specific, Measurable, Attainable, Relevant, Trackable).
• Ensure "SMART" criteria is similar to requirements.
• Example criteria states all code must compile, no high-priority bugs, documentation for all bugs, QA tests, and decreasing defects.

Test Scenario Generation Steps

• Read requirement documents related to the System Under Test (SUT),.
• Figure out user actions/objectives for each requirement.
• Determine technical aspects and system abuse scenarios.
• List test scenarios to verify features and create a traceability matrix to ensure each requirement has a test scenario.
• Have scenarios reviewed by a supervisor and stakeholders.