Module 1 - ISTQB v4.0 - Fundamentals of Testing - Final Version 02-2024.pptx

ISTQB Certified Tester Foundation Level Module 1 – Fundamentals of Testing fdmgroup.com  After completing this module, you will be able to: Understand what testing is Exemplify why testing is neces...

ISTQB Certified Tester Foundation Level Module 1 – Fundamentals of Testing fdmgroup.com  After completing this module, you will be able to: Understand what testing is Exemplify why testing is necessary Module Objectives Explain the seven testing principles Describe the test activities, testware and test roles Identify the essential skills and good practices in testing  Session Aims Why is Testing Testing Test Activities, Essential Skills and What is Testing? Necessary? Principals Testware and Good Practices Test Roles in Testing fdmgroup.com Why is Testing Becoming More Important? Everyday life is dependent on software Business applications e.g. Banking Consumer products e.g. Cars Increasing software complexity Quality as a competitive advantage Software errors can be very expensive fdmgroup.com  What do Software Defects Cause? Software that does not work correctly can lead to many problems Loss of Money Loss of Time Loss of Business reputation Injury or Death Software testing assesses software quality and helps reduce the risk of software failure in operation fdmgroup.com  In OutApril 2018 millions in December the field of TSBseencustomers were locked out of their accounts after In In May 2017, a we have large 2018, ransomware more than 30 attack million called WannaCry O2 users in the(also known UK lost as access atonumber of an IT upgrade examples ofled to an online banking outage. WannaCrypt0r and data services WCry) after ahit NHS England software andthem issue left various organisations unable to use 3Gin the4G and UK software defects. We willand lookaround the world. services. MoreA atplanned info can some system be upgrade found of these at : stories was expected to shut internet and mobile banking toin April 2018, but ended up causing months of services down for one https://www.computerworlduk.com/gall weekend TheCustomers help attack wasofdue you avoid O2 to and these vulnerabilities inthe the found in using other companies Microsoft operating its network systems including: The nationwide outage disruption. also affected Transport for London's eries/infrastructure/top-software- installed future… in millions of computers around the world.live electronic failures-recent-history-3599618/timetables TescoatMobile bus stops. The problems arose from TSB's move to a new banking platform following its According to Microsoft, the Windows versions Sky mobile that were vulnerable to the attack split from Lloyds Banking Group. were versions which were no longer supported Lycamobileby Microsoft such as Windows 8 and Windows XP, which the NHS trusts and Giffgaff O2 subsidiary affected companies seemed to Immediately after the new system was switched on, many customers be running. experienced problems logging in, while others were shown details from other Where affected after a software glitch in equipment supplied by ericsson people's accounts or inaccurate credits and debits on their own. Customers brought the network to its knees. remained locked out of their accounts two weeks after the initial outage. NEWS TS fdmgroup.com fdmgroup.com  What is Software Testing? Software Testing is a set of activities to discover defects and evaluate the quality of the test object (the application) It is a way to assess the quality of software and reduce the risk of software failure during operation Software Testing needs to be properly planned, managed, estimated, monitored and controlled fdmgroup.com  How Do We Test? Dynamic Static Carrying out tests without Carrying Carryingout outtests byexecuting tests by executing executing the software the the software softwareproduct product product Examples Examples include blackand include black andwhite boxbox testing Examples include reviews white testing and static analysis fdmgroup.com  What is involved in Testing? Verification Validation Is the work product meeting Is the work product meeting Carrying out tests by executing the the requirements software product set out for it? the customers needs? Are we building Examples the include application black and white Are we building the RIGHT RIGHT? box testing application? fdmgroup.com  What are the objectives of testing? Evaluating work products such as requirements, user stories, designs and code Triggering failures and finding defects Ensuring required coverage of a test object Reducing the level of risk of inadequate software quality Verifying whether specified requirements have been fulfilled Verifying that a test object complies with contractual, legal and regulatory requirements Providing information to stakeholders to allow them to make informed decisions Building confidence in the quality of the test object Validating whether the test object is complete and works as expected by the stakeholders fdmgroup.com  Varying Test Objectives Context Test objectives Test Level can vary based on a number of Risks factors including: Software development lifecycle model used Corporate structure Competitive considerations Time to market fdmgroup.com  Testing and Debugging Testing Debuggin Performed by testers g Performed by developers Systematic process of finding and When a defect is found using reporting defects within dynamic testing the debugging applications process involves: Reproduction of failure Once defects are fixed by Finding the root cause developers defect re-testing Fixing the cause (confirmation testing) will occur When static testing finds issues Subsequent regression testing in the code they can be fixed can also be performed to check straight away with no need for whether the fixes are causing reproduction of the issue or failures in other parts of the test diagnosis object fdmgroup.com  Quiz Time Undertake The Mentimeter quiz Navigate to Enter the code www.menti.co from your m trainer fdmgroup.com  Session Aims Why is Testing Testing Test Test Process Activities, Essential Skills and What is Testing? Necessary? Principals Testware and Good Practices Test Roles in Testing fdmgroup.com  Why is Testing Necessary ? Testing helps in achieving agreed upon goals within the set scope, time, quality and budget constraints Any stakeholder can use their testing skills to bring the project close to success Testing components, systems and documentation helps to identify defects in software fdmgroup.com Testing’s Contribution to Success Testing provides a cost effective means of detecting defects which then improves the test object Testing provides a means of directly evaluating the quality of the test object at various stages of the SDLC The measures contribute to decisions to move to the next stage of the SDLC, such as release decisions Testers ensure that their understanding of users’ needs are considered throughout the development lifecycle Business representatives may not be available to check the product due to high costs and lack of availability of suitable users Testing may also be required to meet contractual/legal requirements or comply to regulatory standards fdmgroup.com  Quality Control Simulation and Prototyping are some Quality Control is a product- examples of Quality Control orientated, corrective approach Activities support the achievement of the appropriate levels of quality Testing is a major form of quality control Test results are used to fix defects fdmgroup.com  Quality Assurance If a good process is followed correctly, then it will generate a good Quality Assurance (QA) is a process- product orientated, preventative approach Focuses on the implementation and improvement of processes Is the responsibility of everyone on the project Applies to both development and testing processes Test results for QA provide feedback on how well the development and test processes are performing fdmgroup.com  Errors, Defects and Failures A developer makes a mistake/error when writing the code e.g. a semi colon in the wrong place A human makes an Errors by humans can occur for a number of reasons, ERROR including: Time pressure Complexity of work products, processes or infrastructure Tiredness Lack of training fdmgroup.com  Errors, Defects and Failures The ERROR creates a DEFECT in the Defects can be found in documentation, in source software code or in supporting artifacts such as a build file Defects produced earlier in the lifecycle if undetected, often leads to defective artifacts later in the lifecycle Defects are also known as faults or bugs fdmgroup.com  Errors, Defects and Failures If a defect in code is executed, the system may fail to do what it should hence causing a failure The ERROR might cause a FAILURE Some defects may cause a failure under specific in operation circumstances and some may not cause a failure Failures can also be caused by environmental conditions for example when radiation or electromagnetic fields cause defects in firmware fdmgroup.com  Root Causes of Defects A root cause is a fundamental reason for the occurrence of a problem (e.g. a situation that leads to an error) Root causes are found via root cause analysis which is carried out when a defect or failure occurs Further similar defects can be prevented by addressing the root cause Problem Issue Classification Developer misunderstands stakeholder Root cause of a problem requirement Developer writes code for the functionality that Error doesn’t fulfil customer requirement correctly Code will now create functionality that is not Defect correct based on the customers needs Application doesn’t meet customer needs when Failure tested fdmgroup.com  Quiz Time Undertake The Mentimeter quiz Navigate to Enter the code www.menti.co from your m trainer fdmgroup.com  Session Aims Why is Testing Testing Test Activities, Essential Skills and What is Testing? Necessary? Principals Testware and Good Practices Test Roles in Testing  Testing Principles Seven general testing principles have been developed over the past 50 years and offer general 7 guidelines common for all testing Principles Their purpose is to guide testers and prevent the types of problems we have already looked at Very useful to have a good understanding of all seven if you want to be a successful tester fdmgroup.com  Testing Principles Testing shows the presence of defects, not their absence Testing can show that defects are present in an application but cannot prove that there are no defects. Principle Testing reduces the probability of undiscovered defects remaining in the software but, even if no defects are 1 found, testing is not a proof of correctness. Definition of a working system: A system where the next defect has not yet been found fdmgroup.com  General Testing Principles Exhaustive testing is impossible Testing all input combinations is not feasible Risk analysis, test techniques and priorities should be Principle used to focus test efforts For example a field on a website should only accept a 2 valid date and time between 01/01/12 00:00:00 and 31/07/12 23:59:59 inclusive How many tests to be exhaustive? 18,403,200 tests would be needed to cover all inputs! fdmgroup.com  General Testing Principles Early testing saves time and money Principle 3 Defects found early are cheaper to fix Most significant defects are found first Defects are prevented not built in Sometimes referred to as shift left fdmgroup.com  Testing Principles Defects cluster together A small number of system components usually contain most of the defects discovered Principle Helps with risk-based testing as you would focus your testing on the areas that have yielded the greater amount of defects 4 fdmgroup.com  Testing Principles Tests wear out If the same tests are repeated over and over again they become ineffective in detecting new defects Principle May need to change test data or create new tests altogether 5 In some cases repeating tests can be beneficial i.e. automated regression testing fdmgroup.com  Testing Principles Testing is context dependent Testing is done differently in different contexts For example, compare the testing required for a Principle video game against the testing required for an air traffic control system 6 Also, consider the approach for project delivery as testing is done differently in an Agile environment when compared to a Waterfall environment fdmgroup.com  Testing Principles Absence-of-defects fallacy (misconception) Thoroughly testing all the requirements and fixing all the defects found… …could still produce a system that does not fulfil Principle the user’s needs and expectations Defects Requirements 7 fdmgroup.com  Quiz Time Undertake The Mentimeter quiz Navigate to Enter the code www.menti.co from your m trainer fdmgroup.com  Session Aims Why is Testing Testing Test Activities, Essential Skills and What is Testing? Necessary? Principals Testware and Good Practices Test Roles in Testing fdmgroup.com  The Test Process Testing is context dependent, but there are common sets of test activities without which testing is less likely to achieve its objectives There is no one universal test process! These activities form a test process that can be tailored Which test activities are included in the test process is usually decided in the test planning stage Activities are often implemented iteratively or in parallel fdmgroup.com  The Test Process in Context Contextual factors that influence the test process for an organisation include but are not limited to: Stakeholders Project constraints Team members Organisational factors Business domain Software development lifecycle Technical factors Tools fdmgroup.com  The Test Process Test Planning Test Monitoring and Control Test Analysis Test Design Test Implementation Test Execution Test Completion fdmgroup.com  Test Planning Exercise Exercise What is a Test Plan? Research the necessity and describe the content of a Test Plan (testing roadmap) you to 15 minutes have complete fdmgroup.com Test Monitoring and Control Test Monitoring is the ongoing comparison of actual progress against test plan using monitoring metrics Test Control involves taking actions necessary to meet the STOP GO objectives of testing fdmgroup.com  Test Analysis Let me tell you about TheTest Analysis Test Basis is analysed to identify testable features and define/prioritise associated test conditions What? The Yes,Test Basis is also evaluated to identify What! defects That it may is “what we contain along with testability are going to test” Often supported by test techniques (see Module 4) fdmgroup.com  Test Design That doesn’t tell me Test Design does! “how we are Test conditions are going to test” elaborated into high level test cases, often using test design techniques What about Test data requirements, Test environment needs, Infrastructure and We define those during tools required? Test Design too fdmgroup.com  Test Implementation Creates all the Testware necessary for test execution which involves: Developing and prioritising the test procedures and potentially creating automated test automated scripts procedures scripts test automated procedures scripts fdmgroup.com  Test Implementation Creates all the Testware necessary for test execution which involves: test Creating test suites from the test procedures and arranging suites test execution these into an efficient test execution schedule schedule test test execution suites schedule test automated procedures scripts fdmgroup.com  Test Implementation Creates all the Testware necessary for test execution which involves: Building thetest test environment and ensuring it is setup correctly environment test environmen t test test execution suites schedule test automated procedures scripts fdmgroup.com  Test Implementation Creates all the Testware necessary for test execution which involves: Preparing thetest test data and ensure it is properly loaded into the test environment data test test environmen data t test test execution suites schedule test automated procedures scripts fdmgroup.com  Test Execution Test suites are run in accordance with the test execution schedule Can be manual or automated Can take many forms including continuous testing or pair testing sessions Actual test results are compared with expected test results Test results are logged Anomalies are analysed to identify their causes Anomalies are reported fdmgroup.com  Test Completion Occurs at project milestones (e.g. release, end of iteration, test level completion) and includes: Checking whether all defects are closed and taking appropriate action for any defects still open Creating a test completion report to be communicated to stakeholders Test environment is shut down to an agreed state Testware is identified and archived or handed to the appropriate team Analysing lessons learned from testing to support improvement of future iterations, releases or projects fdmgroup.com  Test Work Products Test work products are created as part of the test process and the types of products created will vary depending on the organisation Test Planning Work Products: Test plan, test schedule, risk register and entry and exit criteria Risk register is a list of risks together with risk likelihood, risk impact and risk mitigation Test Monitoring and Control Work Products: Test progress reports Documentation of control directives Risk information Test Analysis Work Products: Test conditions that are prioritised Defect reports regarding defects in the test basis (requirements) fdmgroup.com  Test Work Products Test Design Work Products: Test cases, test charters, coverage items, test data and test environment requirements Test Implementation Work Products: Test procedures, automated test scripts, test suites, tests data, test execution schedule and test environment elements Examples of test environment elements include: stubs, drivers, simulators and service virtualisations Test Execution Work Products Test logs Defect reports Test Completion Work Products: Test completion report Action items for improvement of future projects/iterations Documented lessons learnt Change requests fdmgroup.com  Traceability Throughout Test Process It is important to establish and maintain traceability throughout the test process between each element of the test basis, various test products, test results and defect reports. Good traceability supports: Analysing the impact of changes Making testing auditable Meeting IT governance criteria Improving the understandability of test progress reports and test summary reports to include the status of elements of the test basis Relating the technical aspects of testing to stakeholders in terms that they can understand Providing information to assess product quality, process capability and project progress against business goals Coverage measured by traceability can be a key KPI towards test objectives for example: o Traceability of test cases to requirements can verify that all the requirements have been covered o Traceability of test results can be used to evaluate the level of residual risk in the test object fdmgroup.com  Roles in Testing Test Management Role Mainly focuses on test planning, test monitoring and control and test completion Takes overall responsibility for the test process, test team and leadership of test activities Can depend on context for example in Agile software development some test management tasks can be carried out by the Agile team Tasks that span multiple teams or entire organisation may be performed by test managers outside of the development team Testing Role Takes overall responsibility for the engineering (technical) aspects of testing Mainly focused test analysis, test design test implementation and test execution Possible for one person to take on the role of testing and test management at the same time fdmgroup.com  Quiz Time Undertake The Mentimeter quiz Navigate to Enter the code www.menti.co from your m trainer fdmgroup.com  Session Aims Why is Testing Testing Test Activities, Essential Skills and What is Testing? Necessary? Principals Testware and Good Practices Test Roles in Testing fdmgroup.com Essential Skills and Good practices in Testing Confirmation Bias can make it difficult to accept information that disagrees with currently held beliefs To It try is and a common Testing improve is oftenhuman this seenview, trait as a information to blame the destructive about bearer defects activity of bad even and news, failures though it and information should greatly contributes beproduced communicated by testing tocode project in aoften constructive success contains and productway bad news quality E.g. developers expect their to be correct but they have confirmation bias which means they often struggle to accept when their code is incorrect The output differs Thank Why? you! The output matches That won’t from the I will We now WhatWhy?check have did youa thework! requirements requirements quality do!system fdmgroup.com Essential Skills and Good practices in Testing Thoroughness, Interacts effectively with Good communication Helps to identify defects, stakeholders, conveys Testing Helpsknowledge increase carefulness, curiosity, skills, active listening, especially the ones that information well,team to be effectiveness of testing attention to detail, being a good are difficult to find understood and to report being methodical player and discuss defects Analytical thinking, To be able to understand To increase Knowledge Technical efficiency of Domain knowledge Increase criticaleffectiveness thinking, and to communicate with testing (use of of testing creativity end users/business appropriate tools) representatives fdmgroup.com  What is Testing independence? The level of testing independence refers to how far removed the tester is from the author of the code. The further removed the tester is the higher the level of independence. Independence is not a replacement for familiarity. (developers can efficiently find many defects in their code) fdmgroup.com  Levels of Testing Independence For most projects it is usually best to carry out testing with multiple levels of independence High level of independence Testers from outside the organisation Testers outside of the author’s team but within the organisation Authors peers from the same team The author Low level of independence fdmgroup.com  Independence of Testing Testers likely to recognise different kind of defects to developers due to different backgrounds and technical perspectives Independent testers can verify, challenge or disprove assumptions made by stakeholders Can be isolated from the development team Isolation can lead to lack of collaboration and communication problems Can lead to an adversarial relationship Developers may lose a sense of responsibility for quality Independent testers may be seen as a bottleneck or blamed for delays in a release fdmgroup.com Whole Team approach Any team member with the knowledge and skills can perform any task and everyone is responsible for quality The team members share the same workspace (physical or virtual) which facilitates communication and interaction A practice that comes from Extreme Programming fdmgroup.com  Whole Team approach Testers work with other team members to ensure that quality is achieved which includes: Collaborating with business representatives to help them create suitable acceptance tests Working with developers to agree a test strategy and decide test automation approaches Testers therefore can transfer knowledge to other team members and influence the development of the product May not be suitable based on context of project for example a safety-critical system may require a high level of test independence fdmgroup.com  Quiz Time Undertake The Mentimeter quiz Navigate to Enter the code www.menti.co from your m trainer fdmgroup.com Questions fdmgroup.com  You are now able to:  Understand what testing is  Exemplify why testing is necessary  Explain the seven testing principles Module Objectives  Describe the test activities, testware and test roles  Identify the essential skills and good practices in testing fdmgroup.com fdmgroup.com