combinepdf.pdf

CPSC 310 Intro. to Software Engineering Summer 2024 Chris Kerslake, Instructor [email protected] Lecture 14: Introduction to Testing 2 Outline for today: Housekeeping Midterm Questions Sprint Questions Introduction to Testing 3 ...

CPSC 310 Intro. to Software Engineering Summer 2024 Chris Kerslake, Instructor [email protected] Lecture 14: Introduction to Testing 2 Outline for today: Housekeeping Midterm Questions Sprint Questions Introduction to Testing 3 Housekeeping 4 Housekeeping  Lab 07 Spreadsheet of User Stories, Wednesday  Sprint #1 starts Wednesday.  Midterm next week (Tue, July 2nd)  In-person, 9:30-11:00 in CBTF, ICCS 008 & 014* * 014 for CFA exam students 5 Labs Lab 07: Scrum Planning for Sprint #1 (A4) Lab 08: Scrum Update Next week, you will update your TA in lab. Keep task spreadsheet up to date. Lab 09: Demo & Review (A4 due) Lab 10: Scrum Planning for Sprint #2 (A5) Lab 11: Scrum Update Lab 12: Demo & Review (A5 due) 6 Sprint Questions? Any questions from the lab? Any sprint questions? 7 A3 Stats… CPSC 310 - Summer 2024 - A3 Project Designs 15 teams, 473 pages, 32 pg avg 90 80 80 70 60 50 48 40 40 38 36 34 29 30 30 27 24 25 21 20 18 11 12 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 8 A4 Deliverables Your application via a Docker image A working prototype, based on your sprint plan A video demo (max 3 minutes) An updated project spreadsheet 9 Midterm Questions 10 What will the midterm look like? Multiple-choice questions Short-answer questions 90-minutes long You are responsible for all content presented or discussed during lectures, labs, and assignments. No Docker or Ollama questions 11 Introduction to Testing 12 Testing Unit Overview Proving vs. Testing Types of Testing Testing Tactics Stopping Criteria Who should test the software? 13 Learning Goals By the end of this unit, you should be able to: Explain differences between proving and testing. Explain why testing is important, what its limitations are, and who should be involved in testing. Describe different kinds of testing & when to use each. Generate statement, branch, and basic path test cases. Explain mutation testing and provide mutants for methods. 14 Why are we looking at testing so early? Testing is critical. Testing is often overlooked… Testing is often under-done… Few people like to test… Testing is best done from the start… You will need to “test” your project… 15 Testing vs. Proving 16 Testing vs. Proving Testing Proving Dynamic: while executing Static: analyzed logically Shows presence of bugs Shows absence of errors Widely used in practice Used sparingly Costly Extremely costly In practice, they are complementary methods. 17 Proving Example static int someMethod(int x, int y) { if (x > y) { return x – y; } else if (x < y) { return y – x; } else { return x + y; } } Q: Can this function ever return a negative number? 18 Who uses proofs? Historically, Nuclear software NASA Aviation Medical device manufacturers Automotive and machines Today, Everyone can, it’s built into our tools. 19 How can we “prove” software? “Formal methods” (costly) Using mathematical methods and models Code Reviews (less costly) Inspections, walkthroughs Static analysis of source code (almost free**) Linters – catch common errors 20 Testing 21 Dijkstra on Testing… “Testing shows the presence, not the absence of bugs.” 22 Why do we test? So hospital patients are not killed (Therac-25) So rockets do not explode (recall Ariane 5) So machines keep working (Year 2000) So the Internet is not vulnerable (Heartbleed) To deliver robust systems (ACM Code of Ethics) Because it saves money?! 23 Testing Goals Verification: “Did we build the system correctly?” Verify that the software is behaving correctly. Validation: “Did we build the correct system?” Demonstrate software meets its requirements. 24 Testing Terminology Testing: Execute a program or program unit with a test set to determine if the expected output is produced. Test Case: A set of value assignments to each input parameter with expected values for each output. Test Set: A set of Test Cases. Exhaustive testing: A Test Set which covers all possible inputs. 25 Exhaustive Testing – the Haystack Problem 26 By Chris Light - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=84840659 Is Testing Futile? Testing is like searching for a needle in a haystack. Searching the haystacks (write test cases) Each test case covers some part of the program. Finding a needle (find a bug) Shows the presence of an error Search every inch of haystacks (exhaustive testing) Shows the absence of the errors tested for. 27 Lab Reminders… 28 Reminders… Midterm Tuesday, 9:30 in CBTF, ICCS 008 & 014 Lab 08 Sprint #1 Scrum Meeting COME PREPARED (30-minutes each team) Get coding: Check-in daily (5-minute check-in) Document progress (spreadsheet) Seek help early Test from the start. 29 CPSC 310 Intro. to Software Engineering Summer 2024 Chris Kerslake, Instructor [email protected] Lecture 15: Types of Testing 2 Outline for today: Housekeeping Sprint Questions Types of Testing Mid-term Survey 3 Housekeeping 4 Housekeeping  Lab 08 Scrum & Spreadsheets  Sprint #1 underway...  Prototype Demo next Wednesday (Wed, July 10th) Some teams still not using Docker… it works on my machine… it won’t work on your TA’s machine… 5 Labs Lab 07: Scrum Planning for Sprint #1 (A4) Lab 08: Scrum Update Lab 09: Demo & Review (A4 due) Demo prototype running in Docker Sprint Review Lab 10: Scrum Planning for Sprint #2 (A5) Lab 11: Scrum Update Lab 12: Demo & Review (A5 due) 6 Scrum/Sprint Questions? Any questions from the lab? Any sprint questions? 7 A4 Deliverables Due next Wednesday (July 10th): Your application via a Docker image A working prototype, based on your sprint plan Due next Friday (July 12th): A video demo (max 3 minutes) An updated project spreadsheet Peer Evaluations (TWA first, please) 8 Types of Testing 9 Testing Unit Overview  Proving vs. Testing Types of Testing Testing Tactics Stopping Criteria Who should test the software? 10 Dijkstra on Testing… “Testing shows the presence, not the absence of bugs.” 11 Binder on Testing… “Given a finite amount of time and resources, testing enables validating that a system has an acceptable risk of costly or dangerous defects.” - Bob Binder 12 Types of Testing Unit Testing Regression Testing Integration Testing System or End-To-End Testing Acceptance Testing 13 Types of Tests Regression Fast Unit Integration System or End-to-End Acceptance Slow Targeted Broad 14 Unit Testing “Where you test the smallest functional unit of code.” Review from CPSC 210? Each unit test: 1. Has a meaningful name that describes the point of the test. 2. Setup: the required code to get into the desired state. 3. Execution: Invokes the code-under-test (CUT) 4. Validation: Validates that the CUT did what was expected. Unit tests are grouped into Unit Test Suites. 15 Regression Testing Repetition of known good tests, not new tests. Tests the system after changes to ensure nothing (that is already being tested) broke. This is why your tests should not be “throwaway tests”. This is why you should update your unit tests after bug fixes so you can prevent reintroducing old bugs (regressing). 16 Integration Testing Tests all parts of the system together. Mock objects get replaced with real ones**. This is where Continuous Integration (CI) tools fits… After every commit/merge: Runs unit tests, if successful, then Run integration tests, if successful, then Run performance tests. 17 Integration Testing, in practice… In practice, integration testing, with a CI tool: Might mean spinning up a VM (or container) Installing the application Configuring the application Loading data Starting a browser automation tool (like Selenium) Running scenarios Typically done BEFORE going to QA 18 System or End-to-End Testing This is where Quality Assurance (QA) comes in. Formal testing, with test plans. Test for user needs and requirements. Dev > Test (QA) > Stage (Cx) > Production Testing servers are deployed just like in integration/CI. 19 User Acceptance Testing In many scenarios, there is a User Acceptance Testing phase (UAT) The Customer (Cx) validates or verifies features to be released before they are released. This is often done in a staging environment that mirrors production but is restricted access for Cx. QA may also smoke-test the system in staging before release. 20 Development Processes and Matching Tests Acceptance Client Needs Software Development Steps Testing Types of Testing Requirements System Testing Integration Design Testing Coding Unit Testing 21 Activity: What level of testing do you have? In groups, discuss what level(s) of testing your project currently has, and what you would need to do in order to implement the other levels. 22 Glass Box Testing 23 Testing Unit Overview  Proving vs. Testing  Types of Testing Testing Tactics Stopping Criteria Who should test the software? 24 Black Box Testing (specifications) So far, we have been testing against the specification. Because we know the spec, we can build tests before writing the code. However, it is hard to reason about the strength and weaknesses of the implementation from the specification… We need to see the code. 25 Glass Box Testing (implementation) Looking at the code, we can test the internals, Test code quality through mutation testing, and Check for code coverage. However, complicated by code changes and easy to overfit to the current implementation. 26 Mid-Term Survey (in Canvas) 27 Next Week 28 Next Week Lab 09 Sprint #1 Demo & Review COME PREPARED (30-minutes each team) Get coding: Ensure Docker is working (on all machines) Check-in daily (5-minute check-in) Document progress (spreadsheet) Seek help early Test from the start. 29 CPSC 310 Intro. to Software Engineering Summer 2024 Chris Kerslake, Instructor [email protected] Lecture 16: Testing Tactics 2 Outline for today: Housekeeping Mid-term Survey Testing Tactics 3 Housekeeping 4 A4 Deliverables Your application via a Docker image A working prototype, based on your sprint plan A video demo (max 3 minutes) An updated project spreadsheet 5 Housekeeping Wednesday  Sprint #1 ends  Lab 09 Demo, Spreadsheet, Review (Lab C – Elaine*)  A4 due “A4-submission” branch on GitHub Friday  A4 video  Peer Evaluation 2** 6 Labs Lab 07: Scrum Planning for Sprint #1 (A4) Lab 08: Scrum Update Lab 09: Demo & Review (A4 due) Demo during lab from Docker A4-submission branch in GitHub Lab 10: Scrum Planning for Sprint #2 (A5) Lab 11: Scrum Update Lab 12: Demo & Review (A5 due) 7 Mid-Term Survey 8 Mid-Term Survey – Things not working… Technical overhead Hardware requirement Memorization? Assignment Details Teams… Labs – not structured, maybe tech activities instead? Housekeeping… maybe a Piazza post instead? More coding time (more sprints) 9 Mid-Term Survey – What’s working… Lectures Labs Scrum with TA Worked examples and extra material The project is cool 10 Mid-Term Survey – Coursework Level More than expected 14% About what I expected 23% Less than expected 64% 0% 10% 20% 30% 40% 50% 60% 70% 11 Mid-Term Survey – Midterm Difficulty Harder than expected 14% Easier than expected 14% About what I expected 73% 0% 10% 20% 30% 40% 50% 60% 70% 80% 12 Mid-Term Survey – Team Stages Forming: Still figuring out how to 18% work together Storming: Competing for status 5% and acceptance of ideas Norming: Focused on developing 55% ways of working together Performing: Trust and rely on each 23% other 0% 10% 20% 30% 40% 50% 60% 13 Glass Box Testing 14 Testing Unit Overview  Proving vs. Testing  Types of Testing Testing Tactics Stopping Criteria Who should test the software? 15 Black Box Testing (specifications) So far, we have been testing against the specification. Because we know the spec, we can build tests before writing the code. However, it is hard to reason about the strength and weaknesses of the implementation from the specification… We need to see the code. 16 Glass Box Testing (implementation) Aimed at testing the completeness of the test suite. By looking at the code, we can test the internals. Check for code coverage. Check for typos by breaking code (mutation testing). However, this is complicated by code changes and is easy to overfit to the current implementation. 17 Test Coverage 18 Test Coverage Three forms of code coverage: Line/Statement coverage Branch coverage Path coverage 19 Statement Coverage 20 Statement Coverage examples foo(x, c1, c2) % covered 21 Statement Coverage: one test case foo(0, false, false) 60% 22 Statement Coverage: one test case foo(0, true, true) 100% 23 Statement Coverage: one test case foo(0, false, true) 80% 24 Branch Coverage foo(0, true, true) 50% 25 Branch Coverage foo(0, true, true) foo(0, false, false) 100% 26 Path Coverage foo(0, true, true) foo(0, false, false) foo(0, true, false) foo(0, false, true) Only practical for some functions. 27 Mutation Testing 28 Mutation Testing 29 Mutation Testing 30 Mutation Testing 31 Mutation Testing 32 Mutation Testing 33 Mutation Testing 34 How to make mutations Statement duplication, deletion, addition Boolean replacement: a && b  a || b Variable replacement or change: if (a)  if (b) 35 Boundary Testing 36 Boundary Testing [29, 30, 31] [68, 69, 70] [69, 70, 71] [72, 73, 74] [73, 74, 75] [84, 85, 86] 37 38 Activity: Where is the bug? 39 CPSC 310 Intro. to Software Engineering Summer 2024 Chris Kerslake, Instructor [email protected] Lecture 17: Refactoring 2 Outline for today: Housekeeping Refactoring Cross-Play… 3 Housekeeping 4 Third-Party Cloud-Based LLMs Using Ollama in a Docker container was the goal. SE is all about balancing trade-offs… If you used a cloud-based LLM instead: Did you try other models in Ollama first? Did you disclose this to your users? Did you include your API token/key in your GitHub repo so your TA can still deploy and run your app? 5 Grading… Midterm… TA deadline is Friday, waiting for marks in Canvas so I can release the grades. Have requested CBTF for next week to review. Other marks will be released this weekend 6 Housekeeping Wednesday  Sprint #1 ends  Lab 09 Demo, Spreadsheet, Review  A4 due “A4-submission” branch on GitHub Friday  A4 video  Peer Evaluation 2** 7 Labs Lab 10: Scrum Planning for Sprint #2 (A5) Meet with TA to plan Sprint #2 Lab 11: Scrum Update Lab 12: Demo & Review (A5 due) 8 Refactoring 9 Additional Source 1999 2018 10 What is refactoring? Controlled, behavior-preserving code transformations. “Its essence is applying a series of small behavior- preserving transformations, each of which "too small to be worth doing". However, the cumulative effect of each of these transformations is quite significant. By doing them in small steps you reduce the risk of introducing errors.” 11 Recall: Agile Process Test-Driven Development Write tests first Emergent Design Grow from Minimally-Viable Product (MVP) Refactor Code After initial architectural spike Allow design to emerge, rather than big design. Make pragmatic choices along the way. 12 Code Change is Constant You are tasked to modify a legacy function that generates the invoice text for a movie rental app. Please add children's movies; just follow the existing code… 13 Code Change is Constant No problem, you add the new code, following the existing code. Great! Now we need to add frequent renter points too… 14 Code Change is Constant No problem, I can fit that into the function too... Wow, you are good at this, we are looking to add seniors too… 15 Which leads to… This is great… We need to generate an email using the same logic… So, you might quickly copy the code and tweak it… (don’t pretend you haven’t done this) 16 So, the code changes Of course, you would never duplicate code like that. However, Other people alter your code. You alter other people’s code. Everyone is happy, b/c we are making progress. But, Code issues emerge… And they are expected in the Agile methodology. 17 Code issues like: Duplication of code. Rigidity Over-coupling A general mess… … and you run the risk of introducing bugs. … generating merge conflicts. Code begins to rot, and we call these “code smells”. 18 What is a “Code Smell”? Recognizable indicators that something may be wrong with the code. Code smells are not bugs. Code smells are design “choices” or hacks. Code smells are typically a form of technical debt. 19 What is Technical Debt? Design choices that were made in the interest of time or budget, rather than technical reasons. “We would like to do it ‘right’, but we don’t have time.” Technical debt builds up over time. It typically requires a restructuring of the design. This is not “throw it out, and re-write it” This is restructure as we understand the problem. 20 Types of “Code Smells” Bloaters – code that has grown too big to maintain OO Abusers – incorrect application of OO principles Change Preventers – change in one place requires change in other places too Dispensables – pointless code Couplers – excessive coupling or delegation of classes Source: https://refactoring.guru/refactoring/smells 21 (Some) Common “Code Smell”? Magic numbers Duplicated code Long methods Complicated conditionals Large classes Overly complicated code comments… 22 Example: Magic Numbers The use of a numeric constant in the code. function potentialEnergy(mass, height) { return mass * 9.81 * height; } “Everyone” knows the constant for gravity… on Earth. 23 Example: Duplicated Code 24 Complicated Comments 26 When to refactor? Do it as you code (opportunistic refactoring) If you recognize a “code smell” When you add a new function When you fix a bug When you code review 27 How do we refactor? We know the symptoms. So, we refactor. Ensure all Find code Determine tests pass that smells refactoring Ensure all Refactor tests still pass 28 Don’t refactor when… When the tests are failing. When you should just rewrite the code. When you have impending deadlines. 29 Remember: First smell, then refactor. Beware of ‘over-refactoring’ or the compelling urge to rewrite all the code. Just because you see a potential for refactoring, does not mean you should. Only refactor if the code suffers from a code smell. 30 Cross-Play 31 Cross-Play – Demo & Review Get into your project groups Set up at least one laptop with your project (A4) Leave one person at the laptop as the tour guide The rest visit other teams Switch when you are done Rotate the tour guide every 10 minute or so. Ask questions, give constructive suggestions, rotate. 32 CPSC 310 Intro. to Software Engineering Summer 2024 Chris Kerslake, Instructor [email protected] Lecture 18: Code Smells & Cyclomatic Complexity 2 Outline for today: Housekeeping Code Smells examples Cyclomatic complexity 3 Housekeeping 4 Housekeeping Midterm viewing in my office (ICCS 249) this week: Tuesday – 11AM to 1PM Thursday – 11AM to 1PM 10-minutes per student Any questions, schedule time with TA Make sure you don’t change A4-submission branch 5 Labs Lab 10: Scrum Planning for Sprint #2 (A5) Meet with TA to plan Sprint #2 Lab 11: Scrum Update Lab 12: Demo & Review (A5 due) 6 Code Smells 7 Recall: a “Code Smell” is… Recognizable indicators that something may be wrong with the code. Code smells are not bugs. Code smells are design “choices” or hacks. Code smells are typically a form of technical debt. 8 How do we refactor? We know the symptoms. So, we refactor. Ensure all Find code Determine tests pass that smells refactoring Ensure all Refactor tests still pass 9 (Some) Common “Code Smell”? Magic numbers Complicated conditionals 10 Smell: Magic Numbers The use of a numeric constant in the code. function potentialEnergy(mass, height) { return mass * 9.81 * height; } Replace with a symbolic constant: const GRAVITY_CONSTANT = 9.81; function potentialEnergy(mass, height) { return mass * GRAVITY_CONSTANT * height; } 11 Smell: Complicated Conditionals if (date.before(SUMMER_START) || date.after(SUMMER_END)) charge = quantity * winterRate + winterCharge; else charge = quantity * summerRate; Extract methods from condition: if (notSummer(date)) charge = winterCharge(quantity); else charge = summerCharge(quantity); 12 Activity: What smells? Now that we have looked at code smells and approaches to addressing them, let’s try it… Canvas > Files > Demo > what_smells.js - There are (at least) three smells, identify them and correct the code – making sure the tests still run. 13 Activity: What smells? Three code smells: - Magic numbers. - Complicated conditional. - Unreadable functionality. 14 Cyclomatic Complexity 15 Cyclomatic Complexity (McCabe, 1976) “How to modularize a software system so the resulting modules are both testable and maintainable?” 16 Cyclomatic Complexity (McCabe, 1976) “What is needed is a mathematical technique that will provide a quantifiable basis for modularization and allow us to identify software modules that will be difficult to test or maintain.” 17 Cyclomatic Complexity (McCabe, 1976) v(G) = e – n + p G – program graph e – num edges n – num nodes p – connected components 18 Cyclomatic Complexity For single functions, the formula becomes: V(G) = E – N + 2 E – num decision points (conditionals) Note each test (+1), thus “if (a or b)”  2 N – num exit points (returns) V - minimum number of white box tests required 19 Cyclomatic Complexity (McCabe, 1976) 20 Cyclomatic Complexity 1 1 – start 2 – return Complexity using nodes & edges: 2 Nodes = 2; Edges = 1 V=E–N+21–2+2=1 Complexity using code: Branches = 0; Returns = 1 V=B–R+20–1+2=1 21 Cyclomatic Complexity 1 1 – start 2 – if 3 – true 4 – return 2 3 This simple program has a complexity of 2 v=E–N+24–4+2=2 4 v=B–R+21–1+2=2 22 Cyclomatic Complexity example Canvas > Files > Demo > lastZeroInArray.js v(G) = B – R + 2 E = 2 branches N = 1 return 23 Activity: Rental Statement Complexity… In Canvas > Files > Demo > rental_statementX.js In groups, calculate the complexity for the four rental statement JS files 24 Cyclomatic Complexity The complexity number provides a couple of views: v(G) Complexity Testability Cost & Effort 1 – 10 Low High Low 10 – 20 Complex Medium Medium 20 – 40 Very complex Low High > 40 Overly complex Not at all Very high Also, recall V is also close to minimum number of white box tests required for a function. 25 Uses of Cyclomatic Complexity Determine independent path executions Ensure that all the paths have been tested Helps to identify uncovered paths Improve code coverage Evaluate application risk due to complexity 26 Cyclomatic Complexity tests… Canvas > Files > Demo > lastZeroInArray.js Given v(G) = 3 What tests are they? lastZeroInArray([1, 1]) -> -1 lastZeroInArray([0, 1]) -> 0 lastZeroInArray([1, 0]) -> 1 How about… lastZeroInArray([]) -> -1 Others? 27 Uses of Cyclomatic Complexity Use tools to calculate complexity For example: https://JSHint.com Try the rental_statementX.js code. 28 Reminders 29 Lab 10 Reminder Come prepared… Update Project Backlog Update Sprint 2 Backlog Updated spreadsheet for tasks. 30