Secure Programming Slides PDF

Secure Programming Outline Secure Programming 2 Software Testing ▪ What is testing? ▪ Direct execution of code on test data in a controlled environment ▪ Discussion: Goals of testing ▪ To reveal failures: Most important goal of testing ▪ T...

Secure Programming Outline Secure Programming 2 Software Testing ▪ What is testing? ▪ Direct execution of code on test data in a controlled environment ▪ Discussion: Goals of testing ▪ To reveal failures: Most important goal of testing ▪ To assess quality: Difficult to quantify, but still important ▪ To clarify the specification: Always test with respect to a spec Testing shows inconsistency Either spec or program could be wrong ▪ To learn about program How does it behave under various conditions? Feedback to rest of team goes beyond bugs ▪ To verify contract: Includes customer, legal, standards Secure Programming 3 Goals of testing ▪ Reveal Failures ▪ Find out if there are any mistakes or errors in the software. ▪ If you are using a calculator app. You try to divide a number by zero, expecting to see an error message. If the app stops working (crashes) instead, it means you have found a mistake. ▪ Assess Quality ▪ Check how good or bad the software is. ▪ a photo editing app. If it takes a long time to add a filter to a photo, testing will help you realize that the app needs to be faster so users don’t get frustrated. ▪ Clarify the Specification ▪ Make sure the software does what it’s supposed to do. ▪ A banking app should send an alert when a transaction is over $500. If it sends alerts for transactions below $500, there’s a problem with the app not following the rules it was designed for. Secure Programming 4 ▪ Learn About the Program ▪ See how the software behaves in different situations. ▪ a shopping website. By testing it when many people try to buy items at once, the team can find out if the website slows down or stops working when it’s really busy. ▪ Verify Contract ▪ Ensure the software follows legal rules and user requirements. ▪ A healthcare app must keep patient data private. Testing makes sure that only authorized people can see sensitive information, keeping users’ data safe and meeting privacy laws. Secure Programming 5 Specification ▪ Contains: ▪ Functional behavior: What the system should do (e.g., how it performs tasks). ▪ An online shopping website should allow users to add items to their cart, proceed to checkout, and pay for their items. ▪ Erroneous behavior: How the system should handle errors or unexpected issues. ▪ If a user tries to log in with an incorrect password, the system should display an error message like, “Incorrect password. Please try again.” ▪ Quality attributes: How well the system should perform (e.g., speed, security, etc.). ▪ A mobile app should load within 3 seconds and be able to handle up to 10,000 users at once Secure Programming 6  Desirable attributes ▪ Complete: Describing every feature of a software tool. ▪ Minimal: It only describes relevant functions, not optional or unused ones. ▪ Unambiguous: If the app should alert a user when the account balance is below $50, the specification clearly states this behavior. ▪ Consistent: If one part says “the user can edit details,” another part should not say “the user cannot edit details.” ▪ Testable: If a function is supposed to add two numbers, the specification makes it easy to test by trying different numbers. ▪ Correct: If users want an app to track expenses, the specification should correctly describe how the app records and displays expenses. Secure Programming 7  Types of Testing ▪ Unit Testing: ▪ Individual subsystem ▪ Carried out by developers ▪ Goal: Confirm that subsystems is correctly coded and carries out the intended functionality ▪ a calculator app. In unit testing, the developer tests whether the "addition" function works correctly by adding different number ▪ Integration Testing: ▪ Groups of subsystems (collection of classes) and eventually the entire system ▪ Carried out by developers ▪ Goal: Test the interface among the subsystem Secure Programming 8 System Testing ▪ System Testing: Terminology: ▪ The entire system system testing here = validation testing ▪ Carried out by developers ▪ Goal: Determine if the system meets the requirements (functional and global) 2 kinds of Acceptance testing ▪ Acceptance Testing: ▪ Evaluates the system delivered by developers ▪ Carried out by the client. May involve executing typical transactions on site on a trial basis ▪ Goal: Demonstrate that the system meets customer requirements and is ready to use ▪ Implementation (Coding) and testing go hand in hand Secure Programming 9  Unit Testing ▪ Informal: ▪ Incremental coding Write a little, test a little ▪ Static Analysis: ▪ Hand execution: Reading the source code ▪ Walk-Through (informal presentation to others) ▪ Code Inspection (formal presentation to others) ▪ Automated Tools checking for ▪ syntactic and semantic errors ▪ departure from coding standards ▪ Dynamic Analysis: ▪ Black-box testing (Test the input/output behavior) ▪ White-box testing (Test the internal logic of the subsystem or object) ▪ Data-structure based testing (Data types determine test cases) Secure Programming 10 ▪ Which is More Effective: Static or Dynamic Analysis? ▪ Static Analysis helps find errors early, is quicker, and can catch many coding standard violations before the code runs. ▪ Dynamic Analysis is necessary to check how the code actually works, revealing issues that only show up when the program is running. Secure Programming 11  Black-box Testing ▪ Testing the software without any knowledge of the internal code or logic. The focus is on inputs and outputs—what goes in and what comes out, without knowing how the processing happens inside. ▪ Almost always impossible to generate all possible inputs ("test cases") why? ▪ Goal: Reduce number of test cases by equivalence partitioning: ▪ Divide input conditions into equivalence classes ▪ Choose test cases for each equivalence class. (Example: If an object is supposed to accept a negative number, testing one negative number is enough) Secure Programming 12  Black-box Testing (Continued) ▪ Selection of equivalence classes (No rules, only guidelines): ▪ Input is valid across range of values. Select test cases from 3 equivalence classes: ▪ Below the range Are these complete? ▪ Within the range ▪ Above the range ▪ Input is valid if it is from a discrete set. Select test cases from 2 equivalence classes: ▪ Valid discrete value ▪ Invalid discrete value ▪ Another solution to select only a limited amount of test cases: ▪ Get knowledge about the inner workings of the unit being tested => white-box testing Secure Programming 13 Types of black-box ▪ Requirements-based: ▪ A login page must allow only valid credentials. Test cases ensure login works as required and fails for invalid credentials. ▪ Positive/Negative results: ▪ A form accepts only numbers. Positive testing: input “123”, negative testing: input “abc”. ▪ Boundary value analysis: ▪ An age input accepts values from 18 to 60. Test cases: 17, 18, 60, 61. ▪ Decision tables: ▪ For an online purchase, the decision table covers conditions like payment success, payment failure, or discount availability, leading to different outcomes. ▪ Equivalence partitioning: ▪ A text field accepts inputs from 1 to 100. Test cases include one from below the range (e.g., 0), within (e.g., 50), and above (e.g., 101). ▪ State-based diagrams: ▪ Testing an ATM’s behavior when transitioning between states like idle, card inserted, PIN entered, and cash withdrawn. Secure Programming 14 Types of black-box ▪ Compatibility testing: ▪ A website is tested to ensure it displays correctly in Chrome, Firefox, and Safari browsers. ▪ User documentation testing: ▪ Verify that a user manual's step-by-step instructions for setting up a printer match the actual software interface. ▪ Domain testing: ▪ For a banking app, ensure that transactions align with banking regulations like account overdraft limits and transaction approvals. Secure Programming 15  White-box Testing ▪ Focus: Thoroughness (Coverage). Every statement in the component is executed at least once. ▪ Testing the software with full knowledge of the internal code and logic. The tester can see the structure of the code, understand how it works, and test specific paths and logic. ▪ Four types of white-box testing ▪ Statement Testing ▪ Loop Testing ▪ Path Testing ▪ Branch Testing Secure Programming 16  White-box Testing (Continued) ▪ Statement Testing (Algebraic Testing): Test single statements ▪ Loop Testing: ▪ Cause execution of the loop to be skipped completely. (Exception: Repeat loops) ▪ Loop to be executed exactly once ▪ Loop to be executed more than once ▪ Path testing: ▪ Make sure all paths in the program are executed ▪ Branch Testing (Conditional Testing): Make sure that each possible outcome from a condition is tested at least once if ( i = TRUE) printf("YES\n");else printf("NO\n"); Test cases: 1) i = TRUE; 2) i = FALSE Secure Programming 17  Loop Testing [Pressman] Simple loop Nested Loops Concatenated Why is loop testing important? Loops Unstructured Loops Secure Programming 18 White-box Testing Example FindMean(float Mean, FILE ScoreFile) { SumOfScores = 0.0; NumberOfScores = 0; Mean = 0; Read(ScoreFile, Score); while (! EOF(ScoreFile) { if ( Score > 0.0 ) { SumOfScores = SumOfScores + Score; NumberOfScores++; } Read(ScoreFile, Score); } if (NumberOfScores > 0 ) { Mean = SumOfScores/NumberOfScores; printf("The mean score is %f \n", Mean); } else printf("No scores found in file\n"); } Secure Programming 19 White-box Testing Example: Determining the Paths Secure Programming 20 Constructing the Logic Flow Diagram Start 1 F 2 T 3 T F 4 5 6 7 T F 8 9 Exit Secure Programming 21 Finding the Test Cases Start 1 a (Covered by any data) 2 b (Data set must contain at least one value) (Positive score) d 3 e (Negative score) c 4 5 (Data set must h (Reached if either f or be empty) f g 6 e is reached) 7 j (Total score > 0.0) (Total score < 0.0) i 8 9 k l Exit Secure Programming 22 Comparison of White & Black-box Testing ▪ White-box 25.1.2002 Testing: ▪ Both types of testing are ▪ Potentially infinite number needed of paths have to be tested ▪ White-box testing and black ▪ White-box testing often tests box testing are the extreme what is done, instead of ends of a testing continuum. what should be done ▪ Any choice of test case lies in ▪ Cannot detect missing use between and depends on the cases following: ▪ Black-box Testing: ▪ Number of possible logical ▪ Potential combinatorical paths explosion of test cases (valid ▪ Nature of input data & invalid data) ▪ Amount of computation ▪ Often not clear whether the selected test cases uncover a ▪ Complexity of algorithms and particular error data structures ▪ Does not discover extraneous use cases ("features") Secure Programming 23  The 4 Testing Steps 1. Select what has to be 3. Develop test cases measured ▪ A test case is a set of test ▪ Analysis: Completeness of data or situations that will requirements be used to exercise the ▪ Design: tested for cohesion unit (code, module, ▪ Implementation: Code tests system) being tested or about the attribute being measured 2. Decide how the testing is 4. Create the test oracle done ▪ An oracle contains of the ▪ Code inspection predicted results for a set ▪ Proofs (Design by Contract) of test cases ▪ Black-box, white box, ▪ The test oracle has to be ▪ Select integration testing written down before the strategy (big bang, bottom actual testing takes place up, top down, sandwich) Secure Programming 24  Unit-testing Heuristics 1. Create unit tests as soon as object 4. Desk check your source code design is completed: ▪ Reduces testing time ▪ Black-box test: Test the use 5. Create a test harness cases & functional model ▪ Test drivers and test stubs are ▪ White-box test: Test the needed for integration testing dynamic model ▪ Data-structure test: Test the 6. Describe the test oracle object model ▪ Often the result of the first successfully executed test 2. Develop the test cases 7. Execute the test cases ▪ Goal: Find the minimal number of test cases to ▪ Don’t forget regression testing cover as many paths as ▪ Re-execute test cases every possible time a change is made. Big cost -> what should be done? 3. Cross-check the test cases to eliminate duplicates 8. Compare the results of the test with the test oracle ▪ Don't waste your time! ▪ Automate as much as possible Secure Programming 25