Secure Programming Slides PDF

Secure Programming Dr. Haroon Mahmood Assistant Professor Al-Ain University, UAE Outline Secure Programming 2 Software Testing  What is testing?  Direct execution of code on test data in a controlled environment  Discussion: Goals of testing...

Secure Programming Dr. Haroon Mahmood Assistant Professor Al-Ain University, UAE Outline Secure Programming 2 Software Testing  What is testing?  Direct execution of code on test data in a controlled environment  Discussion: Goals of testing  To reveal failures: Most important goal of testing  To assess quality: Difficult to quantify, but still important  To clarify the specification: Always test with respect to a spec Testing shows inconsistency Either spec or program could be wrong  To learn about program How does it behave under various conditions? Feedback to rest of team goes beyond bugs  To verify contract: Includes customer, legal, standards Secure Programming 3 Specification Secure Programming 4  Types of Testing  Unit Testing:  Individual subsystem  Carried out by developers  Goal: Confirm that subsystems is correctly coded and carries out the intended functionality  Integration Testing:  Groups of subsystems (collection of classes) and eventually the entire system  Carried out by developers  Goal: Test the interface among the subsystem Secure Programming 5 System Testing  System Testing: Terminology:  The entire system system testing here = validation testing  Carried out by developers  Goal: Determine if the system meets the requirements (functional and global)  Acceptance Testing:2 kinds of Acceptance testing  Evaluates the system delivered by developers  Carried out by the client. May involve executing typical transactions on site on a trial basis  Goal: Demonstrate that the system meets customer requirements and is ready to use  Implementation (Coding) and testing go hand in hand Secure Programming 6  Unit Testing  Informal:  Incremental coding Write a little, test a little  Static Analysis:  Hand execution: Reading the source code  Walk-Through (informal presentation to others)  Code Inspection (formal presentation to others)  Automated Tools checking for  syntactic and semantic errors  departure from coding standards  Dynamic Analysis:  Black-box testing (Test the input/output behavior)  White-box testing (Test the internal logic of the subsystem or object)  Data-structure based testing (Data types determine test cases) Which is more effective, static or dynamic analysis? Secure Programming 7  Black-box Testing  Focus: I/O behavior. If for any given input, we can predict the output, then the module passes the test.  Almost always why? impossible to generate all possible inputs ("test cases")  Goal: Reduce number of test cases by equivalence partitioning:  Divide input conditions into equivalence classes  Choose test cases for each equivalence class. (Example: If an object is supposed to accept a negative number, testing one negative number is enough) Secure Programming 8  Black-box Testing (Continued)  Selection of equivalence classes (No rules, only guidelines):  Input is valid across range of values. Select test cases from 3 equivalence classes:  Below the range Are these complete?  Within the range  Above the range  Input is valid if it is from a discrete set. Select test cases from 2 equivalence classes:  Valid discrete value  Invalid discrete value  Another solution to select only a limited amount of test cases:  Get knowledge about the inner workings of the unit being tested => white-box testing9 Secure Programming Types of black-box  requirements based  positive/negative - checks both good/bad results  boundary value analysis  decision tables  equivalence partitioning - group related inputs/outputs  state-based - based on object state diagrams  compatibility testing  user documentation testing  domain testing Secure Programming 10  White-box Testing  Focus: Thoroughness (Coverage). Every statement in the component is executed at least once.  Four types of white-box testing  Statement Testing  Loop Testing  Path Testing  Branch Testing Secure Programming 11  White-box Testing (Continued)  Statement Testing (Algebraic Testing): Test single statements  Loop Testing:  Cause execution of the loop to be skipped completely. (Exception: Repeat loops)  Loop to be executed exactly once  Loop to be executed more than once  Path testing:  Make sure all paths in the program are executed  Branch Testing (Conditional Testing): Make sure that each possible outcome from a condition is tested if ( i = TRUE) at least printf("YES\n");else once printf("NO\n"); Test cases: 1) i = TRUE; 2) i = FALSE Secure Programming 12  Loop Testing [Pressman] Simple loop Nested Loops Concatenated Why is loop testing important? Loops Unstructured Loops Secure Programming 13 White-box Testing Example FindMean(float Mean, FILE ScoreFile) { SumOfScores = 0.0; NumberOfScores = 0; Mean = 0; Read(ScoreFile, Score); while (! EOF(ScoreFile) { if ( Score > 0.0 ) { SumOfScores = SumOfScores + Score; NumberOfScores++; } Read(ScoreFile, Score); } if (NumberOfScores > 0 ) { Mean = SumOfScores/NumberOfScores; printf("The mean score is %f \n", Mean); } else printf("No scores found in file\n"); } Secure Programming 14 White-box Testing Example: Determining the Paths Secure Programming 15 Constructing the Logic Flow Diagram Start 1 F 2 T 3 T F 4 5 6 7 T F 8 9 Exit Secure Programming 16 Finding the Test Cases Start 1 a (Covered by any data) 2 b (Data set must contain at least one value) (Positive score) d 3 e (Negative score) c 4 5 (Data set must h (Reached if either f or be empty) f g 6 e is reached) 7 (Total score < 0.0) i j (Total score > 0.0) 8 9 k l Exit Secure Programming 17 Comparison of White & Black-box  Both White-box typesTesting:Testing of testing are needed 25.1.2002  Potentially  White-box tested extreme infinite testing andnumber black of boxpaths ends of a testing continuum. haveare testing to be the   White-box testing often tests what is done, instead Any choice of test case lies in between and depends of what should be done on the following:  Cannot detect missing use cases  Number of possible logical paths  Black-box  Nature Testing: of input data  Potential combinatorical explosion of test cases Amount of computation (valid & invalid data)  Complexity of algorithms and data structures  Often not clear whether the selected test cases uncover a particular error  Does not discover extraneous use cases ("features") Secure Programming 18  The 4 Testing Steps 1. 1. Select Select what what has has to to 3. 3. Develop Develop test test cases cases be be measured measured  AA test test case case isis aa set set  Analysis: Analysis: of of test test data data oror Completeness Completenessofof situations situations that that will will requirements requirements be be used used toto exercise exercise  Design: Design:tested testedfor for the the unit unit (code, (code, cohesion cohesion module, module, system) system)  Implementation: Implementation: being being tested tested or or Code Codetests tests about about the the attribute attribute being being measured measured 2. 2. Decide Decide how how thethe 4. 4. Create Create the the test test testing testing is is done done oracle oracle  Code Codeinspection inspection  An An oracle oracle contains contains  Proofs Proofs(Design (Designbyby of of the the predicted predicted Contract) Contract) results results for for aa set set of of  Black-box, test cases test cases Black-box,white whitebox, box,  Select integration  The The test test oracle oracle has has Select integration Secure Programming testing testingstrategy strategy(big (big to to be be written written down down19  Unit-testing Heuristics 1. 1. Create Create unit unit tests tests as 4. as 4. Desk Desk checkcheck your your source source soon soon as as object object design design code code is is completed:  Reduces completed: Reduces testingtesting timetime  Black-box Black-boxtest: test:Test 5. the Test 5. Create Create aa test test harness harness theuse usecases cases& &  Test functional model functional model Test drivers drivers and and test test stubs stubs are needed for are needed for  White-box White-boxtest: test:Test Test integration testing the dynamic model integration testing the dynamic model 6.  Data-structure 6. Describe Describe the the test test oracle oracle Data-structuretest: test:  Often Test Testthe theobject objectmodel model Often the the result result of of the the first successfully first successfully 2. 2. Develop Develop the the test test cases cases executed executed test test  Goal: Goal: Find Find the the 7. 7. Execute Execute the the test test cases cases minimal minimal number number of  Don’t of Don’t forget forget test test cases cases to to cover cover Big regression cost -> what regression shouldtesting be done? testing as as many paths as many paths  Re-execute possible as Re-execute test test cases cases possible every every time time aa change change 3. is made. 3. Cross-check Cross-check the the test test is made. cases cases to to eliminate eliminate duplicates duplicates Secure Programming 8. 8. Compare Compare thethe results results of of the the  Don't waste your test with the test oracle 20 test with the test oracle