smile-devsecops_course-3.3-compressé.pdf

DevSecOps Embedded / IoT How to develop secure, safe, sustainable and resilient embedded and IoT systems Vincent Jourdon 12 septembre 2024  Plan Smile : a quick overview...

DevSecOps Embedded / IoT How to develop secure, safe, sustainable and resilient embedded and IoT systems Vincent Jourdon 12 septembre 2024  Plan Smile : a quick overview Training course goals Introduction 2/421  Plan Smile : a quick overview Training course goals Introduction 3/421  Smile : a quick overview 4/421  A complete offer 5/421  Our locations 6/421  ECS : a quick overview 7/421  ECS : Who we are 8/421  ECS : What we do 9/421  ECS : Our references 10/421  Our white papers for embedded software 11/421  Our Linux blog https://linuxembedded.fr/ 12/421  Our internship offers 13/421  Let’s introduce ourselves. 14/421  Vincent Jourdon ENSICaen Smile-ECS Chief Operating Officer Operating, Delivery and Resource Manager Project Director Constrained real-time embedded systems Standards : EN 62304 : Medical device software EN 50128 / EN 50657 : Railways applications ISO 27001 : Cybersecurity [email protected] Code quality and Continuous @vincentjourdon Integration expert Consultant / Auditor : project organization (methods, process, tools, governance, Agile Cycle V/W) 15/421  Round Table Introduce yourself Where are you with software quality ? What are your expectations from this training ? 16/421  Plan Smile : a quick overview Training course goals Introduction 17/421  Training course goal This course is intended to help you to : Understand the interest of investing in code quality Know the different stages from code development to code deployment Know and use the different tools and be able to control and secure a software development process This course aims at people who will develop and manage software development quality. 18/421 Course overview Section 1: DevSecOps Overview Section 2: Continuous Inspection Section 3: Continuous Integration Section 4: Continuous Deployment Section 5: Conclusion 19/421  Feedback on the course I need your help to improve this course. Report mistakes, especially if they affect understanding Do not report typos, but send them via email Do not hesitate to give feedbacks (pace, difficulties, too many/too few practical work) Please help your future fellow students. 20/421  I DevSecOps Overview 21/421  DevSecOps Overview DevSecOps 22/421  Plan DevSecOps What is DevSecOps ? 23/421  Plan DevSecOps What is DevSecOps ? 24/421  DevSecOps : Definition DevSecOps is : Connecting three disciplines : development, security, and operations. Integrating security into Continuous Integration and Continuous Delivery pipelines in both : Pre-production / Developement environment : dev Production environment : ops Aiming to release better and more secure software faster, and to maintain and correct production software faster 25/421  DevSecOps Workflow 26/421  DevSecOps 27/421  DevSecOps : Dev for Development Development refers to the processes of managing software development, including : Coding Testing Continuous Inspection / Continous Integration / Continuous Delivery Implementing development activities in parrallel with operations and security leads to : Reduce operational issues Reduce security vulnerabilities Increase overall efficiency 28/421  DevSecOps : Ops for Operations Operations refers to the processes of managing software functionality throughout its delivery and use life cycle, including : Continuous Deployment Monitoring system performance Repairing defects Testing after updates and changes Tuning the software release system Implementing operations activities in parrallel with software development leads to : Reduce deployment time Increase overall efficiency 29/421  DevSecOps : Sec for Security Security refers to all the tools and techniques needed to : design and build a software that resists attacks detects and respond to defects (or actual intrusions) as quickly as possible. Implementing security activities in parrallel with software development and operations leads to : Reduce reaction time Give a wide-view of security issues and risks, from dev to ops Increase overall efficiency 30/421  DevSecOps Workflow 31/421  II Continuous Inspection 32/421  Continuous Inspection Technical Debt Code Quality Inspection 33/421  Plan Complexity Code Quality Unit testing Code Coverage Good Code vs Bad Conclusion Code Coding Style Coding Rules Code Review Technical Debt Static Code Analysis Dynamic Code Analysis Inspection 34/421  Plan Code Coverage Code Quality Conclusion Good Code vs Bad Code Coding Style Technical Debt Coding Rules Code Review Inspection Static Code Analysis First analysis Dynamic Code Analysis Going further First analysis Complexity Going further Unit testing 35/421  Good Code vs Bad Code 36/421  Good Code vs Bad Code What is a good / bad code ? Why does good code matter ? How do we do good code ? When should we do good code ? 37/421  Bad code : some real examples - comments ! ! ! unsigned char TxBuffer; // Allocate 128 bytes of RAM ErrorValue = false; firstname = user.getFirstName(); // Get user firstname bool isPatternValid(); 38/421  Bad code : some real examples - why ? ! ! ! unsigned char count = -1; $anEmptyArray = array(); return ($anEmptyArray); if ($method == "POST" && $createByXId == true) { // TODO } if ($method == "POST" && $createByXId == false) { // TODO } bool MyApplication::hasValidInformation() { bool InfoStatus = false; if (MyInfo.IsValid() && MyInfo.hasFix()) { InfoStatus = true; } return true; //InfoStatus } 39/421  Bad code : some real examples - No Cybersecurity ! ! ! GOTO FAIL : CVE-ID CVE-2014-1266 Impact : An attacker with a privileged network position may capture or modify data in sessions protected by SSL/TLS (iOS < 6.1.6 et < 7.0.6) 40/421  What is a good code ? Essentially, a code that is considered good is : Readable and understandable Homogeneous (follows a consistent style) Not over complicated (KISS : Keep it simple, Stupid !) Debuggable (the code fails gracefully) Testable (lower testing effort) Reusable (not too many dependencies) Doing only what is asked for Code you didn’t write (don’t reinvent the wheel) 41/421  Why does good code matter ? Good Developers tend to maintain the heritage (good or bad). A software is (almost) never completed. You cannot improve yourself if you make no measurements « broken window » theory Code is not everything but nothing is more important than code 42/421  Why does good code matter ? Good code has a positive impact on all development and testing activities The later defects are found, the more expensive are the fixes This effect is largely exposed and commented in different studies across a lot of industries. BAD CODE GOOD CODE Good code is cost effective 43/421  Good Code : How ? Define and follow coding rules Review code Measure code quality with tools Set up unit tests Inspect your code continuously with CI/CD 44/421  When should we do good code ? BEFORE first line of code, DURING development, even on legacy code AFTER main development (Maintenance in Secure and Operational Condition) 45/421  Plan Code Coverage Code Quality Conclusion Good Code vs Bad Code Coding Style Technical Debt Coding Rules Code Review Inspection Static Code Analysis First analysis Dynamic Code Analysis Going further First analysis Complexity Going further Unit testing 46/421  Coding Style 47/421  Coding Style : coding standards A set of style guidelines/coding rules ensure that software : Safe : causes no harm Secure : can’t be hacked Reliable : functions as it should, every time Testable : can be tested at the code level Maintenable : can be easily modify over years, even as the codebase grows Portable : works the same in every environment KISS : Keep It Simple, Stupid ! 48/421  Coding Style : coding standards examples Examples of of style guidelines/rules themes : Comments and documentation Spacing and indenting Variable Naming and Casing conventions Duplications (DRY - Don’t Repeat Yourself) Nesting... 49/421  Coding Style : coding standards example A positive integer is called an Armstrong number of order n if abcd..... = an + bn + cn + dn... (for example 153, 370, 371, 407) Original source code Code with coding standards #include #include int main() int main() { { int numberToTest, x, temp, sumPowerDigits = 0; int a, x, temp, b=0; printf(" Enter a number [0..999]: "); printf(" Enter a number : "); scanf("%d", &numberToTest); scanf("%d", &a); temp = numberToTest; temp=a; while(temp>0) while (temp > 0) { { x = temp % 10; x=temp%10; sumPowerDigits = sumPowerDigits + (x * x * x); temp b=b+(x*x*x); = temp / 10; temp=temp/10; } } if(a==b){ if (numberToTest == sumPowerDigits) { printf(" %d is Armstrong number\n",a);} printf(" %d is Armstrong number\n", numberToTest); else{ } else { printf(" %d is not Armstrong number\n",a);} return printf(" %d is not Armstrong number\n", numberToTest); 0; } } return 0; } 50/421  Coding Style : good practices Good practices : Define rules to be checked before first line of code NO MANUAL CODE REVIEW for coding styles, so use tools for : Automatic modification at development level (IDE) Automatic checks at Continuous Integration level (CI) Code legacy : consider the possibility to refactor the code using a tool (see below) 51/421  Coding Style : formatting C source code with clang-format 1 Install clang-format Check source file tp-source/c/tp1/nb_armstrong.c coding style versus GNU coding style Format source file tp-source/c/tp1/nb_armstrong.c coding style versus llvm coding style 52/421 1. https://clang.llvm.org/docs/ClangFormat.html  Coding Style : formatting C source code with clang-format Install clang-format 1 apt install clang-format Check source file tp-source/c/tp1/nb_armstrong.c coding style versus GNU coding style Format source file tp-source/c/tp1/nb_armstrong.c coding style versus llvm coding style 52/421 1. https://clang.llvm.org/docs/ClangFormat.html  Coding Style : formatting C source code with clang-format Install clang-format 1 apt install clang-format Check source file tp-source/c/tp1/nb_armstrong.c coding style versus GNU coding style clang-format --style=gnu nb_armstrong.c Format source file tp-source/c/tp1/nb_armstrong.c coding style versus llvm coding style 52/421 1. https://clang.llvm.org/docs/ClangFormat.html  Coding Style : formatting C source code with clang-format Install clang-format 1 apt install clang-format Check source file tp-source/c/tp1/nb_armstrong.c coding style versus GNU coding style clang-format --style=gnu nb_armstrong.c Format source file tp-source/c/tp1/nb_armstrong.c coding style versus llvm coding style clang-format -i --style=llvm nb_armstrong.c 52/421 1. https://clang.llvm.org/docs/ClangFormat.html  Coding Style : Checking Python source code with pylint 2 Install pylint Check source file tp-source/python/test1/calc2.py coding style Inspect results 53/421 2. https://pypi.org/project/pylint/  Coding Style : Checking Python source code with pylint 2 Install pylint pip3 install pylint Check source file tp-source/python/test1/calc2.py coding style Inspect results 53/421 2. https://pypi.org/project/pylint/  Coding Style : Checking Python source code with pylint 2 Install pylint pip3 install pylint Check source file tp-source/python/test1/calc2.py coding style pylint calc2.py Inspect results 53/421 2. https://pypi.org/project/pylint/  Coding Style : Checking Python source code with pylint 2 Install pylint pip3 install pylint Check source file tp-source/python/test1/calc2.py coding style pylint calc2.py Inspect results 53/421 2. https://pypi.org/project/pylint/  Plan Code Coverage Code Quality Conclusion Good Code vs Bad Code Coding Style Technical Debt Coding Rules Code Review Inspection Static Code Analysis First analysis Dynamic Code Analysis Going further First analysis Complexity Going further Unit testing 54/421  Coding Rules 55/421  Coding Rules Coding standards : C: MISRA-C :2012 Guidelines for the use of the C language in critical systems CERT Secure Coding C++ : MISRA-C++ :2008 Guidelines for the use of the C++ language in critical systems CERT Secure Coding Autosar C++ coding guidelines LLVM coding standards : https://llvm.org/docs/CodingStandards.html 56/421  Coding Rules : good practices Use an existing coding rules set Before first line of code : Define “Quality Profile” : the coding rules to apply Define “Quality Gate” : the targeted quality level For a legacy code, monitor only new and modified code by configuring a Quality Gate relative to the previous version of the code (leak period) Start with a full set of coding rules, then exclude the rules that do not suit you Regular change-based code analysis : for example => pull/merge request Use tools for automatic analysis 57/421  Plan Code Coverage Code Quality Conclusion Good Code vs Bad Code Coding Style Technical Debt Coding Rules Code Review Inspection Static Code Analysis First analysis Dynamic Code Analysis Going further First analysis Complexity Going further Unit testing 58/421  Code Review 59/421  Code Review Reviewing a code is : Evaluating code by one or more people with similar competences = peers Checking code quality by reading parts of source code A human activity Code review goals : Improve code quality and maintainability Improve code architecture Increase sense of mutual responsibility Find defects... but you’d better use tools... Learning / knowledge transfer Comply to QA guidelines Code review also helps identify and extend best practices. 60/421  Code Review : good practices Reviewers must not be the code’s author (thanks Captain Obvious !) Define rules to be checked before first line of code Regular change-based code review : for example => pull/merge request of a whole feature Only on tested software : build, automatic analysis and unit tests are OK Only formalized code review : Who are the reviewers When the review was conducted Why : architecture, coding style/rules, best-practice, vulnerability exposure, error detection, well documented, etc. What : part of code, commits to be reviewed Result : remarks, questions, author’s answers, review decisions Manual code review only on axis that are NOT AUTOMATABLE. 61/421  Code Review : example Remarks 3 static int dayOfYear(int month, int dayOfMonth, int year) { if (month == 2) { // we're in February [C2] dayOfMonth += 31; // add days of January [C3] } else if (month == 3) { dayOfMonth += 59; // month is 3 here [C4] } else if (month == 4) { dayOfMonth += 90; } else if (month == 5) { dayOfMonth += 31+28+31+30; } else if (month == 6) { dayOfMonth += 31+28+31+30+31; } else if (month == 7) { dayOfMonth += 31+28+31+30+31+30; } else if (month == 8) { dayOfMonth += 31+28+31+30+31+30+31; } else if (month == 9) { dayOfMonth += 31+28+31+30+31+30+31+31; } else if (month == 10) { dayOfMonth += 31+28+31+30+31+30+31+31+30; } else if (month == 11) { dayOfMonth += 31+28+31+30+31+30+31+31+30+31; } else if (month == 12) { dayOfMonth += 31+28+31+30+31+30+31+31+30+31+31; } return dayOfMonth; // the answer [C5] } 62/421 3. https://web.mit.edu/6.005/www/fa15/classes/04-code-review/  Code Review : example Remarks 3 static int dayOfYear(int month, int dayOfMonth, int year) { if (month == 2) { // we're in February [C2] dayOfMonth += 31; // add days of January [C3] Avoid magic numbers ! } else if (month == 3) { dayOfMonth += 59; // month is 3 here [C4] Use an array like int } else if (month == 4) { dayOfMonth += 90; monthLengths = 31, 28, 31, 30, } else if (month == 5) { dayOfMonth += 31+28+31+30;..., 31 } else if (month == 6) { dayOfMonth += 31+28+31+30+31; Don’t repeat yourself (how many } else if (month == 7) { dayOfMonth += 31+28+31+30+31+30; times is the number of day for } else if (month == 8) { dayOfMonth += 31+28+31+30+31+30+31; } each month ?) else if (month == 9) { dayOfMonth += 31+28+31+30+31+30+31+31; } Useless comments C2, C3, C4, C5 else if (month == 10) { dayOfMonth += 31+28+31+30+31+30+31+31+30; } One purpose for each variable : else if (month == 11) { dayOfMonth += 31+28+31+30+31+30+31+31+30+31; } else if (month == 12) { dayOfMonth should’nt be reused dayOfMonth += 31+28+31+30+31+30+31+31+30+31+31; } Fail fast : static and return dayOfMonth; // the answer [C5] } dynamic checking ! 62/421 3. https://web.mit.edu/6.005/www/fa15/classes/04-code-review/  Plan Code Coverage Code Quality Conclusion Good Code vs Bad Code Coding Style Technical Debt Coding Rules Code Review Inspection Static Code Analysis First analysis Dynamic Code Analysis Going further First analysis Complexity Going further Unit testing 63/421  Static Code Analysis examining the code without executing the program set of rules or coding standards code bugs, code vulnerabilities, code smells (adherence to commonly accepted coding standards) whitebox whole source code analysis unearth future errors that would not emerge in a dynamic test easy to automate in CI process 64/421  Static Code Analysis : build and analyze C source code with clang Install clang 4 Analyze source file tp-source/memory-trouble/main.c with all warnings enabled 65/421 4. https://clang.llvm.org  Static Code Analysis : build and analyze C source code with clang 4 Install clang apt install clang Analyze source file tp-source/memory-trouble/main.c with all warnings enabled 65/421 4. https://clang.llvm.org  Static Code Analysis : build and analyze C source code with clang 4 Install clang apt install clang Analyze source file tp-source/memory-trouble/main.c with all warnings enabled clang main.c --analyze -O0 -gdwarf -Wall -Wextra -Werror 2> clang- report.txt 65/421 4. https://clang.llvm.org  Static Code Analysis : analyze C++ source code with clang-tidy 5 Install clang-tidy Analyze source file tp-source/memory-trouble/main.c with all checks enabled 66/421 5. https://clang.llvm.org/extra/clang-tidy/  Static Code Analysis : analyze C++ source code with clang-tidy 5 Install clang-tidy apt install clang-tidy Analyze source file tp-source/memory-trouble/main.c with all checks enabled 66/421 5. https://clang.llvm.org/extra/clang-tidy/  Static Code Analysis : analyze C++ source code with clang-tidy 5 Install clang-tidy apt install clang-tidy Analyze source file tp-source/memory-trouble/main.c with all checks enabled clang-tidy --checks=* main.c > clang-tidy-report.txt 66/421 5. https://clang.llvm.org/extra/clang-tidy/  Static Code Analysis : analyze C++ source code with cppcheck 6 Install cppcheck Analyze source file tp-source/memory-trouble/main.c with verbosity, all checks enabled and with xml version 2 output file 67/421 6. https://cppcheck.sourceforge.io/#documentation  Static Code Analysis : analyze C++ source code with cppcheck 6 Install cppcheck apt install cppcheck Analyze source file tp-source/memory-trouble/main.c with verbosity, all checks enabled and with xml version 2 output file 67/421 6. https://cppcheck.sourceforge.io/#documentation  Static Code Analysis : analyze C++ source code with cppcheck 6 Install cppcheck apt install cppcheck Analyze source file tp-source/memory-trouble/main.c with verbosity, all checks enabled and with xml version 2 output file cppcheck -v --enable=all --xml-version=2 main.c 2> cppcheck-report.xml 67/421 6. https://cppcheck.sourceforge.io/#documentation  Static Code Analysis : Checking Python source code with pylint 7 Install pylint Analyze source file tp-source/python/test2/prime_number.py Inspect results 68/421 7. https://pypi.org/project/pylint/  Static Code Analysis : Checking Python source code with pylint 7 Install pylint pip3 install pylint Analyze source file tp-source/python/test2/prime_number.py Inspect results 68/421 7. https://pypi.org/project/pylint/  Static Code Analysis : Checking Python source code with pylint 7 Install pylint pip3 install pylint Analyze source file tp-source/python/test2/prime_number.py pylint prime_number.py Inspect results 68/421 7. https://pypi.org/project/pylint/  Static Code Analysis : Checking Python source code with pylint 7 Install pylint pip3 install pylint Analyze source file tp-source/python/test2/prime_number.py pylint prime_number.py Inspect results 68/421 7. https://pypi.org/project/pylint/  Plan Code Coverage Code Quality Conclusion Good Code vs Bad Code Coding Style Technical Debt Coding Rules Code Review Inspection Static Code Analysis First analysis Dynamic Code Analysis Going further First analysis Complexity Going further Unit testing 69/421  Dynamic Code Analysis Examining the code in a runtime environment No set of rules Runtime vulnerabilities that occur due to variations in business context (memory leak,...) Blackbox Limitation : only executed source code analysis Capable of exposing a subtle flaw or vulnerability too complicated for static analysis alone to reveal Hard to automate in CI process 70/421  Dynamic Code Analysis : analyze build with Valgrind Valgrind : memory debugging, memory leak detection and profiling 8 Install valgrind Build tp-source/memory-trouble/main.c into.out binary Analyze build main-Valgrind.out, all checks enables and with xml output file and tracking of origins and fds, and check of all type of leaks 71/421 8. http://valgrind.org/  Dynamic Code Analysis : analyze build with Valgrind Valgrind : memory debugging, memory leak detection and profiling Install valgrind 8 apt install valgrind Build tp-source/memory-trouble/main.c into.out binary Analyze build main-Valgrind.out, all checks enables and with xml output file and tracking of origins and fds, and check of all type of leaks 71/421 8. http://valgrind.org/  Dynamic Code Analysis : analyze build with Valgrind Valgrind : memory debugging, memory leak detection and profiling Install valgrind 8 apt install valgrind Build tp-source/memory-trouble/main.c into.out binary gcc main.c -O0 -gdwarf -Wall -Wextra -Werror -o main-Valgrind.out Analyze build main-Valgrind.out, all checks enables and with xml output file and tracking of origins and fds, and check of all type of leaks 71/421 8. http://valgrind.org/  Dynamic Code Analysis : analyze build with Valgrind Valgrind : memory debugging, memory leak detection and profiling Install valgrind 8 apt install valgrind Build tp-source/memory-trouble/main.c into.out binary gcc main.c -O0 -gdwarf -Wall -Wextra -Werror -o main-Valgrind.out Analyze build main-Valgrind.out, all checks enables and with xml output file and tracking of origins and fds, and check of all type of leaks valgrind --xml=yes --xml-file=valgrind-report.xml --track-origins=yes --leak-check=full --track-fds=yes./main-Valgrind.out Param1 71/421 8. http://valgrind.org/  Dynamic Code Analysis : analyze build with ASan ASan (Address Sanitizer) : memory error detection 9 Install ASan Build tp-source/memory-trouble/main.c into.out binary Analyze build main-Asan.out 72/421 9. https://github.com/google/sanitizers  Dynamic Code Analysis : analyze build with ASan ASan (Address Sanitizer) : memory error detection Install ASan 9 apt install libasan6 Build tp-source/memory-trouble/main.c into.out binary Analyze build main-Asan.out 72/421 9. https://github.com/google/sanitizers  Dynamic Code Analysis : analyze build with ASan ASan (Address Sanitizer) : memory error detection Install ASan 9 apt install libasan6 Build tp-source/memory-trouble/main.c into.out binary gcc main.c -fsanitize=address -fsanitize-recover=all -O0 -gdwarf -Wall - Wextra -Werror -o main-Asan.out Analyze build main-Asan.out 72/421 9. https://github.com/google/sanitizers  Dynamic Code Analysis : analyze build with ASan ASan (Address Sanitizer) : memory error detection Install ASan 9 apt install libasan6 Build tp-source/memory-trouble/main.c into.out binary gcc main.c -fsanitize=address -fsanitize-recover=all -O0 -gdwarf -Wall - Wextra -Werror -o main-Asan.out Analyze build main-Asan.out ASAN_OPTIONS=halt_on_error=false./main-Asan.out Param1 2> asan_report.txt 72/421 9. https://github.com/google/sanitizers  Plan Code Coverage Code Quality Conclusion Good Code vs Bad Code Coding Style Technical Debt Coding Rules Code Review Inspection Static Code Analysis First analysis Dynamic Code Analysis Going further First analysis Complexity Going further Unit testing 73/421  Complexity 74/421  Cyclomatic complexity a quantitative measure of the number of linearly independent paths through a program’s source code. indicate the complexity of a program. A high complexity impacts : Readability Debug tasks (less debug information, less accurate stack trace) Understandability of change information (ie commit description) also a quantitative measure of how difficult your code will be to test the number of different unit test needed for a 100% code coverage. Cyclomatic complexity is about testability 75/421  Cyclomatic complexity : example 76/421  Cyclomatic complexity : calculation Code Flow-chart State diagram statement1(); if (expression1) statement2(); else statement3() statement4(); do statement5(); while (expression2) statement6(); V(G) = e - n + 2 = 10 - 8 + 2 = 4 77/421  Cognitive complexity a qualitative measure of how difficult a unit of code is to be intuitively understood, and therefore to maintain Cognitive complexity is about maintainability 10 78/421 10. https://www.sonarsource.com/docs/CognitiveComplexity.pdf  Cognitive complexity : example Cyclomatic Complexity 4 Cyclomatic Complexity 4 Cognitive Complexity 7 int main() { // Cyclo+1 int i,j,max,sum=0; Cognitive Complexity 1 printf("Enter the maximum value"); scanf("%d",&max); int fooBar(int number) { // Cyclo+1 for(i=2; i90% : Good code coverage 100% : Great code coverage and requested for every critical parts (ex : SIL4) 96/421  C Code coverage : obtain code coverage with gcovr Install gcovr 18 Build ABC_HiCycloComplexity_BadCoding.c and ABC_UnitTests.c together with test coverage options activated Build ABC_LoCycloComplexity_BetterCoding.c and ABC_UnitTests.c together with test coverage options activated 97/421 18. https://gcovr.com/en/5.0/guide.html  C Code coverage : obtain code coverage with gcovr Install gcovr 18 sudo apt install gcovr Build ABC_HiCycloComplexity_BadCoding.c and ABC_UnitTests.c together with test coverage options activated Build ABC_LoCycloComplexity_BetterCoding.c and ABC_UnitTests.c together with test coverage options activated 97/421 18. https://gcovr.com/en/5.0/guide.html  C Code coverage : obtain code coverage with gcovr Install gcovr 18 sudo apt install gcovr Build ABC_HiCycloComplexity_BadCoding.c and ABC_UnitTests.c together with test coverage options activated gcc -Wall -fprofile-arcs -ftest-coverage ABC_HiCycloComplexity_BadCoding.c ABC_UnitTests.c -o UT_BadCoding Build ABC_LoCycloComplexity_BetterCoding.c and ABC_UnitTests.c together with test coverage options activated 97/421 18. https://gcovr.com/en/5.0/guide.html  C Code coverage : obtain code coverage with gcovr Install gcovr 18 sudo apt install gcovr Build ABC_HiCycloComplexity_BadCoding.c and ABC_UnitTests.c together with test coverage options activated gcc -Wall -fprofile-arcs -ftest-coverage ABC_HiCycloComplexity_BadCoding.c ABC_UnitTests.c -o UT_BadCoding Build ABC_LoCycloComplexity_BetterCoding.c and ABC_UnitTests.c together with test coverage options activated gcc -Wall -fprofile-arcs -ftest-coverage ABC_LoCycloComplexity_BetterCoding.c ABC_UnitTests.c -o UT_BetterCoding 97/421 18. https://gcovr.com/en/5.0/guide.html  C Code coverage : obtain code coverage with gcovr Run UT_BadCoding and UT_BetterCoding Run gcovr with no output report 98/421  C Code coverage : obtain code coverage with gcovr Run UT_BadCoding and UT_BetterCoding./UT_BadCoding./UT_BetterCoding Run gcovr with no output report 98/421  C Code coverage : obtain code coverage with gcovr Run UT_BadCoding and UT_BetterCoding./UT_BadCoding./UT_BetterCoding Run gcovr with no output report gcovr. 98/421  C Code Coverage : HTML report Run gcovr with detailed html output report Conclusion about ABC_HiCycloComplexity_BadCoding.c and ABC_LoCycloComplexity_BetterCoding.c coverage ? 99/421  C Code Coverage : HTML report Run gcovr with detailed html output report gcovr. --html --html-details report.html Conclusion about ABC_HiCycloComplexity_BadCoding.c and ABC_LoCycloComplexity_BetterCoding.c coverage ? 99/421  C Code Coverage : HTML report Run gcovr with detailed html output report gcovr. --html --html-details report.html Conclusion about ABC_HiCycloComplexity_BadCoding.c and ABC_LoCycloComplexity_BetterCoding.c coverage ? Same test cases but less coverage for ABC_HiCycloComplexity_BadCoding.c 99/421  C Code Coverage : HTML report - details Inspect which lines are not covered in ABC_HiCycloComplexity_BadCoding.c 100/421  C Code Coverage : HTML report - details Inspect which lines are not covered in ABC_HiCycloComplexity_BadCoding.c 100/421  C Code Coverage : HTML report - details Inspect which branches are not covered in ABC_LoCycloComplexity_BetterCoding.c 101/421  C Code Coverage : HTML report - details Inspect which branches are not covered in ABC_LoCycloComplexity_BetterCoding.c 101/421  Python Code coverage : obtain code coverage with coverage Install coverage 19 Launch tests wih coverage tp-source/python/test2/test_prime_number.py an generate xml and html files Inspect results 102/421 19. https://pypi.org/project/coverage/  Python Code coverage : obtain code coverage with coverage Install coverage 19 pip3 install coverage Launch tests wih coverage tp-source/python/test2/test_prime_number.py an generate xml and html files Inspect results 102/421 19. https://pypi.org/project/coverage/  Python Code coverage : obtain code coverage with coverage Install coverage 19 pip3 install coverage Launch tests wih coverage tp-source/python/test2/test_prime_number.py an generate xml and html files coverage run -m pytest test_prime_number.py coverage xml coverage html Inspect results 102/421 19. https://pypi.org/project/coverage/  Python Code coverage : obtain code coverage with coverage Install coverage 19 pip3 install coverage Launch tests wih coverage tp-source/python/test2/test_prime_number.py an generate xml and html files coverage run -m pytest test_prime_number.py coverage xml coverage html Inspect results coverage report 102/421 19. https://pypi.org/project/coverage/  Python Code Coverage : HTML report - details Inspect which branches are not covered in tp- source/python/test2/test_prime_number.py 103/421  Python Code Coverage : HTML report - details Inspect which branches are not covered in tp- source/python/test2/test_prime_number.py 103/421  Python Code coverage : obtain code coverage with coverage Add tests in order to increase coverage up to 100 How many tests for 100% code coverage ? 104/421  Plan Code Coverage Code Quality Conclusion Good Code vs Bad Code Coding Style Technical Debt Coding Rules Code Review Inspection Static Code Analysis First analysis Dynamic Code Analysis Going further First analysis Complexity Going further Unit testing 105/421  Conclusion What have we learned ? 106/421  Conclusion What have we learned ? 106/421  Conclusion What have we learned ? Using analysis tools 106/421  Conclusion What have we learned ? Using analysis tools Differentiating good and bad coding and why it matters 106/421  Conclusion What have we learned ? Using analysis tools Differentiating good and bad coding and why it matters The importance of complexity and coverage metrics for software quality and easy testing 106/421  Conclusion What have we learned ? Using analysis tools Differentiating good and bad coding and why it matters The importance of complexity and coverage metrics for software quality and easy testing What will we learn next ? 106/421  Conclusion What have we learned ? Using analysis tools Differentiating good and bad coding and why it matters The importance of complexity and coverage metrics for software quality and easy testing What will we learn next ? How to deal with technical debt 106/421  Conclusion What have we learned ? Using analysis tools Differentiating good and bad coding and why it matters The importance of complexity and coverage metrics for software quality and easy testing What will we learn next ? How to deal with technical debt How do we inspect and refactor code 106/421  Plan Technical Debt Code Quality Definition Of Done Conclusion Technical Debt Inspection 107/421  Plan Conclusion Code Quality Inspection First analysis Technical Debt Going further Technical Debt First analysis Definition Of Done Going further 108/421  Technical Debt 109/421  Technical Debt Technical debt is the implied cost of additional rework (and costs) caused by choosing an easy (limited) solution now instead of using a better approach that would take longer : Code smells : No coding styles / rules / Spaghetti architecture / Lack or too much comments... Poor test coverage Lack of software documentation Lack of alignment to standards (ignoring standards frameworks and technologies) High complexity Technical Debt is neither functionality nor testable Technical Debt lacks visibility and therefore hard to evaluate costs and impacts. Technical Debt can’t be fixed at once, and should not be. Often useful in the life of a project, but you’ll got to pay it back at some point... 110/421  Technical Debt : what kind of technical debt are we facing 111/421  Technical Debt 112/421  Technical Debt Root causes Business pressure Obsolescence of solutions / frameworks Team turnover Mooving requirements, last minute specification changes Lack of process / management Lack of skills Parallel development... These issues can accumulate over time if not supervised 113/421  Technical Debt Costs ! The average organization wastes 23-42% of their development time due to technical debt. An issue corrected at validation phase is 10 times more expensive than at development phase An issue corrected at production phase is 100 times more expensive than at development phase Increasing team size leads to increased coordination needs, and oftenly make the development less efficient (Brook’s law). The return on investment is not only cost savings, but also an opportunity to deliver more value with the existing organization Technical Debt is a business problem 114/421  Technical Debt : impacts on team velocity 115/421  Technical Debt : How to control it ? 116/421  Technical Debt : Make It Visible Peer review Automated code analysis Automated unit tests Continuous Code Inspection : Execute continuously automatic analysis and tests View results in a quantitative and qualitative view of code quality. Be transparent about technical debt 117/421  Technical Debt : Analyze Quality Profile : select warnings / configure warning criticity Analyze code quality upon : Reliability (Bugs) Security (Vulnerabilities) Maintenability (Code Smells) Coverage Duplications Differential view : comparison with a reference version Evolution over time Qualimetry and decision analysis : a quantitative and qualitative view of code quality. 118/421  Technical Debt : Choose Choose issues to be corrected according to : Criticity / frequency (Pareto’s Law) Impacts on Reliability and Security First Probability to be stabilized over time : Priorize code that is not going to be updated often Not all technical debt is urgent 119/421  Technical Debt : Control Teach your team how to avoid the most frequent and critical issues at coding level Control continuously your technical debt : DevSecOps Detect Errors / Technical Debt by crossing the “Quality Gate” Configure and automate merge request validation with “Quality Gate” Priorize and schedule technical debt remediation into your backlog Start refactoring problematic code Non-zero debt is acceptable. Beware of over quality ! 120/421  Technical Debt : legacy code Often we inherit a software and we are overwhelmed by a huge number of issues ! LEAK FIRST : Consider your technical debt on new code as if it were new development Use differential view to see, analyze, choose and control new code Reference is before the first line of new code. MUDDY PUDDLE : Technical debt on legacy code See, analyze, choose and control on old code Start refactoring problematic code (critical issues first) Don’t remediate if code is about to change : remediate old code when modifying it or adding new code Sometimes it’s cheaper to recode a feature from scratch 121/421  Plan Conclusion Code Quality Inspection First analysis Technical Debt Going further Technical Debt First analysis Definition Of Done Going further 122/421  DOD : Definition Of Done 123/421  DOD : Definition Of Done DOD is based on the agreed exit-criteria that must be met for a project team to consider that a part of a software development is completed. Example : Code developped and built Documentation completed (user/installation manuel, Readme,...) Automatic code analysis done Code reviewed Testing completed All defects found in previous steps resolved Release notes are complete Technical Debt treated or identified (and put in the backlog) DOD may vary from one team to another but must be consistent within one team. 124/421  Plan Conclusion Code Quality Inspection First analysis Technical Debt Going further Technical Debt First analysis Definition Of Done Going further 125/421  Conclusion What have we learned ? 126/421  Conclusion What have we learned ? How to deal with technical debt 126/421  Conclusion What have we learned ? How to deal with technical debt How do we inspect and refactor code 126/421  Conclusion What have we learned ? How to deal with technical debt How do we inspect and refactor code What will we learn next ? 126/421  Conclusion What have we learned ? How to deal with technical debt How do we inspect and refactor code What will we learn next ? Practicing code inspection with SonarQube 126/421  Conclusion What have we learned ? How to deal with technical debt How do we inspect and refactor code What will we learn next ? Practicing code inspection with SonarQube Setting up a SonarQube analysis 126/421  Conclusion What have we learned ? How to deal with technical debt How do we inspect and refactor code What will we learn next ? Practicing code inspection with SonarQube Setting up a SonarQube analysis Configuring Quality Profile and Quality Gate 126/421  Conclusion What have we learned ? How to deal with technical debt How do we inspect and refactor code What will we learn next ? Practicing code inspection with SonarQube Setting up a SonarQube analysis Configuring Quality Profile and Quality Gate Analyzing Technical Debt 126/421  Conclusion What have we learned ? How to deal with technical debt How do we inspect and refactor code What will we learn next ? Practicing code inspection with SonarQube Setting up a SonarQube analysis Configuring Quality Profile and Quality Gate Analyzing Technical Debt Managing Technical Debt 126/421  Plan Setting up SonarQube configuration Analyzing C source code Code Quality Analyzing Python source code Understanding SonarQube Technical Debt analysis Quality Gate Fixing issues Inspection Using SonarLint Setting up SonarQube Server 127/421  Plan Analyzing C source code First analysis Code Quality Going further Analyzing Python source code Technical Debt First analysis Going further Understanding SonarQube analysis Inspection Quality Gate Setting up SonarQube Server Fixing issues Setting up SonarQube configuration Using SonarLint 128/421  Setting up SonarQube Server This section is mainly practical work... and we will meet a lot of problems, and we will fix them ! Open Sonarqube in your browser : URL provided by the teacher Login with : login/pwd provided by the teacher Install sonar scanner CLI : https://docs.sonarsource.com/sonarqube/latest/analyzing- source-code/scanners/sonarscanner/ 129/421  Plan Analyzing C source code First analysis Code Quality Going further Analyzing Python source code Technical Debt First analysis Going further Understanding SonarQube analysis Inspection Quality Gate Setting up SonarQube Server Fixing issues Setting up SonarQube configuration Using SonarLint 130/421  Configuring sonar-project.properties : generic properties Copy sonarqube properties file tp-source/sonar-project-properties into tp- source/c/abc/ Configure Sonarqube properties 20 into sonar-project-properties : Sonar Host URL [M] : IP address of SonarQube server Sonar login [M] : Token created from SonarQube server http://localhost:9000/account/security/ Sonar Project Key [M] : Your project key (fill it instead of "tp :projectkey") Sonar Project Name : Project name (fill it instead of "PROJECT NAME") Sonar Project Version : Project version 131/421 20. https://docs.sonarqube.org/latest/analysis/analysis-parameters/  Configuring sonar-project.properties : generic properties Copy sonarqube properties file tp-source/sonar-project-properties into tp- source/c/abc/ Configure Sonarqube properties 20 into sonar-project-properties : Sonar Host URL [M] : IP address of SonarQube server sonar.host.url=http://sonar:9000 Sonar login [M] : Token created from SonarQube server http://localhost:9000/account/security/ Sonar Project Key [M] : Your project key (fill it instead of "tp :projectkey") Sonar Project Name : Project name (fill it instead of "PROJECT NAME") Sonar Project Version : Project version 131/421 20. https://docs.sonarqube.org/latest/analysis/analysis-parameters/  Configuring sonar-project.properties : generic properties Copy sonarqube properties file tp-source/sonar-project-properties into tp- source/c/abc/ Configure Sonarqube properties 20 into sonar-project-properties : Sonar Host URL [M] : IP address of SonarQube server sonar.host.url=http://sonar:9000 Sonar login [M] : Token created from SonarQube server http://localhost:9000/account/security/ sonar.login=MY-TOKEN Sonar Project Key [M] : Your project key (fill it instead of "tp :projectkey") Sonar Project Name : Project name (fill it instead of "PROJECT NAME") Sonar Project Version : Project version 131/421 20. https://docs.sonarqube.org/latest/analysis/analysis-parameters/  Configuring sonar-project.properties : generic properties Copy sonarqube properties file tp-source/sonar-project-properties into tp- source/c/abc/ Configure Sonarqube properties 20 into sonar-project-properties : Sonar Host URL [M] : IP address of SonarQube server sonar.host.url=http://sonar:9000 Sonar login [M] : Token created from SonarQube server http://localhost:9000/account/security/ sonar.login=MY-TOKEN Sonar Project Key [M] : Your project key (fill it instead of "tp :projectkey") sonar.projectKey=tp:projectkey Sonar Project Name : Project name (fill it instead of "PROJECT NAME") Sonar Project Version : Project version 131/421 20. https://docs.sonarqube.org/latest/analysis/analysis-parameters/  Configuring sonar-project.properties : generic properties Copy sonarqube properties file tp-source/sonar-project-properties into tp- source/c/abc/ Configure Sonarqube properties 20 into sonar-project-properties : Sonar Host URL [M] : IP address of SonarQube server sonar.host.url=http://sonar:9000 Sonar login [M] : Token created from SonarQube server http://localhost:9000/account/security/ sonar.login=MY-TOKEN Sonar Project Key [M] : Your project key (fill it instead of "tp :projectkey") sonar.projectKey=tp:projectkey Sonar Project Name : Project name (fill it instead of "PROJECT NAME") sonar.projectKey=PROJECT NAME Sonar Project Version : Project version 131/421 20. https://docs.sonarqube.org/latest/analysis/analysis-parameters/  Configuring sonar-project.properties : generic properties Copy sonarqube properties file tp-source/sonar-project-properties into tp- source/c/abc/ Configure Sonarqube properties 20 into sonar-project-properties : Sonar Host URL [M] : IP address of SonarQube server sonar.host.url=http://sonar:9000 Sonar login [M] : Token created from SonarQube server http://localhost:9000/account/security/ sonar.login=MY-TOKEN Sonar Project Key [M] : Your project key (fill it instead of "tp :projectkey") sonar.projectKey=tp:projectkey Sonar Project Name : Project name (fill it instead of "PROJECT NAME") sonar.projectKey=PROJECT NAME Sonar Project Version : Project version sonar.projectVersion=1.0 131/421 20. https://docs.sonarqube.org/latest/analysis/analysis-parameters/  Plan Analyzing C source code First analysis Code Quality Going further Analyzing Python source code Technical Debt First analysis Going further Understanding SonarQube analysis Inspection Quality Gate Setting up SonarQube Server Fixing issues Setting up SonarQube configuration Using SonarLint 132/421  Configuring sonar-project.properties : First analysis Run sonar-scanner Open your project http://localhost:9000/projects 133/421  Configuring sonar-project.properties : First analysis Run sonar-scanner sonar-scanner Open your project http://localhost:9000/projects 133/421  Configuring sonar-project.properties : First analysis Run sonar-scanner sonar-scanner Open your project http://localhost:9000/projects 133/421  Configuring sonar-project.properties : First analysis Run sonar-scanner sonar-scanner Open your project http://localhost:9000/projects No source code analyzed ! 133/421  Configuring sonar-project.properties : First analysis 134/421  Configuring sonar-project.properties : First analysis All files are checked but we only want C files 134/421  Configuring sonar-project.properties : First analysis All files are checked but we only want C files No CXX sensor 134/421  Configuring sonar-project.properties : C language specific properties 21 Configure CXX properties into sonar-project-properties : Source language if C source code [M] Source language if C++ source code [M] Sources : Source code path 135/421 21. https://github.com/SonarOpenCommunity/sonar-cxx  Configuring sonar-project.properties : C language specific properties 21 Configure CXX properties into sonar-project-properties : Source language if C source code [M] sonar.cxx.file.suffixes=.h,.c Source language if C++ source code [M] Sources : Source code path 135/421 21. https://github.com/SonarOpenCommunity/sonar-cxx  Configuring sonar-project.properties : C language specific properties 21 Configure CXX properties into sonar-project-properties : Source language if C source code [M] sonar.cxx.file.suffixes=.h,.c Source language if C++ source code [M] sonar.cxx.file.suffixes=.hpp,.cpp Sources : Source code path 135/421 21. https://github.com/SonarOpenCommunity/sonar-cxx  Configuring sonar-project.properties : C language specific properties 21 Configure CXX properties into sonar-project-properties : Source language if C source code [M] sonar.cxx.file.suffixes=.h,.c Source language if C++ source code [M] sonar.cxx.file.suffixes=.hpp,.cpp Sources : Source code path sonar.sources=. 135/421 21. https://github.com/SonarOpenCommunity/sonar-cxx  Analyzing C source code : Configuring sonar-project.properties Run sonar-scanner Open your project http://localhost:9000/projects 136/421  Analyzing C source code : Configuring sonar-project.properties Run sonar-scanner sonar-scanner Open your project http://localhost:9000/projects 136/421  Analyzing C source code : Configuring sonar-project.properties Run sonar-scanner sonar-scanner Open your project http://localhost:9000/projects 136/421  Analyzing C source code : Configuring sonar-project.properties Run sonar-scanner sonar-scanner Open your project http://localhost:9000/projects Source code analyzed but no warnings at all ! CXX Sensor running, Quality profile = Sonar Way ! 136/421  Analyzing C source code : Configuring sonar-project.properties Analyze ABC_HiCycloComplexity_BadCoding.c with cppcheck Run sonar-scanner Open your project http://localhost:9000/projects 137/421  Analyzing C source code : Configuring sonar-project.properties Analyze ABC_HiCycloComplexity_BadCoding.c with cppcheck cppcheck -v --enable=all --xml-version=2 ABC_HiCycloComplexity_BadCoding.c 2> cppcheck-report.xml Run sonar-scanner Open your project http://localhost:9000/projects 137/421  Analyzing C source code : Configuring sonar-project.properties Analyze ABC_HiCycloComplexity_BadCoding.c with cppcheck cppcheck -v --enable=all --xml-version=2 ABC_HiCycloComplexity_BadCoding.c 2> cppcheck-report.xml Run sonar-scanner sonar-scanner Open your project http://localhost:9000/projects 137/421  Analyzing C source code : Configuring sonar-project.properties Analyze ABC_HiCycloComplexity_BadCoding.c with cppcheck cppcheck -v --enable=all --xml-version=2 ABC_HiCycloComplexity_BadCoding.c 2> cppcheck-report.xml Run sonar-scanner sonar-scanner Open your project http://localhost:9000/projects Source code analyzed but no warnings at all ! CXX Sensor running, Quality profile = Sonar Way ! 137/421  Analyzing C source code : Configuring sonar-project.properties 22 Configure CXX properties into sonar-project-properties : Reports paths : Run sonar-scanner Open your project http://localhost:9000/projects 138/421 22. https://github.com/SonarOpenCommunity/sonar-cxx  Analyzing C source code : Configuring sonar-project.properties 22 Configure CXX properties into sonar-project-properties : Reports paths : sonar.cxx.cppcheck.reportPaths=cppcheck-report.xml Run sonar-scanner Open your project http://localhost:9000/projects 138/421 22. https://github.com/SonarOpenCommunity/sonar-cxx  Analyzing C source code : Configuring sonar-project.properties 22 Configure CXX properties into sonar-project-properties : Reports paths : sonar.cxx.cppcheck.reportPaths=cppcheck-report.xml Run sonar-scanner sonar-scanner Open your project http://localhost:9000/projects 138/421 22. https://github.com/SonarOpenCommunity/sonar-cxx  Analyzing C source code : Configuring sonar-project.properties 22 Configure CXX properties into sonar-project-properties : Reports paths : sonar.cxx.cppcheck.reportPaths=cppcheck-report.xml Run sonar-scanner sonar-scanner Open your project http://localhost:9000/projects Source code analyzed but no warnings at all ! CXX Sensor running, Quality profile = Sonar Way ! 138/421 22. https://github.com/SonarOpenCommunity/sonar-cxx  Analyzing C source code : Configuring Quality Profile List quality profiles http://localhost:9000/profiles Copy CXX profile "Sonar way" into "My way" Add cppcheck rules into "My way" : Configure your project to "My way" Quality Profile 139/421  Analyzing C source code : Configuring Quality Profile List quality profiles http://localhost:9000/profiles Copy CXX profile "Sonar way" into "My way" Add cppcheck rules into "My way" : Activate more -> Tag cppcheck -> Bulk change -> Activate in My way -> Apply Configure your project to "My way" Quality Profile 139/421  Analyzing C source code : Configuring Quality Profile List quality profiles http://localhost:9000/profiles Copy CXX profile "Sonar way" into "My way" Add cppcheck rules into "My way" : Activate more -> Tag cppcheck -> Bulk change -> Activate in My way -> Apply Configure your project to "My way" Quality Profile Go to project dashboard -> Project Settings -> Quality Profiles -> Change profile -> Choose "My way" 139/421  Analyzing C source code : Configuring Quality Profile Run sonar-scanner 140/421  Analyzing C source code : Configuring Quality Profile Run sonar-scanner 140/421  Analyzing C source code : Configuring Quality Profile Source code analyzed ! Run sonar-scanner 140/421  Analyzing C source code : Configuring Quality Profile Source code analyzed ! Run sonar-scanner CXX Sensor running Quality profile = My Way ! 140/421  Analyzing C source code : Configuring Quality Profile Source code analyzed ! Run sonar-scanner CXX Sensor running Quality profile = My Way ! 140/421  Analyzing C source code : Configuring Quality Profile Source code analyzed ! Run sonar-scanner CXX Sensor running Quality profile = My Way ! But... 140/421  Analyzing C source code : Configuring Quality Profile Source code analyzed ! Run sonar-scanner CXX Sensor running Quality profile = My Way ! But...... only 1 C file analyzed and 5 files indexed (including 1 xml file and unit tests C file) 140/421  Analyzing C source code : Configuring Quality Profile Source code analyzed ! Run sonar-scanner CXX Sensor running Quality profile = My Way ! But...... only 1 C file analyzed and 5 files indexed (including 1 xml file and unit tests C file)... no general analysis 140/421  Going further : Quality Profile With "My way" quality profile : Add CXX rules (Langage CXX -> Repository) : Compiler-GCC CXX rules Clang-tidy CXX rules Clang-SA CXX rules Valgrind CXX Common CXX SonarQube CXX Set profile as default 141/421  Going further : Sonar project properties sonar-project.properties for ABC project : Analyze only source files (put your files into a subdirectory) Identify test source files (put your files into a subdirectory) Check your changes by running sonar-scanner 142/421  Going further : Sonar project properties sonar-project.properties for ABC project : Analyze only source files (put your files into a subdirectory) sonar.sources=src Identify test source files (put your files into a subdirectory) Check your changes by running sonar-scanner 142/421  Going further : Sonar project properties sonar-project.properties for ABC project : Analyze only source files (put your files into a subdirectory) sonar.sources=src Identify test source files (put your files into a subdirectory) sonar.tests=test Check your changes by running sonar-scanner 142/421  Going further : Built, Test, Coverage... With ABC project : Analyze ABC*.c with clang-tidy and send report to SonarQube Analyze ABC*.c with cppcheck and send report to SonarQube Build tests and send coverage report to SonarQube 143/421  Going further : sonar-project-properties With ABC project : Configure sonar-project.properties : Configure SonarQube properties (project name / project key / source code / etc...) Configure CXX properties 23 into sonar-project-properties : clang tidy report : cppcheck report : Coverage report : 144/421 23. https://github.com/SonarOpenCommunity/sonar-cxx  Going further : sonar-project-properties With ABC project : Configure sonar-project.properties : Configure SonarQube properties (project name / project key / source code / etc...) Configure CXX properties 23 into sonar-project-properties : clang tidy report : sonar.cxx.clangtidy.reportPaths=clang-tidy-report.txt cppcheck report : Coverage report : 144/421 23. https://github.com/SonarOpenCommunity/sonar-cxx  Going further : sonar-project-properties With ABC project : Configure sonar-project.properties : Configure SonarQube properties (project name / project key / source code / etc...) Configure CXX properties 23 into sonar-project-properties : clang tidy report : sonar.cxx.clangtidy.reportPaths=clang-tidy-report.txt cppcheck report : sonar.cxx.cppcheck.reportPaths=cppcheck-report.xml Coverage report : 144/421 23. https://github.com/SonarOpenCommunity/sonar-cxx  Going further : sonar-project-properties With ABC project : Configure sonar-project.properties : Configure SonarQube properties (project name / project key / source code / etc...) Configure CXX properties 23 into sonar-project-properties : clang tidy report : sonar.cxx.clangtidy.reportPaths=clang-tidy-report.txt cppcheck report : sonar.cxx.cppcheck.reportPaths=cppcheck-report.xml Coverage report : sonar.cxx.cobertura.reportPaths=coverage-report.xml 144/421 23. https://github.com/SonarOpenCommunity/sonar-cxx  Going further : source code analysis With ABC project : Run source code analysis : clang tidy analysis : cppcheck analysis : Build tests : 145/421  Going further : source code analysis With ABC project : Run source code analysis : clang tidy analysis : clang-tidy -checks=* ABC*Cyclo*.c > clang-tidy-report.txt cppcheck analysis : Build tests : 145/421  Going further : source code analysis With ABC project : Run source code analysis : clang tidy analysis : clang-tidy -checks=* ABC*Cyclo*.c > clang-tidy-report.txt cppcheck analysis : cppcheck -v --enable=all --xml-version=2 ABC*Cyclo*.c 2>cppcheck-report.xml Build tests : 145/421  Going further : source code analysis With ABC project : Run source code analysis : clang tidy analysis : clang-tidy -checks=* ABC*Cyclo*.c > clang-tidy-report.txt cppcheck analysis : cppcheck -v --enable=all --xml-version=2 ABC*Cyclo*.c 2>cppcheck-report.xml Build tests : gcc -Wall -fprofile-arcs -ftest-coverage ABC_HiCycloComplexity_BadCoding.c ABC_UnitTests.c -o UT_BadCoding 145/421  Going further : source code analysis With ABC project : Run source code analysis : clang tidy analysis : clang-tidy -checks=* ABC*Cyclo*.c > clang-tidy-report.txt cppcheck analysis : cppcheck -v --enable=all --xml-version=2 ABC*Cyclo*.c 2>cppcheck-report.xml Build tests : gcc -Wall -fprofile-arcs -ftest-coverage ABC_HiCycloComplexity_BadCoding.c ABC_UnitTests.c -o UT_BadCoding gcc -Wall -fprofile-arcs -ftest-coverage ABC_LoCycloComplexity_BetterCoding.c ABC_UnitTests.c -o UT_BetterCoding 145/421  Going further : source code analysis With ABC project : Run source code analysis : Run tests : Coverage analysis : Run sonar-scanner 146/421  Going further : source code analysis With ABC project : Run source code analysis : Run tests :./UT_BadCoding Coverage analysis : Run sonar-scanner 146/421  Going further : source code analysis With ABC project : Run source code analysis : Run tests :./UT_BadCoding./UT_BetterCoding Coverage analysis : Run sonar-scanner 146/421  Going further : source code analysis With ABC project : Run source code analysis : Run tests :./UT_BadCoding./UT_BetterCoding Coverage analysis : gcovr. --xml coverage-report.xml Run sonar-scanner 146/421  Going further : Built, Test, Coverage... With memory-trouble project : Analyze main.c with clang-tidy and send report to SonarQube Analyze main.c with cppcheck and send report to SonarQube Build and analyze with Valgrind and send report to SonarQube... and send coverage report to SonarQube 147/421  Going further : sonar-project-properties With memory-trouble project : Configure sonar-project.properties : Configure SonarQube properties (project name / project key / source code / etc...) 24 Configure CXX properties into sonar-project-properties : clang tidy report : cppcheck report : Valgrind report : Coverage report : 148/421 24. https://github.com/SonarOpenCommunity/sonar-cxx  Going further : sonar-project-properties With memory-trouble project : Configure sonar-project.properties : Configure SonarQube properties (project name / project key / source code / etc...) 24 Configure CXX properties into sonar-project-properties : clang tidy report : sonar.cxx.clangtidy.reportPaths=clang-tidy-report.txt cppcheck report : Valgrind report : Coverage report : 148/421 24. https://github.com/SonarOpenCommunity/sonar-cxx  Going further : sonar-project-properties With memory-trouble project : Configure sonar-project.properties : Configure SonarQube properties (project name / project key / source code / etc...) 24 Configure CXX properties into sonar-project-properties : clang tidy report : sonar.cxx.clangtidy.reportPaths=clang-tidy-report.txt cppcheck report : sonar.cxx.cppcheck.reportPaths=cppcheck-report.xml Valgrind report : Coverage report : 148/421 24. https://github.com/SonarOpenCommunity/sonar-cxx  Going further : sonar-project-properties With memory-trouble project : Configure sonar-project.properties : Configure SonarQube properties (project name / project key / source code / etc...) 24 Configure CXX properties into sonar-project-properties : clang tidy report : sonar.cxx.clangtidy.reportPaths=clang-tidy-report.txt cppcheck report : sonar.cxx.cppcheck.reportPaths=cppcheck-report.xml Valgrind report : sonar.cxx.valgrind.reportPaths=valgrind-report.xml Coverage report : 148/421 24. https://github.com/SonarOpenCommunity/sonar-cxx  Going further : sonar-project-properties With memory-trouble project : Configure sonar-project.properties : Configure SonarQube properties (project name / project key / source code / etc...) 24 Configure CXX properties into sonar-project-properties : clang tidy report : sonar.cxx.clangtidy.reportPaths=clang-tidy-report.txt cppcheck report : sonar.cxx.cppcheck.reportPaths=cppcheck-report.xml Valgrind report : sonar.cxx.valgrind.reportPaths=valgrind-report.xml Coverage report : sonar.cxx.cobertura.reportPaths=coverage-report.xml 148/421 24. https://github.com/SonarOpenCommunity/sonar-cxx  Going further : source code analysis With memory-trouble project : Run source code analysis : clang tidy analysis : cppcheck analysis : Valgrind analysis : Coverage analysis : Run sonar-scanner 149/421  Going further : source code analysis With memory-trouble project : Run source code analysis : clang tidy analysis : clang-tidy -checks=* main.c > clang-tidy-report.txt cppcheck analysis : Valgrind analysis : Coverage analysis : Run sonar-scanner 149/421  Going further : source code analysis With memory-trouble project : Run source code analysis : clang tidy analysis : clang-tidy -checks=* main.c > clang-tidy-report.txt cppcheck analysis : cppcheck -v --enable=all --xml-version=2 main.c 2>cppcheck-report.xml Valgrind analysis : Coverage analysis : Run sonar-scanner 149/421  Going further : source code analysis With memory-trouble project : Run source code analysis : clang tidy analysis : clang-tidy -checks=* main.c > clang-tidy-report.txt cppcheck analysis : cppcheck -v --enable=all --xml-version=2 main.c 2>cppcheck-report.xml Valgrind analysis : gcc main.c -O0 -gdwarf -Wall -Wextra -Werror -fprofile-arcs -ftest- coverage -o main-Valgrind.out Coverage analysis : Run sonar-scanner 149/421  Going further : source code analysis With memory-trouble project : Run source code analysis : clang tidy analysis : clang-tidy -checks=* main.c > clang-tidy-report.txt cppcheck analysis : cppcheck -v --enable=all --xml-version=2 main.c 2>cppcheck-report.xml Valgrind analysis : gcc main.c -O0 -gdwarf -Wall -Wextra -Werror -fprofile-arcs -ftest- coverage -o main-Valgrind.out valgrind --xml=yes --xml-file=valgrind-report.xml --track-origins=yes --leak-check=full --track-fds=yes./main-Valgrind.out Param1 Coverage analysis : Run sonar-scanner 149/421  Going further : source code analysis With memory-trouble project : Run source code analysis : clang tidy analysis : clang-tidy -checks=* main.c > clang-tidy-report.txt cppcheck analysis : cppcheck -v --enable=all --xml-version=2 main.c 2>cppcheck-report.xml Valgrind analysis : gcc main.c -O0 -gdwarf -Wall -Wextra -Werror -fprofile-arcs -ftest- coverage -o main-Valgrind.out valgrind --xml=yes --xml-file=valgrind-report.xml --track-origins=yes --leak-check=full --track-fds=yes./main-Valgrind.out Param1 Coverage analysis : gcovr. --xml coverage-report.xml Run sonar-scanner 149/421  Plan Analyzing C source code First analysis Code Quality Going further Analyzing Python source code Technical Debt First analysis Going further Understanding SonarQube analysis Inspection Quality Gate Setting up SonarQube Server Fixing issues Setting up SonarQube configuration Using SonarLint 150/421  Configuring sonar-project.properties : Python language specific properties 25 Configure Python properties into sonar-project-properties : 151/421 25.  Configuring sonar-project.properties : Python language specific properties Configure Python properties 25 into sonar-project-properties : sonar.python.version=3.8 151/421 25.  Configuring sonar-project.properties : Python language specific properties Configure Python properties 25 into sonar-project-properties : sonar.python.version=3.8 Use python --version to check python version in your console 151/421 25.  Analyzing Python source code : Configuring sonar-project.properties Run sonar-scanner Open your project http://localhost:9000/projects 152/421  Analyzing Python source code : Configuring sonar-project.properties Run sonar-scanner sonar-scanner Open your project http://localhost:9000/projects 152/421  Analyzing Python source code : Configuring sonar-project.properties Run sonar-scanner sonar-scanner Open your project http://localhost:9000/projects 152/421  Analyzing Python source code : Configuring sonar-project.properties Run sonar-scanner sonar-scanner Open your project http://localhost:9000/projects Source code analyzed but no warnings at all ! Python Sensor running, Quality profile = Sonar Way ! 152/421  Analyzing Python source code : Configuring sonar-project.properties Analyze test2/prime_number.py with pylint Run sonar-scanner Open your project http://localhost:9000/projects 153/421  Analyzing Python source code : Configuring sonar-project.properties Analyze test2/prime_number.py with pylint pylint prime_number.py --output-format=parseable > pylint- report.txt Run sonar-scanner Open your project http://localhost:9000/projects 153/421  Analyzing Python source code : Configuring sonar-project.properties Analyze test2/prime_number.py with pylint pylint prime_number.py --output-format=parseable > pylint- report.txt Run sonar-scanner sonar-scanner Open your project http://localhost:9000/projects 153/421  Analyzing Python source code : Configuring sonar-project.properties Analyze test2/prime_number.py with pylint pylint prime_number.py --output-format=parseable > pylint- report.txt Run sonar-scanner sonar-scanner Open your project http://localhost:9000/projects Source code analyzed but no warnings at all ! Python Sensor running, Quality profile = Sonar Way ! 153/421  Analyzing Python source code : Configuring sonar-project.properties 26 Configure Python properties into sonar-project-properties : Reports paths : Run sonar-scanner Open your project http://localhost:9000/projects https://docs.sonarqube.org/8.9/analyzing-source-code/importing-external-issues/importing-third- 154/421 party-issues/  Analyzing Python source code : Configuring sonar-project.properties 26 Configure Python properties into sonar-project-properties : Reports paths : sonar.python.pylint.reportPaths=pylint-report.txt Run sonar-scanner Open your project http://localhost:9000/projects https://docs.sonarqube.org/8.9/analyzing-source-code/importing-external-issues/importing-third- 154/421 party-issues/  Analyzing Python source code : Configuring sonar-project.properties 26 Configure Python properties into sonar-project-properties : Reports paths : sonar.python.pylint.reportPaths=pylint-report.txt Run sonar-scanner sonar-scanner Open your project http://localhost:9000/projects https://docs.sonarqube.org/8.9/analyzing-source-code/importing-external-issues/importing-third- 154/421 party-issues/  Analyzing Python source code : Configuring sonar-project.properties 26 Configure Python properties into sonar-project-properties : Reports paths : sonar.python.pylint.reportPaths=pylint-report.txt Run sonar-scanner sonar-scanner Open your project http://localhost:9000/projects Source code analyzed and 2 warnings are shown (from pylint) ! Python Sensor running, Quality profile = Sonar Way ! https://docs.sonarqube.org/8.9/analyzing-source-code/importing-external-issues/importing-third- 154/421 party-issues/  Analyzing Python source code : Configuring Quality Profile List quality profiles http://localhost:9000/profiles Copy Python profile "Sonar way" into "My way" Add pylint rules into "My way" : Configure your project to "My way" Quality Profile 155/421  Analyzing Python source code : Configuring Quality Profile List quality profiles http://localhost:9000/profiles Copy Python profile "Sonar way" into "My way" Add pylint rules into "My way" : Activate more -> Tag pylint -> Bulk change -> Activate in My way -> Apply Configure your project to "My way" Quality Profile 155/421  Analyzing Python source code : Configuring Quality Profile List quality profiles http://localhost:9000/profiles Copy Python profile "Sonar way" into "My way" Add pylint rules into "My way" : Activate more -> Tag pylint -> Bulk change -> Activate in My way -> Apply Configure your project to "My way" Quality Profile Go to project dashboard -> Project Settings -> Quality Profiles -> Change profile -> Choose "My way" 155/421  Analyzing Python source code : Configuring Quality Profile Run sonar-scanner 156/421  Analyzing Python source code : Configuring Quality Profile Run sonar-scanner 156/421  Analyzing Python source code : Configuring Quality Profile Run sonar-scanner Source code analyzed ! 156/421  Analyzing Python source code : Configuring Quality Profile Run sonar-scanner Source code analyzed ! Python Sensor running Quality profile = My Way ! 156/421  Analyzing Python source code : Configuring Quality Profile Run sonar-scanner Source code analyzed ! Python Sensor running Quality profile = My Way ! 156/421  Analyzing Python source code : Configuring Quality Profile Run sonar-scanner Source code analyzed ! Python Sensor running Quality profile = My Way ! But... 156/421  Analyzing Python source code : Configuring Quality Profile Run sonar-scanner Source code analyzed ! Python Sensor running Quality profile = My Way ! But...... 2 python files analyzed (including unit tests file) 156/421  Going further : Quality Profile With "My way" quality profile : Add Full python rules : Common Python SonarQube Python Set profile as default 157/421  Going further : Sonar project properties sonar-project.properties for project : Analyze only source files (put your files into a subdirectory) Identify test source files (put your files into a subdirectory) Check your changes by running sonar-scanner 158/421  Going further : Sonar project properties sonar-project.properties for project : Analyze only source files (put your files into a subdirectory) sonar.sources=src Identify test source files (put your files into a subdirectory) Check your changes by running sonar-scanner 158/421  Going further : Sonar project properties sonar-project.properties for project : Analyze only source files (put your files into a subdirectory) sonar.sources=src Identify test source files (put your files into a subdirectory) sonar.tests=test Check your changes by running sonar-scanner 158/421  Going further : Built, Test, Coverage... Build tests and send coverage report to SonarQube : Configure SonarQube properties (project name / project key / source code / etc...) Configure Python coverage and test reports properties into sonar-project-properties : Run Coverage analysis : Run sonar-scanner 159/421  Going further : Built, Test, Coverage... Build tests and send coverage report to SonarQube : Configure SonarQube properties (project name / project key / source code / etc...) Configure Python coverage and test reports proper