AP-Lecture1c-IoC+DI+SpringBeans (2)-merged.pdf

Advanced Programming (WBCS053-05) Inversion of Control and Spring Beans Daniel Feitosa Inversion of Control Design principle in which custom code is executed by an external code. The control of parts of your code is delegated to another logical unit. But that requires a predeﬁned interface. Think of when you submit assignment to themis. You did not write the tests in the program, but they still get executed against your program, as long as the input and output processing is correct 2 public class SimpleProduct implements Product { private String name; Inversion of Control private double price; public SimpleProduct (String name, float price) { this.name = name; A } this.price = price; There are several ways to //... implement Inversion of public String getName() { return name; } Control } public double getPrice () { return price; } public abstract class DecoratorPattern implements Product { protected Product product; Example: public DecoratorPattern (Product product ) { this.product = product; B } Where is the product @Override created in code A? public double getPrice () { return product.getPrice(); } And in B? @Override public String getName() { return product.getName(); } } 3 Dependency Injection The example B in the previous slide is what is called Dependency Injection. DI is one form of IoC. In Dependency Injection, the control being inverted is the creation of dependencies. In the previous example, the product object (i.e., the dependency) is not being created by the owner (i.e., DecoratorPattern) but it is being injected by another class (in that case, the class instantiating DecoratorPattern) 4 What enables this Spring Framework such ﬂexibility? Framework for enterprise Java Developers focus on application-level development, i.e., providing logic and underlying complexities are infrastructure support for building Java handled by the framework. For example: applications. Spring Boot: Simpliﬁes the setup and conﬁguration of Spring applications, incl. Web REST APIs. Spring Data: Provides data access and manipulation tools. Spring Security: Offers security features for applications. 5 Beans are like Beans stem cells of spring applications. Beans are the building blocks of spring applications. Beans are instantiated, managed and conﬁgured by Spring's IoC container. The IoC container does not run the application itself; it ensures all components are properly conﬁgured and available. The application is started by an application server or Spring Boot context. 6 Conﬁguring Beans @Service public class ProductService { private Repository repository ; Beans are conﬁgured to assume a public Product getProductById (int id) { } speciﬁc responsibility. public Iterable< Product> getAllProducts () { } } For example, using: @Component, @Service, @RestController public class ProductRestController { @Repository) private BookService bookService ; Java-based conﬁguration public Product getProduct (int id) { } (@Conﬁguration and @Bean } public Iterable< Book> getAllProducts () { } methods). This is an incomplete example. @Service and @RestController conﬁgurations are missing. Do not miss the Spring tutorial. 7 Bean Typical Lifecycle IoC Container IoC Container Starts Stops Bean Custom Init Bean is ready Custom Destroy Instantiated Method to work Method Dependencies Internal Spring Injected Processing 8 Bean Scopes The lifecycle of a bean will be impacted by its scope. For example: singleton beans will be instantiated and destroyed with IoC container prototype beans will be instantiated for each request but not destroyed. 9 Using Spring DO NOT MISS THE TUTORIAL 10 Further reading Book: Chapter 8, "Inversion of Control," in Introduction to Software Design with Java by Martin P. Robillard. Springer Cham, DOI: 10.1007/978-3-030-97899-0 Blog: Inversion Of Control, by Martin Fowler. Available online: https://martinfowler.com/bliki/InversionOfControl.html Documentation: Spring IoC container and Beans. Available online: https://docs.spring.io/spring-framework/reference/core/beans.html 11 Advanced Programming (WBCS053-05) Conﬁguration Management Daniel Feitosa Conﬁguration Management (CM) Conﬁguration management refers to the process by which all artifacts relevant to your project, and the relationships between them, are stored, retrieved, uniquely identiﬁed, and modiﬁed. Humble and Farley, Continuous Delivery 2 What CM implies How to manage changes to your project Record the evolution of the system Govern how the team collaborates For example… For example… Anyone in the team should be able to I want to change a feature. Can I easily reproduce the environments, including ﬁnd the information that need (code, the version of the operating system, the documentation, related dependencies), network conﬁguration, the software make the change, check correctness stack, the applications deployed into it, (tests), check quality, and deploy the and their conﬁguration. change with little to no intervention? 3 Version Control 4 Version Control Systems What it means… Why to control versions… 1. Retain, and provide access to, Reproduction: Reproduce a particular every version of every ﬁle that has state of the software that existed in ever been stored in it. production at some point in time. 2. Allow collaboration across Auditing: What, when, who, and why. locations and time. Provide a complete history of the application. BONUS QUESTION Deletion is cheap! You can What is a "software state"? rollback or copy over from an old software state. 5 To git or not to git… Is X relevant for creating the software? X goes in the VCS source code tests BUT: generated artifacts stay out documentation database scripts build scripts deployment scripts application conﬁguration ﬁles development conﬁguration ﬁles … 6 How to git Atomic Commits Simpler to roll back and change code yes merges; safer refactoring no run tests failed? git pull yes changes? Use Descriptive Messages Facilitates Debugging and Maintenance no Identify author and context Crucial for more rapid debugging Enhances Team Communication and Collaboration git add Paint project history Assist in code reviews Help when author is unavailable git commit -m "" 7 Branching needs clear Be mindful of How to branch merging policies. the dangers of excessive branching. feat:x-endpoint feat:x-endpoint dev main bug:sorting 8 Semantic Versioning v1.0.2-pre-release+build Major Version Minor Version Patch Optional ﬁelds Major changes Minor changes Bug/Security ﬁxes (e.g., breaks API) (e.g., change that does not modify must update interface, new feature) backwards compatible 9 Managing Software Conﬁguration 10 Application changes behavior manipulates Software Conﬁguration Data (binary/script) (at build, deploy or run time) Treat your software conﬁguration the same way you treat your code 11 Conﬁgurability is not free lunch More conﬁgurability 😉 → More ﬂexibility 😁 → More complexity 😨 Ultimate Conﬁgurability Tracking Problems Client: "Add @, #, $, %, & Development team: "We conﬁguration options" can't change 💩 with so Conﬁguration is often text many options to track!" Broken code is more easily detectable Broken conﬁguration might only appear in production Smoke-Test your deployment! 12 Types of Software Conﬁguration Avoid it. Promotes similarity between test and production environments Conﬁguration incorporated during… Build time Packaging time E.g., C++ macros (e.g., "#ifdef DEBUG"), E.g., minifying and bundling CSS and Maven/Gradle proﬁles (incl/excl JavaScript ﬁles differently based on the module/classes). target environment. Deployment(/installation) time Run time E.g., assign connection details, E.g., CLI arguments, environment usernames, or licensing during setup. variables, user conﬁg ﬁles (e.g., conﬁg.yaml, settings.json). 13 DON'T Passwords (and secrets) Handling Conﬁguration do not belong in the VCS. 1. How to represent conﬁguration? Abstract conﬁguration loading from the chosen conﬁguration format. 2. How do deployment scripts access it? 3. How does it vary between versions? getProperty(A) or getPropertyA() Easier to: change conﬁguration fake conﬁguration for testing 14 Managing Application Conﬁguration 15 Application Environments Test environments are close (as possible) replicas of the production environment: ﬁnd bugs as early as possible. test the process to manage production. Includes: Examples: Hardware Software Development Data Test Infrastructure Production External systems QUESTION What are examples of these requirements? 16 Environment Conﬁguration Some environment conﬁguration are common to all environments. Environments also require conﬁguration: For example, core libraries (such as spring boot in the project). Libraries, incl. special dependencies (e.g., tests) Software (e.g., database), incl. versions and conﬁguration Connection to a speciﬁc database Networking conﬁguration Conﬁguration to external services Operating systems, incl. versions, patch levels, and conﬁguration settings... Whenever possible, Treat your environment Creating a new environment environment conﬁguration the same way you treat should be cheaper than should be stored in the VCS. your code repairing an old one. 17 Effective Conﬁguration Management Principles: Core concepts: Keep binary ﬁles independent Provisioning: creation (i.e., deployment) of an environment. from conﬁguration information. Baseline: Environment in properly deployed state. Keep all conﬁguration information in one place. Automated provisioning: ability to (re)establish baselines. Putting an OS in the VCS is unreasonable but not COTS (Commercial off-the-shelf) its conﬁguration Choosing third-party products: Strive to use the same (or similar) Is it deployable? mechanism to conﬁgure the applications Can it be conﬁgured effectively? and environments organization through How to ﬁt it in the deployment automation? the same mechanism 18 Further reading Book: Chapter 2, "Conﬁguration Management," in Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation, by David Farley, Jez Humble. Addison-Wesley Professional, ISBN: 9780321670250 Website: Semantic Versioning 2.0.0. Available online: https://semver.org/ 19 Advanced Programming (WBCS053-05) Container Technology Daniel Feitosa Solving the baseline problem Need to replicate the baseline with ﬁdelity + Cannot put a whole OS in the VCS + Application conﬁguration is known (software + environment) = Automate provisioning by replicating entirely system installation (incl. OS) from scratch 2 What is a process? An instance of an executing program. Application User space Program Bin / Libs Components syscalls Program code (instructions) Process memory (variables, data) Kernel space Kernel Operating System Interaction with OS Kernel access System calls Accessing resources (CPU, Memory, I/O) Hardware CPU RAM DISK NET 3 May interfere Limitations of traditional deployment Misconﬁguration Application User space Program Library version mismatches Needs to be Bin / Libs replicated Conﬂicting binaries Environment differences syscalls Resource Sharing Issues Kernel space Kernel Operating System Processes competing access for Ports, CPU, Memory Hardware CPU RAM DISK NET 4 Virtualization in a nutshell Application Application Bin/Libs Bin/Libs Hosted Hypervisor Hardware Virtualization Guest OS Hypervisors and Virtual Machines (VMs) Hypervisor Guest OS on Host Hardware Host OS Beneﬁts Hardware Environment consistency Isolation and Security image Shortcomings App App Resource intensive B/L B/L Native Hypervisor Slow startup times Guest OS Complex management Hypervisor Host OS Hardware 5 The cold start problem Save cloud costs by matching supply (resources) with demand (requests) Starting a service duplicate or switching to a more powerful machine takes time Start Load Conﬁgure Run virtualization ﬁles application application 6 Containers to the rescue container Containers Deﬁned Application Program Lightweight, Isolated Environments Bin / Libs Share Host OS Kernel syscalls Advantages Over VMs Container Guest OS Engine / Runtime Faster Startup manage Lower Resource Consumption Kernel Host OS Simpliﬁed Deployment access Must be Linux CPU RAM DISK NET 7 Linux Namespaces i.e., forms of isolation Purpose Types Isolate global system resources Mount Process ID (PID) How they work Network Provide processes with isolated Interprocess Communication (IPC) views UTS (Hostname) Enable multiple instances of global User resources Time cgroup e.g., different processes can use same port 8 Linux Control Groups ( cgroups) Purpose Hierarchical Organization Limit, monitor and prioritize Processes grouped into cgroups resource usage Controllers manage resources per cgroup Resource control CPU shares and quotas Memory limits I/O throughput 9 Building blocks of containers Filesystem isolation chroot and mount Process isolation namespaces container PID namespaces Application Program User isolation Bin / Libs User namespaces syscalls Guest Network isolation OS Union ﬁlesystems network namespaces OverlayFS for layered Kernel images Host OS 10 Security considerations Container Security Challenges Exploits can affect all containers on the host. Shared kernel vulnerabilities Breakout risks Malicious processes may escape the container. Best Practices Reduce the Minimal base images attack surface. Regular updates and patch management Limit system calls Mitigation Strategies available to containers. Capabilities dropping Remove unnecessary Seccomp proﬁles privileges from containers. Mandatory Access Controls (SELinux, AppArmor) Apply security policies to restrict container actions. 11 Container images Layers Immutable, layered ﬁlesystem bundles Collection of FS changes read/write Layer Base OS Layer (Minimal Filesystem) Ephemeral read/write layer Images can be stacked 312b87a231ac conﬁgure environment 273531fa653b copy application ﬁles ibmjava:jre-alpine b125cba542fa dependencies (e.g., apt get install python) 6731baf4361a base image (e.g., ubuntu:22.04) alpine:202208220001 myapp:v1.4 12 Containerization infrastructure Container Engines Image Registries Image management Public Private User Interface (CLI, API) Orchestration tasks Container Runtimes Open Container Initiative (OCI) Standardization efforts Low-level container execution Runtime and image speciﬁcations Interface with OS kernel 13 Building images registry Dockerﬁle ﬁles (incl. code) push FROM — – … —--- ---- specify server pull create build 011 101 laptop image 14 Composing images docker-compose simplest approach set up communication uniﬁed conﬁguration in single ﬁle despite isolation simpliﬁed networking through links easy startup via docker-like commands link suitable for small use cases Limitations of docker-compose single-host only Web API Database Container limited scaling features Engine / Runtime complex networking for scaling lacks high availability and fault tolerance mechanisms, like e.g., Kernel Host OS 15 Containers vs Virtual Machines Containers Virtual Machines Share host kernel Include guest OS's kernel Lightweight and fast Resource intensive Application-centric Hardware-centric Use Cases Containers for microservices, CI/CD VMs for full OS isolation, legacy applications 16 Beneﬁts Challenges Portability Security Consistent environments across development, Shared kernel increases potential attack surface. testing, and production. Networking Efficiency Isolation can complicate network conﬁgurations. Reduced overhead compared to VMs. Persistent storage Scalability Managing data persistence outside of ephemeral Rapid deployment and scaling of applications. containers. Resource utilization Orchestration Optimized use of system resources. Coordinating multiple containers requires additional layers. 17 Further reading Book: Chapter 1, "Introduction to Container Technology," in Podman for DevOps, by Gianni Salinetti and Alessandro Arrichiello. Packt, ISBN: 9781803248233 Workshop: Complete Intro to Containers, by Brian Holt. Available online: https://containers-v2.holt.courses/ 18 Advanced Programming (WBCS053-05) Continuous Software Engineering Daniel Feitosa Development dynamics were not always the same 70s e 80s 90s 2000-present Era Mainframe Client/Server Cloud Technology examples COBOL, DB2 em MVS, C++, Oracle, Solaris, Java, MySQL, Red Hat, etc. etc. Ruby on Rails, PhP, etc. Cycle duration 1–5 years 3–12 months 2–12 weeks Cost $1M–$100M $100K–$10M $10K–$1M Risk for Whole company Product line or division Product feature Cost of failure Bankruptcy, mass Revenue loss, CIO Negligible layoffs layoff "Velocity and Volume (or Speed Wins)", presentation at FlowCon, San Francisco, 2013. 2 A typical (ideal) development cycle u re ns e repository ⚙ compilation ve o pr im commit ru n test 3 Solve collaboration with automation how to ensure good commits? u re ns e repository ⚙ compilation quality checks on repository ve o pr only accept good commits im commit ru n test 4 Solve collaboration with automation client (e.g., server) e change s ur e n repository scale ⚙ compilation deploy ve o pr im feedback commit monitor ru n some icons: Flaticon.com test 5 The development and operation cycle 6 source: Pease, 2017 The problem of delivering software Complexities in modern software systems Frequent changes and updates Need for rapid delivery without compromising quality 7 Common release antipatterns Manual deployments Error-prone and time-consuming Late deployment to production-like environments Issues discovered too late Manual conﬁguration management Inconsistencies across environments 8 Continuous Integration (CI) "automation of the integration of code changes into a single software repository." "ideally, continuous integration will promote good quality by enforcing quality gates" Key Principles Maintain a single source repository Automate the build Make the build self-testing 9 Implementing CI Prerequisites Quality gates Version Control System predeﬁned checkpoints with speciﬁc quality criteria a software must pass before it can proceed to the next stage Automated build process of development pipeline compilation unit/integration tests CI server code style checks bug-ﬁndings tools code smells … JetBrains Jenkins TeamCity CI/CD Actions 10 checks quality gates Implementing quality gates commits change detects change CI/CD notiﬁes runs code compiles? passes tests? proper style? bugs, smells? 11 Key CI practices Optimizing CI Making good use of CI Comprehensive automated test suite Regular check-ins Ensure code quality Prevent integration conﬂicts Short build and test process Don't check in on a broken build Quick feedback Maintain code stability Manage development workspace Run commit tests locally before committing Keep environments consistent Catch issues early Wait for commit tests to pass before moving on Ensure code integrity 12 Challenges and Beneﬁts Beneﬁts Challenges Early detection of errors Maintaining fast builds Saves time and resources Optimize build processes Improved code quality Managing dependencies Through regular testing Use tools like Maven for dependency management Enhanced collaboration Team stays aligned Scaling CI Addressing bottlenecks in large projects 13 Continuous Delivery (CD) "automation of the steps necessary to keep the code base deployable for production." Key Principles Automated conformance checking & quality assurance Simulate production environment as much as possible 14 Deploying a system machine API DBMS Kernel Operating System answer API v1 internet DISK NET CPU RAM requests read/write Some important steps network routing Start machine DB Install OS Install dependencies Install application Conﬁgure port forwarding requests … 15 Deploying a system … more accurately development - laptop smoke test - AWS production - AWS machine API v2 DBMS API v2 API Kernel Operating System quality gates - GitLab acceptance test - AWS volume API v2 DBMS DBMS API v2 16 Deployment process (example) source: Humble and Farley, 17 Continuous Delivery Continuous Delivery vs. Continuous Deployment Code is always ready for deployment Code changes are automatically deployed to production Deployment is a manual decision 18 Deployment pipeline "an automation manifestation of the process for getting software from VCS into the hands of your users." source: Humble and Farley, 19 Continuous Delivery Deployment pipeline practices Build binaries once Smoke test deployments Avoid discrepancies between Quick checks to ensure basic environments functionality Deploy similarly to every environment Plan backing out changes Consistency in deployment process Strategies for rollback 20 Beneﬁts of Continuous Software Engineering Faster time to market Rapid and frequent releases brings deployment ﬂexibility Improved product quality Automated tests catch issues early Reduced risk of deployment failures Empowering teams Collaboration and ownership Predictable and reliable releases 21 DevOps combine software development and IT operations to shorten the development life cycle and provide continuous delivery with high quality 22 Development vs. Operations Continuous integration Disaster recovery planning Feature development Design and architecture Version control management Infrastructure management Bug ﬁxing Maintenance Deployment System monitoring Security management Unit testing 23 Development vs. Operations Version control management Infrastructure management Design and architecture Deployment Feature development System monitoring Bug ﬁxing Maintenance Unit testing How do these Security management tasks intersect Continuous integration and impact Disaster recovery planning each other?...... 24 Conway's law Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations. ? — "Conway's law"; Melvin E. Conway, How Do Committees Invent? What is the implication of Conway's Law for DevOps? 25 Conway's law Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations. DevOps can work very well for Agile development — "Conway's law"; Melvin E. Conway, because some of the necessary culture already How Do Committees Invent? exists. Other factors: Microservices (+modularity) Small teams (-friction) Elevator actors (+communication) 26 DevOps is about culture Development and operations work Development and operations share the together to break silos and promote responsibility for the performance and better communication. reliability of the production software. source: icons by Slamlabs e Kandero, Noun Project 27 DevOps is about culture Continuous feedback between teams and The three previous elements produce an stages of the cycle allow better environment that fosters the for the understanding how changes affect the emergence of the best possible design system as a whole. for the users e teams. source: icons by Darningsih e Ifanicon, Noun Project 28 29 CI/CD Monitoring production deployments Failure detection User interaction insights Identify hardware and software failures, Measure user experience and behavior both total and partial for improvements Performance monitoring Intrusion detection Detect degradations in latency, Identify security breaches and throughput, and resource utilization unauthorized access attempts Capacity planning Support short-term scaling and long-term resource management 30 Structure of monitoring systems collect aggregate generate ﬁlter data store data data data alarms Data Collection Methods Data Challenges Agent-Based Monitoring Time Synchronization Software agents collect data directly from Inconsistent clocks in distributed systems systems complicate data correlation Agentless Monitoring Context Correlation Utilize existing protocols without additional Linking related events across multiple agents components External Health Checks Data Volume Management Monitor system status from an external Handling and storing large amounts of monitoring perspective data efficiently 31 Bottom-up monitoring vs. Top-down monitoring Detailed monitoring of individual Monitoring high-level system components performance metrics Early detection of issues but may Easier to manage but may miss early produce excessive data warning signs Combine them Use both methods for comprehensive monitoring 32 Best monitoring practices and solutions Automate monitoring conﬁguration Integrate monitoring into DevOps culture Integrate monitoring setup into deployment Treat monitoring as a shared responsibility pipelines between development and operations Use scripts and tools to adjust settings Use monitoring data to drive continuous dynamically improvement and informed decision-making 33 Best monitoring practices and solutions Leverage Modern Tools Context-Aware Monitoring Distributed logging systems Incorporate operational context (deployments, feature toggles) into Utilize tools like Logstash and Kafka monitoring data for efficient data collection Enhance event correlation and root Big data analytics cause analysis Apply analytics to extract insights from large datasets 34 Further reading Book: Chapter 3, "Continuous Integration," Chapter 4, "Implementing a Testing Strategy," and Chapter 5, "Anatomy of the Deployment Pipeline," in Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation, by David Farley, Jez Humble. Addison-Wesley Professional, ISBN: 9780321670250 Book: Chapter 1, "What is DevOps?," and Chapter 7, "Monitoring," in DevOps - A Software Architect's Perspective, by Len Bass, Ingo Weber, Liming Zhu. Addison-Wesley Professional, ISBN: 9780134049878 Documentation: GitLab Ci/CD. Available online: https://docs.gitlab.com/ee/ci/ 35 Advanced Programming (WBCS053-05) Technical Debt Daniel Feitosa Poll Have you ever written “poor” code to save time? 2 What is Technical Debt (TD)? A collection of design and implementation decisions that solve problems but make future changes more costly or impossible. Based on explanation coined by Cunningham on "The WyCash Portfolio Management System." OOPSLA92 Experience Report. http://c2.com/doc/oopsla92.html symptoms of refactoring needs A trade-off between the short-term beneﬁts of "cutting corners" in software development and the long-term sustainability of a software system. Z. Li, P. Avgeriou, and P. Liang, “A systematic mapping study on technical debt and management,” JSS, 101(C), 2015 https://doi.org/10.1016/j.jss.2014.12.027 3 88% of developers admit technical debt C. Vassallo, F. Zampetti, D. Romano, M. Beller, A. Panichella, M.D. Penta, A. Zaidman, Continuous delivery practices in a large ﬁnancial organization, in: Proceedings of ICSME ’16, pp. 519–528, 2016. 4 Ok! How much should I care? In 2010 Garter estimated the total amount of technical debt worldwide could reach $1 trillion Airliner loses €€€ due to a storage infrastructure debt that rendered the sales system inoperable Knight Capital’s system lost $465 million in 45 minutes, apparently due to miscopied code 5 How much do others care? 6 Not sure I get it… could you give an example? A team chooses MariaDB/MySQL because they know it quite well. However, it doesn't have all features needed for future features. That is, this isn't a problem now and they need to get the MVP off ground. "We'll ﬁx it later" The new features are delayed, and so is the ﬁx. The DB grows and performance is now also hurt, losing potential proﬁt, but that isn't a huge deal. The performance becomes a bigger issue and a MySQL wizardry is used to speed up transactions a bit. Well, time to implement the new feature (and pay back that debt). Let's migrate the DB to PostgreSQL and be done with it! Just that now you gotta deal with: - a huge migration on a DB that is constantly updated - the MySQL wizardry that does not translate (or does it?) to PostgreSQL What do you do? - invest all the time (and $$$$) in a migration that should had not been expensive in the ﬁrst place and that may fail (due to incompatibilities)? - create a second (PostgreSQL) DB that link keys to ﬁrst DB and supports the new features? (YAY! More debt!) 7 interest: the additional The ﬁnance of Technical Debt development effort required to modify the software (adding new features or ﬁxing bugs) Cost of change principal principal: the effort required to eliminate inefficiencies in the t res current design or e int implementation of a software system time Ar. Ampatzoglou, Ap. Ampatzoglou, A. Chatzigeorgiou, P. Avgeriou The ﬁnancial aspect of managing technical debt: A systematic literature 8 review Information and Software Technology, 64 (Aug. 2015), pp. 52-73 https://doi.org/10.1016/j.infsof.2015.04.001 Technical Debt? vulnerabilities and defects are NOT debt UNLESS they make future changes more costly (e.g., require a workaround that has to be removed after ﬁx) 9 Technical Debt is about the symptoms vulnerabilities and defects are NOT debt BUT the decisions that lead to vulnerability exploitation or can result in defects incur debt. "Ok… I think I get what isn't TD, but what is TD? Is there a catalogue of symptoms?" — not entirely, but there are usual suspects 10 How to ﬁnd symptoms (and debt)? visible invisible visible Architecture (design) Code Architecture Smells Low Internal Quality new features Pattern Violations Code Complexity defects additional functionality Structural Complexity Code Smells low external quality Coding Style Violations Other Development Artifacts Testing and Documentation Issues evolution issues: evolvability quality issues: maintainability P. Kruchten, R. L. Nord and I. Ozkaya, "Technical Debt: From Metaphor to Theory and Practice," in IEEE Software, vol. 29, no. 6, pp. 18-21, 2012. https://doi.org/10.1109/MS.2012.167 11 Types of Technical Debt Architectural TD Design TD Code TD Infrastructure TD architecture decisions shortcuts in detailed code that deviate sub-optimal makin compromises desi n rom best codin conﬁ uration o in internal quality practices or rules dev-related processes, technolo ies, supportin tools, etc. Test TD Build TD Versioning TD shortcuts in testin overly complex build sub-optimal system or process versionin or orkin Requirement TD Documentation TD Defect TD Security TD optimal requirements lackin or outdated lin erin de ects, lin erin vulnerab., vs actual system documentation bu s, or ailures that requirin 'patchin ', implementation impact evolution impactin evolution 12 Z. Li, P. Av eriou, and P. Lian , “A systematic mappin study on technical debt and mana ement,” JSS, vol. 101, no. C, pp. 193–220, 2015 https://doi.or /10.1016/j.jss.2014.12.027 P. Avgeriou et al., "An Overview and Comparison of Technical Debt Measurement Tools," in IEEE Software, vol. 38, no. 3, pp. 61-71, 2021. https://doi.org/10.1109/MS.2020.3024958 Are there tools? Tool Type Principal Other quality attributes CAST* Architectural, design, and code Time to remove issues Security, efficiency, changeability, robustness, and transferability SonarGraph Architectural and design Computation of several metrics Changeability NDepend* Architectural, design, and code Estimated person time to ﬁx issues Changeability, robustness, and testability SonarQube Code Time to remove issues Security and reliability SQuORE Design and code Time to remove issues Changeability, reliability, efficiency, portability, security, and testability CodeMRI* Design Not estimated Security, efficiency, robustness, portability, and testability Code Inspector Architectural, design, and code Effort needed to avoid high TD Security, changeability, portability, testability, and maintainability DV8* Architectural Number of affected ﬁles and lines of code Maintainability, evolvability, and security SymfonyInsight Code Time to remove issues Security, maintainability, and reliability 13 * estimate interest https://www.sonarsource.com/products/sonarqube/downloads/ An example tool - SonarQube 14 Quick look around SonarQube's results 15 Technical Debt timeline rrenc e enes s ion diation o ccu awa r decis reme identiﬁcation latency discussion ﬁxing latency TD can be unintentional knowing what to do is just the beginning 17 Technical Debt management TD as liability TD as asset t poin rrenc e enes s ing cision diation ccu wa r p p de reme o a ti identiﬁcation latency discussion ﬁxing latency getting value out of debt suffering from debt 18 How long does TD linger around? J. Tan, D. Feitosa, P. Avgeriou, "The Lifecycle of Technical Debt that Manifests in both Source Code and Issue Trackers," IST, 159, 2023 19 https://doi.org/10.1016/j.infsof.2023.107216 How long does TD linger around? ~1 year a couple a couple of days of days J. Tan, D. Feitosa, P. Avgeriou, "The Lifecycle of Technical Debt that Manifests in both Source Code and Issue Trackers," IST, 159, 2023 21 https://doi.org/10.1016/j.infsof.2023.107216 How long does TD linger around? Most of the TD items (~87%) identiﬁed and mentioned as resolved in issue trackers are indeed paid back in source code (but not all). J. Tan, D. Feitosa, P. Avgeriou, "The Lifecycle of Technical Debt that Manifests in both Source Code and Issue Trackers," IST, 159, 2023 22 https://doi.org/10.1016/j.infsof.2023.107216 Constraints triangle / Devil's triangle / Project triangle scope/ requirements/ eatures n sio ten quality tension tension time bud et/ effort 24 Forces how to widen the ap? 25 Technical Debt Quadrant deliberate "We don't have "We must ship now time to look into it" and deal with consequences" reckless prudent "What's "Now we know how Postgresql?" we should have used Spring Boot" inadvertent Technical Debt Quadrant by Martin Fowler 26 Technical Debt in the development backlog support quality requirements, e.g.: Infrastructure positive value Common elements Libraries Frameworks Making components reusable Architecture Features features (requirements) (design) visible invisible Defects Technical Debt (e.g. bugs) negative value P. Kruchten, "What colour is your backlog?," 2013. Video: https://youtu.be/XWyBkmvNGrk 27 Slides: https://pkruchten.ﬁles.wordpress.com/2013/12/kruchten-colours-yow-sydney.pdf N. A. Ernst, S. Bellomo, I. Ozkaya, R. L. Nord, and I. Gorton, "Measure It? Manage It? Ignore It? Software Practitioners and Technical Debt", ESEC/FSE ’15, Why is Technical Debt incurred? 2015. https://doi.org/10.1145/2786805.2786848 28 J. Tan, D. Feitosa, P. Avgeriou, "Do practitioners intentionally self-ﬁx Why is Technical Debt incurred? Technical Debt and why?", ICSME ’21, 2021. https://doi.org/10.1109/ICSME52107.2021.00029 29 Poll Who manages Technical Debt? 30 Who pays TD? ~50% of Technical Debt in Python & Java code is self-ﬁxed Multi-faceted understanding - managers - product owner - scrum master - technical leader - developers TD can serve as a common language J. Tan, D. Feitosa, P. Avgeriou, "Do practitioners intentionally self-ﬁx Technical Debt and why?", 31 ICSME ’21, 2021. https://doi.org/10.1109/ICSME52107.2021.00029 J. Tan, D. Feitosa, P. Avgeriou, "The Lifecycle of Technical Debt that Manifests in both Source Code and Issue Trackers," IST, 159, 2023 https://doi.org/10.1016/j.infsof.2023.107216 The survival of TD items 32 Is lack of team oversight a symptom? Impala Camel Thrift Hbase Hadoop PMC members 37 43 21 58 124 Committers 65 85 41 101 242 Developers 149 722 353 2992 22,954 Oversight 0.69 0.18 0.18 0.05 0.02 Fixing ratio 96.3% 92.5% 88.2% 86.4% 77.2% PMC + Committers Lack of enough oversight may Developers impact the quality of TD management J. Tan, D. Feitosa, P. Avgeriou, "The Lifecycle of Technical Debt that Manifests in both Source Code and Issue Trackers," IST, 159, 2023 33 https://doi.org/10.1016/j.infsof.2023.107216 In a nutshell Technical Debt is a metaphor wrapped around suboptimal decisions that lead to short-term beneﬁts and long-term consequences (if not managed). Technical Debt accumulates over time and can originate in all software development activities. Paying back all debt is seldom feasible and desirable. It is all about management. Technical Debt can be used as a common denominator to bridge the communication between managers, designers and developers. 35 Further reading Book: Chapter 2, "What is Technical Debt?," in Managing Technical Debt: Reducing Friction in Software Development by Philippe Kruchten, Robert Nord, Ipek Ozkaya. Addison-Wesley Professional, ISBN: 9780135645932 Article: Z. Li, P. Avgeriou, and P. Liang, “A systematic mapping study on technical debt and management,” JSS, vol. 101, no. C, pp. 193–220, 2015. DOI: 10.1016/j.jss.2014.12.027 Article: P. Kruchten, R. L. Nord and I. Ozkaya, "Technical Debt: From Metaphor to Theory and Practice," in IEEE Software, vol. 29, no. 6, pp. 18-21, 2012. DOI: 10.1109/MS.2012.167 Article: J. Tan, D. Feitosa, P. Avgeriou, "The Lifecycle of Technical Debt that Manifests in both Source Code and Issue Trackers," IST, 159, 2023. DOI: 10.1016/j.infsof.2023.107216 36

AP-Lecture1c-IoC+DI+SpringBeans (2)-merged.pdf

Document Details

Tags

Related

Summary

Full Transcript