DevOps Implementation PDF

DevOps Implementation Yin Hua Li Centennial College Week#1 Topic: DevOps Overview 1 ¡ Definition by Gene Kim § DevOps is architecture practices, technical practices, and cultural norms that allow us to increase our ability to deli...

DevOps Implementation Yin Hua Li Centennial College Week#1 Topic: DevOps Overview 1 ¡ Definition by Gene Kim § DevOps is architecture practices, technical practices, and cultural norms that allow us to increase our ability to deliver applications and services quickly and safely ¡ Wikipedia’s definition § DevOps is a methodology integrating and automating the work of software development (Dev) and information technology operations (Ops). It serves as a means for improving and shortening the system development life cycle and providing continuous delivery with high software quality 2 Source: https://www.timetoast.com/timelines/the-history-of-software-engineering 3 ¡ The inventors’ responsibility is to change the system § Create new feature and functionality in “dev” environment § Occasionally deliver new product to operators with instructions § May incorporate feedback from operators in future deliveries § Rewarded for delivering new features https://americanhistory.si.edu/collections/search/object/nmah_1098824 4 ¡ The mechanics is responsible for keeping the system in operation § Receive new product from developers to be installed and operated § Expected to keep production systems up and running § Track problems, deployment failures, and system outages § May provide feedback to the inventors for future consideration § Penalized for downtime 5 ¡ Nobody can be an expert in everything ¡ As computers became more complex, Dev and Ops became necessarily specialized: § Accelerating pace of technology § Increased demand for new features § Huge amounts of data and number of calculations § More and more specialized tools § Increasingly abstract architectures and design patterns 6 7 ¡ Create feedback loops between inventors and mechanics ¡ Expose real-time metrics from Ops enabling Dev to learn from the system running under real world conditions ¡ Expose real-time metrics from Dev enable Ops to anticipate production needs and provide early input ¡ Cross-functional teams collaborate to deliver whole working systems including all infrastructure, software code and configuration ¡ Ultimately, Feature delivery and system stability become shared goals 8 ¡ DevOps is the outcome of applying the most trusted principles from the domain of physical manufacturing and leadership to the IT value stream ¡ DevOps relies on bodies of knowledge from Lean, the Toyota production system, resilience engineering, safety culture, human factors, etc. ¡ Value stream mapping, Kanban boards, and total productive maintenance were codified for the Toyota Production System in 1980s. In 1997, the Lean enterprise Institute started researching applications of Lean to other value streams, e.g., service industry and healthcare 9 § Just-in-Time: "Making only what is needed, only when it is needed, and only in the amount that is needed" § Jidoka (Autonomation): “Automation with a human touch” Source: https://www.leansixsigmadefinition.com/glossary/toyota-production-system/ 10 ¡ TPS is an integrated socio-technical system developed by Toyota to efficiently organize manufacturing and logistics, including the interaction with suppliers and customers, to minimize the cost and waste ¡ The philosophy is to work intelligently and eliminate waste so that only minimal inventory is needed. This increases cash flow and reduces physical space needs, and makes it easier to deliver the required results smoothly through internal processes one piece at a time to the end customer 11 ¡ Lean was born out of manufacturing practices § It encourages the practice of continuous improvement § It is based on the fundamental idea of respect for people ¡ 5 Lean principles § Define value ▪ Value is what the customer is willing to pay for § Identify & map the value stream ▪ Use customer’s value as a reference point and identify all activities that contribute to these values ▪ Activities that do not add value to the end customer are considered waste § Create flow ▪ After removing the wastes from the value stream, the following action is to ensure that the flow of the remaining steps run smoothly without interruption or delays § Use a pull system ▪ Inventory is considered one of the biggest wastes in any production system. The goal of a pull-based system is to limit inventory and work in process (WIP) items while ensuring that the required materials and information are available for a smooth flow of work § Pursue perfection ▪ It makes Lean thinking and continuous process improvement a part of the organizational culture. Every employee should strive towards perfection while delivering products based on the customer needs ▪ The company should be a learning organization and always find ways to get a little better each and every day 12 ¡ Waste is anything that cause delay § Partially done work ▪ Becomes obsolete and loses value as time progress § Extra process ▪ Any additional work being performed in a process that does not add value to the customer ▪ Add effort and increase lead time § Extra features ▪ Features built into the service that are not needed by the customer ▪ Add complexity and effort to testing and managing functionality § Task switching ▪ When people are assigned to multiple projects and value streams, requiring them to context switch and manage dependencies between work, adding additional effort and time into the value stream § Waiting ▪ Any delays between work requiring resources to wait until they can complete the current work. ▪ Increase cycle time and prevent the customer from getting value § Motion ▪ The amount of effort to move information or materials from one work center to another ▪ Handoffs create motion waste and often require additional communication to resolve ambiguities § Defects ▪ Incorrect, missing, or unclear information, materials, or product create waste ▪ The longer the time between defect creation and defect detection, the more difficult it is to resolve the defect 13 ¡ Technology value stream is the process required to convert a business hypothesis into a technology-enabled service or feature that delivers value to the customer ¡ The value is created only when the services are running in production § Ensure that not only fast flow is delivered, but the deployment can also be performed without causing chaos and disruptions § Lead time starts when the request is made and ends when it is fulfilled; while process time starts when we begin work the customer request, omit the waiting to be processed Source: The DevOps Handbook --Figure 1.1 Lead time vs Process Time of a Deployment Operation 14 ¡ Dev help system stability § Respond to bugs and deployment failures quickly § Trust Ops will put code in production quickly ¡ Ops help accelerate feature delivery § Anticipate how new functionality will effect production § Trust Dev will provide good code ¡ Dev and Ops work together to remove causes of bugs and failures 15 ¡ Dev can incorporate needs of the business and customers into development ¡ Ops understands better how to support business goals ¡ Business better understands capability for changes to features and functionalities, and operational capabilities 16 ¡ Modern business § Depend on IT deploying new features § Need fast time-to-value in ever changing world (i.e., new technologies, new business requirements, customers’ demand, etc. ) § Expectations for delivery time continue to decrease § Expect to maintain very low mean-time to repair (MTTR) 17 ¡ The report represents 10 years’ research and data from more than 39,000 professionals worldwide ¡ DORA’s (https://dora.dev/research/2024/dora-report/) 4 key metrics have become the industry standard for measuring software delivery performance 18 ¡ The first way enables fast left-to-right of work from development to operations to the customer ¡ The second way enables the fast and constant flow of feedback from right to left at all stages of the value stream ¡ The third way enables the creation of a generative, high-trust culture that supports a dynamic, disciplined, and scientific approach to experimentation and risk-taking, facilitating the creation of organizational learning, both from success and failure 19 ¡ Make work visible § A significant difference between technology and manufacturing value stream is that technology work is invisible § Use visual work boards (e.g., Kanban boards) to represent work on physical or electronic cards § Work originates on the left, is pulled from work center to work center, and finishes when it reaches the right side of the board 20 ¡ Limit work in Process (WIP) § Studies have shown that the time to complete even simple tasks significantly degrades when multitasking § Use Kanban board to limit multitasking to enforce WIP limits for each column or work center 21 ¡ Reduce batch sizes § Small batch sizes result in less WIP, faster lead times, faster detection of errors, and less rework 22 ¡ Reduce the number of handoffs § Each time the work passes from team to team, all sorts of communication (e.g., requesting, specifying, signaling, coordinating, prioritizing, scheduling, deconflicting, testing, verifying, etc.) are required. The communication may require using different ticketing or project management systems. § Each of these steps is a potential queue where work will wait; furthermore, each handoff may lost some knowledge 23 ¡ Continually identify and elevate constraints ¡ Eliminate hardships and waste in the value stream 24 ¡ Feedback enables to detect and remediate problems when they are smaller, cheaper, and easier to fix; to avert problems before cause catastrophe; and to create organizational learning that can be integrated into future work ¡ Characteristics of a complex system § Defies any single person’s ability to see the system as a whole and understand how the pieces fit together § Doing the same thing twice will not predictably or necessarily lead to the same result 25 ¡ See problems as they occur § Create feedback and feedforward loop into the system § Feedback loops not only enable quick detection and recovery of problems but also inform us on how to prevent these problems from occurring again in the future § 6 types of feedback in software development ▪ Dev tests: did I write the code I intended to write? ▪ CI and testing: did I write the code I intended to write without violating any existing expectations in the code? ▪ Exploratory testing: did we introduce any unintended consequences? ▪ Acceptance testing: Did I get the feature I asked for? ▪ Stakeholder Feedback: As a team, are we headed in the right direction? ▪ User Feedback: Are we producing something our customers/users love? 26 ¡ Each type of feedback takes different amount of time 27 ¡ Swarm and solve problems to build new knowledge § The goal of swarming is to contain problems before they have a chance to spread, and to diagnose and treat the problem so that it cannot recur § A swarm is where you get the whole team together to focus on solving a single problem ▪ Prevent the problem from progressing downstream, where the cost and effort to repair it increases exponentially and technical debt is to accumulate ▪ Prevent the work center from starting new work, which will likely introduce new errors into the system ▪ Prevent the loss of critical information due to fading memories or changing circumstances ▪ If the problem is not addressed, the work center could potentially have the same problem in the next operation 28 ¡ Keep pushing quality closer to the source § Everyone in value stream needs to find and fix problems in his/her area of control as part of his/her daily work; in other words, quality and safety responsibilities and decision-making are pushed to where the work is performed rather than relying on approvals from distant executives § Having developers share responsibility for the quality of systems they build not only improves outcomes but also accelerates learning 29 ¡ Enable optimizing for downstream work centers § Lean defines two types of customers ▪ External customer: who pays for the service we are delivering ▪ Internal customer: who receives and processes the work immediately after us § According to Lean, our most important customer is our next step downstream § In the technology value stream, we optimize for downstream work centers by designing for operations, where operational non-functional requirements are prioritized as highly as user features. 30 ¡ Enabling organizational learning and a safety culture § Dr. Westrum defines three types of culture ▪ Pathological organizations ▪ Characterized by large amounts of fear and threat. People often hoard information, withhold it for political reasons, or distort it to make themselves look better. Failure is often hidden ▪ Bureaucratic organizations ▪ Characterized by rules and processes, often to help individual departments maintain their “turf”. Failure is processed through a system of judgement, resulting in either punishment or justice or mercy ▪ Generative organizations ▪ Characterized by actively seeking and sharing information to better enable the organization to achieve its mission. Responsibilities are shared throughout the value stream, and failure results in reflection and genuine inquiry 31 ¡ In the technology value streams, we should establish the foundation of a generative culture by striving to create a safe system of work 32 ¡ Institutionalize the improvement of daily work § Improve daily work by explicitly reserving time to pay down technical debt, fix defects, refactor and improve problematic areas of our code and environments § The result is that everyone finds and fixes problems in their area of control all the time, as part of their daily work § We can eradicate the less obvious problems from the system when it is not only easier and cheaper but also when the consequences are smaller 33 ¡ Transform local discoveries into global improvements § When new learning are discovered locally, some mechanism is needed to enable the rest of organization to use and benefit from that knowledge. When anyone else does similar work, they do the work with the cumulative and collective experience of everyone in the organization who has ever done the same work § In the technology value stream, it could be achieved by creating shared source code repositories that span the whole organization, where shared code, libraries, and configurations that embody the best collective knowledge of the entire organization can be easily used. 34 ¡ Inject resilience patterns into our daily work § Consider a typical experiment in manufactory plant, suppose there are 2 production lines, each capable of producing 100 units per day. On slow days, they would send all production onto 1 line, experimenting with ways to increase capacity and identify vulnerabilities in their process, knowing that if overloading the line caused it to fail, they could send all production to the second line. By relentless and constant experimentation in their daily work, they were able to continually increase capacity, often without adding any new equipment or hiring more people § In the technology value stream, the same type of tension can be injected into the system by seeking to always reduce deployment lead times, increase test coverage, decrease test execution time, and even by re-architecturing if necessary to increase developer productivity or increase reliability 35 ¡ Leaders reinforce a learning culture § Leaders and frontline works are in complementary working relationship which is necessary because they cannot solve problems alone ▪ Leaders are not close enough to the work, which is required to solve any problem ▪ Frontline workers do not have the broader organizational context or the authority to make change outside of their area of work 36 Source: https://www.guru99.com/agile-vs-devops.html 37 Comparison Agile DevOps Inception 2001 2007 What is it? Agile refers to an iterative approach which DevOps is considered a practice of bringing focus on collaboration, customer feedback, development and operations team together and small, rapid release Task Focuses on constant changes Focuses on constant testing and delivery Feedback Provided by the customer Comes from the internal team Alternatives Waterfall Silo based development and deployment Emphasis Emphasizes on software development DevOps is all about taking software which is methodology for developing software. After ready for release and deploying it in a reliable the software has been released, the agile and secure manner team will not care what happens to it Team size Team of 10 or less Relatively large team as it involves all stakeholders Focus Focuses on functional and non-functional Focus more on operational and business readiness readiness 38 Source: https://www.guru99.com/devops-tutorial.html 39 1. https://www.splunk.com/en_us/blog/learn/state-of-devops.html 2. https://www.youtube.com/watch?v=_DaEoWVpO0s 3. https://theleanway.net/The-Five-Principles-of-Lean 4. https://tobeagile.com/seven-strategies-for-team-swarming/ 40