Introduction to Agile and DevOps

DevOps Implementation Overview

• DevOps is a combination of architectural, technical, and cultural practices.
• It aims to improve the speed and safety of delivering applications and services.
• Wikipedia defines DevOps as a methodology that integrates and automates the work of software development (Dev) and information technology operations (Ops).
• This integration enhances the system development lifecycle, facilitating continuous delivery with high software quality.

Software Engineering Timeline

• Key milestones in software engineering are noted.
• Examples are the first programming language, creation of the Unix operating system, release of Javascript, milestones related to development of the internet, and the introduction of modern software.

Traditional Development

• Traditional development puts the responsibility on inventors to change the system.
• Inventors create new features and functionality.
• Development is done in an environment with occasional product delivery to operators.
• Operators receive products with instructions for use.
• Operators may provide feedback about the product features.
• Inventors are rewarded for delivering new features.

Traditional Operations

• The mechanics (operations team) are responsible for keeping the system running.
• Operations teams receive new products, and install and operate.
• They keep the production systems running.
• Operations monitor problems, deployment failures and outages.
• Operations teams provide feedback for future improvement considerations to the inventors.
• Downtime is penalized.

History of Separation of Dev and Ops

• Dev and Ops become specialized as computers become more complex.
• This specialization is related to the rising pace of technology and the demand for new features; with a rise in data amounts and calculations.
• Increasingly specialized tools and abstract architectures also contributed to this.

Conflicts Caused by Different Goals

• Developers (Dev) want to push more change, while Operations (Ops) resist change.
• This difference in goals can cause conflicts.

Dev & Ops Work Together

• DevOps creates feedback loops between developers and operations.
• Operations share real-time metrics, helping developers learn from real-world conditions.
• Developers provide real-time metrics, helping operations anticipate production needs and provide crucial input.
• Teams collaborate to deliver complete working systems encompassing infrastructure, software codes, and configurations.
• Feature delivery and system stability become shared goals.

DevOps: Convergence of Dev and Ops

• DevOps is achieved by applying principles from physical manufacturing (primarily from the Toyota Production System)
• Knowledge from Lean, resilience engineering, and safety culture are applicable.
• Codified practices such as value stream mapping, Kanban boards, and total productive maintenance are essential for DevOps.

Toyota Production System (TPS)

• Just-in-Time: Making only what is needed, only when needed, and only in the amount that is needed.
• Jidoka (Autonomation): Automation with a human touch.

Toyota Production System (Con't)

• TPS is a socio-technical system to efficiently organize manufacturing and logistics including those working with suppliers and customers.
• Waste is minimized, minimizing cost and increasing efficiency.
• Minimal inventory maximizes cash flow.
• Internal processes are designed to deliver products to the final customer seamlessly.

Lean

• Lean principles focus on continuous improvement and respect for people.
• Defines value as what the customer is willing to pay for.
• Maps the value stream to identify activities that add or do not add value to the customer; removes waste.
• Focuses on creating a smooth flow of work without any interruption.
• Uses the pull system to limit inventory and work-in-process.
• Perfection is part of the culture.

Wastes in Software Development

• Waste is defined as anything that causes delay in development.
• Examples of waste include partially done work, obsolete work, extra steps, features not needed by customers, complicated tasks that require changes (or task switching), delays in operations, motion, handoffs, errors, and long time to detecting errors; all these can create delays.

The Technology Value Stream

• The technology value stream converts a business hypothesis into a technology-enabled service or feature.
• Value is created only when the service operates in production.
• Lead time begins when the request is received and ends when the request is fulfilled.
• Process time starts when the request is worked on, and excludes the waiting time.

Break the Silos: Communication, Collaboration, Integration

• Developers and operations work together to improve system stability.
• They respond to bugs and deployment issues quickly.
• Developers and operations trust each other to produce good code and deploy it reliably.
• Eliminating causes of bugs and failures is a joint effort.

Break Silos: Dev, Ops & Business

• Developers incorporate business and customer needs into development.
• Operations support business goals better by understanding how to support business goals.
• Businesses improve their capacity for handling changes to features, functionality, and operational capabilities.

Business Demand: Continuously Deliver Valuable Software

• Modern businesses rely on IT to deploy new features frequently.
• Fast time to value is critical in a constantly changing world.
• Improved technologies and shifting customer preferences create new business requirements.
• Business expectations for systems delivery need extremely low mean time to repair (MTTR).

Accelerate State of DevOps 2024 Report

• The report gathers data from more than 39,000 IT professionals to analyze their performance.

The Three Ways of DevOps By Gene Kim

• The first way fosters fast, efficient left-to-right workflow for efficient work from development through operations teams.
• The second way promotes constant flow of feedback from right and left at all stages.
• The third way helps build and maintain a high trust culture.

The Principle of Flow/1

• Works are visible in workflows and visualized with tools like Kanban boards.
• Visualizing work helps in understanding where the work is in the workflow.
• Work is pulled through workflows to finish.

The Principle of Flow/2

• Limit work in progress (WIP) to streamline work, reducing multitasking.
• Work limits reduce delays and errors.

The Principle of Flow/3

• Reducing batch sizes optimizes workflow with small batches to reduce work in progress(WIP).
• Identifying errors early reduces rework in production.

The Principle of Flow/4

• Reduce the number of handoffs between teams to reduce delays and keep the workflow running.
• Reduce errors by preventing loss of knowledge at each handoff.

The Principle of Flow/5

• Continually finding and addressing constraints and waste helps to continually improve the workflow and remove waste.

The Principle of Feedback/1

• Feedback is essential for effectively detecting and addressing issues early in workflows.
• Fixing minor problems before they become significant reduces overall workload and effort.
• Learning from problems can impact future development workflows.

The Principle of Feedback/2

• Feedback loops in development are essential to identify problems promptly, and improve development processes.
• Example feedback types include Dev tests, CI and testing, exploratory testing, acceptance testing, stakeholder feedback, user feedback, etc.

The Principle of Feedback/3

• Different types of feedback take varying amount of time.
• Customer opinion feedback takes the longest amount of time.

The Principle of Feedback/4

• Swarming involves teams focusing on solving a single problem collaboratively.
• This reduces the time to fix the issue compared with solving multiple problems at once.
• Fixing problems early prevents accumulating technical debt affecting future development and operations.

The Principle of Feedback/5

• Keep pushing quality closer to where the work is being performed. - Developers and operators are involved in maintaining quality.
• Quality issues are handled closer to where the work takes place.

The Principle of Feedback/6

• Optimize systems and features for later steps in the workflow.
• This enables downstream work centers with improvements in operational and functional requirements and features which support users.

The Principles of Continual Learning and Experimentation/1

• Enabling organizational learning and safety culture is key for continual learning.
• Pathological organizations: characterized by large amounts of fear and threat.
• Bureaucratic organizations: characterized by rules and processes.
• Generative organizations: characterized by actively seeking and sharing information.

The Principles of Continual Learning and Experimentation/2

• Organizations with different cultures handle information in different ways.
• Pathological organizations hide information or even punish those who share problems, while generative organizations support sharing information to resolve problems.

The Principles of Continual Learning and Experimentation/3

• Institutionalizing the improvement of daily work is essential.
• Reserve time for paying down technical debt.
• fixing defects, refactoring, and improving problematic areas frequently to avoid larger, more complex situations later.
• Fixing problems regularly keeps everything on time with smaller, easily manageable issues.

The Principles of Continual Learning and Experimentation/4

• Transforming local discoveries into global improvements ensures efficiency.
• Encouraging and enabling knowledge sharing within the organization ensures the utilization of combined knowledge and experience of everyone from previous iterations.

The Principles of Continual Learning and Experimentation/5

• Injecting resilience patterns into daily work helps deal with uncertainties.
• Experiments are common across the daily work, to ensure that production lines can deal unexpectedly higher than average workload.

The Principles of Continual Learning and Experimentation/6

• Organizations need leadership to reinforce and encourage learning and experimentation culture.
• Leaders need to be closely related to work to solve problems.
• Operational staff needs the broader organizational context and authority to address issues outside their areas of responsibility.

DevOps vs Agile

• Agile focuses on communication between customers and developers, while DevOps connects developers with IT operations.
• Agile promotes iterative development with small releases; DevOps focuses on constant testing and delivery.
• Feedback in Agile comes from the customer, while DevOps feedback comes from operations.

DevOps Lifecycle

• The DevOps lifecycle is a continuous loop representing different stages of the software development process.
• Includes planning, building, testing, releasing, deploying, operating and monitoring.