Agile Software Engineering PDF

Agile Software Engineering © Ian Sommerville 2018  Agile software engineering Software products must be brought to market quickly so rapid software development and delivery is essential. Virtually all software products are now developed using...

Agile Software Engineering © Ian Sommerville 2018  Agile software engineering Software products must be brought to market quickly so rapid software development and delivery is essential. Virtually all software products are now developed using an agile approach. Agile software engineering focuses on delivering functionality quickly, responding to changing product specifications and minimizing development overheads. A large number of ‘agile methods’ have been developed. There is no ‘best’ agile method or technique. It depends on who is using the technique, the development team and the type of product being developed Agile Software Engineering © Ian Sommerville 2018: 2  Agile methods Plan-driven development evolved to support the engineering of large, long-lifetime systems (such as aircraft control systems) where teams may be geographically dispersed and work on the software for several years. This approach is based on controlled and rigorous software development processes that include detailed project planning, requirements specification and analysis and system modelling. However, plan-driven development involves significant overheads and documentation and it does not support the rapid development and delivery of software. Agile methods were developed in the 1990s to address this problem. These methods focus on the software rather than its documentation, develop software in a series of increments and aim to reduce process bureaucracy as much as possible. Agile Software Engineering © Ian Sommerville 2018: 3  Table 2.1 The agile manifesto We are uncovering better ways of developing software by doing it and helping others to do it. Through this work, we have come to value: - individuals and interactions over processes and tools; - working software over comprehensive documentation; - customer collaboration over contract negotiation; - responding to change over following a plan. While there is value on the items on the right, we value the items on the left more. Agile Software Engineering © Ian Sommerville 2018: 4  Incremental development All agile methods are based around incremental development and delivery. Product development focuses on the software features, where a feature does something for the software user. With incremental development, you start by prioritizing the features so that the most important features are implemented first. You only define the details of the feature being implemented in an increment. That feature is then implemented and delivered. Users or surrogate users can try it out and provide feedback to the development team. You then go on to define and implement the next feature of the system. Agile Software Engineering © Ian Sommerville 2018: 5  Figure 2.1 Incremental development Figure 2.1 Incremental development Product feature list Choose features to be included in If all features are increment complete, deliver system release Refine feature Deliver system increment descriptions Integrate feature Implement and into system test feature Agile Software Engineering © Ian Sommerville 2018: 6  Table 2.2 Incremental development activities Choose features to be included in an increment Using the list of features in the planned product, select those features that can be implemented in the next product increment. Refine feature descriptions Add detail to the feature descriptions so that the team have a common understanding of each feature and there is sufficient detail to begin implementation. Implement and test Implement the feature and develop automated tests for that feature that show that its behaviour is consistent with its description. Integrate feature and test Integrate the developed feature with the existing system and test it to check that it works in conjunction with other features. Deliver system increment Deliver the system increment to the customer or product manager for checking and comments. If enough features have been implemented, release a version of the system for customer use. Agile Software Engineering © Ian Sommerville 2018: 7  Table 2.3 Agile development principles Involve the customer Involve customers closely with the software development team. Their role is to provide and prioritize new system requirements and to evaluate each increment of the system. Embrace change Expect the features of the product and the details of these features to change as the development team and the product manager learn more about it. Adapt the software to cope with changes as they are made. Develop and deliver incrementally Always develop software products in increments. Test and evaluate each increment as it is developed and feed back required changes to the development team. Agile Software Engineering © Ian Sommerville 2018: 8  Table 2.3 Agile development principles Maintain simplicity Focus on simplicity in both the software being developed and in the development process. Wherever possible, do what you can to eliminate complexity from the system. Focus on people, not things Trust the development team and do not expect everyone to always do the development process in the same way. Team members should be left to develop their own ways of working without being limited by prescriptive software processes. Agile Software Engineering © Ian Sommerville 2018: 9  Extreme programming The most influential work that has changed software development culture was the development of Extreme Programming (XP). The name was coined by Kent Beck in 1998 because the approach was developed by pushing recognized good practice, such as iterative development, to ‘extreme’ levels. Extreme programming focused on 12 new development techniques that were geared to rapid, incremental software development, change and delivery. Some of these techniques are now widely used; others have been less popular. The most widely used XP techniques (highlighted in red on the following slide) are explained elsewhere in the book. Agile Software Engineering © Ian Sommerville 2018: 10  Figure 2.2 Extreme programming practices Figure 2.2 Extreme programming practices Test-first development Continuous integration Refactoring Incremental Small planning releases Extreme programming Collective Simple ownership design Pair On-site programming customer Sustainable pace Agile Software Engineering © Ian Sommerville 2018: 11  Table 2.4 Widely adopted XP practices Incremental planning/user stories There is no ‘grand plan’ for the system. Instead, what needs to be implemented (the requirements) in each increment are established in discussions with a customer representative. The requirements are written as user stories. The stories to be included in a release are determined by the time available and their relative priority. Small releases The minimal useful set of functionality that provides business value is developed first. Releases of the system are frequent and incrementally add functionality to the previous release. Test-driven development Instead of writing code then tests for that code, developers write the tests first. This helps clarify what the code should actually do and that there is always a ‘tested’ version of the code available. An automated unit test framework is used to run the tests after every change. New code should not ‘break’ code that has already been implemented. Agile Software Engineering © Ian Sommerville 2018: 12  Table 2.4 Widely adopted XP practices Continuous integration As soon as the work on a task is complete, it is integrated into the whole system and a new version of the system is created. All unit tests from all developers are run automatically and must be successful before the new version of the system is accepted. Refactoring Refactoring means improving the structure, readability, efficiency and security of a program. All developers are expected to refactor the code as soon as potential code improvements are found. This keeps the code simple and maintainable. Agile Software Engineering © Ian Sommerville 2018: 13  Scrum Software company managers need information that will help them understand how much it costs to develop a software product, how long it will take and when the product can be brought to market. Plan-driven development provides this information through long-term development plans that identify deliverables - items the team will deliver and when these will be delivered. Plans always change so anything apart from short-term plans are unreliable. Scrum is an agile method that provides a framework for agile project organization and planning. It does not mandate any specific technical practices. Agile Software Engineering © Ian Sommerville 2018: 14  Table 2.5 Scrum terminology Product The software product that is being developed by the Scrum team. Product owner A team member who is responsible for identifying product features and attributes. They review work done and help to test the product. Product backlog A to-do list of items such as bugs, features and product improvements that the Scrum team have not yet completed. Development team A small self-organising team of five to eight people who are responsible for developing the product. Sprint A short period, typically two to four weeks, when a product increment is developed. Agile Software Engineering © Ian Sommerville 2018: 15  Table 2.5 Scrum terminology Scrum A daily team meeting where progress is reviewed and work to be done that day as discussed and agreed. ScrumMaster A team coach who guides the team in the effective use of Scrum. Potentially shippable product increment The output of a sprint which should be of high enough quality to be deployed for customer use. Velocity An estimate of how much work a team can do in a single sprint. Agile Software Engineering © Ian Sommerville 2018: 16  Key roles in Scrum The Product Owner is responsible for ensuring that the development team are always focused on the product they are building rather than diverted into technically interesting but less relevant work. In product development, the product manager should normally take on the Product Owner role. The ScrumMaster is a Scrum expert whose job is to guide the team in the effective use of the Scrum method. The developers of Scrum emphasize that the ScrumMaster is not a conventional project manager but is a coach for the team. They have authority within the team on how Scrum is used. In many companies that use Scrum, the ScrumMaster also has some project management responsibilities. Agile Software Engineering © Ian Sommerville 2018: 17  Scrum and sprints In Scrum, software is developed in sprints, which are fixed-length periods (2 - 4 weeks) in which software features are developed and delivered. During a sprint, the team has daily meetings (Scrums) to review progress and to update the list of work items that are incomplete. Sprints should produce a ‘shippable product increment’. This means that the developed software should be complete and ready to deploy. Agile Software Engineering © Ian Sommerville 2018: 18  Figure 2.3. Scrum cycles Figure 2.3 Scrum cycles Start Review product backlog Select items to implement Review Product sprint backlog Plan sprint Shippable Sprint product increment Test Sprint software Scrum backlog Develop software Agile Software Engineering © Ian Sommerville 2018: 19  Figure 2.4 The top five benefits of using Scrum Figure 2.4 The top-five benefits of using Scrum The product is broken Unstable requirements down into a set of do not hold up Product understandable chunks progress. that stakeholders can relate to. Scrum benefits Progress People Customers see on-time Team communication is Trust between delivery of increments improved because customers and and gain feedback on everyone can see developers is how the product works. everything. established and a positive culture is created. Agile Software Engineering © Ian Sommerville 2018: 20  Key Scrum practices Product backlog This is a to-do list of items to be implemented that is reviewed and updated before each sprint. Timeboxed sprints Fixed-time (2-4 week) periods in which items from the product backlog are implemented, Self-organizing teams Self-organizing teams make their own decisions and work by discussing issues and making decisions by consensus. Agile Software Engineering © Ian Sommerville 2018: 21  Product backlogs The product backlog is a list of what needs to be done to complete the development of the product. The items on this list are called product backlog items (PBIs). The product backlog may include a variety of different items such as product features to be implemented, user requests, essential development activities and desirable engineering improvements. The product backlog should always be prioritized so that the items that be implemented first are at the top of the list. Agile Software Engineering © Ian Sommerville 2018: 22  Table 2.6 Examples of product backlog items 1. As a teacher, I want to be able to configure the group of tools that are available to individual classes. (feature) 2. As a parent, I want to be able to view my childrens’ work and the assessments made by their teachers. (feature) 3. As a teacher of young children, I want a pictorial interface for children with limited reading ability. (user request) 4. Establish criteria for the assessment of open source software that might be used as a basis for parts of this system. (development activity) 5. Refactor user interface code to improve understandability and performance. (engineering improvement) 6. Implement encryption for all personal user data. (engineering improvement) Agile Software Engineering © Ian Sommerville 2018: 23  Table 2.7 Product backlog item states Ready for consideration These are high-level ideas and feature descriptions that will be considered for inclusion in the product. They are tentative so may radically change or may not be included in the final product. Ready for refinement The team has agreed that this is an important item that should be implemented as part of the current development. There is a reasonably clear definition of what is required. However, work is needed to understand and refine the item. Ready for implementation The PBI has enough detail for the team to estimate the effort involved and to implement the item. Dependencies on other items have been identified. Agile Software Engineering © Ian Sommerville 2018: 24  Figure 2.5 Product Figure backlogbacklog 2.5 Product activities activities REVISED PRODUCT BACKLOG PRODUCT BACKLOG PBI 1.1 PBI 1 Refinement PBI 1.2 PBI 2 PBI 2E Estimation PBI 3 PBI 3E PBI 4 Prioritization Creation PBI 4 PBI 5 PBI 5 Creation PBI 6 Agile Software Engineering © Ian Sommerville 2018: 25  Product backlog activities Refinement Existing PBIs are analysed and refined to create more detailed PBIs. This may lead to the creation of new product backlog items. Estimation The team estimate the amount of work required to implement a PBI and add this assessment to each analysed PBI. Creation New items are added to the backlog. These may be new features suggested by the product manager, required feature changes, engineering improvements, or process activities such as the assessment of development tools that might be used. Prioritization The product backlog items are reordered to take new information and changed circumstances into account. Agile Software Engineering © Ian Sommerville 2018: 26  PBI estimation metrics Effort required This may be expressed in person-hours or person-days i.e. the number of hours or days it would take one person to implement that PBI. This is not the same as calendar time. Several people may work on an item, which may shorten the calendar time required. Story points Story points are an arbitrary estimate of the effort involved in implementing a PBI, taking into account the size of the task, its complexity, the technology that may be required and the ‘unknown’ characteristics of the work. They were derived originally by comparing user stories, but they can be used for estimating any kind of PBI. Story points are estimated relatively. The team agree on the story points for a baseline task and other tasks are estimated by comparison with this e.g. more/ less complex, larger/smaller etc. Agile Software Engineering © Ian Sommerville 2018: 27  Timeboxed sprints Products are developed in a series of sprints, each of which delivers an increment of the product or supporting software. Sprints are short duration activities (1-4 weeks) and take place between a defined start and end date. Sprints are timeboxed, which means that development stops at the end of a sprint whether or not the work has been completed. During a sprint, the team work on the items from the product backlog. Agile Software Engineering © Ian Sommerville 2018: 28  Figure Figure 2.6 Benefits 2.6 Benefits of usingof using timeboxed timeboxed sprints sprints There is a tangible output (usually a software demonstrator) that can be delivered at the end of every sprint. Demonstrable progress Time- boxing benefits Problem discovery Work planning If errors and omissions are The team develops an understand- discovered the rework required is ing of how much work they can do limited to the duration of a sprint. in a fixed time period. Agile Software Engineering © Ian Sommerville 2018: 29  Sprint activities Sprint planning Work items to be completed in that sprint are selected and, if necessary, refined to create a sprint backlog. This should not last more than a day at the beginning of the sprint. Sprint execution The team work to implement the sprint backlog items that have been chosen for that sprint. If it is impossible to complete all of the sprint backlog items, the sprint is not extended. The unfinished items are returned to the product backlog and queued for a future sprint. Sprint reviewing The work done in the sprint is reviewed by the team and (possibly) external stakeholders. The team reflect on what went well and what went wrong during the sprint with a view to improving their work process. Agile Software Engineering © Ian Sommerville 2018: 30  Figure 2.7 Sprint activities Figure 2.7 Sprint activities Sprint Sprint Sprint planning backlog review Sprint execution Sprint Integrate Scrum backlog Develop software Agile Software Engineering © Ian Sommerville 2018: 31  Sprint planning Establish an agreed sprint goal Sprint goals may be focused on software functionality, support or performance and reliability,. Decide on the list of items from the product backlog that should be implemented Create a sprint backlog. This is a more detailed version of the product backlog that records the work to be done during the sprint Agile Software Engineering © Ian Sommerville 2018: 32  Figure 2.8 Sprint goals Figure 2.8 Sprint goals Implement user roles so that a user can select their role when they login to the system Functional Sprint goals Support Performance and reliability Develop analytics that maintain Ensure that the login response time is less information about the time users than 10 seconds for all users where there are spend using each feature of the up to 2000 simultaneous login connections. system. Agile Software Engineering © Ian Sommerville 2018: 33  Sprint planning In a sprint plan, the team decides which items in the product backlog should be implemented during that sprint. Key inputs are the effort estimates associated with PBIs and the team’s velocity The output of the sprint planning process is a sprint backlog. The sprint backlog is a breakdown of PBIs to show the what is involved in implementing the PBIs chosen for that sprint. During a sprint, the team have daily meetings (scrums) to coordinate their work. Agile Software Engineering © Ian Sommerville 2018: 34  Table 2.8 Scrums A scrum is a short, daily meeting that is usually held at the beginning of the day. During a scrum, all team members share information, describe their progress since the previous day’s scrum, problems that have arisen and plans for the coming day. This means that everyone on the team knows what is going on and, if problems arise, can re-plan short-term work to cope with them. Scrum meetings should be short and focused. To dissuade team members from getting involved in long discussions, they are sometimes organized as ‘stand-up’ meetings where there are no chairs in the meeting room. During a scrum, the sprint backlog is reviewed. Completed items are removed from it. New items may be added to the backlog as new information emerges. The team then decide who should work on sprint backlog items that day. Agile Software Engineering © Ian Sommerville 2018: 35  Agile activities Scrum does not suggest the technical agile activities that should be used. However, I think there are two practices that should always be used in a sprint. Test automation As far as possible, product testing should be automated. You should develop a suite of executable tests that can be run at any time. Continuous integration Whenever anyone makes changes to the software components they are developing, these components should be immediately integrated with other components to create a system. This system should then be tested to check for unanticipated component interaction problems. Agile Software Engineering © Ian Sommerville 2018: 36  Table 2.9 Code completeness checklist Reviewed The code has been reviewed by another team member who has checked that it meets agreed coding standards, is understandable, includes appropriate comments, and has been refactored if necessary. Unit tested All unit tests have been run automatically and all tests have executed successfully. Integrated The code has been integrated with the project codebase and no integration errors have been reported. Integration tested All integration tests have been run automatically and all tests have executed successfully. Accepted Acceptance tests have been run if appropriate and the product owner or the development team have confirmed that the product backlog item has been completed. Agile Software Engineering © Ian Sommerville 2018: 37  Sprint reviews At the end of each sprint, there is a review meeting, which involves the whole team. This meeting: reviews whether or not the sprint has met its goal. sets out any new problems and issues that have emerged during the sprint. is a way for a team to reflect on how they can improve the way they work. The product owner has the ultimate authority to decide whether or not the goal of the print has been achieved. They should confirm that the implementation of the selected product backlog items is complete. The sprint review should include a process review, in which the team reflects on its own way of working and how Scrum has been used. The aim is to identify ways to improve and to discuss how to use Scrum more productively. Agile Software Engineering © Ian Sommerville 2018: 38  Figure 2.9 Self-organizing teams Figure 2.9 Self-organizing teams Self-organizing team coordinates the work limits the involvement of makes its own decisions of the team members engineers in external on schedule and by discussing tasks and interactions with deliverables. reaching a consensus on management and who should do what. customers. Agile Software Engineering © Ian Sommerville 2018: 39  Team size and composition The ideal Scrum team size is between 5 and 8 people. Teams have to tackle diverse tasks and so usually require people with different skills, such as networking, user experience, database design and so on. They usually involve people with different levels of experience. A team of 5-8 people is large enough to be diverse yet small enough to communicate informally and effectively and to agree on the priorities of the team. The advantage of a self-organizing team is that it can be a cohesive team that can adapt to change. Because the team rather than individuals take responsibility for the work, they can cope with people leaving and joining the team. Good team communication means that team members inevitably learn something about each other’s areas Agile Software Engineering © Ian Sommerville 2018: 40  Team coordination The developers of Scrum assumed that teams would be co-located. They would work in the same room and could communicate informally. Daily scrums mean that the team members know what’s been done and what others are doing. However, the use of daily scrums as a coordination mechanism is based on two assumptions that are not always correct: Scrum assumes that the team will be made up of full-time workers who share a workspace. In reality, team members may be part-time and may work in different places. For a student project team, the team members may take different classes at different times. Scrum assumes that all team members can attend a morning meeting to coordinate the work for the day. However, some team members may work flexible hours (e.g. because of childcare responsibilities) or may work on several projects at the same time. Agile Software Engineering © Ian Sommerville 2018: 41  External interactions External interactions are interactions that team members have with people outside of the team. In Scrum, the idea is that developers should focus on development and only the ScrumMaster and Product Owner should be involved in external interactions. The intention is that the team should be able to work on software development without external interference or distractions. Agile Software Engineering © Ian Sommerville 2018: 42  Figure 2.10 Managing external interactions Figure 2.10 Managing external interactions External interactions Team-focused Product-focused external interactions external interactions ScrumMaster Product owner Agile Software Engineering © Ian Sommerville 2018: 43  Project management In all but the smallest product development companies, there is a need for development teams to report on progress to company management. A self-organizing team has to appoint someone to take on these responsibilities. Because of the need to maintain continuity of communication with people outside of the group, rotating these activities around team members is not a viable approach. The developers of Scrum did not envisage that the ScrumMaster should also have project management responsibilities. In many companies, however, the ScrumMaster has to take on project management responsibilities. They know the work going on and are in the best position to provide accurate information and project plans and progress. Agile Software Engineering © Ian Sommerville 2018: 44  Figure Figure 2.11 Project 2.11 Project management management responsibilities responsibilities Reporting Budget Schedule Risks Problems Progress Project management Administration People Finance Vacations Compliance Absence Procurement Work quality Liaison Reviewing Hiring Agile Software Engineering © Ian Sommerville 2018: 45  Key points 1 The best way to develop software products is to use agile software engineering methods that are geared to rapid product development and delivery. Agile methods are based around iterative development and the minimization of overheads during the development process. Extreme programming (XP) is an influential agile method that introduced agile development practices such as user stories, test-first development and continuous integration. These are now mainstream software development activities. Scrum is an agile method that focuses on agile planning and management. Unlike XP, it does not define the engineering practices to be used. The development team may use any technical practices that they believe are appropriate for the product being developed. In Scrum, work to be done is maintained in a product backlog – a list of work items to be completed. Each increment of the software implements some of the work items from the product backlog. Agile Software Engineering © Ian Sommerville 2018: 46  Key points 2 Sprints are fixed-time activities (usually 2–4 weeks) where a product increment is developed. Increments should be ‘potentially shippable’ i.e. they should not need further work before they are delivered. A self-organizing team is a development team that organizes the work to be done by discussion and agreement amongst team members. Scrum practices such as the product backlog, sprints and self-organizing teams can be used in any agile development process, even if other aspects of Scrum are not used. Agile Software Engineering © Ian Sommerville 2018: 47