Software Evolution Lecture (7) PDF

Software Evolution Lecture (7) Dania Mohamed Ahmed  Software change  Software change is inevitable because:  New requirements: Users find new needs as they use the software.  Business changes: The business environment evolves, affecting software needs.  Error fixes:...

Software Evolution Lecture (7) Dania Mohamed Ahmed  Software change  Software change is inevitable because:  New requirements: Users find new needs as they use the software.  Business changes: The business environment evolves, affecting software needs.  Error fixes: Bugs and issues must be corrected.  New hardware: New computers or equipment may require software updates.  Performance: The system might need improvements for better performance or reliability.  A key problem for organizations is managing and implementing changes to their existing software systems. Importance of evolution  Organisations have huge investments in their software systems - they are critical business assets.  To maintain the value of these assets to the business, they must be changed and updated.  The majority of the software budget in large companies is devoted to changing and evolving existing software rather than developing new software. A spiral model of development and evolution  Evolution and servicing  Describing stages in the lifecycle of software systems: 1. Evolution: This is the phase where the software is actively used and continuously improved. During this stage, new requirements are gathered and implemented, and the system evolves to meet changing needs. This might involve adding new features, improving performance, or making other modifications based on user feedback and changing requirements. 2. Servicing: In this stage, the software is still operational and useful, but changes are limited to maintaining its functionality. This includes fixing bugs, updating for changes in the environment (such as operating system updates), and other maintenance tasks. No new features are added, and the focus is solely on keeping the existing system running smoothly. 3. Phase-out: At this point, the software is still in use, but no further development or changes are made. It may continue to operate as is, but it’s no longer being actively maintained or updated. The software might be gradually replaced or retired, and users might start transitioning to newer solutions. Evolution and servicing  Evolution processes  Evolution processes in software development depend on: 1. Dependence on Software Type:  Application Software: Evolves through feature additions, user interface improvements, and system integrations.  System Software: Focuses on performance enhancements, security updates, and hardware support.  Embedded Software: Requires attention to hardware constraints and real- time requirements. 2. Development Processes:  Agile: Supports iterative changes with frequent feedback and short cycles.  Waterfall: Offers a more structured approach with defined phases for change.  DevOps: Facilitates ongoing evolution through continuous integration, deployment, and automated testing.  Evolution processes 3. Skills and Experience:  Experienced Teams: Handle changes effectively, anticipating impacts and implementing solutions efficiently.  Skill Gaps: May lead to challenges in assessing impacts and managing changes, potentially causing disruptions.  Proposals for Change  Change Proposals: Triggered by new requirements, feedback, or deficiencies. Should be well-documented, outlining the rationale and benefits.  Impact Analysis: Links proposals to affected components to estimate costs and assess impacts on system functionality.  Change Identification and Evolution  Continuous Process: Involves regular monitoring of performance, feedback, and requirements, followed by impact assessment and controlled implementation of changes. Change identification and evolution processes The software evolution process Change implementation  Change implementation 1. Process Overview:  Iteration of Development: The change implementation is akin to a mini- development cycle where changes are designed, implemented, and tested. 2. Program Understanding:  Initial Stage: If the original developers are not involved, understanding the existing system is crucial. This involves:  System Structure: Learning how the software is organized.  Functionality Delivery: Understanding how the software provides its features.  Impact Assessment: Evaluating how the proposed changes will affect the existing system.  Change implementation 3. Implementation Steps:  Design: Create a detailed plan for how the changes will be integrated into the system.  Implementation: Develop the changes according to the design plan.  Testing: Ensure that the changes work as intended and do not introduce new issues.  By thoroughly understanding the existing system and carefully managing the design, implementation, and testing stages, the change process can be more effectively controlled and integrated. Urgent change requests  Immediate Action: Sometimes changes need to be made quickly without following all the usual steps.  System Faults: Fixing critical issues to keep the system running.  Environment Changes: Addressing unexpected problems from things like OS updates.  Business Needs: Responding swiftly to events like a competitor's new product. The emergency repair process  Agile methods and evolution ► Seamless Transition: In Agile, development and evolution are continuous and interconnected. Evolution follows naturally from Agile’s incremental development approach with regular system releases. ► Automated Regression Testing: Essential for ensuring that new changes don’t disrupt existing functionality. It helps maintain system stability as the software evolves. ► User Stories for Changes: New changes are often documented as additional user stories, integrating them smoothly into the development cycle and maintaining alignment with user needs  Handover problems 1. Agile to Plan-Based:  Issue: The development team used Agile, which may not produce extensive documentation.  Problem: The evolution team, preferring a plan-based approach, might expect detailed documentation that isn't available, complicating their work. 2. Plan-Based to Agile:  Issue: The development team used a plan-based approach, which may not include practices like automated testing and code refactoring.  Problem: The evolution team, using Agile, might need to create automated tests from scratch and deal with code that hasn't been simplified or refactored as Agile practices would typically ensure. Program evolution dynamics ► Definition: Program evolution dynamics examines how and why software systems change over time. ► Lehman and Belady's 'Laws': Based on empirical studies, they proposed several 'laws' or observations about system evolution. These are:  Sensible Observations: More accurately described as observations rather than strict laws.  Applicability: Primarily relevant to large systems developed by big organizations. Their relevance to smaller systems or different types of software is less clear. Change is inevitable ► Evolving Requirements: As a system is developed, changes in the environment often lead to evolving requirements, meaning the delivered system may not fully meet the initial specifications. ► Environment Interaction: A system affects its environment when installed, which can alter the environment and, consequently, the system’s requirements. ► Continuous Adaptation: To stay relevant and useful, systems must be updated and changed in response to shifts in their environment.  Lehman’s laws Law Description Continuing change A program that is used in a real-world environment must necessarily change, or else become progressively less useful in that environment. Increasing complexity As an evolving program changes, its structure tends to become more complex. Extra resources must be devoted to preserving and simplifying the structure. Large program Program evolution is a self-regulating process. System attributes evolution such as size, time between releases, and the number of reported errors is approximately invariant for each system release Organizational Over a program’s lifetime, its rate of development is approximately stability constant and independent of the resources devoted to system development  Lehman’s laws Law Description Conservation of Over the lifetime of a system, the incremental change in each familiarity release is approximately constant. Continuing growth The functionality offered by systems has to continually increase to maintain user satisfaction. Declining quality The quality of systems will decline unless they are modified to reflect changes in their operational environment Feedback system Evolution processes incorporate multiagent, multiloop feedback systems and you have to treat them as feedback systems to achieve significant product improvement Applicability of Lehman’s laws  General Fit: Lehman’s laws generally apply to large, custom systems developed by big organizations, as confirmed by Lehman’s FEAST project research in the early 2000s.  Uncertainty for Other Types:  Shrink-Wrapped Software: Packaged software sold to many users.  COTS Systems: Systems using many commercial off-the-shelf components.  Small Organizations: Smaller teams with different dynamics.  Medium-Sized Systems: Systems that are neither small nor large.  The laws might need adjustments for these different scenarios.  Software maintenance  Software maintenance is a critical aspect of the software development lifecycle, focusing on modifying and updating software after its initial deployment.  Definition: Software maintenance refers to the process of making changes to a program after it has been put into use. This can include bug fixes, enhancements, and adjustments to improve performance or adapt to new requirements.  Custom vs. Generic Software: Maintenance is often discussed in the context of custom software, which is tailored for specific user needs. In contrast, generic software products typically evolve through version updates that introduce new features or improvements.  Scope of Changes: Maintenance usually does not involve significant alterations to the system's architecture. Instead, it focuses on modifying existing components or adding new ones to address specific issues or needs. Types of maintenance 1. Corrective Maintenance: Fixing defects or issues that arise in the software. 2. Adaptive Maintenance: Updating the software to work in new environments or with new hardware/software. 3. Perfective Maintenance: Enhancing existing features or adding new capabilities to improve performance or user experience. 4. Preventive Maintenance: Making changes to prevent future problems or to simplify future update Maintenance effort distribution  Maintenance costs  Maintenance costs in software development often exceed initial development expenses, with estimates ranging from 2 to 100 times the original cost. Key factors influencing these costs include: 1. Technical Factors: Complexity, code quality, and architecture can significantly affect maintenance expenses. 2. Non-Technical Factors: Team skill levels, documentation quality, and project management also play a crucial role.  As software is maintained, its structure may degrade, leading to "maintenance corruption," which complicates future updates. Aging software tends to incur high support costs, especially if it relies on outdated technologies.  To mitigate long-term maintenance costs, organizations should invest in good design practices, conduct regular code reviews, and modernize technology stacks. Proper planning for the entire software lifecycle is essential to allocate adequate resources for ongoing maintenance and support, ensuring long-term success and sustainability. Development and maintenance costs  Maintenance cost factors 1. Team Stability: Having the same team work on maintenance over time can lower costs, as they become familiar with the system. 2. Contractual Responsibility: If developers aren't responsible for maintenance, they may not design the software with future changes in mind. 3. Staff Skills: Maintenance teams often lack experience and deep knowledge of the system, which can lead to higher costs. 4. Program age and structure : As software ages, its structure can degrade, making it harder to understand and modify, thus increasing maintenance challenges Maintenance prediction  Purpose: Identify system components likely to fail or incur high maintenance costs.  Change Impact:  Acceptance of changes is influenced by how maintainable the affected components are.  Implementing changes can lower system maintainability.  Cost Factors:  Maintenance costs increase with the number of changes.  The expense of making changes is affected by the maintainability of the components involved. Maintenance prediction Change Prediction ► Purpose: Estimate how many changes a system will need based on its interaction with the environment. ► Key Points:  Tightly coupled systems require updates whenever the environment changes. ► Influencing Factors:  Complexity and number of system interfaces.  Number of unstable system requirements.  Business processes that use the system. Complexity metrics ► Purpose: Assess the maintainability of system components by evaluating their complexity. ► Key Points:  Most maintenance work focuses on a small number of components. ► Complexity Factors:  Control structure complexity.  Data structure complexity.  Size of objects, methods, and modules. Process Metrics ► Purpose: Use metrics to evaluate maintainability based on maintenance activities. ► Key Metrics:  Number of corrective maintenance requests.  Average time for impact analysis.  Average time to implement change requests.  Number of outstanding change requests. ► Implication: Increasing values in these metrics may indicate declining maintainability.  System re-engineering ► Definition: Restructuring or rewriting parts of a legacy system without altering its functionality. ► When to Use: Useful when some subsystems need frequent maintenance, but not all. ► Process: Involves improving maintainability through restructuring and re- documentation. ► Advantages of Re-engineering 1. Reduced Risk: Lower risks compared to new software development, which can face issues with development, staffing, and specifications. 2. Reduced Cost: Often less expensive than developing entirely new software.  Reengineering Process Activities 1. Source Code Translation: Convert existing code into a new programming language. 2. Reverse Engineering: Analyze the program to gain a better understanding of its functionality. 3. Program Structure Improvement: Automatically restructure the code for better readability and understanding. 4. Program Modularization: Reorganize the program into smaller, manageable modules. 5. Data Reengineering: Clean and restructure the system's data for improved quality and organization The reengineering process  Reengineering cost factors 1. Software Quality: The overall quality of the existing software being reengineered. 2. Tool Support: The availability and effectiveness of tools that aid the reengineering process. 3. Data Conversion Needs: The amount of data that requires conversion and cleanup. 4. Expert Staff Availability: Access to knowledgeable staff for reengineering, which can be challenging for older systems using outdated technology. Preventative maintenance by refactoring ► Definition: Refactoring is the process of improving a program to prevent issues from future changes. ► Purpose: Acts as "preventative maintenance" to reduce the risks associated with future modifications. ► Focus: Involves modifying the program to enhance its structure, reduce complexity, and improve understandability. ► Key Principle: Refactoring should not introduce new functionality; the emphasis should be on improving the existing code. Refactoring and reengineering ► Reengineering: Occurs when maintenance costs rise after a system has been in use for a while. It involves using automated tools to update a legacy system into a more maintainable one. ► Refactoring: A continuous improvement process that happens throughout development. It aims to prevent the degradation of structure and code, which can increase maintenance costs and challenges. ‘Bad smells’ in program code 1. Duplicate Code: Similar code appears in multiple places. It should be consolidated into a single method or function. 2. Long Methods: Methods that are too lengthy should be broken down into shorter, more manageable methods. 3. Switch (Case) Statements: Often involve duplication based on value types. In object-oriented programming, polymorphism can replace these statements. 4. Data Clumping: Groups of related data items appear repeatedly. They can be replaced with a single object that contains all the related data. 5. Speculative Generality: Developers add unnecessary general features for potential future use. This can often be removed to simplify the code. Legacy system management  Organizations using legacy systems need to choose a strategy for their evolution: 1. Scrap the System: Completely discard the system and adjust business processes to eliminate its need. 2. Maintain the System: Continue to provide maintenance as needed. 3. Transform the System: Re-engineer the system to enhance its maintainability. 4. Replace the System: Implement a new system in place of the old one.  The chosen strategy should consider the system's quality and its business value Legacy system categories 1. Low Quality, Low Business Value: Should be scrapped. 2. Low Quality, High Business Value: Important for business but costly to maintain. Should be re-engineered or replaced if a suitable alternative exists. 3. High Quality, Low Business Value: Options include replacing with Commercial Off-The-Shelf (COTS) solutions, scrapping, or maintaining the system. 4. High Quality, High Business Value: Continue operating with regular maintenance. An example of a legacy system assessment Business value assessment  Key Viewpoints to Consider:  System end-users  Business customers  Line managers  IT managers  Senior managers  Method: Conduct interviews with various stakeholders and gather their feedback to form a comprehensive assessment. Issues in Business Value Assessment 1. System Use: If used infrequently or by few people, the system may have low business value. 2. Supported Processes: A system linked to inefficient business processes may also be considered low in value. 3. System Dependability: If the system is unreliable and affects customers, its business value decreases. 4. System Outputs: If the business relies heavily on the system's outputs, it is regarded as having high business value. System quality assessment ► Business Process Assessment  How well do current processes support the business goals? ► Environment Assessment  How effective is the system's environment?  What are the maintenance costs? ► Application Assessment  What is the overall quality of the application software?  Business process assessment  Engage Stakeholders for Insights: 1. Defined Process Model  Is there a clear process model, and is it being followed? 2. Process Variations  Do different departments use different processes for the same task? 3. Process Adaptation  How has the process changed over time? 4. Inter-Process Relationships  What are the connections to other business processes, and are they necessary? 5. Support from Legacy Systems  Does the legacy software effectively support this process?  Example:  A travel ordering system may have low value if many users opt for web-based ordering instead.  Factors used in environment assessment Factor Questions Supplier Stability Is the supplier still in business? Are they financially stable? Failure Rate Does the hardware frequently fail? Do software crashes require system restarts? Age How old is the hardware and software? Older systems may be obsolete, even if they still work Performance Is the system’s performance adequate? Do issues significantly impact users? Support What local support is needed? Are support costs high, making Requirements replacement worthwhile? Maintenance Costs What are the costs for hardware maintenance and software licenses? Older systems might incur higher costs. Interoperability Are there issues connecting this system to others? Can current tools work with the latest operating systems? Factors used in application assessment Factor Questions Understandability How easy is it to understand the source code? Are the control structures complex? Do variable names clearly reflect their purpose? Documentation What documentation exists? Is it complete, consistent, and up to date? Data Is there a clear data model? How much data is duplicated? Is the data current and consistent? Performance Is the application’s performance satisfactory? Do issues impact users significantly? Programming Are modern compilers available for the programming language used? Is Language this language still relevant for new developments? Configuration Are all system versions managed properly? Is there a clear description of Management the current component versions? Test Data Is there test data available? Are there records of regression tests conducted for new features? Personnel Skills Are there skilled personnel available to maintain the application? Do they have experience with the system?  System measurement  To assess the quality of the application system, you can collect quantitative data, such as: 1. Number of System Change Requests  How many change requests have been submitted? This can indicate areas of improvement or frequent issues. 2. Number of Different User Interfaces  How many distinct user interfaces does the system have? A high number might suggest inconsistency or complexity. 3. Volume of Data Used  What is the total volume of data processed by the system? This can help evaluate performance and storage needs.