Advanced Software Engineering COMP319

Course Information

• Course title: SOFTWARE ENGINEERING II
• Course code: COMP319
• Academic year: 2024-2025
• Lecturer: Sebastian Coope
• Lecturer email: [email protected]

Delivery

• Lectures: on-campus and online recorded
• Tutorials/Seminars: 1 hour per week

Installation

• Install Eclipse for Java development (latest version)
• Install AspectJ development tools via Eclipse
• Use a source-control resource, like a GitHub account

Assessment

• 80% Written examination
• 20% Object-Oriented (OO) pattern design programming assignment
• Examination based on material from lectures and tutorials.

Module Aims

• Introduce advanced software engineering topics
• Review and analyze research papers in software engineering
• Focus on how to read and understand research in general and how to produce better software in a timely manner.

Module Materials

• Lecture notes
• Selected research papers
• Past exam papers and model answers
• Tutorial sheets

Recommended Books

• No single recommended textbook, but resources should include research and advanced topics.
• Java Design Pattern Essentials (ISBN 0956575803)
• Learning Agile: Understanding Scrum, XP, Lean, and Kanban (ISBN 978-1449331924)
• Actors: A Model of Concurrent Computation in Distributed Systems (ISBN 978-0262511414)

Contents

• Software engineering crisis
• Software cost estimation and project management
• Object-Oriented (OO) design patterns
• Agile and eXtreme Programming (XP)
• Concurrency and the Actor model
• Dependency graphs and program slicing

Software Engineering

• Highly complex systems (many platforms, models, problems, large sizes)
• Examples: iPhone, Android, PC, Linux, HTML5; Standalone, client-server, peer-to-peer; Artificial intelligence (AI) neural nets; Linux kernel (>10 million lines)

Software Crisis and Failures

• Catastrophic failures (e.g., Ariane 5 launch failure): 7 billion USD cost, caused by variable overflow in software.
• Chronic failures: project overruns, functionality problems, poor performance, and poor quality code.

Ways to Fail

• Project incomplete
• Project over budget
• Project buggy
• Project over schedule
• Project does not fully deliver what is required
• Code base unsupportable (badly written, inflexible)
• Code doesn't port well between platforms (e.g., web browser compatibility)

Theoretical Development Scenarios

• Scenario 1: Imperfect specification, perfectly implemented and tested.
• Scenario 2: Perfect specification, but low coding skills, fully implemented and tested.

Failures 2023

• Tesla Autopilot: Several high-profile crashes due to the Autopilot feature failing to detect obstacles (Washington Post, 17 fatal incidents).
• Pfizer COVID-19 Vaccine rollout: Delayed in several countries due to software glitch affecting vaccine distribution tracking.

Failures 2024 (July)

• Microsoft Azure platform: Update released from Crowdstrike caused Windows PC blue screen of death, affecting airlines, medical centres, etc. This impacted global operations. Not an attack. Question: can systems be made less vulnerable?

CNET Money

• Used AI engine to write compound interest article
• Example result: $10,000 deposited in a 3% annually compounding interest account will earn $10,300 in the first year.

Limits of Language Models

• ChatGPT does not understand the meaning of the words/sentences/paragraphs.
• It uses statistical patterns from training data to generate text.

Standish Chaos Report (US)

• Comprehensive report on software development.
• Average cost of projects (large, medium, small companies).
• 31% of projects are cancelled before completion.
• 52.7% cost 189% of original estimate.
• 16.2% are completed on time and on budget.
• For larger companies, only 9% are completed on time and on budget.
• Project resolutions: successful, challenged, incomplete.

Standish CHAOS Report Findings

• Reasons for project overruns and failures.
• Reasons for successful projects, e.g. user involvement, executive support, clear requirements, proper planning, etc.

Reasons for Cancelled/Failed Projects

• Detailed list of reasons for project cancellation/failure, percentages for each one.

Standish Chaos in Perspective

• Critiques of the Standish report; did it only look at bad news? What is the context and extent of the software crisis? How many systems in daily use? How would they be rated?

Chaos Report Analyzed

• Defining failure and overrun (e.g., incomplete features, delivery schedule).
• Techniques for project delivery or estimation failure.

Chaos Report Criticisms

• Classification of incomplete projects.
• Published data completeness (raw or not)
• Measuring failure (forecast/actual).

Estimation and Chaos Report

• Success measured relative to initial estimate.
• Companies sometimes underestimate (under-resourced, over-promised) or overestimate (over-resourced).

Boem's Cone of Uncertainty

• Graphical representation of uncertainty in project scheduling and various stages, e.g., initial product definition to accepted software.

Project Success/Failure

• Success or failure defined by context (budget, estimations, type of company).
• Culture of sales-driven or development-driven estimations, estimation skills, development skills.

Estimation Quality Factor (EQF)

• Formal method for measuring how accurate estimations are.

EQF Example

• Calculating EQF and bias for a sample project data set and outcomes.

Higher EQF Scores

• Indicate better estimations.
• EQF >10 is considered very good.

Bias

• Formula for calculating bias in project estimations (Bias = Mean(Estimator) - ActualValue).
• Percentage for bias is (Mean(Estimator) - ActualValue)/ActualValue

Things That Affect Estimations

• Early estimations harder
• Management pressure affecting estimations.
• Inexperienced developers overestimate/underestimate
• Poor design specifications
• Poor quality specifications create issues, causing difficulty with estimation.

In General

• Negative or positive bias in estimations due to over-complex initial thought, mid-project specification changes, over-optimism(past experience, underestimate), risk-adverse project managers.

Exercise

• Calculate EQF and bias for 3 projects, and draw conclusions.
• Project data sets given.

Rise and Fall of the Chaos Report Figures

• Determination of organizational performance relative to Chaos Report findings.
• Data for projects over two years for 3 organizations, including EQF and other success metrics.

Research and Critical Analysis

• Warnings about headline figures, and considerations for research (cause of results, sample size, random outcomes).

Exercise (Further)

• Designing an experiment to test the effectiveness of pair-programming.
• What is the null hypothesis of that experiment?

Final Point on Using Statistics

• Low birth weight paradox and explanations.
• Babies born to mothers who smoked in pregnancy having lower infant mortality.

Another Example

• Harvard University gender bias application/admission rates data table.
• The breakdown of this data.

What's This To Do With Software Research?

• Comparing software developers/teams/companies.
• Are raw figures adequate? How performance is measured? How to measure the size of code?

Engineering

• Definition of engineering, based on the American Engineers' Council for Professional Development.

Software Engineering

• Definition of software engineering, with the focus on application of principles to design and develop software systems.

Software Engineering History

• Timeline of software engineering, with emphasis on:
  • Pioneer stage (1945-1965)
  • Software crisis (1965-1985) and related research areas.
  • Current stage (1985-today) and relevant research areas.

Software Engineering

• Quote from the NATO Science Committee, Fritz Bauer, explaining the principles of software engineering.

General Engineering Principles

• General engineering principles (specification, design, manufacture/implementation, quality control, testing, modification).

Software Engineering Activities

• Key activities in software engineering, including requirements analysis, software design/implementation, patterns, actor model, AOP, software testing and analysis, program slicing, software management (scheduling, quality assurance, workflow), etc.

Trends Going Forwards

• Trends related to Agile; Test-driven development, good usage of software tools, better languages, incremental development, Scrum, improved estimation & project management.

Summary

• Designing experiments is difficult.
• Context of activities is important in software engineering
• Software engineering: a mixed bag of evidence for approaches.
• Be skeptical of headline/summary results.