Software Research Methods and Gender Bias

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

What is one possible explanation for the low birth weight paradox?

Babies born to smoking mothers may receive more medical attention.

Smoking during pregnancy may cause lower birth weights that correlate with lower infant mortality. (correct)

Low birth weight is always a risk factor for higher infant mortality.

Low birth weight babies are less likely to be born in low socioeconomic conditions.

Which of the following is a reason why raw figures may not be sufficient in software research?

Raw figures can overestimate the difficulties faced by the teams.

Raw figures provide a complete view of the developers' productivity.

Raw figures are always biased based on project funding.

Raw figures do not account for variations in team experience. (correct)

What aspect does a null hypothesis generally pertain to in an experiment?

It hypothesizes an increase in outcomes due to stronger methods.

It claims the experiment will always succeed.

It states that there is no effect or relationship. (correct)

It confirms the alternative hypothesis.

In the context of Harvard's gender bias statistics, which finding is notable?

Men had a larger number of applications but lower acceptance rates. Signup and view all the answers

How can the effectiveness of pair programming be evaluated in an experiment?

By assessing team output and quality of code produced. Signup and view all the answers

What percentage of the overall assessment is attributed to the written examination?

80% Signup and view all the answers

Which of the following is NOT mentioned as a recommended book for the module?

Effective Java Signup and view all the answers

Which software development tool is specifically mentioned as necessary for Java development?

Eclipse Signup and view all the answers

What is one of the focus areas of the module aims?

Producing software in a timely manner Signup and view all the answers

Which topic is included in the contents of the module?

Concurrency and the Actor model Signup and view all the answers

How often are tutorials held in the module?

Once a week Signup and view all the answers

What is the primary aim of reviewing research papers in this module?

To enhance understanding of software engineering Signup and view all the answers

Which source control resource is recommended for this module?

GitHub Signup and view all the answers

What type of software failure was the Ariane 5 launch associated with?

Catastrophic failure Signup and view all the answers

Which of the following is NOT listed as a way for a project to fail?

Project loses all data Signup and view all the answers

What can be a consequence of low quality code in a project?

Chronic functionality problems Signup and view all the answers

Which scenario involves a highly skilled team using the best object-oriented techniques?

Theoretical Development Scenario 1 Signup and view all the answers

What issue arose with the Tesla Autopilot feature in 2023?

Software glitch causing crashes Signup and view all the answers

What type of specification was used in Theoretical Development Scenario 2?

Perfect specification Signup and view all the answers

Which of the following would NOT be considered a hard problem in software development?

Simple user interfaces Signup and view all the answers

What consequence resulted from a software glitch during the rollout of the Pfizer COVID-19 vaccine?

Delay in distribution Signup and view all the answers

What percentage of software projects are completed on time and on budget according to the Standish CHAOS report?

16.2% Signup and view all the answers

What was the primary reason for project overruns as per the Standish CHAOS report?

Bad Planning and Estimating Signup and view all the answers

Which type of project completion is characterized as 'Challenged' in the Standish project resolution framework?

Over time and budget with incomplete features Signup and view all the answers

According to the findings, what is a key factor in successful projects?

Clear Statement of Requirements Signup and view all the answers

What percentage of projects is cited for having 'Incomplete Requirements' as a reason for cancellation or failure?

13.1% Signup and view all the answers

Which aspect is NOT highlighted as a characteristic of successful projects in the Standish CHAOS report?

High Market Demand Signup and view all the answers

Which software platform experienced a significant global failure due to an update from Crowdstrike?

Windows Signup and view all the answers

What is the average cost of a software project for a medium-sized company as stated in the Standish report?

$1,331,000 Signup and view all the answers

What term describes projects that are completed on time and on budget with all features as per the Standish classification?

Successful Signup and view all the answers

Which is a common concern raised due to the glitch affecting vaccine tracking?

Safety and efficacy of the vaccine Signup and view all the answers

What characterizes the pioneer stage of software engineering from 1945 to 1965?

Development of the first computer programs Signup and view all the answers

Which of the following is NOT a general engineering principle?

Software testing Signup and view all the answers

What is the primary focus of the software crisis phase from 1965 to 1985?

Research in formal methods and OO methods Signup and view all the answers

Which of the following activities is part of software management?

Quality assurance Signup and view all the answers

What is a key trend in modern software engineering?

Emphasis on Agile and test-driven development Signup and view all the answers

According to NATO Science Committee, what is considered sound software engineering?

Applying scientific principles for efficiency and reliability Signup and view all the answers

Which activity is performed right after software design?

Implementation Signup and view all the answers

What does the term 'No software silver bullet' imply in the context of software engineering?

Complex software problems require multiple approaches Signup and view all the answers

What is the Chaos Report primarily concerned with?

Project delivery and estimation Signup and view all the answers

How is project success primarily measured according to the Chaos Report?

By comparing budget and time Signup and view all the answers

What does an EQF (Estimation Quality Factor) score greater than 10 indicate?

Very good estimation Signup and view all the answers

Which of the following is NOT listed as a factor influencing project success or failure?

Technological advancements Signup and view all the answers

What does the term 'overrun' typically refer to in the context of project management?

Going beyond the scheduled delivery date Signup and view all the answers

Which statement is accurate about the criticisms of the Chaos Report?

The classification of project success is often incomplete. Signup and view all the answers

What is the formula for calculating the Estimation Quality Factor (EQF)?

EQF = 1 / (Average(Deviation/Actual)) Signup and view all the answers

What does a low EQF indicate about a project?

There was a significant error in estimated versus actual performance. Signup and view all the answers

What is typically encouraged among project managers concerning their estimations?

To overestimate in order to manage expectations. Signup and view all the answers

What can be inferred about projects with an EQF of 0.43?

They had very low success rates. Signup and view all the answers

Study Notes

Course Information

Course Title: SOFTWARE ENGINEERING II
Course Code: COMP319
Academic Year: 2024-2025
Instructor: Sebastian Coope
Email: [email protected]

Delivery

Lectures: On-campus and online recorded lectures.
Tutorials/Seminars: One hour per week.

Installation

Install Eclipse for Java development (latest version).
Install AspectJ development tools.
Use a source control resource (e.g., GitHub).

Assessment

80% Written examination.
20% Object-Oriented (OO) pattern design programming assignment.
Exam material covers lectures and tutorial work.

Module Aims

Introduce advanced software engineering topics.
Review and analyze software engineering research papers.
Focus on how to read research papers and improve software development.

Module Materials

Lecture notes
Selected papers
Past exam papers and model answers
Tutorial sheets

Recommended Books

No single recommended textbook; research and advanced topics are essential.
Java Design Pattern Essentials (ISBN 0956575803)
Learning Agile: Understanding Scrum, XP, Lean, and Kanban (ISBN 978-1449331924)
Actors: A Model of Concurrent Computation in Distributed Systems (ISBN 978-0262511414)

Software engineering crisis.
Software cost estimation and project management techniques.
Object-Oriented (OO) design patterns.
Agile practices (e.g., Scrum, Extreme Programming (XP)).
Concurrency and the Actor model.
Dependency graphs and program slicing

Software Engineering (Slide 9)

High complexity due to numerous platforms (e.g., iPhone, Android).
Use of varied models, such as client-server and peer-to-peer.
Complex problems (e.g., AI, neural networks), as well as large software size (e.g., Linux kernel).

Software Crisis and Failure Classification (Slide 10)

Catastrophic failures (e.g., Ariane 5 rocket failure due to variable overflow).
Chronic failures (e.g., project overruns, poor functionality, low code quality).

Ways to Fail (Slide 11)

Project incompleteness.
Budget overruns.
Buggy projects.
Missed deadlines. Incomplete feature sets.
Code base unsupportability (e.g., poor code, inflexibility).
Poor platform portability.

Theoretical Development Scenarios (Slides 12 and 13)

Presents two scenarios emphasizing different specification and development team qualities.

Failures 2023 (Slides 14 and 15)

Tesla autopilot incidents: Failures to detect obstacles.
Pfizer COVID-19 vaccine rollout delays: Software issue disrupting tracking.

Failures 2024 (July) (Slide 17)

Microsoft Azure update issues causing Windows blue screen of death in many sectors (e.g., Airlines, NHS). These were not related to attacks.

CNET Money (Slide 18)

CNET used AI to generate an article on compound interest calculation. Example demonstrating accuracy and effectiveness.

Limits of Language Models (Slide 19)

Chat GPT, despite sophisticated text generation, relies on statistical patterns from training data; it does not genuinely understand the text generated.

Standish Chaos Report (Slide 20)

Comprehensive study of software development projects; total spend, costs, cancellation rates, and on-time/on-budget project percentages are key aspects.

Standish Project Resolution (Slide 21)

Categorizes software projects into successful, challenged, and incomplete types, based on completion and budget/time adherence.

Standish Chaos Report Findings (Slides 22 and 23)

Detailed causes of project overruns and project success.
The most frequent project overrun reasons and reasons for project success to support projects.

Reasons for Cancelled/Failed Projects (Slide 24)

Primary causes behind project cancellation or failure.

Standish Chaos in Perspective (Slide 25)

Critiques the bias and limitation of Standish metrics.

Chaos Report Analysis (Slide 26)

Critical discussion on challenges around defining failures and overruns.

Chaos Report Criticisms (Slide 27)

Key limitations and issues raised in evaluating the chaos report's data accuracy and categorization.

Estimation and Chaos Report (Slide 28)

Importance of estimation accuracy for software projects.
Companies' tendencies toward either underestimation or overestimation.

Boem's Cone of Uncertainty (Slide 29)

Presents a graphical representation of uncertainty in software project scheduling.

Project Success/Failure (Slide 30)

Project success is dependent on several external factors, including company context, culture, and estimation/development skills.

Estimation Quality Factor (EQF) (Slides 31 and 32)

Provides a measure of estimation accuracy (EQF)
This technique allows for determination of whether an estimate is accurate

EQF Example (Slide 33)

Presents an example of calculating EQF using sample data.

EQF (Slide 34)

Importance of EQF scores for estimation accuracy.
High EQF scores indicate accurate estimation, while low scores point to estimation inaccuracies/inaccuracies

Bias (Slide 35)

Definition and calculation of bias to adjust estimation accuracy.

Example Data from Slide 36

Illustrates the calculation of bias for the project estimations

Things that Effect Estimation (Slide 37)

Factors that influence project estimation accuracy (e.g., timing, management pressure, experience, design details, specification details).

In General (regarding bias, Slide 38)

General summary of possible estimator bias and potential causes.

Exercise (Slides 39 & 40)

Calculation tasks for EQF and bias, and analysis of the results.

Rise and Fall of the Chaos Report Figures (Slides 40 - 42)

Detailed analysis of various studies involving the Chaos report and their outcomes; the findings have been evaluated in numerous studies.

Research and Critical Analysis (Slide 43)

Importance when evaluating research of critically assessing the underlying methodology, sample size, and potential biases.

Exercise (Slide 44)

Includes a pair programming effectiveness experiment setup exercise.

Final Point on Using Statistics (Slide 45)

Presents a paradox regarding low birth weight and smoking during pregnancy.

Another Example (Slide 46)

Discusses Harvard University's admissions data highlighting potential for bias.

Harvard Figures Broken Down (Slide 47)

Provides a more elaborate or detailed breakdown of data.

What's This To Do with Software Research (Slide 48)

Highlights the importance of recognizing hidden biases when comparing different teams' or organizations' performance.

Engineering (Slide 49)

Definition of engineering in a broad sense; in this context, it is related to the broad application of science in creating and building useful systems or products.

Software Engineering (Slide 50)

Definition of Software Engineering.

Software Engineering History (Slide 51)

Overview of software engineering timeline
Phases of development and crucial events.

Software Engineering Definition (Slide 52)

Definition from NATO Science Committee, Fritz Bauer.

General Engineering Principles (Slide 53)

General engineering principles (specification, design, manufacturing, quality, testing, modification).

Software Engineering Activities (Slide 54)

Key activities in the software development process.

Trends Going Forwards (Slide 55)

Key trends in software engineering (e.g., Agile methodologies, software tools, improved languages, incremental development, and project management).

Summary (Slide 56)

Brief summary of main ideas about researching experiments, considering context and being skeptical of results.

Studying That Suits You

Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

Description

This quiz covers various aspects of software research methods, including evaluations of pair programming and the significance of null hypotheses. Additionally, it addresses notable findings related to gender bias statistics from Harvard and other essential module content. Test your understanding of these critical topics in software development and research methodology.