Computer Reliability Chapter 8

The ability of a computer system to perform its intended function without failure.

Mistakes made when entering data into a computer system.

Mistakes made when retrieving data from a computer system. These misinterpretations lead to incorrect conclusions based on pulled data.

Problems in computer programs that cause unexpected or incorrect results.

Complete breakdowns or malfunctions in computer programs that affect the system's functionality.

A medical radiation machine that caused serious accidents and injuries due to software flaws and failures.

Models that use computers to mimic real-world processes or systems.

The discipline of designing, developing, and testing software to ensure its quality and reliability.

Promises that a software vendor makes about the product's quality and performance.

The responsibility of software companies for issues arising from software flaws or failures.

Voters in a 2000 US election who were wrongly identified as felons. This was due to errors in the voter database.

A malfunction of the Patriot missile system during the 1991 Gulf War, resulting in the failure to intercept Scud missiles, ultimately causing the deaths of 28 soldiers.

The Patriot missile system was designed to operate for a limited time, but was used for significantly longer durations while in operation

A software glitch in the Patriot missile system caused inaccurate tracking of the target.

A single, erroneous line of code in the error recovery procedure caused a cascading failure in the AT&T long-distance communication network.

A failure in a system in which one faulty component causes other components to fail.

The automated baggage handling system at Denver International Airport faced issues due to constraints including short timelines which ended with system complexity surpassing development team capabilities.

The timeline for a project is too short leading to issues in completing the project as planned.

A project's size and intricacy outgrew the capabilities or design of the team that built it, leading to problems in the deliverables.

Tesla's software system designed to control a car's speed and steering.

No automation; the driver controls all aspects of the car.

Driver assistance features, like anti-lock brakes and cruise control.

Partial Automation; the car can handle steering and acceleration/deceleration.

Conditional automation, where the car handles everything with human intervention expected for situations needing intervention.

High automation. All driving aspects with driver response not obligatory.

Full automation, the car manages all driving aspects.

The difficulty drivers face regaining control after a period of automated driving.

A fatal car accident involving a Tesla S and a semi-trailer truck on US-27A in Florida.

Driver did not focus ahead, was looking at a computer interface, possibly streaming a talent show. Braked late.

Pedestrian did not look for oncoming traffic until shortly before impact. May have been impaired by drugs.

Uber's Arizona testing program was suspended, and the testing facility was shut down, terminating 300 safety operators. A pedestrian lost their life.

Using computer models to predict events that are too expensive, time-consuming, unethical, or impossible to replicate in the real world.

Ensuring a computer program correctly implements the model.

Confirming the model accurately represents a real-world system.

To model past events, understand the world, and predict the future.

In 1994, only 1/6 of IT projects were completed on time and on budget. 1/3 cancelled before completion, and 1/2 had time/cost overruns

In 2009, 1/3 of IT projects were successful (on time and on budget). 1/4 were cancelled before completion, and 5/12 had time/cost overruns.

Unconscious bias in design decisions, often impacting products' usability for women due to design based on male prototypes.

Women's viewpoints might be overlooked in software development and the development and maintenance process due to time pressure or a lack of female representation.

AI systems trained on biased data can perform poorly with diverse populations. Examples include misidentification and labeling errors.

AI facial recognition systems trained on data with a preponderance of white, male faces resulted in disproportionately higher misidentification rates for darker-skinned females.

Some AI systems, like Google Photos, incorrectly labeled darker-skinned people as ‘gorillas’, highlighting the potential biases even in popular tools.

A cost-benefit analysis of the National Crime Information Center (NCIC) stolen vehicle database, considering the benefits of recovered vehicles and potential harm from false arrests.

Estimated 100,000 cars recovered annually due to the National Crime Information Center's (NCIC) database of stolen vehicles.

A small number of instances where innocent people are wrongly arrested due to inaccuracies in the NCIC database, despite benefits.

Cases where computerized systems malfunctioned despite correct data input.

Example of a software system error: Qwest sent incorrect cell phone bills due to issues with software in the billing system.

Computerized systems can malfunction even with correct data, if the programming has errors.

The potential for computerized systems to cause errors leading to malfunctions and consequences like false arrests and incorrect outputs.

A malfunction of the Patriot missile system during the 1991 Gulf War, leading to a failure to intercept Scud missiles and casualties.

The Patriot missile system was designed for limited operational time but was used for extended periods, leading to errors.

A software error caused inaccurate target tracking in the Patriot missile system.

A single flawed line of code in the error recovery procedure triggered a cascading failure in the AT&T network.

A failure in a system where an initial failure causes a chain reaction of subsequent failures.

The automated baggage handling system at Denver International Airport faced significant challenges due to factors like exceeding team capability and a tight timeline.

Short timelines can lead to significant problems and delays in project completion.

Projects with excessive complexity can outmatch the capability of the design team, leading to challenges in successful implementation.

Uber's initial attempts to operate self-driving cars, first in Pittsburgh and then San Francisco, encountered significant challenges and safety concerns.

Voting systems that record votes electronically without a paper trail.

In Uber's initial self-driving car tests, safety operators monitored the system, intervened when necessary, and logged significant events.

A 2002 U.S. law that provided funding to states for voting system upgrades.

Problems in DRE voting machines, such as failing to record votes, overcounting, or misrecording votes.

Self-driving cars sometimes mistakenly identify harmless situations (like car exhaust or steam) as threats, leading to unnecessary braking.

Self-driving cars may fail to detect actual dangers, which is arguably a worse outcome than false positives.

DRE machines do not create a paper record of votes, which makes them hard to verify in case of irregularities.

DRE machines are susceptible to hacking or malicious manipulation, leading to problems in vote outcomes.

In 2017, Uber removed the second safety operator from its test vehicles in order to improve efficiency.

A medical radiation machine that suffered fatal accidents due to software errors.

The California DMV revoked Uber's self-driving vehicle registrations due to a video of Uber's self-driving vehicle running a red light.

An error where the order of tasks accessing shared resources affects the machine's behavior.

Problems in computer programs that lead to malfunctions or unexpected results.

Tesla's software system for controlling car speed and steering.

No automation; the driver controls everything.

Driver assistance features, like anti-lock brakes or cruise control.

Partial automation handling steering & acceleration/deceleration.

Conditional automation. The car handles aspects but requires human intervention.

High automation; the car can handle all driving tasks, optionally.

Full automation. Everything is handled by the car.

Difficulty drivers face resuming control after automated driving.

A fatal car accident involving a Tesla S and a semi-truck; Autopilot system engaged.

Autopilot was engaged for 37 minutes before the collision.

The system provided 7 warnings to the driver to put their hands on the steering wheel before the collision.

The ability of a computer system to perform its intended function without failure.

Mistakes made when entering data into a computer system.

Mistakes in interpreting data retrieved from a system.

Programming glitches causing unexpected results.

Complete breakdowns in computer programs affecting functionality.

Medical radiation machine with fatal accidents due to software problems.

Computer models mimicking real-world processes.

Designing, developing, and testing software for reliability.

Promises a software vendor makes about its quality.

Responsibility of software companies for issues.

Voters wrongly identified as felons during voting.

Electronic voting systems that record votes digitally without a paper trail.

2002 US law that provided funding for states to upgrade voting systems.

Problems in DRE voting machines, such as incorrect vote counting, recording failures, or wrong vote registrations.

DRE machines lack a paper record of votes, making it hard to check for errors or fraud.

DRE machines can be hacked or manipulated.

In computer programming, an error where the order of multiple tasks accessing shared data can affect the program's behavior.

Problems in computer programs that lead to unexpected or incorrect results.

A medical radiation machine that caused serious accidents and injuries due to software errors.

Tesla's software system that helps control the car's steering and acceleration.

Time period when the Tesla Autopilot system was active from the dashboard view.

Uber's initial attempts to use self-driving cars, first in Pittsburgh, then San Francisco, faced issues and safety concerns.

In Uber's early self-driving tests, safety operators monitored the system and took control during necessary situations.

In 2017, Uber removed a second safety operator from its test vehicles due to concerns about operator alertness levels.

When self-driving cars incorrectly identify common situations as dangers, resulting in unnecessary braking.

Self-driving cars failing to recognize actual threats, a worse outcome than false positives.

The California DMV revoked Uber's self-driving vehicle registrations due to a video of an Uber vehicle running a red light.

A 2016 accident involving a Tesla and a truck, highlighting issues with the driver and warnings in a test vehicle.

The time the Autopilot system was engaged before an accident, in the Tesla test-vehicle accident.

The number of warnings the safety system gave the driver before the 2016 Tesla incident.

An accident involving an Uber self-driving car and a pedestrian in Tempe, Arizona, in March 2018, highlighting challenges in automated systems.

Uber's self-driving cars had their emergency braking systems turned off to reduce 'bad experiences'.

A pedestrian crossed a road without a crosswalk in dark conditions, leading to the accident.

The self-driving car system had difficulty identifying the pedestrian for six seconds before the incident.

The self-driving system lacked a way to alert the safety operator about the need for emergency braking.

Only 1/6 of IT projects completed on time and on budget in 1994. 1/3 were canceled, and 1/2 had time/cost overruns.

1/3 of IT projects were completed on time and on budget in 2009, a significant improvement from 1994. 1/4 cancelled before completion, and 5/12 had time/cost overruns.

Unconscious bias in product design due to prototypes based on male models, making the product less usable for females.

Women's viewpoints might be overlooked in software development.

AI systems trained on biased data can perform poorly with varied populations— leading to misidentification or incorrect labeling of individuals based on biases in the data.

Facial recognition systems trained on primarily white male data misidentified individuals with a higher rate based on gender and skin tone.

Promises about software quality/performance made by vendors (developers) and their responsibility for software defects.