Critical System Validation Overview

Critical System Validation

• Critical systems are characterized by high reliability, holding this reliability without increasing costs
• Failures can lead to significant economic losses, physical damage, or threats to human life
• The key property of critical systems is dependability, including factors like safety, security, and availability
• Unreliable, unsafe, and insecure systems are unacceptable to users
• System failures can originate from user errors, hardware design flaws, and software coding issues
• Critical system validation is crucial for ensuring reliability, safety, and security, as validation costs are also significant

Critical System Types

• Critical systems are categorized based on their roles and consequences of failure
• Examples include safety, mission, and business systems

Safety-Critical Systems

• Failures in these systems can cause injury or death to humans. This is the primary focus of this module
• Examples: medical devices, aerospace, civil aviation, military aviation, manned space travel, chemical industry, nuclear power stations, traffic control (railways, air traffic, roads, automobiles), and other military equipment.

Mission-Critical Systems

• Failures result in the failure of goal-directed activities
• Examples: navigational system of a space probe

Business-Critical Systems

• Failures lead to business failures
• Examples include customer account systems within banks, online shopping carts, areas requiring secrecy (e.g., defense and secret services), sensitive company areas, police records, customer data administration, and student records administration

Critical System Validation Perspectives

• Reliability validation measures how frequently the system fails its specified service. A special dataset is needed that is not used during the initial testing. Measurable reliability needs verification against specifications, and ensuring it's sufficient for user needs. Safety validation verifies the system does not have failures with serious consequences
• Security validation protects the system and its data against internal and external attacks

Critical System Validation Techniques

• Static techniques evaluate code quality without running it
  • Design reviews
  • Documentation checks
  • Mathematical proofs to support validation
• Dynamic techniques analyze system behavior during execution
  • Statistical testing
  • Runtime checking
  • Scenario testing
• Process validation minimizes process errors affecting system dependability through good design and development

Safety Validation

• Proving system safety through testing can be difficult and expensive
• Static methods, like checking for correctness, are needed to identify potential unsafe situations and prevent them
• Reviewing intended function, structures, algorithms, and data structures, ensuring consistency, and evaluating testing efficiency are key static validation steps
• Design principles such as system simplicity and effective software development techniques contribute to safety
• Hazard analysis, a key process, helps assess risk by identifying, analyzing, and addressing potential hazards through means such as hazard avoidance, identification and removal, and damage limitation

Safety Case of the System

• Demonstrating and justifying the safety of a system, crucial for transportation, medical, and other critical industries, requires evidence specific to its environment
• Safety features should be continually updated to maintain effectiveness

Gas Warning System Example

• A system that warns of toxic gases with a sensor, controller, and alarm
• Different danger levels determine actions; low level alerts reduce gas levels and high level triggers evacuation

Gas Sensor Control (Example Code and Logic)

• Detailed example code with loops and conditional statements to continuously monitor and report on gas levels
• Gas levels compared with warning and danger thresholds drive actions

Graphical Argument

• Demonstrates the logic for responding to different gas levels by showing paths and how they should lead to specific actions or warnings.

Condition Checking

• A table for checking the code logic in relation to the gas sensor example and verifying the accuracy of logic to respond to different gas levels. Identifying contradictions in the code logic.

Security Validation

• Security focuses on protecting the system and its data against attacks
• Validation can use static and dynamic analysis methods, or experience based validation from teams with high-level expertise
• Techniques include automatic validation through system tools and assessing the system's ability to handle potential attacks

Security Validation Methods

• Experience-based validation: Expert teams evaluate system vulnerabilities
• Tool-based validation: Security tools analyze the system for vulnerabilities
• Tiger teams: Specialized teams simulate attacks to identify system vulnerabilities

Safety vs. Security

• Safety addresses unplanned events and protecting human lives
• Security is focused on planned events, such as attacks, and Protecting the system, data and information, and includes internal and external sources

Reliability Validation

• Assessing the system's reliability determines if it can consistently perform its tasks as expected
• Dynamic techniques involve running the system then observing failures and their time to evaluate reliability
• Statistical tests measure error count from a defined dataset to determine reliability and failure situations
• Identifying threshold levels of failures help define an acceptably reliable system
• A reliability validation process includes establishing an operational profile, building a testing dataset, executing the system, observing failures, and calculating reliability after a statistically significant number of failures

Problems in Reliability Validation

• Establishing an accurate operational profile that matches actual usage can be challenging
• Comprehensive testing of every possible scenario is often cost-prohibitive
• Statistical uncertainty in the dataset makes truly accurate reliability calculations difficult

Formal Method and Validation

• Formal methods improve system quality by finding and mitigating errors via mathematical analysis of the system specification
• Formal methods are computationally costly but can be used for complex systems
• Formal verification methods and complete test case generation are important applications of formal methods
• Mathematical methods are used for system verification and validation

Validation Considerations

• Human judgment remains important for validating simple systems
• Formal methods offer support for complex systems but may be cost-prohibitive
• Formal methods help in verification, checking for desired system behaviors and safety.
• Problems with formal methods include their complexity, difficulty in understanding by experts, and a cost of use that increases with complexity and system size.

Safety Proofs

• Safety proofs aim to demonstrate the system will not reach unsafe states
• Proof by contradiction is used to analyze system behavior, utilizing diagrams