Safety Engineering PDF

Safety Engineering ================== Safety in software systems is achieved by developing an understanding of the situations that might lead to safety-related failures. The software is engineered so that such failures do not occur. You might therefore think that a safety-critical system is as simp...

Safety Engineering ================== Safety in software systems is achieved by developing an understanding of the situations that might lead to safety-related failures. The software is engineered so that such failures do not occur. You might therefore think that a safety-critical system is as simple as that. System reliability is necessary for safety achievement, but it isn't enough 12.1 Safety-Critical System =========================== \- Safety-critical systems are systems in which it is essential that system operation is always safe. That is, the system should never damage people or the system's environment, irrespective of whether or not the system conforms to its specification. ***Safety-critical software falls into two classes:*** 1. ***Primary safety-critical software.*** This is software that is embedded as a controller in a system. Malfunctioning of such software can cause a hardware malfunction, which results in human injury or environmental damage. 2. ***Secondary safety-critical software.*** This software can indirectly result in injury. An example of such software is a computer-aided engineering design system whose malfunctioning might result in a design fault in the object being designed. This fault may cause injury to people if the designed system malfunctions. A ***hazard*** is a system state that could lead to an accident 1. ***Hazard avoidance.*** The system is designed so that hazards are avoided. 2. ***Hazard detection and removal.*** The system is designed so that hazards are detected and removed before they result in an accident. 3. ***Damage limitation.*** The system may include a protection feature that minimize the damage that may result from an accident. Safety Terminology 12.2 Safety requirements ------------------------ Safety requirements are functional requirements, which define checking and recovery facilities that should be included in the system and features that provide protection against system failures and external attacks. Four(4) Activities in a hazard-driven safety specification process: 1. ***Hazard identification**.* The hazard identification process identifies hazards that may threaten the system. These hazards may b recorded in a hazard register 2. ***Hazard assessment**.* The hazard assessment process decides which hazards are the most dangerous and/or the most likely to occur. 3. ***Hazard analysis**.* This is a process of root-cause analysis that identifies the events that can lead to the occurrence of a hazard. 4. ***Risk reduction**.* This process is based on the outcome of hazard analysis and leads to identification of safety requirements. These requirements may be concerned with ensuring that a hazard does not arise or lead to an accident or that if an accident does occur, the associated damage is minimized. 12.2.2 Hazard assessment ------------------------ The hazard assessment process focuses on understanding the factors that lead to the occurrence of a hazard and the consequences if and accident or incident associated with the hazard should occur There are three risk categories that are used in hazard assessment: 1. ***Intolerable risk*** in safety-critical systems are those that threaten human life. The system must be designed so that such hazards either cannot arise or, that if they do, features in the system will ensure that they are detected before they cause and accident 2. ***As low as reasonably practical (ALARP)*** risk are those that have less serious consequences or that are serious but have a very low probability of an accident arising because of a hazard minimized, subject to other consideration such as cost and delivery. 3. ***Acceptable risk*** are those where the associated accidents normally result in minor damage. e. System designers should take all possible steps to reduce "acceptable" risks, if these measures do not significantly increase costs, delivery time, or other non-functional system attributes 12.2.3 Hazard analysis ---------------------- Hazard analysis is the process of discovering the root causes of hazards in a safety-critical system. Hazard technique analysis most commonly used is *Fault tree analysis\ * 12.2.4 Risk reduction --------------------- Once potential risks and their root causes have been identified, you are then able to derive safety requirements that manage the risks and ensure that incidents or accidents do not occur. You can use three possible strategies: 1. ***Hazard avoidance***, where a system is designed so that the hazard cannot occur. 2. ***Damage limitation***, where a system is designed so that the consequences of an accident is minimized In the insulin delivery system, a safe state is a shutdown state where no insulin is injected. Over a short period, this is not a threat to the diabetic's health. For the software failures that could lead to an incorrect dose of insulin, the following "solutions" might be developed: 1. ***Arithmetic error.*** This error may occur when an arithmetic computation causes a representation failure 2. ***Algorithm error*** This is a more difficult situation as there is no clear program exception that must be handled. This type of error could be detected by comparing the required insulin dose computed with the previously delivered dose. If it is much higher, this may mean that the amount has been computed incorrectly 12.3 Safety engineering processes ================================= The software processes used to develop safety-critical software are based on the processes used in software reliability engineering. In general, a great deal of care is taken in developing a complete, and often very detailed, system specification. The design and implementation of the system usual follow a plan-based, waterfall model, with reviews and checks at each stage in the process. Fault avoidance and fault detection are the drivers of the process Safety engineers work with quality managers to ensure that a detailed configuration management system is used to track all safety-related documentation and keep it in step with the associated technical documentation ### 12.3.1 Safety assurance processes Safety assurance is a set of activities that check that a system will operate safely. Specific safety assurance activities should be included at all stages in the software development process Examples of safety assurance activities are 1. ***Hazard analysis and monitoring,*** where hazards are traced from preliminary hazard analysis through to testing and system validation. 2\. **Safety *reviews***, which are used throughout the development process. 3\. ***Safety certification***, where the safety of critical components is formally certified ### 12.3.2 Formal verification Formal methods of software development, rely on a formal model of the system that serves as a system specification. These formal methods are mainly concerned with mathematically analyzing the specification; with transforming the specification to a more detailed, semantically equivalent representation; or with formally verifying that one representation of the system is semantically equivalent to another representation. Formally verifying programs using a deductive approach is difficult and expensive, but alternative approaches to formal analysis have been developed that are based on a more restricted notion of correctness. ### 12.3.3 Model checking A diagram of a model building Description automatically generated *Model checking* involves creating a formal state model of a system and checking the correctness of that model using specialized software tools ### 12.3.4 Static program analysis Automated static analyzers are software tools that scan sources text of a program and detect possible faults and anomalies. They parse the program text and thus recognize the different types of statements in a program. They can then detect whether or not statements are well formed, make interference about the control flow in the program, and in many cases, compute the set of all possible values for program data.  Three(3) levels of checking may be implemented in static analyzers: 1. ***Characteristics error checking*.** At this level, the static analyzer knows about common errors that are made by programmers in languages such as Java or C 2. ***User-defined error checking*.** In this approach, the users of the static analyzer define error patters to be detected. These may relate to the application domain or may be based on knowledge of the specific system that is being developed. 3. ***Assertion checking.*** This is the most general and most powerful approach to static analysis. Developers include formal assertions (often written as stylized comments) in their program that state relationship that must hold at that point in a program. 12.4 Safety cases ----------------- A safety case refers to a system as a whole, and , as part of that case, there may be a subsidiary software safety case. When constructing a software safety case, you have to relate software failures to wider system failures and demonstrate either that these software failures will not occur or that they will not be propagated in such a way that dangerous system failures may occur. *A documented body of evidence that provides a convincing and valid argument that a system is adequately safe for a given application in a given environment†.* A screenshot of a computer Description automatically generated ### 12.4.1 Structured arguments  The decision on whether a system is operationally safe should be based on logical arguments. These arguments should demonstrate that the evidence present supports the claims about a system's security and dependability. Arguments in a safety case are usually presented as "claim based" arguments. Some claim about system safety is made, and, based on available evidence, an argument is presented as to why that claim holds. ### 12.4.2 Software safety arguments Software safety arguments are a type of structured argument which demonstrates that a program works as intended. It is only necessary to show that program execution cannot result in it reaching a potentially unsafe state Security Engineering ==================== Software Engineers -- designing and implementing systems that were secure As more and more systems were connected to the internet, a variety of different external attacks were devised to threaten these systems. The problem of producing dependable systems were hugely increased. Systems engineers had to consider threats from malicious and technically skilled attackers as well as problems resulting from accidental mistakes in the development process. Three(3) Security dimension: 1. ***Confidentiality.*** Information in a system may be disclosed or made accessible to people or programs that are not authorized to have access to that information. 2. ***Integrity.*** Information in a system may be damaged or corrupted, making it unusual or unreliable. 3. ***Availability.*** Access to a system or its data that is normally available may not be possible. From an organizational perspective, security has to be considered at three(3) levels: 1. ***Infrastructure security,* which is concerned with maintaining the security of all systems and network that provide an infrastructure and a set of shared services to the organization.** 2. ***Application security,*** which is concerned with the security of individual application systems or related groups of systems. 3. ***Operational security,*** which is concerned with the secure operation and use of the organization's systems. A screenshot of a computer Description automatically generated The term ***cybersecurity*** is now commonly used in discussion of system security. 13.1 Security and dependability ------------------------------- Security is a system attribute that reflects the ability of the system to protect itself from malicious internal or external attacks. These external attacks are possible because most computers and mobile devices are networked and are therefore accessible by outsiders. Security is the most important system dependability attribute.  A screenshot of a computer error message Description automatically generated Four types of security threats may arise: 1. ***Interception threats*** that allow an attacker to again access to an asset. 2. ***Interruption threats*** that allow an attacker to make part of the system unavailable. 3. ***Modification threats*** that allow an attacker to tamper with a system asset. 4. ***Fabrication threats*** that allow an attacker to insert false information into a system. The controls that you might put in place to enhance system security are based on the fundamental notions of avoidance, detection, and recovery: 1. ***Vulnerability avoidance.*** Controls that are intended to ensure that attacks are unsuccessful. The strategy here is to design the system so that security problems are avoided. 2. ***Attack detection and neutralization.*** Controls that are intended to detect and repel attacks. 3. ***Exposure limitation and recovery.*** Controls that support recovery from problems. Security is closely related to the other dependability attributes of reliability, availability, safety, and resilience: 1. **Security and reliability** 2. **Security and availability** 3. **Security and safety** 4. **Security and Resilience** 13.2 Security and Organization ------------------------------ Companies assess the risks and losses that may arise from certain types of attacks on system assets. They may then decide that it is cheaper to accept these risks rather than build a secure system that can deter or repel the external attacks Security risk management is therefore a business rather than a technical issue. It has to take into account the financial and reputational losses from a successful system attack as well as the costs of security procedures and technologies that may reduce these losses For risk management to be effective, organizations should have a documented information security policy that sets out: 1. ***The assets that must be protected*** 2. ***The level of protection that is required for different types of assets*** 3. ***The responsibilities of individual users, managers, and the organization*** 4. ***Existing security procedures and technologies that should be maintained*** ### 13.2.1 Security risk assessment Security risk assessment and management are organizational activities that focus on identifying and understanding the risk to information assets (system and data) in the organization. Stages of Risk Assessment: 1. ***Preliminary risk assessment***. The aim of this initial risk assessment is to identify generic risks that are applicable to the system and to decide if an adequate level of security can be achieved at a reasonable cost. At this stage, decisions on the detailed system requirements, the system design, or the implementation technology have not been made. 2. ***Design risk assessment.*** This risk assessment takes place during the system development life cycle and is informed by the technical system design and implementation decisions 3. ***Operational risk assessment.*** This risk assessment process focuses on the use of the system and the possible risks that can arise. 13.3 Security Requirement ------------------------- The specification of security requirements for systems has much in common with the specification of safety requirements. You cannot specify safety or security requirements as probabilities. Like safety requirements, security requirements are often "shall not" requirements that define unacceptable system behavior rather than required system functionality. 10 types of security requirements that may be included in a system specification: 1. ***Identification requirements*** specify whether or not a system should identify its users before interacting with them. 2. ***Authentication requirements*** specify how users are identified. 3. ***Authorization requirements*** specify the privileges and access permissions of identified users. 4. ***Immunity requirements*** specify how a system should protect itself against viruses, worms, and similar threats. 5. ***Integrity requirements*** specify how data corruption can be avoided. 6. ***Intrusion detection requirements*** specify what mechanism should be used to detect attacks on the system. 7. ***Nonrepudiation requirements*** specify that a party in a transaction cannot deny its involvement in that transaction. 8. ***Privacy requirements*** specify how data privacy is to be maintained. 9. ***Security auditing requirements*** specify how system use can be audited and checked. 10. ***System maintenance requirements*** specify how an application can prevent authorized change from accidentally defeating its security mechanism  A risk-driven security requirements process is shown in Figure 13.5. The process stages are: 1. ***Asset identification,*** where the system assets that may require protection are identified. The system itself or system functions may be identified as assets as well as the data associated with the system. 2. ***Asset value assessment,*** where you estimate the value of the identified assets. 3. ***Exposure assessment,*** where you assess the potential losses associated with each asset. This process should consider direct losses such as the theft of information, the cost of recovery, and the possible loss of reputation. 4. ***Threat identification,*** where you identify the threats to system assets. 5. ***Attack assessment,*** where you decompose each threat into attacks that might be made on the system and the possible ways in which these attacks may occur. 6. ***Control Identification,*** where you propose the controls, the might be put in place to protect an asset. 7. ***Feasibility assessment,*** where you assess the technical feasibility and the cost of the proposed controls. 8. ***Security requirement definition,*** where knowledge of the exposure, threats and control assessments is used to derive system security requirements. ### 13.3.1 Misuse cases Misuse cases are scenarios that represent malicious interaction with a system. You can use these scenarios to discuss and identify possible threats and, therefore also determine the system's security requirements. Misuse cases are associated with use case instance and represents threats or attacks associated with these use cases. Misuse cases can be used as part of the security requirements engineering process, but you also need to consider risks that are associated with system stakeholders who do not interact directly with the system. 13.4 Secure System design ------------------------- **Secure system design** means incorporating security measures into a system from the beginning to protect it from threats, ensuring confidentiality, integrity, and availability while balancing performance and usability. Secure system design must be considered from the start, as adding security after implementation is difficult. Key aspects include: 1. **Architectural Design** -- Security decisions at the architecture level impact overall system protection. 2. **Good Practices** -- Following established security guidelines improves system resilience. ### 13.4.1 Design risk assessment **Design risk assessment** is the process of evaluating security risks during system design. As design decisions are made, new risks emerge, requiring adjustments to security requirements. This process is continuous, refining the design until the remaining risks are acceptable. It helps identify vulnerabilities and implement mitigations, such as using multifactor authentication to reduce password-related risks. (Figure 13.10). Preliminary design decisions are made, and the risks associated with these decisions are assessed. This assessment may lead to new requirements to mitigate the risks that have been identified or design changes to reduce these risks. As the system design evolves and is developed in more detail, the risks are reassessed and the results are fed back to the system designers. The design risk assessment process ends when the design is complete and the remaining risks are acceptable A diagram of a project Description automatically generated with medium confidence Figure 13.11 is a model of the design risk assessment process. The key difference between preliminary risk analysis and design risk assessment is that, at the design stage, you now have information about information representation and distribution and the database organization for the high-level assets that have to be protected ### 13.4.2 Architectural design Software architecture design decisions can have profound effects on the emergent properties of a software system. If an inappropriate architecture is used, it may be very difficult to maintain the confidentiality and integrity of information in the system or to guarantee a required level of system availability In designing a system architecture that maintains security, you need to consider two fundamental issues: 1\. Protection---how should the system be organized so that critical assets can be protected against external attack? 2\. Distribution---how should system assets be distributed so that the consequences of a successful attack are minimized?  To access and modify patient records, an attacker has to penetrate three system layers: 1. ***Platform-level protection***. This usually involves a user signing-on to a particular computer. The platform will also normally include support for maintaining the integrity of files on the system, backups, and so on. 2. ***Application-level protection***. The next protection level is built into the application itself. It involves a user accessing the application, being authenticated, and getting authorization to take actions such as viewing or modifying data. Application-specific integrity management support may be available. 3. ***Record-level protection.*** This level is invoked when access to specific records is required, and involves checking that a user is authorized to carry out the requested operations on that record. Protection at this level might also involve encryption to ensure that records cannot be browsed using a file browser. ### 13.4.3 Design guidelines Some general guidelines have wide applicability when designing system security solutions. These guidelines encapsulate good design practice for secure systems engineering. *10 Design Guidelines* - *Guideline 1: Base security decisions on an explicit security policy* - *Guideline 2: Use defense in depth* - *Guideline 3: Fail securely* - *Guideline 4: Balance security and usability* - *Guideline 5: Log user actions* - *Guideline 6: Use redundancy and diversity to reduce risk* - *Guideline 7: Specify the format of system inputs* - *Guideline 8: Compartmentalize your assets* - *Guideline 9: Design for deployment* - *Guideline 10: Design for recovery* ### 13.4.4 Secure systems programming Secure system design means designing security into an application system. Secure system programming means writing software with security in mind to prevent vulnerabilities that attackers can exploit. It involves using safe coding practices, choosing secure programming languages or techniques, and ensuring reliability to enhance security. 13.5 Security testing and assurance ----------------------------------- Security testing and assurance is the process of evaluating a system's ability to withstand attacks. It is challenging because security requirements define what should not happen, making them difficult to test fully. Attackers actively seek vulnerabilities, often using unexpected methods. While security testing cannot guarantee complete protection, it helps identify and reduce risks. You may use a combination of testing, tool-based analysis, and formal verification to check and analyze the security of an application system: 1. ***Experience-based testing***. In this case, the system is analyzed against types of attack that are known to the validation team. This may involve developing test cases or examining the source code of a system. 2. ***Penetration testing.*** This is a form of experience-based testing where it is possible to draw on experience from outside the development team to test an application system. 3. ***Tool-based analysis.*** In this approach, security tools such as password checkers are used to analyze the system. Password checkers detect insecure passwords such as common names or strings of consecutive letters. This approach is really an extension of experience-based validation, where experience of security flaws is embodied in the tools used. 4. ***Formal verification.*** Essentially, this involves making formal, mathematical arguments that demonstrate that a program conforms to its specification.