Operating System Security (CYB 331) PDF

Operating systems security (CYB 331) C H A PT E R 1 Introduction to Operating System Security Dr. Mohamed Abd-Eldayem 1  Operating systems  Operating systems are the software that provides access to the various hardware resources (e.g., CPU, memory, and devices) th...

Operating systems security (CYB 331) C H A PT E R 1 Introduction to Operating System Security Dr. Mohamed Abd-Eldayem 1  Operating systems  Operating systems are the software that provides access to the various hardware resources (e.g., CPU, memory, and devices) that comprise a computer system.  Any program that is run on a computer system has instructions executed by that computer’s CPU, but these programs may also require the use of other peripheral resources of these complex systems. 2  Operating systems  Operating systems run programs in processes.  The challenge for an operating system developer is to permit multiple concurrently executing processes to use these resources in a manner that preserves the independence of these processes while providing fair sharing of these resources. 3  To build any successful operating system  First, the operating system must provide various mechanisms that enable high performance use of computer resources.  Operating systems must provide efficient resource mechanisms, such as file systems, memory management systems, network protocol stacks, etc., that define how processes use the hardware resources.  Second, it is the operating system’s responsibility to switch among the processes fairly, such that the user experiences good performance from each process in concert with access to the computer’s devices.  This second problem is one of scheduling access to computer resources. 4  To build any successful operating system  Third, access to resources should be controlled, such that one process cannot inadvertently or maliciously impact the execution of another.  Is the problem of ensuring the security of all processes run on the system.  Ensuring the secure execution of all processes depends on the correct implementation of resource and scheduling mechanisms.  First, any correct resource mechanism must provide boundaries between its objects and ensure that its operations do not interfere with one another.  Second, scheduling mechanisms must ensure availability of resources to processes to prevent denial of service attacks. 5  SECURE OPERATING SYSTEMS  A secure operating system provides security mechanisms that ensure that the system’s security goals are enforced despite the threats faced by the system.  These security mechanisms are designed to provide such a guarantee in the context of the resource and scheduling mechanisms.  Security goals define the requirements of secure operation for a system for any processes that it may execute  The security mechanisms must ensure these goals regardless of the possible ways that the system may be misused (i.e., is threatened) by attackers. 6  SECURE OPERATING SYSTEMS  The term “secure operating system” is both considered an ideal.  Systems that provide a high degree of assurance in enforcement have been called secure systems, or even more frequently “trusted” systems.  No system of modern complexity is completely secure.  Studying how to build an ideal secure operating system to be useful in assessing operating systems security. 7  SECURITY GOALS  A security goal defines the operations that can be executed by a system while still preventing unauthorized access.  Security goals describe how the system implements accesses to system resources that satisfy the following: secrecy, integrity, and availability.  A system access is traditionally stated in terms of which subjects (e.g., processes and users) can perform which operations (e.g., read and write) on which objects (e.g., files and sockets).  Secrecy requirements limit the objects that individual subjects can read because objects may contain secrets that not all subjects are permitted to know. 8  SECURITY GOALS  Integrity requirements limit the objects that subjects can write because objects may contain information that other subjects depend on for their correct operation.  Some subjects may not be trusted to modify those objects.  Availability requirements limit the system resources (e.g., storage and CPU) that subjects may consume because they may exhaust these resources.  Security goals must be defined and a practical approach for enforcing such goals, that enables the execution of most popular software in reasonable ways, must be identified. 9  TRUST MODEL  System’s trust model defines the set of software and data upon which the system depends for correct enforcement of system security goals.  For an operating system, its trust model is synonymous with the system’s trusted computing base (TCB).  Ideally, a system TCB should consist of the minimal amount of software necessary to enforce the security goals correctly.  The software that must be trusted includes the software that defines the security goals and the software that enforces the security goals (i.e., the operating system’s security mechanism).  Further, software that bootstraps this software must also be trusted.  Thus, an ideal TCB would consist of a bootstrapping mechanism that enables the security goals to be loaded and subsequently enforced for lifetime of the system. 10  TRUST MODEL  In practice, a system TCB consists of a wide variety of software. Fundamentally, the enforcement mechanism is run within the operating system.  Further, a variety of other software running outside the operating system must also be trusted.  For example, the operating system depends on a variety of programs to authenticate the identity of users (e.g., login and SSH). Such programs must be trusted because correct enforcement of security goals depends on correct identification of users.  Also, there are several services that the system must trust to ensure correct enforcement of security goals. For example, windowing systems, such as the X Window System, perform operations on behalf of all processes running on the operating system, and these systems provide mechanisms for sharing that may violate the system’s security goals (e.g., cut-and-paste from one application to another). As a result, the XWindow Systems and a variety of other software must be added to the system’s TCB. 11  Operating system developer  Must prove that their systems have a viable trust model.  This requires that: 1. The system TCB must mediate all security-sensitive operations; 2. Verification of the correctness of the TCB software and its data; 3. Verification that the software’s execution cannot be tampered by processes outside the TCB.  First, identifying the TCB software itself is a nontrivial task for reasons discussed above. 12  Operating system developer  Second, verifying the correctness of TCB software is a complex task.  For general-purpose systems, the amount of TCB software outside the operating system is greater than the operating system software that is impractical to verify formally.  Third, the system must protect the TCB software and its data from modification by processes outside the TCB.  That is, the integrity of the TCB must be protected from the threats to the system, described below.  Otherwise, this software can be tampered, and is no longer trustworthy. 13  THREAT MODEL  A threat model defines a set of operations that an attacker may use to compromise a system.  In this threat model, we assume a powerful attacker who is capable of injecting operations from the network and may be in control of some of the running software on the system (i.e., outside the trusted computing base).  Further, we presume that the attacker is actively working to violate the system security goals.  If an attacker is able to find a vulnerability in the system that provides access to secret information  (i.e., violate secrecy goals) or permits the modification of information that subjects depend on (i.e., violate integrity goals),  then the attacker is said to have compromised the system. 14  The task of a secure operating system developer  Protect the TCB from the types of threats described above.  Protecting the TCB ensures that the system security goals will always be enforced regardless of the behavior of user processes.  Since user processes are untrusted, we cannot depend on them, but we can protect them from threats.  For example, secure operating system can prevent a user process with access to secret data from leaking that data, by limiting the interactions of that process.  However, protecting the TCB is more difficult because it interacts with a variety of untrusted processes. 15  The task of a secure operating system developer  A secure operating system developer must identify such threats, assess their impact on system security, and provide effective countermeasures for such threats.  For example, a trusted computing base component that processes network requests must identify where such untrusted requests are received from the network, determine how such threats can impact the component’s behavior, and provide countermeasures, such as limiting the possible commands and inputs, to protect the component.  The secure operating system developer must ensure that all the components of the trusted computing base prevent such threats correctly. 16  Reference Books  Trent Jaeger. "Operating System Security (Synthesis Lectures on Information Security, Privacy, and Trust), Morgan and Claypool Publishers; 1 edition (October 7, 2008) 17