Computer Security: Isolation and Confinement

Computer Security: Isolation

• Isolation is a confinement principle in computer security.
• Isolation aims to prevent a misbehaving application from harming the rest of the system.

Running Untrusted Code

• Often need to run untrusted code (buggy code, from untrusted sites):
  • Programs from untrusted Internet sites (mobile apps, JavaScript, browser extensions)
  • Exposed applications (browser, PDF viewer, Outlook)
  • Legacy daemons (sendmail, bind)
  • Honeypots
• Goal: if an application misbehaves, kill it.

Approach: Confinement

• Confinement ensures a misbehaving application cannot harm the rest of the system.
• Can be implemented at multiple levels:
  • Hardware: Run application on isolated hardware (air gap). Difficult to manage.
  • Virtual machines: Isolate OS's on a single machine. The virtual machine monitor (hypervisor) manages the isolation.
  • Process (System Call Interposition): Isolate a process within a single operating system. This uses containers (containers isolate process).
  • Threads: Software Fault Isolation (SFI) isolates threads sharing the same address space.
  • Application level: Example of browser sandbox for JavaScript and WebAssembly.

Implementing Confinement

• Key component: Reference monitor.
  • Mediates requests from applications.
  • Enforces confinement.
  • Implements a specified protection policy.
• Must always be invoked: Every application request must be mediated.
• Tamperproof: Reference monitor cannot be killed; or if it is, the monitored process is also killed.

Example: Chroot

• To use chroot (must be root):
  • chroot /tmp/guest
  • su guest
• Adds /tmp/guest to every file system access.
• Prevents the application (e.g., web server) from accessing files outside of the jail.

Escaping from Jails

• Early escapes use relative paths to access files outside of the chroot jail.
• Chroot should be executable only by root. Otherwise, the jailed app can:
  • Create a dummy file.
  • Run chroot /aaa
  • Run su root to become root (bug).

Many Ways to Escape Jails as Root

• Create device to access raw disk.
• Send signals to non-chrooted processes.
• Reboot system.
• Bind to privileged ports.

Freebsd Jail

• Stronger than simple chroot.
• Executes with jail jail-path hostname IP-addr cmd
  • Runs hardened chroot (no "../../" escape).
  • Only binds to sockets with specified IP address and allowed ports.
  • Communicates only with processes inside the jail.
  • Limits root.

Problems with Chroot and Jail

• Coarse policies (all or nothing).
• Inappropriate for applications like web browsers (needs read access outside jail).
• Does not stop malicious apps from network access or host OS crashes.

System Call Interposition

• Sandboxing a process.

System Call Interposition Details

• Observation: To damage the host system (e.g., persistent changes), the application must make system calls.
  • To delete/overwrite files: unlink, open, write.
  • To do network attacks: socket, bind, connect, send.
• Idea: Monitor application's system calls and block unauthorized calls.
• Implementation options:
  • Completely kernel space (e.g., Linux seccomp).
  • Completely user space (e.g., program shepherding).
  • Hybrid (e.g., Systrace).

Early Implementation (Janus)

• Linux ptrace: process tracing on process calls (ptrace(..., pid_t pid,...)).
• Monitors calls and kills the application if the request is disallowed.

Example Policy

• Sample policy file (e.g., for PDF reader):
  • path allow /tmp/*
  • path deny /etc/passwd
  • network deny all
• Manually specifying can be difficult. Recommended default policies are available and can be made more restrictive as needed.

Complications

• If an application forks, the monitor must also fork.
• If the monitor crashes, the application must be killed.
• The monitor must maintain all OS state associated with the application.
  • Current working directory (CWD), UID, EUID, GID.
• When an app uses cd path, the monitor must update its CWD, otherwise relative paths will be interpreted incorrectly.

Problems with Ptrace

• Ptrace traces all system calls or none.
• Inefficient as it traces unnecessary calls.
• Cannot abort system calls without killing the application.
• Race conditions. Example: symlink (TOCTOU).

SCI in Linux: seccomp-BPF

• Linux kernel facility to filter process system calls.
• System call filters written in BPF language (using the BPF compiler).
• Used in Chromium, Docker containers.

BPF Filters (Policy Programs)

• Processes can install multiple BPF filters (all run on every system call).
• Filters cannot be removed after installation.
• Children inherit filters from parent process.
• execve erases all filters.
• Filter input: syscall number, syscall args, architecture.
• Filter returns one of:
  • SECCOMP_RET_KILL: Kills the process.
  • SECCOMP_RET_ERRNO: Returns specified error to the caller.
  • SECCOMP_RET_ALLOW: Allows the syscall.

Installing a BPF Filter

• Must be called before setting BPF filter to ensure the attacker cannot ignore or elevate privileges on subsequent execve.

Docker: Isolating Containers using seccomp-bpf

• Docker isolates containers at the process level.
• Prevents containers from making system calls filtered by seccomp-bpf.
• Allows specifying BPF policies to block many system calls, including ptrace.

Docker SysCall Filtering

• Example using a filter called filter.json

"defaultAction": "SCMP_ACT_ERRNO",
"syscalls": [
    {
        "names": ["accept"],
        "action": "SCMP_ACT_ALLOW",
        "args": []
    },
    ...
]

More Docker Confinement Flags

• Specify as an unprivileged user: docker run --user www nginx
• Limit Linux capabilities: docker run --cap-drop all --cap-add NET_BIND_SERVICE nginx
• Prevent process from becoming privileged: docker run --security-opt=no-new-privileges:true nginx
• Limit resources: docker run --restart=on-failure=<max-retries> --ulimit nofile=<max-fd> --ulimit nproc=<max-proc> nginx

Confinement Via Virtual Machines

• Virtual machines isolate applications within a single hardware platform.
• The virtual machine monitor (VMM, hypervisor) manages the isolation.

Why VMs are Popular

• In the 1960s, VMs allowed multiple users to share a single computer when computers were scarce.
• In the 2000s, availability of many machines makes VM usage common for web, file, print servers, and other applications, in private and public clouds.

Hypervisor Security Assumptions

• Malware can infect guest OS and guest applications.
• But malware cannot escape from the infected VM.
• Cannot infect the host OS or other VMs on the same hardware.

Problem: Covert Channels

• Covert channel: Unintended communication channel between isolated components.
• Can leak classified data from a secure component to a public component.
• Example: Malware using time variations (CPU intensive calculations).

VM Isolation in Practice: Cloud

• VMs can run on the same machine.

VM Isolation in Practice: End-User

• Qubes OS is an example where everything runs as a VM.

Hypervisor Detection

• Can an OS detect whether it is running on a hypervisor with different applications detecting it and refusing to run in a VM to avoid reverse engineering or to protect code, including DRM.

Hypervisor Detection (Red Pill Techniques)

• VM platforms emulate simple hardware. Examples: VMWare emulating older system hardware.
• Hypervisor introduces variances in time latencies and memory cache behavior, leading to relative time variations.
• Hypervisor shares the TLB (translation lookaside buffer) with GuestOS.
• GuestOS can detect a reduced TLB size, indicating a hypervisor could be present.

Hypervisor Detection in the Browser

• Can webpages detect if they are running in a VM through varying timing performance of tasks such as UI rendering in a VM.

Hypervisor Detection (Bottom Line)

• The perfect hypervisor does not exist.
• Focus on compatibility (software) and performance (overhead).
• Anomalies may reveal hypervisor existence.

Software Fault Isolation (SFI)

• Goal is to confine applications sharing an address space.
  • Kernel modules should not corrupt the kernel.
  • Native libraries should not corrupt other components in memory like the Java virtual machine (JVM).
• Simple solution: Runs apps in separate address spaces but this may be inefficient if frequent communication between apps.
• Requires context switching per message if frequent communication is needed.
• Approches include:
  • Code and data segments.
  • Unsafe instructions (jmp, load, store).
  • At compile time, guards are added to prevent unsafe instructions.
  • When loading code, ensure guards are also loaded.
• Segment matching and address sandboxing techniques are techniques used to identify unsafe code/data from another domain.
• Problem: Jump instructions can bypass load guards.

SFI Summary

• Performance is generally good.
• SFI implementation is more difficult in other architectures like x86 because they have varying instruction length.

Confinement Summary

• Many sandboxing techniques exist: physical air gap, virtual air gap (hypervisor), System Call Interposition (SCI), Software Fault Isolation (SFI), and application-specific techniques.
• Complete isolation is often inappropriate because apps need to communicate through regulated interfaces.
• The hardest part of sandboxing is specifying the policy for what an app can and cannot do. Preventing covert channels to be exploited is another main aspect of security in sandboxing.

This quiz covers the principles of isolation in computer security, exploring how confinement can prevent misbehaving applications from causing harm to a system. It discusses various methods, including hardware isolation, virtual machines, and process isolation. Test your knowledge on the approach to running untrusted code safely.