OS Forensics Study Notes PDF

STUDY NOTES 1. Forensics Overview Forensics Overview Definition of Digital Forensics: Digital forensics is the process of using scientifically derived and proven methods to recover, analyse, and present data for use in a court of law. It involves a systematic process to maintain the integrity an...

STUDY NOTES 1. Forensics Overview Forensics Overview Definition of Digital Forensics: Digital forensics is the process of using scientifically derived and proven methods to recover, analyse, and present data for use in a court of law. It involves a systematic process to maintain the integrity and reliability of evidence. Primary Objectives: Preservation: Ensuring data is protected and maintained in its original form to prevent tampering or loss. Collection: Acquiring data from multiple sources while keeping the chain of custody intact. Examination: Using various tools and techniques to examine and identify relevant data. Analysis: Interpreting the extracted data to reconstruct events or answer investigative questions. Reporting: Summarising findings in a clear, organised, and legally acceptable manner. Locard's Exchange Principle: Emphasises that every interaction leaves a trace, forming the basis for forensic analysis. In digital forensics, this could mean evidence left in log files, metadata, or deleted files that leave behind Access to original data should be performed only by competent professionals. An audit trail should be kept of all processes applied to digital evidence.residual data. Acquisitions methods: Integrity Findings frequently used in legal proceedings – must be robust! ACPO Principles – Evidence collection and preservation Scientific approach Why are you collecting/analysing it like that? Are the tools sound? ▪ Chain of Custody Can we prove that the evidence hasn’t been tampered with? Evidence bags - Hashes Hashing - Fingerprint ACPO Principles (Association of Chief Police Officers Guidelines): These principles guide the handling of digital evidence to ensure admissibility in court. They outline that: 1. No action should alter original evidence. 2. 3. Those responsible for investigations should follow established policies and procedures. Artefacts, Evidence and Argument Evidence-Based Arguments Scientific and Technical endeavours rely on evidence to make claims ▪ Chain smaller claims together to make larger assertions ▪ Claim: The subject intended to download and view video A. ▪ Evidence: What artefacts would help demonstrate intent? What artefacts would help us demonstrate downloading? What artefacts would help us demonstrate viewing? ▪ Perpetual problem with Digital forensics: Attribution Usually not as simple as finding specific artefacts E.g. In an office setting, can we prove Employee X was the one who logged in to the machine? Types of Investigation Context, investigative questions, evidence types, all change with the investigation archetype. ▪ Single machine? Multiple devices? An Entire network? ▪ Correlations between devices may be important. Even if only a single machine, we may have additional information/intelligence to corroborate. ▪ Artefact types: Files, logs, network traces? “Logical” or “physical” data? ▪ Purpose of investigation: DFIR – Corporate breach/incident – network based initially, host-based later on.Ongoing/active attacker. CSAM – Illegal media centric – Attribution and source identification. Communications. Malware – Attack vector, malware behaviour, OS manipulations, backdoors/persistence. Embezzlement – Emails, spreadsheets, inappropriate file access. Strange access patterns? Murder or other “meat space” crimes – Cyber-enabled, corroboration of activity, motive,intent/planning. ▪ Types of device: PC, Mobile, Server, Smart-home, IoT (may include CCTV, for example) Investigative Approach Data Reduction - Narrowing scope Forensics Investigations are large Terabyte drives are not rarity Need a process/strategy to focus and reduce There are general approaches, but pivot based on the needs of the investigation. ○ i.e., the purpose of the investigation ▪ What sort of things? Text search Hash lookups (known images/files) Recently Used / Most Used ▪ Use initial findings to pursue further investigation Preview Approach for Illegal Media on Windows (Police Scotland) Under an hour, ideally 1. Begin keyword search of MFT (filesystem index) 2. Filename filters/conditions (indicative names) and preview Filesystem index search, followed by fetching relevant files for preview 3. Keyword search on User or Appdata folders (if User is big). Wait for keyword search to finish first 4. Windows Thumbnail Cache processing 5. Browse images in Recycle Bin and User Profile (while waiting on Thumbcache processing 6. Extract and browse images in browser caches At this point it’s very likely that something has been found if it’s there. Continue to check in case anti-forensics has been used (or there are more subtle indicators) 7. Keyword search on pagefile.sys, hiberfil.sys, unallocated space, volume shadow copies. (keep running until triage is over) 8. Analyse WebCacheV01.dat for indicative access. (IE and Explorer) 9. Examine shortcut (link files) and jump list files 10. Examine Registry and NTUSER.DAT (regripper) RecentDocs, shellbags, etc. 11. Further examination of Web Browsers (SQLite) 12. Follow-up on any leads, original intelligence, peer-to-peer, communications, etc. 2. FAT File System File Allocation Table (FAT) Overview History and Use: ○ FAT (File Allocation Table) was introduced in the 1970s and is commonly used in smaller storage systems like USB drives and SD cards. Variants of FAT include FAT12, FAT16, and FAT32, each supporting progressively larger volumes and file sizes. FAT Versions ○ FAT12: Represents cluster numbers with 12 bits, allowing 4,096 clusters.2 ○ FAT16: Uses 16 bits for cluster numbers, enabling 65,536 clusters.2 ○ FAT32: Employs 28 bits for cluster representation, accommodating 268,435,456 clusters and introducing changes to the system area like the volume boot record and root directory location. Structure of the FAT Filesystem: SYSTEM AREA ○ Boot Sector: Contains boot code and basic volume information  Located at the beginning, contains essential filesystem parameters like bytes per sector and sectors per cluster. Contains Boot Code Contains basic info about how to interpret the volume The first sector of the FAT volume contains essential data structures to help the operating system manage the filesystem. Key Fields include bytes per sector, sectors per cluster, and number of FATs, which are crucial for interpreting the volume.  Calculation Cluster 2 (𝑅𝑒𝑠𝑒𝑟𝑣𝑒𝑑 𝑠𝑒𝑐𝑡𝑜𝑟𝑠) + (𝐹𝐴𝑇 𝑇𝑎𝑏𝑙𝑒𝑠) + (𝑅𝑜𝑜𝑡 𝐷𝑖𝑟𝑒𝑐𝑡𝑜𝑟𝑦) (𝑅𝑒𝑠𝑒𝑟𝑣𝑒𝑑 𝑆𝑒𝑐𝑡𝑜𝑟) + ( 𝑛𝑜 𝑜𝑓 𝐹𝐴𝑇) * (𝑆𝑒𝑐𝑡𝑜𝑟𝑠 𝑝𝑒𝑟 𝐹𝐴𝑇) + (𝑅𝑜𝑜𝑡 𝐸𝑛𝑡𝑟𝑖𝑒𝑠 ÷ 𝐵𝑦𝑡𝑒𝑠 𝑝𝑒𝑟 𝑠𝑒𝑐𝑡𝑜𝑟) * (𝑆𝑖𝑧𝑒 𝑜𝑓 𝑑𝑖𝑟 𝑒𝑛𝑡𝑟𝑦) ○ File Allocation Table (FAT): Tracks each cluster's status, indicating if it’s free, in use, or damaged. Different FAT types use varying sizes (12-bit, 16-bit, or 32-bit entries). Located immediately after the boot sector, the FAT keeps track of each cluster's status (e.g., free, reserved, or bad). Each file entry in the FAT points to clusters that hold file data, enabling the system to follow cluster chains for files larger than one cluster. FAT Table 1: Maintains a list of clusters for files and free space.34 FAT Table n: Backup copy(ies) of the FAT Table. DATA AREA ○ Root Directory:The volume's root folder. In FAT12 and FAT16, the root directory has a fixed location and maximum number of entries. FAT32 stores it within the data area, allowing for flexible size. Holds metadata for files and directories, including file attributes and start cluster. ○ Data Area:Houses all file data and other folders, starting from Cluster 2. The bulk of the FAT volume, where file and directory data are stored in clusters. Stores actual file data, organised into clusters Key Components Partition Boot Record (PBR): Located in the first sector of the FAT volume, the PBR includes boot code and details for interpreting the volume.7 Specific fields within the PBR, such as OEM Name, Bytes per Sector, and Volume Serial Number, can be found at specific offsets.56789 Directory Entries: Each file or folder has a corresponding directory entry within its parent folder, storing its metadata.6 Entries include fields for file name, extension, attributes, creation time, modification time, and starting cluster number.1011 Filenames adhere to the 8.3 DOS format.11 Dates and times are encoded in a specific format, and the starting cluster number is split into high and low values stored in little-endian format.1213 Root Directory: Serves as the main folder on the FAT volume.13 It holds the volume label entry, limited to 11 characters, and its modified date and time.13 ○ Role in FAT: The root directory is the topmost level of the file hierarchy in FAT12 and FAT16 filesystems. It contains directory entries for files and folders within the volume. ○ Limitations: The root directory in FAT12/16 is limited by size and location within the filesystem, which restricts the number of files. ○ Forensic Importance: The root directory often contains the filesystem's primary structure, and analysing it can reveal the structure and some contents of deleted files. Long File Names (LFNs): For filenames exceeding the 8.3 DOS limit, a series of LFN entries are used.14 These entries contain Unicode filename segments, with the last entry marked and a checksum calculated from the short filename for verification.1516 FAT Table: This table tracks cluster usage, including bad clusters and cluster chains for fragmented files.17 Each cluster has an entry, and the table uses different entry sizes for each FAT version (12 bits for FAT12, 16 bits for FAT16, and 32 bits for FAT32).18 Valid values indicate cluster status: unused, in use, bad cluster, or end-of-file marker.18 Forensic Aspects of FAT Filesystem File Deletion: ○ Deleted files are marked by replacing the first character of the file name with a special character (0xE5), but data remains in clusters until overwritten. ○ When a file is deleted, its entry in the FAT table is updated, and the first character of the file name in the directory entry is changed to 0xE5, indicating the file has been removed. However, the data clusters may not be overwritten, making recovery possible. File Slack and Unallocated Space: ○ Composed of RAM slack (unused space within a sector) and drive slack (remaining empty sectors in a cluster), file slack can contain remnants of previously stored data.2728 ○ This is relevant for forensic investigations as it can reveal information about deleted files. However, this is not applicable to SSDs. ○ Important sources of hidden data remnants that could contain residual deleted files. ○ Slack space and unallocated space can contain remnants of deleted files, which are essential in forensic recovery and analysis. Timestamps: Stored in local time and can be misleading as copying a file updates the created time but not the written time.  ○ File timestamps can be inconsistent due to local time storage and operating system updates. ○ FAT stores timestamps in local time format, including creation, modification, and access times. These timestamps, however, may be altered if files are moved between systems. Key FAT Boot Sector Fields: Bytes per Sector (Offset: 0x0B, 2 bytes): Typically 512 bytes. Sectors per Cluster (Offset: 0x0D, 1 byte): Determines the number of sectors that make up a cluster. Reserved Sectors (Offset: 0x0E, 2 bytes): Number of sectors before the start of the FAT tables. Defines the start of the FAT table. Number of FATs (Offset: 0x10, 1 byte): Typically 2 for redundancy. Root Directory Entries (Offset: 0x11, 2 bytes): Maximum number of entries in the root directory (FAT12/16 only). Calculations in FAT Forensics Root Directory Start: ○ Calculate the start sector of the root directory using: Start Sector of Root Directory=Reserved Sectors+(Number of FATs×Sectors per FAT)\text{Start Sector of Root Directory} = \text{Reserved Sectors} + (\text{Number of FATs} \times \text{Sectors per FAT})Start Sector of Root Directory=Reserved Sectors+(Number of FATs×Sectors per FAT) Directory Entries Windows, from XP onwards, uses a next-available allocation for directory entries, leading to numerous old entries.25 Windows reads entries until an all-zero entry is encountered. Directory Entries in FAT Filesystems Definition: Directory entries are structures within a FAT( File Allocation Table) filesystem(fixed size records) that store metadata about files and folders. Each entry spans 32 bytes and contains the filename, file extension, attributes, timestamps, and other information necessary for the operating system to access the file. Structure of Directory Entries: Each entry has specific offsets for information like the file name (stored in 8.3 format for short names), file extension, and attributes. Long filenames, which don’t adhere to the 8.3 format, require additional directory entries (detailed below in “Long File Names”). Components of a Directory Entry: ○ Filename: An 8.3 format filename where the name occupies 8 characters and the extension 3 characters. ○ Attributes: 1-byte field that defines if the entry is a file, folder, or a system or hidden file. Common flags include: 0x01: Read-only 0x02: Hidden 0x04: System  0x10: Directory 0x20: Archive ○ Reserved: 10 bytes reserved for future use or system-specific flags. ○ File Size: A 4-byte field storing the size of the file in bytes. ○ Starting Cluster: The location within the data area where the file begins. FAT12, FAT16, and FAT32 use different numbers of bits to represent cluster numbers (12, 16, and 32 bits, respectively). Long File Names (LFN) Handling Long Filenames: ○ FAT filesystems originally only supported the 8.3 filename structure, but long filenames are now stored by dividing them into multiple directory entries. ○ LFN Entries: Each LFN entry has a specific format, with an order byte to track the sequence of each part of the filename. ○ Checksum: LFN entries include a checksum to verify that they belong to the correct filename, preventing corruption or mix-up in filenames. Forensic Importance: ○ LFNs spread across multiple entries increase the chances of partial recovery if only part of the entry has been overwritten. Relevance in Forensics: ○ Directory entries are often the first place forensic examiners look for information on a file, especially when investigating deletion or access patterns. Key attributes, timestamps, and size information can help establish file usage and modification history. LFNs spread across multiple entries increase the chances of partial recovery if only part of the entry has been overwritten. DOS File Time and DOS File Date DOS File Time: ○ DOS uses a specific format to store file time in 16-bit values. The time is stored in bits as follows: ○ Hour: 5 bits (0-23 hours) ○ Minute: 6 bits (0-59 minutes) ○ Second: 5 bits (0-29 in two-second increments) ○ Forensic Relevance: Understanding this bit structure is critical for accurate time decoding, especially when files are examined outside the original OS. DOS File Date: ○ DOS date is also stored as a 16-bit value, structured into: ○ Year: 7 bits (from 1980 to 2099) ○ Month: 4 bits (1-12) ○ Day: 5 bits (1-31)  ○ Forensic Relevance: These date fields are essential for timestamp verification and understanding the lifecycle of a file. Differences in DOS file time/date fields can indicate edits, copying, or other actions. Directory Entries - Starting and Long File Names Windows, from XP onwards, uses a next-available allocation for directory entries, leading to numerous old entries.25 Windows reads entries until an all-zero entry is encountered. Starting Entries: ○ Every directory, aside from the root, begins with entries for the current directory (`.`) and parent directory (`..`). These entries assist with navigating the directory structure. Forensic Relevance: The structure and location of these entries can provide insight into directory organisation and user behaviour. Long File Names (LFN): ○ FAT filesystems store long filenames by splitting them across multiple directory entries. Each LFN entry has a specific order and checksum to ensure the filename’s integrity. Attributes: ○ Each LFN entry has a flag of 0x0F in the attribute byte, identifying it as part of a long filename. Forensic Relevance: LFNs make recovery more complex, but they also store data across several entries, increasing the chance of partial recovery even if a portion of the filename is overwritten. FAT Table Function of the FAT Table: ○ The FAT table is a data structure that maps clusters in the data area. It tracks the clusters allocated to files and marks clusters that are free, bad, or part of a file’s data chain. Cluster Tracking: ○ The FAT table allows the OS to follow a chain of clusters to locate the full content of a file. Each entry in the FAT table points to the next cluster in the sequence, ending with a marker indicating the end of the file (EOF). Forensic Relevance: - Forensics on the FAT table can help reconstruct the sequence of clusters in files, making it possible to reassemble partially deleted files or identify cluster usage patterns that point to hidden data. File Operations in FAT - Creation, Folder Management, and Deletion File Creation Involves adding a directory entry, writing data to the data area, and updating the FAT table. Process: ○ When a new file is created in a FAT filesystem, an entry is made in the directory table, clusters are allocated in the FAT table, and the data is written to the data area. Forensic Relevance: The date and time stamps in the directory entry reflect the creation time, and the sequence of clusters can be traced through the FAT table. Creating a Folder Similar to file creation, but a folder's directory entry indicates its type Process: ○ Creating a folder involves making an entry in the parent directory and creating a new directory table with references to the current (`.`) and parent (`..`) directories. Forensic Relevance: Folder entries provide insight into file organisation and hierarchy, which can be useful in establishing user intent and behaviour. Deleting a File and Folder File Deletion: Replaces the first character of the filename with 0xE5 and clears the corresponding FAT entries.20 Data in the data area remains untouched. ○ Deleting a file replaces the first character of its name in the directory entry with 0xE5 and marks its clusters in the FAT as free. However, the actual data remains in the data area until it is overwritten. Folder Deletion: Similar to file deletion but applies to the entire folder's directory entry. ○ Similar to file deletion, the directory entry is marked as deleted, but the data remains in the clusters. Entries for subfiles or subfolders are also flagged but not immediately erased. Forensic Relevance: Deleted entries in FAT file systems can often be recovered as long as the clusters remain untouched. Tools that recognize the 0xE5 marker can attempt recovery based on residual cluster chains. Creating and Deleting a File with a Long File Name Creating LFNs: ○ For files with long filenames, multiple directory entries are created to store each segment of the name. These entries are linked by sequence numbers and a checksum. Deleting LFNs: ○ Deleting LFNs replaces the first byte of each segment with 0xE5. Recovery can still be feasible if all segments are intact. Forensic Relevance: Files with LFNs are more complex to recover, but partial recovery is possible, especially if only part of the name is corrupted. File Recovery Techniques in FAT File Recovery - Filename Standard Recovery: ○ Tools can identify deleted files by locating entries with a 0xE5 marker in the filename. The entry still contains the rest of the metadata, which allows the recovery of the filename by adding a placeholder. Forensic Relevance: Filename recovery is crucial for identifying files of interest and confirming the presence of deleted content. File Recovery - Long Filename  Recovery of LFNs: ○ Long filenames split across entries may still be recoverable if the segments remain intact. Since only the first byte is replaced with 0xE5, the rest of the filename segments can often be reconstructed. Forensic Relevance: The ability to reconstruct LFNs even after deletion allows for deeper recovery possibilities in FAT filesystems. File Recovery - Data Recovery Process: ○ Data recovery in FAT is based on re-linking clusters through the FAT table. Even when deleted, clusters retain data until overwritten, allowing reconstruction of entire files or partial content. Forensic Relevance: Data recovery enables a forensic examiner to retrieve evidence that was deliberately deleted or accidentally lost, supporting comprehensive investigations. FAT - Forensic Considerations Cluster Chaining: ○ Each cluster in the FAT table points to the next cluster in a file’s chain, so the entire sequence can be reconstructed unless overwritten. Slack Space: ○ Slack space within a cluster can contain remnants of previously deleted data, as the entire cluster may not be rewritten when a new file is created. Timestamp Integrity: ○ FAT timestamps can be inconsistent due to system changes, so timestamps must be cross-validated with other artefacts. Forensic Relevance: The forensic analysis of FAT requires careful consideration of timestamps, slack space, and residual clusters to recover as much information as possible. Directory Entries: Windows, from XP onwards, uses a next-available allocation for directory entries, leading to numerous old entries.25 Windows reads entries until an all-zero entry is encountered. ExFAT File System The ExFAT file system addresses limitations of FAT, supporting larger volumes and file sizes, faster I/O, and features like extensibility and flexibility.2526 It introduces a new disk structure with components like boot parameters, cluster heap, and VBR hash. Purpose and Background: ○ ExFAT (Extended FAT) was developed to support larger files and larger storage media than traditional FAT32, making it suitable for flash drives and SD cards over 4GB. It is commonly used in portable storage devices. Structure: ○ Boot Sector: Contains file system parameters, such as the volume size, cluster size, and FAT size. ○ Cluster Heap: The main data storage area, where files and directories are stored.  ○ FAT Table: Tracks cluster usage, similar to FAT32, but designed for handling larger files and devices. Directory Entries: ○ ExFAT uses 32-byte directory entries, with each entry including the filename, timestamps, and size. ○ Unlike FAT, ExFAT allows for more flexible filenames and does not restrict file storage by the 4GB limit. Forensic Relevance: ○ ExFAT’s larger addressable space and flexibility make it suitable for modern storage. It allows for simpler recovery of larger files and reduces the limitations 3. NTFS File System New Technology File System (NTFS) Overview Introduction: ○ NTFS, developed by Microsoft, is the standard file system for Windows OS. It offers advanced features like support for large files, security, and data recovery, making it suitable for modern systems. ○ Introduced in Windows NT 3.1 ○ Implements some things required for a multiuser environment. Security, user IDs and Quotas. ○ Implements a journaling system to help prevent errors on the disk. ○ Uses 64 bits to represent clusters so can theoretically go to 264 – 1 in size. (However, XP is limited to 232 – 1. ○ Adds file system features such as hard links, compression and encryption. ○ Adds sparse files to save disk space when large empty files are stored. ○ Has Alternative Data Streams which can be used to store additional data about a file. Concepts - Lazy Write ○ Lazy write is the name given to the technique to caching data in a “write back” cache and then copying the data to the final destination during idle times. ○ Therefore, changes to files are not made immediately on disk. They can be held within the cache to be written during an idle period. ○ This is the main reason that NTFS is not used for removable media. ○ A removable drive can be formatted with NTFS by setting the “optimise for performance” option for the drive. ○ Right click → Properties → Hardware → Select Drive → Properties → Policies → Optimise for Performance Seizure Issues ○ If a file is written and the power is removed before the cache has been cleared, is the data lost? The data may be lostNTFS uses a technique called "lazy write," which caches data to be written to the disk later during idle times. This improves performance but introduces the risk of data loss during power failure.  ○ If files are deleted and the power is removed before the cache has been cleared, is the data deleted? Similar to writing files, the deletion process may also be delayed due to lazy write. The deletion information might still reside in the cache and not be reflected on the disk until the cache is cleared. NTFS Logging ○ Changes are first saved to the logging file before being written to the disk ○ User saves file → Changes are saved to the logging file first → Finally changes are written to the disk ○ This technique allows the operating system to look at the logging file. It can then determine from the disk whether the changes were made successfully and decide whether to roll back the change or try to re-commit it. Structure of NTFS: ○ In NTFS everything is a file ○ NTFS tracks a number of system files that contain the file system metadata. This is data that is used by the file system to access other data. ○ These system files have filenames that begin with $ ○ Every file has a corresponding MFT Entry.  ○ Master File Table (MFT): The $MFT is the heart of the NTFS file system. i.e The main file within the NTFS file system File contains file records for each of the file within the file system(including itself) Central database containing file records for every file and directory on the volume, including itself Contains a record for each file and directory on the volume. Each record holds essential metadata, including file name, timestamps, and permissions. The first few records in the $MFT are reserved for crucial metadata files, including the $MFT itself, its mirror ($MFTMirr), the log file ($LogFile), the volume information ($Volume), and the root directory ($.), among other NTFS Metadata Files NTFS utilises various metadata files, all starting with a $, to maintain file system information7. These files are stored as records within the $MFT 1. $MFT (Master File Table) Master File Table, contains file records for all files and folders - Record Number: 0 - Purpose: Stores records for every file and directory on the NTFS volume, including its own. Each file or directory’s information is maintained in a fixed-size MFT entry. - Filename: $MFT  2. $MFTMirr (Master File Table Mirror) Mirror of the first four $MFT entries, provides redundancy for critical metadata - Record Number: 1 - Purpose: Provides a backup of the first few records of the MFT, usually the first 4 records, to support recovery in case of corruption. - Filename: $MFTMirr 3. $LogFile Stores transaction records for NTFS journaling, enabling recovery from inconsistencies - Record Number: 2 - Purpose: Holds a transaction log that records changes to the file system. Essential for system recovery and maintaining NTFS consistency in case of power failures or crashes. - Filename: $LogFile 4. $Volume Contains information about the volume, such as name and version - Record Number: 3 - Purpose: Contains volume information, such as the volume label and version. This file also stores metadata about the volume’s state and NTFS version. - Filename: $Volume 5. $AttrDef (Attribute Definitions) Defines attribute types and names used in NTFS - Record Number: 4 - Purpose: Defines the various attribute types that NTFS can use to describe file properties, such as `$STANDARD_INFORMATION`, `$FILE_NAME`, etc. - Filename: $AttrDef 6. $Root Directory Represents the root directory of the volume - Record Number: 5 - Purpose: Represents the root directory of the volume, serving as the top-level directory under which all other directories and files reside. - Filename: `.` (the root directory is denoted by a dot) 7. $Bitmap Tracks used and free clusters on the volume, similar to the FAT table but in bitmap format - Record Number: 6 - Purpose: Tracks allocation of clusters on the volume, identifying which clusters are in use or free. It helps NTFS manage available space efficiently. - Filename: $Bitmap 8. $Boot (Boot Sector) Stores the boot sector of the volume - Record Number: 7 - Purpose: Stores the boot sector and bootstrap code used for booting the operating system. It includes essential volume parameters for the file system. - Filename: `$Boot` 9. $BadClus (Bad Cluster File) Contains a bitmap of bad clusters identified on the volume - Record Number: 8 - Purpose: Tracks bad clusters on the disk, marking them as unusable. This prevents NTFS from writing data to damaged areas of the disk. - Filename: $BadClus 10. $Secure (Security Descriptors) Contains a bitmap of bad clusters identified on the volume - Record Number: 9 - Purpose: Manages security descriptors, which define permissions and access control for files and directories. It maintains a database of unique security descriptors used on the volume.  - Filename: `$Secure` 11. $UpCase (Uppercase Table) Provides information for case-insensitive filename comparisons - Record Number: 10 - Purpose: Contains a table mapping lowercase to uppercase characters, which helps NTFS achieve case-insensitive file operations. - Filename: `$UpCase` 12. $Extend Contains optional extensions for NTFS, such as quotas and reparse points - Record Number: 11 - Purpose: A system directory containing additional metadata files, including files used for features like quotas, object IDs, and reparse points. - Filename: $Extend 13. $Quota Stores disk quota information for users - Record Number: 24 - Purpose: Manages disk quotas, allowing administrators to control and limit the amount of disk space used by individual users. - Filename: `$Quota` 14. $ObjId (Object IDs) Manages object IDs for files - Record Number: 25 - Purpose: Stores object identifiers (OIDs) that uniquely identify files or directories, which aids in tracking and linking objects, especially in distributed systems. - Filename: `$ObjId` 15. $Reparse Contains data for reparse points, which redirect file system requests to specific handlers - Record Number: 26 - Purpose: Manages reparse points, which are used for symbolic links, mount points, and other advanced NTFS features. - Filename: $Reparse MFT Entry Structure Each MFT entry is typically 1024 bytes long10. The entry comprises a header and a series of attributes describing the file or directory of Windows (defined within the boot sector). Although an MFT entry can span more than one record. MFT Record Header -The header holds basic file information Signature ("FILE"): Identifies the record as an MFT entry11.  Update Sequence Offset and Count: Manages updates to the record and consistency with the $LogFile11. Log Sequence Number (LSN): Tracks changes for NTFS journaling11. Sequence Number: Aids file usage and deletion, incremented when a file is deleted and its record becomes available for reuse11. Link Count: Tracks the number of hard links pointing to the file11. Attributes Offset: Points to the start of the attribute list within the MFT entry11. Flags: Indicate file allocation status (live or deleted) and whether it's a file or a directory11. Used Size: Indicates the currently used space within the MFT record11. Allocated Size: Represents the total allocated space for the MFT record11. Base File Record: Points to the base MFT record if the file's attributes span multiple records11. Next Attribute ID: Tracks the next available attribute ID for the file11. Update Sequence Number: Ensures data integrity during updates File Attributes:Following the header, each MFT entry contains a variable number of attributes that describe the file's properties and data. Each attribute has a specific type, identified by a unique identifier ○ $STANDARD_INFORMATION: Stores basic file information, including creation, modification, access, and MFT entry modification timestamps, as well as DOS file attributes like read-only, hidden, system, and archive. (basic metadata) ○ $ATTRIBUTE_LIST: Used when a file has more attributes than can fit within a single MFT record, this attribute points to additional MFT records containing the remaining attributes. ○ $FILE_NAME: Stores filename information, including the long file name, short file name (8.3 format), creation, modification, access, and MFT entry modification timestamps, logical size of the file, and flags indicating file type and other properties. (actual file content) ○ $OBJECT_ID: Contains a globally unique identifier for the file, useful for tracking files across different volumes or systems12. ○ $SECURITY_DESCRIPTOR: Defines security permissions for the file, including access control lists (ACLs) that specify which users and groups have what level of access. ○ $VOLUME_NAME: Stores the volume label assigned to the NTFS volume. ○ $VOLUME_INFORMATION: Contains volume-specific information, such as version and flags16. ○ $DATA: Holds the actual file data. This attribute can be resident, meaning the data is stored directly within the MFT record for small files, or non-resident, meaning the data is stored in separate clusters on the volume and the attribute points to the data runs.(actual file content) ○ $INDEX_ROOT: Used for directories, this attribute stores the root of the directory index, which is a B-tree structure used to efficiently locate files within the directory. ○ $INDEX_ALLOCATION: For large directories, this attribute points to clusters that store the directory index, allowing for efficient management of large numbers of files.  ○ $BITMAP: Used for sparse files, this attribute maintains a bitmap indicating which clusters contain data and which are empty. This attribute holds metadata for each file, including timestamps and permissions. Tracks the allocation status of clusters on the volume (tracks cluster usage) ○ $REPARSE_POINT: Stores information for reparse points, which redirect file system requests to specific handlers, enabling features like symbolic links and mount points16. ○ $EA_INFORMATION and $EA: These attributes manage extended attributes associated with files, providing a mechanism to store additional file-specific data. ○ $LOGGED_UTILITY_STREAM: Used for logging purposes, this attribute provides a stream for storing log data related to file operation Attribute Structure: Each attribute, whether resident or non-resident, consists of a header and data. ○ Resident Attribute: The data is stored directly within the MFT record18. The header defines the attribute type, length, and other relevant information18. ○ Non-Resident Attribute: For larger files, data is stored in separate clusters on the disk18. The attribute header contains pointers to the data runs on the disk18. It also stores information about the starting Virtual Cluster Number (VCN), last VCN, data run length, compressed size, and allocated and initialised sizes of the attribute content Advanced Features: File Permissions: NTFS allows granular control over who can access and modify files. Hard links : provide multiple directory entries pointing to the same file data, unlike shortcuts that merely point to the original file location Encryption: Built-in encryption for file security. provides confidentiality for sensitive data by making it unreadable without the proper decryption key. Data Compression: NTFS supports file compression to save space. Reducing disk space usage for large files helps reduce storage space for files. Data Runs: Data runs are sequences of contiguous clusters used to store file data in non-resident attributes. They are represented as a series of byte sequences within the $DATA attribute. Each byte sequence encodes the length and location of a data run, allowing the file system to locate the file data on the disk. Data runs can be fragmented, meaning the data for a single file is scattered across different locations on the disk. The MFT entry, through the $DATA attribute, maintains the map of these data runs. Named Streams (ADS) Alternative Data Streams (ADS) in NTFS allow storing multiple data streams within a single file4. The main data stream, accessed normally, is unnamed. Additional streams are created with names and can store data independently. ADS are not visible in typical file listings but can be accessed and manipulated using specific commands or tools. They are implemented using multiple $DATA attributes within the same MFT entry. Forensic Considerations in NTFS File Recovery: Recovering deleted files in NTFS can be more challenging than in FAT due to the file system's metadata structures and the potential for data to be overwritten by the journaling process. ○ However, remnants of deleted files may still exist in unallocated space, MFT slack, or in the $LogFile Deleted Files: ○ Deleting a file removes the entry in the MFT but leaves the data in clusters, making partial recovery possible. ○ Deleting a file in NTFS only marks the MFT entry as unused, but the data remains on the disk until it is overwritten. This makes it possible to recover deleted files. Alternate Data Streams (ADS): ○ Hidden data can be stored within files, used by malicious actors for concealing information. ○ ADS allows additional data to be attached to files without affecting their main content. It can be used to hide malicious data or metadata, making it important in forensics. ○ Detecting and analysing ADS can reveal hidden data, potentially relevant to an investigation. Forensic tools often have specific features to identify and analyse ADS. Timestamps in NTFS: ○ NTFS records multiple timestamps (created, modified, accessed, and entry modified), crucial for establishing activity timelines. ○ NTFS records four timestamps: created, modified, accessed, and entry modified. These can be valuable in establishing a timeline of file interactions. ○ However, understanding the specific meaning and potential limitations of each timestamp is crucial for accurate analysis. The sources highlight that copying files in NTFS updates the creation timestamp but not the last modified timestamp, which can be misleading in investigations Log File Analysis : ○ Examining the $LogFile can provide insights into file system activities and potentially reveal traces of deleted or modified files. However, the $LogFile can be overwritten over time, limiting its usefulness 4. Windows Registry and Artefacts Windows Registry Overview Purpose: The Windows Registry is a hierarchical database that stores configuration settings and options for the operating system and applications. It is essential for both system performance and forensic analysis. The Windows Registry acts as a centralised database containing crucial system-wide settings, hardware configurations, software installations, user profiles, and more.  Artefacts, in the context of digital forensics, refer to the traces left behind by user activity, system processes, and data, which can be examined to reconstruct events and understand what happened on a computer Registry Structure: The Registry is organised hierarchically, much like a file system, with hives at the top level, followed by keys, subkeys, and values ○ Hives: These are the root-level containers for registry data, analogous to the root directory of a file system. Each hive file is stored on disk and loaded into memory when Windows starts. Common hive files include SYSTEM, SOFTWARE, SAM, SECURITY, and NTUSER.DAT. ○ Keys: Similar to folders, keys are containers within hives that organise related settings and values. Keys can contain subkeys, creating a hierarchical structure. ○ Subkeys: These are nested keys within parent keys, further categorising information. ○ Values: Values represent specific settings or data points within keys. Each value has a name and a data type. Hive Files and Locations The physical representation of the Registry exists in hive files located primarily in %SYSTEMROOT%\System32\config.2 These files have different extensions: ○.none: Complete copy of the hive data.3 ○.alt: Backup copy of the HKEY_LOCAL_MACHINE\System hive.3 ○.log: Transaction log tracking changes to keys and values.3 ○.regtrans-ms: Logs for the transactional registry subsystem, often used by software installers.3 User-specific hives, NTUSER.DAT, are found under each user's profile directory (C:\Users\) Root Keys: Logical View of the Registry ○ Root Keys provide a logical, hierarchical view of the Registry, making it easier to access and manage information. The five primary root keys are: HKEY_LOCAL_MACHINE (HKLM): Contains hardware and system settings. Stores computer-wide hardware and software settings, including information about the operating system, installed devices, and system services. HKEY_CURRENT_USER (HKCU): Contains user-specific settings for the logged-in user. A dynamic link to the hive of the currently logged-in user (NTUSER.DAT). HKEY_CLASSES_ROOT (HKCR): Information on file associations and OLE data. Contains information about file associations, determining which applications are used to open specific file types.  HKEY_USERS (HKU): Stores profile information for all user accounts.Contains user-specific configuration settings for all user profiles on the system. HKEY_CURRENT_CONFIG (HKCC): Hardware profile currently used by the system. Holds hardware and software settings used at startup. Notably, this key is not saved to the hard drive ○ Keys and Values: Keys are similar to folders, and values within each key hold specific information. Keys function like folders that can contain other keys and values. Values are specific data entries within keys, storing information as strings, numbers, or binary data. Forensic Significance of the Registry The Registry is a treasure trove of information for forensic investigators, as it records numerous details about system activities, user actions, and application behaviour. Analysing the Registry can uncover: System Configuration: Operating system details, hardware specifications, installed software, boot settings, and more. Installed Devices: Information about connected devices, including USB devices, hard drives, and network adapters. User Accounts: Usernames, SIDs (Security Identifiers), account creation dates, last logon times, and group memberships. ○ SID- Security ID is used to identify the computer system ○ RID - Relative ID is used to identify the specific user on the computer system User Activities: Evidence of files opened, programs executed, websites visited, searches performed, and network connections established. Application Settings: Preferences and configurations for various applications, potentially revealing user behaviour and actions within those applications. Evidence of Malware: Traces of malware infection, such as modified registry entries or newly created keys associated with malicious software. Important Registry Keys for Forensics User Data and Activity Tracking: ○ Includes user profiles, last login times, and recent activity. ○ NTUSER.DAT (HKCU): Contains user-specific settings and activity records, such as last opened files and application settings. ○ SAM (Security Accounts Manager): Stores hashed passwords and login information, often used for authentication analysis. System Information: ○ Provides insight into installed software, startup programs, and connected devices. ○ System Hive: Holds data about installed devices, system time, and network information. ○ Run Keys (HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Run): Programs configured to start automatically at boot. Time Zone Information: ○ Critical for interpreting timestamps during cross-time-zone investigations.  ○ The TimeZoneInformation key can be critical in forensic investigations for correctly interpreting timestamps across different time zones. Important Registry Keys: ○ Run: Lists programs that run at startup. ○ RecentDocs: Tracks recently accessed files. ○ TimeZoneInformation: Critical for interpreting timestamps during cross-time-zone investigations. Windows Artefacts: Unveiling Digital Traces Artefacts are digital remnants of user activity, system events, and data that provide crucial insights into what occurred on a computer. These artefacts can be scattered across the file system, registry, and various system locations. Examining these artefacts is crucial for reconstructing timelines, understanding user actions, and uncovering potential evidence. File System Artefacts: Recycle Bin: When files are deleted in Windows, they are typically moved to the Recycle Bin before being permanently removed. Examining the Recycle Bin can allow for the recovery of deleted files and provide information about the time and date of deletion. ○ Artefacts within the Recycle Bin folder, such as $I and $R entries, provide details about deleted files, including original file paths and deletion times. Recent Folder: This folder, located at C:\Users\\AppData\Roaming\Microsoft\Windows\Recent, contains shortcuts (.lnk files) to files that have been accessed recently. ○ These link files preserve valuable metadata: Volume serial number of the drive where the target file was located. File size in bytes. MAC (Modified, Accessed, Created) timestamps of the target file. Complete path to the original target file, even if it has been deleted. Temp Folders: Temporary files created by applications during their operation are often stored in temp folders. These folders, typically found in C:\Users\\AppData\Local\Temp and application-specific directories, can contain valuable remnants of user activity, such as partially edited documents, extracted files from archives, or temporary copies of downloaded files. Registry -Based Artefacts Shellbags: Shellbags are registry keys that record folder view preferences, such as window size, position, and view mode.Analysing shellbags can reveal information about folders that a user has accessed, even if those folders have been subsequently deleted. This can be particularly useful for: ○ Identifying the presence of removable media that was connected to the system. ○ Reconstructing user activity timelines based on folder access patterns. ○ Uncovering evidence of deleted or hidden folders.  USB Device Artefacts: When a USB device is connected to a Windows system, information about the device is recorded in various registry keys. Analysing these keys can provide details about the device, including: ○ Vendor ID and Product ID, which can be used to identify the manufacturer and model of the device. ○ Serial number, which can be used to uniquely identify the device. ○ Timestamps of when the device was connected and disconnected. ○ Drive letter assigned to the device. Typed URLs: Internet Explorer stores typed URLs in the registry, which can provide evidence of websites visited by a user. Recent Documents: The registry tracks recently opened documents, providing another source of evidence regarding user activity. RunMRU: The "RunMRU" key stores a history of commands executed using the "Run" dialog box, offering insights into programs launched by the user. System Files Prefetch Files: Prefetch files are created by Windows to optimise application loading times. They are stored in the C:\Windows\Prefetch directory and contain metadata about executed programs. Analysing prefetch files can reveal: ○ The name of the executable file. ○ The last execution time of the application. ○ The number of times the application has been run. Swap File (pagefile.sys): ○ When the system's RAM is fully utilised, Windows starts using a portion of the hard drive as virtual memory. ○ This designated area, known as the swap file or pagefile.sys, can contain remnants of data that were once present in RAM. ○ As data is constantly swapped in and out of the swap file, it is not a reliable source for recovering complete files. ○ However, it can potentially hold fragments of sensitive information that were once in memory. Hibernation File (hiberfil.sys): ○ When a computer enters hibernation mode, the contents of RAM are saved to the hiberfil.sys file. ○ This allows the system to resume quickly from where it left off, preserving the state of open applications and documents. ○ Analysing the hibernation file can potentially reveal information about the system's state at the time of hibernation, including open files, running processes, and data that was present in memory. ○ However, the hibernation file is typically very large, and extracting data from it can be challenging. Thumbcache: ○ Windows maintains a thumbnail cache to store small previews of images and videos, making it faster to display them in File Explorer.  ○ These thumbnail caches can be valuable for forensic investigators because they may contain thumbnails of images that have been subsequently deleted or are located on removable media that is no longer connected to the system. The thumbcache files are stored in ESE (Extensible Storage Engine) database format and can be analysed using specialised tools to recover the cached images. The sources note that the thumbnail cache implementation has changed over different versions of Windows, leading to variations in file formats and storage mechanisms. Registry Artefacts in Forensics Last Known Good Configuration: Stores details on the last successful boot, useful for determining prior system configurations. Recently Accessed Files: The RecentDocs key lists recently accessed files and folders, which can indicate user activity. USB Storage Devices: ○ SYSTEM\CurrentControlSet\Enum\USBSTOR: Records information on connected USB devices, including timestamps and device IDs. Prefetch Files: ○ Located in the \Windows\Prefetch folder, Prefetch files track recently accessed applications and their data locations, helping to establish patterns of activity. Challenges and Best Practices in Registry and Artefact Analysis Time Discrepancies: Analysing timestamps from different sources (system clock, file system, program logs) can be challenging due to potential inconsistencies and variations in timekeeping mechanisms. ○ Understanding the relationship between local time and UTC is essential for accurate timeline construction. Tool Limitations: Forensic tools may have limitations in their ability to parse and interpret specific artefacts, particularly those from less common applications or operating systems. ○ Relying solely on automated tools without a thorough understanding of the underlying data structures can lead to inaccurate or incomplete findings. Anti-Forensics Techniques: Individuals attempting to conceal their actions may employ anti-forensics techniques to obfuscate or erase digital traces. This can make it more difficult to recover and interpret artefacts. Data Recovery Challenges: Recovering deleted data from unallocated space or fragmented files can be complex and may not always be successful. Best Practices for Robust Investigations Use Validated Tools and Techniques: Utilise forensic tools and methodologies that have been tested and validated to ensure accuracy and reliability.  Thorough Documentation: Meticulously document every step of the investigation, including tools used, procedures followed, and findings obtained. This documentation creates an audit trail that can be used to demonstrate the integrity of the investigation. Corroborate Evidence: Whenever possible, seek corroboration from multiple sources to strengthen findings and build a more comprehensive understanding of the events. Consider Context: Analyse artefacts within the context of the investigation and the specific allegations. This contextual understanding helps to determine the relevance and significance of the findings. Continuous Learning: Stay up-to-date with the latest developments in digital forensics, including new technologies, tools, and techniques. The field is constantly evolving, and ongoing professional development is crucial for maintaining expertise. 5. Web Browser Forensics Web browser forensics is a branch of digital forensics that focuses on recovering and analysing evidence from web browsers. This evidence can be used to understand a user’s activity on the internet. Web browsers typically store a record of accessed websites, primarily to allow users to easily revisit them. However, this stored data can also be used for forensic purposes Overview of Browser Artefacts Web browsers store a wealth of forensic information that can reveal user activity on the internet. Common artefacts include browsing history, cache files, cookies, and download history. Browser Cache: ○ Stores temporary files like images, HTML, and scripts. ○ Stores temporary files (e.g., images, HTML files) that can reveal what websites a user visited and when. Each browser (e.g., Chrome, Firefox, Edge) has its own format for cache storage. ○ When the page is accessed again, the browser can retrieve stored page code and embedded files ( such as graphics) from the hard drive rather that the server This speeds up access and save bandwidth - hence the page loads faster ○ Forensic Use - Recover information about what a user was accessing on the internet History: ○ Lists URLs visited, with timestamps. ○ Browser history files log each website visited, typically with timestamps. This is usually stored in an SQLite database, such as History.db in Chrome. Cookies: ○ Contain session data, login tokens, and tracking information. ○ Small text files that store session data and user preferences. They can reveal logins, visited sites, and user tracking information. Allows session information to be recorded between visits to a website. (e.g. keep shopping basket contents) ○ Forensic use:- Understand how the user may have interacted with a particular website.  Plug-ins : ○ Gives the browser additional functionality, e.g Flash, Silverlight, Java, web developer, SQLite ○ Forensic use: - May store locally cached information that can be used by the Forensic Analyst Downloads: ○ Logs files downloaded from the internet, which may include timestamps and source URLs, aiding in identifying suspect file origins. Types of Web Browser Data Used in Forensic Investigations The following types of data are commonly stored by web browsers and are useful for forensic analysts: Tracking pages visited (history): This includes information about which pages were visited, when they were visited, and how often they were visited. Files used in displaying web pages (cache): This includes the code and styling of web pages, as well as any embedded files such as graphics. ○ Examining the cache can help recover information about what a user was accessing on the internet. Information filled in (form history): Avoid signing in all the time. This includes information that a user has entered into forms on web pages, such as their name, address, and credit card number. Automatic identification / authentication information: This includes cookies and saved credentials, which are used to identify and authenticate users on websites. ○ Analysing cookies can help forensic investigators understand how a user interacted with a particular website. ○ However, it is important to note that some cookies may be spurious, such as third-party cookies Preferences: This includes information about a user’s preferences for a particular website, such as their language, font size, and theme. Trackers: This includes information about trackers that have been used to track a user’s activity on the internet How Browsers Store Information Browsers store information in a variety of formats. Older versions of Internet Explorer (IE 4-9) used a proprietary format called index.dat. IE 10 and 11, along with the original Microsoft Edge (Windows 10), use an Extensible Storage Engine (ESE) embedded database in WebCacheV01.dat and WebCacheV24.dat files. The ESE database format is not publicly documented, but it has been mostly decoded. ○ Format: Database header and backup Pages containing: Space tree data Database table data Database index data  Long value data Most modern browsers, including Firefox, Chrome, Safari, and the new Edge (Chromium), use SQLite databases to store data ○ SQLite: A very "light" open source DBMS (DatabaseManagement System) embedded SQL database engine no separate server process; reads and writes directly to ordinary disk files; cross-platform (32-bit / 64-bit, big-endian /little-endian architectures). Very compact – requires little space / RAM Popular for mobile devices Popular as an application file format Used “behind the scenes” of many apps. Forensic Analysis Techniques SQLite Databases: ○ Many modern browsers store data in SQLite databases (e.g., `places.sqlite` in Firefox). ○ Modern browsers, such as Firefox and Chrome, use SQLite databases to store history, cookies, and bookmarks. Forensic tools like Magnet AXIOM or FTK Imager can parse these databases for investigation. Keyword Searches: ○ Searching for specific keywords within cache and history files can reveal patterns in browsing behaviour or attempts to conceal activity. Deleted History Recovery: ○ Deleted browsing history can sometimes be recovered from unallocated space or backup files ○ Even if browsing history is deleted, remnants can still be found in unallocated space, backups, or other files like WebCacheV01.dat in Internet Explorer. Challenges in Web Browser Forensics There are a number of challenges that forensic analysts face when investigating web browser data: Anti-forensics: Some users may attempt to use anti-forensic techniques to hide their web browsing activity. This can include deleting their browsing history, clearing their cache, and using private browsing mode. Attribution: It can be difficult to attribute web browsing activity to a specific user, especially in an office setting where multiple users may share the same computer. Tool limitations: Forensic tools may have limitations in their ability to recover and analyse web browser data. For example, some tools may not be able to recover deleted history or parse certain types of web browser data. Encryption: Some web browsers may encrypt their data, making it difficult for forensic analysts to access Specific Browsers Internet Explorer (IE): Versions 4-9 used the index.dat file to store browsing history. Versions 10 and later use the WebCacheV01.dat file, which is an ESE database56. Forensic  analysts may encounter challenges with IE due to the ESE database format not being publicly documented. Microsoft Edge: The original version of Edge used the same ESE database format as IE 10 and 11. The new Edge, based on Chromium, stores data similarly to Chrome10. Firefox: Firefox uses SQLite databases to store browsing data, primarily in the places.sqlite file11. It also utilises JSON format for some data. Chrome: Chrome, like Firefox, uses SQLite databases to store data. Chrome stores its browsing data in the History file, which is an SQLite database located in the Default folder. The structure of the Chrome database differs from Firefox. It also uses SNSS files for session and tab information. Safari: Safari also utilises SQLite databases to store its data. Resources for Web Browser Forensics A variety of resources are available for those interested in learning more about web browser forensics. These resources include: The Forensic Wiki: This wiki provides a comprehensive overview of web browser forensics, including information about different web browsers, data storage formats, and forensic tools. Digital Detective: This website offers articles, blog posts, and other resources on web browser forensics. Forensic Focus: This website features a forum where practitioners can discuss web browser forensics and other digital forensic topics. The SANS Institute: This organisation offers training courses and certifications on web browser forensics. Best Practices for Web Browser Forensics To ensure the integrity of digital evidence, practitioners should follow these best practices: Use a forensically sound acquisition method: When acquiring a forensic image of a computer, a write-blocker should be used to prevent any changes to the original data. Validate your tools: Forensic tools should be validated to ensure that they are producing accurate results. This can be done by testing the tools against known datasets. Document your findings: All findings should be documented in a clear and concise manner. This documentation should include the tools that were used, the procedures that were followed, and the results that were obtained. 6. Unix System Forensics Unix File System Structure Unix Filesystem Structure  Common Filesystems: Unix systems use filesystems like EXT2, EXT3, and EXT4. These filesystems offer features such as journaling (EXT3 and EXT4) that help recover from crashes but complicate forensics. Directory Layout: ○ Root (/): The root of the Unix filesystem, under which all other directories and files exist. ○ Key Directories: /etc: Contains system configuration files. /var/log: Stores log files, crucial for tracking system activity. /home: Holds user home directories, including personal files and configurations. /tmp: Temporary files, which may reveal recent activity or session information. Unix Permissions and Ownership File Permissions: ○ Unix permissions are represented by three categories (owner, group, others) with read, write, and execute permissions. For example, a file permission of -rwxr-xr-- grants the owner all permissions, the group read and execute, and others read-only. Ownership: ○ Each file has an owner and group assigned, which is critical in tracking user activity and identifying who created or modified files. Key Forensic Artefacts in Unix System Logs: ○ /var/log/syslog: Stores general system messages, including startup and shutdown events. ○ /var/log/auth.log: Tracks authentication attempts, user logins, and failures. ○ /var/log/secure (for some distributions): Holds authentication and authorization messages. Shell History: ○ Each user’s shell history (e.g.,.bash_history in home directories) logs command-line commands entered by the user, providing insight into user activity. Timestamps: ○ Unix systems store file timestamps (creation, modification, access) using epoch time, measured in seconds since January 1, 1970. This timestamping system may need to be converted during investigations. Unix system forensics involves investigating and analysing digital evidence from systems within the Unix family, including Linux and macOS. Unlike Windows systems, Unix systems do not have a centralised registry. Instead, system information and user data are spread across various files and directories. Information Sources on Unix Systems System information: Key system details can be found in: ○ /etc/hostname: Stores the computer's name, which can be correlated with logs and network traffic. ○ /etc/timezone and /etc/localtime: Store the system's timezone, such as "Europe/London" or "America/New_York". These are best parsed on live systems using the zdump command, which links to a tzfile for time translation. ○ /etc/os-release, /etc/lsb-release, and /proc/version: Contain information about the operating system version. Logs: Valuable logs for forensic analysis include: ○ /var/log/messages: Stores general system messages and events, including boot information, hardware changes, and application errors. ○ /var/log/secure: Contains security-related logs, such as login attempts, authentication successes and failures, and sudo command usage. ○ /var/log/auth.log: Records user connections and disconnections, failed logins, and sudo usage. ○ /var/log/utmp, /var/log/wtmp, and /var/log/btmp: /var/log/utmp: Tracks current uptime, logins, and events. /var/log/wtmp: Stores historical data from utmp. /var/log/btmp: Records failed logins. Installed Software: ○ Information about installed software can be obtained from package managers like rpm (used in Redhat and Suse) and dpkg (used in Debian-based systems like Ubuntu).  ○ Logs for these package managers reside in /var/log/dnf.rpm.log and /var/log/dpkg.log, respectively. Mounted Devices: ○ mount command: Shows currently mounted devices and file systems on a live system. ○ /proc/mounts: Stores persistent mount points, including information about tmpfs (temporary file system in RAM). ○ /proc/fstab : List of file systems to be mounted at boot time ○ /etc/mtab : File systems which are currently mounted filesystems, maintained by mount command, may be inconsistent ○ /proc/mounts: Handles by kernel, old versions were buggy More likely to be accurate than mtab Permanent Mounts: ○ /etc/fstab: Defines file systems that are automatically mounted at boot time. This file is essential for understanding how the system was configured Automounted on boot If ramdisks present -> auto deleted evidence tmpfs, ramfs (older), shmfs (older) tmpfs can page to swap Ramfs doesn’t page, no evidence Devices Files and File System Everything is a file, devices too ○ /dev/ Holds device “files” for each device on the system ○ The device file associated with hardware can change with each boot ○ Has a pattern, different device naming conventions for IDE /SATA/ NVMe ○ Pattern : xx[a,b,c,d..][1,2,3,4..] e.g SDA1 xx identifies device type Letter identifies device Number identifies partition on device /dev/ devices types ○ hd: (“classic”) IDE driver (previously used for ATA hard disk drive, ATAPI optical disc drives, etc.) hda: the master device on the first ATA channel (usually identified by major number 3 and minor number 0) hdb: the slave device on the first ATA channel hdc: the master device on the second ATA channel hdc1: first primary partition on this disk (example) hdc5: first logical drive in the extended partition (example) hdd: the slave device on the second ATA channel SCSI driver, also used by libATA (modern PATA/SATA driver), USB, IEEE 1394, etc. sd: mass-storage driver ○ sda: first registered device sda4: last partition on this disk (example) sda6: second logical drive in the extended partition (example) ○ sdb, sdc, etc.: second, third, etc. registered devices  /dev/ devices types ○ NVMe driver nvme0: first registered device's device controller (character device) nvme0n1: first registered device's first namespace (block device) ○ nvme0n1p1: first registered device's first namespace's first partition (block device) fb: frame buffer ○ fd: (platform) floppy disks, though this same abbreviation is also commonly used to refer to file descriptor tty: terminal devices Swap File ○ Page faults cause pages to be moved from a separate partition ○ The “Swap File” partition is equivalent to pagefile.sys on Windows Understanding the EXT File System- Extended Journaling FileSystem The Extended File System (EXT) is a family of journaling file systems commonly used in Linux distributions. EXT Timeline: ○ EXT: Introduced in 1993. ○ EXT2: Introduced in 1994, it lacked journaling but supported file sizes up to 2TB and file system sizes up to 32TB. ○ EXT3: Introduced in 1999, it added journaling while maintaining backward compatibility with EXT2. ○ EXT4: Introduced in 2006, it is the most recent version and offers improved performance, larger file size support (up to 16TB), and file system size up to 1EB. It can also mount EXT2/3 file systems Key Features of EXT4: ○ Extents: This feature allows for contiguous block allocation, reducing fragmentation and improving performance. ○ Persistent Pre-allocation: Reserves disk space for files in advance. ○ Sparse Files: Efficiently stores files with large empty spaces. ○ Delayed Allocation: Improves write performance by delaying the allocation of data blocks. ○ Unlimited subdirectories ○ Improved Timestamps: Uses nanosecond precision and extends the timestamp range beyond the Year 2038 problem ○ Journal checksums File Systems Inodes Inodes: ○ are data structures used in Unix(Linux??) systems to store metadata about files and directories. ○ Each file and directory has a unique inode associated with it. ○ An inode does not contain the file name; the filename is stored in the directory entry.  Inode Content: ○ Addresses of data blocks. ○ Indirect links to additional data blocks. ○ File type and permissions. ○ Ownership information (user and group). ○ Timestamps (creation, modification, access). ○ File size. EXT4 Structure Block Groups: The EXT4 file system is divided into block groups, each containing metadata and data blocks. Metadata Repository: The superblock and group descriptor tables are crucial metadata structures that provide information about the overall file system layout. Extents: Instead of using indirect blocks for large files, EXT4 utilises extents, which represent contiguous ranges of blocks. Extent Tree: EXT4 uses an extent tree to efficiently manage extents. EXT4 Forensic Considerations Deleting and moving Deletion: ○ -> Inode: File size set to zero Timestamps set to same value ○ ->Extent index: Number of extents is zeroed But old data for index is still present ○ -> Index node is not zeroed ○ -> Extent itself is zeroed  Moving: ○ Indoe number stays the same when moved to another directory on the same device Inodes identifies uniques within file system, but not between file systems on the same machine File Recovery ○ Difficult recovery of deleted files due to zeroing of inode data ○ Some data still present in extent index ○ Carving should be fairly successful due to fragmentation avoidance strategies ○ Journal may be useful for data recovery Pre-Allocation : Sparse Files ○ Sparse files in EXT4 do not initialise data blocks by overwriting them. ○ EXT4 is lazy and simply marks the blocks and relevant nodes as allocated ○ Experiments have shown that this can preserve data in the previously unallocated sectors The data may even come from a superuser on the system, when accessed as a regular user! Implications for cloud security? Data Recovery in EXT4 Challenges: Recovering deleted files can be challenging because EXT4 zeroes inode data upon deletion. However, some residual data might remain in the extent index. Carving: File carving techniques can be effective in recovering data due to EXT4's fragmentation avoidance strategies9. Journal Analysis: The EXT4 journal can potentially aid in recovering deleted data9. Sparse Files: Pre-allocated space for sparse files can reveal remnants of deleted data9. Block Boundaries and Slack Space If data doesn’t fill up the entire sector, or cluster (file system allocation unit), then there is wasted space This space may contain previously written data ○ Very useful for forensics ○ Though not on SSDs! Block Boundaries: Files are stored in blocks, and if a file's size is not a multiple of the block size, the remaining space within the last block is called slack space Types of Slack Space: ○ RAM Slack: Unused space at the end of a sector. Modern Windows systems zero this space, but if data is found here, it might have been placed intentionally. ○ Drive Slack: The remaining empty sectors within a cluster. ○ File Slack: A combination of RAM slack and drive slack. MacOS Forensics Root Folder Structure /System: Reserved for use by Apple and contains items which are part of the OSX. /Library: Contains configuration information from 3rd party add-ons that affect all users. /Users: Contains individual user directories for each of the local users. Can also contain a Shared folder that allows sharing of documents between local users. /Network : Contains information about Open Directory and Active Directory.  /Volumes : Standard mount point for attached media. (Equivalent to Linux /mnt or /media.) /Applications: All the installed applications are kept in this folder. (NB. It is possible to run application from anywhere.) Configuration Information System based configuration is stored in the /System/Library folder. 3rd Party applications store their application in the folder /Library User specific configuration is stored within the folder /Users//Library Network information may also be found in /Network/Library if a network server is involved. Interesting Folders /private/var/log – System wide log files are stored here. /private/var/spool/cups - CUPS print spool can contain logs of past print jobs. /private/var/db/dslocal/nodes/Default – Users and Groups are defined in this folder. A Salted-SHA512 hash of the user’s password can be found here. /private/var/folders – Contains a number of cache folders created for users of the system. /private/var/root – Home folder for the root user if it is enabled. /private/var/vm – Sleepimage and swapfile are located within this folder. /private/etc – Contains additional configuration files.Including the underlying Un*x ones as well as some they have added themselves. File Systems HFS+ ○ Mac OS 8.1. 1998 ○ Added journaling Limitations of HFS+ Wasn’t designed for unix-like systems ○ File permissions retrofitted Designed for big-endian CPUs, modern CPU use little endian ○ Needs to switch byte ordering for all reads/writes No nanosecond timestamps, 2040 as max date No concurrent file access, no snapshots No sparse files Thumbnail Cache: Located in /Users/$USER/Library/Caches/com.apple.finder/Cache.db, this database stores thumbnails of files, even deleted ones, which can be helpful for investigations OTHER - File System Types: Common Unix file systems include EXT2, EXT3, and EXT4, each with its own methods for journaling and data recovery. - Directory Structure: Unix follows a hierarchical structure, starting from the root directory `/`. - Important Directories: - /etc: Contains configuration files.  - /var/log: Stores log files crucial for tracing activities. - /home: Stores user data and profiles. - File Permissions: Unix permissions define access levels (read, write, execute) for each file, essential in forensic analysis. Unix Forensic Artefacts - Log Files: Provide a record of system activity, including login attempts, system boot events, and application errors. - Process and System Logs: - syslog: General log file storing system messages. - auth.log: Records authentication events and login attempts. - Timestamps: Unix uses a simpler time-tracking system based on epoch time (seconds since January 1, 1970), making conversions necessary in investigations. DIFFERENCES: Windows Registry vs. Unix System: A Fundamental Difference The core difference between the Windows registry and a Unix system lies in how they manage system-wide configuration settings and user preferences. Windows Registry: A Centralised Database The Windows registry is a centralised hierarchical database that acts as a single repository for a vast array of information, including: ○ Hardware and Software Configuration: Details about installed hardware components, device drivers, software applications, and system settings are stored in the registry. ○ Network Connections: Network settings, active connections, and network shares are recorded in the registry. ○ User Preferences: Each user's profile settings, such as desktop background, themes, and application preferences, are stored in the registry. ○ Setup Information: Information relating to the operating system installation and updates is maintained in the registry. Structure and Terminology: ○ Hives: The registry is physically stored in files called "hives," typically located in the directory %SYSTEMROOT%\System32\config. These hives contain binary data that is structured in a hierarchical format. ○ Keys: Keys are analogous to folders in a file system and organise the registry into logical categories. ○ Subkeys: Subkeys exist within keys and allow for further organisation of settings. ○ Values: Values, analogous to files, store the actual configuration settings within keys and subkeys. Each value has a name, data type, and the actual data. Root Keys: The Windows registry is organised under several root keys, including: ○ HKEY_LOCAL_MACHINE: Contains configuration information for the entire system, regardless of the user logged in. ○ HKEY_USERS: Stores user-specific information for all user profiles on the system. ○ HKEY_CURRENT_USER: A subset of HKEY_USERS that reflects the settings for the currently logged-in user. Unix Systems: A Decentralised Approach In stark contrast to the Windows registry, Unix systems, including Linux and macOS, take a decentralised approach to storing system and user configurations: Configuration Files: Instead of a centralised registry, Unix systems rely on a multitude of individual configuration files, typically plain text files, scattered throughout the file system. Locations: These files are typically found in standardised locations within the directory tree: ○ /etc/: System-wide configuration files for services, applications, and network settings are often stored here.  ○ /usr/local/etc/: Configuration files for locally installed software are frequently found here. ○ /home/$USER/: User-specific configuration files are typically stored in the user's home directory, often within hidden directories starting with a dot (e.g.,.bashrc). Flexibility: This decentralised approach allows for great flexibility and control, as individual settings can be easily modified by editing the corresponding configuration files. Pros and Cons: A Comparative Glance Feature Windows Registry Unix Systems Centralization Centralised, single database Decentralised, many individual files Ease of Can be easily managed through Requires editing text files, may Management GUI tools need root privileges Security Access control can be enforced Security depends on file system permissions Scalability Can become large and complex Remains manageable as the system grows Transparency Binary format, less transparent Plain text files, more transparent Forensic Valuable source of evidence, can Log files and configuration files Implications reveal user activity provide evidence