Malware Analysis Techniques Quiz

CYB. Defensive AI (part 3)

• Course: Master in Artificial Intelligence
• Year: 2024/25
• Institution: ESEL – University of Vigo

AI/ML in Malware Analysis

• Malware is malicious software designed to harm, exploit, or compromise computer systems and data.

Malware: Definition and Types

• Malware can be a mixture of different types.
• Self-replicating:
  • Viruses replicate when infected files execute. Examples include Stuxnet.
  • Worms spread across networks without user interaction. (SQL Slammer is an example).
• Auto-hiding malware:
  • Trojans disguise as legitimate but contain malicious code (like backdoors or data theft). Examples include Qbot/Qakbot, TrickBot.
  • Rootkits hide malicious software, making detection or removal difficult. Examples include Linfo, Pandora, HIDEDRV.
• Designed to harm:
  • Ransomware encrypts a victim's files and demands ransom for decryption (e.g., CryptoLocker, Phobos/Dharma).
  • Botnets are networks of compromised computers used for malicious activities (e.g., DDoS attacks, spam, Mirai, Andromeda).
  • Logic/time bombs are malicious code activated under specific conditions causing system damage.
  • Keyloggers record keystrokes to capture sensitive information.
  • Cryptojacking uses computers for cryptocurrency mining. Example: Kinsing, LoudMiner.
  • Spyware collects information without consent (e.g., CoolWebSearch, Gator).
  • Adware shows unwanted advertisements and collects user data. (e.g. Fireball, Appearch).

Malware Analysis

• Understanding the behavior and purpose of suspicious files is key.
• Static analysis:
  • Examines malware code and characteristics without executing it.
  • This involves studying file structure, strings, metadata, and embedded resources.
  • Identifies known patterns, signatures, indicators (like file names, hashes, strings, IP addresses, domains, and file headers).
  • Tools used for static analysis include disassemblers and static rules (example: Yara Rules).
  • Static analysis can effectively detect known malware via signature-based approaches or heuristic analysis.
  • However, static analysis may miss sophisticated or polymorphic threats.
• Dynamic analysis:
  • Executes malware in a controlled environment (sandbox), observing its actual behavior.
  • This is crucial for preventing harm to the host system.
  • Dynamic analysis monitors file system access, changes, network communication (e.g., TCP, DNS), and system calls.
  • Dynamic analysis is helpful for identifying unknown or evolving malware.
  • A drawback is that dynamic analysis is often resource-intensive.

Malware Analysis (III)

• Resources and online sandboxes:
  • MalwareBazaar, VirusShare.com
  • Microsoft Malware Classification Challenge (BIG 2015) (Kaggle).
  • Cuckoo Sandbox,
  • Mobile Security Framework (MobSF)
  • Joe Sandbox & tools reports
  • Hybrid Analysis, VirusTotal (VT APIv3)

Typical features in Malware analysis

• Static features:
  • Opcode Sequences (binary code operation codes).
  • API Import and Export Functions (API calls for malicious tasks).
  • File Metadata (size, creation dates, certificates).
  • String Analysis
  • Control Flow Graph (CFG) (flow of code).
  • File Headers and Sections (e.g., Portable Executable (PE) headers in Windows).
  • Image Representation (visual patterns in malware).
  • Permissions and Manifest Information (Mobile malware).
• Dynamic features:
  • API Call Sequences & Frequencies (malware system calls).
  • Memory Access Patterns.
  • Network Traffic Patterns (e.g., communication with malicious sites).
  • System Call Behavior (specific system calls more frequent in malware than benign programs).
  • Persistence Mechanisms (startup entries, scheduled tasks).
  • Registry Operations

Microsoft Malware Classification Challenge (BIG 2015)

• Data: Contains ≈400GB of raw binary data and metadata.
• Problem: Multiclass problem with 9 types of malware.
• Dataset: Available at https://www.kaggle.com/c/malware-classification
• Paper: Available at https://arxiv.org/pdf/1802.10135.pdf
• 9 types of malware families in the dataset, with number of train samples for each

AI/ML in Intrusion Detection

• Intrusion Detection/Prevention Systems (IDS/IPS) monitor for dangerous activities.
  • IDS detects and alerts (passive).
  • IPS detects and blocks (proactive).
• IDS types:
  • Network-based IDS (NIDS) monitors network traffic (e.g., Snort, Suricata, Zeek).
  • Host-based IDS (HIDS) monitors host activity like file system, system calls, logs (e.g., Fail2Ban, OSSEC/Wazuh).
• Signature-based IDS: based on known attack patterns or signatures.
• Behavior-based (Anomaly-based) IDS: based on deviations from "normal" baseline.

Behavior-based / Anomaly-based IDS/IPS

• Anomalies are unexpected events
• Data exfiltration, malware activity (e.g., ransomware, virus), botnet activity, etc.
• Baseline establishment: define typical/acceptable behavior by analyzing historical data.
• Behavioral profiling: continuously monitors and profiles user/system behavior.
• Monitor data transfer volumes, protocol usage, system resource usage, login times, frequency, ...

Anomaly detection techniques

• Outlier detection: finding data points significantly different from the majority.
• Novelty detection: finding instances significantly different from training data.
• Types of Anomalies:
  • Point Anomalies: Individual data instances.
  • Contextual Anomalies: Abnormal behavior in a specific context.
  • Collective Anomalies: A set of data points exhibiting anomalous behavior.

Anomaly detection techniques (III and IV and V)

• Various techniques and tools are used, including:
  • Features engineering (metrics/signals from host and OS activity).
  • OS instrumentation (e.g., OSquery), Cross platform endpoint instrumentation (e.g. Audit Daemon).
    • OS signals (Running processes, Active/new user accounts, Permission changes, DNS lookups, Network connections, Kernel mods, System scheduler, Startup, Daemon…etc).
  • Network intrusion detection (features from traffic).
  • Traffic metadata, Aggregated info, Protocol analyzers, Web/application intrusion detection (features from logs).

NIDS Datasets

• NSL-KDD Dataset: improved benchmark for intrusion detection.
  • Collected over ~9 weeks on a simulated network.
  • ~4.9M connection records; raw PCAP captures, ~41 processed high level features.
  • 22 attack types categorized into four broad groups (dos, unauthorized access, privilege escalation, and probing attempts).
• Criticisms and limitations of the KDD Cup 1999 dataset: outdated, limited, lack of context.
• Alternative Datasets: UNSW-NB15, CIC-IDS2017, CSE-CIC-IDS2018, and UGR’16. These can offer a more up to date representations of real-world attacks.