Lecture 10 - OAI Attack Tools.pdf

OFFENSIVE AI LECTURE 10: ATTACK TOOLS – CYBER SECURITY Dr. Yisroel Mirsky [email protected] Today’s Agenda   Confidentiality  Local Tools (Spyware)  Remote Tools Cyber Security Wide variety of OAI tasks Today we will get a taste Dr. Yisroel Mirsky 2 3 Overview OAI Confidentially Local Attack Too...

OFFENSIVE AI LECTURE 10: ATTACK TOOLS – CYBER SECURITY Dr. Yisroel Mirsky [email protected] Today’s Agenda   Confidentiality  Local Tools (Spyware)  Remote Tools Cyber Security Wide variety of OAI tasks Today we will get a taste Dr. Yisroel Mirsky 2 3 Overview OAI Confidentially Local Attack Tools Remote Attack Tools Data collected locally Data collected remotely Dr. Yisroel Mirsky Dr. Yisroel Mirsky 4 Local Attack Tools (Spyware) Dr. Yisroel Mirsky 5 Spyware What is spyware?  A type of malicious software (malware)  Installed without an end user's knowledge  Steals sensitive information and relays it to external users 6 Conventional Spyware Spyware What can Spyware do?  Steal files/media  Record audio/video  Record calls  Track user location  Track user browsing  Record Keystrokes  Passwords  Emails  Documents Dr. Yisroel Mirsky Dr. Yisroel Mirsky 7 Spyware Side Channels A source of information leakage from system flaws/oversights  Sound  Visual  Heat  Radio (EMR)  Timing  Resource access patterns Dr. Yisroel Mirsky 8 Spyware Classic vs Modern Side Channels Classic Modern Today, complex subtle singals are ‘tortured’ to reveal latent information Dr. Yisroel Mirsky 9 Spyware How Does AI Affect Spyware?  Some side channels are complex, analogue, and multi-modal  Machine learning can be used to decode them Secure or not Secure? That is the question... We download apps and limit their access (e.g., Android permissions)  Devices let apps access ‘safe’ resources without permission   E.g., phone acceleration -> location tracking Even with permission, some sensors can be abused  E.g., note app uses microphone -> infer user location Monoco. J. SoK: Keylogging Side Channels. 2018 10 Spyware: Keyloggers Keyloggers Malware that records your keystrokes Channels for Sensing Keystrokes:  Acoustic: microphones  Seismic: accelerometers  Electromagnetic: magnometers, antenna (built into smartphones)  Visual: cameras  Thermal: thermal cameras Monoco. J. SoK: Keylogging Side Channels. 2018 11 Spyware: Keyloggers Keyloggers Steps 1. Keystroke Identification When a key is pressed Binary classification over sliding window 2. Acoustic example: Key Identification Which key was pressed Multiclass classification on window 3. Text reconstruction Correct the text (“spellcheck”) Dictionary, LLM, Markovian Model Monoco. J. SoK: Keylogging Side Channels. 2018 12 Spyware: Keyloggers Keyloggers Key Identification: Triangulation   At least three perspectives are needed to localise a point in 2D The points must be static (unmoving) to model them First Order: Classic Monoco. J. SoK: Keylogging Side Channels. 2018 13 Spyware: Keyloggers Keyloggers Key Identification: Signature (uniqueness)  Each key sounds different  Distance  Shape of button  Location in frame  How hard it is usually pressed Second Order: Hard! Monoco. J. SoK: Keylogging Side Channels. 2018 14 Spyware: Keyloggers Keyloggers Key Identification: Temporal Side Channels “HELLO” → H---E--LL-O ML Features  Timing between keystrokes  Duration of keystroke  Sequence of intervals  Stress on keystroke  User profiling: changes by person Yildiran U. Et al. SIA: Smartwatch-Enabled Inference Attacks on Physical Keyboards Using Acoustic Signals. 2021 15 Spyware: Keyloggers Acoustic SIA Audio captures from smartwatch or phone call MFCC Features and SVM are used to classify sounds Wang C. Et al. Friend or Foe? Your Wearable Devices Reveal Your Personal PIN. 2016 16 Spyware: Keyloggers Seismic (motion)  Phones & wearable devices have accelerometers  Accelerometers do not need special permissions  Can be used to infer user activity Attack Vectors:  Phone left on desk   Collect keyboard strikes Phone screen tapped  Collect touch screen interactions Taps, gestures,... Dr Yisroel Mirsky 17 Seismic (motion) Example Background Foreground TEST Spyware: Keyloggers 6 1 execute mode MTL App Market 2 5 TRAIN Host App train mode 3 4 Euclidean Error in Scale Cardaiolo M, et al. Hand Me Your PIN! Inferring ATM PINs of Users Typing with a Covered Hand. 2021 18 Spyware: Keyloggers Visual  Known that you can infer keystrokes from video Clearshot: Eavesdropping on keyboard input from video. 2008 What if the keys are obscured? Hand Me Your PIN  Keystroke Detection   Acoustic or ‘*’ on screen Key Identification  Video: CNN-LSTM Model Dr. Yisroel Mirsky 19 Spyware: Tracking Tracking   Conventional Spyware track location with  WiFi SSIDs  GPS  Mobile network (cell IDs) Machine learning can leverage side channels:  Motion (prediction where the user is going)  Sound (predict where the user is) Dr. Yisroel Mirsky 20 Spyware: Tracking Tracking: Acoustic Echolocation What if you could see like a bat? Pavlovic M. Et al. Room sound field characterization and clustering using global multifractal parameters. 2019 21 Spyware: Tracking Tracking: Acoustic Echo Analysis 1. Send Excitation signal (covers all frequencies) 2. Record response 3. Extract impulse response (IR): 𝐹𝐹𝑇 𝑋𝑟𝑥 𝐹𝐹𝑇 𝑋𝑡𝑥 IR can be used to:  Locate within room  Identify room Song Q. Ewt al. Deep Room Recognition Using Inaudible Echos. 2018 22 Spyware: Tracking Tracking: Acoustic Example Response only Hua j, et al.We Can Track You If You Take the Metro: Tracking Metro Riders Using Accelerometers on Smartphones. 2015 23 Spyware: Tracking Motion User Tracking  Attacker accessed accelerometer  Correlates it to commuter trips Dr. Yisroel Mirsky 24 Remote Attack Tools Dr. Yisroel Mirsky 25 External Tools A Person’s Digital Fingerprint In 2020, each person generated 1.7 MB of data  Twitter, facebook, instragram,..  Emails  IoT, Connected cars and devices  Web Searches  WhatsApp messages, emails  Wearable devices This data can be used to ...  Track users  Iinfer Private information about those users Dr. Yisroel Mirsky 26 Surveillance Surveillance Since 2000 the US has used AI to annotate and select phone calls [Snowden Documents] Speech to Text  NLP Classifers China uses a network of cameras to track citizens  Object recognition  Licence plate detection  Facial recognition ... https://cs.stanford.edu/people/eroberts/cs181/projects/ethics-of-surveillance/ethics.html 28 Surveillance Ethics of Surveillance "If you haven't done anything wrong, you have nothing to fear."   If everyone  Carried mandatory trackers  Had all calls and internet recorded  All financial and medical records open...then it would be easy to catch criminals So why not? https://cs.stanford.edu/people/eroberts/cs181/projects/ethics-of-surveillance/ethics.html 29 Surveillance Ethics of Surveillance Surveillance can be......invasive, prone to abuse, vulnerable to cyber breaches Examples of abuse  a Troy police lieutenant, had ran her new husband through the background check system  A special agent of the US Department of Commerce, was indicted for tracking the travel patterns of an ex-girlfriend  [2020-2022] ClearView AI, data breach Prof. Gary Marx 30 Surveillance When is Surveillance Ethical? Evaluate with these questions (abridged)    How is it done?  will collection cause physical/phycological harm?  Will the data be protected, used right, collected in an impersonal way? What is the context?  Is there consent?  Golden Rule: would the surveillers also want to be surveilled? What is it for?  Will it survey the community or an individual?  Does the good outweigh the bad? Dr. Yisroel Mirsky 31 Inference Inference  Can train model to predict sensitive information  Apply to new users to reveal their personal lives May be used for: targeted marketing or blackmailing This concept sets a dangerous predicent Gender from Text Do you smoke? When? Sexual orientation from your face Predicting health from from wearables Where you live from tweets https://www.forbes.com/sites/kashmirhill/2012/02/16/how-target-figured-out-a-teen-girl-was-pregnant-before-her-father-did/?sh=6b68446a6668 32 Inference Inference Purchasing history can be used to predict personal information Schuster R. Et al. Beauty and the Burst: Remote Identification of Encrypted Video Streams. 2017 33 Inference Evading Encryption  ML is great at reconstructing latent (hidden) information  Even encryption has subjected to side channels Beauty and the Burst Predict what is being watched on Netflix, Youtube,...by monitoring traffic rates  Video compression changes depending on amount of change between frames  Changes in bandwidth can be monitored  CNN used to classify video being watched  Similar attacks work on video surveillance Vastel A. Et al. FP-STALKER: Tracking Browser Fingerprint Evolutions. 2018 34 Inference Evading Encryption Browser Fingerprinting Attackers detect what version browser you are using to  know which malware to use  track where your visit FP-STALKER: Tracking Browser Fingerprint Evolutions  Uses random forests to track eveolving fingerprints  Works on proxied traffic (such as Tor) Hitaj B. Et al. PassGAN: A Deep Learning Approach for Password Guessing. 2019 35 Inference Evading Encryption Password Guessing  Many passwords are not random  Password leaks show people use  Common words  Birthdays  Phone numbers  Names ... PASSGAN   Use a GAN to help generate password possibilities  Train: password leak DBs  Exec: generate random possibilities Other works use personal information to generate personalized passwords guesses Dr. Yisroel Mirsky 36 Take-aways 1. 2. Machine learning enables more ways to violate confidentiality  Implicit signals can reveal explicit information  Partial info on an individual can be completed using info from [many] others Not every ML attack on confidentiality is practical  Keyloggers: malware must be on phone, have privileges  Echo location: need audio dataset of location before the attack  … Dr. Yisroel Mirsky 37 OAI Cybersecurity of Organizations Dr. Yisroel Mirsky 38 Cybersecurity & Organizations Organization: An organized body of people with a particular purpose  Business  Institution  Government 39 An attack, via cyberspace, targeting an organization’s use of cyberspace for the purpose of...  disrupting,  disabling,  destroying,  controlling, or  stealing...assets, infrastructure, services, etc. Dr. Yisroel Mirsky Cyber Attacks 40 Kill Chains APT: Advanced Persistent Threat Dr. Yisroel Mirsky 41 OAI and the CKC What does AI have to do with this?  20 years of AI for defence  Recent pivot: attackers now use AI  Accessibility → cheap  Effective → profitable Conventional: Manual Tools Conventional adversaries use manual effort, common tools, and expert knowledge AI-capable adversaries use AI to automate tasks, enhance its tools, and evade detection Modern: AI Powered Tools Near Future: Attack Automation Dr. Yisroel Mirsky 42 OAI and the CKC The Three Motivators of using Offensive AI: Coverage Speed Success AI enables an adversary to... ...scale up activity  ...automate campaigns   Automation decreases human labor (cost) E.g., phishing, OSINT collection & analysis...cover more ground  Easier to manage campaigns in parallel Coverage Summary: AI enables attackers to hit organizations with higher precision and smaller workforces Dr. Yisroel Mirsky 43 OAI and the CKC The Three Motivators of using Offensive AI: Coverage Speed Success AI enables an adversary to... ...reach goals faster  E.g., ML can be helped to extract credentials or find zero-day vulnerabilities  Less time in a network means  More time in another organization’s network  Less time to be detected Speed Summary: AI enables attackers to hit more organizations annually Dr. Yisroel Mirsky 44 OAI and the CKC The Three Motivators of using Offensive AI: Coverage Speed Success AI enables an adversary to... ...increase its likelihood of attack success  Stealth: evade detection e.g., camouflage traffic  Utility: improve attacks e.g., identify social engineering targets & vulnerabilities  Novelty: exploit weaknesses (new attack vectors) e.g., deepfake spear phishing  Effectiveness: Plan optimial attack strategies Best Hacker Success Summary: AI gives attackers a higher likelihood of success, encouraging more attacks as a result Dr. Yisroel Mirsky 45 Offensive AI Capabilities (OAC) Back to the Cyber Kill Chain... If you take all of the AI-based capabilites which empower the adversary in the CKC, you can group them into seven categories: 1. Automation 2. Campaign Resilience 3. Credential Theft 4. Exploit Development 5. Information Gathering 6. Social Engineering 7. Stealth Materialized as...  AI-based Tools  AI-driven Bots Dr. Yisroel Mirsky 46 Quiz: (1) Automation Scenario An attacker can have his malware make take screenshot of the victim’s desktop Objective: Exfiltrate sensitive information Challenge: Cannot send images (too much bandwidth!) What is a good OAI approach?  Use a DNN to perform OCR How would you defend?  Look for DNN signatures: large models in binary, CPU usage,... Dr. Yisroel Mirsky 47 Quiz: (2) Campaign Resilience Scenario Bots inside the Org’s network need to find the attacker’s control server This is done by performing a DNS query: “what is the current IP of www.botnet4ever.com” Objective: Make an algorithm which all bots can synchronously use to find/generate the current domain names (periodically updated) Challenge: Cannot use random strings (too obvious) What is a good OAI approach?  Use GAN to match real website domains + noise  If defender uses a classifier, use adversarial ML on a normal domain name How would you defend?  Monitor DNS query patterns on a daily basis Dr. Yisroel Mirsky 48 Quiz: (3) Credential Theft Scenario A hacker wants to let his buddies sneak through a restricted area Objective: Disable the battery powered surveillance cameras Challenge: Cannot make false alarms (e.g., DDoS via FPs) What is a good OAI approach?  Use patch to perform sponge attack to drain battery  Use patch to remove all people from scene How would you defend?  Search for patches (user awareness)  Monitor semantic patterns in models, or splice the inputs Mirsky et al. VulChecker: Graph-based Vulnerability Localization in Source Code. 2023 49 Quiz: (4) Exploit Development Scenario An attacker knows that a company uses a popular open source library for processing emails Objective: find a zero-day vulnerability that can be exploited in a phishing attack Challenge: The software is massive/complex What is a good OAI approach? Dr. Yisroel Mirsky 50 Quiz: (5) Information Gathering Scenario Company A uses a DL Network Intrusion Detection System by company B Objective: breach company A without being detected Challenge: the DL NIDS is very good What is a good OAI approach?  Breach B and steal the model  Generate evasive samples (whitebox attacks) to breach A How would you defend?  Protect B with same model  Rotate Defences in A randomly Dr. Yisroel Mirsky 51 Quiz: (6) Social Engineering Scenario A hacking group wants to automate email phishing on an organization Objective: craft emails that seem like they were written by person A Challenge: the hackers have no time to do it themselves (wide phishing) What is a good OAI approach?  Obtain text from A taken from the web or stolen DBs, then use style transfer How would you defend?  Monitor the metadata (e.g., where was the email sent from?) Hashemi M. Et al. Towards Evaluation of NIDSs in Adversarial Setting. 2019 Rosenberg I. Et al. Generic Black-Box End-to-End Attack Against State of the Art API Call Based Malware Classifiers. 2018 52 Offensive AI Capabilities (OAC) (7) Stealth Methods to Evade Detection: (1) Feature Evasion Mimicry NIDS: PAYL (2) Mimicry (2) Adversarial Examples Feature Evasion Adversarial Example (Host-based Intrusion Detection System) (Network-based Intrusion Detection System) Dr. Yisroel Mirsky 53 Outlook Dr. Yisroel Mirsky 54 Outlook Survey  The threat of OAI in the eyes of academia, government, and industry (35 participants)  19 from Gov & Industry   CISO, Founders, Malware Analysts, AI/Security Researchers, AI Ethics specialists 16 from Academia  Professors and PhDs in cyber security and AI Examples from Industry The rest are anonymous Examples from Academia The rest are anonymous Dr. Yisroel Mirsky 55 Outlook Survey  For each OAI, they were asked to rank (1 low : 7 high) Profit (P): Benefit an attacker gains by using AI  Attack success, flexibility, coverage, automation, persistence,...  Assumes AI tool already implemented Achievability (A): How easy it is for the attacker to use AI for this task  Considering: training, testing, deployment, monetary cost Defeatability (D): How easy it is for the defender to detect or prevent Harm (H): The amount of damage an AI-capable attacker can inflict  Physical, physiological, monetary, cost of new defences,... All 1-7 scores are normalized to [0,1] Dr. Yisroel Mirsky 56 Outlook Survey Measuring Threats How do you measure a Threat? Threat = Harm x Risk 𝑇 = 𝐻𝑅 High Threat: A high likelyhood of occurring (risk) and if it does, will cause significant damage Low H or R cancels the threat In summary: How do you calculate Risk (R)? Risk = Motivation/Defeatability 𝑅= 𝑀 𝐷 How do you calculate Motivation (M)? Motivation = Avg(Profit, Achievability) 𝑀= 1 𝑃+𝐴 2 High Risk: An attack is likely to occur if there is high motivation and it is hard to defeat the attack (won’t be blocked) If 𝑇 > 1 then the threat is significant High Motivation: When profit is high and achievability (cost) is low (and vice versa) Low P or A does not cancel the motivation It means 𝑀 > 𝐷 And the harm is significant w.r.t. The likelihood of occurrence 57 Outlook Survey Results 72% of the OACs are significant threats (T > 1) Top: SE – easy to achieve, most harm, hard to defeat,... Dr. Yisroel Mirsky 58 Top 10 Threats by perspective Outlook Survey Results Both are concerned about Social Engineering Industry is more concerned about OAI software threats Dr. Yisroel Mirsky Dr. Yisroel Mirsky 59 Outlook Industry's Perspective on the Problem  Already observing some advanced malwares that ‘learn’  Automated offensive tactics hard to combat (overwhelm) US National Security Commission Report 2021 “The U.S. government is not prepared to defend the United States in the coming artificial intelligence (AI) era” “Because of AI, adversaries will be able to act with micro-precision, but at macro-scale and with greater speed. They will use AI to enhance cyber attacks and digital disinformation campaigns and to target individuals in new ways” Awareness and Preparation  49% of 102 cybersecurity orgs. See AI as imminent threat [Forrester 2020]  96% of 306 surveyed are already investing in defending against automated attacks Dr. Yisroel Mirsky 61 Extra ML Tasks used Offensively Examples: Examples: Analysis Decision Making Evasion Mining/extracting useful information Plan development & coordination Adversarial ML  XAI to hide artifacts  Target ranking  Clustering to ID targets  Botnet coordination  Avoid H/NIDS  Bypass Biometrics  Disable Models (DoS) Generation Prediction Retrieval Creating content Using historic data to enhance activities Finding semantically similar content  Tamper records  Impersonation  Password guessing  Implicit keyloggers  Vulnerability detection  User tracking  Summarize OSINT (surveillance)