79 Questions
What is the primary objective of model hardening through strategic feature selection?
Maximize the performance of the model
In adversarial feature selection, what is done with feature set F in the initial step?
Features are randomly selected
What is a key consideration for security by design feature selection?
Choosing features that are hard to tamper with
What happens when a feature is removed in adversarial feature selection?
Model performance degrades significantly
Why are feature selection algorithms mentioned to be susceptible to poisoning attacks?
Because they can be manipulated during training
What is one of the downsides to retraining models for model hardening?
Increased model training costs
In Defensive Distillation, what is the main purpose of training a simpler model (student) on the softmax layer of the complex model (teacher)?
To capture only relevant patterns from the teacher model
How does Data Sanitization through preprocessors like Denoising Auto-Encoders help in defending against adversarial attacks?
By reducing the dimensionality to limit the attacker's search space
What did Wagner and Carlini find regarding Defensive Distillation as a defense to adversarial perturbations against deep neural networks?
It is easily broken and not robust to adversarial examples
How do techniques like GAN Based dimensionality reduction help in defending against adversarial attacks?
By reducing the dimensionality to limit the attacker's search space
Which of the following methods uses an outlier robust loss function to reduce learner sensitivity?
Huber
What is the key principle behind the TRIM method for model hardening?
It focuses on samples with low residuals, minimizing the loss on a subset of the data.
What is the intuition behind the Neural Cleanse method for detecting backdoors?
Samples from a class with a backdoor should be easier to perturb with a small perturbation.
What is the key observation that motivated the development of methods like TRIM and RONI for model hardening?
Subsets of the data are less affected by poisoning attacks, especially for small poisoning rates.
Which of the following statements is true about the use of stochastic gradient descent (SGD) for mitigating poisoning attacks?
SGD is effective against poisoning attacks because it breaks the consistency of adversarial points.
What is a common technique used by defences to prevent exploration of gradients in adversarial examples?
Making the gradient noisy to misdirect search
Which of the following methods can be used to evade defences that hide gradients according to the text?
Attack each component simultaneously
In the context of model hardening, what is the purpose of iterative retraining (adversarial retraining) according to the text?
To optimize the model considering potential attacks from adversaries
Which type of defense strategy is considered one of the most robust according to Madry in the text?
Modeling the game via loss directly
What is an essential step in the iterative retraining process mentioned in the text?
Continuously replacing parts of the dataset
What is the main purpose of the retraining process described in the text?
To make the model more robust to adversarial attacks
What is the effect of using a high-capacity network during the retraining process?
It improves the model's generalization to other types of attacks
What is the key insight regarding the selection of attacks used during the retraining process?
Retraining on the strongest available adversary is the best strategy
What is the potential drawback of using a low-capacity network during the retraining process?
It decreases the robustness of the model to adversarial attacks
What is the relationship between model capacity and the transferability of adversarial examples, as discussed in the text?
Increasing model capacity decreases the transferability of adversarial examples
What is the primary objective of the Reactive Arms Race approach?
The defender responds to attacks by deploying defenses after an attack has occurred.
Which principle is emphasized in the Proactive Arms Race approach?
Considering the attacker's potential next steps and defending against anticipated threats.
According to the Principled Approach to Secure Learning, what is the first step?
Threat modeling to enumerate threats, identify the attack surface, and understand the threat actors.
What is the primary goal of conservative design in the context of adversarial machine learning?
Limiting the attacker's options and restricting their potential actions.
According to Kerckhoffs's Principle, what should not be relied upon for security?
Obscurity or secrecy of the defense mechanisms.
In the context of adversarial machine learning, what is the primary goal of proactive analysis?
Considering the attacker's potential next steps and raising the difficulty bar for them.
What is the primary focus of the triage process mentioned in the context of the Proactive Arms Race?
Identifying the most severe threats and addressing them first.
Which statement best describes the motivation behind the Proactive Arms Race approach?
The defender considers the attacker's potential future actions and defends against anticipated threats.
What is one of the key considerations in threat modeling?
Identifying the attack surface, including the model and system components.
Which principle emphasizes avoiding unnecessary assumptions about the attacker's capabilities or methods?
Conservative design to limit the attacker's options.
Which of the following is the primary goal of the defender in a Stackelberg game scenario described in the text?
To choose the optimal leader strategy that minimizes the defender's payoff
Which of the following is a key challenge with using data sanitization as a defense against causative attacks, as described in the text?
Data sanitization can negatively impact the performance on benign samples
What is the primary limitation of using model hardening as a defense against causative attacks?
Model hardening provides little to no guarantees against attacks
In the context of the 'Exec Mode' equation presented in the text, what does the term '$ extbackslash lambda c(x, x')$' represent?
The constraint that enforces the attacker's feature vector to be within a certain distance from the original input
What is the main approach to defending against Black-Box Membership Inference Attacks as discussed in the text?
Adding Laplacian or Gaussian noise to the prediction vector
How is the concept of k-differential privacy defined in the context of machine learning privacy?
Ensuring no single observation is more important than others within a certain bound
What is the primary drawback of achieving sufficient levels of differential privacy in deep neural networks?
Severely harming DNN accuracy
How can machine learning models defend against membership inference attacks through property inference?
By limiting the length of the confidence vector returned and adding noise to it
What are some of the other approaches discussed in the text to increase privacy and security in machine learning models?
Increase regularization, generalization through dropout, optimization terms, and model ensembles/stacking
What is the main characteristic of a Stackelberg Equilibrium in the context of Game Theory?
Leader makes the best possible action given all possible attacks
What is the key challenge in optimizing defense strategies against adversarial attacks in practice?
Optimization is hard
In the context of a Stackelberg Equilibrium, what is the primary goal of the defender?
Performing damage control
What is the main difficulty the defender faces when trying to reach equilibrium with an attacker in the context of adversarial machine learning?
Choosing a defense that covers as much ground as possible
What is the primary concern for the defender when an attacker can potentially bypass the defense mechanisms put in place?
Attacker can just side-step the defense
What is the common mistake made by defenders in the context of game theory according to Yisroel Mirsky?
Defenders assume a reverse Stackelberg game and select known attack methods to develop defense against.
What strategy did the attacker use to outsmart the defender in protecting a DNN from PGD according to Dr. Yisroel Mirsky?
The attacker included the defense in the loss function or gradually increased the delta to find covert solutions.
What advice does Yisroel Mirsky give when developing a new defense in game theory?
Consider techniques that limit the adversary and discuss what adversaries can do to evade the defense.
What is the main takeaway regarding defenses in game theory according to X. Li et al.?
Nearly any defense that has a gradient can be evaded by attackers.
In the context of adversarial machine learning, why are feature selection algorithms mentioned to be susceptible to poisoning attacks?
Feature selection algorithms can be manipulated by attackers to introduce vulnerabilities or biases.
What is the generic defense strategy known as iterative retraining also referred to as?
Adversarial Retraining
According to Madry, which defense strategy is currently considered one of the most robust?
Iterative Retraining (Adversarial Retraining)
What is the purpose of iterative retraining (adversarial retraining) in model hardening?
To find the model parameters that minimize loss while considering the attacker's optimization strategy.
How can defenses that hide gradients be evaded according to the text?
By using techniques like trying repeatedly, using surrogate loss functions, and attacking each component separately.
What is the key principle behind the TRIM method for model hardening?
Removing features that are most vulnerable to attacks.
What is the purpose of Data Sanitization in the context of model hardening?
Detect and remove
Explain the concept of Asymmetry in the context of adversarial attacks.
Attacker normally chooses one Defender must consider all samples
In the context of Game Theory, what is the Stackelberg Game and who are the players involved?
Two players: leader and follower
What is the main principle emphasized in Kerchoff's principle in the context of model security?
Little/no guarantees, Kherchoff's principle
Explain the concept of False Positives in the context of Machine Learning Model Hardening.
False positives are detrimental
What is the key principle behind Kerckhoffs's Principle in the context of secure learning?
Obscurity is not security
What is the primary goal of the Proactive Arms Race approach in adversarial machine learning?
Raise the difficulty bar for the attacker on all fronts
What is the main purpose of conservative design in the context of secure learning?
Limit the attacker’s options
According to the Principled Approach to Secure Learning, what is the first step in ensuring security in machine learning models?
Threat modelling
What is one of the downsides to retraining models for model hardening in adversarial machine learning?
Overfitting to the adversarial examples
What are the two main defence approaches mentioned by Dr. Yisroel Mirsky in the text?
Data Sanitization and Model Hardening
What does LOF stand for in the context of Outlier Detection?
Local Outlier Factor
What is the downside of Data Sub-sampling as a defence mechanism according to the text?
Expensive, requires small alpha and large subsampled data
What is the main purpose of Data Sanitization through preprocessors like Denoising Auto-Encoders in defending against adversarial attacks?
Preventing data corruption
What is the negative impact that Defence Risk (RD) measures?
The negative impact D2 has after modifying f or cleaning D1
What is the significance of Strategic Feature Selection in Model Hardening?
It reduces learner sensitivity
What is the role of Data Provenance in the context of Security according to Dr. Yisroel Mirsky?
Protecting data
What does the term 'Data Integrity' (I) focus on in the context of Defence Risk?
Presence of tampered data in D2 with respect to D0
What is the main concern addressed by Model Hardening through iterative retraining according to the text?
Defending against adversarial attacks
What is the primary goal of Security measures like Digital Signatures in defending against attacks?
Protecting data, model, hypotheses, and theta
Study Notes
- Train Mode, Rest Mode, and Exec Mode present different attack vectors and defense strategies in machine learning models.
- Model Hardening involves strategic feature selection to maximize model performance and minimize attacker impact.
- Game Theory concepts like Stackelberg Game are used in developing defense strategies against attacks.
- Model Hardening techniques include Adversarial Feature Selection and Security by Design Feature Selection to protect models from attacks.
- Downside to retraining models includes increased cost, harm to generalization, and vulnerability to membership inference attacks.
- Transformer Defensive Distillation involves training a simpler model on the knowledge of a more complex model to enhance robustness.
- Data Sanitization methods like Compression and Reduce Learner Sensitivity help limit attacker search space and improve model performance against attacks.
- Generic Defense Strategies like Iterative Retraining and Neural Cleanse are used to make models more resistant to adversarial attacks.
This quiz covers the concept of privacy attacks in machine learning, focusing on model hardening and the downsides of retraining. Topics include the increased cost of training models, negative impacts on generalization, vulnerability to membership inference, and the risk of overfitting to specific samples. Reference is made to Papernot et al.'s work on defensive distillation against adversarial perturbations.
Make Your Own Quizzes and Flashcards
Convert your notes into interactive study material.
Get started for free
