Exam C Study Notes on LLM Security
39 Questions
0 Views

Choose a study mode

Play Quiz
Study Flashcards
Spaced Repetition
Chat to lesson

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

Which prompting attack directly exposes the configured behavior of a large language model (LLM)?

  • Prompted persona switches
  • Extracting the prompt template (correct)
  • Ignoring the prompt template
  • Exploiting friendliness and trust

    • A company wants to use Amazon Bedrock. The company needs to review which security aspects the company is responsible for when using Amazon Bedrock. Which security aspect will the company be responsible for?

  • Provisioning Amazon Bedrock within the company network
  • Patching and updating the versions of Amazon Bedrock
  • Securing the company's data in transit and at rest (correct)
  • Protecting the infrastructure that hosts Amazon Bedrock

    • A social media company wants to use a large language model (LLM) to summarize messages. The company has chosen a few LLMs that are available on Amazon SageMaker JumpStart. The company wants to compare the generated output toxicity of these models. Which strategy gives the company the ability to evaluate the LLMs with the LEAST operational overhead?

  • Crowd-sourced evaluation
  • Model evaluation with human workers
  • Reinforcement learning from human feedback (RLHF)
  • Automatic model evaluation (correct)

    • A company is testing the security of a foundation model (FM). During testing, the company wants to get around the safety features and make harmful content. Which security technique is this an example of?

    <p>Jailbreak (D)</p> Signup and view all the answers

    A company needs to use Amazon SageMaker for model training and inference. The company must comply with regulatory requirements to run SageMaker jobs in an isolated environment without internet access. Which solution will meet these requirements?

    <p>Run SageMaker training and Inference by using network Isolation. (B)</p> Signup and view all the answers

    An ML research team develops custom ML models. The model artifacts are shared with other teams for integration into products and services. The ML team retains the model training code and data. The ML team wants to build a mechanism that the ML team can use to audit models. Which solution should the ML team use when publishing the custom ML models?

    <p>Create Amazon SageMaker Model Cards with intended uses and training and inference details. (A)</p> Signup and view all the answers

    A software company builds tools for customers. The company wants to use AI to increase software development productivity. Which solution will meet these requirements?

    <p>Install code recommendation software in the company's developer tools. (A)</p> Signup and view all the answers

    A retail store wants to predict the demand for a specific product for the next few weeks by using the Amazon SageMaker DeepAR forecasting algorithm. Which type of data will meet this requirement?

    <p>Time series data (B)</p> Signup and view all the answers

    A large retail bank wants to develop an ML system to help the risk management team decide on loan allocations for different demographics. What must the bank do to develop an unbiased ML model?

    <p>Measure class imbalance on the training dataset. Adapt the training process accordingly. (B)</p> Signup and view all the answers

    Which prompting technique can protect against prompt injection attacks?

    <p>Adversarial prompting (D)</p> Signup and view all the answers

    A company has fine-tuned a large language model (LLM) to answer questions for a help desk. The company wants to determine if the fine-tuning has enhanced the model's accuracy. Which metric should the company use for the evaluation?

    <p>F1 score (B)</p> Signup and view all the answers

    A company is using Retrieval Augmented Generation (RAG) with Amazon Bedrock and Stable Diffusion to generate product images based on text descriptions. The results are often random and lack specific details. The company wants to increase the specificity of the generated images. Which solution meets these requirements?

    <p>Increase the classifier-free guidance (CFG) scale. (A)</p> Signup and view all the answers

    A company wants to implement a large language model (LLM) based chatbot to provide customer service agents with real-time contextual responses to customers' inquiries. The company will use the company's policies as the knowledge base. Which solution will meet these requirements MOST cost-effectively?

    <p>Implement Retrieval Augmented Generation (RAG) for in-context responses. (B)</p> Signup and view all the answers

    A company wants to create a new solution by using AWS Glue. The company has minimal programming experience with AWS Glue. Which AWS service can help the company use AWS Glue?

    <p>Amazon Q Developer (D)</p> Signup and view all the answers

    A company is developing a mobile ML app that uses a phone's camera to diagnose and treat insect bites. The company wants to train an image classification model by using a diverse dataset of insect bite photos from different genders, ethnicities, and geographic locations around the world. Which principle of responsible AI does the company demonstrate in this scenario?

    <p>Fairness (D)</p> Signup and view all the answers

    A company is developing an ML model to make loan approvals. The company must implement a solution to detect bias in the model. The company must also be able to explain the model's predictions. Which solution will meet these requirements?

    <p>Amazon SageMaker Clarify (C)</p> Signup and view all the answers

    A company has developed a generative text summarization model by using Amazon Bedrock. The company will use Amazon Bedrock automatic model evaluation capabilities. Which metric should the company use to evaluate the accuracy of the model?

    <p>BERTScore (B)</p> Signup and view all the answers

    An AI practitioner wants to predict the classification of flowers based on petal length, petal width, sepal length, and sepal width. Which algorithm meets these requirements?

    <p>Kneest neighbors (k-NN) (C)</p> Signup and view all the answers

    A company is using custom models in Amazon Bedrock for a generative AI application. The company wants to use a company managed encryption key to encrypt the model artifacts that the model customization jobs create. Which AWS service meets these requirements?

    <p>AWS Key Management Service (AWS KMS) (B)</p> Signup and view all the answers

    A company wants to use large language models (LLMs) to produce code from natural language code comments. Which LLM feature meets these requirements?

    <p>Text generation (A)</p> Signup and view all the answers

    A company is introducing a mobile app that helps users learn foreign languages. The app makes text more coherent by calling a large language model (LLM). The company collected a diverse dataset of text and supplemented the dataset with examples of more readable versions. The company wants the LLM output to resemble the provided examples. Which metric should the company use to assess whether the LLM meets these requirements?

    <p>Recall-Oriented Understudy for Gisting Evaluation (ROUGE) score (B)</p> Signup and view all the answers

    A company notices that its foundation model (FM) generates images that are unrelated to the prompts. The company wants to modify the prompt techniques to decrease unrelated images. Which solution meets these requirements?

    <p>Use negative prompts. (B)</p> Signup and view all the answers

    A company wants to use a large language model (LLM) to generate concise, feature-specific descriptions for the company's products. Which prompt engineering technique meets these requirements?

    <p>Create prompts for each product category that highlight the key features. Include the desired output format and length for each prompt response. (C)</p> Signup and view all the answers

    A company is developing an ML model to predict customer churn. The model performs well on the trainins dataset but does not accurately predict churn for new data. Which solution will resolve this issue?

    <p>Increase the regularization parameter to decrease model complexity. (C)</p> Signup and view all the answers

    A company is implementing intelligent agents to provide conversational search experiences for its customers. The company needs a database service that will support storage and queries of embeddings from a generative AI model as vectors in the database. Which AWS service will meet these requirements?

    <p>Amazon Aurora PostgreSQL (C)</p> Signup and view all the answers

    A financial institution is building an AI solution to make loan approval decisions by using a foundation model (FM). For security and audit purposes, the company needs the AI solution's decisions to be explainable. Which factor relates to the explainability of the AI solution's decisions?

    <p>Model complexity (C)</p> Signup and view all the answers

    A pharmaceutical company wants to analyze user reviews of new medications and provide a concise overview for each medication. Which solution meets these requirements?

    <p>Create medication review summaries by using Amazon Bedrock large language models (LLMs). (C)</p> Signup and view all the answers

    A company wants to build a lead prioritization application for its employees to contact potential customers. The application must give employees the ability to view and adjust the weights assigned to different variables in the model based on domain knowledge and expertise. Which ML model type meets these requirements?

    <p>Logistic regression model (C)</p> Signup and view all the answers

    Which strategy will determine if a foundation model (FM) effectively meets business objectives?

    <p>Assess the model's alignment with specific use cases. (D)</p> Signup and view all the answers

    A company needs to train an ML model to classify images of different types of animals. The company has a large dataset of labeled images and will not label more data. Which type of learning should the company use to train the model?

    <p>Supervised learning (A)</p> Signup and view all the answers

    Which phase of the ML lifecycle determines compliance and regulatory requirements?

    <p>Business goal identification (B)</p> Signup and view all the answers

    A food service company wants to develop an ML model to help decrease daily food waste and increase sales revenue. The company needs to continuously improve the model's accuracy. Which solution meets these requirements?

    <p>Use Amazon SageMaker and iterate with newer data. (B)</p> Signup and view all the answers

    A company has developed an ML model to predict real estate sale prices. The company wants to deploy the model to make predictions without managing servers or infrastructure. Which solution meets these requirements?

    <p>Deploy the model by using an Amazon SageMaker endpoint. (C)</p> Signup and view all the answers

    A company wants to develop an AI application to help its employees check open customer claims, identify details for a specific claim, and access documents for a claim. Which solution meets these requirements?

    <p>Use Agents for Amazon Bedrock with Amazon Bedrock knowledge bases to build the application. (C)</p> Signup and view all the answers

    A manufacturing company uses AI to inspect products and find any damages or defects. Which type of AI application is the company using?

    <p>Computer vision (D)</p> Signup and view all the answers

    A company wants to create an ML model to predict customer satisfaction. The company needs fully automated model tuning. Which AWS service meets these requirements?

    <p>Amazon SageMaker (C)</p> Signup and view all the answers

    Which technique can a company use to lower bias and toxicity in generative AI applications during the Postprocessing ML lifecycle?

    <p>Human-in-the-loop (C)</p> Signup and view all the answers

    A bank has fine-tuned a large language model (LLM) to expedite the loan approval process. During an external audit of the model, the company discovered that the model was approving loans at a faster pace for a specific demographic than for other demographics. How should the bank fix this issue MOST cost-effectively?

    <p>Include more diverse training data. Fine-tune the model again by using the new data. (B)</p> Signup and view all the answers

    A company needs to log all requests made to its Amazon Bedrock API. The company must retain the logs securely for 5 years at the lowest possible cost. Which combination of AWS service and storage class meets these requirements? (Choose two.)

    <p>Amazon S3 Intelligent-Tiering (A), AWS CloudTrail (D)</p> Signup and view all the answers

    Study Notes

    Exam C Study Notes

    • Question 1: Prompting attack directly exposing large language model (LLM) behavior is extracting the prompt template.

    • Question 2: Security aspect the company is responsible for when using Amazon Bedrock is securing the company's data in transit and at rest.

    • Question 3: Crowd-sourced evaluation is not the least operationally intensive method for comparing LLM toxicity. Automatic model evaluation is best suited for minimized operational overhead.

    • Question 4: Fuzzing training data is an example of testing security in a foundation model to identify vulnerabilities in safety features.

    • Question 5: Running SageMaker training and inference using network isolation meets regulatory requirements to isolate jobs from internet access.

    • Question 6: Using Amazon SageMaker Model Cards when publishing ML models provides transparency on intended uses and training details.

    • Question 7: Installing a code recommendation software tool enhances development productivity.

    • Question 8: Text data is not suitable for product demand prediction using the DeepAR algorithm. Image data is not suitable either.

    • Question 9: Measuring class imbalance on the training dataset and adapting the training process is crucial to develop unbiased ML models.

    • Question 10: Adversarial prompting is a technique to protect against prompt injection attacks.

    • Question 11: F1 score is the metric for evaluating LLM accuracy. (F1 measures the balance between precision and recall.)

    • Question 12: Increasing the specificity of generated images requires using the MASK_IMAGE_BLACK mask source option.

    • Question 13: Implementing Retrieval Augmented Generation (RAG) is the most cost-effective solution for LLM-based chatbot interactions using the company's policy knowledge base.

    • Question 14: Amazon Q Developer can assist in using AWS Glue, a data processing service.

    • Question 15: Demonstrating fairness in data gathering for ML models is crucial to prevent bias, especially in scenarios of insect bite photo classification

    • Question 16: A solution that detects bias is needed but must also be explainable.

    • Question 17: BERTScore is the metric to assess the accuracy of generative text summary models, by using Amazon Bedrock's automatic model evaluation capabilities

    • Question 18: K-Nearest Neighbors (K-NN) is suitable for flower classification based on given parameters.

    • Question 19: AWS Key Management Service (AWS KMS) encrypts model artifacts.

    • Question 20: Text generation is an appropriate LLM feature for producing code from comments.

    • Question 21: Recall-Oriented Understudy for Gisting Evaluation (ROUGE) score is a metric used to assess the quality and coherence of generated text, as per the example.

    • Question 22: Using negative prompts helps decrease unrelated images generated.

    • Question 23: Creating separate prompts for each product type, outlining features, and describing the desired format enhances accuracy and product specificity.

    • Question 24: Increasing the regularization parameter can resolve the model under-fitting issue.

    • Question 25: Amazon Aurora PostgreSQL supports storing and querying embeddings from generative AI models.

    • Question 26: Model complexity is a factor impacting the explainability of the AI loan approval decisions.

    • Question 27: Using Amazon Bedrock large language models (LLMs) directly produces medication review summaries

    • Question 28: Logistic regression models offer customization capabilities for variable weights in lead prioritization applications.

    • Question 29: Assessing model alignment with specific use cases verifies the effectiveness and suitability of the foundation model.

    • Question 30: Supervised learning uses labeled data to train a model for identifying animal classifications.

    • Question 31: Data Collection is where compliance and regulatory requirements are determined.

    • Question 32: Iterating with newer data using Amazon SageMaker results in improved model accuracy regarding food waste predictions.

    • Question 33: Deploying a model through an Amazon SageMaker endpoint results in a serverless deployment method for real-estate price predictions.

    • Question 34: Agents that utilize Amazon Bedrock knowledge bases effectively facilitates knowledge-driven claim checks.

    • Question 35: Image processing effectively identifies damages or defects in manufacturing inspections.

    • Question 36: Amazon SageMaker automates model tuning.

    • Question 37: Lowering the bias and toxicity in generative AI applications involves techniques like human-in-the-loop.

    • Question 38: Using more diverse training data and retraining the model is the most cost-effective method for addressing biased loan approvals.

    • Question 39: Using both Amazon CloudTrail and Amazon S3 Intelligent-Tiering provides secure and cost-effective logging of Amazon Bedrock API requests.

    Studying That Suits You

    Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

    Quiz Team

    Related Documents

    Exam C PDF

    Description

    This quiz covers essential topics related to the security of large language models (LLM) and their operational evaluation. It includes questions on prompt extraction, data security, and best practices for model evaluation. Ideal for anyone preparing for an examination in machine learning security.

    More Like This

    Use Quizgecko on...
    Browser
    Open
    Browser
    Browser