20 Questions
When making decisions based on data, what is the process called?
AI inference
What is the process of teaching a machine how to do something called?
Training
True or false: CPUs are better suited than GPUs for AI inference tasks.
False
What type of AI inference is used for applications that run on the user's device?
Client App Inference
What are the four main types of AI inference?
Server application inference, client app inference, streaming inference, and edge inference
True or false: Automated Model Updates are done at the beginning of a model development cycle.
False
What is the process of teaching a machine how to do something called?
AI training
What is the most difficult type of compute to counter?
ARM products
True or false: ARM products are the most difficult to counter compute type.
True
What type of instances will outperform CPU-only instances when raw throughput or latency is a factor?
GPUs
True or false: Cloud deployment of AI workloads is becoming less popular.
False
What are the main factors to consider when deciding if an AI workload is suitable for cloud deployment?
Type of AI and latency
What is the most difficult to counter compute type?
ARM products
What are the most popular cloud platforms for deploying AI workloads?
Amazon Web Services (AWS) and Microsoft Azure
True or false: Google Cloud Platform and Microsoft Azure are the only cloud platforms available for AI workload deployment.
False
What are the three most popular cloud platforms for deploying AI workloads?
AWS, GCP, and Microsoft Azure
What type of tradeoff is often considered when deciding if an AI workload is well suited for cloud deployment?
Price and performance
What is usually done at the end of a model development cycle?
AI training
What is the purpose of retraining/automated model update?
To update a model that has been trained previously
What is the power band of Intel's abilities?
Between GPUs and ARM products
Study Notes
- AI inference is the process of making decisions based on data.
- There are four main types of AI inference: server application inference, client app inference, batch or streaming inference, and edge inference.
- AI training is the process of teaching a machine how to do something.
- Training is done in a single direction, and is usually done at the end of a model development cycle.
- Retraining/Automated Model Update is the process of updating a model that has been trained previously.
- CPUs can compete up until raw throughput or latency is a factor—at that point GPU instances will outperform CPU-only instances.
- For AI tasks of inference, CPUs can compete in the space unless raw throughput or time to train is a factor—otherwise GPU instances will outperform CPU-only instances.
- Training cost is a unique problem—is less time on a GPU instance equivalent to a longer training time on a CPU instance? In this case you may need an expert to evaluate the situation.
- If their IT staff is already in the Capex to Opex conversion with their compute, then a play against a GPU instance can only be made with larger scale problems that exceed GPU networking—again call in an expert to evaluate the compute.
- For Intel-based instances, one will need to balance the above data with the mixed server generations that have different behaviors with the end-customer. Intel’s abilities are in a specific “power band” of the AI compute graph above, bookended by GPUs on one end and ARM on the other. Additionally, the most difficult to counter compute type is not an Nvidia GPU or AMD product (as Intel has GPU and x86 lines)—it is ARM products.
- Cloud deployment of AI workloads is becoming more popular due to the price performance tradeoff.
- There are a few factors to consider when deciding if an AI workload is well suited for cloud deployment, including the type of AI, the latency and bandwidth requirements, and the price.
- Cloud instances are typically run on Amazon Web Services (AWS), Google Cloud Platform (GCP), or Microsoft Azure.
- Each platform has its own advantages and disadvantages, so it is important to evaluate the workload before choosing a platform.
Test your knowledge of AI inference, training, and cloud deployment with this quiz. Explore different types of AI inference, the process of AI training, and factors to consider when deploying AI workloads in the cloud.
Make Your Own Quizzes and Flashcards
Convert your notes into interactive study material.
Get started for free
