AI - AI in the Cloud

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

When making decisions based on data, what is the process called?

AI inference (correct)

AI training

Retraining

Cloud deployment

What is the process of teaching a machine how to do something called?

Training (correct)

Inference

Retraining

Automated Model Update

True or false: CPUs are better suited than GPUs for AI inference tasks.

False

What type of AI inference is used for applications that run on the user's device?

Client App Inference Signup and view all the answers

What are the four main types of AI inference?

Server application inference, client app inference, streaming inference, and edge inference Signup and view all the answers

True or false: Automated Model Updates are done at the beginning of a model development cycle.

False Signup and view all the answers

What is the process of teaching a machine how to do something called?

AI training Signup and view all the answers

What is the most difficult type of compute to counter?

ARM products Signup and view all the answers

True or false: ARM products are the most difficult to counter compute type.

True Signup and view all the answers

What type of instances will outperform CPU-only instances when raw throughput or latency is a factor?

GPUs Signup and view all the answers

True or false: Cloud deployment of AI workloads is becoming less popular.

False Signup and view all the answers

What are the main factors to consider when deciding if an AI workload is suitable for cloud deployment?

Type of AI and latency Signup and view all the answers

What is the most difficult to counter compute type?

ARM products Signup and view all the answers

What are the most popular cloud platforms for deploying AI workloads?

Amazon Web Services (AWS) and Microsoft Azure Signup and view all the answers

True or false: Google Cloud Platform and Microsoft Azure are the only cloud platforms available for AI workload deployment.

False Signup and view all the answers

What are the three most popular cloud platforms for deploying AI workloads?

AWS, GCP, and Microsoft Azure Signup and view all the answers

What type of tradeoff is often considered when deciding if an AI workload is well suited for cloud deployment?

Price and performance Signup and view all the answers

What is usually done at the end of a model development cycle?

AI training Signup and view all the answers

What is the purpose of retraining/automated model update?

To update a model that has been trained previously Signup and view all the answers

What is the power band of Intel's abilities?

Between GPUs and ARM products Signup and view all the answers

Study Notes

AI inference is the process of making decisions based on data.
There are four main types of AI inference: server application inference, client app inference, batch or streaming inference, and edge inference.
AI training is the process of teaching a machine how to do something.
Training is done in a single direction, and is usually done at the end of a model development cycle.
Retraining/Automated Model Update is the process of updating a model that has been trained previously.
CPUs can compete up until raw throughput or latency is a factor—at that point GPU instances will outperform CPU-only instances.
For AI tasks of inference, CPUs can compete in the space unless raw throughput or time to train is a factor—otherwise GPU instances will outperform CPU-only instances.
Training cost is a unique problem—is less time on a GPU instance equivalent to a longer training time on a CPU instance? In this case you may need an expert to evaluate the situation.
If their IT staff is already in the Capex to Opex conversion with their compute, then a play against a GPU instance can only be made with larger scale problems that exceed GPU networking—again call in an expert to evaluate the compute.
For Intel-based instances, one will need to balance the above data with the mixed server generations that have different behaviors with the end-customer. Intel’s abilities are in a specific “power band” of the AI compute graph above, bookended by GPUs on one end and ARM on the other. Additionally, the most difficult to counter compute type is not an Nvidia GPU or AMD product (as Intel has GPU and x86 lines)—it is ARM products.
Cloud deployment of AI workloads is becoming more popular due to the price performance tradeoff.
There are a few factors to consider when deciding if an AI workload is well suited for cloud deployment, including the type of AI, the latency and bandwidth requirements, and the price.
Cloud instances are typically run on Amazon Web Services (AWS), Google Cloud Platform (GCP), or Microsoft Azure.
Each platform has its own advantages and disadvantages, so it is important to evaluate the workload before choosing a platform.

Studying That Suits You

Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

Description

Test your knowledge of AI inference, training, and cloud deployment with this quiz. Explore different types of AI inference, the process of AI training, and factors to consider when deploying AI workloads in the cloud.