Model Deployment

Model deployment types

Model deployment is the integration of the model and its resources into a production environment so that it can be used to create predictions.

Self-hosted API

In a self-hosted API approach, you deploy and host your ML models on your own infrastructure, either on premises or in the cloud (using virtual machines or containers). This approach involves setting up and managing the necessary infrastructure, such as web servers, load balancers, and databases, to serve your ML models as APIs.

Managed API

Managed API services are cloud-based services that provide a fully managed environment for deploying and hosting your ML models as APIs. SageMaker AI is an example. These services abstract away the underlying infrastructure management so you can focus on building and deploying your models.

Advantages of self-hosted APIs include greater control over the infrastructure, potential cost savings (depending on usage), and the ability to customize the deployment environment. However, this approach requires more operational overhead and responsibility for managing and maintaining the infrastructure.The choice between a managed API service or a self-hosted API for ML deployment depends on factors such as the specific requirements of your use case, the level of control and customization needed, the available resources and expertise, and cost considerations.

SageMaker AI

SageMaker AI is a fully managed ML service. With SageMaker AI, data scientists and developers can quickly and confidently build, train, and deploy ML models into a production-ready, hosted environment. Within a few steps, you can deploy a model into a secure and scalable environment.

SageMaker AI provides the following:

• Deployment with one click or a single API call
• Automatic scaling
• Model hosting services
• HTTPS endpoints that can host multiple models

You can use SageMaker AI to deploy a model to get predictions in several ways.

Real-time

Real-time inference is ideal for inference workloads where you have real-time, interactive, and low latency requirements.

Batch transform

Use batch transform when you need to get inferences from large datasets and don't need a persistent endpoint. You can also use it when you need to preprocess datasets to remove noise or bias that interferes with training or inference from your dataset.

Asynchronous

SageMaker AI asynchronous inference is a capability in SageMaker AI that queues incoming requests and processes them asynchronously. This option is ideal for requests with large payload sizes (up to 1GB), long processing times (up to one hour), and near real-time latency requirements.

Serverless

On-demand serverless inference is ideal for workloads that have idle periods between traffic spurts and can tolerate cold starts. It is a purpose-built inference option that you can use to deploy and scale ML models without configuring or managing any of the underlying infrastructure.

In the next lesson, you will learn about Fundamental Concepts of MLOps.