Developing ML solutions
Model Deployment
Deployment puts the model and its resources into production so it can make predictions.
Self-hosted versus managed
- Self-hosted API: you run the model on your own infrastructure (on premises or VMs and containers in the cloud). More control and customization, and possibly lower cost, but more operational overhead.
- Managed API: a cloud service such as SageMaker AI handles the infrastructure. It offers one-click or single-API deployment, automatic scaling, model hosting, and HTTPS endpoints that can host multiple models.
SageMaker inference options
| Option | Best for |
|---|---|
| Real-time | Interactive, low-latency requests on a persistent endpoint |
| Batch transform | Predictions on large datasets with no persistent endpoint, or preprocessing datasets |
| Asynchronous | Large payloads (up to 1 GB) and long processing (up to 1 hour), with near real-time latency. Requests are queued |
| Serverless | Intermittent traffic with idle periods, when cold starts are acceptable. No infrastructure to manage |