pault-tech / SageMaker-Deployment

Compilation of examples of SageMaker inference options and other features.

SageMaker-Deployment

Compilation of examples of SageMaker inference options and other features (Forked).

Inference Options

• Real-Time Inference
  • Multi-Model Endpoints(MME)
  • Multi-Container Endpoints(MCE)
  • Serial Inference Pipelines
• Serverless Inference
• Asynchronous Inference
• Batch Transform

What Inference Option Is For You?

SageMaker Inference Options

Performance Optimization/Advanced Functionality

This is most pertinent for Real-Time Inference in regards to handling traffic spikes and meeting latency requirements. More techniques + tools will be added to help optimize performance for your use-case.

AutoScaling

Custom Inference Scripts

Load Testing with Locust

How To Deploy ML Models on Amazon SageMaker?

SageMaker SDK vs Boto3 Python SDK