AWS SageMaker

SageMaker provides fully managed notebook instances that run on EC2 with pre-installed ML libraries (TensorFlow, PyTorch, scikit-learn, XGBoost), Git integration, and automatic lifecycle management. Notebooks are elastically scaled and can be paused/resumed without losing state, with built-in IAM role attachment for direct AWS service access (S3, DynamoDB, Secrets Manager). The architecture uses EBS-backed storage and VPC networking for security isolation.

SageMaker Training abstracts away cluster provisioning by accepting training scripts (Python, TensorFlow, PyTorch, XGBoost) and automatically spinning up distributed training jobs across multiple EC2 instances with built-in support for data parallelism, model parallelism, and pipeline parallelism. It handles inter-node communication via Horovod or native framework distributed APIs, manages spot instance interruption recovery, and logs metrics to CloudWatch. The service uses a container-based architecture where user code runs in Docker images (AWS-managed or custom ECR images).

SageMaker Clarify computes feature importance and SHAP values to explain model predictions at the instance and global levels. It supports tabular, text, and image models and uses multiple explanation methods (SHAP, permutation importance, partial dependence). Clarify integrates with SageMaker training and inference to automatically generate explanations during model evaluation and can be invoked on-demand for specific predictions. Explanations are visualized in SageMaker Studio dashboards and exported as JSON for downstream analysis.

SageMaker Neo compiles trained models to optimized formats for edge devices (AWS Greengrass, IoT devices, mobile) and on-premises servers. It uses compiler technology to reduce model size by 2-10x and improve inference latency by 2-25x without retraining. Neo supports TensorFlow, PyTorch, XGBoost, and MXNet models and targets multiple hardware platforms (ARM, x86, NVIDIA GPUs). Compiled models run via SageMaker Runtime, a lightweight inference library that handles model loading and prediction.

SageMaker Experiments tracks training runs with hyperparameters, metrics, artifacts, and code versions, enabling comparison across experiments. SageMaker Model Registry stores trained models with metadata (framework, input schema, performance metrics, approval status) and integrates with CI/CD pipelines for automated model promotion. The service maintains full lineage from raw data through feature engineering, training, and deployment, enabling reproducibility and audit trails. Models can be versioned and approved for production via workflow-based approval gates.

SageMaker Automatic Model Tuning (AMT) uses Bayesian optimization to search hyperparameter spaces by training multiple model variants in parallel and iteratively refining the search based on objective metrics (accuracy, F1, AUC). It supports categorical, continuous, and integer parameter types, defines search bounds, and can optimize for multiple objectives with weighted trade-offs. The service manages the training job queue, early stopping of unpromising trials, and warm-pooling of instances to reduce launch overhead.

SageMaker Model Registry stores trained models with metadata (framework, input schema, performance metrics), and SageMaker Endpoints provision containerized inference servers on managed EC2 instances with automatic load balancing, health checks, and horizontal scaling based on CloudWatch metrics (CPU, memory, custom metrics). Deployment uses a blue-green strategy for zero-downtime updates, supports A/B testing via traffic splitting, and includes built-in monitoring for model drift and prediction latency. The service handles container orchestration, SSL/TLS termination, and request batching.

SageMaker Feature Store is a centralized repository for ML features with two storage tiers: Online Store (low-latency DynamoDB for real-time inference) and Offline Store (S3 for batch training). It automatically handles feature versioning, point-in-time joins to prevent data leakage, and event-time semantics for time-series features. Features are organized into FeatureGroups with schema definitions, and the service provides Python SDK methods to ingest, retrieve, and join features across groups. Ingestion supports batch (Spark, Glue) and streaming (Kinesis, EventBridge) sources.

SageMaker Pipelines is a DAG-based workflow engine that chains together training, evaluation, and deployment steps using Python SDK definitions. Pipelines support conditional execution (if model accuracy > threshold, deploy; else, retrain), parameter sweeps (grid/random search over step inputs), and caching of step outputs to avoid re-running expensive computations. Steps are containerized and run on managed compute; the service integrates with CloudWatch for monitoring, SNS for notifications, and EventBridge for triggering on external events. Pipelines are versioned and can be scheduled via EventBridge or triggered manually.

SageMaker Batch Transform processes large datasets (GB to TB scale) through trained models without provisioning persistent endpoints. It reads input data from S3, partitions it across multiple workers, applies the model, and writes predictions back to S3. The service supports data preprocessing/postprocessing via Lambda functions, automatic input/output format conversion (CSV to JSON and vice versa), and spot instances for cost reduction. Batch jobs are asynchronous and can process data in parallel across multiple instances with configurable batch sizes.

SageMaker Model Monitor captures prediction data from endpoints, compares feature distributions and prediction outputs against baseline statistics (computed during training), and detects data drift, model drift, and feature attribution drift. It uses statistical tests (Kolmogorov-Smirnov, Chi-squared) to identify distribution shifts, triggers CloudWatch alarms when drift exceeds thresholds, and integrates with EventBridge to automatically trigger retraining pipelines. Monitoring data is stored in S3 and visualized in SageMaker Studio dashboards.

SageMaker Multi-Model Endpoints (MME) host multiple models on a single endpoint, dynamically loading models into GPU/CPU memory based on incoming requests. The service uses a model loading cache to minimize latency for frequently accessed models and automatically unloads idle models to free resources. MME reduces infrastructure costs by 50-70% compared to single-model endpoints when hosting many small models. The architecture uses a model server (e.g., TensorFlow Serving, TorchServe) that routes requests to the appropriate model based on request path.

SageMaker Ground Truth provides managed data labeling with support for image classification, object detection, semantic segmentation, text classification, and custom labeling tasks. It integrates with Amazon Mechanical Turk for crowdsourced labeling and supports private labeling teams. The service includes active learning to automatically select the most informative samples for labeling, reducing annotation costs by 40-60%. Labeling jobs output annotations in standard formats (COCO, Pascal VOC, YOLO) and integrate with SageMaker training pipelines.