ML Exam: 3 - Sagemaker AI

 

ML Exam: 3 

Sagemaker AI

 SageMaker JumpStart vs. Bedrock

Choose between Jumpstart and Bedrock for GenAI and FMs: 
  1) Bedrock: Best for serverless, API-driven access to FMs (Anthropic, Cohere, Meta, Amazon). No infrastructure to manage.
  2) SageMaker JumpStart: Best for complete control. Hub to deploy, fine-tune, and host open-source models (like Llama or Mistral) on dedicated SageMaker instances.

Sagemaker Concepts:

SHAP baseline - To understand how features changes over time, SageMaker needs a point of comparison which is the SHAP baseline. SHAP is (SHapley Additive exPlanations). SHAP is cooperative game theory math. Breaks down a model's prediction and assigns "importance score" (SHAP value) to each input feature. Used to quantify the contribution of each feature in a prediction.

Shadow Testing = evaluates a new model against a production model with minimal operational overhead. Allows traffic to be routed to multiple models without managing more endpoints.  With a shadow variant, the new model receives a copy of live traffic but does not affect production responses. Latency, accuracy, and error rate metrics is compared directly against the current model using CloudWatch metrics. Natively supported.

    Serverless endpoints scale independently in a fully serverless manner. Additionally, the memory requirements fit within the 6 GB memory and 200 maximum concurrency limits of serverless endpoints.

Sagemaker Products: 

SageMaker AI = IDE plugin. fully managed service that automates ML lifecycle (from data prep to production) with "no-code" environment and handles infrastructure to streamline building, tuning, and deploying models. Offers pre-built containers for its built-in algorithms. Capabilities: Predictive analytics, computer vision, NLP, and fraud detection. Steps to Start: 1) pick labeled S3 dataset in CSV, Parquet or other, 2) set algorithm 3) set hyperparameters. 4) Pick compute resources (e.g., instance type), 5) run training job by picking pre-built container.   

   Core Features: auto-training and integrated logging. Secured via IAM; Data: integrates with S3 (storage), including Apache Parquet, Lambda (triggers), RecordIO-protobuf, CloudWatch (monitoring) and API Gateway (endpoints).  

    Scaling Policies: 1) Target Tracking Scaling (benchmarks like CPU %), 2) Step Scaling (thresholds at different tiers), 3) Scheduled Scaling (at different times) with cool downs of scale out and scale in to increase capacity.

    Input Modes: 1) File Mode (default), 2) Pipe Mode (obsolete), 3) Fast File Mode, 4) S3 Express One Zone, 5) FSx for Lustre 6) EFS.   

   Features: 1) "Spot Instances" option: manages the EC2 Spot Instance lifecycle. "Checkpointing Enabled" sub option: saves state and progress to S3 to cover interruptions and auto restarts with last checkpoint. To use on spot instances, requires a) algorithm, b) hyperparameter range, and c) performance metric. 2) "Network Isolation" mode for preventing data exfiltration. 3) "Bring Your Own Container" approach - However, requires Docker image creation, container configuration, and deployment. Use SDK to call the TensorFlow or other model in the custom container. 4) "Managed Warm Pools" pre-warm training instances, so ready train immediately. 5) "Complete On Convergence": Auto stops when more training unlikely to get 1+% over best found so far.

   If super large, then might just mount the volume that holds S3 data.   

    Auto Pilot = fully managed ML service. auto process of building, training, and tuning ML models. Good for quickly exploring different models and algorithms to find the best one for your task. uses Clarify to show how ML models could make predictions. uses SHAP values. Auto finds the best hyperparameters.

    Auto Model Tuning/Hyperparameter Tuning = UI location: Training tab - Hyperparameter tuning section - add tuning job. Steps: 1) Job (IAM role, VPC settings, and min/max metric), 2) algorithm or custom script, 3) tuning limits, performance metric, and search strategies with option to Autotune. Only works with ML and not data prep or data analysis. Adjusts learning rate and batch size.

    Canvas/Data Wrangler = Canvas is "No-code" ML tool to create predictions and generate data insights. Data Wrangler is older version of the tool. Imports data from S3, Redshift, and Feature Store. Has AutoML piece that is focused on models. Requires model registered in Model Registry. 

        Feature engineering options: 1) rebalances data for undersampling, 2) oversampling, 3) systemic minority oversampling (SMOTE), 4) categorical data to numeric using one-hot encoding. (if see feature dominates due to large numbers, then feature normalize with Min-Max scaling), 5) impute values with mean, median, or interpolated values. 6) cleans and transforms raw imported CSV data. 6) "Similarity Encode" for misspellings. 5) "Corrupt Image" (impulse noise option), 7) "Featurize date/time" transform extracts day, month, weekday, etc. from a timestamp. 

        Features: 1) "Quick Model" auto trains a fast, lightweight ML model to create Feature Importance Plot, so see each feature's contribution (Gini importance) before committing to a full training run.  (old Data Wrangler way only did Feature Importance Plot w/o auto train model). 2) after data import, you can add analysis for "Data Quality and Insights" to get report.

         UI: 1) Line plots show how numbers change over time or across ordered categories. 2) Histograms show numerical data distribution.

         Security: Canvas user must have access to S3 bucket of model. 

         Limits: 1) Not for schema tracking. 2) if super large, use Pipelines instead.

    forecasting algorithms:

       parameters: ARIMA, DeepAR+ (time-series), ETS, and LSTNet. 

       non-parameters: CNN-QR (predicts quadriles), NPTS, and Prophet (time-series).

     

    Clarify = evaluates FMs for accuracy, bias, robustness, toxicity, and features and creates reports with results. Helps with transparency and explainability. monitors for bias drift. Can check for class imbalance. Partial dependence plots (PDPs): Show diff in predicted outcome as feature changes. Diff in proportions of labels (DPL): Measures imbalance of + outcomes between different facet values.

    Code Editor = connects to VS Code

    Console = main screen. Under Network Options or Algorithm Options section when creating a Training, Processing, or Tuning Job, check "Enable Network Isolation" to block internet access and external network access.    

    Debugger = debugs the code. Built-in rules for real-time monitoring of Vanishing Gradients (and auto reduce the Learning Rate), underutilized GPUs, and Overfitting (watches the loss curve) triggering alerts or actions when thresholds exceeded. create a hook in the training script to capture and log tensors, to get insights into model‘s performance. If debugger shows low GPU use, adjust batch size and data loading pipeline to ensure GPU is used efficiently. 

    DeepAR model = time-series forecasting model

    Deploy locally trained models = Steps: 1) Serialize Model: Save scikit-learn model and compress to .tar.gz file. Upload file to S3. 2) Build Container: Create  Dockerfile containing Python, scikit-learn, and an HTTP server wrapper. 3) Push to ECR: Authenticate your local Docker client to AWS, tag the image, and push it to ECR. 4) Deploy Endpoint: Create a Model pointing to the S3 bucket path and the ECR image URI, then deploy to a live endpoint.

    Endpoints = fully managed service for ML. via HTTPS URLs. call endpoints to deploy your model then later allow your apps to send data to ML model and receive a prediction. 

       Endpoint Features: 1) call  "Update Endpoint" to auto blue/green deploy. 2) can set min and max capacity via auto scaling. 3) can set "Max Concurrency" to 1 to be singleton. 4) to do multiple production variants, create a endpoint config, set Production Variants, add multiple target models to it, optional set of Infrastructure => Traffic Split for load balancing, then select new endpoint config in endpoint. 5) use "Elastic Inference accelerators" to attach a small GPU acceleration to CPU-based instances. 6) "Invocations Per Instance" metric to add more instances in peak times. 7) increase "Desired Weight" from 0 to allow more to an endpoint. 

       Endpoint Types: 

          1) Real Time endpoint (Max payload is 6 MB. Max response timeout 60 sec. Supports Multi-Model and Multi-Container endpoints. for low-latency responses. auto scales), 

          2) Asynchronous endpoint (large or long-running, Max payload is 1 GB. Max response timeout 15 mins, Supports Scale-to-Zero. Ideal for large-scale, offline processing tasks (like video processing). auto scales. Feature Inference Recommends alt. models, not model quality), 

          3) Serverless endpoint (intermittent or long inactive, Max payload is 30 MB. Max response timeout 60 sec. Only pay for compute used in ms. Feature: "Provisioned concurrency" sets min available instances and avoids cold start overhead.) , 

         4) Batch endpoint (massive or long offline, batch, deploys model asynch for inference, only pay for compute used). 

         5) VPC endpoint. Security: When VPC created, the auto created ENI is governed by a IAM group. You can add a VPC endpoint policy (policy statement and inline policy) to allow access to IAM users. 

       High Availability = deploy endpoint instance across multiple AZ in same region.

    Estimator = If Sagemaker Profiler to work, set profiling duration in seconds for the CPU and GPU. If train using Python code in Sagemaker, then: 1) Python code must be in separate files, 2) Create requirements.txt manifest that SageMaker auto installs. 3) Set entry point script that imports and executes functions. 

    Experiments = Track different configurations, hyperparameters, and algorithms used in each experiment. You can see the visual results in Studio.  "Trial" is a single training run with specific configuration. "Trial components" are individual steps or artifacts in a trial (e.g., metrics, outputs) for different workflow stages such as data processing or model training. "Trackers" functionality auto log hyperparameters, datasets, metrics, and code changes for each trial. 

    Feature Store = stores and shares features/variables of a model to team. Steps: 1) create feature group, 2) ingest, 3) access online or offline store for training data. Offline store is for batch. Online store is for real-time.  ′GetRecord′ API always returns latest feature version.

    Ground Truth = data labeling service creates high-quality training datasets by sending the most difficult ones (hardest 30%) to crowd-sourced humans by outsourcing data labeling tasks. Has special computer vision labeling section.

     Horovod distributed framework = Popular OSS distributed DL framework. Efficient scaling of TensorFlow training across multiple GPUs. User-friendly.

     HyperPod = managed infrastructure service for accelerated distributed training and fine-tuning of FMs. 

     Inference or Deployment options = deploys trained models as hosted services with 4 options: 1) Real-Time: Good for low-latency, but has low payload. 2) Serverless: Good for intermittent traffic with idle periods. 3) Asynchronous: Good for large payloads or long-running. 4) Batch Transform: Good for offline massive datasets. Works for bulk data, scheduled. Not for real-time.

    Inference Recommender = recommends inference types. 

        Default = runs predefined load tests and instance types. NO custom for known traffic patterns or specific latency/throughput requirements.

        Advanced = for precise knowledge of your workload. You can create a custom load test using your *known traffic patterns* and specify target latency and throughput requirements.

    JumpStart = Gen AI. ML hub with 100s of FMs and pre-built MLs (vision, NLP, and tabular data) deployable with a few clicks. Not a low code option. Created code should feature threat detection and data protection. Some of evaluation types such as automatic model evaluation. Can restrict FMs. Comes with pre-built pipelines for whole ML process.

   JupyterLab = run code cells, visualize data, and tweak preprocessing workflows w/o worrying about underlying infrastructure. Limitation: 1) totally unmanaged, while Studio notebook is managed.

   Lineage Tracking = auto records data lineage, model parameters, and artifacts across the workflow. Good for audit trails, governance, and compliance verifications, tracking the exact lifecycle of a model from raw data to deployment. Better than Experiments if already in Sagemaker since gives better workflow management via DAG.

   Managed Warm Pools = keeps provisioned ML compute instances active and "warm" for a Keep-Alive period after a model training job completes.

    MLOps  = DevOps for ML

    Model Dashboard = sharing team info on production model behavior in one place.

      Model Cards = your model property info for documentation purposes

      Model Registry = store, manage, tracks your model versions and through deploy ML lifecycle. Hierarchy: 1) collection and 2) model groups to catalog the models. You must give unique tags for each model version. Really is a cataloging step, not deployment step. Feature: "Model Approval".  Pipelines can auto register the model into registry.

    Model Invocation Logging = you can turn this on.

    Model Monitor = monitors production models for data drift (missing values and outliers in new data, see bias drift by data pattern changes, "concept drift" in customer data, and deviations from the baseline training data), model drift, model quality (performance) loss, and feature change drift (called model explainability). If feature drift, then alert the team and retrain the model with updated data to ensure feature importance remains balanced. If new model, then must do another baseline. For deployed models and not for training/evaluation metrics. To measure model quality (accuracy, precision, recall, F1 score), you must merge predictions with ground truth labels. Steps for other AWS accounts and models: 1) create "model group". 2) create new AWS account with this as central catalog, 3) attach cross-account resource policy to each model group in other AWS accounts,   

    Neo - for edge devices. compiles the model into a platform-specific format, reducing its size and for faster performance.

    Neural Topic Model (NTM): auto blog post tagging OR product category creation. 

    Notebook = WARNING - different from Studio Notebook!!!  Is older version, not for collaboration. Accessed by short-lived presigned URL via CLI or SDK. Instances: 1) General Purpose, 2) Memory Optimized, 3) Compute Optimized, and 4) Accelerated Computing. 

        Notebook Rights 

        Location: Notebook instances are in an EC2 instance in a AWS-managed service account, not your ABC Company customer account.

        Rights for individual: 1) Create single IAM role with permissions or policy to access the notebook and read to S3 data. 2) Attach to each notebook instance. 3) If S3 data encrypted with KMS, then grant rights to KMS key policy to IAM role. 

        Rights for shared notebook: IAM groups do permission mgmt. for multiple users. 

        Rights for no internet: Put on private subnet in a VPC with S3 and/or SageMaker S3 VPC endpoint inside it. IAM groups of VPC endpoint to only allow outbound connections. 

        Rights On Creation: When create notebook, its EC2 instance and its attached EBS volume is in AWS-owned service account, not directly in your AWS customer account.

      Lifecycle Config = Create config with on-create or on-start scripts for a notebook.

    Pipelines = serverless, purpose-built MLOps and LLMOps workflow orchestration service.  DAG done in JSON or Python. once passes Model Registry, then process waits until "Approved" status. Can automate all ML workflow. 

          Steps for other pipelines: 1) pipeline definition such as name, data pre-processing, model training, and model registration, 2) steps for each action, 3) parameters for step configuration. Steps to Code Pipeline: 1) define steps in this, 2) add explicit step to this in Code Pipeline, 3) trigger via Code Commit, 4) monitor and validate. CI/CD steps: 1) S3 event triggers pipeline, 2) CodeBuild processes/trains model using data in S3, 3) Model registered in SageMaker Model Registry.  

          Features: 1) Runs batch transforms for cost-effective, scheduled inference on entire data. 2) Can trigger retraining pipelines when data drift detected or when new data arrives. 3) Can auto register a new model in Model Registry and then trigger deployment to real-time endpoint based on predefined steps. (It looks at Model Lineage for formal details.)

       Pipeline Parameters = create variables that can be overridden when running pipeline (without modifying the pipeline).

       Pipeline Steps = define actions in the pipeline, but do not offer parameter flexibility. 

    Processing = fully managed feature designed to run data engineering, feature engineering, data validation, and model evaluation workloads at scale. 

          Steps: 0) Triggers prep script, 1) spins up instance, 2) reads from S3, Athena, or Redshift. 3) spins up Docker or custom container. 4) Runs script, 5) writes output back to S3, 6) auto teardown of temp EC2 instances.

    Profiler = for profiling (CPU and GPU use, memory usage, and I/O) and visualization 

          Requires: 1) Import necessary modules 2) add Start and Stop profiling commands in PyTorch training script.  

          Monitor: 1) GPU utilization and adjust the data pipeline if GPU underutilization is detected, 2) disk IO to prevent bottlenecks in data loading. 

          Limits: 1) not for changing hyperparameters, 2) not for detecting overfitting

    Purchase Provisioned Throughput = dedicated capacity bought. required for using custom models.

    Python SDK Estimator = Define the profiler config, specifying profiling duration in seconds for the CPU and GPU. can have endpoints.

    Role Manager = define min. permissions

    Script Mode = gives pre-configured Python and PyTorch containers to hold your existing custom training and inference scripts to help reuse. can train custom ML models. 

    SDK

       Estimator = high-level Python interface

    Serverless Inference = built for infrequent, intermittent, or predictable workloads (such as a single nightly run). Auto handles runs and scales down to absolute zero when not in use. Setting the 

Max Concurrency parameter to 1 limits the endpoint to a single concurrent container invocation.

    Studio = web-based IDE for ML with tools for data prep, model building, training, and deployment. Its "Feature Importance" charts and "Summary" plots use SHAP values. On "Autopilot" tab then "Training Mode section" then "Hyperparameter Tuning" section, you can set "Early Stopping" to Enabled or Auto to stop early.

       Studio notebooks = WARNING - different from Notebook!!! Collaborative with Jupyter notebooks with ML libraries, persistent storage, and integrated tools. Fully managed environment, unlike Jupyter Labs. Use Studio notebooks to write, run, and share code for data exploration, model training, and deployment. Studio notebooks are integrated with SageMaker Studio, allowing persistent storage and multiple notebooks per project, while standalone notebook instances operate independently.

       Studio JupyterLab environment = No need for manual infrastructure setup. Has Matplotlib and Seaborn libraries for data visualizations. Has pre‑installed libraries such as TensorFlow and PyTorch. Simplifies infrastructure setup, auto save work to S3, and allow users to resume their work later.

    Tensor Board = visual tool. track, debug, and optimize DL models over "epochs" (training loops).

    Training = trains ML models on various compute instances, including GPU accelerated instances using distributed service.

    Training Compiler = built in feature. takes DL from PyTorch or TensorFlow and compiles into EXE.

    Users can have resource tagging tracking for threshold detection.

SageMaker Input Modes: (for training data from S3)

1. "File Mode" - Default. Copies entire S3 data to onto the training instance's local EBS volume in Docker container. Then starts training. Terrible when training dataset is huge!
2. "Pipe Mode" - Streams real-time data from S3 straight to training. Data not stored on local storage of training instance. Obsolete, use Fast File. Requires either training.lst file, validation.lst, or an augmented manifest file in JSON lines format.  RecordIO (.rec) format is most efficient and recommended for built-in image classification algorithm. Create two RecordIO files, training.rec and validation.rec, from the manifest files by using the im2rec Apache MXNet utility tool.
3. "Fast File Mode" - Real-time. Training begins before all data loaded in parallel so decreases startup time.  Combines benefits of both File and Pipe modes. Can access entire files (like File mode) and streams data to algorithm (like Pipe mode). Can do random access (but best with sequential access).
4. S3 Express One Zone = Fast storage class in one AZ. Combines with an S3 mode (File, Pipe or Fast File)
5. FSx for Lustre = Scales to high performance (100s GB of throughput and millions of IOPS) with low latency, Single AZ, Requires VPC
6. EFS = Requires VPC.

Main SageMaker Algorithms

Regression & Classification (Tabular Data):
  Linear Learner = for classification and regression. optimizes variants in parallel. Parameters 1) Increasing "Target Precision" parameter minimizes false positives. 

  XGBoost = SL. Decision tree. Parameters: 1) max_depth that controls tree complexity. 2) 

Increasing reg_lambda counteracts overfitting.
  Factorization Machines = Good for click-through-rate (CTR) and high volume. Models interactions between features and is effective for sparse datasets, such as recommendation systems.

  K-Nearest Neighbors (KNN) = SL. Classification (common) or rarely in Progression. classifies data point on how its features are similar to others (neighbors). Classification answer is 0 to 1.

  Object2Vec = Converts pairs of words, customer IDs, and tokens into 1 or 0 for similar words.

UL and Clustering:

  KMeans = UL. No event planning by 1) K = number of K leaders for people to cluster around, 2) each data point finds closest leader, and 3) the leader moves to the Means = math mean (center) of their group. Good for: finds hidden or unlabeled patterns, customer segmentation, risk grouping, and pattern discovery.

  Principal Component Analysis (PCA) = UL. think: looks for Patterns, Compressing it (reducing the dimensions), on the Anonymous data (so UL).  PC1 = trend of points, PC2 = perpendicular and sub-trend. Ex: Does not care about labels (of "height" and "weight"), but rather creates single dimension of size (so seeing the trend) which is PC1. Then tracks data that is not explainable by size (say "body shape") that is PC2.  Only cares about where the data is most spread out (variance).

  Principal Components = new, independent axes (directions) that rank the data's most important trends (patterns) from highest to lowest spread (variance)."

  Random Cut Forest = anomaly detection. Good for id outliers or unusual behavior.

NLP & Topic: 

   BlazingText: An ultra-fast word embedding and text classification engine optimized for GPUs. It can scale across multi-node clusters to generate Word2Vec vectors or categorize text items (e.g., web queries, sentiment tags) at scale.

   Neural Topic Model (NTM): organizes large text doc collections into distinct thematic topic categories. It maps hidden word associations without requiring pre-existing manual index labels.

   Latent Dirichlet Allocation (LDA): UL. NLP. Dirichlet is a lazy (so UL) bible reader that looks through text (so NLP), finding different topics, and finds the theme by associations between topics. 

   Sequence-to-Sequence (Seq2Seq): supervised neural framework mapping an input sequence of tokens directly to an output sequence. Good for translations, summarization models, and speech-to-text workflows.

Vision:

  Image Classification: Assigns one or more categorical labels to a whole image using deep CNN (ResNet). Supports transfer learning from pre-trained nodes or full custom initializations.

  Object Detection: Ids, bounds, and classifies multiple distinct elements inside a single frame. It produces standard pixel-coordinate bounding boxes tagged with categorical confidence markers.

  Semantic Segmentation: Pixel-level structural tracking, tagging every individual pixel in an image with a class category. Good for autonomous driving maps or medical scan line tracking. 

Time Series Forecasting: 

  DeepAR: Needs historical data. Optimized for predicting future values.

Other:

   LightGBM: tree-based algorithm. Can be configured with oversampling the minority class or adjusting class weights on imbalanced classes.  Captures complex relationships and interactions between features.  Fast and efficient. Quickly train models.
   IP Insights: UL, IPv4 addresses and associates them with entities like user IDs.

   Bias: Jensen-Shannon divergence, Kullback-Leibler divergence, and total variation distance for loan bias.