ML Exam Prep: 9 - Misc

 

ML Exam Prep

Misc

Search Types

1) Lexical search: matches exact words

2) Semantic search: understands concept 

3) Contextual search: adapts results using user data or surrounding conversation

Deployment Types

Canary = deploy to alpha sites (small subset of users or servers).

A/B testing = concurrent. some goes to new and rest to production.

Shadow = incoming are mirrored or copied and sent to production and new potential. New potential is logged and analyzed on metrics, but *only* answers from production are returned to end-users.

Blue/Green = All goes to “blue“ (production). Once "green" (new version) is ready, traffic switched over. Has a safe rollback mechanism and min downtime. No parallel testing of new model. 

Hyperparameters

Temperature = chaos\creative of response

Top K = is # of tokens. More tokens = more diverse/chaos of responses.

Misc

Categorical model = classification model.

Data Bias & Security

Pre-Training Bias

MetricsClass Imbalance (CI): One category drastically outnumbers others in data.
Diff in Proportions of Labels (DPL): Measures imbalanced positive outcomes across facets.
SageMaker Clarify: Primary AWS tool to detect and mitigate ML bias.

Handling Bias

Resampling: Oversample minority classes or undersample majority classes.

Synthetic Data: Use SMOTE to generate fake minority class examples.
Augmentation: Crop, flip, or rotate images to expand training variety.

Security & Compliance

Encryption: Use S3 server side encryption with KMS for data at rest and TLS for data in transit.

Anonymization: Mask or tokenize PII (Personally Identifiable Information).
Compliance: Ensure data storage locations respect PHI (HIPAA) and data residency laws.

External-account: For external company access to S3 bucket in the company's AWS account:

  1) Use an identity-based policy to attach to IAM users that grants or denies access to the S3 bucket.

  2) Create new IAM role with a trust policy in the company's account for the external account with permissions and specify the external company's account ID.