ML Exam Prep: 10 - Key Algorithms

ML Exam Prep

Key Algorithms 

Common Algorithms:

K-Means algorithm = UL. No event planning by 1) K = # of K leaders for people to cluster around, 2) each data point finds closest leader, and 3) the leader moves to the Means = math mean (center) of their group. finds hidden or unlabeled patterns. 

KNN algorithm = SL. Classification. finds absolute distance of data point to class of its neighbors. Ex answers: "Apple", "Orange", "Spam", "Not Spam".

Label encoding = City name ("Dallas", "Paris", "London") to single column of City of (1, 2, 3). 

Linear regression algorithm = SL. predicts "best-fit" line of min diff between input and output. Can be negative. Answer is continuous range.  Disadv: fails with skewed data. 

Logistic regression algorithm = SL. Classification. estimates % that input is in category by predicting binary outcome of using Logistic function and Log-odds. Answer is 0 to 1.

One-Hot Encoding: "City Name"  ("NY", "Paris", "Tokyo") to  City_NY, City_Paris, City_Tokyo columns with only single 1 row.

Recurrent Neural Network (RNN): for sequential data (such as time series, NLP, speech, or text). 

eXtreme Gradient Boosting (XGBoost): SL. builds trees sequentially, with each new tree correcting previous errors. Good at regression, classifying, and ranking. Good on tabular data. Gradient boosted trees algorithm. Set "Objective" to multi:softmax if product categorization.

Too Many Features or Highly Correlated/Multicollinearity (regression) Data:

Pearson Correlation Coefficient: calcs feature correlation. Is 0 = no (so independent), 1 (or -1) = strong. Doesn't change data. Use Naive Bayesian Model if independent, otherwise use full Bayesian network if dependent.

Principal Component Analysis (PCA): Drops dimensions, keeps data variance. Sensitive to large swings so scale with Min Max Scaler transform. By creating new uncorrelated PCs, it also rids multi-collinearity.

Recursive Feature Elimination (RFE): iteratively trains the model, ranks features by importance (e.g., based on coefficients in logistic regression), removes least important features, and repeats the process until target number.

Singular Value Decomposition (SVD) = looks at single value at a time to find impact.

Outliers:

Random Cut Forest:  outlier detection.

IP Insights: if IP. 

Recommendations:

Factorization Machines (FM): For recommendations and sparse data.

Collaborative Filtering: For customer recommendations given past customer data.

Images:

Convolutional neural network (CNN): DL for images; uses neural network filters to grid pixels (3x3 block) for pattern recognition in images. Filters scale from local to abstract. Ideal for computer vision, image classification, OCR, and medical imaging. Settings: 1) Softmax is for binary classifications. 2) Linear is for regression (rare).

ResNet-50: CNN. Takes image and classifies into categories.

Semantic segmentation: Precise object boundary detection such as people edges in images but complex model.

Single Shot MultiBox Detector (SSD): real-time object detection algorithm.

Time Series Forecasting Terms (Traditional Statistics):

Autoregressive Integrated Moving Average (ARIMA) : Auto Regression (AR) + differencing (I) + moving average (MA). For simple, single-time series.

Convolutional Neural Network - Quantile Regression (CNN-QR): very complex time series for numbers. Main CNN is images.  very good if cost asymmetry requirements.

DeepAR+: very complex time series. Can learn from similar sales history if no history. Missing values in target should be ′null′ literals or ′NaN′ strings in JSON. 

ExponenTial Smoothing (ETS) = family of models. Error, Trend, and Seasonal pieces. calcs by exponentially decreasing weights over time smoothing by doing weighted average over past observations.

Prophet: time-series plus holiday and seasons

Text:

BlazingText = SL. text tasks such as classification (multi-class and multi-label). Captures word embeddings (via Word2Vec and FastText). Additional Modes: 1) continuous bag-of-words (CBOW) mode = Predicts a word from surrounding context; 2) skip-gram mode = Captures local word context

Latent Dirichlet Allocation (LDA): UL. NLP. topics. Dirichlet is a lazy (so UL) bible reader that looks through text (so NLP), finding different topics, and finds the theme by associations between topics. 

Neural Topic Model (NTM): UL. topic modeling into fixed, predefined categories using labeled historical data

Object2Vec = Captures semantic meaning and sequential dependencies

Misc:

Validation LogLoss = for rare events in imbalanced data