AI Exam Prep: 3E - Metrics

 

AI Practitioner Exam Prep - Metrics

Text Metric Terms

BERT = Text. scores semantic similarity (using cosine) between two sentences. Good for text generation, summarization, translation, and filling in missing words. think brainy Bert. Counts: words that mean the same.

BLEU = Text. scores exact word translation did against human translated sample(s). Counts: exact words only. think translation because it is foreign spelling of blue. 

METEOR = Text. evaluates by exact matches, stemmed matches, and synonyms. focus on both recall and precision. think poetry meter. Counts: synonyms and word order.

Perplexity = Text. calcs how well probability model predicts a sample token, not the text quality. Counts: Confusion level of model of its prediction.

ROUGE = Text. uses F1, precision, and recall versus a reference text. Good for text summary. think rogue makeup is for highlights (summaries). Counts: summary points.

TF-IDF = Text. scores word importance in docs by doing local count x log of (global rarity). Lessens filler words like "the" or "an".

Other Metric Terms

Conversion rate = measures customers that do action

RWK = measures if output aligns with known, real-world facts and info. high is accurate to real world.