AI Exam 5 - AI Aspects
AI Practitioner Exam Prep - Aspects of AI
Cost Considerations
Cost factors are responsiveness and availability, redundancy and regional coverage, performance, token-based pricing, provisioned throughput, and refining your custom models.
Responsible AI
Aspects: controllability, explainability, fairness, governance, interpretability\transparency, privacy\security (theft and exposure risk), safety, and veracity\robustness. Business Benefits: trust, regulatory complain, mitigate risks, competitive advantage, improved decision making, and improved products.
Model Selection: a) Narrow the use case to tune your model to your use case. Ex: favor recall or precision, b) pick by performance with some test data sets, c) responsible agency, d) environmental reasons, e) economical reasons.
Dataset Prep: want balanced dataset so inclusive and diverse in data collection, curating by 1) preprocessing, 2) augmentation, and 3) regular auditing.
Model Tradeoffs: a) interpretability vs. performance (wrong - see Lonestar), b) safety vs. transparency (wrong - see open source), c) control over the model.
Human Centered Design: amplified decision making, unbiased decision making, RLHF.
Benchmarking Datasets
Way 1: Humans evaluate by 1) human SMEs create ?s. 2) context identified, 3) answer created.
Way 2: LLMs as judges approach grades looking at answers compared to benchmarking datasets.
Security and Compliance in AI
AWS supports 143 security standards and compliance certifications, such as GDPR, HIPPA, ISO, PCI DSS, etc.
Security scopes: Consumer app, Enterprise app, Pre-trained models, Fine-tuned models, and self-trained models.
OWASP Top 10 AI Security risks
Prompt injection: Malicious inputs that change model behavior.
Insecure output: Failing to sanitize or validate model outputs.
Data poisoning: Injecting bad data into training to cause bad behavior.
Model DoS: Disrupting availability by exploiting architectural flaws.
Supply chain gaps: Weaknesses in the software, hardware, or services used.
Data disclosure: Leaking sensitive info via outputs or unintended channels.
Insecure plugins: Flaws in optional model components.
Excessive agency: Granting a model too much autonomy.
Overreliance: Over-dependence on a model's capabilities.
Model theft: Unauthorized copying of model parameters or architecture.
Insecure output: Failing to sanitize or validate model outputs.
Data poisoning: Injecting bad data into training to cause bad behavior.
Model DoS: Disrupting availability by exploiting architectural flaws.
Supply chain gaps: Weaknesses in the software, hardware, or services used.
Data disclosure: Leaking sensitive info via outputs or unintended channels.
Insecure plugins: Flaws in optional model components.
Excessive agency: Granting a model too much autonomy.
Overreliance: Over-dependence on a model's capabilities.
Model theft: Unauthorized copying of model parameters or architecture.
Prompt template theft is possibility.
Overall the risks are fake content, prompt injection, and AI model weaknesses.
Overall to secure data, you should control user access to the data and ensure data integrity.
Data Governance
Strategies: data quality and integrity, data protection and security, data lifecycle mgmt., responsible AI, governance structures and roles, and data sharing and collection.
Approaches: policies, review cadence, review strategies, transparency, and team training standards.
MLOps
MLOps = features: 1) model versioning: ensures reproducibility and rollbacks, 2) automated testing: validates models before deploy.
Misc
AI agents in multi-step tasks = are important in task coordination such as task sequence.
Model pruning = reducing model size and complexity.
Comments
Post a Comment