Azure Environment
Azure Environment
- External Identity Provider Integration: Okta acts as the centralized OAuth 2.0 and OpenID Connect (OIDC) Auth Server, decoupling identity mgmt from Azure hosting.
- JWT Validation: Microservices validate incoming JSON Web Tokens (JWTs) statelessly using the public keys published via Okta's JSON Web Key Set (JWKS) endpoint.
- Token Verification: Apps must validate token signatures, expiration (
exp), activation (nbf), issuer (iss), and target audience (aud) prior to processing requests.
- Layered Edge Defense: Ingress points (such as Azure API Mgmt) enforce dual verification by requiring an explicit subscription API Key alongside the identity token.
- Separation of Concerns: API Keys handle North-South traffic routing, rate limiting, and client id. Bearer tokens handle East-West service comms, identification, and contextual authorization.
- Rotation Policies: API keys require deterministic rotation schedules handled through secure automation, avoiding hardcoded dependencies in app code.
- Granular Permission Enforcement: Fine-grained auth relies on custom scopes (e.g.,
read:reports,write:orders) embedded within the token claims payload. - Verification Points: App checks presence and scope validity before doing business domain use cases.
- Principle of Least Privilege: Clients get min scope for trans type rather than broad admin rights.
Azure Storage Queues: Simple, high-scale. Max 64KB payload. No native Dead-Letter Queue. Simple polling model. No native trans support. Should use Idempotent Consumer Pattern: 1) create key, 2) if exist check, 3) insert or skip.
Azure Service Bus: Enterprise-grade. Max 100MB payload. Dup detection looking at message ID. Message sessions (with FIFO ordering of messages). Pub/Sub via Topics and Subscriptions.
idempotent = can run multiple times w/o changing the final outcome.
Retries and Exponential Backoff Strategies
Failures (network drops, database throttling) are handled with retries that back off over time to avoid overwhelming downstream services.
Azure Functions Built-In Retries = set Retry policies in the host.json file or via code attributes on individual functions.
ServiceBusClient or QueueClient.- Native Isolation: Creates an auto sub-queue called
$DeadLetterQueuefor every queue or subscription. - Trigger Mechanisms: Messages are moved to the DLQ when:
MaxDeliveryCountis exceeded.- Message expires (
TimeToLivepasses) and dead-lettering on expiry is enabled. - Consumer app explicitly calls
.DeadLetterAsync()after catching a known invalid payload exception.
- Lack of Native DLQs: Storage Queues do not have a secondary sub-queue.
- Handling Poison Messages: When a message fails, its
DequeueCountproperty increments. Your consumer app (or the Azure Functions Storage Queue trigger) must check this property. - Default: If Azure Function fails to run 5 times for a Storage Queue message, the Function runtime writes poison message to a separate poison queue.
- Trigger Types: Use
ServiceBusTriggerfor Service Bus Queues/Topics, andQueueTriggerfor Storage Queues. - Message Settlement (Service Bus):
- Peek-Lock (Default): Locks the message. If success, the runtime calls
Completeto delete it. If error, the lock expires and message returns to the queue. - Receive-and-Delete: Message is deleted from the broker the instant it is fetched. Do not use this if you need reliability guarantees.
- Peek-Lock (Default): Locks the message. If success, the runtime calls
- Concurrency Controls: Prevent scaling out too rapidly and overwhelming downstream DBs by parameters
maxConcurrentCalls(Service Bus) orbatchSize(Storage Queues) inside thehost.jsonfile.
- Centralized State Store: Consolidates app settings and feature flags into a secure, unified Azure resource.
- Feature Management: Enables dynamic feature flags (decoupling code deployment from feature activation).
- Dynamic Config Refresh: Sets cache expiration intervals and sentinel keys to allow running apps to pull config updates w/o restarts.
- Key-Vault: Secures secrets by mapping App Config pointers to keys in Azure Key Vault (preventing secret exposure).
- Immutable Build Packages: CI compile tasks generate a single, signed build artifact that propagates through testing to production.
- Config Extraction Pattern: Build packages remain entirely decoupled from environment details; config injection occurs strictly during the CD release stage.
- Safe Deployment Topologies: Release pipelines mandate structured progressive rollouts via multi-stage environments protected by automated gate checks and approval steps.
- Distributed Tracing Execution: Gens unique IDs that cross network and boundary lines, mapping end-to-end distributed system call trees.
- Core Telemetry Ingestion: Captures 3 distinct telemetry types natively: metrics (numerical aggregates), logs (structured string statements), and traces (system tracks).
- Auto-Collection Capabilities: Out-of-the-box tracking captures external dependency call latencies, unhandled app exceptions, and HTTP server response rates.
- Log Context Enrichment: Custom properties, execution correlations, and user context keys get injected into all trace states to simplify diagnostic querying.
- Sampling Optimization: Configures adaptive sampling rates to reduce storage ingest bills and protect app throughput without losing anomalous event signals.
- Impact-Driven Classifications: Severity ratings align directly to calc business impact, separating critical outages (Sev-1) from minor bugs (Sev-3).
- Incident Commander Model: Establishes a single leader to delegate tasks, manage external updates, and isolate engineers working on technical fixes.
- Communication Mechanics: Maintains active, separate bridges for internal debugging operations and external customer-facing progress updates.
- Blameless Culture Standards: Analysis sessions isolate structural system gaps, broken automated configurations, and code bugs rather than human mistakes.
- Timeline Compilation: Builds a breakdown by minute showing system telemetry changes, alert triggers, human actions, and remediation steps.
- Actionable Corrective Items: Post-mortem reviews generate explicit, prioritized backlog tickets to prevent the exact failure pattern from repeating.
- Executable Runbooks: Tech manuals outline step-by-step procedures for alert validation, manual failovers, configs, and service restorations.
- Service Level Objectives (SLOs): Performance benchmarks (e.g. 99.9% success rate).
- Service Level Indicators (SLIs): Direct, real-time measurements (such as HTTP 5xx error counts) to track compliance against established SLO limits.
- Error Budget Mgmt: If incidents exhaust the defined SLO error budget, engineering teams pause feature dev to focus entirely on stability.
Comments
Post a Comment