ASP.NET Docker containers in AWS

Integrating AI into a containerized .NET app on AWS creates a highly scalable, modern architecture. By leveraging AWS managed AI services (like Bedrock, SageMaker, Rekognition, or Polly) via the AWS SDK for .NET, you offload heavy ML compute to AWS while keeping your Docker containers lightweight. However, moving this architecture into a production-grade Docker and AWS environment introduces specific engineering challenges.

Architecture Overview

[ Frontend / Client ]
       │ (HTTPS)
       ▼
[ AWS ECS / EKS (Fargate) ] ──(VPC Endpoint)──► [ AWS AI Services ]
  └─► Docker Container                               (Bedrock, Rekognition, etc.)
       └─► ASP.NET Core API

Application: ASP.NET Core Web API running the official .NET 8 or .NET 9 runtime.
Container: Docker images deployed to ECS (Elastic Container Service) or EKS (Elastic Kubernetes Service) using Fargate (serverless containers).
AI Layer: The .NET application interacts with AWS AI services using the AWSSDK.BedrockRuntime or AWSSDK.Rekognition NuGet packages.

Challenges

Challenge 1: Securely Managing AWS Credentials in Docker

Problem: Hardcoding AWS access keys (AWS_ACCESS_KEY_ID) inside Dockerfiles or passing them via static environment variables creates a severe security vulnerability. If a container image is leaked, your entire AWS infrastructure could be compromised.
Solution: IAM Roles for Tasks. Assign an ECS Task Role (or an EKS Service Account via IRSA) directly to your container.
- In your C# code, initialize the AWS client using the DefaultAWSCredentials class.
- The AWS SDK for .NET is smart enough to auto detect and inherit the temporary, auto-rotating credentials provided by the ECS/EKS container environment.

Challenge 2: Network Latency & Model Timeouts

Problem: AI operations—especially Gen AI text generation via LLMs on Amazon Bedrock—are fundamentally slow. A standard HTTP request-response cycle in ASP.NET Core will time out if an AI model takes 30 seconds to generate a response, leading to a terrible user experience and thread pool starvation.
Solution: Streaming Responses and Asynchronous Processing.
- For Real-time AI (e.g., Chatbots): Use Bedrock's streaming APIs (InvokeModelWithResponseStreamAsync) combined with ASP.NET Core SignalR or IAsyncEnumerable. This streams words to the frontend client as they are being generated by the model, preventing timeouts.
- For Batch AI (e.g., Video/Document Analysis): Implement an asynchronous worker pattern. Have your API drop the task into an Amazon SQS queue, return an immediate 202 Accepted status to the client, and use a background .NET BackgroundService (Hosted Service) inside the container to process the AI task off the queue.

Challenge 3: Cold Starts and Docker Image Optimization

Problem: The standard .NET SDK container image is large. If you bundle heavy AI dependencies, third-party python scripts, or local fallback models inside your Docker image, your container size will balloon past 1 GB. This causes slow AWS Fargate scaling times (cold starts) when traffic spikes.
Solution: Multi-Stage Builds and Managed Endpoints.
- Keep your container strictly a "thin client". Use Docker multi-stage builds to compile the app in the SDK image, but deploy it using the lightweight, stripped-down .NET ASP.NET Runtime base image.
- Delegate 100% of the ML heavy lifting to AWS endpoints. If you must use custom models, host them on SageMaker Endpoints and query them via HTTP REST/gRPC rather than running ML models inside the ASP.NET container itself.

Challenge 4: Data Privacy and VPC Isolation

Problem: By default, when your container calls an AWS AI service, that traffic travels over the public internet to reach the AWS service endpoint.
Solution: VPC Interface Endpoints (AWS PrivateLink).

Configure VPC Endpoints inside your private subnets for the specific AI services you are using (e.g., com.amazonaws.us-east-1.bedrock-runtime).
This routes all AI payload data entirely through AWS’s private, internal fiber network. Your ASP.NET Docker container can communicate with the AI services securely without ever needing an internet gateway or public IP address.

Connection Specifics

To connect your containerized ASP.NET app to AWS AI services, you do not need to manually configure raw HTTP clients, manage TLS certificates, or explicitly pass authentication tokens. The AWS SDK for .NET handles all of this automatically under the hood. Here is exactly how C# resolves the endpoints, leverages the ECS Task Role, and executes the code.

1. Download AWS SDK for .NET

https://aws.amazon.com/sdk-for-net/

2. How C# Knows the AI's Endpoint

You do not need to hardcode specific URLs (like https://amazonaws.com) into your C# code.

Instead, the AWS SDK constructs the endpoint dynamically based on two things:

The Service Client: The specific class you instantiate (e.g., AmazonBedrockRuntimeClient, AmazonRekognitionClient).
The AWS Region: The region you pass to that client configuration.

When you configure your app, you simply specify the region (e.g., RegionEndpoint.USEast1), and the SDK auto maps it to the correct, highly available global AWS endpoint for that service.

3. How it Shares the ECS Task Role Credentials

When your container runs on AWS ECS, AWS injects an environment variable called AWS_CONTAINER_CREDENTIALS_RELATIVE_URI into your Docker container. This variable points to a local link-local metadata address hosted by the AWS ECS Agent.

The AWS SDK for .NET uses a fallback system called the Default Credentials Provider Chain. When you initialize an AWS client in C# without passing keys, it looks for credentials in this exact order:

System environment variables (AWS_ACCESS_KEY_ID).
IAM roles for Amazon ECS tasks / EKS Service accounts.
EC2 Instance Metadata Service (IMDS).

Because you assigned an IAM Task Role to your ECS Task Definition, the SDK auto hits the internal ECS metadata endpoint, fetches temporary, self-rotating credentials, and signs every outgoing AI request using AWS Signature Version 4 (SigV4). No manual credential sharing is required.

4. C# Code

To do this in modern ASP.NET Core (.NET 8+), use the official AWS Dependency Injection packages.

Step 1: Install the NuGet Packages

dotnet add package AWSSDK.Extensions.NETCore.Setup

dotnet add package AWSSDK.BedrockRuntime

dotnet add package AWSSDK.Rekognition

Step 2: Register Services in Program.cs

By calling AddAWSService, ASP.NET Core auto wires up the default credentials chain and looks at your config files for the default region.

var builder = WebApplication.CreateBuilder(builder.Args);

// 1. Read AWS options from appsettings.json (e.g., {"AWS": {"Region": "us-east-1"}})

var awsOptions = builder.Configuration.GetAWSOptions();

// 2. Register the AWS AI clients into the DI container

builder.Services.AddDefaultAWSOptions(awsOptions);

builder.Services.AddAWSService<IAmazonBedrockRuntime>();

builder.Services.AddAWSService<IAmazonRekognition>();

var app = builder.Build();

Step 3: Call the AI Service via a Controller or Endpoint

Inject the AI client directly into your classes. The SDK handles the endpoint and the ECS credentials behind the scenes.

Ex: Calls Amazon Bedrock (Anthropic Claude) asynchronously:

using Amazon.BedrockRuntime;

using Amazon.BedrockRuntime.Model;

using Microsoft.AspNetCore.Mvc;

using System.Text.Json;

[ApiController]

[Route("api/[controller]")]

public class AiController : ControllerBase

{

private readonly IAmazonBedrockRuntime _bedrockClient;

// The SDK client is injected automatically via ASP.NET Core DI

public AiController(IAmazonBedrockRuntime bedrockClient)

{

_bedrockClient = bedrockClient;

}

[HttpPost("generate-text")]

public async Task<IActionResult> GenerateText([FromBody] string userPrompt, CancellationToken cancellationToken)

{

// Construct the payload for the specific foundation model

var nativeRequest = new

{

anthropic_version = "bedrock-2023-05-31",

max_tokens = 500,

messages = new[] { new { role = "user", content = userPrompt } }

};

var requestJson = JsonSerializer.Serialize(nativeRequest);

// Define the request wrapper. The SDK targets the correct endpoint automatically.

var request = new InvokeModelRequest

{

ModelId = "anthropic.claude-3-sonnet-20240229-v1:0", // Specify the model ID

ContentType = "application/json",

Accept = "application/json",

Body = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(requestJson))

};

// The SDK uses the ECS Task Role credentials to securely sign this call

InvokeModelResponse response = await _bedrockClient.InvokeModelAsync(request, cancellationToken);

using var reader = new StreamReader(response.Body);

var responseBody = await reader.ReadToEndAsync();

return Ok(responseBody);

}

Summary

So overall:

"We use the standard AWS SDK for .NET packages. We don't manage credentials or endpoints manually. The SDK uses the Default Credentials Provider Chain to automatically detect and use the temporary tokens from the ECS Task Role environment variables. It also automatically builds the correct target URL based on the RegionEndpoint we pass to the client registration in Program.cs."

AWS Prep

How to secure an ASP.NET Core Web API hosted on AWS EC2?

Network: Place the EC2 instances inside a private subnet within a VPC.
Access: Route traffic through an Application Load Balancer sitting in a public subnet.
Firewall: Lock down EC2 Security Groups to accept inbound traffic only from the ALB.
Identity: Use AWS Secrets Manager to store the database connection strings securely instead of leaving them in appsettings.json.

Which AWS database service would you choose for a .NET application?

Relational: Use Amazon RDS (SQL Server) if you require complex joins, transaction integrity (ACID), and seamless migration for existing SQL schemas.
NoSQL: Use Amazon DynamoDB if the app requires single-digit millisecond latency, infinite scaling, and handles unstructured or document key-value data structures.

Search This Blog

Ones and Zeros