AI Environment

  I was asked how to create an environment for AI that would I do for a business firm that is not a software firm. Thought it was a great question faced by many people.  This blog entry is an attempt to answer that question:

1) Get and Install Docker

Download Docker.

2) Create Docker Container (Sandbox) with Config file

   The safest and easiest way to isolate your AI is running Ollama in a Docker container.  

  Why Docker? Docker creates a strict security sandbox. The AI cannot see or touch the host computer's files unless you explicitly map a specific folder to it.  You can configure the Docker container with a restricted network bridge, preventing the AI from reaching the internet or sensitive company servers. Docker 

   A suggested Docker config file is below.

3) Get Ollama to Drive Your AI

       Why Ollama? Ollama is an open-source tool and framework designed to run, manage, and orchestrate various LLMs locally.  Ollama runs dozens of different open-weight LLMs, including Meta's Llama, Alibaba's Qwen, Google's Gemma, and Microsoft's Phi. Ollama requires LLMs to be quantized into the GGUF format (the format used by llama.cpp) and configured with a Modelfile. Ollama does not fine-tune LLMs.

4) Get Open-Weight LLM (in GGUF format) Tailored for Your Needs

https://huggingface.co/

5) Do Safety Check of LLM

6) Get Workflow Harness: n8n or Flowise

Since the company does not have a software engineering team, avoid complex coding frameworks like LangChain or AutoGen. Use a Low-Code/No-Code workflow harness. 

• n8n (Community Edition): A visual node-based automation tool you can host locally. It lets employees build AI workflows (e.g., "Read this email, summarize it, and save it to this folder") using a drag-and-drop interface. Excels at tasks like: "Watch this shared folder, when a new PDF appears, run it through the local AI, and type the summary into an Excel sheet."
• Flowise / Dify: Graphic user interfaces built specifically for stitching together local AI models, memory blocks, and company documents into functional internal apps. Good for internal ChatGPT-style chatbot clone for your employees that uses your local company documents (RAG) as its knowledge base. 

7) Setup Workflow Harness: n8n or Flowise

1. The Input: You build a simple visual form in n8n where an employee uploads a PDF file using their web browser.
2. The Processing: n8n takes that uploaded file binary, feeds the text into Ollama (Llama 3), and gets the summary.
3. The Output: n8n passes that text summary to the Filesystem MCP Server container, which writes it directly to your company's network storage drive or a designated archival folder.

8) OS Actions: Filesystem MCP Server

Yes to MCP, No to Direct Code Execution. Allowing a local AI to write and execute arbitrary Python or Bash code to perform OS tasks (like creating directories) is highly dangerous. Instead use Model Context Protocol (MCP) or static tool bindings.

• Filesystem MCP Server: The best at the moment.
• Git MCP Server: Allows the AI to read, search, and manage local version-controlled repositories safely without terminal access. 
• Docker MCP Server: A highly rated community server that gives an AI the ability to safely monitor, start, or stop specific containerized services on your local machine without full root terminal privileges. 
• Fetch MCP Server: An official utility tool that allows the local AI to securely pull down web pages and convert them to clean text for processing, without giving the AI an open web browser. 

  TRAPS: 1) Dependency on Node.js: Official open-source MCP servers are written in TypeScript/JavaScript. Your Docker container will need Node.js installed inside it (as shown in the node:20-alpine example above) to run the npx command.  2) "Docker in Docker" Trap: If you decide to use the Docker MCP server (to let the AI manage other containers), that specific container needs access to the host's /var/run/docker.sock. Only do this if you absolutely trust the workflows, as it gives the AI a way to bypass standard container restrictions.

9) Prompt Filtering & Guardrails: Llama Guard

To prevent bad actors, malicious prompts, or accidental data leaks, implement a two-tier defense layer right before the prompt hits your main model.

• The Guardrail Model (Llama Guard): Run a tiny, specialized safety model alongside your main AI. Every user prompt is first sent to Llama Guard, which classifies it as safe or unsafe based on company policies. If unsafe, the request is instantly blocked.
• Input Regex & Keyword Filtering: Use your workflow harness (n8n) to scan incoming prompts for specific forbidden keywords, sensitive data formats (like social security numbers or credit cards), or known system injection phrases (e.g., "ignore previous instructions").
• Output Sanitization: Run the AI's response back through Llama Guard before showing it to the user to guarantee the model didn't generate harmful, inaccurate, or forbidden data.

Security Layers:

Static Code & Image Scan   Snyk Container    Finds out-of-date or buggy code packages inside your Docker layers before deployment.

Runtime Security       Falco / KubeArmor     Blocks unauthorized file access or sudden terminal executions while the containers are running.

AI Text & Alignment  Llama Guard   Blocks malicious user text, prompt injections, and data leaks.

Network Security   Docker Internal Bridge  Prevents outside hackers on your Wi-Fi from directly touching the AI container infrastructure.

Suggested Docker Config file:

version: '3.8'

services:

  # 1. THE AI ENGINE

  ollama:

    image: ollama/ollama:latest

    container_name: local-ai-engine

    volumes:

      - ollama_storage:/root/.ollama

    # If using a computer with an Nvidia GPU, uncomment the lines below to make it fast:

    # deploy:

    #   resources:

    #     reservations:

    #       devices:

    #         - driver: nvidia

    #           count: all

    #           capabilities: [gpu]

    networks:

      - ai-network

  # 2. THE WORKFLOW HARNESS (n8n)

  n8n:

    image: docker.n8n.io/n8nio/n8n:latest

    container_name: local-ai-workflow

    ports:

      - "5678:5678" # This opens a website at http://localhost:5678 to build workflows

    volumes:

      - n8n_storage:/home/node/.n8n

    environment:

      - N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=true

    depends_on:

      - ollama

    networks:

      - ai-network

  # 3. THE OS FILESYSTEM MCP SERVER

  mcp-filesystem:

    image: node:20-alpine

    container_name: local-ai-os-tools

    volumes:

      # Maps a physical folder on your computer to a safe sandbox inside the container

      - C:\CompanyData\AI_Sandbox:/data/sandbox

    # Command runs the official Anthropic file tools restricted ONLY to the sandbox folder

    command: npx -y @modelcontextprotocol/server-filesystem /data/sandbox

    networks:

      - ai-network

volumes:

  ollama_storage:

  n8n_storage:

networks:

  ai-network:

    driver: bridge