On-Premises Development Options

This document outlines the options and configurations for on-premises development of the AI Agent Orchestration Platform.

Overview

On-premises development provides complete control over the development environment, data security, and infrastructure. This approach is particularly valuable for organizations with strict security requirements, existing on-premises infrastructure, or specialized hardware needs (such as GPU acceleration for AI workloads).

Development Environment Options

1. Docker Compose Development Environment

The simplest and most portable option for on-premises development uses Docker Compose to create a consistent local environment.

Requirements

Docker Engine (20.10+)
Docker Compose (2.0+)
Git
8GB+ RAM
50GB+ disk space

Setup

Clone the repository:

git clone https://github.com/yourusername/meta-agent-platform.git
cd meta-agent-platform

Copy the example environment file:
```
cp .env.example .env
```
Edit the .env file to configure your environment variables.

Start the development environment:

docker-compose -f docker-compose.dev.yml up -d

Access the services:
Frontend: http://localhost:3000
Backend API: http://localhost:8000
API Documentation: http://localhost:8000/docs
PostgreSQL: localhost:5432
Monitoring: http://localhost:9090 (Prometheus), http://localhost:3001 (Grafana)

Docker Compose Configuration

# docker-compose.dev.yml
version: '3.8'

services:
  frontend:
    build:
      context: ./frontend
      dockerfile: Dockerfile.dev
    ports:
      - "3000:3000"
    volumes:
      - ./frontend:/app
      - /app/node_modules
    environment:
      - NODE_ENV=development
      - REACT_APP_API_URL=http://localhost:8000
    depends_on:
      - backend

  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile.dev
    ports:
      - "8000:8000"
    volumes:
      - ./backend:/app
      - /app/.venv
    environment:
      - ENVIRONMENT=development
      - DATABASE_URL=postgresql://postgres:postgres@db:5432/meta_agent_platform
      - CORS_ORIGINS=http://localhost:3000
    depends_on:
      - db

  db:
    image: postgres:15
    ports:
      - "5432:5432"
    volumes:
      - postgres_data:/var/lib/postgresql/data
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
      - POSTGRES_DB=meta_agent_platform

  prometheus:
    image: prom/prometheus
    ports:
      - "9090:9090"
    volumes:
      - ./infra/prometheus:/etc/prometheus
      - prometheus_data:/prometheus

  grafana:
    image: grafana/grafana
    ports:
      - "3001:3000"
    volumes:
      - ./infra/grafana:/etc/grafana/provisioning
      - grafana_data:/var/lib/grafana
    depends_on:
      - prometheus

volumes:
  postgres_data:
  prometheus_data:
  grafana_data:

2. Kubernetes Development Environment

For teams familiar with Kubernetes or requiring a development environment that closely mirrors production, a local Kubernetes cluster provides a more sophisticated option.

Requirements

Kubernetes cluster (Minikube, k3s, k3d, or Kind)
kubectl CLI
Helm
16GB+ RAM
100GB+ disk space

Setup with Minikube

Install Minikube:

# macOS
brew install minikube

# Linux
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikube

Start Minikube with sufficient resources:

minikube start --cpus 4 --memory 8192 --disk-size 50g

Enable necessary addons:

minikube addons enable ingress
minikube addons enable metrics-server

Deploy the platform using Helm:

# Add the Helm repository (if using a private repository)
helm repo add meta-agent-platform https://your-helm-repo.example.com
helm repo update

# Install the platform
helm install meta-agent-platform ./infra/helm/meta-agent-platform \
  --values ./infra/helm/meta-agent-platform/values.dev.yaml

Access the services:

# Get the ingress URL
minikube service list

# Or set up port forwarding
kubectl port-forward svc/meta-agent-platform-frontend 3000:80
kubectl port-forward svc/meta-agent-platform-backend 8000:80

3. Bare Metal Development Environment

For organizations with existing hardware or requiring maximum performance, a bare metal installation provides direct access to system resources.

Requirements

Linux server/workstation (Ubuntu 22.04 LTS recommended)
Python 3.10+
Node.js 18+
PostgreSQL 15+
16GB+ RAM
100GB+ SSD storage
Optional: NVIDIA GPU with CUDA support for AI workloads

Setup

Install system dependencies:

# Ubuntu/Debian
sudo apt update
sudo apt install -y python3 python3-pip python3-venv nodejs npm postgresql postgresql-contrib

# Optional: NVIDIA CUDA for GPU support
sudo apt install -y nvidia-cuda-toolkit

Clone the repository:

git clone https://github.com/yourusername/meta-agent-platform.git
cd meta-agent-platform

Set up the backend:

cd backend
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# Configure environment variables
cp .env.example .env
# Edit .env with your configuration

# Initialize the database
python manage.py migrate

# Start the backend server
python manage.py runserver 0.0.0.0:8000

Set up the frontend:

cd ../frontend
npm install

# Configure environment variables
cp .env.example .env
# Edit .env with your configuration

# Start the frontend development server
npm run dev

Set up monitoring (optional):

# Install Prometheus
cd ../infra/prometheus
docker run -d -p 9090:9090 -v $(pwd)/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus

# Install Grafana
cd ../grafana
docker run -d -p 3001:3000 -v $(pwd)/provisioning:/etc/grafana/provisioning grafana/grafana

4. Virtual Machine Development Environment

For teams requiring isolation without the overhead of containerization, virtual machines provide a good middle ground.

Requirements

Virtualization software (VMware, VirtualBox, Hyper-V)
Ubuntu 22.04 LTS VM image
8GB+ RAM allocated to VM
50GB+ disk space allocated to VM

Setup with Vagrant

Vagrant provides a consistent way to create and manage virtual machine environments.

Install Vagrant and VirtualBox:

# macOS
brew install vagrant virtualbox

# Ubuntu/Debian
sudo apt install vagrant virtualbox

Create a Vagrantfile:

# Vagrantfile
Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/jammy64"
  config.vm.hostname = "meta-agent-dev"

  # Network configuration
  config.vm.network "forwarded_port", guest: 3000, host: 3000  # Frontend
  config.vm.network "forwarded_port", guest: 8000, host: 8000  # Backend
  config.vm.network "forwarded_port", guest: 5432, host: 5432  # PostgreSQL
  config.vm.network "forwarded_port", guest: 9090, host: 9090  # Prometheus
  config.vm.network "forwarded_port", guest: 3001, host: 3001  # Grafana

  # Resource allocation
  config.vm.provider "virtualbox" do |vb|
    vb.memory = 8192
    vb.cpus = 4
  end

  # Provisioning script
  config.vm.provision "shell", inline: <<-SHELL
    # Update system
    apt-get update
    apt-get upgrade -y

    # Install dependencies
    apt-get install -y python3 python3-pip python3-venv nodejs npm postgresql postgresql-contrib docker.io docker-compose

    # Enable Docker
    systemctl enable docker
    systemctl start docker
    usermod -aG docker vagrant

    # Clone repository
    git clone https://github.com/yourusername/meta-agent-platform.git /home/vagrant/meta-agent-platform
    chown -R vagrant:vagrant /home/vagrant/meta-agent-platform
  SHELL
end

Start the VM:
```
vagrant up
```

SSH into the VM and start the development environment:

vagrant ssh
cd meta-agent-platform
docker-compose -f docker-compose.dev.yml up -d

Development Tools and Utilities

Local Development Scripts

The platform includes several scripts to simplify development tasks:

infra/scripts/setup-dev.sh: Set up the development environment
infra/scripts/start-dev.sh: Start all development services
infra/scripts/stop-dev.sh: Stop all development services
infra/scripts/reset-db.sh: Reset the development database
infra/scripts/run-tests.sh: Run the test suite

Example usage:

# Set up development environment
./infra/scripts/setup-dev.sh

# Start development services
./infra/scripts/start-dev.sh

# Run tests
./infra/scripts/run-tests.sh

Development Database Management

For database management during development:

Database Migrations:

# Create a new migration
cd backend
python manage.py makemigrations

# Apply migrations
python manage.py migrate

Seed Data:

# Load seed data
python manage.py loaddata initial_data

Database Backup and Restore:

# Backup
pg_dump -U postgres -d meta_agent_platform > backup.sql

# Restore
psql -U postgres -d meta_agent_platform < backup.sql

Local Testing

The platform includes comprehensive testing capabilities:

Backend Tests:
```
cd backend
pytest
```
Frontend Tests:
```
cd frontend
npm test
```

End-to-End Tests:

# Start the development environment
docker-compose -f docker-compose.dev.yml up -d

# Run E2E tests
cd e2e
npm test

GPU Support for AI Workloads

For development environments requiring GPU acceleration for AI workloads:

Docker Compose with GPU Support

# docker-compose.gpu.yml
version: '3.8'

services:
  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile.dev
    ports:
      - "8000:8000"
    volumes:
      - ./backend:/app
      - /app/.venv
    environment:
      - ENVIRONMENT=development
      - DATABASE_URL=postgresql://postgres:postgres@db:5432/meta_agent_platform
      - CORS_ORIGINS=http://localhost:3000
      - CUDA_VISIBLE_DEVICES=0
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    depends_on:
      - db

  # Other services...

Start with GPU support:

docker-compose -f docker-compose.gpu.yml up -d

Bare Metal with GPU Support

For bare metal installations, ensure NVIDIA drivers and CUDA are properly installed:

# Check NVIDIA driver installation
nvidia-smi

# Check CUDA installation
nvcc --version

# Install PyTorch with CUDA support
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

Multi-User Development Environment

For teams requiring a shared development environment:

GitPod Configuration

# .gitpod.yml
image:
  file: .gitpod.Dockerfile

ports:
  - port: 3000
    onOpen: open-preview
  - port: 8000
    onOpen: ignore
  - port: 5432
    onOpen: ignore

tasks:
  - name: Backend
    init: |
      cd backend
      python -m venv .venv
      source .venv/bin/activate
      pip install -r requirements.txt
    command: |
      cd backend
      source .venv/bin/activate
      python manage.py migrate
      python manage.py runserver 0.0.0.0:8000

  - name: Frontend
    init: |
      cd frontend
      npm install
    command: |
      cd frontend
      npm run dev

vscode:
  extensions:
    - ms-python.python
    - dbaeumer.vscode-eslint
    - esbenp.prettier-vscode

VS Code Dev Containers

// .devcontainer/devcontainer.json
{
  "name": "Meta Agent Platform Development",
  "dockerComposeFile": "../docker-compose.dev.yml",
  "service": "backend",
  "workspaceFolder": "/app",
  "settings": {
    "python.defaultInterpreterPath": "/app/.venv/bin/python",
    "python.linting.enabled": true,
    "python.linting.pylintEnabled": true
  },
  "extensions": [
    "ms-python.python",
    "ms-python.vscode-pylance",
    "dbaeumer.vscode-eslint",
    "esbenp.prettier-vscode"
  ],
  "forwardPorts": [3000, 8000, 5432, 9090, 3001],
  "postCreateCommand": "pip install -r requirements.txt"
}

Troubleshooting Common Issues

Docker Compose Issues

Container fails to start:

# Check container logs
docker-compose logs [service_name]

# Rebuild container
docker-compose build --no-cache [service_name]

Port conflicts:

# Find process using the port
lsof -i :3000

# Kill the process
kill -9 [PID]

Database Issues

Connection errors:

# Check PostgreSQL service
sudo systemctl status postgresql

# Restart PostgreSQL
sudo systemctl restart postgresql

Migration errors:

# Reset migrations
cd backend
python manage.py migrate --fake zero
python manage.py migrate

Environment Setup Issues

Python virtual environment problems:

# Remove and recreate virtual environment
rm -rf .venv
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Node.js dependency issues:

# Clear npm cache
npm cache clean --force

# Reinstall dependencies
rm -rf node_modules
npm install

References

Last updated: 2025-04-18