attune/docs/QUICKREF-containerized-workers.md

Quick Reference: Containerized Workers

Worker Variants

Variant	Runtimes	Size	Use Case
worker-base	shell	~580MB	Shell scripts, basic automation
worker-python	shell, python	~1.2GB	Python actions, scripts with deps
worker-node	shell, node	~760MB	JavaScript/TypeScript actions
worker-full	shell, python, node, native	~1.6GB	Multi-language packs

Quick Start

# Build all worker images
make docker-build-workers

# Start specific worker
docker-compose up -d worker-python

# Start all workers
docker-compose up -d worker-shell worker-python worker-node worker-full

# Check worker status
docker-compose exec postgres psql -U attune -d attune -c \
  "SELECT name, worker_type, status, capabilities->>'runtimes' as runtimes FROM worker;"

Environment Variables

Variable	Description	Example
ATTUNE_WORKER_NAME	Unique worker ID	worker-python-01
ATTUNE_WORKER_RUNTIMES	Runtime capabilities	shell,python
ATTUNE_WORKER_TYPE	Worker deployment type	container
ATTUNE__DATABASE__URL	Database connection	postgresql://...
ATTUNE__MESSAGE_QUEUE__URL	RabbitMQ connection	amqp://...
RUST_LOG	Logging level	info, debug

Runtime Capability Declaration

Priority order:

1. Environment variable (highest): ATTUNE_WORKER_RUNTIMES="shell,python"
2. Config file: worker.capabilities.runtimes: ["shell", "python"]
3. Auto-detection (fallback): Checks for runtime binaries

Build Commands

# Individual builds
make docker-build-worker-base      # Shell only
make docker-build-worker-python    # Python + shell
make docker-build-worker-node      # Node.js + shell
make docker-build-worker-full      # All runtimes

# Direct Docker build
DOCKER_BUILDKIT=1 docker build \
  --target worker-python \
  -t attune-worker:python \
  -f docker/Dockerfile.worker .

Scaling Workers

# Scale specific worker type
docker-compose up -d --scale worker-python=3

# Scale multiple types
docker-compose up -d \
  --scale worker-python=3 \
  --scale worker-shell=2

Custom Worker Images

# Extend Python worker with ML packages
FROM attune-worker:python

USER root
RUN pip install --no-cache-dir pandas numpy scikit-learn
USER attune

ENV ATTUNE_WORKER_RUNTIMES="shell,python,ml"

Monitoring

# View logs
docker-compose logs -f worker-python

# Check container status
docker-compose ps | grep worker

# Resource usage
docker stats attune-worker-python

# Worker registration
docker-compose exec postgres psql -U attune -d attune -c \
  "SELECT name, status, last_heartbeat, capabilities FROM worker;"

# Active executions per worker
docker-compose exec postgres psql -U attune -d attune -c \
  "SELECT w.name, COUNT(e.id) as active 
   FROM worker w 
   LEFT JOIN execution e ON e.worker = w.id AND e.status = 'running' 
   GROUP BY w.name;"

Troubleshooting

Worker not registering

# Check database connectivity
docker-compose logs worker-python | grep -i database

# Verify environment
docker-compose exec worker-python env | grep ATTUNE

# Force restart
docker-compose restart worker-python

Runtime not detected

# Check runtime availability
docker-compose exec worker-python python3 --version

# Force runtime declaration
docker-compose exec worker-python sh -c \
  'echo $ATTUNE_WORKER_RUNTIMES'

Actions not scheduled

-- Check runtime compatibility
SELECT a.ref, r.name as required_runtime
FROM action a
JOIN runtime r ON a.runtime = r.id
WHERE a.ref = 'core.my_action';

-- Check worker capabilities
SELECT name, capabilities->>'runtimes' as available_runtimes
FROM worker
WHERE status = 'active';

Resource Limits

# docker-compose.override.yml
services:
  worker-python:
    deploy:
      resources:
        limits:
          cpus: '2.0'
          memory: 2G
        reservations:
          cpus: '0.5'
          memory: 512M

Security

• ✅ Non-root execution (user attune, UID 1000)
• ✅ Read-only pack mounts (:ro flag)
• ✅ Network isolation (dedicated Docker network)
• ✅ Health checks (process monitoring)
• ✅ Minimal base images (Debian slim variants)

Common Tasks

# Stop all workers
docker-compose stop worker-shell worker-python worker-node worker-full

# Remove workers
docker-compose rm -f worker-python

# Rebuild and restart
docker-compose up -d --build worker-python

# Execute shell in worker
docker-compose exec worker-python bash

# View worker config
docker-compose exec worker-python cat /opt/attune/config.yaml

How It Works

1. Worker starts → Reads ATTUNE_WORKER_RUNTIMES or detects runtimes
2. Registration → Stores capabilities in worker.capabilities JSON field
3. Executor schedules action → Checks action's runtime requirement
4. Worker selection → Finds workers with matching runtime in capabilities
5. Action execution → Sends to compatible worker via message queue

Files

• docker/Dockerfile.worker - Multi-stage worker Dockerfile
• docker-compose.yaml - Worker service definitions
• crates/worker/src/registration.rs - Capability detection logic
• docs/worker-containerization.md - Full design document
• docs/worker-containers-quickstart.md - Detailed guide

Further Reading

• Worker Containerization Design
• Worker Containers Quick Start
• Worker Service Architecture
• Docker README