Files

David Culbreth 8ba7e3bb84 [wip] universal workers

2026-03-21 07:32:11 -05:00

7.2 KiB

Raw Blame History

Quick Reference: Agent-Based Workers

TL;DR: Inject the attune-agent binary into any container image to turn it into an Attune worker. No Dockerfiles. No Rust compilation. ~12 lines of YAML.

How It Works

The init-agent service (in docker-compose.yaml) builds the statically-linked attune-agent binary and copies it into the agent_bin volume
Your worker service mounts agent_bin read-only and uses the agent as its entrypoint
On startup, the agent auto-detects available runtimes (Python, Ruby, Node.js, Shell, etc.)
The worker registers with Attune and starts processing executions

Quick Start

Option A: Use the override file

# Start all services including the example Ruby agent worker
docker compose -f docker-compose.yaml -f docker-compose.agent.yaml up -d

The docker-compose.agent.yaml file includes a ready-to-use Ruby worker and commented-out templates for Python 3.12, GPU, and custom images.

Option B: Add to docker-compose.override.yaml

Create a docker-compose.override.yaml in the project root:

services:
  worker-my-runtime:
    image: my-org/my-custom-image:latest
    container_name: attune-worker-my-runtime
    depends_on:
      init-agent:
        condition: service_completed_successfully
      init-packs:
        condition: service_completed_successfully
      migrations:
        condition: service_completed_successfully
      postgres:
        condition: service_healthy
      rabbitmq:
        condition: service_healthy
    entrypoint: ["/opt/attune/agent/attune-agent"]
    stop_grace_period: 45s
    environment:
      RUST_LOG: info
      ATTUNE_CONFIG: /opt/attune/config/config.yaml
      ATTUNE_WORKER_NAME: worker-my-runtime-01
      ATTUNE_WORKER_TYPE: container
      ATTUNE__SECURITY__JWT_SECRET: ${JWT_SECRET:-docker-dev-secret-change-in-production}
      ATTUNE__SECURITY__ENCRYPTION_KEY: ${ENCRYPTION_KEY:-docker-dev-encryption-key-please-change-in-production-32plus}
      ATTUNE__DATABASE__URL: postgresql://attune:attune@postgres:5432/attune
      ATTUNE__MESSAGE_QUEUE__URL: amqp://attune:attune@rabbitmq:5672
      ATTUNE_API_URL: http://attune-api:8080
    volumes:
      - agent_bin:/opt/attune/agent:ro
      - ${ATTUNE_DOCKER_CONFIG_PATH:-./config.docker.yaml}:/opt/attune/config/config.yaml:ro
      - packs_data:/opt/attune/packs:ro
      - runtime_envs:/opt/attune/runtime_envs
      - artifacts_data:/opt/attune/artifacts
    healthcheck:
      test: ["CMD-SHELL", "pgrep -f attune-agent || exit 1"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 20s
    networks:
      - attune-network
    restart: unless-stopped

Then run:

docker compose up -d

Docker Compose automatically merges docker-compose.override.yaml.

Required Volumes

Every agent worker needs these volumes:

Volume	Mount Path	Mode	Purpose
`agent_bin`	`/opt/attune/agent`	`ro`	The statically-linked agent binary
`packs_data`	`/opt/attune/packs`	`ro`	Pack files (actions, workflows, etc.)
`runtime_envs`	`/opt/attune/runtime_envs`	`rw`	Isolated runtime environments (venvs, node_modules)
`artifacts_data`	`/opt/attune/artifacts`	`rw`	File-backed artifact storage
Config YAML	`/opt/attune/config/config.yaml`	`ro`	Attune configuration

Required Environment Variables

Variable	Description	Example
`ATTUNE_CONFIG`	Path to config file inside container	`/opt/attune/config/config.yaml`
`ATTUNE_WORKER_NAME`	Unique worker name	`worker-ruby-01`
`ATTUNE_WORKER_TYPE`	Worker type	`container`
`ATTUNE__DATABASE__URL`	PostgreSQL connection string	`postgresql://attune:attune@postgres:5432/attune`
`ATTUNE__MESSAGE_QUEUE__URL`	RabbitMQ connection string	`amqp://attune:attune@rabbitmq:5672`
`ATTUNE__SECURITY__JWT_SECRET`	JWT signing secret	(use env var)
`ATTUNE__SECURITY__ENCRYPTION_KEY`	Encryption key for secrets	(use env var)

Optional Environment Variables

Variable	Description	Default
`ATTUNE_WORKER_RUNTIMES`	Override auto-detection	Auto-detected
`ATTUNE_API_URL`	API URL for token generation	`http://attune-api:8080`
`RUST_LOG`	Log level	`info`

Runtime Auto-Detection

The agent probes for these runtimes automatically:

Runtime	Probed Binaries
Shell	`bash`, `sh`
Python	`python3`, `python`
Node.js	`node`, `nodejs`
Ruby	`ruby`
Go	`go`
Java	`java`
R	`Rscript`
Perl	`perl`

To override, set ATTUNE_WORKER_RUNTIMES:

environment:
  ATTUNE_WORKER_RUNTIMES: python,shell  # Only advertise Python and Shell

Testing Detection

Run the agent in detect-only mode to see what it finds:

# In a running container
docker exec <container> /opt/attune/agent/attune-agent --detect-only

# Or start a throwaway container
docker run --rm -v agent_bin:/opt/attune/agent:ro ruby:3.3-slim /opt/attune/agent/attune-agent --detect-only

Examples

Ruby Worker

worker-ruby:
  image: ruby:3.3-slim
  entrypoint: ["/opt/attune/agent/attune-agent"]
  # ... (standard depends_on, volumes, env, networks)

Node.js 22 Worker

worker-node22:
  image: node:22-slim
  entrypoint: ["/opt/attune/agent/attune-agent"]
  # ...

GPU Worker (NVIDIA CUDA)

worker-gpu:
  image: nvidia/cuda:12.3.1-runtime-ubuntu22.04
  runtime: nvidia
  entrypoint: ["/opt/attune/agent/attune-agent"]
  environment:
    ATTUNE_WORKER_RUNTIMES: python,shell  # Override — CUDA image has python
  # ...

Multi-Runtime Custom Image

worker-data-science:
  image: my-org/data-science:latest  # Has Python, R, and Julia
  entrypoint: ["/opt/attune/agent/attune-agent"]
  # Agent auto-detects all available runtimes
  # ...

Comparison: Traditional vs Agent Workers

Aspect	Traditional Worker	Agent Worker
Docker build	Required (5+ min)	None
Dockerfile	Custom per runtime	Not needed
Base image	`debian:bookworm-slim`	Any image
Runtime install	Via apt/NodeSource	Pre-installed in image
Configuration	Manual `ATTUNE_WORKER_RUNTIMES`	Auto-detected
Binary	Compiled into image	Injected via volume
Update cycle	Rebuild image	Restart `init-agent`

Troubleshooting

Agent binary not found

exec /opt/attune/agent/attune-agent: no such file or directory

The init-agent service hasn't completed. Check:

docker compose logs init-agent

"No runtimes detected"

The container image doesn't have any recognized interpreters in $PATH. Either:

Use an image that includes your runtime (e.g., ruby:3.3-slim)
Set ATTUNE_WORKER_RUNTIMES manually

Connection refused to PostgreSQL/RabbitMQ

Ensure your depends_on conditions include postgres and rabbitmq health checks, and that the container is on the attune-network.

7.2 KiB Raw Blame History