attune/docker/README.md

Attune Docker Configuration

This directory contains Docker-related files for building and running Attune services.

⚠️ Important: When building multiple services in parallel, you may encounter race conditions. See DOCKER_BUILD_RACE_CONDITIONS.md for solutions and recommended workflows.

Quick Start

Default User Credentials

When you start Attune with Docker Compose, a default test user is automatically created:

• Login: test@attune.local
• Password: TestPass123!

This happens via the init-user service which runs after database migrations complete.

Test Login

curl -X POST http://localhost:8080/auth/login \
  -H 'Content-Type: application/json' \
  -d '{"login":"test@attune.local","password":"TestPass123!"}'

⚠️ Security Note: This default user is for development/testing only. Never use these credentials in production!

Files

Dockerfiles

• Dockerfile - Multi-stage Dockerfile for all Rust services (API, Executor, Worker, Sensor, Notifier)

  • Uses build argument SERVICE to specify which service to build
  • Example: docker build --build-arg SERVICE=api -f docker/Dockerfile -t attune-api .

• Dockerfile.worker - Multi-stage Dockerfile for containerized workers with different runtime capabilities

  • Supports 4 variants: worker-base, worker-python, worker-node, worker-full
  • See README.worker.md for details

• Dockerfile.web - Multi-stage Dockerfile for React Web UI

  • Builds with Node.js and serves with Nginx
  • Includes runtime environment variable injection

Configuration Files

• nginx.conf - Nginx configuration for serving Web UI and proxying API/WebSocket requests

  • Serves static React assets
  • Proxies /api/* to API service
  • Proxies /ws/* to Notifier service (WebSocket)
  • Includes security headers and compression

• inject-env.sh - Script to inject runtime environment variables into Web UI

  • Runs at container startup
  • Creates runtime-config.js with API and WebSocket URLs

Initialization Scripts

• init-db.sh - Database initialization script (optional)

  • Waits for PostgreSQL readiness
  • Creates schema and runs migrations
  • Can be used for manual DB setup

• init-user.sh - Default user initialization script

  • Automatically creates test user on first startup
  • Idempotent - safe to run multiple times
  • Creates user: test@attune.local / TestPass123!
  • Uses pre-computed Argon2id password hash
  • Skips creation if user already exists

• run-migrations.sh - Database migration runner

  • Runs SQLx migrations automatically on startup
  • Used by the migrations service in docker-compose

Docker Compose

The main docker compose.yaml is in the project root. It orchestrates:

• Infrastructure: PostgreSQL, RabbitMQ, Redis
• Services: API, Executor, Worker, Sensor, Notifier
• Web UI: React frontend with Nginx

Building Images

Build All Services (Recommended Method)

To avoid race conditions during parallel builds, pre-warm the cache first:

cd /path/to/attune

# Enable BuildKit for faster incremental builds (recommended)
export DOCKER_BUILDKIT=1
export COMPOSE_DOCKER_CLI_BUILD=1

# Step 1: Pre-warm the build cache (builds API service only)
make docker-cache-warm

# Step 2: Build all services (faster and more reliable)
make docker-build

Or build directly with docker compose:

docker compose build

Note: The Dockerfile uses sharing=locked on cache mounts to prevent race conditions. This makes parallel builds sequential but ensures 100% reliability. See DOCKER_BUILD_RACE_CONDITIONS.md for details.

Build Individual Service

# Enable BuildKit first
export DOCKER_BUILDKIT=1

# API service
docker compose build api

# Web UI
docker compose build web

# Worker service
docker compose build worker

Build with Custom Args

# Build API with specific Rust version
DOCKER_BUILDKIT=1 docker build \
  --build-arg SERVICE=api \
  --build-arg RUST_VERSION=1.92 \
  -f docker/Dockerfile \
  -t attune-api:custom \
  .

Enable BuildKit Globally

BuildKit dramatically speeds up incremental builds by caching compilation artifacts.

# Run the configuration script
./docker/enable-buildkit.sh

# Or manually add to your shell profile (~/.bashrc, ~/.zshrc, etc.)
export DOCKER_BUILDKIT=1
export COMPOSE_DOCKER_CLI_BUILD=1

# Apply changes
source ~/.bashrc  # or ~/.zshrc

Image Structure

Rust Services

Builder Stage:

• Base: rust:1.92-bookworm
• Installs build dependencies
• Compiles the specified service in release mode
• Uses BuildKit cache mounts for incremental builds
• Build time:
  • First build: ~5-6 minutes
  • Incremental builds (with BuildKit): ~30-60 seconds
  • Without BuildKit: ~5-6 minutes every time

Runtime Stage:

• Base: debian:bookworm-slim
• Minimal runtime dependencies (ca-certificates, libssl3, curl)
• Runs as non-root user attune (UID 1000)
• Binary copied to /usr/local/bin/attune-service
• Configuration in /opt/attune/
• Packs directory: /opt/attune/packs

Web UI

Builder Stage:

• Base: node:20-alpine
• Installs npm dependencies
• Builds React application with Vite

Runtime Stage:

• Base: nginx:1.25-alpine
• Custom Nginx configuration
• Static files in /usr/share/nginx/html
• Environment injection script at startup

Environment Variables

Rust services support environment-based configuration with ATTUNE__ prefix:

# Database
ATTUNE__DATABASE__URL=postgresql://user:pass@host:5432/db

# Message Queue
ATTUNE__MESSAGE_QUEUE__URL=amqp://user:pass@host:5672

# Security
JWT_SECRET=your-secret-here
ENCRYPTION_KEY=your-32-char-key-here

# Logging
RUST_LOG=debug

Web UI environment variables:

API_URL=http://localhost:8080
WS_URL=ws://localhost:8081
ENVIRONMENT=production

Volumes

The following volumes are used:

Data Volumes:

• postgres_data - PostgreSQL database files
• rabbitmq_data - RabbitMQ data
• redis_data - Redis persistence

Log Volumes:

• api_logs, executor_logs, worker_logs, sensor_logs, notifier_logs

Temporary:

• worker_temp - Worker service temporary files

Bind Mounts:

• ./packs:/opt/attune/packs:ro - Read-only pack files

Networking

All services run on the attune-network bridge network with subnet 172.28.0.0/16.

Service Communication:

• Services communicate using container names as hostnames
• Example: API connects to postgres:5432, rabbitmq:5672

External Access:

• API: localhost:8080
• Notifier WebSocket: localhost:8081
• Web UI: localhost:3000
• RabbitMQ Management: localhost:15672
• PostgreSQL: localhost:5432

Health Checks

All services have health checks configured:

API:

curl -f http://localhost:8080/health

Web UI:

wget --spider http://localhost:80/health

PostgreSQL:

pg_isready -U attune

RabbitMQ:

rabbitmq-diagnostics -q ping

Background Services: Process existence check with pgrep

Security

Non-Root User

All services run as non-root user attune (UID 1000) for security.

Secrets Management

Development:

• Secrets in .env file (not committed to git)
• Default values for testing only

Production:

• Use Docker secrets or external secrets manager
• Never hardcode secrets in images
• Rotate secrets regularly

Network Security

• Services isolated on private network
• Only necessary ports exposed to host
• Use TLS/SSL for external connections
• Dedicated bridge network
• Proper startup dependencies

Optimization

BuildKit Cache Mounts (Recommended)

Enable BuildKit for dramatically faster incremental builds:

export DOCKER_BUILDKIT=1
docker compose build

How it works:

• Persists /usr/local/cargo/registry (downloaded crates, ~1-2GB)
• Persists /usr/local/cargo/git (git dependencies)
• Persists /build/target (compilation artifacts, ~5-10GB)

Performance improvement:

• First build: ~5-6 minutes
• Code-only changes: ~30-60 seconds (vs 5+ minutes without caching)
• Dependency changes: ~2-3 minutes (vs full rebuild)

Manage cache:

# View cache size
docker system df

# Clear build cache
docker builder prune

# Clear specific cache
docker builder prune --filter type=exec.cachemount

Layer Caching

The Dockerfiles are also optimized for Docker layer caching:

1. Copy manifests first
2. Download and compile dependencies
3. Copy actual source code last
4. Source code changes don't invalidate dependency layers

Image Size

Rust Services:

• Multi-stage build reduces size
• Only runtime dependencies in final image
• Typical size: 140-180MB per service

Web UI:

• Static files only in final image
• Alpine-based Nginx
• Typical size: 50-80MB

Build Time

With BuildKit (recommended):

• First build: ~5-6 minutes
• Code-only changes: ~30-60 seconds
• Dependency changes: ~2-3 minutes

Without BuildKit:

• Every build: ~5-6 minutes

Enable BuildKit:

./docker/enable-buildkit.sh
# or
export DOCKER_BUILDKIT=1

Troubleshooting

Build Failures

Slow builds / No caching: If builds always take 5+ minutes even for small code changes, BuildKit may not be enabled.

Solution:

# Check if BuildKit is enabled
echo $DOCKER_BUILDKIT

# Enable BuildKit
export DOCKER_BUILDKIT=1
export COMPOSE_DOCKER_CLI_BUILD=1

# Add to shell profile for persistence
echo 'export DOCKER_BUILDKIT=1' >> ~/.bashrc
echo 'export COMPOSE_DOCKER_CLI_BUILD=1' >> ~/.bashrc

# Or use the helper script
./docker/enable-buildkit.sh

# Rebuild
docker compose build

Cargo.lock version error:

error: failed to parse lock file at: /build/Cargo.lock
Caused by:
  lock file version `4` was found, but this version of Cargo does not understand this lock file

Solution: Update Rust version in Dockerfile

# Edit docker/Dockerfile and change:
ARG RUST_VERSION=1.75
# to:
ARG RUST_VERSION=1.92

Cargo.lock version 4 requires Rust 1.82+. The project uses Rust 1.92.

Cargo dependencies fail:

# Clear Docker build cache
docker builder prune -a

# Rebuild without cache
docker compose build --no-cache

SQLx compile-time verification fails:

• Ensure .sqlx/ directory is present
• Regenerate with cargo sqlx prepare if needed

Runtime Issues

Service won't start:

# Check logs
docker compose logs <service-name>

# Check health
docker compose ps

Database connection fails:

# Verify PostgreSQL is ready
docker compose exec postgres pg_isready -U attune

# Check connection from service
docker compose exec api /bin/sh
# Then: curl postgres:5432

Permission errors:

# Fix volume permissions
sudo chown -R 1000:1000 ./packs ./logs

Development Workflow

Local Development with Docker

# Start infrastructure only
docker compose up -d postgres rabbitmq redis

# Run services locally
cargo run --bin attune-api
cargo run --bin attune-worker

# Or start everything
docker compose up -d

Rebuilding After Code Changes

# Rebuild and restart specific service
docker compose build api
docker compose up -d api

# Rebuild all
docker compose build
docker compose up -d

Debugging

# Access service shell
docker compose exec api /bin/sh

# View logs in real-time
docker compose logs -f api worker

# Check resource usage
docker stats

Production Deployment

See Docker Deployment Guide for:

• Production configuration
• Security hardening
• Scaling strategies
• Monitoring setup
• Backup procedures
• High availability

CI/CD Integration

Example GitHub Actions workflow:

- name: Build Docker images
  run: docker compose build

- name: Run tests
  run: docker compose run --rm api cargo test

- name: Push to registry
  run: |
    docker tag attune-api:latest registry.example.com/attune-api:${{ github.sha }}
    docker push registry.example.com/attune-api:${{ github.sha }}

Maintenance

Updating Images

# Pull latest base images
docker compose pull

# Rebuild services
docker compose build --pull

# Restart with new images
docker compose up -d

Cleaning Up

# Remove stopped containers
docker compose down

# Remove volumes (WARNING: deletes data)
docker compose down -v

# Clean up unused images
docker image prune -a

# Full cleanup
docker system prune -a --volumes

References

• Docker Compose Documentation
• Multi-stage Builds
• Dockerfile Best Practices
• Main Documentation