trying to make the pipeline builds work, desperately.
This commit is contained in:
@@ -50,8 +50,8 @@ web/node_modules/
|
|||||||
web/dist/
|
web/dist/
|
||||||
web/.vite/
|
web/.vite/
|
||||||
|
|
||||||
# SQLx offline data (generated at build time)
|
# SQLx offline data (generated when using `cargo sqlx prepare`)
|
||||||
#.sqlx/
|
# .sqlx/
|
||||||
|
|
||||||
# Configuration files (copied selectively)
|
# Configuration files (copied selectively)
|
||||||
config.development.yaml
|
config.development.yaml
|
||||||
|
|||||||
@@ -9,10 +9,12 @@ on:
|
|||||||
|
|
||||||
env:
|
env:
|
||||||
CARGO_TERM_COLOR: always
|
CARGO_TERM_COLOR: always
|
||||||
RUST_MIN_STACK: 16777216
|
RUST_MIN_STACK: 67108864
|
||||||
CARGO_INCREMENTAL: 0
|
CARGO_INCREMENTAL: 0
|
||||||
CARGO_NET_RETRY: 10
|
CARGO_NET_RETRY: 10
|
||||||
RUSTUP_MAX_RETRIES: 10
|
RUSTUP_MAX_RETRIES: 10
|
||||||
|
# Gitea Actions runner tool cache. Actions like setup-node/setup-python can reuse this.
|
||||||
|
RUNNER_TOOL_CACHE: /toolcache
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
rust-fmt:
|
rust-fmt:
|
||||||
@@ -22,6 +24,17 @@ jobs:
|
|||||||
- name: Checkout
|
- name: Checkout
|
||||||
uses: actions/checkout@v4
|
uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Cache Rust toolchain
|
||||||
|
uses: actions/cache@v4
|
||||||
|
with:
|
||||||
|
path: |
|
||||||
|
~/.rustup/toolchains
|
||||||
|
~/.rustup/update-hashes
|
||||||
|
key: rustup-rustfmt-${{ runner.os }}-stable-v1
|
||||||
|
restore-keys: |
|
||||||
|
rustup-${{ runner.os }}-stable-v1
|
||||||
|
rustup-
|
||||||
|
|
||||||
- name: Setup Rust
|
- name: Setup Rust
|
||||||
uses: dtolnay/rust-toolchain@stable
|
uses: dtolnay/rust-toolchain@stable
|
||||||
with:
|
with:
|
||||||
@@ -37,6 +50,17 @@ jobs:
|
|||||||
- name: Checkout
|
- name: Checkout
|
||||||
uses: actions/checkout@v4
|
uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Cache Rust toolchain
|
||||||
|
uses: actions/cache@v4
|
||||||
|
with:
|
||||||
|
path: |
|
||||||
|
~/.rustup/toolchains
|
||||||
|
~/.rustup/update-hashes
|
||||||
|
key: rustup-clippy-${{ runner.os }}-stable-v1
|
||||||
|
restore-keys: |
|
||||||
|
rustup-${{ runner.os }}-stable-v1
|
||||||
|
rustup-
|
||||||
|
|
||||||
- name: Setup Rust
|
- name: Setup Rust
|
||||||
uses: dtolnay/rust-toolchain@stable
|
uses: dtolnay/rust-toolchain@stable
|
||||||
with:
|
with:
|
||||||
@@ -72,6 +96,17 @@ jobs:
|
|||||||
- name: Checkout
|
- name: Checkout
|
||||||
uses: actions/checkout@v4
|
uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Cache Rust toolchain
|
||||||
|
uses: actions/cache@v4
|
||||||
|
with:
|
||||||
|
path: |
|
||||||
|
~/.rustup/toolchains
|
||||||
|
~/.rustup/update-hashes
|
||||||
|
key: rustup-test-${{ runner.os }}-stable-v1
|
||||||
|
restore-keys: |
|
||||||
|
rustup-${{ runner.os }}-stable-v1
|
||||||
|
rustup-
|
||||||
|
|
||||||
- name: Setup Rust
|
- name: Setup Rust
|
||||||
uses: dtolnay/rust-toolchain@stable
|
uses: dtolnay/rust-toolchain@stable
|
||||||
|
|
||||||
@@ -105,6 +140,17 @@ jobs:
|
|||||||
- name: Checkout
|
- name: Checkout
|
||||||
uses: actions/checkout@v4
|
uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Cache Rust toolchain
|
||||||
|
uses: actions/cache@v4
|
||||||
|
with:
|
||||||
|
path: |
|
||||||
|
~/.rustup/toolchains
|
||||||
|
~/.rustup/update-hashes
|
||||||
|
key: rustup-audit-${{ runner.os }}-stable-v1
|
||||||
|
restore-keys: |
|
||||||
|
rustup-${{ runner.os }}-stable-v1
|
||||||
|
rustup-
|
||||||
|
|
||||||
- name: Setup Rust
|
- name: Setup Rust
|
||||||
uses: dtolnay/rust-toolchain@stable
|
uses: dtolnay/rust-toolchain@stable
|
||||||
|
|
||||||
|
|||||||
@@ -102,7 +102,7 @@ docker compose logs -f <svc> # View logs
|
|||||||
- **BuildKit cache mounts**: Persist cargo registry and compilation artifacts between builds
|
- **BuildKit cache mounts**: Persist cargo registry and compilation artifacts between builds
|
||||||
- **Cache strategy**: `sharing=shared` for registry/git (concurrent-safe), service-specific IDs for target caches
|
- **Cache strategy**: `sharing=shared` for registry/git (concurrent-safe), service-specific IDs for target caches
|
||||||
- **Parallel builds**: 4x faster than old `sharing=locked` strategy - no serialization overhead
|
- **Parallel builds**: 4x faster than old `sharing=locked` strategy - no serialization overhead
|
||||||
- **Rustc stack size**: All Rust Dockerfiles set `ENV RUST_MIN_STACK=16777216` (16 MiB) in the build stage to prevent `rustc` SIGSEGV crashes during release compilation. The `Makefile` also exports this variable for local builds.
|
- **Rustc stack size**: All Rust Dockerfiles set `ENV RUST_MIN_STACK=67108864` (64 MiB) in the build stage to prevent `rustc` SIGSEGV crashes during release compilation. The `Makefile` also exports this variable for local builds.
|
||||||
- **Documentation**: See `docs/docker-layer-optimization.md`, `docs/QUICKREF-docker-optimization.md`, `docs/QUICKREF-buildkit-cache-strategy.md`
|
- **Documentation**: See `docs/docker-layer-optimization.md`, `docs/QUICKREF-docker-optimization.md`, `docs/QUICKREF-buildkit-cache-strategy.md`
|
||||||
|
|
||||||
### Docker Runtime Standardization
|
### Docker Runtime Standardization
|
||||||
|
|||||||
2
Makefile
2
Makefile
@@ -69,7 +69,7 @@ help:
|
|||||||
@echo ""
|
@echo ""
|
||||||
|
|
||||||
# Increase rustc stack size to prevent SIGSEGV during compilation
|
# Increase rustc stack size to prevent SIGSEGV during compilation
|
||||||
export RUST_MIN_STACK := 16777216
|
export RUST_MIN_STACK:=67108864
|
||||||
|
|
||||||
# Building
|
# Building
|
||||||
build:
|
build:
|
||||||
|
|||||||
148
docker/Dockerfile
Normal file
148
docker/Dockerfile
Normal file
@@ -0,0 +1,148 @@
|
|||||||
|
# Multi-stage Dockerfile for Attune Rust services
|
||||||
|
# This Dockerfile can build any of the Attune services by specifying a build argument
|
||||||
|
# Usage: DOCKER_BUILDKIT=1 docker build --build-arg SERVICE=api -f docker/Dockerfile -t attune-api .
|
||||||
|
#
|
||||||
|
# BuildKit cache mounts are used to speed up incremental builds by persisting:
|
||||||
|
# - Cargo registry and git cache (with sharing=locked to prevent race conditions)
|
||||||
|
# - Rust incremental compilation artifacts
|
||||||
|
#
|
||||||
|
# This dramatically reduces rebuild times from ~5 minutes to ~30 seconds for code-only changes.
|
||||||
|
|
||||||
|
ARG RUST_VERSION=1.92
|
||||||
|
ARG DEBIAN_VERSION=bookworm
|
||||||
|
|
||||||
|
# ============================================================================
|
||||||
|
# Stage 1: Builder - Compile the Rust services
|
||||||
|
# ============================================================================
|
||||||
|
FROM rust:${RUST_VERSION}-${DEBIAN_VERSION} AS builder
|
||||||
|
|
||||||
|
# Install build dependencies
|
||||||
|
RUN apt-get update && apt-get install -y \
|
||||||
|
pkg-config \
|
||||||
|
libssl-dev \
|
||||||
|
ca-certificates \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
WORKDIR /build
|
||||||
|
|
||||||
|
# Increase rustc stack size to prevent SIGSEGV during release builds
|
||||||
|
ENV RUST_MIN_STACK=67108864
|
||||||
|
|
||||||
|
# Copy workspace manifests and source code
|
||||||
|
COPY Cargo.toml Cargo.lock ./
|
||||||
|
COPY crates/ ./crates/
|
||||||
|
COPY migrations/ ./migrations/
|
||||||
|
COPY .sqlx/ ./.sqlx/
|
||||||
|
|
||||||
|
# Build argument to specify which service to build
|
||||||
|
ARG SERVICE=api
|
||||||
|
|
||||||
|
# Build the specified service with BuildKit cache mounts
|
||||||
|
# Cache mount sharing modes prevent race conditions during parallel builds:
|
||||||
|
# - sharing=locked: Only one build can access the cache at a time (prevents file conflicts)
|
||||||
|
# - cargo registry/git: Locked to prevent "File exists" errors when extracting dependencies
|
||||||
|
# - target: Locked to prevent compilation artifact conflicts
|
||||||
|
#
|
||||||
|
# This is slower than parallel builds but eliminates race conditions.
|
||||||
|
# Alternative: Use docker-compose --build with --no-parallel flag, or build sequentially.
|
||||||
|
#
|
||||||
|
# First build: ~5-6 minutes
|
||||||
|
# Incremental builds (code changes only): ~30-60 seconds
|
||||||
|
RUN --mount=type=cache,target=/usr/local/cargo/registry,sharing=locked \
|
||||||
|
--mount=type=cache,target=/usr/local/cargo/git,sharing=locked \
|
||||||
|
--mount=type=cache,target=/build/target,sharing=locked \
|
||||||
|
cargo build --release --bin attune-${SERVICE} && \
|
||||||
|
cp /build/target/release/attune-${SERVICE} /build/attune-service-binary
|
||||||
|
|
||||||
|
# ============================================================================
|
||||||
|
# Stage 2: Pack Binaries Builder - Build native pack binaries with GLIBC 2.36
|
||||||
|
# ============================================================================
|
||||||
|
FROM rust:${RUST_VERSION}-${DEBIAN_VERSION} AS pack-builder
|
||||||
|
|
||||||
|
# Install build dependencies
|
||||||
|
RUN apt-get update && apt-get install -y \
|
||||||
|
pkg-config \
|
||||||
|
libssl-dev \
|
||||||
|
ca-certificates \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
WORKDIR /build
|
||||||
|
|
||||||
|
# Increase rustc stack size to prevent SIGSEGV during release builds
|
||||||
|
ENV RUST_MIN_STACK=67108864
|
||||||
|
|
||||||
|
# Copy workspace files
|
||||||
|
COPY Cargo.toml Cargo.lock ./
|
||||||
|
COPY crates/ ./crates/
|
||||||
|
COPY .sqlx/ ./.sqlx/
|
||||||
|
|
||||||
|
# Build pack binaries (sensors, etc.) with GLIBC 2.36 for maximum compatibility
|
||||||
|
# These binaries will work on any system with GLIBC 2.36 or newer
|
||||||
|
# IMPORTANT: Copy binaries WITHIN the cache mount, before it's unmounted
|
||||||
|
RUN --mount=type=cache,target=/usr/local/cargo/registry,sharing=locked \
|
||||||
|
--mount=type=cache,target=/usr/local/cargo/git,sharing=locked \
|
||||||
|
--mount=type=cache,target=/build/target,sharing=locked \
|
||||||
|
mkdir -p /build/pack-binaries && \
|
||||||
|
cargo build --release --bin attune-core-timer-sensor && \
|
||||||
|
cp /build/target/release/attune-core-timer-sensor /build/pack-binaries/attune-core-timer-sensor && \
|
||||||
|
ls -lh /build/pack-binaries/
|
||||||
|
|
||||||
|
# Verify binaries were copied successfully (after cache unmount)
|
||||||
|
RUN ls -lah /build/pack-binaries/ && \
|
||||||
|
test -f /build/pack-binaries/attune-core-timer-sensor && \
|
||||||
|
echo "Timer sensor binary built successfully"
|
||||||
|
|
||||||
|
# ============================================================================
|
||||||
|
# Stage 3: Runtime - Create minimal runtime image
|
||||||
|
# ============================================================================
|
||||||
|
FROM debian:${DEBIAN_VERSION}-slim AS runtime
|
||||||
|
|
||||||
|
# Install runtime dependencies
|
||||||
|
RUN apt-get update && apt-get install -y \
|
||||||
|
ca-certificates \
|
||||||
|
libssl3 \
|
||||||
|
curl \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# Create non-root user
|
||||||
|
RUN useradd -m -u 1000 attune && \
|
||||||
|
mkdir -p /opt/attune/packs /opt/attune/logs /opt/attune/config && \
|
||||||
|
chown -R attune:attune /opt/attune
|
||||||
|
|
||||||
|
WORKDIR /opt/attune
|
||||||
|
|
||||||
|
# Copy the service binary from builder
|
||||||
|
# Note: We copy from /build/attune-service-binary because the cache mount is not available in COPY
|
||||||
|
COPY --from=builder /build/attune-service-binary /usr/local/bin/attune-service
|
||||||
|
|
||||||
|
# Copy migrations for services that need them.
|
||||||
|
# Runtime config is mounted via Docker Compose.
|
||||||
|
COPY migrations/ ./migrations/
|
||||||
|
|
||||||
|
# Copy packs directory (excluding binaries that will be overwritten)
|
||||||
|
COPY packs/ ./packs/
|
||||||
|
|
||||||
|
# Overwrite pack binaries with ones built with compatible GLIBC from pack-builder stage
|
||||||
|
# Copy individual files to ensure they overwrite existing ones
|
||||||
|
COPY --from=pack-builder /build/pack-binaries/attune-core-timer-sensor ./packs/core/sensors/attune-core-timer-sensor
|
||||||
|
|
||||||
|
# Make binaries executable and set ownership
|
||||||
|
RUN chmod +x ./packs/core/sensors/attune-core-timer-sensor && \
|
||||||
|
chown -R attune:attune /opt/attune
|
||||||
|
|
||||||
|
# Switch to non-root user
|
||||||
|
USER attune
|
||||||
|
|
||||||
|
# Environment variables (can be overridden at runtime)
|
||||||
|
ENV RUST_LOG=info
|
||||||
|
ENV ATTUNE_CONFIG=/opt/attune/config/config.yaml
|
||||||
|
|
||||||
|
# Health check (will be overridden per service in docker-compose)
|
||||||
|
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
|
||||||
|
CMD curl -f http://localhost:8080/health || exit 1
|
||||||
|
|
||||||
|
# Expose default port (override per service)
|
||||||
|
EXPOSE 8080
|
||||||
|
|
||||||
|
# Run the service
|
||||||
|
CMD ["/usr/local/bin/attune-service"]
|
||||||
@@ -28,7 +28,7 @@ RUN apt-get update && apt-get install -y \
|
|||||||
WORKDIR /build
|
WORKDIR /build
|
||||||
|
|
||||||
# Increase rustc stack size to prevent SIGSEGV during release builds
|
# Increase rustc stack size to prevent SIGSEGV during release builds
|
||||||
ENV RUST_MIN_STACK=33554432
|
ENV RUST_MIN_STACK=67108864
|
||||||
|
|
||||||
# Copy dependency metadata first so `cargo fetch` layer is cached
|
# Copy dependency metadata first so `cargo fetch` layer is cached
|
||||||
# when only source code changes (Cargo.toml/Cargo.lock stay the same)
|
# when only source code changes (Cargo.toml/Cargo.lock stay the same)
|
||||||
@@ -60,8 +60,7 @@ RUN --mount=type=cache,target=/usr/local/cargo/registry,sharing=shared \
|
|||||||
--mount=type=cache,target=/usr/local/cargo/git,sharing=shared \
|
--mount=type=cache,target=/usr/local/cargo/git,sharing=shared \
|
||||||
cargo fetch
|
cargo fetch
|
||||||
|
|
||||||
# Now copy the real source code, SQLx metadata, and migrations
|
# Now copy the real source code and migrations
|
||||||
COPY .sqlx/ ./.sqlx/
|
|
||||||
COPY migrations/ ./migrations/
|
COPY migrations/ ./migrations/
|
||||||
COPY crates/ ./crates/
|
COPY crates/ ./crates/
|
||||||
|
|
||||||
|
|||||||
@@ -30,7 +30,7 @@ RUN apt-get update && apt-get install -y \
|
|||||||
WORKDIR /build
|
WORKDIR /build
|
||||||
|
|
||||||
# Increase rustc stack size to prevent SIGSEGV during release builds
|
# Increase rustc stack size to prevent SIGSEGV during release builds
|
||||||
ENV RUST_MIN_STACK=16777216
|
ENV RUST_MIN_STACK=67108864
|
||||||
|
|
||||||
# Copy workspace configuration
|
# Copy workspace configuration
|
||||||
COPY Cargo.toml Cargo.lock ./
|
COPY Cargo.toml Cargo.lock ./
|
||||||
@@ -54,9 +54,6 @@ RUN mkdir -p crates/worker/src && echo "fn main() {}" > crates/worker/src/main.r
|
|||||||
RUN mkdir -p crates/notifier/src && echo "fn main() {}" > crates/notifier/src/main.rs
|
RUN mkdir -p crates/notifier/src && echo "fn main() {}" > crates/notifier/src/main.rs
|
||||||
RUN mkdir -p crates/cli/src && echo "fn main() {}" > crates/cli/src/main.rs
|
RUN mkdir -p crates/cli/src && echo "fn main() {}" > crates/cli/src/main.rs
|
||||||
|
|
||||||
# Copy SQLx metadata for compile-time query checking
|
|
||||||
COPY .sqlx/ ./.sqlx/
|
|
||||||
|
|
||||||
# Copy only the source code needed for pack binaries
|
# Copy only the source code needed for pack binaries
|
||||||
COPY crates/common/ ./crates/common/
|
COPY crates/common/ ./crates/common/
|
||||||
COPY crates/core-timer-sensor/ ./crates/core-timer-sensor/
|
COPY crates/core-timer-sensor/ ./crates/core-timer-sensor/
|
||||||
|
|||||||
@@ -31,12 +31,11 @@ RUN apt-get update && apt-get install -y \
|
|||||||
WORKDIR /build
|
WORKDIR /build
|
||||||
|
|
||||||
# Increase rustc stack size to prevent SIGSEGV during release builds
|
# Increase rustc stack size to prevent SIGSEGV during release builds
|
||||||
ENV RUST_MIN_STACK=16777216
|
ENV RUST_MIN_STACK=67108864
|
||||||
|
|
||||||
# Copy workspace files
|
# Copy workspace files
|
||||||
COPY Cargo.toml Cargo.lock ./
|
COPY Cargo.toml Cargo.lock ./
|
||||||
COPY crates/ ./crates/
|
COPY crates/ ./crates/
|
||||||
COPY .sqlx/ ./.sqlx/
|
|
||||||
|
|
||||||
# Build all pack binaries in release mode with BuildKit cache
|
# Build all pack binaries in release mode with BuildKit cache
|
||||||
RUN --mount=type=cache,target=/usr/local/cargo/registry,sharing=locked \
|
RUN --mount=type=cache,target=/usr/local/cargo/registry,sharing=locked \
|
||||||
|
|||||||
@@ -31,7 +31,7 @@ RUN apt-get update && apt-get install -y \
|
|||||||
WORKDIR /build
|
WORKDIR /build
|
||||||
|
|
||||||
# Increase rustc stack size to prevent SIGSEGV during release builds
|
# Increase rustc stack size to prevent SIGSEGV during release builds
|
||||||
ENV RUST_MIN_STACK=33554432
|
ENV RUST_MIN_STACK=67108864
|
||||||
|
|
||||||
# Copy dependency metadata first so `cargo fetch` layer is cached
|
# Copy dependency metadata first so `cargo fetch` layer is cached
|
||||||
# when only source code changes (Cargo.toml/Cargo.lock stay the same)
|
# when only source code changes (Cargo.toml/Cargo.lock stay the same)
|
||||||
@@ -62,8 +62,7 @@ RUN --mount=type=cache,target=/usr/local/cargo/registry,sharing=shared \
|
|||||||
--mount=type=cache,target=/usr/local/cargo/git,sharing=shared \
|
--mount=type=cache,target=/usr/local/cargo/git,sharing=shared \
|
||||||
cargo fetch
|
cargo fetch
|
||||||
|
|
||||||
# Now copy the real source code, SQLx metadata, and migrations
|
# Now copy the real source code and migrations
|
||||||
COPY .sqlx/ ./.sqlx/
|
|
||||||
COPY migrations/ ./migrations/
|
COPY migrations/ ./migrations/
|
||||||
COPY crates/ ./crates/
|
COPY crates/ ./crates/
|
||||||
|
|
||||||
|
|||||||
288
docker/Dockerfile.worker
Normal file
288
docker/Dockerfile.worker
Normal file
@@ -0,0 +1,288 @@
|
|||||||
|
# Multi-stage Dockerfile for Attune workers
|
||||||
|
# Supports building different worker variants with different runtime capabilities
|
||||||
|
#
|
||||||
|
# Usage:
|
||||||
|
# docker build --target worker-base -t attune-worker:base -f docker/Dockerfile.worker .
|
||||||
|
# docker build --target worker-python -t attune-worker:python -f docker/Dockerfile.worker .
|
||||||
|
# docker build --target worker-node -t attune-worker:node -f docker/Dockerfile.worker .
|
||||||
|
# docker build --target worker-full -t attune-worker:full -f docker/Dockerfile.worker .
|
||||||
|
#
|
||||||
|
# BuildKit cache mounts are used to speed up incremental builds.
|
||||||
|
|
||||||
|
ARG RUST_VERSION=1.92
|
||||||
|
ARG DEBIAN_VERSION=bookworm
|
||||||
|
ARG NODE_VERSION=20
|
||||||
|
|
||||||
|
# ============================================================================
|
||||||
|
# Stage 1: Builder - Compile the worker binary
|
||||||
|
# ============================================================================
|
||||||
|
FROM rust:${RUST_VERSION}-${DEBIAN_VERSION} AS builder
|
||||||
|
|
||||||
|
# Install build dependencies
|
||||||
|
RUN apt-get update && apt-get install -y \
|
||||||
|
pkg-config \
|
||||||
|
libssl-dev \
|
||||||
|
ca-certificates \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
WORKDIR /build
|
||||||
|
|
||||||
|
# Increase rustc stack size to prevent SIGSEGV during release builds
|
||||||
|
ENV RUST_MIN_STACK=67108864
|
||||||
|
|
||||||
|
# Copy workspace manifests and source code
|
||||||
|
COPY Cargo.toml Cargo.lock ./
|
||||||
|
COPY crates/ ./crates/
|
||||||
|
COPY migrations/ ./migrations/
|
||||||
|
COPY .sqlx/ ./.sqlx/
|
||||||
|
|
||||||
|
# Build the worker binary with BuildKit cache mounts
|
||||||
|
# sharing=locked prevents race conditions during parallel builds
|
||||||
|
RUN --mount=type=cache,target=/usr/local/cargo/registry,sharing=locked \
|
||||||
|
--mount=type=cache,target=/usr/local/cargo/git,sharing=locked \
|
||||||
|
--mount=type=cache,target=/build/target,sharing=locked \
|
||||||
|
cargo build --release --bin attune-worker && \
|
||||||
|
cp /build/target/release/attune-worker /build/attune-worker
|
||||||
|
|
||||||
|
# Verify the binary was built
|
||||||
|
RUN ls -lh /build/attune-worker && \
|
||||||
|
file /build/attune-worker && \
|
||||||
|
/build/attune-worker --version || echo "Version check skipped"
|
||||||
|
|
||||||
|
# ============================================================================
|
||||||
|
# Stage 2a: Base Worker (Shell only)
|
||||||
|
# Runtime capabilities: shell
|
||||||
|
# Use case: Lightweight workers for shell scripts and basic automation
|
||||||
|
# ============================================================================
|
||||||
|
FROM debian:${DEBIAN_VERSION}-slim AS worker-base
|
||||||
|
|
||||||
|
# Install runtime dependencies
|
||||||
|
RUN apt-get update && apt-get install -y \
|
||||||
|
ca-certificates \
|
||||||
|
libssl3 \
|
||||||
|
curl \
|
||||||
|
bash \
|
||||||
|
procps \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# Create worker user and directories
|
||||||
|
RUN useradd -m -u 1000 attune && \
|
||||||
|
mkdir -p /opt/attune/packs /opt/attune/logs /opt/attune/config && \
|
||||||
|
chown -R attune:attune /opt/attune
|
||||||
|
|
||||||
|
WORKDIR /opt/attune
|
||||||
|
|
||||||
|
# Copy worker binary from builder
|
||||||
|
COPY --from=builder /build/attune-worker /usr/local/bin/attune-worker
|
||||||
|
|
||||||
|
# Copy packs directory
|
||||||
|
COPY packs/ ./packs/
|
||||||
|
|
||||||
|
# Set ownership
|
||||||
|
RUN chown -R attune:attune /opt/attune
|
||||||
|
|
||||||
|
# Switch to non-root user
|
||||||
|
USER attune
|
||||||
|
|
||||||
|
# Environment variables
|
||||||
|
ENV ATTUNE_WORKER_RUNTIMES="shell"
|
||||||
|
ENV ATTUNE_WORKER_TYPE="container"
|
||||||
|
ENV RUST_LOG=info
|
||||||
|
ENV ATTUNE_CONFIG=/opt/attune/config/config.yaml
|
||||||
|
|
||||||
|
# Health check
|
||||||
|
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
|
||||||
|
CMD pgrep -f attune-worker || exit 1
|
||||||
|
|
||||||
|
# Run the worker
|
||||||
|
CMD ["/usr/local/bin/attune-worker"]
|
||||||
|
|
||||||
|
# ============================================================================
|
||||||
|
# Stage 2b: Python Worker (Shell + Python)
|
||||||
|
# Runtime capabilities: shell, python
|
||||||
|
# Use case: Python actions and scripts with dependencies
|
||||||
|
#
|
||||||
|
# Uses debian-slim + apt python3 (NOT the python: Docker image) so that
|
||||||
|
# python3 lives at /usr/bin/python3 — the same path as worker-full.
|
||||||
|
# This avoids broken venv symlinks when multiple workers share the
|
||||||
|
# runtime_envs volume.
|
||||||
|
# ============================================================================
|
||||||
|
FROM debian:${DEBIAN_VERSION}-slim AS worker-python
|
||||||
|
|
||||||
|
# Install system dependencies including Python
|
||||||
|
RUN apt-get update && apt-get install -y \
|
||||||
|
ca-certificates \
|
||||||
|
libssl3 \
|
||||||
|
curl \
|
||||||
|
build-essential \
|
||||||
|
python3 \
|
||||||
|
python3-pip \
|
||||||
|
python3-venv \
|
||||||
|
procps \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# Create python symlink for convenience
|
||||||
|
RUN ln -sf /usr/bin/python3 /usr/bin/python
|
||||||
|
|
||||||
|
# Install common Python packages
|
||||||
|
# Use --break-system-packages for Debian 12+ pip-in-system-python restrictions
|
||||||
|
RUN pip3 install --no-cache-dir --break-system-packages \
|
||||||
|
requests>=2.31.0 \
|
||||||
|
pyyaml>=6.0 \
|
||||||
|
jinja2>=3.1.0 \
|
||||||
|
python-dateutil>=2.8.0
|
||||||
|
|
||||||
|
# Create worker user and directories
|
||||||
|
RUN useradd -m -u 1000 attune && \
|
||||||
|
mkdir -p /opt/attune/packs /opt/attune/logs /opt/attune/runtime_envs /opt/attune/config && \
|
||||||
|
chown -R attune:attune /opt/attune
|
||||||
|
|
||||||
|
WORKDIR /opt/attune
|
||||||
|
|
||||||
|
# Copy worker binary from builder
|
||||||
|
COPY --from=builder /build/attune-worker /usr/local/bin/attune-worker
|
||||||
|
|
||||||
|
# Copy packs directory
|
||||||
|
COPY packs/ ./packs/
|
||||||
|
|
||||||
|
# Set ownership
|
||||||
|
RUN chown -R attune:attune /opt/attune
|
||||||
|
|
||||||
|
# Switch to non-root user
|
||||||
|
USER attune
|
||||||
|
|
||||||
|
# Environment variables
|
||||||
|
ENV ATTUNE_WORKER_RUNTIMES="shell,python"
|
||||||
|
ENV ATTUNE_WORKER_TYPE="container"
|
||||||
|
ENV RUST_LOG=info
|
||||||
|
ENV ATTUNE_CONFIG=/opt/attune/config/config.yaml
|
||||||
|
|
||||||
|
# Health check
|
||||||
|
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
|
||||||
|
CMD pgrep -f attune-worker || exit 1
|
||||||
|
|
||||||
|
# Run the worker
|
||||||
|
CMD ["/usr/local/bin/attune-worker"]
|
||||||
|
|
||||||
|
# ============================================================================
|
||||||
|
# Stage 2c: Node Worker (Shell + Node.js)
|
||||||
|
# Runtime capabilities: shell, node
|
||||||
|
# Use case: JavaScript/TypeScript actions and npm packages
|
||||||
|
#
|
||||||
|
# Uses debian-slim + NodeSource apt repo (NOT the node: Docker image) so that
|
||||||
|
# node lives at /usr/bin/node — the same path as worker-full.
|
||||||
|
# This avoids path mismatches when multiple workers share volumes.
|
||||||
|
# ============================================================================
|
||||||
|
FROM debian:${DEBIAN_VERSION}-slim AS worker-node
|
||||||
|
|
||||||
|
# Install system dependencies
|
||||||
|
RUN apt-get update && apt-get install -y \
|
||||||
|
ca-certificates \
|
||||||
|
libssl3 \
|
||||||
|
curl \
|
||||||
|
procps \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# Install Node.js from NodeSource (same method as worker-full)
|
||||||
|
RUN curl -fsSL https://deb.nodesource.com/setup_${NODE_VERSION}.x | bash - && \
|
||||||
|
apt-get install -y nodejs && \
|
||||||
|
rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# Create worker user and directories
|
||||||
|
RUN useradd -m -u 1000 attune && \
|
||||||
|
mkdir -p /opt/attune/packs /opt/attune/logs /opt/attune/runtime_envs /opt/attune/config && \
|
||||||
|
chown -R attune:attune /opt/attune
|
||||||
|
|
||||||
|
WORKDIR /opt/attune
|
||||||
|
|
||||||
|
# Copy worker binary from builder
|
||||||
|
COPY --from=builder /build/attune-worker /usr/local/bin/attune-worker
|
||||||
|
|
||||||
|
# Copy packs directory
|
||||||
|
COPY packs/ ./packs/
|
||||||
|
|
||||||
|
# Set ownership
|
||||||
|
RUN chown -R attune:attune /opt/attune
|
||||||
|
|
||||||
|
# Switch to non-root user
|
||||||
|
USER attune
|
||||||
|
|
||||||
|
# Environment variables
|
||||||
|
ENV ATTUNE_WORKER_RUNTIMES="shell,node"
|
||||||
|
ENV ATTUNE_WORKER_TYPE="container"
|
||||||
|
ENV RUST_LOG=info
|
||||||
|
ENV ATTUNE_CONFIG=/opt/attune/config/config.yaml
|
||||||
|
|
||||||
|
# Health check
|
||||||
|
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
|
||||||
|
CMD pgrep -f attune-worker || exit 1
|
||||||
|
|
||||||
|
# Run the worker
|
||||||
|
CMD ["/usr/local/bin/attune-worker"]
|
||||||
|
|
||||||
|
# ============================================================================
|
||||||
|
# Stage 2d: Full Worker (All runtimes)
|
||||||
|
# Runtime capabilities: shell, python, node, native
|
||||||
|
# Use case: General-purpose automation with multi-language support
|
||||||
|
# ============================================================================
|
||||||
|
FROM debian:${DEBIAN_VERSION} AS worker-full
|
||||||
|
|
||||||
|
# Install system dependencies including Python and Node.js
|
||||||
|
RUN apt-get update && apt-get install -y \
|
||||||
|
ca-certificates \
|
||||||
|
libssl3 \
|
||||||
|
curl \
|
||||||
|
build-essential \
|
||||||
|
python3 \
|
||||||
|
python3-pip \
|
||||||
|
python3-venv \
|
||||||
|
procps \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# Install Node.js from NodeSource (same method and version as worker-node)
|
||||||
|
RUN curl -fsSL https://deb.nodesource.com/setup_${NODE_VERSION}.x | bash - && \
|
||||||
|
apt-get install -y nodejs && \
|
||||||
|
rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# Create python symlink for convenience
|
||||||
|
RUN ln -sf /usr/bin/python3 /usr/bin/python
|
||||||
|
|
||||||
|
# Install common Python packages
|
||||||
|
# Use --break-system-packages for Debian 12+ pip-in-system-python restrictions
|
||||||
|
RUN pip3 install --no-cache-dir --break-system-packages \
|
||||||
|
requests>=2.31.0 \
|
||||||
|
pyyaml>=6.0 \
|
||||||
|
jinja2>=3.1.0 \
|
||||||
|
python-dateutil>=2.8.0
|
||||||
|
|
||||||
|
# Create worker user and directories
|
||||||
|
RUN useradd -m -u 1000 attune && \
|
||||||
|
mkdir -p /opt/attune/packs /opt/attune/logs /opt/attune/runtime_envs /opt/attune/config && \
|
||||||
|
chown -R attune:attune /opt/attune
|
||||||
|
|
||||||
|
WORKDIR /opt/attune
|
||||||
|
|
||||||
|
# Copy worker binary from builder
|
||||||
|
COPY --from=builder /build/attune-worker /usr/local/bin/attune-worker
|
||||||
|
|
||||||
|
# Copy packs directory
|
||||||
|
COPY packs/ ./packs/
|
||||||
|
|
||||||
|
# Set ownership
|
||||||
|
RUN chown -R attune:attune /opt/attune
|
||||||
|
|
||||||
|
# Switch to non-root user
|
||||||
|
USER attune
|
||||||
|
|
||||||
|
# Environment variables
|
||||||
|
ENV ATTUNE_WORKER_RUNTIMES="shell,python,node,native"
|
||||||
|
ENV ATTUNE_WORKER_TYPE="container"
|
||||||
|
ENV RUST_LOG=info
|
||||||
|
ENV ATTUNE_CONFIG=/opt/attune/config/config.yaml
|
||||||
|
|
||||||
|
# Health check
|
||||||
|
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
|
||||||
|
CMD pgrep -f attune-worker || exit 1
|
||||||
|
|
||||||
|
# Run the worker
|
||||||
|
CMD ["/usr/local/bin/attune-worker"]
|
||||||
@@ -36,7 +36,7 @@ RUN apt-get update && apt-get install -y \
|
|||||||
WORKDIR /build
|
WORKDIR /build
|
||||||
|
|
||||||
# Increase rustc stack size to prevent SIGSEGV during release builds
|
# Increase rustc stack size to prevent SIGSEGV during release builds
|
||||||
ENV RUST_MIN_STACK=33554432
|
ENV RUST_MIN_STACK=67108864
|
||||||
|
|
||||||
# Copy dependency metadata first so `cargo fetch` layer is cached
|
# Copy dependency metadata first so `cargo fetch` layer is cached
|
||||||
# when only source code changes (Cargo.toml/Cargo.lock stay the same)
|
# when only source code changes (Cargo.toml/Cargo.lock stay the same)
|
||||||
@@ -67,8 +67,7 @@ RUN --mount=type=cache,target=/usr/local/cargo/registry,sharing=shared \
|
|||||||
--mount=type=cache,target=/usr/local/cargo/git,sharing=shared \
|
--mount=type=cache,target=/usr/local/cargo/git,sharing=shared \
|
||||||
cargo fetch
|
cargo fetch
|
||||||
|
|
||||||
# Now copy the real source code, SQLx metadata, and migrations
|
# Now copy the real source code and migrations
|
||||||
COPY .sqlx/ ./.sqlx/
|
|
||||||
COPY migrations/ ./migrations/
|
COPY migrations/ ./migrations/
|
||||||
COPY crates/ ./crates/
|
COPY crates/ ./crates/
|
||||||
|
|
||||||
|
|||||||
@@ -425,10 +425,6 @@ docker builder prune -a
|
|||||||
docker compose build --no-cache
|
docker compose build --no-cache
|
||||||
```
|
```
|
||||||
|
|
||||||
**SQLx compile-time verification fails:**
|
|
||||||
- Ensure `.sqlx/` directory is present
|
|
||||||
- Regenerate with `cargo sqlx prepare` if needed
|
|
||||||
|
|
||||||
### Runtime Issues
|
### Runtime Issues
|
||||||
|
|
||||||
**Service won't start:**
|
**Service won't start:**
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ Docker Compose builds were failing with `rustc interrupted by SIGSEGV` during re
|
|||||||
|
|
||||||
### Fix
|
### Fix
|
||||||
|
|
||||||
Added `ENV RUST_MIN_STACK=16777216` to the build stage of all 7 Rust Dockerfiles:
|
Added `ENV RUST_MIN_STACK=67108864` to the build stage of all 7 Rust Dockerfiles:
|
||||||
|
|
||||||
- `docker/Dockerfile` (both build stages)
|
- `docker/Dockerfile` (both build stages)
|
||||||
- `docker/Dockerfile.optimized`
|
- `docker/Dockerfile.optimized`
|
||||||
@@ -117,4 +117,4 @@ The visual workflow builder and API save endpoints now produce the action-linked
|
|||||||
- All 598 workspace lib tests pass
|
- All 598 workspace lib tests pass
|
||||||
- Zero TypeScript errors
|
- Zero TypeScript errors
|
||||||
- Zero compiler warnings
|
- Zero compiler warnings
|
||||||
- Zero build errors
|
- Zero build errors
|
||||||
|
|||||||
Reference in New Issue
Block a user