From 4d5a3b1bf55f7b08373863bef7ef77467846328c Mon Sep 17 00:00:00 2001 From: David Culbreth Date: Sat, 21 Mar 2026 08:27:20 -0500 Subject: [PATCH] agent-style workers --- AGENTS.md | 6 +- crates/api/src/routes/agent.rs | 160 ++++++++++++++++++ crates/api/tests/agent_tests.rs | 138 +++++++++++++++ crates/common/src/pack_environment.rs | 25 ++- crates/common/src/repositories/runtime.rs | 150 ++++++---------- crates/common/src/runtime_detection.rs | 17 +- crates/worker/src/agent_main.rs | 10 +- crates/worker/src/executor.rs | 19 +-- crates/worker/src/registration.rs | 68 ++++++++ migrations/20250101000002_pack_system.sql | 15 ++ ...20250101000012_agent_runtime_detection.sql | 22 --- 11 files changed, 469 insertions(+), 161 deletions(-) create mode 100644 crates/api/tests/agent_tests.rs delete mode 100644 migrations/20250101000012_agent_runtime_detection.sql diff --git a/AGENTS.md b/AGENTS.md index 0d40da9..249c7a4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -250,7 +250,7 @@ Completion listener advances workflow → Schedules successor tasks → Complete - **History Large-Field Guardrails**: The `execution` history trigger stores a compact **digest summary** instead of the full value for the `result` column (which can be arbitrarily large). The digest is produced by the `_jsonb_digest_summary(JSONB)` helper function and has the shape `{"digest": "md5:", "size": , "type": ""}`. This preserves change-detection semantics while avoiding history table bloat. The full result is always available on the live `execution` row. When adding new large JSONB columns to history triggers, use `_jsonb_digest_summary()` instead of storing the raw value. - **Nullable FK Fields**: `rule.action` and `rule.trigger` are nullable (`Option` in Rust) — a rule with NULL action/trigger is non-functional but preserved for traceability. `execution.action`, `execution.parent`, `execution.enforcement`, `execution.started_at`, and `event.source` are also nullable. `enforcement.event` is nullable but has no FK constraint (event is a hypertable). `execution.enforcement` is nullable but has no FK constraint (enforcement is a hypertable). All FK columns on the execution table (`action`, `parent`, `original_execution`, `enforcement`, `executor`, `workflow_def`) have no FK constraints (execution is a hypertable). `inquiry.execution` and `workflow_execution.execution` also have no FK constraints. `enforcement.resolved_at` is nullable — `None` while status is `created`, set when resolved. `execution.started_at` is nullable — `None` until the worker sets status to `running`. **Table Count**: 21 tables total in the schema (including `runtime_version`, `artifact_version`, 2 `*_history` hypertables, and the `event`, `enforcement`, + `execution` hypertables) -**Migration Count**: 12 migrations (`000001` through `000012`) — see `migrations/` directory +**Migration Count**: 11 migrations (`000001` through `000011`) — see `migrations/` directory - **Artifact System**: The `artifact` table stores metadata + structured data (progress entries via JSONB `data` column). The `artifact_version` table stores immutable content snapshots — either on disk (via `file_path` column) or in DB (via `content` BYTEA / `content_json` JSONB). Version numbering is auto-assigned via `next_artifact_version()` SQL function. A DB trigger (`enforce_artifact_retention`) auto-deletes oldest versions when count exceeds the artifact's `retention_limit`. `artifact.execution` is a plain BIGINT (no FK — execution is a hypertable). Progress-type artifacts use `artifact.data` (atomic JSON array append); file-type artifacts use `artifact_version` rows with `file_path` set. Binary content is excluded from default queries for performance (`SELECT_COLUMNS` vs `SELECT_COLUMNS_WITH_CONTENT`). **Visibility**: Each artifact has a `visibility` column (`artifact_visibility_enum`: `public` or `private`, DB default `private`). The `CreateArtifactRequest` DTO accepts `visibility` as `Option` — when omitted the API route handler applies a **type-aware default**: `public` for Progress artifacts (informational status indicators), `private` for all other types. Callers can always override explicitly. Public artifacts are viewable by all authenticated users; private artifacts are restricted based on the artifact's `scope` (Identity, Pack, Action, Sensor) and `owner` fields. The visibility field is filterable via the search/list API (`?visibility=public`). Full RBAC enforcement is deferred — the column and basic query filtering are in place for future permission checks. **Notifications**: `artifact_created` and `artifact_updated` DB triggers (in migration `000008`) fire PostgreSQL NOTIFY with entity_type `artifact` and include `visibility` in the payload. The `artifact_updated` trigger extracts a progress summary (`progress_percent`, `progress_message`, `progress_entries`) from the last entry of the `data` JSONB array for progress-type artifacts. The Web UI `ExecutionProgressBar` component (`web/src/components/executions/ExecutionProgressBar.tsx`) renders an inline progress bar in the Execution Details card using the `useArtifactStream` hook (`web/src/hooks/useArtifactStream.ts`) for real-time WebSocket updates, with polling fallback via `useExecutionArtifacts`. - **File-Based Artifact Storage**: File-type artifacts (FileBinary, FileDataTable, FileImage, FileText) use a shared filesystem volume instead of PostgreSQL BYTEA. The `artifact_version.file_path` column stores the relative path from the `artifacts_dir` root (e.g., `mypack/build_log/v1.txt`). Pattern: `{ref_with_dots_as_dirs}/v{version}.{ext}`. The artifact ref (globally unique) is used as the directory key — no execution ID in the path, so artifacts can outlive executions and be shared across them. **Endpoint**: `POST /api/v1/artifacts/{id}/versions/file` allocates a version number and file path without any file content; the execution process writes the file to `$ATTUNE_ARTIFACTS_DIR/{file_path}`. **Download**: `GET /api/v1/artifacts/{id}/download` and version-specific downloads check `file_path` first (read from disk), fall back to DB BYTEA/JSON. **Finalization**: After execution exits, the worker stats all file-backed versions for that execution and updates `size_bytes` on both `artifact_version` and parent `artifact` rows via direct DB access. **Cleanup**: Delete endpoints remove disk files before deleting DB rows; empty parent directories are cleaned up. **Backward compatible**: Existing DB-stored artifacts (`file_path = NULL`) continue to work unchanged. - **Pack Component Loading Order**: Runtimes → Triggers → Actions (+ workflow definitions) → Sensors (dependency order). Both `PackComponentLoader` (Rust) and `load_core_pack.py` (Python) follow this order. When an action YAML contains a `workflow_file` field, the loader creates/updates the referenced `workflow_definition` record and links it to the action during the Actions phase. @@ -320,7 +320,7 @@ Completion listener advances workflow → Schedules successor tasks → Complete - **Worker Startup Sequence**: (1) Connect to DB and MQ, (2) Load runtimes from DB → create `ProcessRuntime` instances, (3) Register worker and set up MQ infrastructure, (4) **Verify runtime versions** — run verification commands from `distributions` JSONB for each `RuntimeVersion` row and update `available` flag (`crates/worker/src/version_verify.rs`), (5) **Set up runtime environments** — create per-version environments for packs, (6) Start heartbeat, execution consumer, and pack registration consumer. - **Agent Startup Sequence** (`attune-agent`): (0) **Auto-detect runtimes** — probes the container for interpreter binaries using `runtime_detect::detect_runtimes()`, sets `ATTUNE_WORKER_RUNTIMES` env var with discovered names, (0b) **Dynamic runtime registration** — calls `auto_register_detected_runtimes()` to ensure each detected runtime has a DB entry (from template or minimal), then (1–6) follows the same startup sequence as `attune-worker`. If `ATTUNE_WORKER_RUNTIMES` is already set, auto-detection is skipped (explicit override). The `--detect-only` flag runs detection, prints a report, and exits without starting the worker. - **Agent Runtime Auto-Detection** (`crates/worker/src/runtime_detect.rs`): Database-free runtime discovery for the agent. Probes 8 interpreter families in order: shell (`bash`/`sh`), python (`python3`/`python`), node (`node`/`nodejs`), ruby, go, java, r (`Rscript`), perl. Uses `which`-style PATH lookup with fallbacks for absolute paths (`/bin/bash`, `/bin/sh`) and `command -v`. Captures version strings via interpreter-specific version commands. Returns `Vec` with name, path, and optional version. The `format_as_env_value()` helper converts to comma-separated format for `ATTUNE_WORKER_RUNTIMES`. -- **Dynamic Runtime Registration** (`crates/worker/src/dynamic_runtime.rs`): When the agent detects a runtime that has no corresponding entry in the database, `auto_register_detected_runtimes()` auto-registers it before `WorkerService::new()`. Strategy: (1) look up by normalized name — if found, skip; (2) look for a template runtime in loaded packs (e.g., `core.ruby`) — if found, clone with `auto_detected = true` and the detected binary path substituted into the execution config; (3) if no template, create a minimal runtime with just the interpreter binary and file extension. Auto-registered runtimes use ref format `auto.` (e.g., `auto.ruby`). The `Runtime` model has `auto_detected: bool` and `detection_config: JsonDict` columns (migration `000012`). The `detection_config` JSONB stores `detected_path`, `detected_name`, and optional `detected_version`. +- **Dynamic Runtime Registration** (`crates/worker/src/dynamic_runtime.rs`): When the agent detects a runtime that has no corresponding entry in the database, `auto_register_detected_runtimes()` auto-registers it before `WorkerService::new()`. Strategy: (1) look up by normalized name — if found, skip; (2) look for a template runtime in loaded packs (e.g., `core.ruby`) — if found, clone with `auto_detected = true` and the detected binary path substituted into the execution config; (3) if no template, create a minimal runtime with just the interpreter binary and file extension. Auto-registered runtimes use ref format `auto.` (e.g., `auto.ruby`). The `Runtime` model has `auto_detected: bool` and `detection_config: JsonDict` columns (migration `000002`). The `detection_config` JSONB stores `detected_path`, `detected_name`, and optional `detected_version`. - **Runtime Name Normalization**: The `ATTUNE_WORKER_RUNTIMES` filter (e.g., `shell,node`) uses alias-aware matching via `normalize_runtime_name()` in `crates/common/src/runtime_detection.rs`. This ensures that filter value `"node"` matches DB runtime name `"Node.js"` (lowercased to `"node.js"`). Alias groups: `node`/`nodejs`/`node.js` → `node`, `python`/`python3` → `python`, `shell`/`bash`/`sh` → `shell`, `native`/`builtin`/`standalone` → `native`, `ruby`/`rb` → `ruby`, `go`/`golang` → `go`, `java`/`jdk`/`openjdk` → `java`, `perl`/`perl5` → `perl`, `r`/`rscript` → `r`. Used in worker service runtime loading and environment setup. - **Runtime Execution Environment Variables**: `RuntimeExecutionConfig.env_vars` (HashMap) specifies template-based environment variables injected during action execution. Example: `{"NODE_PATH": "{env_dir}/node_modules"}` ensures Node.js finds packages in the isolated environment. Template variables (`{env_dir}`, `{pack_dir}`, `{interpreter}`, `{manifest_path}`) are resolved at execution time by `ProcessRuntime::execute`. - **Native Runtime Detection**: Runtime detection is purely data-driven via `execution_config` in the runtime table. A runtime with empty `execution_config` (or empty `interpreter.binary`) is native — the entrypoint is executed directly without an interpreter. There is no special "builtin" runtime concept. @@ -719,7 +719,7 @@ When reporting, ask: "Should I fix this first or continue with [original task]?" - **Web UI**: Static files served separately or via API service ## Current Development Status -- ✅ **Complete**: Database migrations (21 tables, 12 migration files), API service (most endpoints), common library, message queue infrastructure, repository layer, JWT auth, CLI tool, Web UI (basic + workflow builder + workflow timeline DAG), Executor service (core functionality + workflow orchestration), Worker service (shell/Python execution), Runtime version data model, constraint matching, worker version selection pipeline, version verification at startup, per-version environment isolation, TimescaleDB entity history tracking (execution, worker), Event, enforcement, and execution tables as TimescaleDB hypertables (time-series with retention/compression), History API endpoints (generic + entity-specific with pagination & filtering), History UI panels on entity detail pages (execution), TimescaleDB continuous aggregates (6 hourly rollup views with auto-refresh policies), Analytics API endpoints (7 endpoints under `/api/v1/analytics/` — dashboard, execution status/throughput/failure-rate, event volume, worker status, enforcement volume), Analytics dashboard widgets (bar charts, stacked status charts, failure rate ring gauge, time range selector), Workflow execution orchestration (scheduler detects workflow actions, creates child task executions, completion listener advances workflow via transitions), Workflow template resolution (type-preserving `{{ }}` rendering in task inputs), Workflow `with_items` expansion (parallel child executions per item), Workflow `with_items` concurrency limiting (sliding-window dispatch with pending items stored in workflow variables), Workflow `publish` directive processing (variable propagation between tasks), Workflow function expressions (`result()`, `succeeded()`, `failed()`, `timed_out()`), Workflow expression engine (full arithmetic/comparison/boolean/membership operators, 30+ built-in functions, recursive-descent parser), Canonical workflow namespaces (`parameters`, `workflow`, `task`, `config`, `keystore`, `item`, `index`, `system`), Artifact content system (versioned file/JSON storage, progress-append semantics, binary upload/download, retention enforcement, execution-linked artifacts, 18 API endpoints under `/api/v1/artifacts/`, file-backed disk storage via shared volume for file-type artifacts), CLI artifact management (`attune artifact list/show/create/upload/download/delete` + `attune artifact version list/show/upload/create-json/download/delete` — full CRUD for artifacts and their versions with multipart file upload, binary download, JSON version creation, auto-detected MIME types, human-readable size formatting, and pagination), CLI `--wait` flag (WebSocket-first with polling fallback — connects to notifier on port 8081, subscribes to execution, returns immediately on terminal status; falls back to exponential-backoff REST polling if WS unavailable; polling always gets at least 10s budget regardless of how long WS path ran), Workflow Timeline DAG visualization (Prefect-style time-aligned Gantt+DAG on execution detail page, pure SVG, transition-aware edge coloring from workflow definition metadata, hover tooltips, click-to-highlight path, zoom/pan), Universal Worker Agent Phase 1 (static binary build infrastructure — `attune-agent` binary target in worker crate with musl cross-compilation, runtime auto-detection module probing 8 interpreter families, `--detect-only` diagnostic flag, `docker/Dockerfile.agent` multi-stage build, Makefile targets `build-agent`/`docker-build-agent`/`run-agent`), Universal Worker Agent Phase 2 (runtime auto-detection integration with worker registration — `DetectedRuntime` is serializable, `WorkerRegistration.set_detected_runtimes()` stores structured `detected_interpreters` capability with binary paths and versions, `WorkerRegistration.set_agent_mode()` sets `agent_mode` boolean capability, `WorkerService.with_detected_runtimes()` builder method passes agent detection results to registration during `start()`, `agent_main.rs` passes auto-detected runtimes through to worker registration), Universal Worker Agent Phase 3 (`WorkerService` dual-mode refactor — `StartupMode` enum with `Worker` and `Agent { detected_runtimes }` variants, `WorkerService.startup_mode` field replaces `detected_runtimes` option, `with_detected_runtimes()` sets `StartupMode::Agent`, `start()` conditionally skips proactive version verification and environment setup in agent mode, `ProcessRuntime::execute()` performs lazy on-demand environment creation when env dir is missing instead of just warning, `StartupMode` re-exported from `attune_worker` crate), Universal Worker Agent Phase 4 (Docker Compose integration — `init-agent` service in `docker-compose.yaml` builds statically-linked agent binary and populates `agent_bin` volume, `docker-compose.agent.yaml` override file with example agent-based workers for Ruby/Python/GPU/custom images, Makefile targets `docker-up-agent`/`docker-down-agent`, quick-reference docs at `docs/QUICKREF-agent-workers.md`), Universal Worker Agent Phase 5 (API binary download endpoint — `GET /api/v1/agent/binary` streams the statically-linked agent binary with architecture selection via `?arch=x86_64|aarch64`, falls back from arch-specific to generic binary name, optional bootstrap token auth via `X-Agent-Token` header or `token` query param configured in `agent.bootstrap_token`; `GET /api/v1/agent/info` returns available architectures and binary sizes; `AgentConfig` in common config with `binary_dir` and `bootstrap_token`; `agent_bin` volume mounted read-only in API container; `scripts/attune-agent-wrapper.sh` bootstrap script with auto-detection of architecture, retry-based download from API, and `curl`/`wget` fallback), Universal Worker Agent Phase 6 (database & runtime registry extensions — `runtime.auto_detected` BOOLEAN column to distinguish agent-created vs. pack-loaded runtimes, `runtime.detection_config` JSONB column for detection metadata (detected path, version, binary name), runtime template packs for Ruby/Go/Java/Perl/R in `packs/core/runtimes/`, dynamic runtime registration module `crates/worker/src/dynamic_runtime.rs` with `auto_register_detected_runtimes()` that runs before `WorkerService::new()` in agent mode — looks up detected runtimes by alias-aware name matching, clones from pack template if available or creates minimal entry, marks auto-registered runtimes with `auto_detected = true`, `normalize_runtime_name()` extended with 5 new alias groups for the new runtimes, `SELECT_COLUMNS` constant added to RuntimeRepository), Universal Worker Agent Phase 7 (Kubernetes support — Helm chart `agent-workers.yaml` template creates a Deployment per `agentWorkers[]` values entry using the InitContainer pattern: `agent-loader` init container copies the statically-linked binary from the `attune-agent` image into an `emptyDir` volume, worker container runs any user-specified image with the agent as entrypoint; supports `nodeSelector`, `tolerations`, `runtimeClassName` for GPU/specialized scheduling, custom env vars, resource limits, runtime auto-detect or explicit override; `images.agent` added to `values.yaml` for registry-aware image resolution; `attune-agent` image added to Gitea Actions publish workflow as `agent-init` target; quick-reference docs at `docs/QUICKREF-kubernetes-agent-workers.md`) +- ✅ **Complete**: Database migrations (21 tables, 11 migration files), API service (most endpoints), common library, message queue infrastructure, repository layer, JWT auth, CLI tool, Web UI (basic + workflow builder + workflow timeline DAG), Executor service (core functionality + workflow orchestration), Worker service (shell/Python execution), Runtime version data model, constraint matching, worker version selection pipeline, version verification at startup, per-version environment isolation, TimescaleDB entity history tracking (execution, worker), Event, enforcement, and execution tables as TimescaleDB hypertables (time-series with retention/compression), History API endpoints (generic + entity-specific with pagination & filtering), History UI panels on entity detail pages (execution), TimescaleDB continuous aggregates (6 hourly rollup views with auto-refresh policies), Analytics API endpoints (7 endpoints under `/api/v1/analytics/` — dashboard, execution status/throughput/failure-rate, event volume, worker status, enforcement volume), Analytics dashboard widgets (bar charts, stacked status charts, failure rate ring gauge, time range selector), Workflow execution orchestration (scheduler detects workflow actions, creates child task executions, completion listener advances workflow via transitions), Workflow template resolution (type-preserving `{{ }}` rendering in task inputs), Workflow `with_items` expansion (parallel child executions per item), Workflow `with_items` concurrency limiting (sliding-window dispatch with pending items stored in workflow variables), Workflow `publish` directive processing (variable propagation between tasks), Workflow function expressions (`result()`, `succeeded()`, `failed()`, `timed_out()`), Workflow expression engine (full arithmetic/comparison/boolean/membership operators, 30+ built-in functions, recursive-descent parser), Canonical workflow namespaces (`parameters`, `workflow`, `task`, `config`, `keystore`, `item`, `index`, `system`), Artifact content system (versioned file/JSON storage, progress-append semantics, binary upload/download, retention enforcement, execution-linked artifacts, 18 API endpoints under `/api/v1/artifacts/`, file-backed disk storage via shared volume for file-type artifacts), CLI artifact management (`attune artifact list/show/create/upload/download/delete` + `attune artifact version list/show/upload/create-json/download/delete` — full CRUD for artifacts and their versions with multipart file upload, binary download, JSON version creation, auto-detected MIME types, human-readable size formatting, and pagination), CLI `--wait` flag (WebSocket-first with polling fallback — connects to notifier on port 8081, subscribes to execution, returns immediately on terminal status; falls back to exponential-backoff REST polling if WS unavailable; polling always gets at least 10s budget regardless of how long WS path ran), Workflow Timeline DAG visualization (Prefect-style time-aligned Gantt+DAG on execution detail page, pure SVG, transition-aware edge coloring from workflow definition metadata, hover tooltips, click-to-highlight path, zoom/pan), Universal Worker Agent Phase 1 (static binary build infrastructure — `attune-agent` binary target in worker crate with musl cross-compilation, runtime auto-detection module probing 8 interpreter families, `--detect-only` diagnostic flag, `docker/Dockerfile.agent` multi-stage build, Makefile targets `build-agent`/`docker-build-agent`/`run-agent`), Universal Worker Agent Phase 2 (runtime auto-detection integration with worker registration — `DetectedRuntime` is serializable, `WorkerRegistration.set_detected_runtimes()` stores structured `detected_interpreters` capability with binary paths and versions, `WorkerRegistration.set_agent_mode()` sets `agent_mode` boolean capability, `WorkerService.with_detected_runtimes()` builder method passes agent detection results to registration during `start()`, `agent_main.rs` passes auto-detected runtimes through to worker registration), Universal Worker Agent Phase 3 (`WorkerService` dual-mode refactor — `StartupMode` enum with `Worker` and `Agent { detected_runtimes }` variants, `WorkerService.startup_mode` field replaces `detected_runtimes` option, `with_detected_runtimes()` sets `StartupMode::Agent`, `start()` conditionally skips proactive version verification and environment setup in agent mode, `ProcessRuntime::execute()` performs lazy on-demand environment creation when env dir is missing instead of just warning, `StartupMode` re-exported from `attune_worker` crate), Universal Worker Agent Phase 4 (Docker Compose integration — `init-agent` service in `docker-compose.yaml` builds statically-linked agent binary and populates `agent_bin` volume, `docker-compose.agent.yaml` override file with example agent-based workers for Ruby/Python/GPU/custom images, Makefile targets `docker-up-agent`/`docker-down-agent`, quick-reference docs at `docs/QUICKREF-agent-workers.md`), Universal Worker Agent Phase 5 (API binary download endpoint — `GET /api/v1/agent/binary` streams the statically-linked agent binary with architecture selection via `?arch=x86_64|aarch64`, falls back from arch-specific to generic binary name, optional bootstrap token auth via `X-Agent-Token` header or `token` query param configured in `agent.bootstrap_token`; `GET /api/v1/agent/info` returns available architectures and binary sizes; `AgentConfig` in common config with `binary_dir` and `bootstrap_token`; `agent_bin` volume mounted read-only in API container; `scripts/attune-agent-wrapper.sh` bootstrap script with auto-detection of architecture, retry-based download from API, and `curl`/`wget` fallback), Universal Worker Agent Phase 6 (database & runtime registry extensions — `runtime.auto_detected` BOOLEAN column to distinguish agent-created vs. pack-loaded runtimes, `runtime.detection_config` JSONB column for detection metadata (detected path, version, binary name), runtime template packs for Ruby/Go/Java/Perl/R in `packs/core/runtimes/`, dynamic runtime registration module `crates/worker/src/dynamic_runtime.rs` with `auto_register_detected_runtimes()` that runs before `WorkerService::new()` in agent mode — looks up detected runtimes by alias-aware name matching, clones from pack template if available or creates minimal entry, marks auto-registered runtimes with `auto_detected = true`, `normalize_runtime_name()` extended with 5 new alias groups for the new runtimes, `SELECT_COLUMNS` constant added to RuntimeRepository), Universal Worker Agent Phase 7 (Kubernetes support — Helm chart `agent-workers.yaml` template creates a Deployment per `agentWorkers[]` values entry using the InitContainer pattern: `agent-loader` init container copies the statically-linked binary from the `attune-agent` image into an `emptyDir` volume, worker container runs any user-specified image with the agent as entrypoint; supports `nodeSelector`, `tolerations`, `runtimeClassName` for GPU/specialized scheduling, custom env vars, resource limits, runtime auto-detect or explicit override; `images.agent` added to `values.yaml` for registry-aware image resolution; `attune-agent` image added to Gitea Actions publish workflow as `agent-init` target; quick-reference docs at `docs/QUICKREF-kubernetes-agent-workers.md`) - 🔄 **In Progress**: Sensor service, advanced workflow features (nested workflow context propagation), Python runtime dependency management, API/UI endpoints for runtime version management, Artifact UI (web UI for browsing/downloading artifacts), Notifier service WebSocket (functional but lacks auth — the WS connection is unauthenticated; the subscribe filter controls visibility) - 📋 **Planned**: Execution policies, monitoring, pack registry system, configurable retention periods via admin settings, export/archival to external storage diff --git a/crates/api/src/routes/agent.rs b/crates/api/src/routes/agent.rs index 83a8a5a..eb5b75d 100644 --- a/crates/api/src/routes/agent.rs +++ b/crates/api/src/routes/agent.rs @@ -302,3 +302,163 @@ pub fn routes() -> Router> { .route("/agent/binary", get(download_agent_binary)) .route("/agent/info", get(agent_info)) } + +#[cfg(test)] +mod tests { + use super::*; + use attune_common::config::AgentConfig; + use axum::http::{HeaderMap, HeaderValue}; + + // ── validate_arch tests ───────────────────────────────────────── + + #[test] + fn test_validate_arch_valid_x86_64() { + let result = validate_arch("x86_64"); + assert!(result.is_ok()); + assert_eq!(result.unwrap(), "x86_64"); + } + + #[test] + fn test_validate_arch_valid_aarch64() { + let result = validate_arch("aarch64"); + assert!(result.is_ok()); + assert_eq!(result.unwrap(), "aarch64"); + } + + #[test] + fn test_validate_arch_arm64_alias() { + // "arm64" is an alias for "aarch64" + let result = validate_arch("arm64"); + assert!(result.is_ok()); + assert_eq!(result.unwrap(), "aarch64"); + } + + #[test] + fn test_validate_arch_invalid() { + let result = validate_arch("mips"); + assert!(result.is_err()); + let (status, body) = result.unwrap_err(); + assert_eq!(status, StatusCode::BAD_REQUEST); + assert_eq!(body.0["error"], "Invalid architecture"); + } + + // ── validate_token tests ──────────────────────────────────────── + + /// Helper: build a minimal Config with the given agent config. + /// Only the `agent` field is relevant for `validate_token`. + fn test_config(agent: Option) -> attune_common::config::Config { + let manifest_dir = std::env::var("CARGO_MANIFEST_DIR").unwrap_or_else(|_| ".".to_string()); + let config_path = format!("{}/../../config.test.yaml", manifest_dir); + let mut config = attune_common::config::Config::load_from_file(&config_path) + .expect("Failed to load test config"); + config.agent = agent; + config + } + + #[test] + fn test_validate_token_no_config() { + // When no agent config is set at all, no token is required. + let config = test_config(None); + let headers = HeaderMap::new(); + let query_token = None; + + let result = validate_token(&config, &headers, &query_token); + assert!(result.is_ok()); + } + + #[test] + fn test_validate_token_no_bootstrap_token_configured() { + // Agent config exists but bootstrap_token is None → no token required. + let config = test_config(Some(AgentConfig { + binary_dir: "/tmp/test".to_string(), + bootstrap_token: None, + })); + let headers = HeaderMap::new(); + let query_token = None; + + let result = validate_token(&config, &headers, &query_token); + assert!(result.is_ok()); + } + + #[test] + fn test_validate_token_valid_from_header() { + let config = test_config(Some(AgentConfig { + binary_dir: "/tmp/test".to_string(), + bootstrap_token: Some("s3cret-bootstrap".to_string()), + })); + let mut headers = HeaderMap::new(); + headers.insert( + "x-agent-token", + HeaderValue::from_static("s3cret-bootstrap"), + ); + let query_token = None; + + let result = validate_token(&config, &headers, &query_token); + assert!(result.is_ok()); + } + + #[test] + fn test_validate_token_valid_from_query() { + let config = test_config(Some(AgentConfig { + binary_dir: "/tmp/test".to_string(), + bootstrap_token: Some("s3cret-bootstrap".to_string()), + })); + let headers = HeaderMap::new(); + let query_token = Some("s3cret-bootstrap".to_string()); + + let result = validate_token(&config, &headers, &query_token); + assert!(result.is_ok()); + } + + #[test] + fn test_validate_token_invalid() { + let config = test_config(Some(AgentConfig { + binary_dir: "/tmp/test".to_string(), + bootstrap_token: Some("correct-token".to_string()), + })); + let mut headers = HeaderMap::new(); + headers.insert("x-agent-token", HeaderValue::from_static("wrong-token")); + let query_token = None; + + let result = validate_token(&config, &headers, &query_token); + assert!(result.is_err()); + let (status, body) = result.unwrap_err(); + assert_eq!(status, StatusCode::UNAUTHORIZED); + assert_eq!(body.0["error"], "Invalid token"); + } + + #[test] + fn test_validate_token_missing_when_required() { + // bootstrap_token is configured but caller provides nothing. + let config = test_config(Some(AgentConfig { + binary_dir: "/tmp/test".to_string(), + bootstrap_token: Some("required-token".to_string()), + })); + let headers = HeaderMap::new(); + let query_token = None; + + let result = validate_token(&config, &headers, &query_token); + assert!(result.is_err()); + let (status, body) = result.unwrap_err(); + assert_eq!(status, StatusCode::UNAUTHORIZED); + assert_eq!(body.0["error"], "Token required"); + } + + #[test] + fn test_validate_token_header_takes_precedence_over_query() { + // When both header and query provide a token, the header value is + // checked first (it appears first in the or_else chain). Provide a + // valid token in the header and an invalid one in the query — should + // succeed because the header matches. + let config = test_config(Some(AgentConfig { + binary_dir: "/tmp/test".to_string(), + bootstrap_token: Some("the-real-token".to_string()), + })); + let mut headers = HeaderMap::new(); + headers.insert("x-agent-token", HeaderValue::from_static("the-real-token")); + let query_token = Some("wrong-token".to_string()); + + let result = validate_token(&config, &headers, &query_token); + assert!(result.is_ok()); + } +} diff --git a/crates/api/tests/agent_tests.rs b/crates/api/tests/agent_tests.rs new file mode 100644 index 0000000..1d74d41 --- /dev/null +++ b/crates/api/tests/agent_tests.rs @@ -0,0 +1,138 @@ +//! Integration tests for agent binary distribution endpoints +//! +//! The agent endpoints (`/api/v1/agent/binary` and `/api/v1/agent/info`) are +//! intentionally unauthenticated — the agent needs to download its binary +//! before it has JWT credentials. An optional `bootstrap_token` can restrict +//! access, but that is validated inside the handler, not via RequireAuth +//! middleware. +//! +//! The test configuration (`config.test.yaml`) does NOT include an `agent` +//! section, so both endpoints return 503 Service Unavailable. This is the +//! correct behaviour: the endpoints are reachable (no 401/404 from middleware) +//! but the feature is not configured. + +use axum::http::StatusCode; + +#[allow(dead_code)] +mod helpers; +use helpers::TestContext; + +// ── /api/v1/agent/info ────────────────────────────────────────────── + +#[tokio::test] +#[ignore = "integration test — requires database"] +async fn test_agent_info_not_configured() { + let ctx = TestContext::new() + .await + .expect("Failed to create test context"); + + let response = ctx + .get("/api/v1/agent/info", None) + .await + .expect("Failed to make request"); + + // Agent config is not set in config.test.yaml, so the handler returns 503. + assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE); + + let body: serde_json::Value = response.json().await.expect("Failed to parse JSON"); + assert_eq!(body["error"], "Not configured"); +} + +#[tokio::test] +#[ignore = "integration test — requires database"] +async fn test_agent_info_no_auth_required() { + // Verify that the endpoint is reachable WITHOUT any JWT token. + // If RequireAuth middleware were applied, this would return 401. + // Instead we expect 503 (not configured) — proving the endpoint + // is publicly accessible. + let ctx = TestContext::new() + .await + .expect("Failed to create test context"); + + let response = ctx + .get("/api/v1/agent/info", None) + .await + .expect("Failed to make request"); + + // Must NOT be 401 Unauthorized — the endpoint has no auth middleware. + assert_ne!( + response.status(), + StatusCode::UNAUTHORIZED, + "agent/info should not require authentication" + ); + // Should be 503 because agent config is absent. + assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE); +} + +// ── /api/v1/agent/binary ──────────────────────────────────────────── + +#[tokio::test] +#[ignore = "integration test — requires database"] +async fn test_agent_binary_not_configured() { + let ctx = TestContext::new() + .await + .expect("Failed to create test context"); + + let response = ctx + .get("/api/v1/agent/binary", None) + .await + .expect("Failed to make request"); + + // Agent config is not set in config.test.yaml, so the handler returns 503. + assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE); + + let body: serde_json::Value = response.json().await.expect("Failed to parse JSON"); + assert_eq!(body["error"], "Not configured"); +} + +#[tokio::test] +#[ignore = "integration test — requires database"] +async fn test_agent_binary_no_auth_required() { + // Same reasoning as test_agent_info_no_auth_required: the binary + // download endpoint must be publicly accessible (no RequireAuth). + // When no bootstrap_token is configured, any caller can reach the + // handler. We still get 503 because the agent feature itself is + // not configured in the test environment. + let ctx = TestContext::new() + .await + .expect("Failed to create test context"); + + let response = ctx + .get("/api/v1/agent/binary", None) + .await + .expect("Failed to make request"); + + // Must NOT be 401 Unauthorized — the endpoint has no auth middleware. + assert_ne!( + response.status(), + StatusCode::UNAUTHORIZED, + "agent/binary should not require authentication when no bootstrap_token is configured" + ); + // Should be 503 because agent config is absent. + assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE); +} + +#[tokio::test] +#[ignore = "integration test — requires database"] +async fn test_agent_binary_invalid_arch() { + // Architecture validation (`validate_arch`) rejects unsupported values + // with 400 Bad Request. However, in the handler the execution order is: + // 1. validate_token (passes — no bootstrap_token configured) + // 2. check agent config (fails with 503 — not configured) + // 3. validate_arch (never reached) + // + // So even with an invalid arch like "mips", we get 503 from the config + // check before the arch is ever validated. The arch validation is covered + // by unit tests in routes/agent.rs instead. + let ctx = TestContext::new() + .await + .expect("Failed to create test context"); + + let response = ctx + .get("/api/v1/agent/binary?arch=mips", None) + .await + .expect("Failed to make request"); + + // 503 from the agent-config-not-set check, NOT 400 from arch validation. + assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE); +} diff --git a/crates/common/src/pack_environment.rs b/crates/common/src/pack_environment.rs index 4077e9b..90c1cdb 100644 --- a/crates/common/src/pack_environment.rs +++ b/crates/common/src/pack_environment.rs @@ -10,7 +10,7 @@ use crate::config::Config; use crate::error::{Error, Result}; use crate::models::Runtime; use crate::repositories::action::ActionRepository; -use crate::repositories::runtime::RuntimeRepository; +use crate::repositories::runtime::{self, RuntimeRepository}; use crate::repositories::FindById as _; use serde_json::Value as JsonValue; use sqlx::{PgPool, Row}; @@ -370,20 +370,15 @@ impl PackEnvironmentManager { // ======================================================================== async fn get_runtime(&self, runtime_id: i64) -> Result { - sqlx::query_as::<_, Runtime>( - r#" - SELECT id, ref, pack, pack_ref, description, name, - distributions, installation, installers, execution_config, - auto_detected, detection_config, - created, updated - FROM runtime - WHERE id = $1 - "#, - ) - .bind(runtime_id) - .fetch_one(&self.pool) - .await - .map_err(|e| Error::Internal(format!("Failed to fetch runtime: {}", e))) + let query = format!( + "SELECT {} FROM runtime WHERE id = $1", + runtime::SELECT_COLUMNS + ); + sqlx::query_as::<_, Runtime>(&query) + .bind(runtime_id) + .fetch_one(&self.pool) + .await + .map_err(|e| Error::Internal(format!("Failed to fetch runtime: {}", e))) } fn runtime_requires_environment(&self, runtime: &Runtime) -> Result { diff --git a/crates/common/src/repositories/runtime.rs b/crates/common/src/repositories/runtime.rs index 6f7de6e..350cdd3 100644 --- a/crates/common/src/repositories/runtime.rs +++ b/crates/common/src/repositories/runtime.rs @@ -63,19 +63,11 @@ impl FindById for RuntimeRepository { where E: Executor<'e, Database = Postgres> + 'e, { - let runtime = sqlx::query_as::<_, Runtime>( - r#" - SELECT id, ref, pack, pack_ref, description, name, - distributions, installation, installers, execution_config, - auto_detected, detection_config, - created, updated - FROM runtime - WHERE id = $1 - "#, - ) - .bind(id) - .fetch_optional(executor) - .await?; + let query = format!("SELECT {} FROM runtime WHERE id = $1", SELECT_COLUMNS); + let runtime = sqlx::query_as::<_, Runtime>(&query) + .bind(id) + .fetch_optional(executor) + .await?; Ok(runtime) } @@ -87,19 +79,11 @@ impl FindByRef for RuntimeRepository { where E: Executor<'e, Database = Postgres> + 'e, { - let runtime = sqlx::query_as::<_, Runtime>( - r#" - SELECT id, ref, pack, pack_ref, description, name, - distributions, installation, installers, execution_config, - auto_detected, detection_config, - created, updated - FROM runtime - WHERE ref = $1 - "#, - ) - .bind(ref_str) - .fetch_optional(executor) - .await?; + let query = format!("SELECT {} FROM runtime WHERE ref = $1", SELECT_COLUMNS); + let runtime = sqlx::query_as::<_, Runtime>(&query) + .bind(ref_str) + .fetch_optional(executor) + .await?; Ok(runtime) } @@ -111,18 +95,10 @@ impl List for RuntimeRepository { where E: Executor<'e, Database = Postgres> + 'e, { - let runtimes = sqlx::query_as::<_, Runtime>( - r#" - SELECT id, ref, pack, pack_ref, description, name, - distributions, installation, installers, execution_config, - auto_detected, detection_config, - created, updated - FROM runtime - ORDER BY ref ASC - "#, - ) - .fetch_all(executor) - .await?; + let query = format!("SELECT {} FROM runtime ORDER BY ref ASC", SELECT_COLUMNS); + let runtimes = sqlx::query_as::<_, Runtime>(&query) + .fetch_all(executor) + .await?; Ok(runtimes) } @@ -136,31 +112,28 @@ impl Create for RuntimeRepository { where E: Executor<'e, Database = Postgres> + 'e, { - let runtime = sqlx::query_as::<_, Runtime>( - r#" - INSERT INTO runtime (ref, pack, pack_ref, description, name, - distributions, installation, installers, execution_config, - auto_detected, detection_config) - VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11) - RETURNING id, ref, pack, pack_ref, description, name, - distributions, installation, installers, execution_config, - auto_detected, detection_config, - created, updated - "#, - ) - .bind(&input.r#ref) - .bind(input.pack) - .bind(&input.pack_ref) - .bind(&input.description) - .bind(&input.name) - .bind(&input.distributions) - .bind(&input.installation) - .bind(serde_json::json!({})) - .bind(&input.execution_config) - .bind(input.auto_detected) - .bind(&input.detection_config) - .fetch_one(executor) - .await?; + let query = format!( + "INSERT INTO runtime (ref, pack, pack_ref, description, name, \ + distributions, installation, installers, execution_config, \ + auto_detected, detection_config) \ + VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11) \ + RETURNING {}", + SELECT_COLUMNS + ); + let runtime = sqlx::query_as::<_, Runtime>(&query) + .bind(&input.r#ref) + .bind(input.pack) + .bind(&input.pack_ref) + .bind(&input.description) + .bind(&input.name) + .bind(&input.distributions) + .bind(&input.installation) + .bind(serde_json::json!({})) + .bind(&input.execution_config) + .bind(input.auto_detected) + .bind(&input.detection_config) + .fetch_one(executor) + .await?; Ok(runtime) } @@ -252,12 +225,7 @@ impl Update for RuntimeRepository { query.push(", updated = NOW() WHERE id = "); query.push_bind(id); - query.push( - " RETURNING id, ref, pack, pack_ref, description, name, \ - distributions, installation, installers, execution_config, \ - auto_detected, detection_config, \ - created, updated", - ); + query.push(&format!(" RETURNING {}", SELECT_COLUMNS)); let runtime = query .build_query_as::() @@ -289,20 +257,14 @@ impl RuntimeRepository { where E: Executor<'e, Database = Postgres> + 'e, { - let runtimes = sqlx::query_as::<_, Runtime>( - r#" - SELECT id, ref, pack, pack_ref, description, name, - distributions, installation, installers, execution_config, - auto_detected, detection_config, - created, updated - FROM runtime - WHERE pack = $1 - ORDER BY ref ASC - "#, - ) - .bind(pack_id) - .fetch_all(executor) - .await?; + let query = format!( + "SELECT {} FROM runtime WHERE pack = $1 ORDER BY ref ASC", + SELECT_COLUMNS + ); + let runtimes = sqlx::query_as::<_, Runtime>(&query) + .bind(pack_id) + .fetch_all(executor) + .await?; Ok(runtimes) } @@ -312,20 +274,14 @@ impl RuntimeRepository { where E: Executor<'e, Database = Postgres> + 'e, { - let runtime = sqlx::query_as::<_, Runtime>( - r#" - SELECT id, ref, pack, pack_ref, description, name, - distributions, installation, installers, execution_config, - auto_detected, detection_config, - created, updated - FROM runtime - WHERE LOWER(name) = LOWER($1) - LIMIT 1 - "#, - ) - .bind(name) - .fetch_optional(executor) - .await?; + let query = format!( + "SELECT {} FROM runtime WHERE LOWER(name) = LOWER($1) LIMIT 1", + SELECT_COLUMNS + ); + let runtime = sqlx::query_as::<_, Runtime>(&query) + .bind(name) + .fetch_optional(executor) + .await?; Ok(runtime) } diff --git a/crates/common/src/runtime_detection.rs b/crates/common/src/runtime_detection.rs index 1f05a84..2fba9f6 100644 --- a/crates/common/src/runtime_detection.rs +++ b/crates/common/src/runtime_detection.rs @@ -12,6 +12,7 @@ use crate::config::Config; use crate::error::Result; use crate::models::Runtime; +use crate::repositories::runtime::SELECT_COLUMNS; use serde_json::json; use sqlx::PgPool; use std::collections::HashMap; @@ -161,18 +162,10 @@ impl RuntimeDetector { info!("Querying database for runtime definitions..."); // Query all runtimes from database - let runtimes = sqlx::query_as::<_, Runtime>( - r#" - SELECT id, ref, pack, pack_ref, description, name, - distributions, installation, installers, execution_config, - auto_detected, detection_config, - created, updated - FROM runtime - ORDER BY ref - "#, - ) - .fetch_all(&self.pool) - .await?; + let query = format!("SELECT {} FROM runtime ORDER BY ref", SELECT_COLUMNS); + let runtimes = sqlx::query_as::<_, Runtime>(&query) + .fetch_all(&self.pool) + .await?; info!("Found {} runtime(s) in database", runtimes.len()); diff --git a/crates/worker/src/agent_main.rs b/crates/worker/src/agent_main.rs index 174ba0a..5bbad99 100644 --- a/crates/worker/src/agent_main.rs +++ b/crates/worker/src/agent_main.rs @@ -113,6 +113,10 @@ async fn main() -> Result<()> { let runtime_list: Vec<&str> = detected.iter().map(|r| r.name.as_str()).collect(); let runtime_csv = runtime_list.join(","); info!("Setting ATTUNE_WORKER_RUNTIMES={}", runtime_csv); + // SAFETY: std::env::set_var is safe in Rust 2021 edition. If upgrading + // to edition 2024+, this call will need to be wrapped in `unsafe {}`. + // It's sound here because detection runs single-threaded before tokio + // starts any worker tasks. std::env::set_var("ATTUNE_WORKER_RUNTIMES", &runtime_csv); // Stash for Phase 2: pass to WorkerService for rich capability registration @@ -133,9 +137,10 @@ async fn main() -> Result<()> { println!(); let detected = detect_runtimes(); print_detection_report(&detected); + } else if let Some(ref detected) = agent_detected_runtimes { + print_detection_report(detected); } else { - // We already ran detection above; re-run to get a fresh Vec for the report - // (the previous one was consumed by env var setup). + // No detection ran (empty results), run it fresh let detected = detect_runtimes(); print_detection_report(&detected); } @@ -144,6 +149,7 @@ async fn main() -> Result<()> { // --- Phase 2: Load configuration --- if let Some(config_path) = args.config { + // SAFETY: std::env::set_var is safe in Rust 2021 edition. See note above. std::env::set_var("ATTUNE_CONFIG", config_path); } diff --git a/crates/worker/src/executor.rs b/crates/worker/src/executor.rs index d5e68b2..6ee9888 100644 --- a/crates/worker/src/executor.rs +++ b/crates/worker/src/executor.rs @@ -19,6 +19,7 @@ use attune_common::models::runtime::RuntimeExecutionConfig; use attune_common::models::{runtime::Runtime as RuntimeModel, Action, Execution, ExecutionStatus}; use attune_common::repositories::artifact::{ArtifactRepository, ArtifactVersionRepository}; use attune_common::repositories::execution::{ExecutionRepository, UpdateExecutionInput}; +use attune_common::repositories::runtime::SELECT_COLUMNS as RUNTIME_SELECT_COLUMNS; use attune_common::repositories::runtime_version::RuntimeVersionRepository; use attune_common::repositories::{FindById, Update}; use attune_common::version_matching::select_best_version; @@ -410,16 +411,14 @@ impl ActionExecutor { // Load runtime information if specified let runtime_record = if let Some(runtime_id) = action.runtime { - match sqlx::query_as::<_, RuntimeModel>( - r#"SELECT id, ref, pack, pack_ref, description, name, - distributions, installation, installers, execution_config, - auto_detected, detection_config, - created, updated - FROM runtime WHERE id = $1"#, - ) - .bind(runtime_id) - .fetch_optional(&self.pool) - .await + let query = format!( + "SELECT {} FROM runtime WHERE id = $1", + RUNTIME_SELECT_COLUMNS + ); + match sqlx::query_as::<_, RuntimeModel>(&query) + .bind(runtime_id) + .fetch_optional(&self.pool) + .await { Ok(Some(runtime)) => { debug!( diff --git a/crates/worker/src/registration.rs b/crates/worker/src/registration.rs index fdad853..6456cbc 100644 --- a/crates/worker/src/registration.rs +++ b/crates/worker/src/registration.rs @@ -393,4 +393,72 @@ mod tests { registration.deregister().await.unwrap(); } + + #[test] + fn test_detected_runtimes_json_structure() { + // Test the JSON structure that set_detected_runtimes builds + let runtimes = vec![ + DetectedRuntime { + name: "python".to_string(), + path: "/usr/bin/python3".to_string(), + version: Some("3.12.1".to_string()), + }, + DetectedRuntime { + name: "shell".to_string(), + path: "/bin/bash".to_string(), + version: None, + }, + ]; + + let interpreters: Vec = runtimes + .iter() + .map(|rt| { + json!({ + "name": rt.name, + "path": rt.path, + "version": rt.version, + }) + }) + .collect(); + + let json_value = json!(interpreters); + + // Verify structure + let arr = json_value.as_array().unwrap(); + assert_eq!(arr.len(), 2); + assert_eq!(arr[0]["name"], "python"); + assert_eq!(arr[0]["path"], "/usr/bin/python3"); + assert_eq!(arr[0]["version"], "3.12.1"); + assert_eq!(arr[1]["name"], "shell"); + assert_eq!(arr[1]["path"], "/bin/bash"); + assert!(arr[1]["version"].is_null()); + } + + #[test] + fn test_detected_runtimes_empty() { + let runtimes: Vec = vec![]; + let interpreters: Vec = runtimes + .iter() + .map(|rt| { + json!({ + "name": rt.name, + "path": rt.path, + "version": rt.version, + }) + }) + .collect(); + + let json_value = json!(interpreters); + assert_eq!(json_value.as_array().unwrap().len(), 0); + } + + #[test] + fn test_agent_mode_capability_value() { + // Verify the JSON value for agent_mode capability + let value = json!(true); + assert_eq!(value, true); + + let value = json!(false); + assert_eq!(value, false); + } } diff --git a/migrations/20250101000002_pack_system.sql b/migrations/20250101000002_pack_system.sql index 8890e82..875d765 100644 --- a/migrations/20250101000002_pack_system.sql +++ b/migrations/20250101000002_pack_system.sql @@ -131,6 +131,17 @@ CREATE TABLE runtime ( -- {manifest_path} - absolute path to the dependency manifest file execution_config JSONB NOT NULL DEFAULT '{}'::jsonb, + -- Whether this runtime was auto-registered by an agent + -- (vs. loaded from a pack's YAML file during pack registration) + auto_detected BOOLEAN NOT NULL DEFAULT FALSE, + + -- Detection metadata for auto-discovered runtimes. + -- Stores how the agent discovered this runtime (binary path, version, etc.) + -- enables re-verification on restart. + -- Example: { "detected_path": "/usr/bin/ruby", "detected_name": "ruby", + -- "detected_version": "3.3.0" } + detection_config JSONB NOT NULL DEFAULT '{}'::jsonb, + created TIMESTAMPTZ NOT NULL DEFAULT NOW(), updated TIMESTAMPTZ NOT NULL DEFAULT NOW(), @@ -145,6 +156,8 @@ CREATE INDEX idx_runtime_created ON runtime(created DESC); CREATE INDEX idx_runtime_name ON runtime(name); CREATE INDEX idx_runtime_verification ON runtime USING GIN ((distributions->'verification')); CREATE INDEX idx_runtime_execution_config ON runtime USING GIN (execution_config); +CREATE INDEX idx_runtime_auto_detected ON runtime(auto_detected); +CREATE INDEX idx_runtime_detection_config ON runtime USING GIN (detection_config); -- Trigger CREATE TRIGGER update_runtime_updated @@ -160,6 +173,8 @@ COMMENT ON COLUMN runtime.distributions IS 'Runtime distribution metadata includ COMMENT ON COLUMN runtime.installation IS 'Installation requirements and instructions including package managers and setup steps'; COMMENT ON COLUMN runtime.installers IS 'Array of installer actions to create pack-specific runtime environments. Each installer defines commands to set up isolated environments (e.g., Python venv, npm install).'; COMMENT ON COLUMN runtime.execution_config IS 'Execution configuration: interpreter, environment setup, and dependency management. Drives how the worker executes actions and how pack install sets up environments.'; +COMMENT ON COLUMN runtime.auto_detected IS 'Whether this runtime was auto-registered by an agent (true) vs. loaded from a pack YAML (false)'; +COMMENT ON COLUMN runtime.detection_config IS 'Detection metadata for auto-discovered runtimes: binaries probed, version regex, detected path/version'; -- ============================================================================ -- RUNTIME VERSION TABLE diff --git a/migrations/20250101000012_agent_runtime_detection.sql b/migrations/20250101000012_agent_runtime_detection.sql deleted file mode 100644 index a367b6f..0000000 --- a/migrations/20250101000012_agent_runtime_detection.sql +++ /dev/null @@ -1,22 +0,0 @@ --- Migration: 000012_agent_runtime_detection --- Adds columns to support agent auto-detected runtimes - --- Track whether a runtime was auto-registered by an agent --- (vs. loaded from a pack's YAML file during pack registration) -ALTER TABLE runtime ADD COLUMN IF NOT EXISTS auto_detected BOOLEAN NOT NULL DEFAULT FALSE; - --- Store detection configuration for auto-discovered runtimes. --- Used by agents to identify how they discovered the runtime and --- enables re-verification on restart. --- Example: { "binaries": ["ruby", "ruby3.2"], "version_command": "--version", --- "version_regex": "ruby (\\d+\\.\\d+\\.\\d+)", --- "detected_path": "/usr/bin/ruby", --- "detected_version": "3.3.0" } -ALTER TABLE runtime ADD COLUMN IF NOT EXISTS detection_config JSONB NOT NULL DEFAULT '{}'::jsonb; - --- Index for filtering auto-detected vs. pack-registered runtimes -CREATE INDEX IF NOT EXISTS idx_runtime_auto_detected ON runtime(auto_detected); - --- Comments -COMMENT ON COLUMN runtime.auto_detected IS 'Whether this runtime was auto-registered by an agent (true) vs. loaded from a pack YAML (false)'; -COMMENT ON COLUMN runtime.detection_config IS 'Detection metadata for auto-discovered runtimes: binaries probed, version regex, detected path/version';